It's 9 in the evening. Do you know if your backups work? Do you know when you last made a successful backup of all of your data? Do you know whether you can restore from that backup? If not, how well can you sleep? You should verify your backups, and do it regularly, not just when you first set up the backup system. Verification means doing whatever you need to do to ensure all of your precious data has been backed up and can be correctly restored from the backups. The simplest way to do that is to restore all your data, and compare it with your live data, and note any differences. That requires you have enough free disk space to restore everything, but it's the almost the only way to be really sure. It's also a great way to ensure the restoring actually works. If you don't test that, don't be sure you can do it when needed. Some backup software has a verification mode, which doesn't actually write the restored data to disk, but compares it with live data directly. This can save space, and can be a reasonable compromise, but you should test actual restoring as well. You could do that by restoring only some of the live data, perhaps choosing files randomly for each test. If you have the disk space to do a complete restore, doing so is a great way to excercise your disaster recovery process in general. Here's one way of doing it:

• On your main computer, do a backup.
• On a second computer, perhaps borrowed for this, restore all your data, without using your main computer at all.
• Start using the restored data as your live data. Do real work, and do all the things you normally do. Pretend your main computer was eaten by your pet shark.
• If you notice something missing, or being corrupt, or being too old, get the files from your main computer, and fix your backup process so that the next time you won't have that problem.

How often should you do that? That, again, depends on how you feel about your data, and how much you trust your backup tools and processes. If it's really important that you can recover from a disaster, you need to verify more frequently. If data loss is merely inconvenient and not life-changingly disastrous, you can verify less often.

Cory Doctorow published a letter from a 14yo who had just read his novel Homeland [1]. I haven t had anything insightful to say about Aaron Swartz, so I think that this link will do [2]. Seth Godin gave an interesting TED talk about leading tribes [3]. I think everyone who is active in the FOSS community should watch this talk. Ron Garrett wrote an interesting post about the risk of being hit by a dinosaur killer [4]. We really need to do something about this and the cost of defending against asteroids is almost nothing compared to defence spending. Afra Raymond gave an interesting TED talk about corruption [5]. He focussed on his country Trinidad and Tobago but the lessons apply everywhere. Wikihouse is an interesting project that is based around sharing designs for houses that can be implemented using CNC milling machines [6]. It seems to be at the early stages but it has a lot of potential to change the building industry. Here is a TED blog post summarising Dan Pallotta s TED talk about fundraising for nonprofits [7]. His key point is that moral objections to advertising for charities significantly reduce their ability to raise funds and impacts the charitable mission. I don t entirely agree with his talk which is very positive towards spending on promotion but I think that he makes some good points which people should consider. Here is a TED blog post summarising Peter Singer s TED talk about effective altruism [8]. His focus seems to be on ways of cheaply making a significant difference which doesn t seem to agree with Dan Pallotta s ideas. Patton Oswalt wrote an insightful article about the culture of stand-up comedians which starts with joke stealing and heckling and ends with the issue of rape jokes [9]. Karen Eng wrote an interesting TED blog post about Anthony Vipin s invention of HAPTIC shoes for blind people [10]. The vibration of the shoes tells the person which way to walk and a computer sees obstacles that need to be avoided. David Blaine gave an interesting TED talk about how he prepared for a stunt of holding his breath for 17 minutes [11].

• [1] http://craphound.com/?p=4521
• [2] https://torforge.wordpress.com/2013/02/04/cory-doctorow-on-aaron-swartz/
• [3] http://www.ted.com/talks/seth_godin_on_the_tribes_we_lead.html
• [4] http://blog.rongarret.info/2013/02/its-question-of-when-not-if.html
• [5] http://www.ted.com/talks/afra_raymond_three_myths_about_corruption.html
• [6] http://www.wikihouse.cc/
• [7] http://tinyurl.com/d3jnmqt
• [8] http://blog.ted.com/2013/03/01/effective-altruism-peter-singer-at-ted2013/
• [9] http://tinyurl.com/krxvrnx
• [10] http://tinyurl.com/nlrq5k6
• [11] http://www.ted.com/talks/david_blaine_how_i_held_my_breath_for_17_min.html

A friend asked yesterday if I knew of a tool to print a web page as a single-page PDF, i.e., making the PDF page as tall as necessary to keep everything on one page. As a result, I know that Obnam's bug list is 4915 mm tall.

wkhtmltopdf --page-height 4915 --page-width 210 \
    http://liw.fi/obnam/bugs/ foo.pdf \
pdfinfo foo.pdf   grep '^Pages:

How long is yours? (wkhtmltopdf, in Debian, wasn't good enough for him, though: it has no way to disable the print style sheet, and he needs the page printed as shown in the browser.)

If we can t write diversity into sci-fi, then what s the point? You don t create new worlds to give them all the same limits of the old ones.
Jane Espenson (from interview with Advocate)

Basically reblogged from here: I couldn't resist, because it's just oh-so-damn-right.
And this should always apply when we imagine new worlds, no matter if for writing them or for fighting for them.

A random book haul, for no particular reason other than that it had been a while and several books came out that I have been eagerly awaiting (in one case for many years): Ann Aguirre Wanderlust (sff)
Ann Aguirre Killbox (sff)
Ann Aguirre Doubleblind (sff)
Ann Aguirre Aftermath (sff)
Ann Aguirre Endgame (sff)
Yoon Ha Lee A Conservation of Shadows (sff collection)
M.J. Locke Up Against It (sff)
China Mi ville Railsea (sff)
Chris Moriarty Ghost Spin (sff)
Paul Nahin An Imaginary Tale (non-fiction) Yes, Ghost Spin finally exists, and I have held it in my hands. Now if I can just remember what was happening in Spin State and Spin Control. I may just have to re-read them, which would be such a hardship.... The explanation of all the Aguirre purchases is that if you say sensible things amid the latest round of obnoxious sexism in the SFF community, and particularly when anonymous cowards claim your writing is obviously bad because of what you say, there are people like me who may well buy your entire series, new, as a show of support. Particularly if it was already on their radar as something to read. I already had the first book, but I bought all the rest sight unseen, based just on the strength of her commentary and some of the things other people have said about the books. This afternoon, I finished reading Grimspace, the first in the series, and now I'm delighted that I already have the others in hand. Excellent stuff; a bit trying on suspension of disbelief in a few spots, but the pacing was so good I didn't care. Review coming eventually when I catch up further. Yoon Ha Lee is my absolute favorite SF short fiction author writing currently, and that's saying something since I generally don't like short fiction. I've been eagerly awaiting her first collection ever since I heard it had gone to print. Also, in the context of picking up eight more SF novels or collections by women, I want to note just how irritating it is when people claim that women don't write science fiction. I read quite a lot of science fiction, and I haven't had any trouble finding female authors. I haven't even needed to try; the books I read run about half female authors with no special attention or effort on my part. If someone honestly hasn't heard of those writers, I have a bunch of suggestions in my collected book reviews, but too often it seems less a plea for help and more a rhetorical point about how science fiction is somehow more naturally male. Which is complete bullshit, and I have the library and reviews to prove it. Last year I read 13 unambiguous science fiction novels by women (not counting any fantasy). Some of them I liked, some of them I didn't, some of them were more believable than others, and some of them were harder (in the sense of being more scientifically plausible) than others just like the 11 science fiction novels, not counting fantasy, by men that I read in the same time frame. So far this year, I've reviewed one science fiction novel by a woman and two by men, and I have two more by women (and one by a man) waiting to be reviewed. I'm not doing anything special, just reading what's published, what's been recommended to me, and what wins awards. Anyone else can do exactly the same thing.

Look! A tiny TARDIS-powered release, as if we were a few years back, when Perl was hype! It s called IPC::GimpFu and abstracts talking to Gimp s script-fu server, making it possible to run a few commands/scripts without having to care too much about Gimp s start-up time, and without having to deal manually with the script-fu server protocol. At the moment, it was mainly tested against a local server, and without caring much about the reply (there s the nasty #583778 anyway), and it needs some polishing to make reports from CPAN testers greener Source is at http://git.mraw.org/ (git clone git://git.mraw.org/ipc-gimpfu). More on the reasons why I needed this module and those Gimp filters in a later post.

So I have a new desktop computer. I installed Wheezy on it via a USB stick, and everything worked. All the hardware. Yay. I guess we take it for granted when things like sound, disks, and network cards just work these days. I remmeber fighting with distros in the past, where such things were not necessarily straightforward. The only minor complication is the graphics card. I bought a cheap/random GeForce card for the new machine ( 30):

$ lspci -nn   grep VGA
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF119 [GeForce GT 610] [10de:104a] (rev a1)

Booting up I get a working X.org and GNOME 3.x, but the open graphics driver is "too bad" so I get fallback GNOME; with "Applications" & "Places" menus. Installing the proprietry driver gave me a full GNOME 3.x experience. But I didn't like it so for the moment I'm running:

• GNOME fallback mode.
• Bluetile.
• Open (nvidia) drivers only.

The plan was to install awesome, or similar, but I'm just a creature of habit and I'm still cloning git/mercurial repos and selectively restoring backups. My old desktop has been given to my partner to replace the EeeeeePC she's been using for the past year. I'll fettle over the weekend until I'm back up and running properly; but for the moment I'm good. All my videos/music are ported across. I can print, and I have access to the repos I'm currently working on. (Mostly lumail which will have a new release over the weekend.)

remctl is the middleware layer that we use all over the place at Stanford. It's a simple GSS-API client/server application that provides a very low-weight way of running a specific command using GSS-API authentication and strong ACL checking. The primary point of this release is to fix a very long-standing occasional data corruption issue. Due to the way that the remctl server checked for the completion of a command, a command that produced a lot of output immediately before exiting could have the end of its output cut off when sending it back to the client. That's fixed in this release. Also in this release, I finally figured out the thread-safe way of specifying to the GSS-API layer which Kerberos ticket cache to use for authentication. This means that the remctl_set_ccache API is now thread-safe if there were Kerberos libraries available during the remctl build and gss_krb5_import_cred was supported. I also fixed the way that the Perl modules are versioned, which resulted in a one-time move backwards of the module versions from Perl's perspective (from 3.4 to 3.05). Hopefully that won't be a problem for anyone. The Perl module version will match the package version with an adjustment so that it sorts properly as a number from here on. You can get the latest version from the remctl distribution page.

You're not the only one who cares about your data. A variety of governments, corporations, criminals, and overly curious snoopers are probably also interested. (It's sometimes hard to tell them apart.) They might be interested in it to data in order to find evidence against you, blackmail you, or just curious about what you're talking about with your other friends. They might be interested in your data from a statistical point of view, and don't particularly care about your specifically. Or they might be interested only in you. Instead of reading your files and e-mail, or looking at your photos and videos, they might be interested in preventing your access to them, or to destroy your data. They might even want to corrupt your data, perhaps by planting child porn in your photo archive. You protect your computer as well as you can to prevent these and other bad things from happening. You need to protect your backups with equal care. If you back up to a USB drive, you should probably make the drive be encrypted. Likewise, if you back up to online storage. There are many forms of encryption, and I'm unqualified to give advice on this, but any of the common, modern ones should suffice except for quite determined attackers. Instead of, or in addition to, encryption, you could ensure the physical security of your backup storage. Keep the USB drive in a safe, perhaps, or a safe deposit box. The multiple backups you need to protect yourself against earthquakes, floods, and roving gangs of tricycle-riding clowns, are also useful against attackers. They might corrupt your live data, and the backups at your home, but probably won't be able to touch the USB drive encased in concrete and buried in the ground at a secret place only you know about. The other side of the coin is that you might want to, or need to, ensure others do have access to your backed up data. For example, if the clown gang kidnaps you, your spouse might need access to you backups to be able to contact your MI6 handler to ask them to rescue you. Arranging safe access to (some) backups is an interesting problem to which there are various solutions. You could give your spouse the encryption passphrase, or give the passphrase to a trusted friend or your lawyer. You could also use something like libgfshare to escrow encryption keys more safely.

A few months ago, I was looking for a piece of software to spawn long-running tasks on behalf of some daemon with the benefit of the tasks not being interrupted when this daemon is restarted. Here are the specifications:

• No special privilege is needed to submit a task.
• Submitted tasks are not known beforehand.
• Task output is redirect to a log file.
• Submitted tasks are identified by a provided name.
• With only the name as a reference, a task can be checked for existence or killed.
• Tasks should not be interrupted by an unrelated event, like a configuration change or a software upgrade.

The last requirement explains why tasks are not spawned directly by the daemon requesting them: its complexity or the way it needs to be operated may need frequent restarts. Even if it is possible to re-execute a daemon while keeping its children, like stateful re-exec support in Upstart, this is quite difficult: internal state should be serialized and restored. Part of this state can be contained into a third-party library. While Upstart or systemd seemed to be good candidates for this purpose, I didn t find a straightforward way to run arbitrary tasks without any privilege¹. Here comes lan o². It is a very simple task launcher. It can run any tasks, stop them and check if they are still running. It leverages cgroups in recent Linux kernels and avoids the use of any daemon.

Quick tour Before looking at how lan o works, let s have a look at how it can be used. To avoid usage conflicts, each task is run in the context of a namespace that needs to be initialized:

$ sudo lanco testns init -u $(id -un) -g $(id -gn)

This is the only command that needs to be run as root. Subsequent ones can be run as a normal user. Let s run some task:

$ lanco testns run first-task openssl speed aes
$ lanco testns check first-task && echo "Still running"
Still running
$ lanco testns ls
testns
   first-task
      28456 openssl speed aes 
  

The output of the task is logged into a file:

$ head -3 /var/log/lanco-testns/task-first-task.log
Doing aes-128 cbc for 3s on 16 size blocks: 8678442 aes-128 cbc's in 2.85s
Doing aes-128 cbc for 3s on 64 size blocks: 2478283 aes-128 cbc's in 2.99s
Doing aes-128 cbc for 3s on 256 size blocks: 628105 aes-128 cbc's in 3.00s

If the task is too long, we can kill it:

$ lanco testns run first-task openssl speed aes
$ lanco testns stop first-task

You cannot run a task that already exists or kill a task that does not exist:

$ lanco testns run first-task openssl speed aes
2013-06-09T22:50:34 [WARN/run] task first-task is already running
$ lanco testns stop second-task
2013-06-09T22:50:45 [WARN/stop] task second-task is not running

Thanks to the use of cgroups, lan o is able to track multiple processes even when they are forking away³:

$ lanco testns run second-task sh -c \
>  "while true; do (sleep 30 &)& sleep 1; done"
$ lanco testns ls
testns
   first-task
      28456 openssl speed aes 
   second-task
      29572 sh -c while true; do (sleep 30 &)& sleep 1; done 
      29575 sleep 30 
      29593 sleep 30 
      29596 sleep 30 
      29599 sleep 30 
      29622 sleep 30 
      29644 sleep 1 
      29645 sleep 30 
  
$ lanco testns stop second-task
$ lanco testns check second-task   echo "Killed!"
Killed!

Also, there is a top-like command (lanco testns top):

Using cgroups Control groups (cgroups) is a mechanism to partition a set of tasks and their future children into hierarchical groups with a set of parameters.

Hierarchy Let s start with the hierarchical stuff first. To create a new hierarchy, you have to mount the cgroup filesystem on some empty directory. Usually, cgroups hierarchies are setup in /sys/fs/cgroups which is a tmpfs filesystem:

# mount -t tmpfs tmpfs /sys/fs/cgroup -o nosuid,nodev,noexec,relatime,mode=755

Now, we can create our first hierarchy:

# cd /sys/fs/cgroup
# mkdir my-first-hierarchy
# mount -t cgroup cgroup my-first-hierarchy -o name=my-first-hierarchy,none
# ls -1 my-first-hierarchy
cgroup.clone_children
cgroup.event_control
cgroup.procs
notify_on_release
release_agent
tasks

We ll see later the purpose of none. The most interesting file is tasks. It contains the PIDs of all processes in our group. Since there are currently no sub-group, all processes are part of it. Let s create a sub-group and attach a process to it:

# mkdir first-child
# cd first-child
# ls -1
cgroup.clone_children
cgroup.event_control
cgroup.procs
notify_on_release
tasks
# echo $$ > tasks
# cat tasks
23184
23311
# cat /proc/$$/cgroup 
9:name=my-first-hierarchy:/first-child
8:perf_event:/
7:blkio:/
6:net_cls:/
5:freezer:/
4:devices:/
3:cpuacct,cpu:/
2:cpuset:/
1:name=systemd:/user/bernat/1

We have added our shell to the new cgroup. Moreover, all its children will also be part of this group. This explains why we have two tasks: the shell and cat. The last command is quite interesting: for each hierarchy, it shows which cgroup the task belongs to. For a given hierarchy, each task is exactly in one group. The most useful features of lan o are done using just this: a namespace is a named hierarchy and each task is enclosed into a cgroup so it can be tracked properly.

Subsystems A subsystem makes use of the task grouping provided by cgroups to assign resources to a set of tasks. Each available subsystem can only be in one hierarchy, so the usual way to setup things is to assign a hierarchy for each subsystem⁴. Let s have a look at the cpuset subsystem: it allows to assign tasks to specific CPUs and memory nodes (for NUMA systems). Let s suppose we have 4 cores and we want to assign the first core to common system tasks and the three remaining ones to nginx:

# cd /sys/fs/cgroup
# mkdir cpuset
# mount -t cgroup cgroup cpuset -o cpuset
# echo 0-3 > cpuset/cpuset.cpus
# echo 0 > cpuset/cpuset.mems
# mkdir cpuset/system
# echo 0 > cpuset/system/cpuset.cpus
# echo 0 > cpuset/system/cpuset.mems
# for task in $(cat cpuset/tasks); do
>    echo $task > cpuset/system/tasks
> done
# mkdir cpuset/nginx
# echo 1-3 > cpuset/nginx/cpuset.cpus
# echo 0 > cpuset/nginx/cpuset.mems
# for task in $(pidof nginx); do
>    echo $task > cpuset/nginx/tasks
> done

The first mount is not needed if something else (like systemd) has setup the appropriate subsystem. Now, even if some system process is going crazy, it won t affect the performance of your webserver. The kernel comes with a more complete documentation if you need additional details on cgroups. There are high level tools to manipulate them, like the tools provided by libcg, but they are currently quite buggy and invasive.

Use in lan o Here are the features used by lan o:

• For each namespace, a specific named hierarchy is created.
• By setting the appropriate permissions on the hierarchy, an unprivileged user can create subgroups.
• Each submitted task has its own sub-group for tracking purpose.
• Actions can be executed when a task terminates by using the release agent mechanism⁵.
• The cpuacct subsystem is used to track CPU usage: a group is created in this subsystem for this purpose.

---

1. Both systemd and Upstart are difficult to run without being PID 1 and without being root. They both support user sessions but this requires to use them as PID 1 as well. I discovered later than runit would have been a good fit: it does not require to be PID 1, it does not need to be run as root and the service directory can be specified by an environment variable. It could have been tweaked to meet the above requirements.
2. Lan o means launcher in esperanto. The unusual diacritic on the letter c is not present in many web fonts, so the rendering may be a little odd.
3. Linux cgroups do not provide any facility to kill a whole group. We need to iterate through the processes and kill them one by one. lan o does not try to freeze processes before killing them and therefore may be inefficient against violent fork bombs.
4. The cpu and cpuacct subsystems are the notable exception. There seems to be little use to keep them in separate hierarchies, so they are part of the same hierarchy.
5. Unfortunately, only one global release agent per hierarchy can be used. Since lan o provides with the ability to execute an arbitrary command when a task terminates, the command needs to be stored on the filesystem for the release agent of the namespace to execute it.

After converting Lintian to using autodie, I was a bit trouble by the effects it had on Lintian s (start-up) performance. This overhead accumulated to at least a 1s when checking packages. Indeed, I am not the first to complain about this problem. For those who have not heard of it, autodie is a Perl pragma that basically alters subs and built-ins from a check return value for errors -sub (e.g. like C) to throws exception on error -sub (e.g. like Python). So, you can replace open( ) or die with open( ) when autodie is in effect. autodie achieves this by exporting a little sub that wraps the target sub (or built-in). This little wrapper is compiled[1] into the importing package. The problem with this solution is that it ends up compiling these subs once per importing package. Secondly, all of these wrappers are wrapped in something called a leak guard . As autodie is a lexical pragma it is not permitted to leak across file boundaries. The leak guards take care of that by doing a runtime check of the caller and reverting to the original sub/built-in if a leak occurred. These leak guards were also compiled into the importing package. It is probably not much of a surprise to anyone that autodie spent quite a bit of its runtime compiling these wrappers and their leak guards. Together with upstream, I have addressed some of these problems. The first optimization was to allow some wrappers for built-ins to be reused that does not take glob or bareword arguments[2]. Secondly, we started using closures to generate the leak guards. Combined these two effectively cut the load time in half for subsequent loads of autodie, though it did have a negative effect on the call overhead for these wrappers (~1s for 1M calls). That landed in autodie v2.18. The extra runtime overhead of v2.18 was basically caused by the leak guard being split into two (a small closure, which called the real leak guard with an extra argument). We managed to merge these two parts into one without (too much) loss of code maintainability. With that, the call overhead of these wrappers dropped by ~3s (for 1M calls) and thus the wrappers in autodie v2.20 were even faster than the ones in v2.17. Beyond these improvements, I have sent upstream several patches (still pending review) that will improve the performance of autodie even further. If merged, we are looking at another 1.5s (per 1M calls) removed and almost a factor 4 reduction on the load time of autodie. This factor 4 comes from lazily compiling the wrapper on demand, so some of the price is moved to the call overhead. That said, it is a net gain since it is highly unlikely that every module using autodie will use every single wrapped built-in (how often do you use both read and sysread in the same module)? [1] It is compiled via a call to eval using some template code. [2] For the rest of the built-ins, it could probably just use Symbol::qualify to qualify bareword filehandles. Perhaps that will come in a future version of autodie.

tl; dr version: shutdown works, Debian GNU/Hurd booting using sysvinit:

Loading GNU Mach ...
Loading the Hurd ...
GNU Mach 1.3.99-486
AT386 boot: physical memory map from 0x0 to 0x9f400
AT386 boot: physical memory map from 0x100000 to 0x1fffe000
AT386 boot: physical memory from 0x0 to 0x1fffe000
Enabling FXSR
pcibios_init : BIOS32 Service Directory structure at 0xfcff0
pcibios_init : BIOS32 Service Directory entry at 0xfc7ba
pcibios_init : PCI BIOS revision 2.10 entry at 0xfc78c
Probing PCI hardware.
ide: Intel 82371 PIIX3 (dual FIFO) DMA Bus Mastering IDE
    Controller on PCI bus 0 function 9
ide: BM-DMA feature is not enabled (BIOS), enabling
    ide0: BM-DMA at 0xc100-0xc107
    ide1: BM-DMA at 0xc108-0xc10f
hd0: got CHS=762/128/63 CTL=c8 from BIOS
hd0: QEMU HARDDISK, 3001MB w/256kB Cache, CHS=762/128/63, DMA
hd2: QEMU DVD-ROM, ATAPI CDROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
Floppy drive(s): fd0 is 1.44M
FDC 0 is a S82078B
probing scsi 5/22: Adapteco1542FInvalidiaddresseforishpnt.with 1542.
Invalid address for shpnt with 1542.
probing scsi 18/22:AWesternxDigitalAWD-7000nFailed9initialization of WD-7000 SCSI card!
probing eata on/1f0EATA0:Daddress 0x1f04in3use, skipping probe.
probing eata on 170EATA0: address 0x170 in use, skipping probe.
probing scsi 21/22: Iomega7parport ZIP drive ppa: Version 1.42
ppa: Probing port 03bc
ppa: Probing port 0378
ppa:         SPP port present
ppa:         PS/2 bidirectional port present
ppa: Probing port 0278
done
scsi : 0 hosts.
scsi : detected total.
Partition check (DOS partitions):
 hd0: hd0s1 hd0s2 < hd0s5 >
lpr0: at atbus0, port = 378, spl = 6, pic = 7.
2omultibootxmodulesd/execl$(exec-task=task-create)ne=$ kernel-command-line  --host-priv-port=$ host-port  --device-master-port=$ device-port  --exec-server-task=$ exec-task  -T typed $ root  $(task-create) $(task-resume)               task loaded: ext2fs --readonly --multiboot-command-line=root=device:hd0s1 console=com0 --host-priv-port=1 --device-master-port=2 --exec-server-task=3 -T typed device:hd0s1
task loaded: exec /hurd/exec
start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec init proc auth
INIT: version 2.88 booting
Using makefile-style concurrent boot in runlevel S.
Activating swap...done.
Checking root file system...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
ext2fs_check_if_mount: Can't check if filesystem is mounted due to missing mtab file while determining whether /dev/hd0s1 is mounted.
/dev/hd0s1: clean, 43930/181056 files, 281248/723200 blocks
done.
Cleaning up temporary files... /tmp.
/etc/rcS.d/S06mtab.sh: 48: /etc/rcS.d/S06mtab.sh: cannot open /proc/mounts: No such file
Activating lvm and md swap...(default pager): Already paging to partition hd0s5!
done.
Checking file systems...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
done.
Mounting local filesystems...mount: invalid option -- 'O'
Try  mount --help' or  mount --usage' for more information.
failed.
Activating swapfile swap...(default pager): Already paging to partition hd0s5!
done.
df: Warning: cannot read table of mounted file systems: No such file or directory
Cleaning up temporary files....
Configuring network interfaces...inetutils-ifconfig: invalid arguments
ifup: failed to open pid file /run/network/ifup-/dev/eth0.pid: No such file or directory
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
DHCPDISCOVER on /dev/eth0 to 255.255.255.255 port 67 interval 6
DHCPREQUEST on /dev/eth0 to 255.255.255.255 port 67
DHCPOFFER from 10.0.2.2
DHCPACK from 10.0.2.2
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
bound to 10.0.2.15 -- renewal in 38544 seconds.
done.
Cleaning up temporary files....
Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
INIT: Entering runlevel: 2
Using makefile-style concurrent boot in runlevel 2.
Starting enhanced syslogd: rsyslogd.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting system message bus: dbusFailed to set socket option"/var/run/dbus/system_bus_socket": Protocol not available.
Starting OpenBSD Secure Shell server: sshd.
GNU 0.3 (debian) (console)
login:

This is the result of:

• A patch by Pino Toscano that adds the -e and --ifexists flags to Hurds swapon: http://lists.gnu.org/archive/html/bug-hurd/2012-01/msg00060.html
• Pino Toscanos runsystem.sysv that starts sysvinits init (see below).
• A patch series to allow setting options of procfs translators at runtime (the hurdish equivalent of "remounting"): http://lists.gnu.org/archive/html/bug-hurd/2013-06/msg00119.html
• A trivial patch to tmpfs, also for updating the options at runtime: http://lists.gnu.org/archive/html/bug-hurd/2013-06/msg00126.html
• A patch series adding --fake and --no-mtab to our mount utility, fixing -oremount with one parameter (e. g. mount -oremount,ro /) and finally fixing a subtle bug in an internal look-up function: http://lists.gnu.org/archive/html/bug-hurd/2013-06/msg00135.html
• A minor tweak to checkroot.sh.
• A minor tweak to mountall.sh. Turns out that there are two init processes at this point, pidof -s takes care of that. Not sure why this does not happen on Linux, maybe it even does? Or it is related to us using inits undocumented -i flag to make it ignore the fact that it isn't pid 1. Needs further investigation.
• Disabling killprocs and sendsigs. See below.
• A minor tweak to mount-functions.sh. Same as last week though.
• Using the same inittab as Debian/Linux uses (we seem to ship our own, though I must admit I'm not sure it comes from, dpkg doesn't know about this file on my installations).
• Using the reboot and halt utilities from the sysvinit package instead of our own.

This yields a surprisingly functional Debian/Hurd system with sysvinit. There are still lot's of loose ends, I populated my status page with them. I spent quite some time in #hurd on freenode and Pino Toscano offered to send me some old notes of his about the sysvinit issue. This turned out to be most helpful as his notes were about plugging sysvinits init into the boot process. This was the next logical thing to do for me, so first thing Monday morning I just put his runsystem.sysv file with some very minor tweaks (my version) into my overlay and had a working system in front of my eyes. That certainly was a nice way to start a week of work :), so many thanks again Pino! For your convenience I've created an overlay with all necessary changes and recompiled binaries. Be careful though, you might want to backup /sbin/reboot first or you won't be able to reboot your system! Of course my tool does that for you, and I've managed to create a reasonably small (253 mb) image thanks to the zerofree tool, the process of compacting images is implemented in hurdtest as the compact subcommand. I also added sysvinit specific tests. So if you want to try hurdtest, download the image and unxz it, my tool expects the image to be in the current working directory and be named image.qcow2. Download and untar an overlay and execute:

% hurdtest -logfile log -sysvinit overlay path/to/second-week-overlay

If you specify -logfile you can execute tail -f path/to/log to see what the VM is printing to the console in real time. The sysvinit specific tests demonstrate that indeed sysvinit is managing the system and that e. g. init 0 or shutdown -r now works. There are at least three major problems that I will need to tackle for this project:

• /bin/init is not running as pid 1.
• kill(-1, SIGTERM) kills essential processes.
• There is no /proc/mounts.

I think the last issue is what I'm going to look at next week. There's information about this in the Hurd wiki and I've spoken with Richard Braun about it and he drilled me to think about the hurdish way to solve this. I think I've got a pretty good picture of how to implement this in my head and am eager to give it a go. This will involve adding a function to Hurds filesystem interface that allows one to query an active translator about his mount options and "source" and about all active and passive translators that are attached to one of its nodes. This is where the fun actually begins. This is changing an integral part of an operating system, but this is okay since it is just another program written in c, operating in userspace. But hey I know that, that's just c and if something goes wrong (and it always does) I fire up gdb and have a look. And mind you, I'm running all this as an unprivileged user, no need to jeopardize the entire system for that. Besides implementing /proc/mounts I will also add -O, --test-opts to mount, and -v, --verbose to swapo n,ff . PS: My blog is still not properly registered to planet.d.o. I asked planet@ about that, but got no response. Any help? I never imagined blogging being so stressful, I care about losing potential readers :/

Once again, it's been some time since the last FusionForge update in here. The main explanation is that news is slow on that front. Debian Wheezy was released without FusionForge packages, as previously announced, and even Sid hadn't seen any update on the package for way too long. The latter has just been fixed though: the freshly-released 5.2.2 upstream version is on its way to Sid (via NEW, since it adds a new plugin to allow authenticating against a CAS server). If and when it reaches Debian Jessie (the current testing ), I'll work on backporting it for Wheezy. At some point I'll also start uploading snapshots of the upstream master branch to experimental , to give adventurous users a glimpse of what is to come in the future releases, although I'll stick to only uploading when the automated tests all pass.

Review: This Alien Shore, by C.S. Friedman

Publisher:	DAW
Copyright:	1998
Printing:	July 1999
ISBN:	0-88677-799-2
Format:	Mass market
Pages:	564

Jamisia is an orphan taken in by the Shido corporation, but she has a fairly comfortable and happy life in Shido Station. She lives in the care of her tutor, has friends, has something that feels like a relatively normal life. Then the station is attacked and destroyed by a rival corporation and she barely escapes in a one-person lifeboat, sent inexplicably by her tutor with a fake identity to rendezvous with a spaceship heading for the up-and-out. That begins a long and quite confusing flight away from everything she's known, and a discovery that her life is far more complicated than she thought. Meanwhile, far from the backwater of corporate tyranny that Earth has become, a new computer virus is wreaking havoc on Guild outpilots. The Guild navigate the ainniq: a sort of subspace that allows ships to slip between nodes without regard to distance. This is the only acceptable method of interstellar travel now that humans know that the more conventional Hausman Drive causes massive genetic mutations, and only Guerans have the secret of navigating it safely. But a virus of a sophistication no one has seen before appears to be targeting outpilot brains through their implants. This Alien Shore is told in two threads but multiple characters and viewpoints. Jamisia (yes, I stumbled over that name the entire book) spends nearly the whole book running while discovering that her head is a very complicated place for very bad reasons. Meanwhile, various other people (most notably the careful and meticulous Kio Masada) are analyzing and tracing the virus, which is of deep concern to the powers that be in the Guaran-controlled network of human colonies. The threads do come together eventually, helped by a young and much more impulsive hacker named Phoenix who is also on the trail of the virus, but it takes much longer than I expected. The book is a bit of cyberpunk, a bit of space opera, and is stuffed to the brim with world-building ideas. There's always something that feels slightly emotionally off about C.S. Friedman novels. Her characters are a bit too intense, a bit too full of primary colors and inherent natures. They don't do a lot of changing; they're just a little too much purely themselves to feel quite accurate. Sometimes it matters and sometimes it doesn't. This is one of the books where it mostly doesn't, partly because it's full of such an amazing variety of characters, and partly because Friedman does a great job adding situational complexity by playing them off of each other and showing them through each other's eyes. Pyschological realism, though, this isn't, nor would I look to Friedman for that. That's somewhat unfortunate given how central psychology is to this book. The focus that slowly emerges from this novel is the whole-hearted embrace of psychological difference as a force for creativity, power, and perspective. Acceptance, respect for, and integration of difference borders on a religious belief for Guerans, which makes this book one of the most wholeheartedly inclusive stories (at least for psychology) I've read. I loved that message enough to overlook the fact that those psychological differences are not treated with much subtlety. The most central of the differences (a very mild spoiler, but one that's essentially given away on the back cover and that is easy to guess early in the book) is multiple personalities. Sadly, I thought that was also the least successful. Friedman treats the topic respectfully and as a difficult asset, which I liked, but it's bog-standard fictional multiple personalities. The presentation owes much more to popular fiction than to the reality of disassociative identity, and the notes that Friedman hits (and the characteristics that the multiples embody) are predictable. However, her second major psychological idea is simply brilliant. The Guerans have developed a coding system for people to communicate their psychological profile to anyone who sees them, and all of the Gueran characters in the novel actively engage with that system and make extensive use of it. Friedman hints at a great deal of subtlety: a wealth of primary patterns (many of which are described in quite enjoyable animal metaphors as chapter epigraphs), secondary patterns that can be layered underneath, and a complex interaction of base patterns and patterns for daily mood. Here, I think Friedman's mild tendency towards archetypal characters helps, since it makes the psychological coding system clearer for the reader. I wish it had been more central to the story and explained further, although I can see the risk of turning the novel into an RPG sourcebook. (Although, now that I mention it, I would love to read an RPG sourcebook for this universe.) The other point for which I have to give Friedman huge credit is that at least half of the book is about a computer virus, the analysis of that virus, and the hacking culture of her future universe, and at no point during the story did I want to start throwing things. There are no completely unbelievable magic computer programs, the sense of movement of data and virtual presence through computer networks seemed vaguely plausible and less steeped in unrealistic VR metaphors, the sense of aesthetics of code is captured remarkably well for a story of this sort, and Friedman even brings some variety to her computer-savvy characters. There are a few mildly stereotyped young hackers (although even those have a tendency to have more depth than the stereotype), but Friedman also populates her electronic world with the other types you rarely see in these novels: thoughtful, brilliant, but meticulous researchers, competent technicians who don't engage as much on the aesthetic aspects, and even moderately-intelligent users who pay attention to the computers their lives are built around. I could actually believe in both these people and the software they use with only the suspension of disbelief required for any future SF. That's not to say there are no flaws, quibbles, or places where computers probably won't ever work that way. But that's part of what makes me so impressed. Friedman is still using computers and hacking in the direct service of a fairly typical SF plot to achieve specific dramatic ends and to create a mystery. She's not building the story around actual technology; she's staying fairly close to the sort of technology and plots that a lot of cyberpunk and hacking novels have. And yet she manages not to make the computing entirely stupid or unbelievable. It's existence proof that an SF author can pay attention to experts and research without compromising the story they want to tell. I want to take this book and smack Robert Sawyer upside the head with it. This is a great idea book with a heavily computer-driven plot strand that I quite enjoyed. It is, however, not without its flaws. The virus-hunting part of the plot is significantly more interesting than Jamisia's flight, particularly given that she keeps running, and running, and running, encountering various people who seem important and end up not being important at all. It takes more than half the book before anything particularly interesting happens to her; up until then, I was dragging myself through her segments so that I could get back to Masada and the other Guerans. She's also annoyingly dim about some things. I figured out what the deep secret in her brain was within the first four or five chapters (it's entirely obvious from the political structure of Friedman's universe), and I have no idea why it took everyone else so long to figure it out. There is also, as mentioned, the slightly off feeling of the characters that I find typical of Friedman, and a cut-throat power struggle at the heart of the book that just didn't have enough psychological believability to fully engage me. But the ideas are wonderful: technology-driven political structures, corporate-controlled states, ideological clashes, lots of believable cultural interactions, and a neat two-option interstellar travel system (even if the nature of the Hausman Drive damage bears no resemblence to any genetic science known to our reality). And that's on top of a wonderful visual language of psychology and discussions of computer security that I actually enjoyed reading. Friedman's the sort of author who won't be to everyone's tastes. You have to have some tolerance for her tendency to write grand clashes of power between highly flawed, aggressively emotional people. Sometimes you have to put up with rather a lot of setup. But I really liked this novel. I suspect I'm editing out a lot of the boring bits in my memory, but I fondly remember that edited version as a ton of fun. Rating: 8 out of 10

I think that hexadecimal is a fairly poor pre-encoding for information exchange via data to speech and speech to data engines (aka voice boxes, brains and fingers). Reading out and typing long strings of hexadecimal digits at OpenPGP keysignings is tedious and annoying. There have been some experiments using photography and QR codes for this, which I think is pretty cool but not always practical since not everyone has a camera and QR code software installed. An alternative to this might be to pre-encode using a different scheme that encodes to less words in English speech. Diceware is one possibility that I recently experimented with. Diceware is a password generation scheme that encodes data from a random number generator (aka some dice) using a list of 7776 words. Each word thus represents a 5 digit number in base 6. Diceware is mainly used for generating strong and easier to remember passwords. So Diceware is the coupling between a non-digital random number generator and an interesting encoding scheme. Below are my fingerprints in Diceware and Hexadecimal form. The Diceware form is longer to type at 69 characters, 40 for hex. The Diceware form has the advantage that it is only 16 words to say while the hexadecimal form is 40. I don't know if this will be more practical than hex but I can almost remember my entire fingerprint after reading it a few times so hopefully that will translate to practical use. A rough script for encoding your fingerprint in the Diceware encoding is available but I haven't implemented the reverse yet. I would be glad if someone could check it for correctness.

Diceware:    frame maze bear usgs deter wag prissy bush hoyt mayo upton child indy
Hexadecimal: 610B 28B5 5CFC FE45 EA1B  563B 3116 BA5E 9FFA 69A3

If you want to discuss this topic and try it out in person and attempt to understand my accent, I'll be at DebConf13 and OHM2013. Some downsides that I can think of are accents, multiple spellings and the inclusion of non-words and special characters in the wordlist. These can be solved by using a different wordlist created specifically for OpenPGP fingerprint exchange that only includes suitable words. Update: there is a standard called PGP Word List that is much better than Diceware for this purpose. It maps 256 words to individual bytes, with different lists for odd and even bytes. Unfortunately it doesn't appear to be supported by GnuPG or signing-party. This post was inspired by the screenshots for RedPhone. You can comment on this post on debian-project.

This is part 3 in a series on organizational design and growth. What it means

Taking responsibility for action, and following through. Every time. Why it s important

Commitments are a cornerstone of collaborative teamwork. Knowing what to expect from others makes it possible to plan beyond your own work. If you aren t sure whether a task will be completed, you carry that uncertainty as low-grade anxiety, which accumulates and creates cognitive load. You may even spend extra time checking back to make sure the need has been met. Old status quo

Most work happens without an explicit commitment at all. Consistency is highly variable from one individual, team or circumstance to another. Tasks fall through the cracks, and we may not even realize it until much later. New status quo

When something needs to get done, someone agrees to take responsibility for it. When this happens, everyone involved can trust that it will get done. When exceptions happen, we acknowledge them, and seek to understand what happened so that we can do better in the future. Behaviors that help

• Make commitments explicit: say I will take care of that , and record that commitment somewhere, preferably in a shared work space where everyone concerned can see it. For example, meetings should generally result in decisions and commitments to act. Write them down, and check back on the commitments at the next meeting to confirm that they re done.

• Hold yourself accountable: Follow through on your commitments visibly, e.g. reporting back the next time you see each other. If you re unable to deliver for some reason, apologize and explain.

• Hold others accountable: Expect these same behaviors from others.

• Celebrate success: Thank others for following through on their commitments.

• Learn from failure: When commitments are broken, it should be treated as an exception. Something went wrong, and we should work to prevent it in the future.

• Balance your workload: Doing fewer things at a time will help you complete them much faster and more consistently. If you don t have the bandwidth to take something on, say no. Make space for someone else to take it, or ask for help. If you are unsure about whether you can commit, you probably shouldn t.

Obstacles that stand in our way

• Fatigue: It s hard to commit to something new if you already feel overwhelmed by the status quo.

  • Slack is an essential component of any change. If we re 100% busy just keeping the lights on, we re not getting better at what we do, and that road leads to mediocrity

• Cynicism: If past commitments have been made but not kept, people can become cynical about setting goals and priorities.
  • This needs to change. Setting and achieving goals needs to be normal, respected and valued behavior in the company.

• Unclear roles and responsibilities: This makes it hard to tell who should take responsibility for getting things done

Further reading

• http://en.wikipedia.org/wiki/Getting_Things_Done

• http://www.amazon.com/Slack-Getting-Burnout-Busywork-Efficiency/dp/0932633617