digest version 0.6.4 is now on CRAN and in Debian. This is a pure maintenance release which should help with a build issue affecting users on Solaris. CRANberries provides the usual summary of changes to version 0.6.3. Our package is available via the R-Forge page leading to svn and tarball access, my digest page, the local directory here as well as via Debian and its mirrors.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

I ve just read Bartleby the Scrivener which is a short story about a scrivener who refused to work saying I d prefer not to . It reminded me of some situations in the computer industry. I ve never seen a single case where someone preferred not to work when everyone around them (colleagues and management) wanted them to work. But then the incidence of having an entire team and management wanting to work efficiently isn t nearly as common as one might imagine. In some cases it s desired that someone not work, such as a former colleague who was hired as a sysadmin but did nothing but change backup tapes (a few hours work per week). Not having him login as root improved the general reliability of the servers but it was fortunate that we never needed to restore from backups One time I had a colleague who preferred to spend most of his time in the office searching the Internet for videos of street fights. I have often told colleagues that I would prefer them to work, but in the case of a guy who s only hobby is street-fighting I decided to let it go. Managing people can be difficult, particularly for someone who doesn t like disagreements. Some managers that I ve reported to seemed to prefer not to manage in an apparent attempt to avoid disputes. One time when I complained about a colleague not even having a suitable computer to permit doing any work a manager responded with the rhetorical question what do you expect me to do? . That manager didn t do any annual reviews of staff for over a year, he only eventually did some reviews because he was told that his scheduled promotion was on hold until he got them done. I got the impression that at least two levels of management preferred not to work at that company. Sometimes it just gets weird though, such as the occasion when I was the only member of a team and the manager who was supposedly managing no-one but me never seemed to have time to have a meeting with me. But he didn t want me to bypass him and talk directly to other people in the company, so he preferred not to work and not to have anyone else do his job. Most of the companies that I ve worked for in a full-time capacity didn t seem to have any effective technical interviews (note that I ve mostly worked for financial companies and ISPs not free software companies). So it seems that anyone with minimal computer skills who wants a well paying job could just send out a CV to a bunch of recruiting agencies, get interviewed by enough companies to eventually hit one without a technical interview process and then find a job that doesn t require work. Depression The Wikipedia page about Bartleby the Scrivener [2] suggests that Bartleby was depressed. I wonder how much of the lack of performance I ve witnessed has been due to depression. There appears to be a strong correlation between work environments that cause depression and people preferring not to work. Maybe managers should be considering how to make work less depressing to try and get more effective employees (in terms of quality and quantity of work). One example of this is the sysadmin team death spiral I ve witnessed where no-one can automate solving problems (EG by cron jobs to manage resource usage and analysis tools to find minor problems before they become major problems) because everyone is dedicated to fixing things that break needlessly (EG systems crashing due to lack of disk space). When people start getting control over recurring problems and automating things then the work becomes increasingly about solving problems and less about implementing the same manual processes every day/week and it s more fun and effective for everyone. At the BoF on depression at LCA 2013 one delegate stated that many companies have people in HR who can arrange support for depressed employees. Apparently if you are depressed and you work for a company that s large enough to have a HR department then it can be beneficial to talk to HR about it. That probably works well in the case where an employee is depressed but the company is working well. But in the case where the company isn t working well it seems unlikely to help. David Graeber wrote an interesting article about Bullshit Jobs [3]. He goes a bit far, I don t think that late night pizza delivery is a bullshit job and actuaries are useful to society. But his points about the existence of useless jobs are reasonable. Management Levels I sometimes wonder whether there is some benefit in establishing social norms about working and then having management take little interest in how it happens. If a team works well together then management could just set deadlines (which would be negotiated with employees who know what s possible) and let the team work out how to do it. Then instead of having one manager for each team of ~10 people who theoretically tracks what everyone is doing you could have one manager for a dozen teams who just tracks overall team performance essentially remove a layer of management. Valve is famous for having no formal management structure and for getting things done, unfortunately that apparently allows school-style cliques to block actions [4]. But I think that the Valve experiment is useful and provides some ideas that can be used by other companies. Maybe if instead of requiring consensus of the entire company for hiring decisions they only required consensus of the team things would have worked better. Of course another down-side to such things is that hierarchical management can be good for avoiding discrimination and bullying. The article I cited about Valve compares it to high-school. It could be that Valve employees were all nice people who only hired other nice people. But if similar systems were implemented in many companies then some would surely end up being like a typical high school with all the bullying and mistreatment of minority groups that entails. Michael O. Church wrote an interesting article in which he divides employees into four categories, loser , clueless , psychopaths , and technocrats (note that he didn t invent the first three names) [5]. In his model the clueless category includes most middle-management. I think that there are some problems with Michael s model and I m not arguing for a technocracy (which is how this post might be interpreted in terms of his ideas). But I think he demonstrates some of the real problems in the way companies are managed and in his model the losers prefer not to work as long as they can get paid. Conclusion I don t have any good solutions to these problems to offer. It seems that the best we can hope for is incremental change to make work less depressing, to have the minimal amount of management, and to avoid bullshit jobs .

• [1] http://www.gutenberg.org/ebooks/11231
• [2] http://en.wikipedia.org/wiki/Bartleby,_the_Scrivener
• [3] http://www.strikemag.org/bullshit-jobs/
• [4] http://www.wired.com/gamelife/2013/07/wireduk-valve-jeri-ellsworth/
• [5] http://tinyurl.com/kxcduch

I have implemented really pretty histograms in the venerable ping software, something I never thought could be improved, until I discovered prettyping.sh, something that was just begging for improvements. Which are now done. But first some history... First, in 1983 (!), there was ping and network operators rejoiced, as they could see if a host was down or not, and have all sorts of geeky statistics:

PING koumbit.net (209.44.112.66): 48 data bytes
56 bytes from 209.44.112.66: icmp_seq=0 ttl=52 time=25.076 ms
56 bytes from 209.44.112.66: icmp_seq=1 ttl=52 time=24.006 ms
56 bytes from 209.44.112.66: icmp_seq=2 ttl=52 time=24.106 ms
^C--- koumbit.net ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 24.006/24.396/25.076/0.483 ms

Then, in 2006, there was noping, and things were, well, not much better, but we had colors, and there was some rejoicing. Then, in october 2013, there was prettyping.sh, and things got really flashy and "oh wow, you can do that?" There was much rejoicing. Then, tonight, I learned ncurses and there was much headaches. But then, I reimplemented prettyping in noping, and I am so happy that I wrote this blog post: Unless you haven't figured out how cool this is, let me break it down for you:

1. it supports IPv4 and IPv6
2. it allows you to track multiple hosts at the same time, and compare them
3. this allows you to easily track down failure points in a network, something for which you usually need smokeping (needs a webserver) or mtr (doesn't have colors)
4. it allows you to track a lot (the last minute at least) of history by default
5. it is visually easy to track, even from a distance

You may know of that hack that can make "ping" ring a bell when it receives a packet? This is better: you can see the packets latency (or when they are just dropped!) from a distance, using an intuitive color code. The code is up for review here:

git clone -b prettyping git://src.anarc.at/liboping

Thanks to the well architectured noping, the patches were not that complicated to implemented. I have contacted upstream to get those changes merged, and hopefully this will be in your favorite Debian distribution soon. Rejoice!

I ve did something I surely will (financially) regret, next year, and designated the Neo900 to be the successor to my PocketPC, due to the latter having only 64 MiB RAM and Geocaching applications being quite hungry. It s got a lovely hardware keyboard, a pen display like the PocketPC (as opposed to the wishy-washy displays that Android and iPhone have), not only GPS but also GLONASS, fully free software with mostly free firmware (I m okay with that, mostly), a Ctrl key (useful in ssh and locally and my text editor; ^I is Tab, so it s useful in shell, too), WLAN, UMTS (I don t think I need LTE and would rather it have the more RAM), USB host (OTG), and lots of other nice features. In short, it s a tinkerable device: one I can not only hack at, but also hack on. Since I use a dumbphone for mobile phone anyway (pro: separate battery from the toy PocketPC/Smartphone we re talking two+ weeks of battery time when using it here, and easier use and less bugs, and a reliable fallback when I tinker too much ), this is perfect for me. I m reposting this in the wlog mostly because it s an interesting technical and OSS project, and because if 1000 people want one it will get less expensive for all of us (while here shameless plug any sponsors willing to contribute some EUR so I don t ruin myself with this, in exchange for services of some kind?). I ll probably run Debian on it (unless it goes systemd), maybe in a chroot if the native OS has functionality needed that I can t simply put into packages; they say Maemo has much better power management, but considering most use will have GPS, GLONASS and backlight on, battery isn t going to last long anyway or maybe even native I ve been wanting to know what this freesmartphone stuff my m68k (Atari VM) buildd has been happily compiling, anyway and some sort of Geocaching application (ideally a cross between something online, CacheWolf and an offline OSM (with most of Europe, but uninteresting tags stripped) and possibly access to the GS Live API but nevertheless supporting TC, NC, OC, gpsgames too), and my usual mksh(1), GNU screen, jupp(1), lynx(1), ssh(1) toolchain.) Delivery is expected for mid to end of 2014, but once it s there I ll keep you informed

BackupPC is a pretty good backup system. Its configuration is rather flexible, it has nice expiry policies, and it can store duplicated file contents only once (for files that are shared across hosts or don't change in time) within a compressed pool of data. However, it doesn't do much to help pushing the data to off-site storage, or at least not very efficiently. So if you have a BackupPC instance running on a Raspberry Pi or a plug computer at home, it's a bit tricky to protect your data against loss due to burglary or home fire. The obvious solution would be to rsync the storage pool to a remote site. However, the current pooling system relies heavily on hardlinks, and rsync is notoriously inefficient with those. In the home backup server scenario, this means that even if the computer is more powerful than a Pi and can handle the memory requirements of rsync, you'll often end up transferring way too much data. So, since the obvious solution doesn't work straight away, what do we do? Why, we fix it, of course. With a little look into the storage pool, we notice that the bulk of the data is stored in files with an abstract name (related to the contents) within a $prefix/pool directory; the files with concrete names looking much like their original are stored within $prefix/pc, and they're actually the same files because they're hardlinks. Knowing this (that rsync doesn't), we can make a smarter replication tool, by

1. pushing only the pool with standard rsync;
2. storing locally, and recreating remotely, the structure of hardlinks;
3. pushing everything again with standard rsync.

Steps 1 and 3 are simple invocations of rsync -aH; step 2 can be implemented using the following two scripts. Run store-hardlinks.pl locally, push the links file, then run restore-hardlinks.pl on the remote server. This will ensure that files already present in the pool are also hardlinked in their natural location. store-hardlinks.pl:

#! /usr/bin/perl -w
use strict;
use Storable qw(nstore);
use File::Find;
use vars qw/$prefix $poolpath $pcpath %i2cpool %todo $store/;
$prefix = '/var/lib/backuppc';
$poolpath = '$prefix/cpool';
$pcpath = '$prefix/pc';
$store = '$prefix/links';
# for the convenience of &wanted calls, including -eval statements:
use vars qw/*name *dir *prune/;
*name   = *File::Find::name;
*dir    = *File::Find::dir;
*prune  = *File::Find::prune;
# Scan pool
File::Find::find( wanted => \&wanted_pool , $poolpath);
# Scan PC dirs
File::Find::find( wanted => \&wanted_pc , $pcpath);
nstore \%todo, $store;
exit;
sub wanted_pc  
    my ($dev,$ino,$mode,$nlink,$uid,$gid);
    (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($_)) &&
      -f _ &&
      ($nlink > 1) &&
      do  
      $name =~ s,$pcpath/,,;
      if (defined $i2cpool $ino )  
      $todo $name  = $i2cpool $ino ;
       
     
 
sub wanted_pool  
    my ($dev,$ino,$mode,$nlink,$uid,$gid);
    (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($_)) &&
      -f _ &&
      ($nlink > 1) &&
      do  
      $name =~ s,$poolpath/,,;
      $i2cpool $ino  = $name;
     
 

restore-hardlinks.pl:

#! /usr/bin/perl -w
use strict;
use Storable;
use File::Path qw/make_path/;
use vars qw/$prefix $poolpath $pcpath %todo $store/;
$prefix = '/srv/backuppc-mirror';
$poolpath = "$prefix/cpool";
$pcpath = "$prefix/pc";
$store = "$prefix/links";
%todo = % retrieve ($store) ;
my ($dev,$ino,$mode,$nlink,$uid,$gid);
foreach my $src (keys %todo)  
    my $inode;
    my $dest = $todo $src ;
    my $dpath = "$poolpath/$dest";
    my $spath = "$pcpath/$src";
    my $sdir = $spath;
    $sdir =~ s,/[^/]*?$,,;
    make_path ($sdir);
    next unless -e $dpath;
    if (! -e $spath)  
      link $dpath, $spath;
      next;
     
    (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($spath));
    $inode = $ino;
    (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($dpath));
    if ($ino != $inode)  
      unlink $spath;
      link $dpath, $spath;
     
 

The initial transfer can still take forever if the pool is large (and if you're pushing it through the small end of an ADSL link ), but at least the files are only transferred once. Note: This is only useful for current versions of BackupPC. Apparently BackupPC 4 will have a different pooling system without hardlinks, and the following hack will no longer be required. For now, though, here it is.

In early October, I had an opportunity to talk about Rcpp and RcppArmadillo at the Statistical Computing Seminar at the Booth School of Business at the University of Chicago. And then two weeks ago, I had an invitation to talk at the Center for Research Methods and Data Analysis at the University of Kansas where I covered similar material as well as ongoing work on the RcppZiggurat package (for which I should have an updated version soon). Slides from both talks are now at the top of my talks / presentations page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Replacing my phone My first Android phone was a ZTE Blade (sold as Orange San Francisco here in the UK). It originally shipped with Android 2.1 but was upgradable to 2.3 thanks to CyanogenMod (or other unofficial mods). However there's little sign of an upgrade to 4.x; the apps I want to use are pushing the limits of its CPU, RAM and internal storage; and I've never got very good at typing on a soft keyboard. So it seemed like time to get a newer phone with a hard keyboard (and still with a SD slot). After a little research with the CyanogenMod compatibility list and some time looking at reviews, I settled on the Samsung Galaxy S Relay 4G, which was exclusive to T-Mobile USA but is being resold through eBay. It shipped with Android 4.0 but I immediately installed CyanogenMod 10.2 (Android 4.3). This was a bit of an adventure as I bought the Blade with CM already installed and wasn't familiar with the multiple steps that were necessary. Copying my data The last step, and the real subject of this entry, was to move my data across. Much of this was on a SD card which I could simply plug into the Relay. The internal files had to be backed up onto this card and then restored, using the recovery environment (ClockworkMod) on each phone. After rebooting the Relay into the full Android system, my apps and settings were mostly present but I was immediately confronted with a series of error dialogs reporting that 'Unfortunately, Dialler has stopped' - and the same for 'Clock' and 'the process android.process.acore' (whatever that is). What went wrong There are thankfully more detailed error logs in the filesystem, under /data/system/dropbox (this is for logging at the Java level; if a process is terminated by a fatal signal it's logged to /data/tombstones). There are several ways to get at them, but I used recovery mode and adb to get a root shell where I could easily read and write files as necessary. The Contacts (aka People) database upgrade fails with an SQLiteException, apparently because it doesn't account for upgrading from the schema used in 2.3:

Process: android.process.acore
Flags: 0x883e45
Package: com.android.providers.applications v18 (4.3.1-f963020e38)
Package: com.android.providers.contacts v18 (4.3.1-f963020e38)
Package: com.android.providers.userdictionary v18 (4.3.1-f963020e38)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
android.database.sqlite.SQLiteException: no such column: phonebook_label (code 1): , while compiling: UPDATE raw_contacts SET display_name_source=?,display_name=?,display_name_alt=?,phonetic_name=?,phonetic_name_style=?,sort_key=?,phonebook_label=?,phonebook_bucket=?,sort_key_alt=?,phonebook_label_alt=?,phonebook_bucket_alt=? WHERE _id=?
...
        at com.android.providers.contacts.ContactsDatabaseHelper.updateRawContactDisplayName(ContactsDatabaseHelper.java:5344)
        at com.android.providers.contacts.ContactsDatabaseHelper.upgradeToVersion504(ContactsDatabaseHelper.java:3580)
        at com.android.providers.contacts.ContactsDatabaseHelper.onUpgrade(ContactsDatabaseHelper.java:2261)
...

Clock fails somewhat similarly though it apparently didn't try to upgrade:

Process: com.android.deskclock
Flags: 0xc8be45
Package: com.android.deskclock v203 (2.0.3)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
android.database.sqlite.SQLiteException: no such column: incvol (code 1): , while compiling: SELECT _id, hour, minutes, daysofweek, alarmtime, enabled, vibrate, message, alert, incvol, profile FROM alarms WHERE (enabled=1)
...
        at com.android.deskclock.AlarmProvider.query(AlarmProvider.java:73)
        at android.content.ContentProvider.query(ContentProvider.java:744)
        at android.content.ContentProvider$Transport.query(ContentProvider.java:199)
        at android.content.ContentResolver.query(ContentResolver.java:414)
        at android.content.ContentResolver.query(ContentResolver.java:357)
        at com.android.deskclock.Alarms.getFilteredAlarmsCursor(Alarms.java:165)
        at com.android.deskclock.Alarms.calculateNextAlert(Alarms.java:344)
        at com.android.deskclock.Alarms.setNextAlert(Alarms.java:417)
        at com.android.deskclock.Alarms.saveSnoozeAlert(Alarms.java:510)
        at com.android.deskclock.AlarmInitReceiver$1.run(AlarmInitReceiver.java:49)
...

The media player fails with a NullPointerException:

Process: com.andrew.apollo:main
Flags: 0x98be65
Package: com.andrew.apollo v2 (1.1)
Build: samsung/apexqtmo/apexqtmo:4.1.2/JZO54K/T699UVBMC5:user/release-keys
java.lang.NullPointerException
        at com.andrew.apollo.MusicPlaybackService.stop(MusicPlaybackService.java:878)
        at com.andrew.apollo.MusicPlaybackService.openCurrentAndMaybeNext(MusicPlaybackService.java:1048)
        at com.andrew.apollo.MusicPlaybackService.openCurrentAndNext(MusicPlaybackService.java:1031)
        at com.andrew.apollo.MusicPlaybackService.access$1500(MusicPlaybackService.java:74)
        at com.andrew.apollo.MusicPlaybackService$MusicPlayerHandler.handleMessage(MusicPlaybackService.java:2337)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:137)
        at android.os.HandlerThread.run(HandlerThread.java:61)

Fixing the problem Maybe I could have worked out how to upgrade the relevant databases myself, but I went for simpler solutions. The clock settings are easy to re-enter and most of the media player state is regenerated by scanning files on the SD card. So I just deleted those on the Relay:

  rm -rf /data/data/com.android.deskclock
  rm -rf /data/data/com.android.providers.media

The contacts were what I really cared about, and there are actually specific menu items in Contacts to export and import those (using VCF format), side-stepping the database upgrade. So I did:

1. Insert SD card in Blade
2. Open Contacts and export to VCF (I forget where this is in the menus but it was easy to find)
3. Remove broken Contacts database on Relay:
   rm -rf /data/data/com.android.providers.contacts
4. Insert SD card in Relay
5. Move exported contacts to internal storage:
   mv /storage/sdcard1/*.vcf /storage/emulated/legacy
6. Open People and tap the menu key, 'Import/export', 'Import from storage', then the filename

This may not include all data, and in particular it doesn't seem to include attached photos. But that was good enough for me.

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (44.52 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. The Debian Administrator s Handbook Wheezy update completed. Roland and I completed the update of the Debian Administrator s Handbook for Debian 7 Wheezy. We still have some proofreading work to do but you can already enjoy the result here: http://debian-handbook.info/browse/wheezy/ Feel free to report back any problem that you discover. You can also submit us patches ready to apply if you want to go one step further. Publican contributions. The book is generated with publican and I maintain its Debian package. This month I got a release critical bug because it stopped working it turns out that the problem lied in libxml-treebuilder-perl and I thus reassigned #728885 while providing a tentative patch to the upstream author. After a few days without action from the pkg-perl team, and after having received a FTBFS bug on debian-handbook (of course publican was broken in unstable!), I prepared a fixed package myself and I uploaded it (I m still part of the pkg-perl team although I m inactive). Since I used publican heavily this month, I filed two tickets in its bugzilla. I requested a new feature in #1034836 (the possibility to keep around the former string for fuzzy strings to update), and I reported a problem with the handling of \n in PO files in #1036150. Debian France Galette update. I updated the galette package and its paypal plugin, and I deployed those on france.debian.net. It had some fixes for the reminder mails sent to members. Bylaws update. I also resumed my work on preparing new bylaws for Debian France. Sylvestre Ledru came up with a draft (with the help of a lawyer) a few months ago and I m reviewing/improving them now. The main goal is to clarify that Debian France is meant to be a Trusted Organization for the Debian project. Debian France Shop. We had the idea since a few months already but Sylvestre did the leg work to open a Debian France shop with the help of EnVenteLibre. I asked our members to prepare some CSS that better match the Debian colors and this should be fixed in a few days. The first goodies will also start to appear shortly, just in time for Christmas! Misc Debian work Distro Tracker. In the continuation of the Google Summer of Code, I asked the DSA team to setup a new virtual machine to host tracker.debian.org, an instance of Distro Tracker, the rewritten Package Tracking System. They have done their part of the job (except the mail setup), it s now waiting on me to find some time to complete some cleanups and deploy the thing. WordPress. I packaged wordpress 3.7.1 and sent a call for help on debian-mentors. I got 3 replies, I gave them some initial direction but I haven t heard back anything since. WordPress 3.8 is expected in a few days, hopefully one of the new volunteers will take care of preparing the next update. Dpkg regressions. I haven t done anything for multiple months but at least I keep running the git version of dpkg and I detected two regressions. Good to have them squashed before the upcoming 1.17.2 upload to unstable. PTS fix. I fixed some warnings that the PTS code started generating since the upgrade of its host to wheezy. They were generating some annoying backscatter mails to users of the pts@qa.debian.org bot. Ruby security update. I helped the ruby team to prepare the required security updates of ruby1.8 and ruby1.9.1 (see #730178 and #730189). This work was sponsored by Kali/Offensive Security. Smartcard setup. I bought 2 OpenPGP smartcards with a reader and I moved all my private keys on those devices (one card with the master key for signature/certification to be kept at home, one card for daily/mobile usage with the subkeys for encryption/signature/authentication). My laptop s harddrive doesn t contain any private key anymore. I have kept the required offline backup in a safe place, but in the end, my private keys are much harder to steal. I should write down my findings in another article Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

I recently bought a Cubieboard to replace my old Sheevaplug which has finally blown a power supply capacitor (this appears to be a common defect of Sheevaplugs), so I am publishing these instructions which show how to install Debian on sunxi systems (i.e. based on the Allwinner A10 SoC or one of its newer versions) with no need for cross compilers, emulators or ugly FAT partitions. This should work on any sunxi system as long as U-Boot is at least version 2012.10. The first step is to erase the beginning of SD card to remove anything in the unpartitioned space which may confuse U-Boot, partition and format it as desired. The first partition must begin at 1MB (1024*1024/512=2048 sectors) because the leading unpartitioned space is used by the boot loaders.

dd if=/dev/zero of=/dev/mmcblk0 bs=1M count=1
parted /dev/mmcblk0
  mklabel msdos
  mkpart primary ext4 2048s 15G
  unit s
  print
  mkpart primary linux-swap ... -1
mkfs.ext4 -L root /dev/mmcblk0p1
mkswap --label swap /dev/mmcblk0p2

Download the boot loaders and an initial kernel and install them:

tar xf cubieboard_hwpack.tar.xz
dd if=bootloader/sunxi-spl.bin of=/dev/mmcblk0 bs=1024 seek=8
dd if=bootloader/u-boot.bin of=/dev/mmcblk0 bs=1024 seek=32
mount /dev/mmcblk0p1 /mnt
mkdir /mnt/boot/
cp kernel/script.bin kernel/uImage /mnt/boot/

script.bin is Allwinner's proprietary equivalent of the device tree: it will be needed until sunxi support will be fully merged in mainline kernels. U-Boot needs to be configured to load the kernel from the ext4 file system (join the lines at \\, this is not a supported syntax!):

cat << END > /mnt/boot/uEnv.txt
# kernel=uImage
root=/dev/mmcblk0p1 rootwait
boot_mmc=ext4load mmc 0:1 0x43000000 boot/script.bin && ext4load mmc 0:1 0x48000000 boot/$ kernel  \\
  && watchdog 0 && bootm 0x48000000
END

Now the system is bootable: add your own root file system or build one with debootstrap. My old Sheevaplug tutorial shows how to do this without a working ARM system or emulator (beware: the other parts are quite obsolete and should not be trusted blindly). If you have an old armel install around it will work as well, and you can easily cross-grade it to armhf as long as it is up to date to at least wheezy (the newer, the better). You can also just use busybox for a quick test:

mkdir /mnt/bin/
dpkg-deb -x .../busybox-static_1.21.0-1_armhf.deb .
cp bin/busybox /mnt/bin/
ln -s busybox /mnt/bin/sh

After booting the busybox root file system you can run busybox --install /bin/ to install links for all the supported commands. Until Debian kernels will support sunxi (do not hold your breath: there are still many parts which are not yet in mainline) I recommend to install one of Roman's kernels:

dpkg -i linux-image-3.4.67-r0-s-rm2+_3.4.67-r0-s-rm2+-10.00.Custom_armhf.deb
mkimage -A arm -O linux -T kernel -C none -a 40008000 -e 40008000 \
  -n uImage -d /boot/vmlinuz-3.4.67-r0-s-rm2+ /boot/uImage-3.4.67-r0-s-rm2+

It is not needed with these kernels for most setups, but an initramfs can be created with:

update-initramfs -c -k 3.4.67-r0-s-rm2+
mkimage -A arm -T ramdisk -C none -n uInitrd \
  -d /boot/initrd.img-3.4.67-r0-s-rm2+ /boot/uInitrd-3.4.67-r0-s-rm2+

/boot/uEnv.txt will have to be updated to load the initramfs. Since the Cubieboard lacks a factory-burned MAC address you should either configure one in script.bin or (much easier) add it to /etc/network/interfaces:

iface eth0 inet dhcp
        hwaddress ether xx:xx:xx:xx:xx:xx

To learn more about the Allwinner SoCs boot process you can consult 1 and 2.

Blogging from a phone... first time ever. I don't want to forget some specifics for this :) I have just completed an exam to try to enter a postgraduate program (I'll talk more about it once it becomes real ). The exam is administered by CENEVAL, the same evaluation agency Where I presented my graduation equivalency exam some years ago - Only this exam is for all of the postgraduate studies on many national universities and is thus basically just a psychometric test. The exam had 162 questions, all to be filled in a optical reader sheet, on five subjects: mathematical reasoning, Spanish grammar and comprehension, Project management, Computers and technology, And English reading and understanding. It was all in all a fun exam to take, mostly due to the math reasoning part. But... on the Subject I Know I am an expert, I have to complain (and intend to find a easy to do so formally). First, I spotted two absolute mistakes (and answered based on What I knew others would, but knowing the answer is wrong technically). One was a subtlety, on how and why have hard drives should be defragmented (and part of my quip is that it's an obsolete habit, but besides, the answers were all erroneous), but a second one was... just wrong. It asked on what should not be part of an "Internet link" (can only guess they meant An URL). The 4 options were valid parts of a URL - including one very seldom used by most people, but very often by many of us: the @ sign. Anyway, answered it, but my other main gripe is that most of the section was in specific use of Office software. Not only In Office-like, which would be bad enough to begin with, but on specific ways of using Mainly Excel And PowerPoint. Syntax issues, or the name of the menu under which to look for specific functions.
Anyway, I will wait the stipulated 10 days for the exam to be rated, but will anyway look die a way to contact the very opaque and secretive CENEVAL. Not to demand to be better treated, but to try to correct those known mistakes and errors.

Backups are fun and exciting! Restores are exciting, in a terrifying and stressful way. Fixing serious bugs is gratifying, in a depressing way. I am glad to announce two software releases: Obnam version 1.6.1 and larch version 1.20131130. Obnam is my backup application, larch is a Python copy-on-write B-tree implementation that Obnam uses. I've uploaded the new versions to Debian unstable and to my own apt repository at [code.liw.fi] (also for wheezy). (I don't have the energy to upload to Debian backports: help is welcome there.) These are mainly bug fix releases, and minor improvements. The larch release fixes serious problems, and everyone should be upgrading. NEWS for larch:

• Serious bug fixed: the "KeyError" crash for reference counts. This was false memory use optimisation, which triggered a rare bug in related code. Repeatable test case by Rob Kendrick, and helpful analysis by Itamar Turing-Trauring.
• Serious bug fixed: another "node missing" bug. This crash was caused by a bug that overwrote on-disk reference count groups with zeroes. Repeatable test case by Rob Kendrick.
• Fixes to fsck from Antoine Brenner.

NEWS for Obnam (combining versions 1.6 and 1.6.1, both released today):

• Fix Debian package dependencies correctly.
• Stop logging paramiko exceptions that get converted into another type of exception by the SFTP plugin in Obnam.
• obnam-benchmark can now use an installed version of larch. Patch by Lars Kruse.
• Obnam has been ported to FreeBSD by Itamar Turner-Trauring of HybridCluster.
• Backup progress reporting now reports scanned file data, not just backed up file data. This will hopefully be less confusing to people.
• The list-keys, client-keys, and list-toplevels commands now obey a new option, --key-details, to show the usernames attached to each public key. Patch by Lars Kruse.
• New option --ssh-command to set the command Obnam runs when invoking ssh. patch by Lars Kruse.
• obnam clients can now be used without being an existing client. Patch by Itamar Turner-Trauring.
• New option --ssh-host-keys-check to better specify how SSH host keys should be checked. Patch by Itamar Turner-Trauring.

Bug fixes:

• Fix"obnam list-toplevels so it doesn't give an error when it's unable to read the per-client directory of another client, when encryption is used. Fix by Lars Kruse.
• Fix the encryption plugin to give a better error message when it looks for client directories but fails to find them. Fix by Lars Kruse.
• obnam list-toplevels got confused when the repository contained extra files, such as "lock" (left there by a previous, crashed Obnam run). It no longer does. Fix by Lars Kruse.
• The SFTP plugin now handles another error code (EACCESS) when writing a file and the directory it should go into not existing. Patch by Armin Gr linger.
• Obnam's manual page now explains about breaking long logical lines into multiple physical ones.
• The /~/ path prefix in SFTP URLs works again, at least with sufficiently new versions of Paramiko (1.7.7.1 in Debian wheezy is OK). Reported by Lars Kruse.
• The Nagios plugin to report errors in a way Nagios expects. Patch by Martijn Grendelman.
• The Nagios plugin for Obnam now correctly handles the case where a backup repository for a client exists, but does not have a backup yet. Patch by Lars Kruse.
• obnam ls now handles trailing slashes in filename arguments. Reported by Biltong.
• When restoring a backup, Obnam will now continue past errors, instead of aborting with the first one. Patch by Itamar Turner-Trauring.

Right on the heels of release 0.2.1 of RcppCNPy, a new version 0.2.2 is now on CRAN. RcppCNPy uses the CNPY library by Carl Rogers to provide R with easy read and write access to NumPy files. The reason for the new version that I had experimented with a different way to test endianness (as needed for the NumPy file headers) but accidentally sent an interim tarball to CRAN which still wanted to include endian.h promptly breaking Windows builds. So now we do something even simpler and just rely on the (even more complete) test for endianness when R is built, which prevents all sorts of complications for us and builds everywhere (with thanks to Brian Ripley for the suggestion). While we were at it, we also added a new unit test. Full changes are listed below.

Changes in version 0.2.2 (2013-11-29)

• Switched to using the result from the compile-time configuration for R to determine big or little endian (as needed for the NPy headers)
• Added a new test (and test validation result file) for a complete save-reload cycle and comparison

CRANberries also provides a diffstat report for 0.2.2 relative to 0.2.1. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Following up on my port of the crystalhd plugin to the gstreamer 1.0 api I realized that the CrystalHD repo is pretty dormant. After reading slomo's nice article about GStreamer and hardware integration and a short off list mail exchange I decided to split the GStreamer part out of the CrystalHD repo and to try to get the plugin into gst-plugins-bad. Since the kernel part is already in linux kernel's staging area there would not be much left in the repo except for the libcrystalhd library itself and the firmware blobs. So I split them out as well and started to clean them up a bit by moving it to autoconf/automake, dropping the need for a C++ compiler and adding symbol versioning among other things. So up to know video is still smooth with:

gst-launch-1.0 filesrc location=sample.mp4 ! decodebin ! xvimagesink

after jhbuilding up to gst-plugins-bad. There are #ifdefs for macosx and windows but I doubt they're functional but in case anybody is building libcrystalhd on these these platforms it'd be great to know if it still works. Should these efforts lead to the crystalhd plugin being merged into GStreamer getting the kernel driver out of staging would be a great next step. This blog is flattr enabled.

If you want the ability to electronically communicate directly with your neighbors and friends using a network controlled by your peers in stead of centrally controlled by a few corporations, or would like to experiment with interesting network technology, the Dugnasnett for alle i Oslo might be project for you. 39 mesh nodes are currently being planned, in the freshly started initiative from NUUG and Hackeriet to create a wireless community network. The work is inspired by Freifunk, Athens Wireless Metropolitan Network, Roofnet and other successful mesh networks around the globe. Two days ago we held a workshop to try to get people started on setting up their own mesh node, and there we decided to create a new mailing list dugnadsnett (at) nuug.no and IRC channel #dugnadsnett.no to coordinate the work. See also the NUUG blog post announcing the mailing list and IRC channel.

Shanley wrote an insightful article about microagressions and management [1]. It s interesting to read that and think of past work experiences, even the best managers do it. Bill Stone gave an inspiring TED talk about exploring huge caves, autonamous probes to explore underground lakes (which can be used on Europa) and building a refuelling station on the Moon [2]. Simon Lewis gave an interesting TED talk about consciousness and the technology needed to help him recover from injuries sustained in a serious car crash [3]. Paul Wayper wrote an interesting article about reforming the patent system [4]. He also notes that the patent system is claimed to be protecting the mythical home inventor when it s really about patent trolls (and ex-inventors who work for them). This is similar to the way that ex-musicians work for organisations that promote extreme copyright legislation. Amanda Palmer gave an interesting TED talk about asking for donations/assistance, and the interactions between musicians and the audience [5]. Some part of this are NSFW. Hans Rakers wrote a useful post about how to solve a Dovecot problem with too many files open [6]. His solution was for a Red Hat based system, for Debian you can do the same but by editing /etc/init.d/dovecot. The use of the /proc/N/limits file was interesting, I ve never had a cause to deliberately use that file before. Krebs on Security has an interesting article about Android malware being used to defeat SMS systems to prevent bank fraud [7]. Apparently an infected PC will instruct the user to install an Android app to complete the process. Rick Falkvinge wrote an interesting article about how to apply basic economics terminology to so-called Intellectual Property [8]. Matthew Garrett wrote an interesting post about the way that Ubuntu gets a better result than Debian and Fedora because it has clear fixed goals [9]. He states that many people regard Fedora as a playground to produce a range of niche derivatives , probably a large portion of the Fedora and Debian developers consider this a feature not a bug. Ming Thein wrote an interesting article about the demise of the DSLR [10]. Bruce Schneier wrote an interesting post on the detention of David Miranda by the British authorities [11]. It s mostly speculation as to why they would do such a thing (which seems to go against their own best interests) and whether the NSA even knows which documents Edward Snowden copied. Jaclyn Friedman wrote an interesting article on Mens Rights Movements (MRAs) and how they are bad for MEN as well as for women [12]. Rodney S. Tucker wrote an insightful article for the IEEE about the NBN [13]. Basically the Liberal party are going to spend most of the tax money needed for a full NBN but get a significantly less than the full benefit. Lauren Drell wrote an interesting article for Mashable about TellSpec, a portable spectrometer that communicates with an Android phone to analyse food for allergens [14]. I guess this will stop schools from banning phones. Katie McDonough wrote an interesting article for Salon about the Pope s statements about the problems with unchecked capitalism [15]. His ideas are really nothing new to anyone who has read the Bible and read the news. It seems to me that the most newsworthy part of this is that most Christian leaders don t make similar statements. Daniel Leidert wrote an interesting post about power saving when running Debian on a HP Microserver [16]. Most of it is relevant to other AMD64 hardware too, I ll have to investigate the PCIE ASPM and spin down options on some of my systems that are mostly idle.

• [1] https://medium.com/about-work/65d4740f7a2f
• [2] http://www.ted.com/talks/bill_stone_explores_the_earth_and_space.html
• [3] http://tinyurl.com/6zgcb3f
• [4] http://mabula.net/tbfw/2013/08/01
• [5] http://www.ted.com/talks/amanda_palmer_the_art_of_asking.html
• [6] http://tinyurl.com/k3xz9a8
• [7] http://krebsonsecurity.com/2013/08/a-closer-look-perkele-android-malware-kit/
• [8] http://tinyurl.com/orf9gds
• [9] http://mjg59.dreamwidth.org/27517.html
• [10] http://blog.mingthein.com/2013/08/24/the-demise-of-the-dslr/
• [11] https://www.schneier.com/blog/archives/2013/08/detaining_david.html
• [12] http://prospect.org/article/good-mens-rights-movement-hard-find
• [13] http://tinyurl.com/q49b9fa
• [14] http://mashable.com/2013/11/26/tellspec/
• [15] http://tinyurl.com/nosxkj2
• [16] http://www.wgdd.de/2013/08/hp-n54l-microserver-energy-efficiency.html

Today was my last day working at Bytemark, and I found it a lot harder than expected. For better or worse I finished earlier than expected; having been gradually removing my accounts and privileges over the past few weeks I'd revoked my OpenVPN key this morning. Mid-afternoon my openvpn connection tried to renegotiate session keys, or similar, and failed. So I stopped work a few hours early. That meant I managed to avoid sending my "goodbye world" email, which is probably for the best - after all a lovely company, lovely people, and a good environment, what can you say besides things that are lovely? I think I largely wrapped things up neatly, and I'm pleased that one of my photos is hanging on the office wall. (I look forward to seeing that actually, I've only rarely made canvas prints.) The only other thing of note this week has been the sharp rise in blogspam I've detected. Black Friday alive and well, on the internets ..

Axel just said that PDiffs are really useful for stable because it changes seldom. The truth is that there are no PDiffs for stable. I consider this a bug because you need to download a huge blob on every point release even though only few stanzas changed.

The real problem of PDiffs is the count that needs to be applied. We run "dinstall" much more often now than we used to and we cannot currently skip diffs in the middle when looking at a dak-maintained repository. It would be more useful if you could fetch a single diff from some revision to the current state and apply this instead of those (download, apply, hash check)+ cycles. 56 available index diffs are also not particularly helpful in this regard. They cover 14 days with four dinstalls each, but it takes a very long time to apply them, even on modern machines.

I know that there was ongoing work improving PDiffs that wanted to use git to generate the intermediate diffs. The current algorithm used in dak does not require old versions except one to be kept, but keeping everything is the far other end of the spectrum where you have a git repository that potentially gets huge unless you rewrite it. It should be possible to just implement a rolling ring buffer of Packages files to diff against, reprepro already generates the PDiffs in the right way, the Index format is flexible enough to support this. So someone would just need to make that bit of work in dak.

This is one thing I have wanted to do for a long time: configure Mutt to encrypt all messages sent to addresses for which I have a valid public key. Well, here is an awk-based script to generate that configuration. (Yes, I know, a similar script was already written. But I did not see it at first, and I find mine more readable. :-) )Code

#! /bin/sh
set -e
# Specific hooks set global configuration values for Mutt, which are not
# reset for new messages, so a default hook is needed to set the default
# value (do not try encrypting all messages).
printf "send-hook . unset crypt_autoencrypt\n\n"
cat <<EOF
# This group contains all known PGP users, and can be used to define
# hooks, for instance to encrypt all messages sent to PGP users, or to
# add a specific signature to promote PGP usage to non-PGP users.
EOF
# Generate a machine-readable list of known keys
gpg2 --batch --with-colons --list-keys \
      awk -- '
        BEGIN  
            FS = ":"
            # A flag that indicates whether or not we are in a public
            # key with usable encryption capability
            usable = 0
         
        # Detect beginning of public keys with usable encryption
        # capability
        $1 == "pub" && $12 ~ /E/   usable = 1  
        # Detect beginning of public keys without usable encryption
        # capability
        $1 == "pub" && $12 !~ /E/   usable = 0  
        
        # Only consider user IDs from public keys with usable encryption
        # capability and with regular, marginal, full or ultimate
        # validity (n, m, f or u in the second field)
        usable && $1 == "uid" && $2 ~ /[nmfu]/ && $10 ~ /@/  
            # Remove everything before and after the email address in
            # brackets
            sub(/^[^<]*</, "", $10)
            sub(/>[^>]*$/, "", $10)
            # Finally print the email address
            print $10
         ' \
      sort -u \
       
        # Group addresses by ten to reduce the number of Mutt commands
        finished=0
        while [ $finished -ne 1 ]
        do
            addresses=""
            for i in 0 1 2 3 4 5 6 7 8 9
            do
                if read address
                then
                    addresses="$addresses $address"
                else
                    finished=1
                    break
                fi
            done
            # Add PGP users' addresses to a pgp-users group
            printf "group -group pgp-users -addr %s\n" \
                "$addresses"
        done
     
# Build a hook to try encrypting messages sent to PGP users
printf "\n# Encrypt all messages sent to known PGP users\n"
printf "send-hook \"%%C pgp-users\" set crypt_autoencrypt\n" \
printf "\n# Add a specific signature to promote PGP usage to non-PGP users\n"
printf "#send-hook \"!%%C pgp-users\" set signature=~/.signature.use-pgp\n"

Usage Script usage This script generates uses GnuPG to print a list of all known keys, takes only valid ones, and uses them to build a series of configuration instructions for Mutt, that define hooks that are triggered when you write a message to them. This configuration has to be written to your Mutt configuration file:

$ bin/gen-pgp-hooks.sh >> ~/.muttrc

Or better (far better, in fact), to a dedicated configuration file you will refer to in your main one:

$ bin/gen-pgp-hooks.sh > ~/.muttrc.crypt-recipients
$ echo "source ~/.muttrc.crypt-recipients" >> ~/.muttrc

You may want to adapt this script to it directly writes to that file rather than having to redirect it. Also, remember to run it when you refresh your public keyring or add new keys to it. Mutt usage These hooks will make Mutt try to encrypt all messages you send to at least one known PGP user. That means it will try it even for messages you send to a PGP user and to non-PGP ones. In these cases, it will not find keys for all the recipient, and ask you which key to use; as you have none, you will have to quit that prompt by typing ^G, and manually disable encryption by accessing the PGP options menu with the key p.

Back on Black (Friday) Event Altus Metrum is pleased to announce our Back on Black (Friday) event! For the first time since the Black Forest fire in June, we re re-opening our web store this weekend with a host of new and classic Altus Metrum products, including a special pre-order discount on our latest-and-greatest flight computer design, TeleMega. This weekend only, Friday, 29 November 2013 through Monday, 2 December, 2013, the first 40 TeleMega direct orders placed through our web store will receive a special $50 pre-order discount (regular $400, now only $350!). TeleMega is an advanced flight computer with 9-axis IMU, 6 pyro channels, uBlox Max 7Q GPS and 40mW telemetry system. We designed TeleMega to be the ideal flight computer for sustainers and other complex projects. TeleMega production is currently in process, and we expect to be ready to ship in mid-December. Pre-order now and we won t charge you until we ship. Learn more about TeleMega at: http://altusmetrum.org/TeleMega/ We are also pleased to announce that TeleBT is back in stock. Priced at $150, TeleBT is our latest ground station that connects to your laptop over USB or your Android device over BlueTooth. Learn more about TeleBT at http://altusmetrum.org/TeleBT/ Another new product we re thrilled to announce is EasyMini! Priced at only $80, EasyMini is a two-channel flight computer with built-in data logging and USB data download. Like our more advanced flight computers, EasyMini is loaded with sophisticated electronics and firmware, designed to be very simple to use yet capable enough for high performance airframes. Perfect as a first flight computer, EasyMini is also great as a backup deployment controller in complext projects. Learn more about EasyMini at: http://altusmetrum.org/EasyMini/ Also in stock for immediate shipment is MicroPeak, our 1.9 gram recording altimeter available for $50. The MicroPeak USB adapter, also $50, has been improved to make data downloading a snap. Read more about these at: http://altusmetrum.org/MicroPeak http://altusmetrum.org/MicroPeakUSB You can learn more about these and all our other Altus Metrum products at http://altusmetrum.org. The special discount on TeleMega pre-orders is available only on orders placed directly through Bdale s web store at http://shop.gag.com Thank you all for your support of Altus Metrum during 2013. It s been a rough year, but we re having a great time updating our existing products and designing new stuff! We look forward to returning products like TeleMetrum and TeleMini to the market soon, and plan to introduce even more new products soon.