Search Results: "reg"

12 July 2014

Dirk Eddelbuettel: RcppArmadillo 0.4.320.0

While I was out at the (immensely impressive and equally enjoyable) useR! 2014 conference at UCLA, Conrad provided a bug-fix release 4.320 of Armadillo, the nifty templated C++ library for linear algebra. I quickly rolled that into RcppArmadillo release 0.4.320.0 which has been on CRAN and in Debian for a good week now. This release fixes some minor things with sparse and dense Eigen solvers (as well as one RNG issue probably of lesser interest to R users deploying the RNGs from R) as shown in the NEWS entry below.
Changes in RcppArmadillo version 0.4.320.0 (2014-07-03)
  • Upgraded to Armadillo release Version 4.320 (Daintree Tea Raider)
    • expanded eigs_sym() and eigs_gen() to use an optional tolerance parameter
    • expanded eig_sym() to automatically fall back to standard decomposition method if divide-and-conquer fails
    • automatic installer enables use of C++11 random number generator when using gcc 4.8.3+ in C++11 mode
Courtesy of CRANberries, there is also a diffstat report for the most recent release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

11 July 2014

Russ Allbery: Net::Duo 1.00

This is the first release of Net::Duo, which provides an object-oriented Perl interface for the Duo Security REST APIs. It attempts to abstract some of the API details and provide an object-oriented view of the returned objects in order to make use of the API in Perl code more natural than dealing with JSON data structures directly. Currently, some parts of the Auth and Admin APIs are implemented alongside with generic methods to call any of the JSON-based APIs. The approach I took with this module was a bit of a science experiment, and I'm still not entirely sure what I think about the results. Duo Security offers sample Perl code that provides the equivalent of the call and call_json Net::Duo methods but stops there. One sends in data structures and gets back data structures from JSON and manipulates everything in that format. I prefer a more object-oriented style, and want the module to do a bit more of the work for me, so this implementation wraps some of the APIs in objects with method calls. For updates, there are setters for the object itself and then a commit method to push the changes to Duo. This requires more implementation effort, and each API that should get richer treatment has to be modelled, but the resulting code looks like more natural object-oriented code. I wasn't completely sure going in if the effort to reward tradeoff made sense, and having finished the module sufficiently for Stanford's immediate needs, I'm still not sure. It was certainly more effort to write the base module this way, but on the other hand it also meant that I could map Perl notions of true and false to Duo's and provide much simpler methods for common operations. I still think this will make the code more maintainable in the long run, but I think it's within the margin of difference of opinion. Regardless, you can get the latest version from the Net::Duo distribution page and shortly from CPAN as well.

Russell Coker: Improving Computer Reliability

In a comment on my post about Taxing Inferior Products [1] Ben pointed out that most crashes are due to software bugs. Both Ben and I work on the Debian project and have had significant experience of software causing system crashes for Debian users. But I still think that the widespread adoption of ECC RAM is a good first step towards improving the reliability of the computing infrastructure. Currently when software developers receive bug reports they always wonder whether the bug was caused by defective hardware. So when bugs can t be reproduced (or can t be reproduced in a way that matches the bug report) they often get put in a list of random crash reports and no further attention is paid to them. When a system has ECC RAM and a filesystem that uses checksums for all data and metadata we can have greater confidence that random bugs aren t due to hardware problems. For example if a user reports a file corruption bug they can t repeat that occurred when using the Ext3 filesystem on a typical desktop PC I ll wonder about the reliability of storage and RAM in their system. If however the same bug report came from someone who had ECC RAM and used the ZFS filesystem then I would be more likely to consider it a software bug. The current situation is that every part of a typical PC is unreliable. When a bug can be attributed to one of several pieces of hardware, the OS kernel and even malware (in the case of MS Windows) it s hard to know where to start in tracking down a bug. Most users have given up and accepted that crashing periodically is just what computers do. Even experienced Linux users sometimes give up on trying to track down bugs properly because it s sometimes very difficult to file a good bug report. For the typical computer user (who doesn t have the power that a skilled Linux user has) it s much worse, filing a bug report seems about as useful as praying. One of the features of ECC RAM is that the motherboard can inform the user (either at boot time, after a NMI reboot, or through system diagnostics) of the problem so it can be fixed. A feature of filesystems such as ZFS and BTRFS is that they can inform the user of drive corruption problems, sometimes before any data is lost. My recommendation of BTRFS in regard to system integrity does have a significant caveat, currently the system reliability decrease due to crashes outweighs the reliability increase due to checksums at this time. This isn t all bad because at least when BTRFS crashes you know what the problem is, and BTRFS is rapidly improving in this regard. When I discuss BTRFS in posts like this one I m considering the theoretical issues related to the design not the practical issues of software bugs. That said I ve twice had a BTRFS filesystem seriously corrupted by a faulty DIMM on a system without ECC RAM.

10 July 2014

Steve Kemp: Blogspam moved, redis alternatives being examined

As my previous post suggested I'd been running a service for a few years, using Redis as a key-value store. Redis is lovely. If your dataset will fit in RAM. Otherwise it dies hard. Inspired by Memcached, which is a simple key=value store, redis allows for more operations: using sets, using hashes, etc, etc. As it transpires I mostly set keys to values, so it crossed my mind last night an alternative to rewriting the service to use a non-RAM-constrained service might be to juggle redis out and replace it with something else. If it were possible to have a redis-compatible API which secretly stored the data in leveldb, sqlite, or even Berkley DB, then that would solve my problem of RAM-constraints, and also be useful. Looking around there are a few projects in this area nds fork of redis, ssdb, etc. I was hoping to find a Perl Redis::Server module, but sadly nothing exists. I should look at the various node.js stub-servers which exist as they might be easy to hack too. Anyway the short version is that this might be a way forward, the real solution might be to use sqlite or postgres, but that would take a few days work. For the moment the service has been moved to a donated guest and has 2Gb of RAM instead of the paltry 512Mb it was running on previously. Happily the server is installed/maintained by my slaughter tool so reinstalling took about ten minutes - the only hard part was migrating the Redis-contents, and that's trivial thanks to the integrated "slave of" support. (I should write that up regardless though.)

Russell Coker: A Linux Conference as a Ritual

Sociological Images has an interesting post by Jay Livingston PhD about a tennis final as a ritual [1]. The main point is that you can get a much better view of the match on your TV at home with more comfort and less inconvenience, so what you get for the price of the ticket (and all the effort of getting there) is participating in the event as a spectator. It seems to me that the same idea applies to community Linux conferences (such as LCA) and some Linux users group meetings. In terms of watching a lecture there are real benefits to downloading it after the conference so that you can pause it and study related web sites or repeat sections that you didn t understand. Also wherever you might sit at home to watch a video of a conference lecture you will be a lot more comfortable than a university lecture hall. Some people don t attend conferences and users group meetings because they would rather watch a video at home. Benefits of Attending (Apart from a Ritual) One of the benefits of attending a lecture is the ability to ask questions. But that seems to mostly apply to the high status people who ask most questions. I ve previously written about speaking stacks and my observations about who asks questions vs the number that can reasonably be asked [2]. I expect that most delegates ask no questions for the entire conference. I created a SurveyMonkey survey to discover how many questions people ask [3]. I count LCA as a 3 day conference because I am only counting the days where there are presentations that have been directly approved by the papers committee, approving a mini-conf (and thus delegating the ability to approve speeches) is different. Another benefit of attending is the so-called hallway track where people talk to random other people. But that seems to be of most benefit to people who have some combination of high status in the community and good social skills. In the past I ve attended the Professional Delegates Networking Session which is an event for speakers and people who pay the Professional registration fee. Sometimes at such events there has seemed to be a great divide between speakers (who mostly knew each other before the conference) and Professional Delegates which diminishes the value of the event to anyone who couldn t achieve similar benefits without it. How to Optimise a Conference as a Ritual To get involvement of people who have the ritualistic approach one could emphasise the issue of being part of the event. For example to get people to attend the morning keynote speeches (which are sometimes poorly attended due to partying the night before) one could emphasise that anyone who doesn t attend the keynote isn t really attending the conference. Conference shirts seem to be strongly correlated with the ritual aspect of conferences, the more corporate conferences don t seem to offer branded clothing to delegates. If an item of branded schwag was given out before each keynote then that would increase the attendance by everyone who follows the ritual aspect (as well as everyone who just likes free stuff). Note that I m not suggesting that organisers of LCA or other conferences go to the effort of giving everyone schwag before the morning keynote, that would be a lot of work. Just telling people that anyone who misses the keynote isn t really attending the conference would probably do. I ve always wondered why conference organisers want people to attend the keynotes and award prizes to random delegates who attend them. Is a keynote lecture a ritual that is incomplete if the attendance isn t good enough?

Paul Tagliamonte: Dell XPS 13

More hardware adventures. I got my Dell XPS13. Amazing. The good news: This MacBook Air clone is clearly an Air competitor, and easily slightly better in nearly every regard except for the battery.
The bad news is that the Intel Wireless card needs non-free (I ll be replacing that shortly), and the touchpad s driver isn t totally implemented until Kernel 3.16. I m currently building a 3.14 kernel with the patch to send to the kind Debian kernel people. We ll see if that works. Ubuntu Trusty already has the patch, but it didn t get upstreamed. That kinda sucks. It also shipped with UEFI disabled, and was defaulting to boot in legacy mode. It shipped with Ubuntu, a bit disappointed to not see Ubuntu keys on the machine. Touchscreen works; in short -stunning. I think I found my new travel buddy. Debian unstable runs great, stable had some issues.

6 July 2014

Ian Campbell: Setting absolute date based Amazon S3 bucket lifecycles with curl

For my local backup regimen I use flexbackup to create a full backup twice a year and differential/incremental backups on a weekly/monthly basis. I then upload these to a new amazon S3 bucket for each half year (so each bucket corresponds to the a full backup plus the associated differentials and incrementals). I then set the bucket's lifecycle to archive to glacier (cheaper offline storage) from the month after that half year has ended (reducing costs) and to delete it a year after the half ends. It used to be possible to do this via the S3 web interface but the absolute date based options seem to have been removed in favour of time since last update, which is not what I want. However the UI will still display such lifecycles if they are configured and directs you to the REST API to set them up. I had a look around but couldn't any existing CLI tools to do this directly but I figured it must be possible with curl. A little bit of reading later I found that it was possible but it involved some faff calculating signatures etc. Luckily EricW has written Amazon S3 Authentication Tool for Curl (AKA s3curl) which automates the majority of that faff. The tool is "New BSD" licensed according to that page or Apache 2.0 license according to the included LICENSE file and code comments. Setup Following the included README setup ~/.s3curl containing your id and secret key (I called mine personal which I then use below). Getting the existing lifecycle Retrieving an existing lifecycle is pretty easy. For the bucket which I used for the first half of 2014:
$ s3curl --id=personal -- --silent http://$bucket.s3.amazonaws.com/?lifecycle   xmllint --format -
<?xml version="1.0" encoding="UTF-8"?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Rule>
    <ID>Archive and Expire</ID>
    <Prefix/>
    <Status>Enabled</Status>
    <Transition>
      <Date>2014-07-31T00:00:00.000Z</Date>
      <StorageClass>GLACIER</StorageClass>
    </Transition>
    <Expiration>
      <Date>2015-01-31T00:00:00.000Z</Date>
    </Expiration>
  </Rule>
</LifecycleConfiguration>
See GET Bucket Lifecycle for details of the XML. Setting a new lifecycle The desired configuration needs to be written to a file. For example to set the lifecycle for the bucket I'm going to use for the second half of 2014:
$ cat s3.lifecycle
<LifecycleConfiguration>
  <Rule>
    <ID>Archive and Expire</ID>
    <Prefix/>
    <Status>Enabled</Status>
    <Transition>
      <Date>2015-01-31T00:00:00.000Z</Date>
      <StorageClass>GLACIER</StorageClass>
    </Transition>
    <Expiration>
      <Date>2015-07-31T00:00:00.000Z</Date>
    </Expiration>
  </Rule>
</LifecycleConfiguration>
$ s3curl --id=personal --put s3.lifecycle --calculateContentMd5 -- http://$bucket.s3.amazonaws.com/?lifecycle
See PUT Bucket Lifecycle for details of the XML.

Matthew Palmer: Witness the security of this fully DNSSEC-enabled zone!

After dealing with the client side of the DNSSEC puzzle last week, I thought it behooved me to also go about getting DNSSEC going on the domains I run DNS for. Like the resolver configuration, the server side work is straightforward enough once you know how, but boy howdy are there some landmines to be aware of. One thing that made my job a little less ordinary is that I use and love tinydns. It s an amazingly small and simple authoritative DNS server, strong in the Unix tradition of do one thing and do it well . Unfortunately, DNSSEC is anything but small and simple and so tinydns doesn t support DNSSEC out of the box. However, Peter Conrad has produced a patch for tinydns to do DNSSEC, and that does the trick very nicely. A brief aside about tinydns and DNSSEC, if I may Poor key security is probably the single biggest compromise vector for crypto. So you want to keep your keys secure. A great way to keep keys secure is to not put them on machines that run public-facing network services (like DNS servers). So, you want to keep your keys away from your public DNS servers. A really great way of doing that would be to have all of your DNS records somewhere out of the way, and when they change regenerate the zone file, re-sign it, and push it out to all your DNS servers. That happens to be exactly how tinydns works. I happen to think that tinydns fits very nicely into a DNSSEC-enabled world. Anyway, back to the story. Once I d patched the tinydns source and built updated packages, it was time to start DNSSEC-enabling zones. This breaks down into a few simple steps:
  1. Generate a key for each zone. This will produce a private key (which, as the name suggests, you should keep to yourself), a public key in a DNSKEY DNS record, and a DS DNS record. More on those in a minute. One thing to be wary of, if you re like me and don t want or need separate Key Signing and Zone Signing keys. You must generate a Key Signing key this is a key with a flags value of 257. Doing this wrong will result in all sorts of odd-ball problems. I wanted to just sign zones, so I generated a Zone Signing key, which has a flags value of 256. Big mistake. Also, the DS record is a hash of everything in the DNSKEY record, so don t just think you can change the 256 to a 257 and everything will still work. It won t.
  2. Add the key records to the zone data. For tinydns, this is just a matter of copying the zone records from the generated key into the zone file itself, and adding an extra pseudo record (it s all covered in the tinydnssec howto).
  3. Publish the zone data. Reload your BIND config, run tinydns-sign and tinydns-data then rsync, or do whatever it is PowerDNS people do (kick the database until replication starts working again?).
  4. Test everything. I found the Verisign Labs DNSSEC Debugger to be very helpful. You want ticks everywhere except for where it s looking for DS records for your zone in the higher-level zone. If there are any other freak-outs, you ll want to fix those because broken DNSSEC will take your domain off the Internet in no time.
  5. Tell the world about your DNSSEC keys. This is simply a matter of giving your DS record to your domain registrar, for them to add it to the zone data for your domain s parent. Wherever you d normally go to edit the nameservers or contact details for your domain, you probably want to do to the same place and look for something about DS or Domain Signer records. Copy and paste the details from the DS record in your zone into there, submit, and wait a minute or two for the records to get published.
  6. Test again. Before you pat yourself on the back, make sure you ve got a full board of green ticks in the DNSSEC Debugger. if anything s wrong, you want to rollback immediately, because broken DNSSEC means that anyone using a DNSSEC-enabled resolver just lost the ability to see your domain.
That s it! There s a lot of complicated crypto going on behind the scenes, and DNSSEC seems to revel in the number of acronyms and concepts that it introduces, but the actual execution of DNSSEC-enabling your domains is quite straightforward.

3 July 2014

Ian Donnelly: Hello Planet Debian!

Hi Planet Debian! My name is Ian Donnelly. I am going into my Senior Year at Florida State University this fall but for the summer I am working on a Google Summer of Code project with Debian. My mentor is Markus Raab and I am working with the Elektra team to implement a great new feature for Debian! I am spending this summer working on a system to allow automatic conffile merging by utilizing the Elektra Framework. Elektra is designed to provide a universal and secure framework to store configuration parameters in a global, hierarchical key database. For this project, conffiles will be mounted to the Elektra framework as KeySets. I have written code that performs a three-way merge on these KeySets which is much more robust than merging individual lines. A conffile can easily be mounted to the Key database using the kdb mount command, which I detailed in a previous blog post. Elektra has many included plugins which allow all types of conffiles to be mounted to the Key database. With our newest release, we even added an augeas plugin which will provide compatibility for most of the major conffiles. Once a conffile is mounted, any changes to the Elektra KeySet it is contained in will be updated in the actual conffile and visa-versa. So far this summer, I have written code for a three-way merge of Elektra KeySets and we have begun to look at the next step of implementing a automatic merge into dpkg. We are currently working on identifying exactly how the merge should take place and where we need to modify dpkg. The basic idea of what will change is that: Once dpkg has been patched, we will modify some packages and thoroughly test the new merge operation. It is designed to be non-destructive and to not infringe with any of the advantages of the current system. Package maintainers would need to opt in to using this feature to insure that all packages get tested before using it. However, we have designed this system to require minimal modification in order for maintainers to utilize it. This project will benefit Debian by removing a large pain-point with users. For experienced uses, merging changes in conffiles can be a tedious process. For newer users, the current system can be downright confusing. The goal of our project is to make Debian easier to use for everybody while not loosing the features that make it such a powerful platform. Please let us know what you think of the project or if you have any questions. Sincerely,
Ian S. Donnelly

2 July 2014

Osamu Aoki: debmake -- checking source against DEP-5 copyright

It has been already more than a year after my initial uploading of the debmake package which offers semi-automatic generation of the DEP-5 copyright file.

One recent feature addition is the "debmake -k" command which audits debian/copyright contents against the latest source. If, for example, the upstream changed the license from GPL-2.0+ to 3,0+, this command will tell you this change. (In the debian/copyright, the more specific entry should be listed after the generic entry since parser uses the last definition as the valid one.)

I admit that this package had some regressions in some previous versions. It is now a stable tool to help making multiarch aware Debian packages of any types. If you had negative experiences, please try this again.

Let me quote from its documentation for the features. (Also available in the package.)
The debmake command is intended to replace functions offered historically by deb-make and dh_make commands. Its features include:
  • use of dh syntax under the new debhelper (> 9.0) package
    • extensive check of copyright for DEP-5 (debian/copyright)
    • substvar supports for binary packages (debian/control)
    • support of compiler hardening options (debian/rules)
  • keep pre-existing Debian package configuration files untouched
    • automatic generation of the missing template packaging files
    • easy verification of the debian/copyright file against the current source. (-k option)
  • easy packaging command line UI supporting
    • non-stop execution with clean results
    • direct operation on the tarball archive
    • direct operation on the source tree from VCS
    • the multiarch Debian package
    • the multi binary Debian package
    • the non-native Debian packages from the VCS snapshot
    • seamless work with debuild, pdebuild, etc.
Note
I wrote this debmake command because there was no easy command like "python setup.py bdist_deb" to create the Debian binary package. Now "debmake -d -s -b":python" -i debuild" does the job for me.
This documentation comes with detailed packaging examples, too.

1 July 2014

Russell Coker: Blog Comments

John Scalzi wrote an insightful post about the utility of blog comments with the way the Internet works nowadays [1]. He starts out focusing on hate comments that could reasonably be described as terrorism (death threats with the aim of preventing people writing about politics meet any reasonable definition of terrorism ). Terrorists on the Internet are a significant problem but it s one that doesn t get much attention as it generally only affects people who aren t straight-acting white men. Blogging About Technology One corner case that John doesn t seem to consider is that of writing about technology. Issues related to programming often aren t related to politics and are often testable so comments will be based on things that have been shown to work rather than stuff people invent or want to believe. I ve received many useful and educational comments on my technical posts with little hostility. Even getting a snarky comment is rare when writing a strictly technical blog post. The comments problem for technology blogging is spam. I ve been using the WordPress plugin Block Spam by Math [2] (which is obsolete but still works) for years. Initially it stopped almost all spam, but now I m getting at least 20 spam comments a day. Extremely Popular Blogs The comments section of a blog is sometimes described as a conversation . When a blog post gets comments from less than 10 people it is possible for them to have something that resembles a conversation with the author that is of benefit to other readers and doesn t take excessive amounts of time for the author. When a blog is very popular and every post gets comments from 50+ people it s not really possible. So a traditional blog comment section seems to work best when the blog is primarily read by a small well connected group of people who sometimes comment and some casual readers who never comment (but sometimes find value in the comments of others). Discussions of blog comment systems usually include a reference to a post written by someone who disabled comments on their blog and found it to be a good thing, it always seems that the person who writes such a post has a large and varied audience who s comments would take a lot of time to moderate. John followed the usual form in this regard by linking to a reasonably popular SF author who would presumably have a lot of fans with good net access. I m not going to criticise anyone for disabling comments when their blog becomes really popular, but any advice that they have to offer about such things won t apply to the vast majority of blogs. Due to the long-tail effect the small blogs would probably comprise the majority of all comments so in terms of the way the blog environment works I don t think it makes much difference when the small minority of very popular blogs disable comments. The vast majority of blogs that I regularly read only have a small number of comments. One thing that should be noted is that getting a lot of readers shouldn t be the only factor for writing a successful blog. For example some of my blog posts about SE Linux are aimed at a small audience of Linux programmers and have an even smaller number of people who are qualified to comment. When I write a post that can only receive comments other than please explain more because I don t understand from a few dozen people that doesn t make it any less important. Sometimes the few dozen people who know a topic well need to work together to educate the few thousand who can implement the ideas for the benefit of millions of users of the software. Disabling Comments on Contentious Posts One interesting method John uses is to disable comments early when posting about contentious issues. It s a general practice when running a blog to disable comments on posts after a certain period of time (3 months to 1 year seem to be common time limits for comments). This means that the moderators can concentrate on recent posts and not be bothered with spam bots hitting ancient posts as the interest in writing legitimate comments on an old post is vanishingly small. John has a practice of disabling comments after a couple of days when the comments start to lose quality. No matter how contentious the issue is I m not likely to get the 400+ comments that John gets. But the idea of closing comments quickly still has some merit for my blog and other blogs with less traffic. Not Accepting Comments While Asleep John has a practice of closing comments while he s asleep to avoid allowing a troll to get 8 hours of viewing for a nasty comment. The most immediate down-side to that is that it inconveniences people who don t want to wait 8 hours to comment and prioritises comments from people in the same time zone, this makes me think of Cory Doctorow s novel Eastern Standard Tribe (which is available for free download and I highly recommend reading it) [3]. It seems that a better solution to that problem would be to have a team of moderators to watch things 24*7 which is what a lot of popular blogs that allow comments do. The WordPress capabilities model doesn t support granting a user no special privileges other than moderating comments [4], as WordPress is the most popular self-hosted blog software this limits the possibilities for people moderating comments on other people s blogs. No variation of this would work for me. I have lots of things that require my ongoing attention and don t want to add my blog to the list. If I have other things to work on for a few days I want to just not bother with my blog. This means that my blog needs to be able to run on autopilot for days at a time however I do monitor my blog closely after publishing a post that is likely to attract nasty comments. One extra problem that I have is that the Android client for WordPress has problems in synchronising comments. Using a Forum for Comments Popular Planet installations such as Planet Debian and Planet Linux Australia syndicate more than a few blogs that have comments disabled. A forum installation for such a Planet would be useful to allow people to comment on all posts and also support bloggers who are thinking of disabling comments. While the use of a forum for blog comments has been proven to work well for Boing Boing forums have their own issues of spam and anti-social behavior. Debian already has a forum [5], if a section of that was devoted to discussing blog posts from Planet Debian then it shouldn t make much of an increase to the work of the forum administrators while providing a benefit to the community. Also if the Debian forum had such a section it would probably attract use from more Debian Developers, I would use that forum if it was a place to comment on blogs that don t have a comment section and I might also comment on other forum discussions. It would be good if there was a forum for discussing Linux in Australia. I m not volunteering to run it but I would help out if someone else wants to be the main sysadmin and I can offer free hosting.

30 June 2014

Russell Coker: Links June 2014

Russ Albery wrote an insightful blog post about trust, computer security, and training programmers [1]. He makes a good case that social problems in our community decrease the availability of skilled people to write and audit security code. The Lawfare blog has an insightful article by Dan Geer about Heartbleed as a Metaphor [2]. He makes some good points about security and design, ways of potentially solving some flaws and problems with the various solutions. Eben Moglen wrote an insightful article for The Guardian about the way that the NSA spying is a direct threat to democracy [3] The TED blog has an interesting interview with Kitra Cahana about her work living with and photographing nomads in the US [4]. I was surprised to learn that there s an active nomad community in the US based on the culture that started in the Great Depression. Apparently people are using Youtube to learn about nomad culture before joining. Dave Johnson wrote an interesting Salon article about why CEOs make 300* as much money as workers [5]. Note that actually contributing to the financial success of the company is not one of the reasons. Maia Szalavitz wrote an interesting Slate article about Autism and Anorexia [6]. Apparently some people on the Autism Spectrum are mis-diagnosed with Anorexia due to food intolerance. Groups of four professors have applied for the job of president and vice-chancellor of the University of Alberta [7]. While it was a joke to apply in that way, 1/4 of the university president s salary is greater than the salary of a professor and the university would get a team of 4 people to do the job so it would really make sense to hire them. Of course the university could just pay a more reasonable salary for the president and hire an extra 3 professors. But the same argument applies for lots of highly paid jobs. Is a CEO who gets paid $10M per annim really going to do a better job than a team of 100 people who are paid $100K? Joel on Software wrote an insightful article explaining why hiring 1/200 applicants doesn t mean you hire the top 0.5% of workers [8]. He suggests that the best employees almost never apply through regular channels so an intern program is the only way to get a chance of hiring the best people. Chaotic Idealism has an interesting article on some of the bogus claims about autism and violence [9]. Salon has an interesting articleby Lindsay Abrams about the way the food industry in the US lobbies for laws to prevent employees from reporting animal cruelty or contamination of the food supply and how drones will now be used for investigative journalism [10]. Jacobin Mag has an interesting article by Geoff Shullenberger about the Voluntariat , the people who volunteer their time to help commercial organisations [11]. I don t object to people voluntarily helping companies, but when they are exploited or when the company also requires voluntary help from the government it becomes a problem. We need some legislation about this. Laura Hudson wrote an insightful article about how Riot Games solved their online abuse problem [12]. There are ideas in this that can apply to all online communities. Matt LeMay wrote an interesting article for Medium titled What (Else) Can Men Do? Grow The Fuck Up [13]. It s a general commentary on the treatment of women in geek communities (and most other places). Foz Meadows wrote an insightful analysis of the attempts of bigots to influence science-fiction [14]. If I had more spare time I d read some of the books by bigoted authors on the Sad Puppy Slate (from a library of course) and see if they lack talent in the same way that Orson Scott Card does. Racialicious has an interesting article by Phenderson Djeli Clark about the horrible racism and bigotry of H.P. Lovecraft [15]. I have only read two HP Lovecraft stories, one was mediocre and the other (The Horroe at Red Hook) was quite poor largely due to his inability to write about non-white people. Grace Wyler wrote an insightful article for Vice magazine about the right-wing terrorists in the US killing cops [16]. Paul Rosenberg wrote an interesting and amusing (for people outside the US) article about the gun crazies in the US [17]. Maybe the gun crazies should have a loaded assault rifles and tequila party to follow up on their gun appreciation day . A US TV show made a 4 minute clip of some of the stupid things that Tony Abbott has done [18]. Tony is almost as stupid as Dubya.

28 June 2014

Russ Allbery: Review: The Emergency Sasquatch Ordinance

Review: The Emergency Sasquatch Ordinance, by Kevin Underhill
Publisher: American Bar Association
Copyright: 2013
ISBN: 1-62722-269-3
Format: Trade paperback
Pages: 334
First, if you have not read Lowering the Bar, you should do so. There aren't many blogs where I will start reading, end up reading numerous posts out loud to anyone in the vicinity, and then find myself systematically reading the entire backlog (which, in this case, is substantial). Kevin Underhill is a lawyer with civil libertarian leanings, a wicked sense of humor, and a knack for finding the most absurd legal stories. He varies between weary incredulity and caustic sarcasm, almost always manages to make me smile, and often makes me laugh out loud. The Emergency Sasquatch Ordinance, and Other Real Laws that Human Beings Have Actually Dreamed Up, Enacted, and Sometimes Even Enforced is, as one might expect, more of the same sort of thing that one gets from Lowering the Bar. However, unlike a lot of books released by long-standing bloggers, it is not a collection of previously published material. This is an entirely new collection of legal absurdities, drawing on history, US federal, state, and local law, and some non-US laws, and complete with Underhill's trademark commentary. Long-time readers of Lowering the Bar have seen several posts about this sort of thing before, which also means there's a great sample up on the web. See posts tagged "Law (Dumb)" for a preview of the sort of thing you'll get, although as mentioned the material in the book is original. My one complaint about the book version of this post type is that I think the blog posts tend to be longer and contain a bit more analysis. Underhill gets in about one punch line per law in the book, and I frequently wished for a somewhat longer analysis. The blog posts tend to get funnier and more biting once he works up a head of steam on a topic. The book is, alas, a bit uniform in its style. It's composed of a large number of "chapters," each of which is usually a page or two and covers a single law. Those are divided into ancient, pre-modern, US federal, US state, US city, and non-US laws, but with considerably funnier section headings than those. Each chapter has a title that's usually a brief summary of the law (often funny), an introduction, a quote (or several), and a punch line. There's not much variation, which makes it good bathroom reading you can pick it up anywhere, read a few pages, and put it back down without fear of losing context but lacks variety when read in large gulps. I find Lowering the Bar itself funnier than this book. I think that's because the blog posts are longer, more varied, and mix weird laws with caustic political commentary and entertaining stories of the extremely ill-advised things people do around, within, and against the law. The odd laws are entertaining; the stories of defendants, the accounts of weird legal proceedings, and Underhill's exasperated sarcasm about legal and political stupidities are even funnier. If you had to choose between the book and the blog, I'd go with the blog. That said, you don't have to choose, and if you've been reading the blog for a while, more material is always welcome. A book also has the substantial advantage of being a way to throw some money at a blogger as a reward for hours of entertainment, with a bonus of getting an enjoyable physical object in return. I certainly don't regret the purchase, and I've already had an opportunity to mention Millard Fillmore's State of the Union address on guano. I'm not sure if that means the book made me smarter, but I do believe it made me more entertaining. TL,DR version: Read Lowering the Bar. If you like that, there's this book. I think you can figure out the rest from there. Rating: 7 out of 10

27 June 2014

Russ Allbery: Review: Due Diligence

Review: Due Diligence, by David Roodman
Publisher: Center for Global Development
Copyright: 2012
ISBN: 1-933286-48-2
Format: Trade paperback
Pages: 335
Due Diligence: An Impertinent Inquiry into Microfinance is a detailed analysis of the effects, philosophy, approaches, and merits of microfinance as an institution and as, increasingly, a charitable focus of the developed world. The author, David Roodman, was a senior fellow at the Center for Global Development at the time of writing this book, and is an expert in microfinance and economic development. This book starts as an overview and a history of the spectrum of microfinance institutions (something I found very valuable, since I knew very little about microfinance prior to Kiva's popularity), and then attempts a critical, data-driven analysis of whether, to what extent, and by which measures microfinance is successful. The most common aspect of microfinance, made popular and prominent by Kiva among others, is microcredit: small loans to people in developing countries. The underlying theory of microcredit is that lack of financial services handicaps prospective entrepreneurs and business owners in the third world, preventing them from growing businesses, funding expansion, and moving out of poverty. By extending relatively small amounts of credit to these people, or so the theory goes, they can better smooth out the uncertainties of day-to-day life and get the necessary capital to expand small local business. Microcredit is not the only type of microfinance, though, and Roodman's careful discussion of that point was one of the most informative parts of this book for me. It can also come in the form of microsavings tiny bank accounts or small-scale savings plans or microinsurance. Roodman makes a very interesting argument that these three forms of microfinance form two opposing continuums. Microcredit is the easiest to provide and microinsurance is the hardest, for many reasons he discusses including some that were surprising to me, but microinsurance is arguably the most helpful and beneficial to the individual poor and microcredit is the least. The opening history and survey does an excellent job showing the range of techniques and approaches used in microfinance, and showing the challenges of providing it. I had some vague understanding of the logistical challenges in serving areas far away from traditional infrastructure. I had much less understanding of the fascinating array of techniques in use, both inside and outside what's become the microfinance industry. Some of these techniques are brilliant examples of human economic creativity and skill at balancing incentives with risk protection. There are a much wider variety of approaches and instruments available than the individual or group loans that one typically sees on a site like Kiva, all with different trade-offs in group size, flexibility, rigidity, risk, and complexity. This book is well worth reading just to get a feel for the huge array of financial arrangements possible outside of the traditional banking structures that most readers of this review are used to. Another question Roodman answers thoroughly is why microfinance is so focused on microcredit. My vague unease with that was strengthened by reading David Graeber's Debt, which among many other things makes clear some of the social perils and power imbalances in lending. Due Diligence both crystallized that discomfort and explained why alternatives are difficult. Both microsavings and microinsurance necessarily involve considerably more regulation (which has both advantages and complexity), since they involve taking and safeguarding people's money and promising service in the future. Lenders require less trust; lenders instead extend trust and assume that risk themselves, and are in a position to do so. Microinsurance is also harder to explain and harder to target, although it is potentially the most helpful financial instrument when well-targeted. And microcredit can absorb huge influxes of capital from the outside, lending it out in a region, while both microsavings and microinsurance are necessarily built on the money already in the local economy and are inherently more resistant to external involvement. (I'm of the opinion this is a significant feature, but it makes it far harder for them to catch on as a charitable trend.) The heart of the book is a hard look at whether microfinance, and particularly microcredit, works. Roodman attempts this analysis for three different definitions of working: lifting people out of poverty (the most commonly claimed success), giving people more freedom and control over their lives even if their poverty level does not change in absolute terms, and developing the local economy. Sadly for charities like Kiva, microcredit does surprisingly poorly on the first two metrics. There is still little clear data, and the degree to which data from one region is applicable to a different region is highly debatable, but what evidence we have does not support the theory that microcredit lifts people out of poverty. In fact, the more intense the microcredit, the more likely it appears to be that credit pathologies common in the developed world, such as taking out loans from one organization to pay off loans from another, spirals of debt that is unlikely to ever be repaid, and aggressive debt collection methods, turn up in microcredit as well. The vision of the entrepreneur who succeeds through new-found access to capital is largely a myth. Indeed, the current packaging and presentation of loans to distant charities in developed countries means that even the supposed purpose of the loans is dubious. Loans are almost always presented as being for some sort of business activity, since that's what external charities want to fund, but money is fungible and the funds are more often used for all the things people normally need to use funds for: weddings, funerals, food for the family in hard times, paying off other loans to relatives, and other daily expenses. On the empowerment front, microcredit fairs little better, and here I particularly appreciated Roodman's clear-eyed and honest portrayal of common microcredit practices and their benefits for the creditors. For example, microcredit makes much of loaning to women, and presents this as a story of female empowerment. Roodman argues that there is some reality behind that, but it's also clear that microcredit organizations loan to women because women are more likely to repay not due to any inherent gender trait, but because women are easier to intimidate and shame into repaying. Similarly, the much-vaunted group loans, usually presented as a way for the community to support each other and teach each other financial skills, are most common in the poorest regions because they represent an outsourcing of costs. The main risk to the lender is that poor people have no such thing as credit histories, and detailed investigations of someone's trustworthiness is expensive. Groups are jointly liable and therefore will police themselves and check each other's reliability at no cost to the lender. Groups will also use intimidation tactics and even violence against members who don't or can't repay their share, without the lender having to get involved or have any responsibility for those tactics. Roodman takes a philosophical tone on this topic, noting several times that services for the very poor are generally of low quality and finance is no exception. I'm less inclined to be philosophical; after reading this book, I find it very hard to support group loans. This leaves economic development as the most compelling argument left for microcredit. This is weak tea for people like myself who wanted to believe in the transformative power of microcredit, but Roodman does a solid job explaining why it's still important. There is a great deal to be said for the perspective that only industrialization actually lifts people out of poverty on any substantial scale, and therefore the most useful type of aid is aid that somehow contributes towards industrialization. Building a native financial industry could arguably be such a step. And there seems to be some real merit and lasting effect to having a richer and more capable set of local financial institutions. However, here, it's not at all clear that external money helps. In fact, there's quite a bit of evidence that external money hurts, since it pushes microfinance institutions to find ways of using that money and pleasing external donors instead of building their infrastructure and capabilities as financial institutions. The earlier institutions, such as the Grameen Bank, seem to have done a better job at being local institutions woven into the local economy than have the later organizations that are largely supported by western donors. Roodman makes a good case for the usefulness of microfinance as a part of economic development, but it comes with a strong argument against too much external funding, and particularly against the practice of developed world donors funding specific loans. I came away from this book feeling like I had a much better understanding of the core problems and techniques of financial services for the poor, and with a strong desire to see more microsavings than microcredit. The two are closer in nature than I had expected, and Roodman's analysis of the similarities was fascinating, but microsavings seems less vulnerable to some of the credit pathologies. There is also some statistical evidence that microsavings does provide some modest reductions in poverty, while the evidence for microcredit is much less compelling. Sadly, from the perspective of a resident of the developed world such as myself who would very much like to help, neither microsavings nor effective microcredit seems likely to benefit from my contributions, and particularly not in the form of a site like Kiva. The end of this book is a long appendix that reprints multiple blog posts on related topics, including additional discussion of the problems with the Kiva model. Effectively, Kiva is a way for people like myself to subsidize microcredit organizations. It's not at all clear that subsidies are needed, and there's some reason to believe that they encourage negative and destructive behavior. The way they are marketed involves a certain amount of lying, or at least distortion, on the part of multiple actors in the microcredit picture. And they're a distraction from microsavings, and from microcredit based on loaning out savings from within the same local economy, both of which seem to have considerably more positive properties. Due Diligence is a book with a somewhat narrow target audience. It will appeal primarily to people with an interest in economics, particularly development economics, or who are evaluating microfinance charities or analyzing their support of groups like Kiva. But within that context, this is an excellent book, and I recommend it to your attention. Roodman is clear, thorough, and fair within my ability to measure that, and while no one book can be the final word on a topic, he adds considerable data and thoughtful analysis. If you are a supporter of Kiva, like I was, I fear you'll find this book rather discouraging. But it's better to have the information than not, and I found it a good reminder of the inherent drawbacks of loans and debt. There is, sadly, nothing magical about microcredit that removes those problems. Rating: 9 out of 10

26 June 2014

Jonathan McDowell: Forms of communication

I am struck by the fragmentation in communication mechanisms. Let's look at how I have communicated with my friends in the past few days:
  • Phone call
    Tried and tested, though I tend to avoid them. I've made some deliberate calls to sort out immediate plans, and at least one accidental call caused by user error which resulted in talking to someone it was good to hear from.
  • Text message
    Again, reasonably tried and tested. I miss the inability to use Google Voice when I'm in the UK; I'd much rather read and compose text messages from my web browser when I'm near a computer than type them on my phone, even if it does have a keyboard.
  • Email
    One of my favourite methods of communication. Suitable for quick messages or longer screeds. I can throw links in and expect you to be able to click them. I can put lots of detail so that everything is covered easily. I can confuse you by quoting correctly. I guess while I do read email on my phone I'm less likely to reply there as I'm always a bit embarrassed how the clients cope with replies.
  • IRC
    Like, I suspect, many readers of my blog posts, I'm still a daily user of IRC. There are friends I keep in touch with mostly via this method. It's great. It's like Twitter for old people and much better in many ways.
  • Skype
    This started out as a work thing. It was the way in which the Belfast office communicated with the US, it become the way the Belfast office communicated with each other and when I moved on it was the way in which I kept in contact with a group of people I consider good friends. It's great for calls (I feel bad saying that, but it's an idea executed well across multiple platforms and any other VOIP stuff I've played with has been much more of a hassle), but the one to one and group chat functionality is pretty spot on as well. Also has the advantage that I can turn it off and mostly not end up with work queries.
  • Google Hangouts
    I actually quite like these. They work on my phone, I can poke them from a web browser, I can dump more than just text into them. IRC is better in some ways, but I do like the additional flexibility I get from a Hangout. It doesn't play well with people who haven't drunk the Google koolaid, which is the main reason I haven't managed to convince the Skype group chat group to move it over here.
  • Facebook messenger
    I hate this. On the face of it there's not a lot of difference between it and Hangouts, but the app wants more and more privileges, I'm less likely to be logged into Facebook (e.g. I avoid it at work, whereas there are good reasons I'd be logged into my Google account there, though less so since the demise of Reader) and I don't think it's as nicely implemented. However there are a few people who it's easiest to get hold of via this method. And there's a certain amount of mesmerisation by the floaty wee faces it invokes on my phone.
While some of these work better for me than others really what I'd like is to use fewer of them, and I can't see that happening any time soon. I don't want to have to run a handful of different messaging apps on my phone. I also don't want to be limited to only using my laptop or my phone for something - I'd much prefer to be able to pickup the phone, laptop or tablet depending on what I'm up to and have my full range of communication available. Some of these things can be aggregated together, but that will then lose some of the advantages. And I'm sure that even if I got rid of one or two of the above there'd be something to fill the gap along shortly (I have, for example, so far completely avoided WhatsApp).

Russ Allbery: Review: A People's History of the Supreme Court

Review: A People's History of the Supreme Court, by Peter Irons
Publisher: Penguin
Copyright: 1999, 2006
Printing: 2006
ISBN: 0-14-303738-2
Format: Trade paperback
Pages: 553
I first encountered Peter Irons via his Teaching Company course, The History of the Supreme Court. I listened to that and enjoyed it, and would recommend it as an excellent overview. When I later ran across this book, I was excited: I usually prefer books to lectures on topics that can benefit from greater depth, and A People's History of the United States is one of my favorite history books. A book that talked about the Supreme Court from a similar bottom-up approach was appealing. Unfortunately, I think the title oversells this book. It is a history of the Supreme Court, and, as with Zinn's book, it carries its bias openly (if not as forcefully or compellingly as Zinn). It is a legal history concerned with individual rights, and Irons openly expresses his opinions of the merits of certain decisions. But it's not the full-throated cry for justice and for the voice of the general population that Zinn's book was, nor did I think it entirely delivered on the introductory promise to look at the people and situations underlying the case. It does provide more background and biographical sketch of the litigants in famous cases than some other histories, but those sketches are generally brief and focused on the background and relevant facts for the case. This is, in short, a history of the Supreme Court, with some attention paid to the people involved and open comfort with an authorial viewpoint, but without the eye-opening and convention-defying freshness of Zinn's refusal to point the camera at the usual suspects. It also largely duplicates Irons's course for the Teaching Company. That's only really a problem for those, like me, who listened to that course already and came to this book looking for additional material. If you haven't listened to the course, this is a reasonable medium in which to provide the same content, although you do miss the audio recordings of actual Supreme Court argument in some of the most famous cases. But after having listened or read both, I think I prefer the course. It felt a bit more focused; the book is not padded, but the additional material is also not horribly memorable. That said, as a history, I found this a solid book. Irons opens with a surprisingly detailed look at the constitutional convention and the debates over the wording of the US Constitution, focusing on those sections that will become the heart of later controversy. Some of them were known to be controversial at the time and were discussed and argued at great length, such as anything related to slavery and the role of a bill of rights. But others, including many sections at the heart of modern controversies, were barely considered. Despite being a bit off-topic for the book, I found this section very interesting and am now wanting to seek out a good, full history of the convention. Irons's history from there is primarily chronological, although he does shift the order slightly to group some major cases into themes. He doesn't quite provide the biographies of each Supreme Court justice that he discussed in the introduction, but he comes close, along with discussion of the surrounding politics of the nomination and the political climate in which they were sitting. There's a bit of an overview of the types of cases each court saw, although not as much as I would have liked. Most of the history is, as one might expect, focused on more detailed histories of the major cases. Here, I was sometimes left satisfied and sometimes left a bit annoyed. The discussion of the Japanese internment cases is excellent, as you might expect given Irons's personal role in getting them overturned. The discussion of the segregation cases is also excellent; in general, I found Section V the strongest part of this book. Irons also does a good job throughout in showing how clearly political the court was and has always been, and the degree to which many major decisions were pre-decided by politics rather than reasoned legal judgment. Where I think he fails, however, is that much of this book has little sense of narrative arc. This is admittedly difficult in a history of a body that takes on widely varying and almost random cases, but there are some clear narratives that run through judicial thinking, and I don't think Irons does a good enough job telling those stories for the readers. For example, I remembered the evolution of interpretation of the 14th Amendment from freedom of contract to the keystone of application of the Bill of Rights to the states as a compelling and coherent story from his course, and here it felt scattered and less clear. In general, I liked the earlier sections of this book better than the later, with Section V on the Warren court as the best and last strong section. Beyond that point in the history, it felt like Irons was running out of steam, and it was harder and harder to see an underlying structure to the book. He would describe a few cases, in a factual but somewhat dry manner, make a few comments on the merits of the decision that felt more superficial than earlier sections of the book, and then move on to another case that might be largely unrelated. Recent judicial history is fascinating, but I don't think this is the book for that. Irons is much stronger in the 19th and first half of the 20th centuries; beyond that, I prefer Jeffrey Toobin's The Nine, which also has far deeper and more interesting biographies of recent justices. This is not a bad history of the Supreme Court. If you're looking for one, I can recommend it. But if you're flexible about format, I recommend Irons's course more. I think it's less dry, better presented, and flows better, and I don't feel like you'll miss much in the transformation of this book into an 18 hour course. It's also remarkable to hear the actual voices of the lawyers and justices in some of the landmark cases of the middle of the 20th century. But if you specifically want a book, this does the job. Do also read The Nine, though. It's a good complement to Irons's straight history, showing much more of the inner workings, political maneuverings, and day-to-day struggles of the justices. Rating: 7 out of 10

23 June 2014

Steve Kemp: So I accidentally ... a service.

This post is partly introspection, and partly advertising. Skip if it either annoys you. Back in February I was thinking about what to do with myself. I had two main options "Get a job", and "Start a service". Because I didn't have any ideas that seemed terribly interesting I asked people what they would pay for. There were several replies, largely based "infrastructure hosting" (which was pretty much 50/50 split between "DNS hosting", and project hosting with something like trac, redmine, or similar). At the time DNS seemed hard, and later I discovered there were already at least two well-regarded people doing DNS things, with revision control. So I shelved the idea, after reaching out to both companies to no avail. (This later lead to drama, but we'll pretend it didn't.) Ultimately I sought and acquired gainful employment. Then, during the course of my gainful employment I was exposed to Amazons Route53 service. It looked like I was going to be doing many things with this, so I wanted to understand it more thoroughly than I did. That lead to the creation of a Dynamic-DNS service - which seemed to be about the simplest thing you could do with the ability to programatically add/edit/delete DNS records via an API. As this was a random hack put together over the course of a couple of nights I didn't really expect it to be any more popular than anything else I'd deployed, and with the sudden influx of users I wanted to see if I could charge people. Ultimately many people pretended they'd pay, but nobody actually committed. So on that basis I released the source code and decided to ignore the two main missing features - lack of MX records, and lack of sub-sub-domains. (Isn't it amazing how people who claim they want "open source" so frequently mean they want something with zero cost, they can run, and never modify and contribute toward?) The experience of doing that though, and the reminder of the popularity of the original idea made me think that I could do a useful job with Git + DNS combined. That lead to DNS-API - GitHub based DNS hosting. It is early days, but it looks like I have a few users, and if I can get more then I'll be happy. So if you want to to store your DNS records in a (public) GitHub repository, and get them hosted on geographically diverse anycasted servers .. well you know where to go: Github-based DNS hosting.

Francesca Ciceri: Adventures in Mozillaland #2

Time for an update from my internship at Mozilla, as part of the OPW. The last weeks have been a bit rough: I have my usual migraines to thank for that. It's not easy to work with them: you are either stoned by the meds, or cannot look at a monitor. And while you're trying to sleep your headaches away, the world keeps rolling. Silly world.
As Liz, my mentor, suggested I tried to stick with the little things and to do a bit of something everyday.
"Waiting for the miracle", you know.
So, here's what I've been up to: Triaging I probably failed my personal 5-bugs-a-day policy, in the last two weeks, but beside that I'm pretty satisfied of my progress:
  • I learnt how to find regression windows
  • I verified some more bugs (mostly Developer Tools or Theme related) on both Linux and Windows (yeah. Don't.Ask.)
  • I triaged a lot of them. Also botched a couple, and felt an idiot for that.
All in all, I realized that I really love triaging/verifying bugs: being not a developer nor a simple user, but a bit of both, gives me the right mindset - I think - to mediate between these two worlds. Community On the community front, I finally managed to meet - virtually - the Mozilla Italian community. The guys are great and beside running the forum to give user support, they do a whole lot of activities related to localization. They are also trying to encourage participation of Italian users and enthusiasts to Mozilla events: don't miss, for instance, the upcoming Marketplace Day when a couple of us will be available to help other Italian users with the day's activities. Read Daniele's post on the forum for more information in Italian. Documentation On the documentation front, I finally managed to get out the Bugdays FAQ, draft a guide on how to run different versions of Firefox and multiple profiles for triaging purposes on Linux and Windows - still have to finish this one, though -, and participate to a very interesting discussion on the current state of QMO - the entry point for QA contributors in Mozilla.
The site, in my opinion, needs some love and I'd very much like to help in that sense.
Check out the discussion, and give us some feedback about the website on the dev-quality mailing list! Lessons learned
  • when you're not feeling well, stick with the little things and don't feel guilty for that
  • don't try to reproduce a bug when you're tired, you'll end up Doing It Wrong

Simon Josefsson: Offline GnuPG Master Key and Subkeys on YubiKey NEO Smartcard

I have moved to a new OpenPGP key. There are many tutorials and blog posts on GnuPG key generation around, but none of them matched exactly the setup I wanted to have. So I wrote down the steps I took, to remember them if I need to in the future. Briefly my requirements were as follows:
  • The new master GnuPG key is on an USB stick.
  • The USB stick is only ever used on an offline computer.
  • There are subkeys stored on a YubiKey NEO smartcard for daily use.
  • I want to generate the subkeys using GnuPG so I have a backup.
  • Some non-default hash/cipher preferences encoded into the public key.
After writing down the notes below, I posted about how to create a small JPEG image to embed in my OpenPGP key. I was planning to go live with the first key I generated, however as was gently pointed out to me, the JPEG image I generated was not optimal (too low quality and not sufficiently compressed). I have decided to retake the photo so I have a color image as a basis for size optimization. I don t want to postpone using the new key though, so I stepped through all of these steps again (except adding the photo) to get a new key. This is why the notes below are for a key 1C5C4717 that is now revoked. My new real key is 54265E8C. I will add the photo to my 54265E8C key once I have a JPEG file that I m happy with. Offline machine The offline machine setup I use is a Live CD on a machine that is physically well protected. I m using the Debian Live CD version 7.5.0 GNOME Desktop. The password for the auto-logged in user is live which you need if the screen-saver kicks in. Configure the keyboard layout if you need to. Insert an USB memory stick. I m using a VFAT filesystem to keep things simple; and for this writeup it happened to be mounted as /media/FA21-BEC7 so you will have to replace that path with something that points to your USB stick. Open a terminal since the rest of this writeup will be done from a terminal window. GnuPG configuration Set your GnuPG home directory to point at the USB memory device. You will need to do this in every terminal windows you open that you want to use GnuPG in.
user@debian:~$ export GNUPGHOME=/media/FA21-BEC7/gnupghome
user@debian:~$ mkdir $GNUPGHOME
user@debian:~$ 
The GnuPG defaults (as of version 1.4.16) to rank SHA1 higher than SHA384, SHA512, and SHA224 in the default hash preference list. To be precise, the default hash preference order is SHA256, SHA1, SHA384, SHA512, SHA224. I consider SHA1 broken so I don t advertise it all, although I believe that will not prevent some implementations of using SHA1 anyway since it is the mandatory to implement hash algorithm. Regarding symmetric ciphers, the default order is AES256, AES192, AES128, CAST5, 3DES. I don t like ciphers with 64-bit block lengths, so I don t advertise them but similarily, I believe this will not prevent some implementations of using CAST5 or 3DES anyway. I also advertise support for Twofish and Camellia in case someone wants to use them, they are 128-bit block length and relatively modern ciphers after all. The default-preference-list keyword is used to override the default settings, which will be recorded into any newly generated keys. GnuPG self-sign keys with SHA1 by default, and I prefer to use a member of the SHA2 family, hence the cert-digest-algo keyword. Further down below we will use the GnuPG Agent to talk to the smartcard, so configure GnuPG to use it with the use-agent keyword. GnuPG prints ugly warning messages about locking (gpg: DBG: locking for /media/FA21-BEC7/gnupghome/secring.gpg.lock' done via O_EXCL), presumably because of the VFAT filesystem, so I use lock-never to silence that.
user@debian:~$ cat > $GNUPGHOME/gpg.conf
default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAMELLIA256 CAMELLIA192 CAMELLIA128 TWOFISH
cert-digest-algo SHA512
use-agent
lock-never
user@debian:~$ 
Generate master key Below I will use a 3744 bit RSA key, where the key size is selected based on the assumption that people will focus efforts to crack RSA keys on the usual power-of-two key sizes. I have chosen to not generate an encryption key, since I will use subkeys on a smartcard. With my old B565716F key I noticed that sometimes people will encrypt to my main encryption key even though I have encryption subkeys. Presumably this happens due to implementation flaws or user configuration mistakes. It could happen intentionally if someone had a public key from me with an expired subkeys but not expired main keys. This could be a reason to use the same expiration day for all your keys. Still, I chose to not generate an encryption key at all at this point. For additional protection, I m using a passphrase on the key.
user@debian:~$ gpg --gen-key
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
gpg: keyring  /media/FA21-BEC7/gnupghome/secring.gpg' created
gpg: keyring  /media/FA21-BEC7/gnupghome/pubring.gpg' created
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 3744
Requested keysize is 3744 bits
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 100
Key expires at Fri 26 Sep 2014 10:50:22 PM UTC
Is this correct? (y/N) y
You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
    "Heinrich Heine (Der Dichter) "
Real name: Simon Josefsson
Email address: simon@josefsson.org
Comment: 
You selected this USER-ID:
    "Simon Josefsson "
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
You need a Passphrase to protect your secret key.
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
...
gpg: /media/FA21-BEC7/gnupghome/trustdb.gpg: trustdb created
gpg: key 1C5C4717 marked as ultimately trusted
public and secret key created and signed.
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: next trustdb check due at 2014-09-26
pub   3744R/1C5C4717 2014-06-18 [expires: 2014-09-26]
      Key fingerprint = EF0A 1996 7B3B 4BAD 9D5C  A97F 1A44 08DD 1C5C 4717
uid                  Simon Josefsson 
Note that this key cannot be used for encryption.  You may want to use
the command "--edit-key" to generate a subkey for this purpose.
user@debian:~$ 
Add photo I m in the process of creating a better JPEG photo, so I skipped this step for my new key. However the notes here are correct anyway.
user@debian:~$ gpg --edit-key 1C5C4717
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
[ultimate] (1). Simon Josefsson 
gpg> addphoto
Pick an image to use for your photo ID.  The image must be a JPEG file.
Remember that the image is stored within your public key.  If you use a
very large picture, your key will become very large as well!
Keeping the image close to 240x288 is a good size to use.
Enter JPEG filename for photo ID: /media/FA21-BEC7/simon-gpg.jpg
Is this photo correct (y/N/q)? y
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 1C5C4717, created 2014-06-18
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
[ultimate] (1). Simon Josefsson 
[ unknown] (2)  [jpeg image of size 6048]
gpg> save
user@debian:~$ 
Add another identity Most people have multiple email addresses, and this needs to be reflected in the GnuPG key. Use the primary command to specify your main User ID.
user@debian:~$ gpg --edit-key 1C5C4717
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  [jpeg image of size 6048]
gpg> adduid
Real name: Simon Josefsson
Email address: simon@yubico.com
Comment: 
You selected this USER-ID:
    "Simon Josefsson "
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 1C5C4717, created 2014-06-18
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
[ultimate] (1)  Simon Josefsson 
[ultimate] (2)  [jpeg image of size 6048]
[ unknown] (3). Simon Josefsson 
gpg> uid 1
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
[ultimate] (1)* Simon Josefsson 
[ultimate] (2)  [jpeg image of size 6048]
[ unknown] (3). Simon Josefsson 
gpg> primary
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 1C5C4717, created 2014-06-18
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
[ultimate] (1)* Simon Josefsson 
[ultimate] (2)  [jpeg image of size 6048]
[ unknown] (3)  Simon Josefsson 
gpg> save
user@debian:~$ 
Create a revocation certificate It is good practice to generate a revocation certificate in case you lose your key. Store this in a safe place, possibly printed out on paper.
user@debian:~$ gpg --output $GNUPGHOME/../revocation-certificate.txt --gen-revoke 1C5C4717
sec  3744R/1C5C4717 2014-06-18 Simon Josefsson 
Create a revocation certificate for this key? (y/N) y
Please select the reason for the revocation:
  0 = No reason specified
  1 = Key has been compromised
  2 = Key is superseded
  3 = Key is no longer used
  Q = Cancel
(Probably you want to select 1 here)
Your decision? 1
Enter an optional description; end it with an empty line:
> Created during key creation, emergency use only.
> 
Reason for revocation: Key has been compromised
Created during key creation, emergency use only.
Is this okay? (y/N) y
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 1C5C4717, created 2014-06-18
ASCII armored output forced.
Revocation certificate created.
Please move it to a medium which you can hide away; if Mallory gets
access to this certificate he can use it to make your key unusable.
It is smart to print this certificate and store it away, just in case
your media become unreadable.  But have some caution:  The print system of
your machine might store the data and make it available to others!
user@debian:~$ 
Make a backup of the master key To have an easy way to move back and forward in time in GnuPG, I both export the key to a stable data format and keep a backup of the actual GnuPG home directory.
user@debian:~$ gpg -a --export-secret-keys 1C5C4717 > $GNUPGHOME/../masterkey.txt
user@debian:~$ cp -a $GNUPGHOME $GNUPGHOME-backup-masterkey
user@debian:~$ 
Create subkeys Now I will generate three keys that will go onto the smartcard. I have chosen to generate these using GnuPG and then move the keys onto the smartcards, instead of generating the keys directly on the card. The difference is that with this approach, I get a backup of the keys and can import them to another key in the future if I need to. Each key has its own purpose: Signature, Encryption, and Authentication. Smartcards typically have limitation on key sizes, so I select 2048 as a widely supported size. Expert mode is required to generate authentication subkeys.
user@debian:~$ gpg --expert --edit-key 1C5C4717
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  [jpeg image of size 6048]
[ultimate] (3)  Simon Josefsson 
gpg> addkey
Key is protected.
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 1C5C4717, created 2014-06-18
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 100
Key expires at Fri 26 Sep 2014 11:03:16 PM UTC
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
..+++++
....+++++
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/72D5245B  created: 2014-06-18  expires: 2014-09-26  usage: S   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  [jpeg image of size 6048]
[ultimate] (3)  Simon Josefsson 
gpg> addkey
Key is protected.
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 1C5C4717, created 2014-06-18
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
Your selection? 6
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 100
Key expires at Fri 26 Sep 2014 11:03:31 PM UTC
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
......+++++
Not enough random bytes available.  Please do some other work to give
the OS a chance to collect more entropy! (Need 7 more bytes)
.+++++
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/72D5245B  created: 2014-06-18  expires: 2014-09-26  usage: S   
sub  2048R/A11F46D2  created: 2014-06-18  expires: 2014-09-26  usage: E   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  [jpeg image of size 6048]
[ultimate] (3)  Simon Josefsson 
gpg> addkey
Key is protected.
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 1C5C4717, created 2014-06-18
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
Your selection? 8
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: Sign Encrypt 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
Your selection? s
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: Encrypt 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
Your selection? e
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
Your selection? a
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: Authenticate 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
Your selection? q
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 100
Key expires at Fri 26 Sep 2014 11:03:59 PM UTC
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
+++++
Not enough random bytes available.  Please do some other work to give
the OS a chance to collect more entropy! (Need 56 more bytes)
+++++
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/72D5245B  created: 2014-06-18  expires: 2014-09-26  usage: S   
sub  2048R/A11F46D2  created: 2014-06-18  expires: 2014-09-26  usage: E   
sub  2048R/D6987A02  created: 2014-06-18  expires: 2014-09-26  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  [jpeg image of size 6048]
[ultimate] (3)  Simon Josefsson 
gpg> save
user@debian:~$ 
Export subkeys for backup This is a good time to save a restore point for your key. Note in the output of --list-secret-keys the keywords sec and ssb which means the main key and the subkeys are available. If the secret keyring contained only stubs, it would be sec> and sec#.
user@debian:~$ gpg --list-keys
/media/FA21-BEC7/gnupghome/pubring.gpg
--------------------------------------
pub   3744R/1C5C4717 2014-06-18 [expires: 2014-09-26]
uid                  Simon Josefsson 
uid                  [jpeg image of size 6048]
uid                  Simon Josefsson 
sub   2048R/72D5245B 2014-06-18 [expires: 2014-09-26]
sub   2048R/A11F46D2 2014-06-18 [expires: 2014-09-26]
sub   2048R/D6987A02 2014-06-18 [expires: 2014-09-26]
user@debian:~$ gpg --list-secret-keys
/media/FA21-BEC7/gnupghome/secring.gpg
--------------------------------------
sec   3744R/1C5C4717 2014-06-18 [expires: 2014-09-26]
uid                  Simon Josefsson 
uid                  [jpeg image of size 6048]
uid                  Simon Josefsson 
ssb   2048R/72D5245B 2014-06-18
ssb   2048R/A11F46D2 2014-06-18
ssb   2048R/D6987A02 2014-06-18
user@debian:~$ gpg -a --export-secret-keys 1C5C4717 > $GNUPGHOME/../mastersubkeys.txt
user@debian:~$ gpg -a --export-secret-subkeys 1C5C4717 > $GNUPGHOME/../subkeys.txt
user@debian:~$ cp -a $GNUPGHOME $GNUPGHOME-backup-mastersubkeys
user@debian:~$ 
Configure machine for smartcards The YubiKey NEO requires that RSA keys are imported with some additional parameters, used for CRT speedups. This was fixed in GnuPG 2.0.22. Unfortunately, it is not fixed in GnuPG 1.x. However, GnuPG 1.x can use gpg-agent and scdaemon from GnuPG to communicate with the smartcard. So let s work around the limitation in GnuPG 1.x by installing parts from GnuPG 2.x and use those. You will need to install the following packages: gnupg-agent, libpth20, pinentry-curses, libccid, pcscd, scdaemon, libksba8. Make sure that scdaemon is version 2.0.22 or later (get it from backports). I downloaded these packages and put them on the USB stick. Unfortunately, libccid in Debian is a bit outdated, and does not contain the USB device vendor/product ID in /etc/libccid_Info.plist. You will need to manually add this, and restart pcscd. user@debian:~$ sudo gedit /etc/libccid_Info.plist
user@debian:~$ sudo service pcscd restart Start gnupg-agent and setup the environment variable for this session:
user@debian:~$ gpg-agent --daemon
gpg-agent[22556]: directory  /media/FA21-BEC7/gnupghome/private-keys-v1.d' created
GPG_AGENT_INFO=/tmp/gpg-wGji5C/S.gpg-agent:22557:1; export GPG_AGENT_INFO;
gpg-agent[22557]: gpg-agent (GnuPG) 2.0.22 started
user@debian:~$ GPG_AGENT_INFO=/tmp/gpg-wGji5C/S.gpg-agent:22557:1; export GPG_AGENT_INFO;
user@debian:~$ 
Prepare YubiKey NEO Make sure you have a recent firmware version, 3.1.8 or later; use lsusb -v to find out. Make sure the device is in OTP/CCID or CCID mode, use ykpersonalize -m from the YubiKey Personalization project to switch. Make sure you have the OpenPGP applet loaded properly, otherwise see the YubiKey NEO OpenPGP applet project on installing it. You may want to set a proper Application ID, see herlo s ssh-gpg-smartcard-config github repository for some hints. Configure OpenPGP applet This also changes the PIN and Admin codes.
user@debian:~$ gpg --card-edit
Application ID ...: D2760001240102000060000000420000
Version ..........: 2.0
Manufacturer .....: unknown
Serial number ....: 00000042
Name of cardholder: [not set]
Language prefs ...: [not set]
Sex ..............: unspecified
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: 2048R 2048R 2048R
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 0
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]
gpg/card> admin
Admin commands are allowed
gpg/card> passwd
gpg: OpenPGP card no. D2760001240102000060000000420000 detected
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection? 3
PIN changed.
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection? 1
PIN changed.
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection? q
gpg/card> name
Cardholder's surname: Josefsson
Cardholder's given name: Simon
gpg/card> lang
Language preferences: sv
gpg/card> url
URL to retrieve public key: https://josefsson.org/1c5c4717.txt
gpg/card> sex
Sex ((M)ale, (F)emale or space): m
gpg/card> login
Login data (account name): jas
gpg/card> 
Application ID ...: D2760001240102000060000000420000
Version ..........: 2.0
Manufacturer .....: unknown
Serial number ....: 00000042
Name of cardholder: Simon Josefsson
Language prefs ...: sv
Sex ..............: male
URL of public key : https://josefsson.org/1c5c4717.txt
Login data .......: jas
Signature PIN ....: forced
Key attributes ...: 2048R 2048R 2048R
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 0
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]
gpg/card> quit
user@debian:~$ 
Move subkeys to YubiKey NEO Moving subkeys to a NEO is a destructive operation, so make sure you took backups of the subkeys as above. After this step, your GnuPG keyring will contain stubs for the subkeys.
user@debian:~$ gpg --edit-key 1C5C4717
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: ultimate
sub  2048R/72D5245B  created: 2014-06-18  expires: 2014-09-26  usage: S   
sub  2048R/A11F46D2  created: 2014-06-18  expires: 2014-09-26  usage: E   
sub  2048R/D6987A02  created: 2014-06-18  expires: 2014-09-26  usage: A   
[ultimate] (1). Simon Josefsson 
[ultimate] (2)  [jpeg image of size 6048]
[ultimate] (3)  Simon Josefsson 
gpg> toggle
sec  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26
ssb  2048R/72D5245B  created: 2014-06-18  expires: never     
ssb  2048R/A11F46D2  created: 2014-06-18  expires: never     
ssb  2048R/D6987A02  created: 2014-06-18  expires: never     
(1)  Simon Josefsson 
(2)  [jpeg image of size 6048]
(3)  Simon Josefsson 
gpg> key 1
sec  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26
ssb* 2048R/72D5245B  created: 2014-06-18  expires: never     
ssb  2048R/A11F46D2  created: 2014-06-18  expires: never     
ssb  2048R/D6987A02  created: 2014-06-18  expires: never     
(1)  Simon Josefsson 
(2)  [jpeg image of size 6048]
(3)  Simon Josefsson 
gpg> keytocard
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
Please select where to store the key:
   (1) Signature key
   (3) Authentication key
Your selection? 1
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
2048-bit RSA key, ID 72D5245B, created 2014-06-18
sec  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26
ssb* 2048R/72D5245B  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb  2048R/A11F46D2  created: 2014-06-18  expires: never     
ssb  2048R/D6987A02  created: 2014-06-18  expires: never     
(1)  Simon Josefsson 
(2)  [jpeg image of size 6048]
(3)  Simon Josefsson 
gpg> key 1
sec  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26
ssb  2048R/72D5245B  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb  2048R/A11F46D2  created: 2014-06-18  expires: never     
ssb  2048R/D6987A02  created: 2014-06-18  expires: never     
(1)  Simon Josefsson 
(2)  [jpeg image of size 6048]
(3)  Simon Josefsson 
gpg> key 2
sec  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26
ssb  2048R/72D5245B  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb* 2048R/A11F46D2  created: 2014-06-18  expires: never     
ssb  2048R/D6987A02  created: 2014-06-18  expires: never     
(1)  Simon Josefsson 
(2)  [jpeg image of size 6048]
(3)  Simon Josefsson 
gpg> keytocard
Signature key ....: EF34 D1F7 95C0 3392 E52A  54FE DFF1 6372 72D5 245B
Encryption key....: [none]
Authentication key: [none]
Please select where to store the key:
   (2) Encryption key
Your selection? 2
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
2048-bit RSA key, ID A11F46D2, created 2014-06-18
sec  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26
ssb  2048R/72D5245B  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb* 2048R/A11F46D2  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb  2048R/D6987A02  created: 2014-06-18  expires: never     
(1)  Simon Josefsson 
(2)  [jpeg image of size 6048]
(3)  Simon Josefsson 
gpg> key 2
sec  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26
ssb  2048R/72D5245B  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb  2048R/A11F46D2  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb  2048R/D6987A02  created: 2014-06-18  expires: never     
(1)  Simon Josefsson 
(2)  [jpeg image of size 6048]
(3)  Simon Josefsson 
gpg> key 3
sec  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26
ssb  2048R/72D5245B  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb  2048R/A11F46D2  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb* 2048R/D6987A02  created: 2014-06-18  expires: never     
(1)  Simon Josefsson 
(2)  [jpeg image of size 6048]
(3)  Simon Josefsson 
gpg> keytocard
Signature key ....: EF34 D1F7 95C0 3392 E52A  54FE DFF1 6372 72D5 245B
Encryption key....: E24D 5135 C2FC 905C 8995  ACD8 EC96 9E77 A11F 46D2
Authentication key: [none]
Please select where to store the key:
   (3) Authentication key
Your selection? 3
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
2048-bit RSA key, ID D6987A02, created 2014-06-18
sec  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26
ssb  2048R/72D5245B  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb  2048R/A11F46D2  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
ssb* 2048R/D6987A02  created: 2014-06-18  expires: never     
                     card-no: 0060 00000042
(1)  Simon Josefsson 
(2)  [jpeg image of size 6048]
(3)  Simon Josefsson 
gpg> save
user@debian:~$ 
Take another backup Can you tell yet that I like having backup options? Note that the subkeys are now marked ssb> indicating they are stubs for a smartcard key.
user@debian:~$ gpg --list-secret-keys
/media/FA21-BEC7/gnupghome/secring.gpg
--------------------------------------
sec   3744R/1C5C4717 2014-06-18 [expires: 2014-09-26]
uid                  Simon Josefsson 
uid                  [jpeg image of size 6048]
uid                  Simon Josefsson 
ssb>  2048R/72D5245B 2014-06-18
ssb>  2048R/A11F46D2 2014-06-18
ssb>  2048R/D6987A02 2014-06-18
user@debian:~$ gpg -a --export-secret-keys 1C5C4717 > $GNUPGHOME/../masterstubs.txt
user@debian:~$ gpg -a --export-secret-subkeys 1C5C4717 > $GNUPGHOME/../subkeysstubs.txt
user@debian:~$ gpg -a --export 1C5C4717 > $GNUPGHOME/../publickey.txt
user@debian:~$ cp -a $GNUPGHOME $GNUPGHOME-backup-masterstubs
Transfer to daily machine Copy publickey.txt to your day-to-day laptop and import it.
jas@latte:~$ gpg --import < publickey.txt 
gpg: key 1C5C4717: public key "Simon Josefsson " imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
jas@latte:~$ 
Insert the YubiKey NEO and generate secret key stubs:
jas@latte:~$ gpg --card-status
Application ID ...: D2760001240102000060000000420000
Version ..........: 2.0
Manufacturer .....: unknown
Serial number ....: 00000042
Name of cardholder: Simon Josefsson
Language prefs ...: sv
Sex ..............: male
URL of public key : https://josefsson.org/1c5c4717.txt
Login data .......: jas
Signature PIN ....: forced
Key attributes ...: 2048R 2048R 2048R
Max. PIN lengths .: 0 0 0
PIN retry counter : 0 0 0
Signature counter : 0
Signature key ....: EF34 D1F7 95C0 3392 E52A  54FE DFF1 6372 72D5 245B
      created ....: 2014-06-18 23:03:16
Encryption key....: E24D 5135 C2FC 905C 8995  ACD8 EC96 9E77 A11F 46D2
      created ....: 2014-06-18 23:03:31
Authentication key: 2768 2EF9 415C 19FC F0CC  9CA5 DA81 BA39 D698 7A02
      created ....: 2014-06-18 23:03:59
General key info..: pub  2048R/72D5245B 2014-06-18 Simon Josefsson 
sec#  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26
ssb>  2048R/72D5245B  created: 2014-06-18  expires: 2014-09-26
                      card-no: 0060 00000042
ssb>  2048R/A11F46D2  created: 2014-06-18  expires: 2014-09-26
                      card-no: 0060 00000042
ssb>  2048R/D6987A02  created: 2014-06-18  expires: 2014-09-26
                      card-no: 0060 00000042
jas@latte:~$ 
Now you should have a offline master key with subkey stubs. Note that the master key is not available (sec#) and the subkeys are stubs for smartcard keys (ssb>).
jas@latte:~$ gpg --list-secret-keys 1c5c4717
sec#  3744R/1C5C4717 2014-06-18 [expires: 2014-09-26]
uid                  Simon Josefsson 
uid                  [jpeg image of size 6048]
uid                  Simon Josefsson 
ssb>  2048R/72D5245B 2014-06-18 [expires: 2014-09-26]
ssb>  2048R/A11F46D2 2014-06-18 [expires: 2014-09-26]
ssb>  2048R/D6987A02 2014-06-18 [expires: 2014-09-26]
jas@latte:~$ 
Mark the key as ultimately trusted.
jas@latte:~$ gpg --edit-key 1c5c4717
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: unknown       validity: unknown
sub  2048R/72D5245B  created: 2014-06-18  expires: 2014-09-26  usage: S   
sub  2048R/A11F46D2  created: 2014-06-18  expires: 2014-09-26  usage: E   
sub  2048R/D6987A02  created: 2014-06-18  expires: 2014-09-26  usage: A   
[ unknown] (1). Simon Josefsson 
[ unknown] (2)  [jpeg image of size 6048]
[ unknown] (3)  Simon Josefsson 
gpg> trust
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: unknown       validity: unknown
sub  2048R/72D5245B  created: 2014-06-18  expires: 2014-09-26  usage: S   
sub  2048R/A11F46D2  created: 2014-06-18  expires: 2014-09-26  usage: E   
sub  2048R/D6987A02  created: 2014-06-18  expires: 2014-09-26  usage: A   
[ unknown] (1). Simon Josefsson 
[ unknown] (2)  [jpeg image of size 6048]
[ unknown] (3)  Simon Josefsson 
Please decide how far you trust this user to correctly verify other users' keys
(by looking at passports, checking fingerprints from different sources, etc.)
  1 = I don't know or won't say
  2 = I do NOT trust
  3 = I trust marginally
  4 = I trust fully
  5 = I trust ultimately
  m = back to the main menu
Your decision? 5
Do you really want to set this key to ultimate trust? (y/N) y
pub  3744R/1C5C4717  created: 2014-06-18  expires: 2014-09-26  usage: SC  
                     trust: ultimate      validity: unknown
sub  2048R/72D5245B  created: 2014-06-18  expires: 2014-09-26  usage: S   
sub  2048R/A11F46D2  created: 2014-06-18  expires: 2014-09-26  usage: E   
sub  2048R/D6987A02  created: 2014-06-18  expires: 2014-09-26  usage: A   
[ unknown] (1). Simon Josefsson 
[ unknown] (2)  [jpeg image of size 6048]
[ unknown] (3)  Simon Josefsson 
Please note that the shown key validity is not necessarily correct
unless you restart the program.
gpg> quit
jas@latte:~$ 
Signing keys This needs to be done using your master key, since it is your certification key that will be used. So boot the Live CD and make the usual GnuPG configurations. Below I m signing my own old key (0xB565716F) so the output may look a bit confusing with me signing my own key, but there is really two different keys involved here. The same process apply if you want to sign someone else s key too. Before signing the key, you need to put the public key on a USB stick and move it to the secure machine. On your laptop:
jas@latte:~$ gpg -a --export b565716f > /media/KINGSTON/b565716f.txt
jas@latte:~$ 
On the disconnected machine:
user@debian:~$ gpg --import < /media/KINGSTON/b565716f.txt 
gpg: key B565716F: public key "Simon Josefsson " imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: next trustdb check due at 2014-09-26
user@debian:~$ gpg --sign-key b565716f
pub  1280R/B565716F  created: 2002-05-05  expires: 2014-11-10  usage: SC  
                     trust: unknown       validity: unknown
sub  2048R/105E722E  created: 2012-03-13  expires: 2014-11-10  usage: S   
sub  2048R/728AB82C  created: 2012-03-13  expires: 2014-11-10  usage: E   
sub  2048R/9394F626  created: 2012-03-13  expires: 2014-11-10  usage: A   
sub  1280R/4D5D40AE  created: 2002-05-05  expires: 2014-11-10  usage: E   
sub  1024R/09CC4670  created: 2006-03-18  expired: 2011-05-23  usage: A   
sub  1024R/AABB1F7B  created: 2006-03-18  expired: 2011-05-23  usage: S   
sub  1024R/A14C401A  created: 2006-03-18  expired: 2011-05-23  usage: E   
[ unknown] (1). Simon Josefsson 
[ unknown] (2)  Simon Josefsson 
[ revoked] (3)  Simon Josefsson 
Really sign all user IDs? (y/N) y
User ID "Simon Josefsson " is revoked.  Unable to sign.
pub  1280R/B565716F  created: 2002-05-05  expires: 2014-11-10  usage: SC  
                     trust: unknown       validity: unknown
 Primary key fingerprint: 0424 D4EE 81A0 E3D1 19C6  F835 EDA2 1E94 B565 716F
     Simon Josefsson 
     Simon Josefsson 
This key is due to expire on 2014-11-10.
Are you sure that you want to sign this key with your
key "Simon Josefsson " (1C5C4717)
Really sign? (y/N) y
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 1C5C4717, created 2014-06-18
user@debian:~$ 
Then export the newly signed key back to your laptop for further distribution.
user@debian:~$ gpg -a --export b565716f > /media/KINGSTON/signed-b565716f.txt
user@debian:~$ 
On your laptop, either email it encrypted to the other person, or upload it to keyservers directly depending on your preference. By emailing it encrypted to the other person, they need to prove posession of the key before receiving your signature. In my case, I m the other person, so I just import the signed key and then send the key:
jas@latte:~$ gpg --import < /media/KINGSTON/signed-b565716f.txt
jas@latte:~$ gpg --send-keys b565716f
Key transition Since I'm migrating from an key to a new, I sign my new key using my old key, and publish that signature on keyservers. This allows people to trust my new key more easily. To let the world know about your key transition, I created a key transition statement. The transition statement should be signed by both keys. I created a new temporary GnuPG home directory and imported both master keys, and clearsigned the file. Note that I used "54265e8c!" to make GnuPG use the master key for signing rather than a subkey, which it would normally do.
user@debian:~$ export GNUPGHOME=/tmp/kts
user@debian:~$ mkdir $GNUPGHOME
user@debian:~$ gpg --import b565716f.txt 
gpg: WARNING: unsafe permissions on homedir  /tmp/kts'
gpg: keyring  /tmp/kts/secring.gpg' created
gpg: keyring  /tmp/kts/pubring.gpg' created
gpg: key B565716F: secret key imported
gpg: /tmp/kts/trustdb.gpg: trustdb created
gpg: key B565716F: public key "Simon Josefsson " imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
gpg:       secret keys read: 1
gpg:   secret keys imported: 1
user@debian:~$ gpg --import /media/FA21-AE97/secret-master-subkeys.txt 
gpg: WARNING: unsafe permissions on homedir  /tmp/kts'
gpg: key 54265E8C: secret key imported
gpg: key 54265E8C: public key "Simon Josefsson " imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
gpg:       secret keys read: 1
gpg:   secret keys imported: 1
user@debian:~$ cat key-transition-2014-06-22-unsigned.txt   gpg --clearsign --personal-digest-preferences "SHA512" --local-user b565716f --local-user 54265e8c! > key-transition-2014-06-22.txt 
gpg: WARNING: unsafe permissions on homedir  /tmp/kts'
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
3744-bit RSA key, ID 54265E8C, created 2014-06-22
                  
You need a passphrase to unlock the secret key for
user: "Simon Josefsson "
1280-bit RSA key, ID B565716F, created 2002-05-05
user@debian:~$    
My statement is available as https://josefsson.org/key-transition-2014-06-22.txt if you want to download the signed text file directly. Feel free to base your own document on it, as I based mine on earlier examples. flattr this!

20 June 2014

Ian Donnelly: Week 4: New Release Work

Hi All, Team Elektra recently released version 0.8.6 to download, more information on that here: http://sourceforge.net/p/registry/mailman/message/32418614/ However, since I am working with Debian, and the Debian package isn t ready yet I have spent a large chunk of the week working on that. Namely I fixed a few minor lintian errors and worked on packaging the lua and python bindings for Elektra. These two packages, libelektra-python4 and libelektra-lua4 uses a tool called SWIG. SWIG allows C and C++ programs to connect to programs written in higher-level languages such as Python. So these two packages use the power of SWIG to bring the features of the Elektra API (which is written in C) to Python and Lua developers. I personally think it is a really neat feature and allows Elektra to be used more widely than ever before. As our project develops Elektra will support more languages and we are even moving to allowing Developers to write Elektra plugins using other languages. Additionally, I worked on improving the MergeTools library for Elektra. This library just contains the necessary functions to allow developers to complete a three-way merge of KeySets. This library also serves as the backbone of the kdb merge command. First, the KeySetMerge command now returns an empty KeySet if its unsuccessful. I added operator overloads for == and != to the KeySet.hpp class so developers can tell if the returned keyset is empty. I created a new function that takes in 3 KeySets, their parent keys (for the purposes of merging), and a merge_root key which will be where the merged keys are stored under. The prototype for this function is KeySet(KeySet base, Key base_root, KeySet ours, Key our_root, KeySet theirs, Key their_root, Key merge_root). A function still exists that just takes in 3 KeySets and a key, merge_root. That function attempts to find each root key and then calls the previously mentioned function. Why? This week I finalized the kdb merge command (not including options). This command takes in 4 root keys, one for base, one for ours, one for theirs, and one for where the merged keys should be stored. The merge command now uses the new function to be able to handle more complex KeySets and so far it works great! In this case we don t need to make any assumptions about the KeySet since the user needs to pass in the root keys (this is how the kdb commands typically work) so we make good use of this information. Now the kdb merge command only appends the merge KeySet to the database upon successful merge, otherwise it tells you what went wrong. Also, looking into the main feature of my GSoC project, automatic conffile merging, it seems that it will require patching dpkg. Originally I though package maintainers would have to edit their packages to not define their conffiles as such in order to use the automatic merge. However this is obviously a messy workaround. Patching dpkg however would allow very clean integration of this feature. If this project is successful and the patch makes its way upstream, all Debian packages that utilize conffiles (most of them) will basically allow automatic merging practically overnight. dpkg would attempt a merge on the conffiles during an upgrade and if unsuccessful would just prompt the user like it does now. In most situations however, because of the features of Elektra, conffiles should merge fine. This would make Debian much more user-friendly and could save users a lot of time manually merging conffiles. Oh, and look for a little tutorial and explanation of the kdb mount command in the next few days. I will post it here. Until Then,
Ian S. Donnelly

Next.

Previous.