Review: Clean Sweep, by Ilona Andrews

Series:	Innkeeper Chronicles #1
Publisher:	NYLA
Copyright:	2013
ISBN:	1-62517-343-1
Format:	Kindle
Pages:	228

Dina owns a bed and breakfast in a small town in Texas. As the book opens, her neighbor's dogs are being killed by some beast. As far as Dina is concerned, this should be the problem of the local werewolf, given that this is his territory, and she takes him to task for apparently doing nothing about it. The werewolf doesn't take that well, nor wants to admit to being a werewolf, and reacts by threatening Dina. Given Dina's position and the nature of her bed and breakfast, that turns out to be ill-advised. At this point, you're doubtless thinking urban fantasy, and you're not exactly wrong. You're probably also thinking love interest, and you're not entirely wrong there either. The dead dogs are, of course, an early warning sign of an evil monster who has moved into the area and whose violence may escalate. And if I mention that a vampire shows up later in the story and the werewolf and vampire dislike each other at first sight, you'll probably start rolling your eyes. But this also isn't quite what it looks like on the surface. I'm going to gleefully spoil what you find out about a fifth of the way into the book, since it's what got me to buy this book: neither the werewolves nor the vampires are magical creatures. They're aliens. Werewolves are bio-engineered soldiers; vampires are members of a militaristic order with advanced technology and almost imperceptible circulatory systems. And Dina doesn't truly have a bed and breakfast. She maintains an inn: a refuge for aliens traveling on Earth, part of the secrecy that keeps them out of the eyes of normal humans, and an institution sworn to neutrality in local problems. Dina is straining the last part. The alien creatures who are hunting and killing her neighbors' pets aren't exactly threatening the inn, but she's not willing to stand idly by while her neighbors are threatened. It's a dangerous path, since her inn is struggling already. Failure to prioritize correctly and protect her guests might lower her inn's rating, possibly fatally. She's also without a guide ever since her parents (prominent innkeepers themselves) mysteriously disappeared. But she does have an inn. It may be a weaker one, coaxed out of dormancy and desperately in need of more visitors, but it still has considerable technological resources and other abilities that are indistinguishable from magic. It's also sentient, although more like a large animal than a person, and it knows its innkeeper. This is a light, straightforward sort of book that does not take itself particularly seriously, as you might have guessed. It is very aware of the conventions of urban fantasy and is both following them and intentionally poking fun at them. The authors (Ilona Andrews is a husband and wife writing team) play an impish game of wedging fantasy tropes into a science fiction framework, preserving much of the outward appearance while playing with the explanations. I particularly liked the clannish reworking of vampires into a sort of crusader knight, which works considerably better than it has any right to. Clean Sweep is at its best when the story seems to be going down well-trodden urban fantasy paths and then takes an abrupt right turn into science fiction: Dina going through a dimensional gate to an interstellar marketplace to stock up on weapons, for instance. Also, it has a sentient house. This is one of my favorite story elements ever, provided that the story isn't horror (which is all too often the case with sentient houses, but which is not at all the case here). I found the early parts of this book, during which Sean is insulting Dina and Dina is unimpressed but not doing anything about it, rather tedious. Thankfully, once Dina finally loses her patience and knocks some sense into him it gets a lot better. It never becomes great literature, but sometimes that's a feature. If you're in the mood for some urban fantasy with an adequate, if hand-waved, re-spun science fiction justification and an uncomplicated and loyal sentient house, I can wholeheartedly recommend Clean Sweep. I liked it better than I had expected to. Followed by Sweep in Peace. Rating: 7 out of 10

For many years now, I have been driving a diesel based VW Passat Estate. It has served me very well and been as reliable as I might have hoped given how awful I am with cars. Sadly Gunther was reaching the point where it was going to cost more per year to maintain than the car was worth, and also I've been being more and more irked by not having a car from the future. I spent many months doing spreadsheets, trying to convince myself I could somehow afford a Tesla of some variety. Sadly I never quite managed it. As such I set my sights on the more viable BEVs such as the Nissan Leaf. For a while, nothing I saw was something I wanted. I am quite unusual it seems, in that I don't want a car which is a "Look at me, I'm driving an electric car" fashion statement. I felt like I'd never get something which looked like a normal car, but happened to be a BEV. Then along came the Hyundai Ioniq. Hybrid, Plug-in Hybrid, and BEV all looking basically the same, and not in-your-face-special. I began to covet. Eventually I caved and arranged a test drive of an Ioniq plug-in hybrid because the BEV was basically on 9 month lead. I enjoyed the drive and was instantly very sad because I didn't want a plug-in hybrid, I wanted a BEV. Despondent, I left the dealership and went home. I went online and found a small number of second-hand Ioniq BEVs but as I scrolled through the list, none seemed to be of the right trim level. Then, just as I was ready to give up hope, I saw a new listing, no photo, of the right thing. One snag, it was 200 miles away. No matter, I rang the place, confirmed it was available, and agreed to sleep on the decision. The following morning, I hadn't decided to not buy, so I called them up, put down a deposit to hold the car until I could test drive it, and then began the long and awkward process of working out how I would charge the car given I have no off-street parking so I can't charge at home. (Yeah yeah, you'd think I'd have checked that first, but no I'm just not that sensible). Over the week I convinced myself I could do it, I ordered RFID cards for various schemes, signed up with a number of services, and then, on Friday last week, I drove down to a hotel near the dealership and had a fitful night's sleep. I rocked up to the dealership exactly as they opened for business, shook the hand of the very helpful salesman who had gone through the purchase process with me over the phone during the week, and got to see the car. Instant want coursed through me as I sat in it and decided "Yes, this somehow feels right". I took the car for about a 45 minute test drive just to see how it felt relative to the plug-in hybrid I'd driven the week before and it was like night and day. The BEV felt so much better to drive. I was hooked. Back to the dealership and we began the paperwork. Emptying Gunther of all of the bits and bobs scattered throughout his nooks and crannies took a while and gave me a chance to say goodbye to a car which, on reflection, had actually been a pleasure to own, even when its expensive things went wrong, more than once. But once I'd closed the Passat for the last time, and handed the keys over, it was quite a bittersweet moment as the salesman drove off in what still felt like my car, despite (by this point) it not being so. Sitting in the Ioniq though, I headed off for the 200 mile journey back home. With about 90% charge left after the test drive, I had two stops planned at rapid chargers and I headed toward my first. Unfortunately disaster struck, the rapid (50KW) charger refused to initialise, and I ended up with my car on the slower (7KW) charger to get enough juice into it to drive on to the next rapid charger enabled service station. When I got the message that my maximum charge period (45m) had elapsed, I headed back to the car to discover I couldn't persuade it to unlock from the car. Much hassle later, and an AA man came and together we learned that it takes 2 to tango, one to pull the emergency release in the boot, the other to then unplug the cable. Armed with this knowledge, I headed on my way to a rapid charger I'd found on the map which wasn't run by the same company. Vainly hoping that this would work better, I plugged the car in, set the charger going, and headed into the adjacent shop for a rest break. I went back to the car about 20 minutes later to see the charger wasn't running. Horror of horrors. I imagined maybe some nasty little oik had pressed 'stop' so I started the charger up again, and sat in the car to read my book. After about 3 minutes, the charge stopped. Turns out that charger was slightly iffy and couldn't cope with the charge current and kept emergency-stopping as a result. The lovely lady I spoke to about it directed me to a nearby (12 miles or so, easily done with the charge I had) charger in the grounds of a gorgeous chateau hotel. That one worked perfectly and I filled up. I drove on to my second planned stop and that charge went perfectly too. In fact, every charge since has gone flawlessly. So perhaps my baptism of failed charges has hardened me to the problems with owning a BEV. I've spent the past few days trying different charge points around Manchester enjoying my free charge capability, and trying different names for the car before I finally settled on which is a reasonable Korean boy's name (since the car is Korean I even got to set that as the bluetooth ID) and it's roughly pronounced sock/gin which are two wonderful things in life. I'm currently sat in a pub, eating a burger, enjoying myself while suckles on the teat of "free" electricity to make up for the fact that I've developed a non-trivial habit of leaving Audi drivers in the dust at traffic lights. Further updates may happen as I play with Android Auto and other toys in an attempt to eventually be able to ask the car to please "freeze my buttocks" (a feature it has in the form of air-conditioned seats.)

Review: Raven Stratagem, by Yoon Ha Lee

Series:	Machineries of Empire #2
Publisher:	Solaris
Copyright:	2017
ISBN:	1-78618-046-4
Format:	Kindle
Pages:	400

Raven Stratagem is the sequel to Ninefox Gambit and will make very little sense if you've not read the previous book. In fact, I'll add a rare warning, one that I wish I'd had and followed: you need to be deeply familiar with the details of the end of Ninefox Gambit or large chunks of this book will make no sense. I read the previous book earlier this year, but my memory of some of the specifics of the ending had slipped, and I ended up having to skim the ending of the previous book several times. Consider re-reading that bit before starting this book if you share my lack of memory for plot specifics and it's been more than a few months. I unfortunately can't provide a plot summary, since there's almost nothing I can say about the plot that doesn't spoil Ninefox Gambit. It is basically the story that you would expect from the very end of Ninefox Gambit, though, although the way it's carried out is not quite as dramatic as I was expecting. I wanted to like this book a great deal. Ninefox Gambit introduced a beautiful magitech system, but it was otherwise mostly setup and one extended battle. Its ending promised engagement with larger universe politics, and prospects of doing something about the deep unfairness of the current political system. I was hoping this book would contain the payoff of that escalation. It does deliver on that payoff, but something about it didn't quite work for me. The best description I've been able to come up with is that Raven Stratagem skitters. Some of this is an increase in the number of viewpoint characters, which made it harder for me to get into a reading rhythm with any of them. But most of it, I think, is that the characters have so many layers of deception, emotional shielding, weariness, and resignation that it's hard to find the true emotional beats of the story except in retrospect. I kept bouncing off surfaces, so many different and conflicting surfaces. This book is full of people who are not being honest with each other, or sometimes with themselves, and who are pretending to motives they don't really have. As a reader, I wanted to be placed in a privileged position where I could experience the character emotions even when they were lying about them. For good or ill, Raven Stratagem doesn't do that. There was also something about the dramatic structure of the story that didn't work for me. When describing the book to a friend, I said that the main plot climax was off-camera. In skimming the book again for this review, I found that wasn't the case, but it still felt that way. I think that's because, despite some event build-up, the emotional build-up wasn't in place for me, so I wasn't ready as a reader for the climax when it came. The build-up to the climax is partly sacrificed to keep the secrecy of a couple of long-awaited revelations. I very much enjoyed those revelations (one satisfying one was set up with Cheris's behavior at the start of Ninefox Gambit), but I wanted the catharsis of the climax as well. As written, the strongest emotional hit was from a somewhat ancillary climax, and that involved characters who mattered considerably less to me than Cheris. The climax also involves quite a lot of hand-waving. While some of that is expected in magitech, I would have liked to understand the mechanisms of what happened, not just the effects. Lee introduces several new viewpoint characters here, including two very contrasting Kel. I warmed to them by the end of the book, but I liked Cheris as a viewpoint character considerably better than either of them. Both of them spend most of this book in conditions of varying powerlessness; Cheris, despite difficult circumstances, was at least driving the plot. I can kind of see why Lee picked the viewpoint characters he did, but I still feel grumbly about it. I would have loved to have a servitor as a primary viewpoint character in this story. All that said, I'm still glad I read this book. The climax is satisfying, as is the growing respect of the characters and the growing realization of just how the universe is being changed. I wanted more of that on camera rather than being held for dramatic surprise, but I still savored it when it happened. The mechanisms of the Hexarchate, particularly formation instinct, are considerably creepier than Ninefox Gambit reveals, which is saying something, and yet oddly logical in their own magitech way. I liked all the pieces; I just wanted them to have more emotional oomph and momentum. Instead, I felt like I was bringing my own emotion to the story rather than letting the story sweep me away, which meant being more analytical and less engrossed than I prefer to be in a novel. I'm still going to read the third book, though. By the end of Raven Stratagem, Lee has set most of the scenery on fire, and I want to see what sort of world rises from the flames. Rating: 6 out of 10

The other day I wanted to add support for tests of javascript functionality in a (dockerized) rails application using rspec-rails. Since rails 5.1 includes system tests with niceties like automatically taking a screenshot on failed tests, I hoped for a way to benefit from this
features without changing to another test framework. Lucky me only recently the authors of rspec-rails added support for so-called system specs. There is not much documentation so far (but there are a lot of useful information in the corresponding bug report #1838 and a friendly guy named Thomas Walpole (@twalpole) is helpfully answering to questions in that issue. To make things a little bit more complicated: the application in question is usually running in a docker container and thus the tests of the application are also run in a docker container. I didn t want to change this, so here is what it took me to get this running. Overview Let s see what we want to achieve exactly: From a technical point of view, we will have the application under test (AUT) and the tests in one container (let s call it: web) and we will need another container running a javascript-capable browser (let s call it browser). Thus we need the tests to drive a remote running browser (at least when running in a docker environment) which needs to access the application under a different address than usually. Namely an address reachable by the chrome-container, since it will not be reachable via 127.0.0.1 as is (rightfully) assumed by default. If we want Warden authentication stubbing to work (as we do, since our application uses Devise) and transactional fixtures as well (e.g. rails handling database cleanup between tests without database_cleaner gem) we also need to ensure that the application server is being started by the tests and the tests are actually run against that server. Otherwise we might run into problems. Getting the containers ready Assuming you already have a container setup (and are using docker-compose like we do) there is not that much to change on the docker front. Basically you need to add a new service called chrome and point it to an appropriate image and add a link to it in your existing web-container. I ve decided to use standalone-chrome for the browser part, for which there are docker images provided by the selenium project (they also have images for other browsers). Kudos for that.

...
services:
  chrome:
    image: selenium/standalone-chrome
   
  web:
   
    links:
      - chrome

The link ensures that the chrome instance is available before we run the tests and that the the web-container is able to resolve the name of this container. Unfortunately this is not true for the other way round, so we need some magic in our test code to find out the ip-address of the web-container. More to this later. Other than that, you probably want to configure a volume for you to be able to access the screenshots, which get saved to tmp/screenshots
in the application directory. Preparing the application for running system tests There is a bit more to do on the application side. The steps are roughly:

1. Add necessary depends / version constraints
2. Register a driver for the remote chrome
3. Configure capybara to use the appropriate host for your tests (and your configured driver)
4. Add actual tests with type: :system and js: true

Let s walk them through. Add necessary depends What we need is the following:

• rspec-rails version >= 3.7.1
• rails itself > 5.1.4 (unreleased at time of writing)
• capybara and capybara-selenium

The required features are already part of 3.7.0, but this version is the version I used and it contains a bugfix, which may or may not be relevant. One comment about the rails version: for the tests to properly work it s viable to have puma use certain settings. In rails 5.1.4 (the version released, at time of writing this) uses the settings from config/puma.rb which most likely collides with the necessary settings. You can ensure these settings yourself or use rails from branch 5-1-stable which includes this change. I decided for the latter and pinned my Gemfile to the then current commit. Register a driver for the remote chrome To register the required driver, you ll have to add some lines to your rails_helper.rb:

if ENV['DOCKER']
  selenium_url =
      "http://chrome:4444/wd/hub"
  Capybara.register_driver :selenium_remote do  app 
    Capybara::Selenium::Driver.new(app,
         :url => selenium_url, :browser => :remote, desired_capabilities: :chrome )
  end
end

Note that I added those lines conditionally (since I still want to be able to use a local chrome via chromedriver) if an environment variable DOCKER is set. We defined that environment variable in our Dockerfile and thus you might need to adapt this to your case. Also note that the selenium_url is hard-coded. You could very well take a different approach, e.g. using an externally specified SELENIUM_URL, but ultimately the requirement is that the driver needs to know that the chrome instance is running on host chrome, port 4444 (the containers default). Configure capybara to use the appropriate host and driver The next step is to ensure that javascript-requiring system tests are actually run with the given driver and use the right host. To achieve that we need to add a before-hook to the corresponding tests or we can configure rspec accordingly to always include such a hook by modifying the rspec-configuration in rails_helper.rb like this:

RSpec.configure do  config 
  ...
  config.before(:each, type: :system, js: true) do
    if ENV['DOCKER']
      driven_by :selenium_remote
      ip = Socket.ip_address_list.detect addr  addr.ipv4_private?  .ip_address
      host! "http://# ip :# Capybara.server_port "
    else
      driven_by :headless_chrome
    end
  end

Note the part with the ip-address: it tries to find an IPv4 private address for the web-container (the container running the tests) to ensure the chrome-container uses this address to access the application. The Capybara.server_port is important here, since it will correspond to the puma instance launched by the tests. That heuristic (first ipv4 private address) works for us at the moment, but it might not work for you. It is basically a workaround to the fact that I couldn t get web resolvable for the chrome container which may be fixable on the docker side, but I was to lazy to further investigate that. If you change it: Just make sure the host! method uses an URI pointing to an address of the web-container that is reachable to the chrome-container. Define tests with type: :system and js: true Last but certainly not least, you need actual tests of the required type and with or without js: true. This can be achieved by creating tests files starting like this:

RSpec.feature "Foobar", type: :system, js: true do

Since the new rspec-style system tests are based around the feature-specs which used to be around previously, the rest of the tests is exactly like it is described for feature specs. Run the tests To run the tests a commandline like the following should do: docker-compose run web rspec It won t make a big noise about running the tests against chrome, unless something fails. In that case you ll see a message telling you where the screenshot has been placed. Troubleshooting Below I add some hints about problems I ve seen during configuring that: Test failing, screenshot shows login screen In that case puma might be configured wrongly or you are not using transactional fixtures. See the hints above about the rails version to use which also includes some pointers to helpful explanations. Note that rspec-rails by default does not output the puma startup output as it clutters the tests. For debugging purposes it might be helpful to change that by adding the following line to your tests:

ActionDispatch::SystemTesting::Server.silence_puma = false

Error message: Unable to find chromedriver This indicates that your driver is not configured properly, because the default for system tests is to be driven_by selenium, which tries to spawn an own chrome instance and is suitable for non-dockerized tests. Check if your tests are marked as js: true (if you followed the instructions above) and that you properly added the before-hook to your rspec-configuration. Collisions with VCR If you happen to have tests that make use of the vcr gem you might see it complaining about not knowing what to do with the requests between the driver and the chrome instance. You can fix this, by telling VCR to ignore that requests, by adding a line where you configured VCR:

VCR.configure do  config 
# required so we don't collide with capybara tests
config.ignore_hosts 'chrome'
...

Review: Algorithms to Live By, by Brian Christian & Tom Griffiths

Publisher:	Henry Holt and Company
Copyright:	April 2016
ISBN:	1-62779-037-3
Format:	Kindle
Pages:	255

Another read for the work book club. This was my favorite to date, apart from the books I recommended myself. One of the foundations of computer science as a field of study is research into algorithms: how do we solve problems efficiently using computer programs? This is a largely mathematical field, but it's often less about ideal or theoretical solutions and more about making the most efficient use of limited resources and arriving at an adequate, if not perfect, answer. Many of these problems are either day-to-day human problems or are closely related to them; after all, the purpose of computer science is to solve practical problems with computers. The question asked by Algorithms to Live By is "can we reverse this?": can we learn lessons from computer science's approach to problems that would help us make day-to-day decisions? There's a lot of interesting material in the eleven chapters of this book, but there's also an amusing theme: humans are already very good at this. Many chapters start with an examination of algorithms and mathematical analysis of problems, dive into a discussion of how we can use those results to make better decisions, then talk about studies of the decisions humans actually make... and discover that humans are already applying ad hoc versions of the best algorithms we've come up with, given the constraints of typical life situations. It tends to undermine the stated goal of the book. Thankfully, it in no way undermines interesting discussion of general classes of problems, how computer science has tackled them, and what we've learned about the mathematical and technical shapes of those problems. There's a bit less self-help utility here than I think the authors had intended, but lots of food for thought. (That said, it's worth considering whether this congruence is less because humans are already good at this and more because our algorithms are designed from human intuition. Maybe our best algorithms just reflect human thinking. In some cases we've checked our solutions against mathematical ideals, but in other cases they're still just our best guesses to date.) This is the sort of a book where a chapter listing is an important part of the review. The areas of algorithms discussed here are optimal stopping, explore/exploit decisions (when to go with the best thing you've found and when to look for something better), sorting, caching, scheduling, Bayes's rule (and prediction in general), overfitting when building models, relaxation (solving an easier problem than your actual problem), randomized algorithms, a collection of networking algorithms, and finally game theory. Each of these has useful insights and thought-provoking discussion of how these sometimes-theoretical concepts map surprisingly well onto daily problems. The book concludes with a discussion of "computational kindness": an encouragement to reduce the required computation and complexity penalty for both yourself and the people you interact with. If you have a computer science background (as I do), many of these will be familiar concepts, and you might be dubious that a popularization would tell you much that's new. Give this book a shot, though; the analogies are less stretched than you might fear, and the authors are both careful and smart about how they apply these principles. This book passes with flying colors a key sanity check: the chapters on topics that I know well or have thought about a lot make few or no obvious errors and say useful and important things. For example, the scheduling chapter, which unsurprisingly is about time management, surpasses more than half of the time management literature by jumping straight to the heart of most time management problems: if you're going to do everything on a list, it rarely matters the order in which you do it, so the hardest scheduling problems are about deciding what not to do rather than deciding order. The point in the book where the authors won my heart completely was in the chapter on Bayes's rule. Much of the chapter is about Bayesian priors, and how one's knowledge of past events is a vital part of analysis of future probabilities. The authors then discuss the (in)famous marshmallow experiment, in which children are given one marshmallow and told that if they refrain from eating it until the researcher returns, they'll get two marshmallows. Refraining from eating the marshmallow (delayed gratification, in the psychological literature) was found to be associated with better life outcomes years down the road. This experiment has been used and abused for years for all sorts of propaganda about how trading immediate pleasure for future gains leads to a successful life, and how failure in life is because of inability to delay gratification. More evil analyses have (of course) tied that capability to ethnicity, with predictably racist results. I have kind of a thing about the marshmallow experiment. It's a topic that reliably sends me off into angry rants. Algorithms to Live By is the only book I have ever read to mention the marshmallow experiment and then apply the analysis that I find far more convincing. This is not a test of innate capability in the children; it's a test of their Bayesian priors. When does it make perfect sense to eat the marshmallow immediately instead of waiting for a reward? When their past experience tells them that adults are unreliable, can't be trusted, disappear for unpredictable lengths of time, and lie. And, even better, the authors supported this analysis with both a follow-up study I hadn't heard of before and with the observation that some children would wait for some time and then "give in." This makes perfect sense if they were subconsciously using a Bayesian model with poor priors. This is a great book. It may try a bit too hard in places (applicability of the math of optimal stopping to everyday life is more contingent and strained than I think the authors want to admit), and some of this will be familiar if you've studied algorithms. But the writing is clear, succinct, and very well-edited. No part of the book outlives its welcome; the discussion moves right along. If you find yourself going "I know all this already," you'll still probably encounter a new concept or neat explanation in a few more pages. And sometimes the authors make connections that never would have occurred to me but feel right in retrospect, such as relating exponential backoff in networking protocols to choosing punishments in the criminal justice system. Or the realization that our modern communication world is not constantly connected, it's constantly buffered, and many of us are suffering from the characteristic signs of buffer bloat. I don't think you have to be a CS major, or know much about math, to read this book. There is a lot of mathematical details in the end notes if you want to dive in, but the main text is almost always readable and clear, at least so far as I could tell (as someone who was a CS major and has taken a lot of math, so a grain of salt may be indicated). And it still has a lot to offer even if you've studied algorithms for years. The more I read of this book, the more I liked it. Definitely recommended if you like reading this sort of analysis of life. Rating: 9 out of 10

Here s my weekly report for week 42 of 2017. In this week I have replaced my spacebar, failed to replace a HDD and begun the process to replace my YubiKey.

Debian Eariler in the week I blogged about powerline-taskwarrior . There is a new upstream version available that includes the patches I had produced for Python 2 support and I have filed #879225 to remind me to package this. The state of emscripten is still not great, and as I don t have the time to chase this up and I certainly don t have the time to fix it myself, I ve converted the ITP for csdr to an RFP. As I no longer have the time to maintain map.debian.net, I have released this domain name and published the sources behind the service.

• Bugs opened: #879225

Tor Project There was a request to remove the $ from family fingerprint on Atlas. These actually come from Onionoo and we have decided to fix this in Onionoo, but I did push a small fix for Atlas this week that makes sure that Atlas doesn t care if there are $ prefixes or not. I requested that a Trac component be created for metrics-bot. I wrote a seperate post about metrics-bot. I also attended the weekly metrics team meeting.

• Bugs opened: #23926, #23929, #23933, #23934, #23935, #23936, #23937, #23938, #23939, #23941

Sustainability I believe it is important to be clear not only about the work I have already completed but also about the sustainability of this work into the future. I plan to include a short report on the current sustainability of my work in each weekly report. I have not had any free software related expenses this week. The current funds I have available for equipment, travel and other free software expenses remains 60.52. I do not believe that any hardware I rely on is looking at imminent failure. I do not find it likely that I ll be travelling to Cambridge for the miniDebConf as the train alone would be around 350 and hotel accomodation a further 600 (to include both me and Ana).

In the pkg-go team, we are currently discussing which workflows we should standardize on. One of the considerations is what goes into the upstream Git branch of our repositories: should it track the upstream Git repository, or should it contain orig tarball imports? Now, tracking the upstream Git repository only works if upstream actually uses Git. The go tool, which is widely used within the Go community for managing Go packages, supports Git, Mercurial, Bazaar and Subversion. But which of these are actually used in practice? Let s find out! Option 1: If you have the sources lists of all suites locally anyway

/usr/lib/apt/apt-helper cat-file \
  $(apt-get indextargets --format '$(FILENAME)' 'ShortDesc: Sources' 'Origin: Debian') \
    sed -n 's,Go-Import-Path: ,,gp' \
    sort -u

Option 2: If you prefer to use a relational database over textfiles This is the harder option, but also the more complete one. First, we ll need the Go package import paths of all Go packages which are in Debian. We can get them from the ProjectB database, Debian s main PostgreSQL database containing all of the state about the Debian archive. Unfortunately, only Debian Developers have SSH access to a mirror of ProjectB at the moment. I contacted DSA to ask about providing public ProjectB access.

  ssh mirror.ftp-master.debian.org "echo \"SELECT value FROM source_metadata \
  LEFT JOIN metadata_keys ON (source_metadata.key_id = metadata_keys.key_id) \
  WHERE metadata_keys.key = 'Go-Import-Path' GROUP BY value\"   \
    psql -A -t service=projectb" > go_import_path.txt

I uploaded a copy of resulting go_import_path.txt, if you re curious. Now, let s come up with a little bit of Go to print the VCS responsible for each specified Go import path:

go get -u golang.org/x/tools/go/vcs
cat >vcs4.go <<'EOT'
package main
import (
	"fmt"
	"log"
	"os"
	"sync"
	"golang.org/x/tools/go/vcs"
)
func main()  
	var wg sync.WaitGroup
	for _, arg := range os.Args[1:]  
		wg.Add(1)
		go func(arg string)  
			defer wg.Done()
			rr, err := vcs.RepoRootForImportPath(arg, false)
			if err != nil  
				log.Println(err)
				return
			 
			fmt.Println(rr.VCS.Name)
		 (arg)
	 
	wg.Wait()
 
EOT

Lastly, run it in combination with uniq(1) to discover

go run vcs4.go $(tr '\n' ' ' < go_import_path.txt)   sort   uniq -c
    760 Git
      1 Mercurial

Here s are some of the bugs against the Debian Policy Manual. In particular, there really are quite a few patches needing seconds from DDs. Consensus has been reached and help is needed to write a patch: #759316 Document the use of /etc/default for cron jobs #761219 document versioned Provides #767839 Linking documentation of arch:any package to arch:all #770440 policy should mention systemd timers #773557 Avoid unsafe RPATH/RUNPATH #780725 PATH used for building is not specified #793499 The Installed-Size algorithm is out-of-date #810381 Update wording of 5.6.26 VCS-* fields to recommend encryption #823256 Update maintscript arguments with dpkg >= 1.18.5 #833401 virtual packages: dbus-session-bus, dbus-default-session-bus #835451 Building as root should be discouraged #838777 Policy 11.8.4 for x-window-manager needs update for freedesktop menus #845715 Please document that packages are not allowed to write outside thei #853779 Clarify requirements about update-rc.d and invoke-rc.d usage in mai #874019 Note that the -e argument to x-terminal-emulator works like #874206 allow a trailing comma in package relationship fields Wording proposed, awaiting review from anyone and/or seconds by DDs: #688251 Built-Using description too aggressive #737796 copyright-format: support Files: paragraph with both abbreviated na #756835 Extension of the syntax of the Packages-List field. #786470 [copyright-format] Add an optional License-Grant field #810381 Update wording of 5.6.26 VCS-* fields to recommend encryption #835451 Building as root should be discouraged #845255 Include best practices for packaging database applications #846970 Proposal for a Build-Indep-Architecture: control file field #850729 Documenting special version number suffixes #864615 please update version of posix standard for scripts (section 10.4) #874090 Clarify wording of some passages #874095 copyright-format: Use the synopsis term established in the de Merged for the next release: #683495 perl scripts: #!/usr/bin/perl MUST or SHOULD? #877674 [debian-policy] update links to the pdf and other formats of the do #878523 [PATCH] Spelling fixes

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In August, about 170 work hours have been dispatched among 13 paid contributors. Their reports are available:

• Antoine Beaupr did 7h (out of 15.75h allocated, thus keeping 8.75h for October).
• Ben Hutchings did 12 hours (out of 15h allocated + 6 extra hours, thus keeping 9 extra hours for October).
• Brian May did 10 hours.
• Chris Lamb did 15.75 hours.
• Emilio Pozuelo Monfort did 27 hours (out of 15.75 hours allocated + 12.75 hours remaining, thus keeping 1.5 hours for October).
• Guido G nther did 10 hours (out of 11h allocated, thus keeping one hour for October).
• Hugo Lefeuvre did 15h.
• Lucas Kanashiro did 5 hours.
• Markus Koschany did 15.75 hours.
• Ola Lundqvist did 13.5h (out of 7h allocated + 7 extra hours, thus keeping 0.5 extra hours for October).
• Rapha l Hertzog did 10.5 hours (out of 12h allocated, thus keeping 1.5 hours for October).
• Roberto C. Sanchez did 10 hours (out of 15.75 hours allocated + 9 hours remaining, thus keeping 14.75 extra hours for October).
• Thorsten Alteholz did 15.75 hours.

Evolution of the situation The number of sponsored hours is the same as last month. But we have a new sponsor in the pipe. The security tracker currently lists 52 packages with a known CVE and the dla-needed.txt file 49. The number of packages with open issues decreased slightly compared to last month but we re not yet back to the usual situation. Thanks to our sponsors New sponsors are in bold.

• Platinum sponsors:
• TOSHIBA (for 24 months)
• GitHub (for 15 months)
• Gold sponsors:
• The Positive Internet (for 40 months)
• Blablacar (for 39 months)
• Linode (for 29 months)
• Babiel GmbH (for 18 months)
• Plat Home (for 18 months)
• Silver sponsors:
• Domeneshop AS (for 40 months)
• Universit Lille 3 (for 39 months)
• Trollweb Solutions (for 37 months)
• Nantes M tropole (for 34 months)
• Dalenys (for 30 months)
• Univention GmbH (for 25 months)
• Universit Jean Monnet de St Etienne (for 25 months)
• Sonus Networks (for 19 months)
• maxcluster GmbH (for 13 months)
• Exonet B.V. (for 9 months)
• Leibniz Rechenzentrum (for 3 months)
• Bronze sponsors:
• David Ayers IntarS Austria (for 40 months)
• Evolix (for 40 months)
• Offensive Security (for 40 months)
• Seznam.cz, a.s. (for 40 months)
• Freeside Internet Service (for 39 months)
• MyTux (for 39 months)
• Intevation GmbH (for 37 months)
• Linuxhotel GmbH (for 37 months)
• Daevel SARL (for 36 months)
• Bitfolk LTD (for 34 months)
• Megaspace Internet Services GmbH (for 34 months)
• NUMLOG (for 34 months)
• Greenbone Networks GmbH (for 33 months)
• WinGo AG (for 33 months)
• Ecole Centrale de Nantes LHEEA (for 29 months)
• Sig-I/O (for 27 months)
• Entr ouvert (for 24 months)
• Adfinis SyGroup AG (for 22 months)
• GNI MEDIA (for 16 months)
• Laboratoire LEGI UMR 5519 / CNRS (for 16 months)
• Quarantainenet BV (for 16 months)
• RHX Srl (for 13 months)
• Bearstech (for 8 months)
• LiHAS (for 7 months)
• People Doc (for 4 months)
• Catalyst IT Ltd

No comment Liked this article? Click here. My blog is Flattr-enabled.

I just finished the presentation at the Neo4j Online Meetup on getting the Debian UDD into a Neo4j graph database. Besides the usual technical quibbles it did work out quite well. The code for pulling the data from the UDD, as well as converting and importing it into Neo4j is available on Github Debian-Graph. The slides are also available on Github: preining-debian-packages-neo4j.pdf. There are still some things I want to implement, time permitting, because it would be a great tool for better integration for Debian. In any case, graph databases are lots of fun to play around.

It has been ages that I haven t updated the MegaUpdate package for Kobo. Now that a new and seemingly rather bug-free and quick firmware release (4.6.9995) has been released, I finally took the time to update the whole package to the latest releases of all the included items. The update includes all my favorite patches and features: Kobo Start Menu, koreader, coolreader, pbchess, ssh access, custom dictionaries, and some side-loaded fonts. So what are all these items:

• firmware (thread): the basic software of the device, shipped by Kobo company
• Metazoa firmware patches (thread): fix some layout options and functionalities, see below for details.
• Kobo Start Menu (V08, update 5b thread): a menu that pops up before the reading software (nickel) starts, which allows to start alternative readers (like koreader) etc.
• KOreader (koreader-nightly-20171004, thread): an alternative document reader that supports epub, azw, pdf, djvu and many more
• pbchess and CoolReader (2017.10.14, thread): a chess program and another alternative reader, bundled together with several other games
• kobohack (web site): I only use the ssh server
• ssh access (old post: makes a full computer from your device by allowing you to log into it via ssh
• custom dictionaries (thread): this fix updates dictionaries from the folder customdicts to the Kobo dictionary folder. For creating your own Japanese-English dictionary, see this blog entry
• side-loaded fonts: GentiumBasic and GentiumBookBasic, Verdana, DroidSerif, and Charter-eInk

Install procedure Download Mark6 Kobo GloHD firmware: Kobo 4.6.9995 for GloHD Mega update: Kobo-4.6.9995-combined/Mark6/KoboRoot.tgz Mark5 Aura firmware: Kobo 4.6.9995 for Aura Mega update: Kobo-4.6.9995-combined/Mark5/KoboRoot.tgz Mark4 Kobo Glo, Aura HD firmware: Kobo 4.6.9995 for Glo and AuraHD Mega update: Kobo-4.6.9995-combined/Mark4/KoboRoot.tgz Latest firmware Warning: Sideloading or crossloading the incorrect firmware can break/brick your device. The link below is for Kobo GloHD ONLY. The first step is to update the Kobo to the latest firmware. This can easily be done by just getting the latest firmware from the links above and unpacking the zip file into the .kobo directory on your device. Eject and enjoy the updating procedure. Mega update Get the combined KoboRoot.tgz for your device from the links above and put it into the .kobo directory, then eject and enjoy the updating procedure again. After this the device should reboot and you will be kicked into KSM, from where after some time of waiting Nickel will be started. If you consider the fonts too small, select Configure, then the General, and add item, then select kobomenuFontsize=55 and save. Remarks to some of the items included The full list of included things is above, here are only some notes about what specific I have done.

• Metazoa firmware patches Included patches from the Metazoa firmware patches:

  Custom left & right margins
  Fix three KePub fullScreenReading bugs
  Change dicthtml strings to micthtml
  Default ePub monospace font (Courier)
  Custom reading footer style
  Dictionary pop-up frame size increase
  Increase The Cover Size In Library
  Increasing The View Details Container
  New home screen increasing cover size
  Reading stats/Author name cut when the series is showing bug fix 
  New home screen subtitle custom font
  Custom font to Collection and Authors names
  

  If you need/want different patches, you need to do the patching yourself.
• kobohack-h Kobohack (latest version 20150110) originally provided updated libraries and optimizations, but unfortunately it is now completely outdated and using it is not recommended for the library part. I only include the ssh server (dropbear) so that connections to the Kobo via ssh.
• ssh fixes See the detailed instructions here, the necessary files are already included in the mega upload. It updates the /etc/inittab to run also /etc/init.d/rcS2, and this one again starts the inetd server and run user supplied commands in /mnt/onboard/run.sh which is where your documents are.
• Custom dictionaries The necessary directories and scripts are already included in the above combined KoboRoot.tgz, so nothing to be done but dropping updated, fixed, changed dictionaries into your Kobo root into the directory customdict. After this you need to reboot to get the actual dictionaries updated. See this thread for more information. The adaptions and script mentioned in this post are included in the mega update.

WARNINGS If this is the first time you install this patch, you to fix the password for root and disable telnet. This is an important step, here are the steps you have to take (taken from this old post):

1. Turn on Wifi on the Kobo and find IP address
   Go to Settings Connect and after this is done, go to Settings Device Information where you will see something like
IP Address: 192.168.1.NN
   (numbers change!)
2. telnet into your device
   telnet 192.168.1.NN
   it will ask you the user name, enter root (without the quotes) and no password
3. (ON THE GLO) change home directory of root
   edit /etc/passwd with vi and change the entry for root by changing the 6th field from: / to /root (without the quotes). After this procedure the line should look like
   root::0:0:root:/root:/bin/sh
   don t forget to save the file
4. (ON THE GLO) create ssh keys for dropbear
   [root@(none) ~]# mkdir /etc/dropbear
[root@(none) ~]# cd /etc/dropbear
[root@(none) ~]# dropbearkey -t dss -f dropbear_dss_host_key
[root@(none) ~]# dropbearkey -t rsa -f dropbear_rsa_host_key
5. (ON YOUR PERSONAL COMPUTER) check that you can log in with ssh
   ssh root@192.168.1.NN
   You should get dropped into your device again
6. (ON THE GLO) log out of the telnet session (the first one you did)
   [root@(none) ~]# exit
7. (ON THE GLO) in your ssh session, change the password of root
   [root@(none) ~]# passwd
   you will have to enter the new password two times. Remember it well, you will not be easily able to recover it without opening your device.
8. (ON THE GLO) disable telnet login
   edit the file /etc/inetd.conf.local on the GLO (using vi) and remove the telnet line (the line starting with 23).
9. restart your device

The combined KoboRoot.tgz is provided without warranty. If you need to reset your device, don t blame me!

Confession time: I started making these posts (eons ago) because a close friend did as well, and I enjoyed reading them. But the main reason why I continue is because the primary way I have to keep track of the books I've bought and avoid duplicates is, well, grep on these posts. I should come up with a non-bullshit way of doing this, but time to do more elegant things is in short supply, and, well, it's my blog. So I'm boring all of you who read this in various places with my internal bookkeeping. I do try to at least add a bit of commentary. This one will be more tedious than most since it includes five separate Humble Bundles, which increases the volume a lot. (I just realized I'd forgotten to record those purchases from the past several months.) First, the individual books I bought directly: Ilona Andrews Sweep in Peace (sff)
Ilona Andrews One Fell Sweep (sff)
Steven Brust Vallista (sff)
Nicky Drayden The Prey of Gods (sff)
Meg Elison The Book of the Unnamed Midwife (sff)
Pat Green Night Moves (nonfiction)
Ann Leckie Provenance (sff)
Seanan McGuire Once Broken Faith (sff)
Seanan McGuire The Brightest Fell (sff)
K. Arsenault Rivera The Tiger's Daughter (sff)
Matthew Walker Why We Sleep (nonfiction)
Some new books by favorite authors, a few new releases I heard good things about, and two (Night Moves and Why We Sleep) from references in on-line articles that impressed me. The books from security bundles (this is mostly work reading, assuming I'll get to any of it), including a blockchain bundle: Wil Allsop Unauthorised Access (nonfiction)
Ross Anderson Security Engineering (nonfiction)
Chris Anley, et al. The Shellcoder's Handbook (nonfiction)
Conrad Barsky & Chris Wilmer Bitcoin for the Befuddled (nonfiction)
Imran Bashir Mastering Blockchain (nonfiction)
Richard Bejtlich The Practice of Network Security (nonfiction)
Kariappa Bheemaiah The Blockchain Alternative (nonfiction)
Violet Blue Smart Girl's Guide to Privacy (nonfiction)
Richard Caetano Learning Bitcoin (nonfiction)
Nick Cano Game Hacking (nonfiction)
Bruce Dang, et al. Practical Reverse Engineering (nonfiction)
Chris Dannen Introducing Ethereum and Solidity (nonfiction)
Daniel Drescher Blockchain Basics (nonfiction)
Chris Eagle The IDA Pro Book, 2nd Edition (nonfiction)
Nikolay Elenkov Android Security Internals (nonfiction)
Jon Erickson Hacking, 2nd Edition (nonfiction)
Pedro Franco Understanding Bitcoin (nonfiction)
Christopher Hadnagy Social Engineering (nonfiction)
Peter N.M. Hansteen The Book of PF (nonfiction)
Brian Kelly The Bitcoin Big Bang (nonfiction)
David Kennedy, et al. Metasploit (nonfiction)
Manul Laphroaig (ed.) PoC GTFO (nonfiction)
Michael Hale Ligh, et al. The Art of Memory Forensics (nonfiction)
Michael Hale Ligh, et al. Malware Analyst's Cookbook (nonfiction)
Michael W. Lucas Absolute OpenBSD, 2nd Edition (nonfiction)
Bruce Nikkel Practical Forensic Imaging (nonfiction)
Sean-Philip Oriyano CEHv9 (nonfiction)
Kevin D. Mitnick The Art of Deception (nonfiction)
Narayan Prusty Building Blockchain Projects (nonfiction)
Prypto Bitcoin for Dummies (nonfiction)
Chris Sanders Practical Packet Analysis, 3rd Edition (nonfiction)
Bruce Schneier Applied Cryptography (nonfiction)
Adam Shostack Threat Modeling (nonfiction)
Craig Smith The Car Hacker's Handbook (nonfiction)
Dafydd Stuttard & Marcus Pinto The Web Application Hacker's Handbook (nonfiction)
Albert Szmigielski Bitcoin Essentials (nonfiction)
David Thiel iOS Application Security (nonfiction)
Georgia Weidman Penetration Testing (nonfiction)
Finally, the two SF bundles: Buzz Aldrin & John Barnes Encounter with Tiber (sff)
Poul Anderson Orion Shall Rise (sff)
Greg Bear The Forge of God (sff)
Octavia E. Butler Dawn (sff)
William C. Dietz Steelheart (sff)
J.L. Doty A Choice of Treasons (sff)
Harlan Ellison The City on the Edge of Forever (sff)
Toh Enjoe Self-Reference ENGINE (sff)
David Feintuch Midshipman's Hope (sff)
Alan Dean Foster Icerigger (sff)
Alan Dean Foster Mission to Moulokin (sff)
Alan Dean Foster The Deluge Drivers (sff)
Taiyo Fujii Orbital Cloud (sff)
Hideo Furukawa Belka, Why Don't You Bark? (sff)
Haikasoru (ed.) Saiensu Fikushon 2016 (sff anthology)
Joe Haldeman All My Sins Remembered (sff)
Jyouji Hayashi The Ouroboros Wave (sff)
Sergei Lukyanenko The Genome (sff)
Chohei Kambayashi Good Luck, Yukikaze (sff)
Chohei Kambayashi Yukikaze (sff)
Sakyo Komatsu Virus (sff)
Miyuki Miyabe The Book of Heroes (sff)
Kazuki Sakuraba Red Girls (sff)
Robert Silverberg Across a Billion Years (sff)
Allen Steele Orbital Decay (sff)
Bruce Sterling Schismatrix Plus (sff)
Michael Swanwick Vacuum Flowers (sff)
Yoshiki Tanaka Legend of the Galactic Heroes, Volume 1: Dawn (sff)
Yoshiki Tanaka Legend of the Galactic Heroes, Volume 2: Ambition (sff)
Yoshiki Tanaka Legend of the Galactic Heroes, Volume 3: Endurance (sff)
Tow Ubukata Mardock Scramble (sff)
Sayuri Ueda The Cage of Zeus (sff)
Sean Williams & Shane Dix Echoes of Earth (sff)
Hiroshi Yamamoto MM9 (sff)
Timothy Zahn Blackcollar (sff)
Phew. Okay, all caught up, and hopefully won't have to dump something like this again in the near future. Also, more books than I have any actual time to read, but what else is new.

Last saturday the Japanese TeX User Meeting took place in Fujisawa, Kanagawa. For those who have been at the TUG 2013 in Tokyo you will remember that the Japanese TeX community is quite big and vibrant. On Saturday about 50 users and developers gathered for a set of talks on a variety of topics. The first talk was by Keiichiro Shikano ( ) on using Markup text to generate (La)TeX and HTML. He presented a variety of markup formats, including his own tool xml2tex. The second talk was my Masamichi Hosoda ( ) on reducing the size of PDF files using PDFmark extraction. As a contributor to many projects including Texinfo and LilyPond, Masamichi Hosoda tells us horror stories about multiple font embedding in the manual of LilyPond, the permanent need for adaption to newer Ghostscript versions, and the very recent development in Ghostscript prohibiting the merge of font definitions in PDF files. Next up was Yusuke Terada ( ) on grading exams using TeX. Working through hundreds and hundreds of exams and do the grading is something many of us are used to and I think nobody really enjoys it. Yusuke Terada has combined various tools, including scans, pdf merging using pdfpages, to generate gradable PDF which were then checked on an iPad. On the way he did hit some limits in dvipdfmx on the number of images, but this was obviously only a small bump on the road. Now if that could be automatized as a nice application, it would be a big hit I guess! The forth talk was by Satoshi Yamashita ( ) on the preparation of slides using KETpic. KETpic is a long running project by Setsuo Takato ( ) for the generation of graphics, in particular using Cinderella. KETpic and KETcindy integrates with lots of algebraic and statistical programs (R, Maxima, SciLab, ) and has a long history of development. Currently there are activities to incorporate it into TeX Live. The fifth talk was by Takuto Asakura ( ) on programming TeX using expl3, the main building block of the LaTeX3 project and already adopted by many TeX developers. Takuto Asakura came to fame on this years TUG/BachoTeX 2017 when he won the W. J. Martin Prize for his presentation Implementing bioinformatics algorithms in TeX. I think we can expect great new developments from Takuto! The last talk was by myself on fmtutil and updmap, two of the main management programs in any TeX installation, presenting the changes introduced over the last year, including the most recent release of TeX Live. Details have been posted on my blog, and a lengthy article in TUGboat 38:2, 2017 is available on this topic, too. After the conference about half of the participants joined a social dinner in a nearby Izakaya, followed by a after-dinner beer tasting at a local craft beer place. Thanks to Tatsuyoshi Hamada for the organization. As usual, the Japanese TeX User Meetings are a great opportunity to discuss new features and make new friends. I am always grateful to be part of this very nice community! I am looking forward to the next year s meeting.

An earlier article showed that private key storage is an important problem to solve in any cryptographic system and established keycards as a good way to store private key material offline. But which keycard should we use? This article examines the form factor, openness, and performance of four keycards to try to help readers choose the one that will fit their needs. I have personally been using a YubiKey NEO, since a 2015 announcement on GitHub promoting two-factor authentication. I was also able to hook up my SSH authentication key into the YubiKey's 2048 bit RSA slot. It seemed natural to move the other subkeys onto the keycard, provided that performance was sufficient. The mail client that I use, (Notmuch), blocks when decrypting messages, which could be a serious problems on large email threads from encrypted mailing lists. So I built a test harness and got access to some more keycards: I bought a FST-01 from its creator, Yutaka Niibe, at the last DebConf and Nitrokey donated a Nitrokey Pro. I also bought a YubiKey 4 when I got the NEO. There are of course other keycards out there, but those are the ones I could get my hands on. You'll notice none of those keycards have a physical keypad to enter passwords, so they are all vulnerable to keyloggers that could extract the key's PIN. Keep in mind, however, that even with the PIN, an attacker could only ask the keycard to decrypt or sign material but not extract the key that is protected by the card's firmware.

Form factor The four keycards have similar form factors: they all connect to a standard USB port, although both YubiKey keycards have a capacitive button by which the user triggers two-factor authentication and the YubiKey 4 can also require a button press to confirm private key use. The YubiKeys feel sturdier than the other two. The NEO has withstood two years of punishment in my pockets along with the rest of my "real" keyring and there is only minimal wear on the keycard in the picture. It's also thinner so it fits well on the keyring. The FST-01 stands out from the other two with its minimal design. Out of the box, the FST-01 comes without a case, so the circuitry is exposed. This is deliberate: one of its goals is to be as transparent as possible, both in terms of software and hardware design and you definitely get that feeling at the physical level. Unfortunately, that does mean it feels more brittle than other models: I wouldn't carry it in my pocket all the time, although there is a case that may protect the key a little better, but it does not provide an easy way to hook it into a keyring. In the group picture above, the FST-01 is the pink plastic thing, which is a rubbery casing I received along with the device when I got it. Notice how the USB connectors of the YubiKeys differ from the other two: while the FST-01 and the Nitrokey have standard USB connectors, the YubiKey has only a "half-connector", which is what makes it thinner than the other two. The "Nano" form factor takes this even further and almost disappears in the USB port. Unfortunately, this arrangement means the YubiKey NEO often comes loose and falls out of the USB port, especially when connected to a laptop. On my workstation, however, it usually stays put even with my whole keyring hanging off of it. I suspect this adds more strain to the host's USB port but that's a tradeoff I've lived with without any noticeable wear so far. Finally, the NEO has this peculiar feature of supporting NFC for certain operations, as LWN previously covered, but I haven't used that feature yet. The Nitrokey Pro looks like a normal USB key, in contrast with the other two devices. It does feel a little brittle when compared with the YubiKey, although only time will tell how much of a beating it can take. It has a small ring in the case so it is possible to carry it directly on your keyring, but I would be worried the cap would come off eventually. Nitrokey devices are also two times thicker than the Yubico models which makes them less convenient to carry around on keyrings.

Open and closed designs The FST-01 is as open as hardware comes, down to the PCB design available as KiCad files in this Git repository. The software running on the card is the Gnuk firmware that implements the OpenPGP card protocol, but you can also get it with firmware implementing a true random number generator (TRNG) called NeuG (pronounced "noisy"); the device is programmable through a standard Serial Wire Debug (SWD) port. The Nitrokey Start model also runs the Gnuk firmware. However, the Nitrokey website announces only ECC and RSA 2048-bit support for the Start, while the FST-01 also supports RSA-4096. Nitrokey's founder Jan Suhr, in a private email, explained that this is because "Gnuk doesn't support RSA-3072 or larger at a reasonable speed". Its devices (the Pro, Start, and HSM models) use a similar chip to the FST-01: the STM32F103 microcontroller. Nitrokey also publishes its hardware designs, on GitHub, which shows the Pro is basically a fork of the FST-01, according to the ChangeLog. I opened the case to confirm it was using the STM MCU, something I should warn you against; I broke one of the pins holding it together when opening it so now it's even more fragile. But at least, I was able to confirm it was built using the STM32F103TBU6 MCU, like the FST-01. But this is where the comparison ends: on the back side, we find a SIM card reader that holds the OpenPGP card that, in turn, holds the private key material and does the cryptographic operations. So, in effect, the Nitrokey Pro is really a evolution of the original OpenPGP card readers. Nitrokey confirmed the OpenPGP card featured in the Pro is the same as the one shipped by the Free Software Foundation Europe (FSFE): the BasicCard built by ZeitControl. Those cards, however, are covered by NDAs and the firmware is only partially open source. This makes the Nitrokey Pro less open than the FST-01, but that's an inevitable tradeoff when choosing a design based on the OpenPGP cards, which Suhr described to me as "pretty proprietary". There are other keycards out there, however, for example the SLJ52GDL150-150k smartcard suggested by Debian developer Yves-Alexis Perez, which he prefers as it is certified by French and German authorities. In that blog post, he also said he was experimenting with the GPL-licensed OpenPGP applet implemented by the French ANSSI. But the YubiKey devices are even further away in the closed-design direction. Both the hardware designs and firmware are proprietary. The YubiKey NEO, for example, cannot be upgraded at all, even though it is based on an open firmware. According to Yubico's FAQ, this is due to "best security practices": "There is a 'no upgrade' policy for our devices since nothing, including malware, can write to the firmware." I find this decision questionable in a context where security updates are often more important than trying to design a bulletproof design, which may simply be impossible. And the YubiKey NEO did suffer from critical security issue that allowed attackers to bypass the PIN protection on the card, which raises the question of the actual protection of the private key material on those cards. According to Niibe, "some OpenPGP cards store the private key unencrypted. It is a common attitude for many smartcard implementations", which was confirmed by Suhr: "the private key is protected by hardware mechanisms which prevent its extraction and misuse". He is referring to the use of tamper resistance. After that security issue, there was no other option for YubiKey NEO users than to get a new keycard (for free, thankfully) from Yubico, which also meant discarding the private key material on the key. For OpenPGP keys, this may mean having to bootstrap the web of trust from scratch if the keycard was responsible for the main certification key. But at least the NEO is running free software based on the OpenPGP card applet and the source is still available on GitHub. The YubiKey 4, on the other hand, is now closed source, which was controversial when the new model was announced last year. It led the main Linux Foundation system administrator, Konstantin Ryabitsev, to withdraw his endorsement of Yubico products. In response, Yubico argued that this approach was essential to the security of its devices, which are now based on "a secure chip, which has built-in countermeasures to mitigate a long list of attacks". In particular, it claims that:

A commercial-grade AVR or ARM controller is unfit to be used in a security product. In most cases, these controllers are easy to attack, from breaking in via a debug/JTAG/TAP port to probing memory contents. Various forms of fault injection and side-channel analysis are possible, sometimes allowing for a complete key recovery in a shockingly short period of time.

While I understand those concerns, they eventually come down to the trust you have in an organization. Not only do we have to trust Yubico, but also hardware manufacturers and designs they have chosen. Every step in the hidden supply chain is then trusted to make correct technical decisions and not introduce any backdoors. History, unfortunately, is not on Yubico's side: Snowden revealed the example of RSA security accepting what renowned cryptographer Bruce Schneier described as a "bribe" from the NSA to weaken its ECC implementation, by using the presumably backdoored Dual_EC_DRBG algorithm. What makes Yubico or its suppliers so different from RSA Security? Remember that RSA Security used to be an adamant opponent to the degradation of encryption standards, campaigning against the Clipper chip in the first crypto wars. Even if we trust the Yubico supply chain, how can we trust a closed design using what basically amounts to security through obscurity? Publicly auditable designs are an important tradition in cryptography, and that principle shouldn't stop when software is frozen into silicon. In fact, a critical vulnerability called ROCA disclosed recently affects closed "smartcards" like the YubiKey 4 and allows full private key recovery from the public key if the key was generated on a vulnerable keycard. When speaking with Ars Technica, the researchers outlined the importance of open designs and questioned the reliability of certification:

Our work highlights the dangers of keeping the design secret and the implementation closed-source, even if both are thoroughly analyzed and certified by experts. The lack of public information causes a delay in the discovery of flaws (and hinders the process of checking for them), thereby increasing the number of already deployed and affected devices at the time of detection.

This issue with open hardware designs seems to be recurring topic of conversation on the Gnuk mailing list. For example, there was a discussion in September 2017 regarding possible hardware vulnerabilities in the STM MCU that would allow extraction of encrypted key material from the key. Niibe referred to a talk presented at the WOOT 17 workshop, where Johannes Obermaier and Stefan Tatschner, from the Fraunhofer Institute, demonstrated attacks against the STMF0 family MCUs. It is still unclear if those attacks also apply to the older STMF1 design used in the FST-01, however. Furthermore, extracted private key material is still protected by user passphrase, but the Gnuk uses a weak key derivation function, so brute-forcing attacks may be possible. Fortunately, there is work in progress to make GnuPG hash the passphrase before sending it to the keycard, which should make such attacks harder if not completely pointless. When asked about the Yubico claims in a private email, Niibe did recognize that "it is true that there are more weak points in general purpose implementations than special implementations". During the last DebConf in Montreal, Niibe explained:

If you don't trust me, you should not buy from me. Source code availability is only a single factor: someone can maliciously replace the firmware to enable advanced attacks.

Niibe recommends to "build the firmware yourself", also saying the design of the FST-01 uses normal hardware that "everyone can replicate". Those advantages are hard to deny for a cryptographic system: using more generic components makes it harder for hostile parties to mount targeted attacks. A counter-argument here is that it can be difficult for a regular user to audit such designs, let alone physically build the device from scratch but, in a mailing list discussion, Debian developer Ian Jackson explained that:

You don't need to be able to validate it personally. The thing spooks most hate is discovery. Backdooring supposedly-free hardware is harder (more costly) because it comes with greater risk of discovery. To put it concretely: if they backdoor all of them, someone (not necessarily you) might notice. (Backdooring only yours involves messing with the shipping arrangements and so on, and supposes that you specifically are of interest.)

Since that, as far as we know, the STM microcontrollers are not backdoored, I would tend to favor those devices instead of proprietary ones, as such a backdoor would be more easily detectable than in a closed design. Even though physical attacks may be possible against those microcontrollers, in the end, if an attacker has physical access to a keycard, I consider the key compromised, even if it has the best chip on the market. In our email exchange, Niibe argued that "when a token is lost, it is better to revoke keys, even if the token is considered secure enough". So like any other device, physical compromise of tokens may mean compromise of the key and should trigger key-revocation procedures.

Algorithms and performance To establish reliable performance results, I wrote a benchmark program naively called crypto-bench that could produce comparable results between the different keys. The program takes each algorithm/keycard combination and runs 1000 decryptions of a 16-byte file (one AES-128 block) using GnuPG, after priming it to get the password cached. I assume the overhead of GnuPG calls to be negligible, as it should be the same across all tokens, so comparisons are possible. AES encryption is constant across all tests as it is always performed on the host and fast enough to be irrelevant in the tests. I used the following:

• Intel(R) Core(TM) i3-6100U CPU @ 2.30GHz running Debian 9 ("stretch"/stable amd64), using GnuPG 2.1.18-6 (from the stable Debian package)
• Nitrokey Pro 0.8 (latest firmware)
• FST-01, running Gnuk version 1.2.5 (latest firmware)
• YubiKey NEO OpenPGP applet 1.0.10 (not upgradable)
• YubiKey 4 4.2.6 (not upgradable)

I ran crypto-bench for each keycard, which resulted in the following:

Algorithm	Device	Mean time (s)
ECDH-Curve25519	CPU	0.036
	FST-01	0.135
RSA-2048	CPU	0.016
	YubiKey-4	0.162
	Nitrokey-Pro	0.610
	YubiKey-NEO	0.736
	FST-01	1.265
RSA-4096	CPU	0.043
	YubiKey-4	0.875
	Nitrokey-Pro	3.150
	FST-01	8.218

There we see the performance of the four keycards I tested, compared with the same operations done without a keycard: the "CPU" device. That provides the baseline time of GnuPG decrypting the file. The first obvious observation is that using a keycard is slower: in the best scenario (FST-01 + ECC) we see a four-fold slowdown, but in the worst case (also FST-01, but RSA-4096), we see a catastrophic 200-fold slowdown. When I presented the results on the Gnuk mailing list, GnuPG developer Werner Koch confirmed those "numbers are as expected":

With a crypto chip RSA is much faster. By design the Gnuk can't be as fast - it is just a simple MCU. However, using Curve25519 Gnuk is really fast.

And yes, the FST-01 is really fast at doing ECC, but it's also the only keycard that handles ECC in my tests; the Nitrokey Start and Nitrokey HSM should support it as well, but I haven't been able to test those devices. Also note that the YubiKey NEO doesn't support RSA-4096 at all, so we can only compare RSA-2048 across keycards. We should note, however, that ECC is slower than RSA on the CPU, which suggests the Gnuk ECC implementation used by the FST-01 is exceptionally fast. In discussions about improving the performance of the FST-01, Niibe estimated the user tolerance threshold to be "2 seconds decryption time". In a new design using the STM32L432 microcontroller, Aurelien Jarno was able to bring the numbers for RSA-2048 decryption from 1.27s down to 0.65s, and for RSA-4096, from 8.22s down to 3.87s seconds. RSA-4096 is still beyond the two-second threshold, but at least it brings the FST-01 close to the YubiKey NEO and Nitrokey Pro performance levels. We should also underline the superior performance of the YubiKey 4: whatever that thing is doing, it's doing it faster than anyone else. It does RSA-4096 faster than the FST-01 does RSA-2048, and almost as fast as the Nitrokey Pro does RSA-2048. We should also note that the Nitrokey Pro also fails to cross the two-second threshold for RSA-4096 decryption. For me, the FST-01's stellar performance with ECC outshines the other devices. Maybe it says more about the efficiency of the algorithm than the FST-01 or Gnuk's design, but it's definitely an interesting avenue for people who want to deploy those modern algorithms. So, in terms of performance, it is clear that both the YubiKey 4 and the FST-01 take the prize in their own areas (RSA and ECC, respectively).

Conclusion In the above presentation, I have evaluated four cryptographic keycards for use with various OpenPGP operations. What the results show is that the only efficient way of storing a 4096-bit encryption key on a keycard would be to use the YubiKey 4. Unfortunately, I do not feel we should put our trust in such closed designs so I would argue you should either stick with 2048-bit encryption subkeys or keep the keys on disk. Considering that losing such a key would be catastrophic, this might be a good approach anyway. You should also consider switching to ECC encryption: even though it may not be supported everywhere, GnuPG supports having multiple encryption subkeys on a keyring: if one algorithm is unsupported (e.g. GnuPG 1.4 doesn't support ECC), it will fall back to a supported algorithm (e.g. RSA). Do not forget your previously encrypted material doesn't magically re-encrypt itself using your new encryption subkey, however. For authentication and signing keys, speed is not such an issue, so I would warmly recommend either the Nitrokey Pro or Start, or the FST-01, depending on whether you want to start experimenting with ECC algorithms. Availability also seems to be an issue for the FST-01. While you can generally get the device when you meet Niibe in person for a few bucks (I bought mine for around \$30 Canadian), the Seeed online shop says the device is out of stock at the time of this writing, even though Jonathan McDowell said that may be inaccurate in a debian-project discussion. Nevertheless, this issue may make the Nitrokey devices more attractive. When deciding on using the Pro or Start, Suhr offered the following advice:

In practice smart card security has been proven to work well (at least if you use a decent smart card). Therefore the Nitrokey Pro should be used for high security cases. If you don't trust the smart card or if Nitrokey Start is just sufficient for you, you can choose that one. This is why we offer both models.

So far, I have created a signing subkey and moved that and my authentication key to the YubiKey NEO, because it's a device I physically trust to keep itself together in my pockets and I was already using it. It has served me well so far, especially with its extra features like U2F and HOTP support, which I use frequently. Those features are also available on the Nitrokey Pro, so that may be an alternative if I lose the YubiKey. I will probably move my main certification key to the FST-01 and a LUKS-encrypted USB disk, to keep that certification key offline but backed up on two different devices. As for the encryption key, I'll wait for keycard performance to improve, or simply switch my whole keyring to ECC and use the FST-01 or Nitrokey Start for that purpose.

---

[The author would like to thank Nitrokey for providing hardware for testing.] This article first appeared in the Linux Weekly News.

I said that I was going to start writing these regularly, so I'm going to stick to it, even when the results are rather underwhelming. One of the goals is to make the time for more free software work, and I do better at doing things that I record. The only piece of free software work for September was that I made rra-c-util compile cleanly with the Clang static analyzer. This was fairly tedious work that mostly involved unconfusing the compiler or converting (semi-intentional) crashes into explicit asserts, but it unblocks using the Clang static analyzer as part of the automated test suite of my other projects that are downstream of rra-c-util. One of the semantic changes I made was that the vector utilities in rra-c-util (which maintain a resizable array of strings) now always allocate room for at least one string pointer. This wastes a small amount of memory for empty vectors that are never used, but ensures that the strings struct member is always valid. This isn't, strictly speaking, a correctness fix, since all the checks were correct, but after some thought, I decided that humans might have the same problem that the static analyzer had. It's a lot easier to reason about a field that's never NULL. Similarly, the replacement function for a missing reallocarray now does an allocation of size 1 if given a size of 0, just to avoid edge case behavior. (I'm sure the behavior of a realloc with size 0 is defined somewhere in the C standard, but if I have to look it up, I'd rather not make a human reason about it.) I started on, but didn't finish, making rra-c-util compile without Clang warnings (at least for a chosen set of warnings). By far the hardest problem here are the Clang warnings for comparisons between unsigned and signed integers. In theory, I like this warning, since it's the cause of a lot of very obscure bugs. In practice, gah does C ever do this all over the place, and it's incredibly painful to avoid. (One of the biggest offenders is write, which returns a ssize_t that you almost always want to compare against a size_t.) I did a bunch of mechanical work, but I now have a lot of bits of code like:

     if (status < 0)
         return;
    written = (size_t) status;
    if (written < avail)
        buffer->left += written;

which is ugly and unsatisfying. And I also have a ton of casts, such as with:

    buffer_resize(buffer, (size_t) st.st_size + used);

since st.st_size is an off_t, which may be signed. This is all deeply unsatisfying and ugly, and I think it makes the code moderately harder to read, but I do think the warning will potentially catch bugs and even security issues. I'm still torn. Maybe I can find some nice macros or programming styles to avoid the worst of this problem. It definitely requires more thought, rather than just committing this huge mechanical change with lots of ugly code. Mostly, this kind of nonsense makes me want to stop working on C code and go finish learning Rust.... Anyway, apart from work, the biggest thing I managed to do last month that was vaguely related to free software was upgrading my personal servers to stretch (finally). That mostly went okay; only a few things made it unnecessarily exciting. The first was that one of my systems had a very tiny / partition that was too small to hold the downloaded debs for the upgrade, so I had to resize it (VM disk, partition, and file system), and that was a bit exciting because it has an old-style DOS partition table that isn't aligned (hmmm, which is probably why disk I/O is so slow on those VMs), so I had to use the obsolete fdisk -c=dos mode because I wasn't up for replacing the partition right then. The second was that my first try at an upgrade died with a segfault during the libc6 postinst and then every executable segfaulted. A mild panic and a rescue disk later (and thirty minutes and a lot of swearing), I tracked the problem down to libc6-xen. Nothing in the dependency structure between jessie and stretch forces libc6-xen to be upgraded in lockstep or removed, but it's earlier in the search path. So ld.so gets upgraded, and then finds the old libc6 from the libc6-xen package, and the mismatch causes immediate segfaults. A chroot dpkg --purge from the rescue disk solved the problem as soon as I knew what was going on, but that was a stressful half-hour. The third problem was something I should have known was going to be an issue: an old Perl program that does some internal stuff for one of the services I ran had a defined @array test that has been warning for eons and that I never fixed. That became a full syntax error with the most recent Perl, and then I fixed it incorrectly the first time and had a bunch of trouble tracking down what I'd broken. All sorted out now, and everything is happily running stretch. (ejabberd, which other folks had mentioned was a problem, went completely smoothly, although I suspect I now have too many of the plugin packages installed and should do a purging.)

FOSDEM is a free software event that offers open source communities a place to meet, share ideas and collaborate. It is renown for being highly developer-oriented and brings together 8000+ participants from all over the world. It is held in the city of Brussels (Belgium). FOSDEM 2018 will take place during the weekend of February 3rd-4th 2018. More details about the event can be found at http://www.fosdem.org Call for Participation The Retrocomputing DevRoom is a first-timer at FOSDEM, with talks about use of older computing hardware and software in modern times. Presentation topics could include but are not limited to:

• Emulation of old systems to run videogames, legacy software, etc
• Open source software emulation/simulation
• Open hardware
• Operating systems/executives for retrocomputers/retrosystems
• Uses of retrocomputing today
• Other retrosystems topics
• Opportunities in retrocomputing
• Round tables

You are not limited to slide presentations, of course. Be creative. However, FOSDEM is an open source conference, therefore we ask you to stay clear of marketing presentations. We are not afraid of technical stuff: devrooms are a place for development teams to meet, discuss, hack and publicly present their project s latest improvements and future directions. If you will have special needs for your talk (e. g. because you will need to plug some sort of a system), please note that clearly in your proposal so that we can provide it. You can use the Wikipedia definition of retrocomputing as a reference definition to see if you talk qualifies, although it is not exclusive: https://en.wikipedia.org/wiki/Retrocomputing Important dates

• 25 Nov 2017: submission deadline for talk proposals
• 15 Dec 2017: announcement of the final schedule
• 4 Feb 2018: Retrocomputing dev room

Useful information Use the FOSDEM Pentabarf tool to submit your proposal: https://penta.fosdem.org/submission/FOSDEM18 If necessary, create a Pentabarf account and activate it. Please reuse your account from previous years if you have already created it. Details on submission:

• In the Person section, provide First name, Last name (in the General tab), Email (in the Contact tab) and Bio ( Abstract field in the Description tab).
• Submit a proposal by clicking on Create event .
• Important! Select the Retrocomputing DevRoom track (on the General tab). If you do not select a track, then nobody, from any track, will look at your submission!
• Provide the title of your talk ( Event title in the General tab).
• Provide a description of the subject of the talk and the intended audience (in the Abstract field of the Description tab)
• Provide a rough outline of the talk or goals of the session (a short list of bullet points covering topics that will be discussed) in the Full description field in the Description tab
• Provide an expected length of your talk in the Duration field, including discussion. The default duration is 30 minutes.

Please note neither FOSDEM nor the Retrocomputing DevRoom will reimburse any expenses you incur Recording of talks The FOSDEM organizers plan to have live streaming and recording fully working, both for remote/later viewing of talks, and so that people can watch streams in the hallways when rooms are full. This requires speakers to consent to being recorded and streamed. If you plan to be a speaker, please understand that by doing so you implicitly give consent for your talk to be recorded and streamed. The recordings will be published under the same license as all FOSDEM content (CC-BY). Contact The Retrocomputing DevRoom is managed by Pau Garcia Quiles (retrocomputing-devroom-manager@fosdem.org). A mailing list of speakers, audience and the curious is available, please subscribe at https://lists.fosdem.org/listinfo/retrocomputing-devroom Hope to hear from you soon! And please forward this announcement.

Here s my weekly report for week 41 of 2017. In this week I have explored some Java 8 features, looked at automatic updates in a few Linux distributions and decided that actually I don t need swap anymore.

Debian The issue that was preventing the migration of the Tasktools Packaging Team s mailing list from Alioth to Savannah has now been resolved. Ana s chkservice package that I sponsored last week has been ACCEPTED into unstable and since MIGRATED to testing.

Tor Project I have produced a patch for the Tor Project website to update links to the Onionoo documentation now this has moved (#23802 ). I ve updated the Debian and Ubuntu relay configuration instructions to use systemctl instead of service where appropriate (#23048 ). When a Tor relay is less than 2 years old, an alert will now appear on Atlas to link to the new relay lifecycle blog post (#23767 ). This should hopefully help new relay operators understand why their relay is not immediately fully loaded but instead it takes some time to ramp up. I have gone through the tickets for Tor Cloud and did not find any tickets that contain any important information that would be useful to someone reviving the project. I have closed out these tickets and the Tor Cloud component no longer has any non-closed tickets (#7763, #8544, #8768, #9064, #9751, #10282, #10637, #11153, #11502, #13391, #14035, #14036, #14073, #15821 ). I ve continued to work on turning the Atlas application into an integrated part of Tor Metrics (#23518 ) and you can see some progress here. Finally, I ve continued hacking on a Twitter bot to tweet factoids about the public Tor network and you can now enjoy some JavaDoc documentation if you d like to learn a little about its internals. I am still waiting for a git repository to be created (#23799 ) but will be publishing the sources shortly after that ticket is actioned.

• Bugs opened: #23799, #23802, #23809, #23830, #23831
• Bugs closed (fixed/wontfix): #7763, #8544, #8768, #9064, #9751, #10282, #10637, #11153, #11502, #13391, #14035, #14036, #14073, #15821, #23048, #23767

Sustainability I believe it is important to be clear not only about the work I have already completed but also about the sustainability of this work into the future. I plan to include a short report on the current sustainability of my work in each weekly report. I have not had any free software related expenses this week. The current funds I have available for equipment, travel and other free software expenses remains 60.52. I do not believe that any hardware I rely on is looking at imminent failure. I d like to thank Digital Ocean for providing me with futher credit for their platform to support my open source work. I do not find it likely that I ll be travelling to Cambridge for the miniDebConf as the train alone would be around 350 and hotel accomodation a further 600 (to include both me and Ana).

With the development of TLCockpit continuing, I found the need for and easy exchange format between the TeX Live Manager tlmgr and frontend programs like TLCockpit. Thus, I have implemented JSON output for the tlmgr info command. While the format is not 100% stable I might change some thing I consider it pretty settled. The output of tlmgr info --data json is a JSON array with JSON objects for each package requested (default is to list all).

[ TLPackageObj, TLPackageObj, ... ]

The structure of the JSON object TLPackageObj reflects the internal Perl hash. Guaranteed to be present keys are name (String) and avilable (Boolean). In case the package is available, there are the following further keys sorted by their type:

• String type: name, shortdesc, longdesc, category, catalogue, containerchecksum, srccontainerchecksum, doccontainerchecksum
• Number type: revision, runsize, docsize, srcsize, containersize, srccontainersize, doccontainersize
• Boolean type: available, installed, relocated
• Array type: runfiles (Strings), docfiles (Strings), srcfiles (Strings), executes (Strings), depends (Strings), postactions (Strings)
• Object type:
  • binfiles: keys are architecture names, values are arrays of strings (list of binfiles)
  • binsize: keys are architecture names, values or numbers
  • docfiledata: keys are docfile names, values are objects with optional keys details and lang
  • cataloguedata: optional keys aare topics, version, license, ctan, date, values are all strings

A rather long example showing the output for the package latex, formatted with json_pp and having the list of files and the long description shortened:

[
    
      "installed" : true,
      "doccontainerchecksum" : "5bdfea6b85c431a0af2abc8f8df160b297ad73f6a324ca88df990f01f24611c9ae80d2f6d12c7b3767308fbe3de3fca3d11664b923ea4080fb13fd056a1d0c3d",
      "docfiles" : [
         "texmf-dist/doc/latex/base/README.txt",
         ....
         "texmf-dist/doc/latex/base/webcomp.pdf"
      ],
      "containersize" : 163892,
      "depends" : [
         "luatex",
         "pdftex",
         "latexconfig",
         "latex-fonts"
      ],
      "runsize" : 414,
      "relocated" : false,
      "doccontainersize" : 12812184,
      "srcsize" : 752,
      "revision" : 43813,
      "srcfiles" : [
         "texmf-dist/source/latex/base/alltt.dtx",
         ....
         "texmf-dist/source/latex/base/utf8ienc.dtx"
      ],
      "category" : "Package",
      "cataloguedata" :  
         "version" : "2017/01/01 PL1",
         "topics" : "format",
         "license" : "lppl1.3",
         "date" : "2017-01-25 23:33:57 +0100"
       ,
      "srccontainerchecksum" : "1d145b567cf48d6ee71582a1f329fe5cf002d6259269a71d2e4a69e6e6bd65abeb92461d31d7137f3803503534282bc0c5546e5d2d1aa2604e896e607c53b041",
      "postactions" : [],
      "binsize" :  ,
      "longdesc" : "LaTeX is a widely-used macro package for TeX, [...]",
      "srccontainersize" : 516036,
      "containerchecksum" : "af0ac85f89b7620eb7699c8bca6348f8913352c473af1056b7a90f28567d3f3e21d60be1f44e056107766b1dce8d87d367e7f8a82f777d565a2d4597feb24558",
      "executes" : [],
      "binfiles" :  ,
      "name" : "latex",
      "catalogue" : null,
      "docsize" : 3799,
      "available" : true,
      "runfiles" : [
         "texmf-dist/makeindex/latex/gglo.ist",
         ...
         "texmf-dist/tex/latex/base/x2enc.dfu"
      ],
      "shortdesc" : "A TeX macro package that defines LaTeX"
    
]

What is currently not available via tlmgr info and thus also not via the JSON output is access to virtual TeX Live databases with several member databases (multiple repositories). I am thinking about how to incorporate this information. These changes are currently available in the tlcritical repository, but will enter proper TeX Live repositories soon. Using this JSON output I will rewrite the current TLCockpit tlmgr interface to display more complete information.

I had a bit hard time to get ScalaFX to display a list of items in a scrollable space, and each item can be clicked. I use this in TLCockpit to display the list of documentation files in a TeX Live package, and open it directly from the application. Unfortunately there is not a huge amount of examples using ScalaFX out there in the web, so it took me a bit. My first try was using a VBox with various Labels in there, but this is not scrollable. In other areas I have used TreeTableView, so in this case using ListView should be fine. What I finally came up is the following code:

import scalafx.application.JFXApp
import scalafx.application.JFXApp.PrimaryStage
import scalafx.collections.ObservableBuffer
import scalafx.geometry.Orientation
import scalafx.scene.control. ListCell, ListView 
import scalafx.scene.input.MouseEvent
import scalafx.scene. Cursor, Scene 
import scalafx.scene.paint.Color
import scalafx.Includes._
object ApplicationMain extends JFXApp  
  val SomeStrings: Seq[String] = Seq("Hello", "World", "Enjoy")
  stage = new PrimaryStage  
    title = "ListViewExample"
    scene = new Scene  
      root =  
        new ListView[String]  
          orientation = Orientation.Vertical
          cellFactory =  
            p =>  
              val cell = new ListCell[String]
              cell.textFill = Color.Blue
              cell.cursor = Cursor.Hand
              cell.item.onChange   (_, _, str) => cell.text = str  
              cell.onMouseClicked =   me: MouseEvent => println("Do something with " + cell.text.value)  
              cell
             
           
          items = ObservableBuffer(SomeStrings)
         
       
     
   
 

Some comments to the code, at least as far I understand it:

• line 9: Importing scalafx.Includes._ seems to simplify some things, in particular the event handler routines can be written more straight forward.
• line 20: Many more properties can be set here, for example the preferred height and max height, both of which I am using.
• line 23: In my case a ListCell was enough for my needs (changing color, cursor, and allowing for mouse clicks), but if one needs something more complicated here it is bst to create an arbitrary object and asign it to the graphic field.
• line 26: The essential part to actually fill the cells is the routine cell.item.onChange, which takes three arguments of which the last is the new value. It is used to update the cell text.
• line 31: Last but not least one needs to assign some Observable to the items, in this case I use ObservableBuffer around the lost of strings.

Once managed, it doesn t look so complicated, but took me some time.