This is a copy of an email I posted to the nikita-noark mailing list. Please follow up there if you would like to discuss this topic. The background is that we are making a free software archive system based on the Norwegian Noark 5 standard for government archives. I've been wondering a bit lately how trusted timestamps could be stored in Noark 5. Trusted timestamps can be used to verify that some information (document/file/checksum/metadata) have not been changed since a specific time in the past. This is useful to verify the integrity of the documents in the archive. Then it occured to me, perhaps the trusted timestamps could be stored as dokument variants (ie dokumentobjekt referered to from dokumentbeskrivelse) with the filename set to the hash it is stamping? Given a "dokumentbeskrivelse" with an associated "dokumentobjekt", a new dokumentobjekt is associated with "dokumentbeskrivelse" with the same attributes as the stamped dokumentobjekt except these attributes:

• format -> "RFC3161"
• mimeType -> "application/timestamp-reply"
• formatDetaljer -> "<source URL for timestamp service>"
• filenavn -> "<sjekksum>.tsr"

This assume a service following IETF RFC 3161 is used, which specifiy the given MIME type for replies and the .tsr file ending for the content of such trusted timestamp. As far as I can tell from the Noark 5 specifications, it is OK to have several variants/renderings of a dokument attached to a given dokumentbeskrivelse objekt. It might be stretching it a bit to make some of these variants represent crypto-signatures useful for verifying the document integrity instead of representing the dokument itself. Using the source of the service in formatDetaljer allow several timestamping services to be used. This is useful to spread the risk of key compromise over several organisations. It would only be a problem to trust the timestamps if all of the organisations are compromised. The following oneliner on Linux can be used to generate the tsr file. $input is the path to the file to checksum, and $sha256 is the SHA-256 checksum of the file (ie the ".tsr" value mentioned above).

openssl ts -query -data "$inputfile" -cert -sha256 -no_nonce \
    curl -s -H "Content-Type: application/timestamp-query" \
      --data-binary "@-" http://zeitstempel.dfn.de > $sha256.tsr

To verify the timestamp, you first need to download the public key of the trusted timestamp service, for example using this command:

wget -O ca-cert.txt \
  https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt

Note, the public key should be stored alongside the timestamps in the archive to make sure it is also available 100 years from now. It is probably a good idea to standardise how and were to store such public keys, to make it easier to find for those trying to verify documents 100 or 1000 years from now. :) The verification itself is a simple openssl command:

openssl ts -verify -data $inputfile -in $sha256.tsr \
  -CAfile ca-cert.txt -text

Is there any reason this approach would not work? Is it somehow against the Noark 5 specification?

During last weeks I was finally able to push out some releases of Gammu and related tools. Those were mostly waiting for quite some time in Git, but still will be useful for many users. Gammu 1.38.3 brings several SMSD performance improvements (I'd like to hear feedback here how much it helped in real world workloads), fixes USSD related crashes and adds support for new devices. python-gammu 2.8 finally brings emoji fixes to Python users as well. Those were fixed in Gammu quite some time ago, but the Python API still didn't handle them properly. There was also simplification in the calls API. Wammu release will follow shortly in next days, the most important change there being license change to GPLv3 or later.

Filed under: Debian English Gammu python-gammu Wammu

Dear Lazyweb, About a week ago, I learnt about the release of an interesting book by the fine people at Creative Commons: Made with Creative Commons. The book itself is, of course, CC BY-SA-licensed. I downloaded it and started reading right away. Some minutes later, I ordered a dead-tree copy, which arrived a couple of days ago. (I'm linking to the publisher's page, but bought it from Amazon M xico... Shipping it from Denmark would not have been as cheap and fast, I guess). Anyway, given my workplace, given the community I know, given I know something like this is much needed... I will start a Spanish translation of the content. There are at least two other people interested in participating, and I haven't yet publicized my intentions (this is the first public statement about it). So, dear Lazyweb: What I need is a good framework for doing this. I started by creating a Git repository, and we were discussing to translate to Markdown (to later format according to the desired output) but then I thought... If we want the translation to be updateable, and to be able to properly accept other people's work, maybe a better format is warranted? So, my current idea is to create a Markdown version for the English original, and find a way to shoehorn^Wseparate it by paragraphs and feed it to Gettext, which is the best translation framework I have used (but is meant for code translation, not for full-text)... Dear lazyweb: What tools do you recommend me to use? Quite important to me: Are they Free tools? Are they easy to use by third-parties, maybe incorporating work via Git? Or, at least, via a Web front-end that allows me as a project lead to review and approve/fix/reject strings? Thanks, lazyweb!

Here's what happened in the Reproducible Builds effort between Sunday May 28 and Saturday June 3 2017: Past an upcoming events

• On June 9th, Chris Lamb will present at the Hong Kong Open Source Conference 2017 on reproducible builds.
• We restarted our IRC meetings, now with a monthly schedule where we meet on the first Thursday of the month at a varying time. Our next meeting will be on the first 6th of July at 17:00 UTC. At the June meeting we had the following agenda, and if you are interested there is an automated summary and full logs too.
  1. Introductions
  2. Feedback for the reproducible.json spec format - is that suitable for Guix and F-Droid as well? LEDE/OpenWrt? Coreboot?
  3. writing parser + user interface for reproducible.json
  4. tests.r-b.o/Debian once Stretch has been released
  5. Reproducible Builds Summit 2017
  6. Next meeting
  7. Any other business

Documentation updates

• Holger added a link to the blog post with a report about the recent hackathon to our website.

Toolchain development and fixes

• Chris Lamb wrote a proof-of-concept implementation for #863622 ("apt: warn when installing packages that are not reproducible"). Try it, it's fun!
• Russ worked on making the debian-policy package build reproducibly which is a good opportunity to shamelessly plug that #844431 ("debian-policy: Packages should be reproducible") is still a work in progress.

Patches and bugs filed

• Bernhard M. Wiedemann:
  • pycryptopp
  • rpm
  • rpm
  • bam
  • boost/jam
  • premake
  • ruby/rdoc
  • ruby/rdoc
  • sdcc (merged)

4 package reviews have been added, 6 have been updated and 25 have been removed in this week, adding to our knowledge about identified issues. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (2)
• Chris Lamb (1)

diffoscope development

• Chris Lamb:
  • Add comparator for Fontconfig cache files.
• anthraxx:
  • tools: extend external Arch Linux tools
• Ximin Luo:
  • Add a --exclude-command CLI for filtering out long-running commands like "readelf --debug-dump=info"
  • Use unicode chars for the +/- controls instead of hacky punctuation
  • Don't show +/- controls for differences with no children
  • Fix create_limited_print_func
  • Add a size() method to Difference and check that self._visuals is empty in get_reverse()
  • Add a reader for the JSON format
  • Allow the "source" param to overridden compare() methods to be given as a positional argument

tests.reproducible-builds.org Mattia Rizzolo:

• packages-tests: go back to the shorter way of specifying JUnit jobs.
• update_jdn: call jenkins-jobs instead of jenkins-job-builder, as that's how it is named in 1.6.1 version.
• Upload jenkins-job-builder 1.6.1-1~bpo8+1 to jessie-backports.

Daniel Kahn Gillmor:

• Contribute a few typo fixes.

Vagrant Cascadian:

• Update README to reflect switch to PostgreSQL (some time ago).
• Add three new boards for armhf tests:
  • ff64a-armhf-rb.debian.net: Firefly-rk3399, Rockchip six-core (Cortex-A72 x2 + Cortex-A53 x4), 2GB RAM, USB-sata (future plans for native sata.)
  • jtx1a-armhf-rb.debian.net: Jetson-tx1, quad-core (big.LITTLE Cortex-A53/A57), ~3.5GB RAM, native SATA ~500GB disk
  • odc2a-armhf-rb.debian.net: Odroic-C2, quad-core (Cortex-A53), ~2GB ram, 128GB eMMC
• Ressurect rpi2c-armhf-rb.debian.net (Raspberry PI 2B, broadcom bcm2836 quad-core (cortex-A7), 1GB RAM) from the dead.

Holger Levsen:

• Configure the rc.local service on all build nodes to only start when the network is actually up, as configuring half of them to run in the future requires networking to determine the real current date. See Running Services After the Network is up if you want to learn more about networking with systemd. Somehow this doesnt work yet on the Ubuntu 16.05 arm64 nodes; help welcome, either on #debian-qa or #debian-reproducible.
• Add rpi2c back to the armhf network.
• Also add ff64a, odc2a & jtx1a to the armhf network.
  • Add pbuilder/schroot-setup jobs for the new nodes.
  • Add 10 new armhf builder jobs.
  • Disable all build jobs on odc2a again as haveged segfaults on 4.12~rc2, which is needed for this board & setup.
• Reproducible_cleanup_nodes.sh: adapt for new build service
• Make performance.html show if build jobs are down due to remote node problems and make code to count enabled jobs more robust.
• reproducible_build_service: Make it cope with disabled workers.
• jenkins-master-wrapper: Fail loudly if remote node is not accessable.
• Increase maximum scheduling queue sizes, to adjust for faster scheduler.
• New script, init_node, to initialize new build nodes (derived from refactored existing script).
• Update INSTALL documentation to reflect that we're testing arm64 now too.

Misc. This week's edition was written by Chris Lamb, Bernhard M. Wiedemann and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Review: Star Healer, by James White

Series:	Sector General #6
Publisher:	Orb
Copyright:	1984
Printing:	2002
ISBN:	0-312-87770-6
Format:	Trade paperback
Pages:	206

Star Healer is the sixth book of the Sector General series, and I think it may be the first novel in the series that was written as a novel instead of a fix-up of short stories. That makes it not a bad place to start in the series if one would rather not deal with fix-ups or barely-disguised short story collections. There isn't a huge amount of character development over the course of this series (at least to this point), so the main thing you would lose by starting here is some built-up reason for caring about the main character. This is the third book in the Alien Emergencies omnibus (the book referenced in the publication information here). Most of the previous stories have focused on Conway, a Senior Physician in the sprawling and wonderfully well-equipped multi-species hospital called Sector General and, in recent stories, the head physician in the hospital's ambulance ship. Becoming a Senior Physician at Sector General is quite the accomplishment, and a fine point to reach in the career of any doctor specializing in varied life forms, but there is another tier above: the Diagnosticians, who are the elite of Sector General. The difference is education tapes. Deep knowledge of even one specific type of life is a lot to ask of a doctor, as shown by the increasing specialization of human medicine. Sector General, which deals with wildly varying ailments of thousands of species including entirely unknown ones (if, admittedly, primarily trauma, at least in the stories shown), would be an impossible task. White realizes this and works around it with education tapes that temporarily embed in a doctor's head the experience of a doctor of another species entirely. This provides the native expertise missing, but it comes with the full personality of the doctor who recorded the tape, including preferences for food and romantic attachment that may be highly disorienting. Senior Physicians use a tape at a time, and then have it erased again when they don't need it. Diagnosticians juggle four or more tapes at the same time, and keep them for long periods or even permanently, allowing them to do ground-breaking original research. The opening of Star Healer is an offer from the intimidating Chief Psychologist of Sector General: he has a shot at Diagnostician. But it's a major decision that he should think over first, so the next step is to take a vacation of sorts on a quiet world with a small human scientific station. Oh, and there's a native medical problem, although not one with much urgency. Conway doesn't do a lot of resting, because of course he gets pulled into trying to understand the mystery of an alien species that is solitary to the point of deep and unbreakable social taboos against even standing close to other people. This is a nice cultural puzzle in line with the rest of the series, but it also leaves Conway with a new ally: an alien healer in a society in which being a doctor is difficult to the point of near hopelessness. It's not much of a spoiler to say that of course Conway decides to try for Diagnostician after his "vacation." The rest of the book is him juggling multiple cases with his new and often conflicting modes of thinking, and tackling problems that require a bit less in the way of puzzle-solving and a bit more in the way of hard trade-off decisions and quick surgical action. Senior Physicians may be able to concentrate on just one puzzle at a time; Diagnosticians have to juggle several. And they're larger, more long-term problems, focusing on how to improve a general problem for a whole species rather than just heal a specific injured alien. One interesting aspect of this series, which is very much on display here, is that Sector General most definitely does not have a Prime Directive. They are cautious about making contact with particularly primitive civilizations for fear that spacefarers would give them an inferiority complex, but sometimes they do anyway. And if they run into some biological system that offends their sensibilities, they try to fix it, not just observe it. White frequently shows species caught in what the characters call "biological traps," unable to develop farther because of some biological adaptation that gets in their way, and Sector General tries to fix those. It's an interesting ethical problem that I wish they'd think about a bit more. It's not clear they're wrong, and I think it's correct to take an expansive view of the mission to heal, but there's also a sense in which Sector General is modifying culture and biology to make aliens more like them. (The parallels between this and all the abusive paternalism that human cultures do around disability is a little too close to home to be comfortable, and now I kind of wish it hadn't occurred to me.) The gender roles, sadly, continue to be dire, although mostly ignorable because the one major female character is generally just shown as another doctor with little attention to sex. But apparently women (of every species!) cannot become Diagnosticians because they have an insurmountable biological aversion to sharing their minds with a learning tape from any doctor who doesn't find them physically attractive, which is just... sigh. It's sad that someone who could write an otherwise remarkably open-minded and pacifist series of stories, in sharp contrast with most of SF history, would still have that large of a blind spot. Apart from the times gender comes up, I liked this book more than the rest of the series, in part because I strongly prefer novels to short stories. There's more room to develop the story, and while characterization continues to not be White's strong point and that space mostly goes to more puzzles instead, he does provide an interesting set of interlocking puzzles. The problem posed by the aliens Conway meets on his "vacation" isn't fully resolved here (presumably that's for a future book), but he does solve several other significant problems and develops his own problem-solving style in more depth than in previous stories. Mildly recommended, particularly if you like this series in general. Followed by Code Blue - Emergency. Rating: 7 out of 10

This has been a spring with times to relax, times to be busy, times of anticipation of a new baby, and times of enjoying our family. Rather than write a lot of words about it, I m telling the story in photos. To view, click here, then click Show Info in the upper right to see captions. You can pause it with the button in the lower left, and use arrow keys to advance. Alternatively, there s a captionless slideshow available here. Here s one photo to get you started:

A new version of the anytime package is now on CRAN. It marks the eleventh release since the inaugural version late last summer. anytime is a very focused package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, ... format to either POSIXct or Date objects -- and to do so without requiring a format string. See the anytime page, or the GitHub README.md for a few examples. This release brings a little more consistency to how numeric or integer arguments are handled. Previously, we were overly eager in accepting something such as 20150605 (i.e. today) as a (numerical or integer) input to both anytime() and anydate(). That is well-intentioned, but ultimately foolish. We relied on heuristic cutoffs to determine whether input was "meant to be" a date or time offset. There lies madness. We now differentiate whether we were called via anytime() (in which case numerical data is second offset to the epoch, just as.POSICct()) or anytime() (in which case it is days offset to the (date) epoch, just like as.Date()). The previous behaviour can be restored via a options, both function-local as well as global are supported. And of course, there is no change for all other (and more common) input formats, notably character or factor. A full list of changes follows.

Changes in anytime version 0.3.0 (2017-06-05)

• Numeric input is now always an offset to epoch, with anytime() using seconds, and anydate() using dates. (#65 fixing #63).
• Old behaviour can be re-enabled with an option also supporting a global setting getOption("anytimeOldHeuristic")
• RStudio versions 1.1.129 or later can run all functions without fear of crashing due to a change in their use of Boost.
• Replaced init.c with registration code inside of RcppExports.cpp thanks to Rcpp 0.12.11.

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

For a while now, I've wanted to increase my frequency of blogging. I've got roughly 24 draft posts which I haven't finished writing, stretching back to at least 2009, and several other topics that I haven't even begun to write about but have something in the back of my mind which I'd like to express. I have been self-censoring to an extent for two reasons:

• I'd been publishing my blog posts (or links to them) elsewhere, in particular on Facebook, and I felt discouraged from writing on topics that I thought would not be interesting to many of those who I am connected to on Facebook (school friends, etc.)
• I used twitter, and expressed shorter ideas there instead.

Neither of these are good enough reasons to do so: in the case of Facebook, I make very little use of it, and if people didn't find my posts interesting they can mute me, block me or ignore them. In the case of Twitter, my interest in it as a platform has waned since they have made it increasingly difficult to interact with them as a developer. I've given up trying to archive my own tweets, and so on longer think of it as a good place to express anything in isolation. Going forward I'm going to make an effort to ignore their influence on me.

Hey Sega, Why the Hell is There No Skies of Arcadia Sequel? USgamer

There was still plenty of world left to be explored, if it came to that

I don t understand this remark there really wasn t much left. They d have to make changes in order to squash more in, as I ve found when trying to convert the world of Arcadia to D&D.

Picture one Sunday morning. Three guys are seemingly-randomly walking into a Mennonite church in rural Nebraska. One with long hair and well-maintained clothes from the 70s. Another dressed well enough to be preaching. And the third simply dressed to be comfortable, with short hair showing evidence of having worn a headset for a couple of hours that morning. This was the scene as we made a spur-of-the-moment visit to that church which resulted in quite some surprise all around, since my brother knew a number of people there. For instance:

Pastor: Peter! What are you doing here? Peter: [jokingly] Is that how you greet visitors here?

And then, of course, Peter would say, Well, we were flying home from South Dakota and figured we d stop in at Beatrice for fuel. And drop in on you. Followed by some surprise that we would stop at their little airport (which is quite a nice one). This all happened because it was windy. This is the fun adventure of aviation. Sometimes you plan to go to Texas, but the weather there is terrible, so you discover a 100-year-old landmark in Indiana instead. Or sometimes, like a couple of weeks ago, we planned to fly straight home but spent a few hours exploring rural Nebraska. The three of us flew to Sioux Falls, SD, in a little Cessna to visit my uncle and aunt up there. On our flight up, we stopped at the little airport in Seward, NE. It was complete with this unique elevated deck. In my imagination, this is used for people to drink beer while watching the planes land. In South Dakota, we had a weekend full of card and board games, horseshoes, and Crokinole with my uncle and aunt, who are always fun to visit. We had many memories of visits up there as children and the pleasant enjoyment of the fact that we didn t need an 8-hour drive to get there. We flew back with a huge bag of large rhubarb from their garden (that too is something of a tradition!) It was a fun weekend to spend with my brothers first time we d been able to do this in a long while. And it marked the 11th state I ve flown into, and over 17,000 miles of flying.

The new RcppArmadillo release 0.7.900.2.0 is now on CRAN, and the Debian package was just updated as well. Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language--and is widely used by (currently) 350 other packages on CRAN---an increase of 32 since the last CRAN release of 0.7.800.2.0 in April! With the 7.900.* series of Armadillo, Conrad has started to more fully utilize OpenMP (also see Wikipedia on OpenMP) for operations that can be parallelized. To use this in your package you need to update its src/Makevars ,.win file similarly to what the skeleton default now uses

PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS) 
PKG_LIBS = $(SHLIB_OPENMP_CFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)

and you may want to enable C++11 while you are at it---though this may pose issues with older-than-ancient RHEL installations which are still (way too) pervasive so we do not do it by default (yet). Here, we once again rely on the build infrastructure automagically provided by R itself: if and when OpenMP is available, R will use it via $(SHLIB_OPENMP_CXXFLAGS) etc; see the fine WRE manual for details. That said, some operating systems make this harder than other, and macOS usually takes the crown. See for example this blog post by James for surviving in that environment. I am a little short of details because on Linux these things just work, and have for well over a decade. The rcpp-devel mailing list will be the best place for questions. Changes in this release relative to the previous CRAN release are as follows:

Changes in RcppArmadillo version 0.7.900.2.0 (2017-06-02)

• Upgraded to Armadillo release 7.900.2 (Evil Banana Republic)
  • Expanded clamp() to handle cubes
  • Computationally expensive element-wise functions (such as exp(), log(), cos(), etc) can now be automatically sped up via OpenMP; this requires a C++11/C++14 compiler with OpenMP 3.0+ support for GCC and clang compilers
  • One caveat: when using GCC, use of -march=native in conjunction with -fopenmp may lead to speed regressions on recent processors
• Added gcc 7 to support compiler check (James Balamuta in #128 addressing #126).
• A unit test helper function for rmultinom was corrected (#133).
• OpenMP support was added to the skeleton helper in inline.R

Courtesy of CRANberries, there is a diffstat report. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

tl;dr: Get vmdebootstrap replacement from http://git.liw.fi/vmdb2 and run it from the source tree. Tell me if something doesn't work. Send patches. Many years ago I wrote vmdebootstrap, a tool for installing Debian on a disk image for virtual machines. I had a clear personal need: I was setting up a CI system and it needed six workers: one each for Debian oldstable, stable, and unstable, on two architectures (i386, amd64). Installing Debian six times in the same way is a lot of work, so I figured how difficult can it be to automate it. Turns out that not difficult at all, except to install a bootloader. (Don't ask me why I didn't use any of the other tools for this. It was long ago, and while some of the tools that now exist probably did exist then, I like writing code and learning things while doing it.) After a while I was happy with what the program did, but didn't want to upload it to Debian, and didn't want to add the kinds of things other people wanted, so I turned vmdebootstrap over to Neil Williams, who added a ton of new features. Unfortunately, it turned out that my initial architecture was not scaleable, and also the code I wrote wasn't very good, and there weren't any tests. Neil did heroic work forcing my crappy software into doing things I never envisioned. Last year he needed a break and asked me to take vmdebootstrap back. I did, and have been hiding from the public eye ever since, since I was so ashamed of the code. (I created a new identity and pretended to be an international assassin and backup specialist, travelling the world forcing people to have at least one tested backup of their system. If you've noticed reports in the press about people reporting near-death experiences while holding a shiny new USB drive, that would've been my fault.) Pop quiz: if you have a program with ten boolean options ("do this, except if that option is given, do the other thing"), how many black box tests do you need to test all the functionality? If one run of the program takes half an hour, how long will a full test suite run? I did some hard thinking about vmdebootstrap, and came to the sad conclusion that it had reached the end of its useful life as a living software project. There was no reasonable way to add most of the additional functionality people were asking for, and even maintaining the current code was too tedious a task to consider seriously. It was time to make a clean break of the past and start over, without caring about backwards compatibility. After all, the old code wasn't going anywhere so anyone who needed it could still use it. There was no need to burden a new program with my past mistakes. All new mistakes were called for. At the Cambridge mini-Debconf of November, 2016, I gave a short presentation of what I was going to do. I also posted about my plans to the debian-cloud list. In short, I would write a new, more flexible and cleaner replacement to be called vmdb2. For various personal reasons, I've not been able to spend as much time on vmdb2 as I'd like to, but I've now reached the point where I'd like to announce the first alpha version publically. The source code is hosted here: http://git.liw.fi/vmdb2 . There are .deb packages at my personal public APT repo (http://liw.fi/code/), but vmdb2 is easy enough to run directly from a git checkout:

sudo ./vmdb2 foo.vmdb --output foo.img

There's no need to install it to try it. What works:

• vmdb2 can build a disk image with Debian installed, for amd64 only at this time
• the boot loader is GRUB, either for UEFI or BIOS
• the image boots under Qemu / KVM and also on actual hardware

What doesn't work:

• other architecures, including building for a foreign archiecture
• live CD building (no squashfs support)

I'm not opposed to adding support for those, but they're not directly interesting to me. For example, I only have amd64 machines. The best way to get support for additional features is to tell me how, preferably in the form of patches. (If I have to read tons of docs, or other people's code, and then write code and iterate while other people tell me it doesn't work, it's probably not happening.) Why would you be interested in vmdb2? There's a lot of other tools to do what it does, so perhaps you shouldn't care. That's fine. I like writing tools for myself. But if this kind of tool is of interest to you, please do have a look. A short tutorial: vmdb2 wants you to give it a "specification file" (conventionally suffixed .vmdb, because someone stole the .spec suffix, but vmdb2 doesn't care about the name). Below is an example. vmdb2 image specification files are in YAML, since I like YAML, and specify a sequence of steps to take to build the image. Each step is a tiny piece of self-contained functionality provided by a plugin.

steps:
  - mkimg: "  output  "
    size: 4G
  - mklabel: msdos
    device: "  output  "
  - mkpart: primary
    device: "  output  "
    start: 0%
    end: 100%
    part-tag: root-part

The above create an image (name is specified with the --output option), which is four gigabytes in size, and create a partitition table and a single partition that fills the whole disk. The "tag" is given so that later steps can easily refer to the partition. If you prefer another way to partition the disk, you can achieve that by adding more "mkpart" steps. For example, for UEFI you'll want to have an EFI partition.

  - mkfs: ext4
    partition: root-part
  - mount: root-part
    fs-tag: root-fs

The above formats the partition with the ext4 filesystem, and then mounts it. The mount point will be a temporary directory created by vmdb2, and a tag is again given to the mount point so it can be referred to.

  - unpack-rootfs: root-fs

The above unpacks a tar archive to put content into the filesystem, if the tar archive exists. The tar archive is specified with the --rootfs-tarball command line option.

  - debootstrap: stretch
    mirror: http://http.debian.net/debian
    target: root-fs
    unless: rootfs_unpacked
  - apt: linux-image-amd64
    fs-tag: root-fs
    unless: rootfs_unpacked
  - cache-rootfs: root-fs
    unless: rootfs_unpacked

The above will run debootstrap and install a kernel into the filesystem, but skip doing that if the rootfs tarball was used. Also, the tarball is created if it didn't exist. This way the tarball is used by all but the first run, which saves a bit of time. On my laptop and with a local mirror, debootstrap and kernel installation takes on the order of nine minutes (500 to 600 seconds), whereas unpacking the tar archive is a bit faster (takes around 30 seconds). When iterating over things other than debootstrap, this speeds things up something wonderful, and seems worth the complexity. The "unless:" mechanism is generic. All the steps share some state, and the unpack-rootfs step sets the "rootfs_unpacked" flag in the shared state. The "unless:" field tells vmdb2 to check for the flag and if it is not set, or if it is set to false ("unless it is set to true"), vmdb2 will execute the step. vmdb2 may get more such flags in the future, if there's need.

  - chroot: root-fs
    shell:  
      sed -i '/^root:[^:]*:/s//root::/' /etc/passwd
      echo pc-vmdb2 > /etc/hostname

The above executes a couple of shell commands in a chroot of the root filesystem we've just created. In this case they remove a login password from root, and set the hostname. This is a replacement of the vmdebootstrap "customize" script, but it can be inserted anywhere into the sequence of steps. There's boot chroot and non-chroot variants of the step. This is a good point to mention that writing customize scripts gets quite repetitive and tedious after a while, so vmdb2 has a plugin to run Ansible instead. You can customize your image with that instead, while the image is being built and not have to wait until you boot the image and running Ansible over ssh.

  - grub: bios
    root-fs: root-fs
    root-part: root-part
    device: "  output  "
    console: serial

Finally, install a boot loader, grub. This shows the BIOS variant, UEFI is also supported. This also configures grub and the kernel to use a serial console. There's a "yarn" (test suite) to build and smoke test an image with vmdb2 to make sure at least the basic functionality works. The smoke test boots the image under Qemu, logs in as root, and tells the VM to power off. Very, very basic, but has already found actual bugs in vmdb2. The smoke test needs the serial console to work. As with vmdebootstrap originally, I don't particularly want to maintain the package in Debian. I've added Debian packaging (so that I can install it on my own machines), but I already have enough packages to maintain, so I'm hoping someone else will volunteer to take on the Debian maintainership and bug handling duties. If you would like vmdb2 to do more things to suit you better, I'm happy to explain how to write plugins to provide more types of steps. If you are currently using vmdebootstrap, either directly or as part of another tool, I encourage you to have a look at vmdb2. In the long term, I would like to retire vmdebootstrap entirely, once vmdb2 can do everything vmdebootstrap can do, and few people use vmdebootstrap. This may take a while. In any case, whether you want a new image building tool or not, happy hacking.

TeX Live 2017 has been released! CTAN mirrors are busy updating. Get out bottles of good wine, and enjoy a good long download
Besides the huge amount of package updates due to the 2 month testing hiatus, I want to pick a few changes before copying the complete changelog entry: Additional TEXMF trees tlmgr got a new functionality to easily add and remove additional TEXMF trees to the search path. MikTeX had this features since long, and it was often requested. As it turned out, it is a rather trivial thing to achieve by some texmf.cnf lines. The rest is just front end in tlmgr. Here a few invocations (not very intelligent usage, though):

$ tlmgr conf auxtrees show
tlmgr: no auxiliary texmf trees defined.
$ tlmgr conf auxtrees add /projects/book-abc
$ tlmgr conf auxtrees
List of auxiliary texmf trees:
  /projects/book-abc
$ tlmgr conf auxtrees remove /projects/book-abc
$ tlmgr conf auxtrees show
tlmgr: no auxiliary texmf trees defined.

TeX Live Manager interactive shell tlmgr also got an interactive shell now, that can also be used for scripting. Available commands are all the usual command line actions, plus a few more (see documentation). Again, here a simple session:

$ tlmgr shell
protocol 1
tlmgr> load local
OK
tlmgr> load remote
tlmgr: package repositories
	main = /home/norbert/public_html/tlnet (verified)
	koma = http://www.komascript.de/repository/texlive/2017 (verified)
OK
tlmgr> update --list
tlmgr: package repositories
	main = /home/norbert/public_html/tlnet (verified)
	koma = http://www.komascript.de/repository/texlive/2017 (verified)
tlmgr: saving backups to /home/norbert/tl/2017/tlpkg/backups
tlmgr: no updates available
OK
tlmgr> byebye
$

User versus System mode for updmap and fmtutil I have reported on this during BachoTeX/TUG 2017, here are the slides, that the two central configuration programs updmap and fmtutil will change their operation mode slightly by disallowing invocations without mde specification. That is, one either needs to call updmap -sys (or updmap-sys, nothing changed here from previous years) or updmap -user (or updmap-user, new in TeX Live 2017). We hope by this and the accompanying web page of recommendations to help users not to shoot themselves to often by calling updmap without knowing the consequences. The upcoming proceedings of the BachoTeX will also comtain an article fully documenting both updmap and fmtutil including the most recent changes.

---

These were only a few changes where I had my fingers in the development. For the full list, read on below. Now get ready for partying, and also for the end of the peace, because daily package updates will restart in the next days. Enjoy Changes since TeX Live 2016 The following list of changes is directly from the TeX Live documentation.

• LuaTEX: More callbacks, more typesetting control, more access to internals; ffi library for dynamic code loading added on some platforms.
• pdfTEX: Environment variable SOURCE_DATE_EPOCH_TEX_PRIMITIVES from last year renamed to FORCE_SOURCE_DATE, with no changes in functionality; if the \pdfpageattr token list contains the string /MediaBox, omit output of the default /MediaBox.
• XeTEX: Unicode/OpenType math now based on HarfBuzz s MATH table support; some bug fixes.
• Dvips: Make the last papersize special win, for consistency with dvipdfmx and package expectations; the -L0 option (L0 config setting) restores the previous behavior of the first special winning.
• epTEX, eupTEX: New primitives \pdfuniformdeviate, \pdfnormaldeviate, \pdfrandomseed, \pdfsetrandomseed, \pdfelapsedtime, \pdfresettimer, from pdfTEX.
• MacTEX: As of this year, only Mac OS X releases for which Apple still releases security patches will be supported in MacTEX, under the platform name x86_64-darwin; currently this means Yosemite, El Capitan, and Sierra (10.10 and newer). Binaries for older Mac OS X versions are not included in MacTEX, but are still available in TEX Live (x86_64-darwinlegacy, i386-darwin, powerpc-darwin).
• Infrastructure: The TEXMFLOCAL tree is now searched before TEXMFSYSCONFIG and TEXMFSYSVAR (by default); the hope is that this will better match expectations of local files overriding system files. Also, tlmgr has a new mode shell for interactive and scripted use, and a new action conf auxtrees to easily add and remove extra trees.
• updmap and fmtutil: These scripts now give a warning when invoked without explicitly specifying either so-called system mode (updmap-sys, fmtutil-sys, or option -sys), or user mode (updmap-user, fmtutil-user, or option -user). The hope is that this will reduce the perennial problem of invoking user mode by accident and thus losing future system updates. See http://tug.org/texlive/scripts-sys-user.html and an upcoming TUGboat article for details.
• install-tl: Personal paths are now set to MacTEX values (~/Library/...) by default on Macs. New option -init-from-profile to start an installation with the values from a given profile; new command P to explicitly save a profile; new profile variable names (but previous ones are still accepted).
• SyncTEX: the temporary file name now looks like foo.synctex(busy), instead of foo.synctex.gz(busy) (no .gz). Front ends and build systems that want to remove temp files may need adjusting.
• Other utilities: texosquery-jre8 is a new cross-platform program for retrieving locale and other OS information from a TEX document; it is included in shell_escape_commands by default for restricted shell execution. (Older JRE versions are supported by texosquery, but cannot be enabled in restricted mode, as they are no longer supported by Oracle, even for security issues.)
• Platforms: See MacTEX entry above; no other changes.

Git-cinnabar is a git remote helper to interact with mercurial repositories. It allows to clone, pull and push from/to mercurial remote repositories, using git. Get it on github. These release notes are also available on the git-cinnabar wiki. What s new since 0.4.0?

• git-cinnabar-helper is now mandatory. You can either download one with git cinnabar download on supported platforms or build one with make helper.
• Metadata changes require to run git cinnabar fsck.
• Mercurial tags are consolidated in a separate (fake) repository. See the README file.
• Updated git to 2.13.0 for git-cinnabar-helper.
• Improved memory consumption and performance.
• Improved experimental support for pushing merges.
• Experimental support for clonebundles.
• Removed support for the .git/hgrc file for mercurial specific configuration.
• Support any version of Git (was previously limited to 1.8.5 minimum)

Stretch So I have done my (hopefully) last uploads for Stretch (r0) today and want to share how pleased I've been with this release cycle. It was the smoothest freeze I've witnessed so far, I really liked the clear and documented rules from the Release Team, their regular release infos on d-d-a and publically logged IRC meetings plus the easy-to-remember URL http://release.debian.org (nicely redirecting to https), where one can easily find those relevant dates, emails and rules. Plus, useful bug views via https://udd.debian.org. There is more, but to me the last (and probably most important) factor to all this has been that most of this actually hasn't been new in this release cycle, but rather that by now a significant number of developers are aware of this and mostly have acted accordingly and communicated this further. Kudos and many thanks to the Release team, you have been amazing! Many thanks also to everyone who has contributed nicely to those 24838 source packages! You rock as well. And then I'll also like the upcoming quieter two weeks, also to prepare the Debian Edu Stretch announcement And then, finally, I really really look forward to the next development cycle and being able to meaningful start working on reproducible Buster!

FTP assistant This month I only marked 39 packages for accept and rejected 5 packages. Debian LTS This was my thirty-fifth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 27.25h. During that time I did LTS uploads or prepared one for Jessie/Sid:

• [DLA 934-1] radicale security update for one CVE
• [DLA 942-1] jbig2dec security update for three CVEs
• [DLA 947-1] icu security update for two CVEs
• [DLA 950-1] libtasn1-3 security update for one CVE
• [DSA 3861-1] libtasn1-6 security update for one CVE
• [DLA 956-1] libsndfile security update for four CVE
• [DLA 957-1] bind9 security update for three CVEs
• [DLA 962-1] tnef security update for one CVE
• [DSA 3869-1] tnef security update for one CVE

For [DLA 948-1] dropbear and [DLA 958-1] libonig I only did the LTS bookkeeping and sent the DLA. The icu upload would not have been possible without the help of Roberto. I also tried to work on jasper, libxml2, libytnef and swftools but unfortunately all upstreams did not finish their respective patches this month, so maybe there will be an upload in June. Other stuff Again this has been a busy LTS month, so I only uploaded a new version of smstools, which closed most of its bugs and adopted adopted ptpd as DOPOM. As a prerequisite of wview I uploaded radlib. Unfortunately I could not do anything for wview, so work on this has to be postponed. Another new package is te923con, which I hope is able to read data from my weather station. Last but no least I fixed an RC bug in alljoyn-services-1504.

Tonight I read this weeks LWN quotes-page a little later than usual because I was busy at work for most of the day. Anyway as always LWNs content was awesome, and this particular list lead to an interesting discussion about a new Linux-Security-Module (LSM).

That read weirdly, what I was trying to say was that every Thursday morning I like to read LWN at work. Tonight is the first chance I had to get round to it.

One of the later replies in the thread was particularly interesting as it said:

Suggestion:
Create an security module that looks for the attribute
    security.WHITELISTED
on things being executed/mmapped and denys it if the attribute
isn't present. Create a program (whitelistd) that reads
/etc/whitelist.conf and scans the system to ensure that only
things on the list have the attribute.

So I figured that was a simple idea, and it didn't seem too hard even for myself as a non-kernel non-developer. There are several linux security modules included in the kernel-releases, beneath the top-level security/ directory, so I assumed I could copy & paste code around them to get something working. During the course of all this work, which took about 90 minutes from start to Finnish (that pun never gets old), this online documentation was enormously useful:

• http://elixir.free-electrons.com/linux/v4.11.3/source

Brief attr primer If you're not familiar with the attr tool it's pretty simple. You can assign values to arbitrary labels on files. The only annoying thing is you have to use extra-flags to commands like rsync, tar, cp, etc, to preserve the damn things. Set three attributes on the file named moi:

$ touch moi
$ attr -s forename -V "Steve"      moi
$ attr -s surname  -V "Kemp"       moi
$ attr -s name     -V "Steve Kemp" moi

Now list the attributes present:

$ attr -l moi
Attribute "name" has a 10 byte value for moi
Attribute "forename" has a 5 byte value for moi
Attribute "surname" has a 4 byte value for moi

And retrieve one?

$ attr -q -g name moi
Steve Kemp

LSM Skeleton My initial starting point was to create "steve_lsm.c", with the following contents:

 #include <linux/lsm_hooks.h>
 /*
  * Log things for the moment.
  */
 static int steve_bprm_check_security(struct linux_binprm *bprm)
  
     printk(KERN_INFO "STEVE LSM check of %s\n", bprm->filename);
     return 0;
  
 /*
  * Only check exec().
  */
 static struct security_hook_list steve_hooks[] =  
     LSM_HOOK_INIT(bprm_check_security, steve_bprm_check_security),
  ;
 /*
  * Somebody set us up the bomb.
  */
 static void __init steve_init(void)
  
     security_add_hooks(steve_hooks, ARRAY_SIZE(steve_hooks), "steve");
     printk(KERN_INFO "STEVE LSM initialized\n");
  

With that in place I had to modify the various KBuild files beneath security/ to make sure this could be selected as an LSM, and add in a Makefile to the new directory security/steve/. With the boiler-plate done though, and the host machine rebooted into my new kernel it was simple to test things out. Obviously the first step, post-boot, is to make sure that the module is active, which can be done in two ways, looking at the output of dmesg, and explicitly listing the modules available:

 ~# dmesg   grep STEVE   head -n2
 STEVE LSM initialized
 STEVE LSM check of /init
 $ echo $(cat /sys/kernel/security/lsm)
 capability,steve

Making the LSM functional The next step was to make the module do more than mere logging. In short this is what we want:

• If a binary is invoked by root - allow it.
  • Although note that this might leave a hole, if the user can enter a new namespace where their UID is 0..
• If a binary is invoked by a non-root user look for an extended attribute on the target-file named security.WHITELISTED.
  • If this is present we allow the execution.
  • If this is missing we deny the execution.

NOTE we don't care what the content of the extended attribute is, we just care whether it exists or not. Reading the extended attribute is pretty simple, using the __vfs_getxattr function. All in all our module becomes this:

  #include <linux/xattr.h>
  #include <linux/binfmts.h>
  #include <linux/lsm_hooks.h>
  #include <linux/sysctl.h>
  #include <linux/ptrace.h>
  #include <linux/prctl.h>
  #include <linux/ratelimit.h>
  #include <linux/workqueue.h>
  #include <linux/string_helpers.h>
  #include <linux/task_work.h>
  #include <linux/sched.h>
  #include <linux/spinlock.h>
  #include <linux/lsm_hooks.h>
  /*
   * Perform a check of a program execution/map.
   *
   * Return 0 if it should be allowed, -EPERM on block.
   */
  static int steve_bprm_check_security(struct linux_binprm *bprm)
   
         // The current task & the UID it is running as.
         const struct task_struct *task = current;
         kuid_t uid = task->cred->uid;
         // The target we're checking
         struct dentry *dentry = bprm->file->f_path.dentry;
         struct inode *inode = d_backing_inode(dentry);
         // The size of the label-value (if any).
         int size = 0;
         // Root can access everything.
         if ( uid.val == 0 )
            return 0;
         size = __vfs_getxattr(dentry, inode, "user.whitelisted", NULL, 0);
         if ( size >= 0 )
          
             printk(KERN_INFO "STEVE LSM check of %s resulted in %d bytes from 'user.whitelisted' - permitting access for UID %d\n", bprm->filename, size, uid.val );
             return 0;
          
         printk(KERN_INFO "STEVE LSM check of %s denying access for UID %d [ERRO:%d] \n", bprm->filename, uid.val, size );
         return -EPERM;
   
  /*
   * The hooks we wish to be installed.
   */
  static struct security_hook_list steve_hooks[] =  
       LSM_HOOK_INIT(bprm_check_security, steve_bprm_check_security),
   ;
  /*
   * Initialize our module.
   */
  void __init steve_add_hooks(void)
   
       /* register ourselves with the security framework */
       security_add_hooks(steve_hooks, ARRAY_SIZE(steve_hooks), "steve");
       printk(KERN_INFO "STEVE LSM initialized\n");
   

Once again we reboot with this new kernel, and we test that the LSM is active. After the basic testing, as before, we can now test real functionality. By default no binaries will have the attribute we look for present - so we'd expect ALL commands to fail, unless executed by root. Let us test that:

~# su - nobody -s /bin/sh
No directory, logging in with HOME=/
Cannot execute /bin/sh: Operation not permitted

That looks like it worked. Let us allow users to run /bin/sh:

 ~# attr -s whitelisted -V 1 /bin/sh

Unfortunately that fails, because symlinks are weird, but repeating the test with /bin/dash works as expected:

 ~# su - nobody -s /bin/dash
 No directory, logging in with HOME=/
 Cannot execute /bin/dash: Operation not permitted
 ~# attr -s whitelisted -V 1 /bin/dash
 ~# attr -s whitelisted -V 1 /usr/bin/id
 ~# su - nobody -s /bin/dash
 No directory, logging in with HOME=/
 $ id
 uid=65534(nobody) gid=65534(nogroup) groups=65534(nogroup)
 $ uptime
 -su: 2: uptime: Operation not permitted

And our logging shows the useful results as we'd expect:

  STEVE LSM check of /usr/bin/id resulted in 1 bytes from 'user.WHITELISTED' - permitting access for UID 65534
  STEVE LSM check of /usr/bin/uptime denying access for UID 65534 [ERRO:-95]

Surprises If you were paying careful attention you'll see that we changed what we did part-way through this guide.

• The initial suggestion said to look for security.WHITELISTED.
• But in the kernel module I look for user.whitelisted.
  • And when setting the attribute I only set whitelisted.

Not sure what is going on there, but it was very confusing. It appears to be the case that when you set an attribute a secret user. prefix is added to the name. Could be worth some research by somebody with more time on their hands than I have. Anyway I don't expect this is a terribly useful module, but it was my first, and I think it should be pretty stable. Feedback on my code certainly welcome!