FTP master This month I accepted 384 packages and rejected 47. The overall number of packages that got accepted was 457. Debian LTS This was my seventieth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 28.75h. During that time I did LTS uploads of:

• [DLA 2183-1] libgsf security update for one CVE
• [DLA 2184-1] jsch security update for one CVE
• [DLA 2185-1] eog security update for one CVE
• [DLA 2187-1] radicale security update for one CVE
• [DLA 2186-1] ncmpc security update for one CVE
• [DLA 2188-1] php5 security update for three CVEs
• [DLA 2189-1] rzip security update for one CVE
• [DLA 2195-1] w3m security update for two CVEs
• [DLA 2194-1] yodl security update for one CVE
• [DLA 2197-1] miniupnpc security update for one CVE
• [DLA 2196-1] pound security update for one CVE

As there have been lots of no-dsa-CVEs I continued my work on wireshark. Last but not least I did some days of frontdesk duties. Debian ELTS This month was the twenty second ELTS month. During my small allocated time I only uploaded:

• ELA-227-1 for php5 fixing four CVEs

I also did some days of frontdesk duties. Other stuff Unfortunately this month again strange things happened outside Debian and I only got some stuff done. I improved packaging of

• libctl
• meep

I sponsored uploads of

• ulfius

I uploaded the new package

• puppet-module-cirrax-gitolite

On my Go challenge I uploaded:
golang-github-facebookgo-subset, golang-github-facebookgo-ensure, golang-github-shurcool-gopherjslib, golang-github-grafana-grafana-plugin-model, golang-github-crewjam-httperr, golang-github-hashicorp-terraform-svchost, golang-github-neelance-sourcemap, golang-github-neelance-astrewrite, golang-github-kisielk-gotool, golang-github-gopherjs-gopherjs, golang-github-yvasiyarov-newrelic-platform-go, golang-github-rhnvrm-simples3, golang-github-robfig-go-cache, golang-github-xorcare-pointer, golang-github-goburrow-serial

Welcome to the April 2020 report from the Reproducible Builds project. In our regular reports we outline the most important things that we and the rest of the community have been up to over the past month. What are reproducible builds? One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. But whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into seemingly secure software during the various compilation and distribution processes.

News It was discovered that more than 725 malicious packages were downloaded thousands of times from RubyGems, the official channel for distributing code for the Ruby programming language. Attackers used a variation of typosquatting and replaced hyphens and underscores (for example, uploading a malevolent atlas-client in place of atlas_client) that executed a script that intercepted Bitcoin payments. (Ars Technica report) Bernhard M. Wiedemann launched ismypackagereproducibleyet.org, a service that takes a package name as input and displays whether the package is reproducible in a number of distributions. For example, it can quickly show the status of Perl as being reproducible on openSUSE but not in Debian. Bernhard also improved the documentation of his unreproducible package to add some example patches for hash issues. [ ]. There was a post on Chaos Computer Club s website listing Ten requirements for the evaluation of Contact Tracing apps in relation to the SARS-CoV-2 epidemic. In particular:

4. Transparency and verifiability: The complete source code for the app and infrastructure must be freely available without access restrictions to allow audits by all interested parties. Reproducible build techniques must be used to ensure that users can verify that the app they download has been built from the audited source code.

Elsewhere, Nicolas Boulenguez wrote a patch for the Ada programming language component of the GCC compiler to skip -f.*-prefix-map options when writing Ada Library Information files. Amongst other properties, these .ali files embed the compiler flags used at the time of the build which results in the absolute build path being recorded via -ffile-prefix-map, -fdebug-prefix-map, etc. In the Arch Linux project, kpcyrd reported that they held their first rebuilder workshop . The session was held on IRC and participants were provided a document with instructions on how to install and use Arch s repro tool. The meeting resulted in multiple people with no prior experience of Reproducible Builds validate their first package. Later in the month he also announced that it was now possible to run independent rebuilders under Arch in a hands-off, everything just works solution to distributed package verification. Mathias Lang submitted a pull request against dmd, the canonical compiler for the D programming languageto add support for our SOURCE_DATE_EPOCH environment variable as well the other C preprocessor tokens such __DATE__, __TIME__ and __TIMESTAMP__ which was subsequently merged. SOURCE_DATE_EPOCH defines a distribution-agnostic standard for build toolchains to consume and emit timestamps in situations where they are deemed to be necessary. [ ] The Telegram instant-messaging platform announced that they had updated to version 5.1.1 continuing their claim that they are reproducible according to their full instructions and therefore verifying that its original source code is exactly the same code that is used to build the versions available on the Apple App Store and Google Play distribution platforms respectfully. Lastly, Herv Boutemy reported that 97% of the current development versions of various Maven packages appear to have a reproducible build. [ ]

Distribution work In Debian this month, 89 reviews of Debian packages were added, 21 were updated and 33 were removed this month adding to our knowledge about identified issues. Many issue types were noticed, categorised and updated by Chris Lamb, including:

• captures_build_path_in_hd5_database_files
• cargo_installs_crates2_json
• nondeterministic_devhelp_documentation_generated_by_gtk_doc
• ros_dynamic_reconfigure_captures_build_path

In addition, Holger Levsen filed a feature request against debrebuild, a tool for rebuilding a Debian package given a .buildinfo file, proposing to add --standalone or --one-shot-mode functionality.
In openSUSE, Bernhard M. Wiedemann made the following changes:

• blender (sort C readdir call, rejected upstream)
• guile/guix (parallelism race condition)
• mingw32-filesystem/mingw32-binutils (sort readdir, filesystem, toolchain)
• mingw64-filesystem/mingw64-binutils (sort readdir, filesystem, toolchain)
• musescore (non-deterministic .zip files)
• OBS (FTBFS in rebuild)
• perl-Image-Sane (report hung build on a single core VM)
• ruby2.7 (date, already upstream)
• vtk (drop unreproducible .pyc file)

In Arch Linux, a rebuilder instance has been setup at reproducible.archlinux.org that is rebuilding Arch s [core] repository directly. The first rebuild has led to approximately 90% packages reproducible contrasting with 94% on the Reproducible Build s project own ArchLinux status page on tests.reproducible-builds.org that continiously builds packages and does not verify Arch Linux packages. More information may be found on the corresponding wiki page and the underlying decisions were explained on our mailing list.

Software development

diffoscope Chris Lamb made the following changes to diffoscope, the Reproducible Builds project s in-depth and content-aware diff utility that can locate and diagnose reproducibility issues (including preparing and uploading versions 139, 140, 141, 142 and 143 to Debian which were subsequently uploaded to the backports repository):

• Comparison improvements:
  • Dalvik .dex files can also serve as APK containers so restrict the narrower identification of .dex files to files ending with this extension and widen the identification of APK files to when file(1) discovers a Dalvik file. (#28)
  • Add support for Hierarchical Data Format (HD5) files. (#95)
  • Add support for .p7c and .p7b certificates. (#94)
  • Strip paths from the output of zipinfo(1) warnings. (#97)
  • Don t uselessly include the JSON similarity percentage if it is 0.0% . [ ]
  • Render multi-line difference comments in a way to show indentation. (#101)
• Testsuite improvements:
  • Add pdftotext as a requirement to run the PDF test_metadata text. (#99)
  • apktool 2.5.0 changed the handling of output of XML schemas so update and restrict the corresponding test to match. (#96)
  • Explicitly list python3-h5py in debian/tests/control.in to ensure that we have this module installed during a test run to generate the fixtures in these tests. [ ]
  • Correct parsing of ./setup.py test --pytest-args arguments. [ ]
• Misc:
  • Capitalise Ordering differences only in text comparison comments. [ ]
  • Improve documentation of FILE_TYPE_HEADER_PREFIX and FALLBACK_FILE_TYPE_HEADER_PREFIX to highlight that only the first 16 bytes are used. [ ]

Michael Osipov created a well-researched merge request to return diffoscope to using zipinfo directly instead of piping input via /dev/stdin in order to ensure portability to the BSD operating system [ ]. In addition, Ben Hutchings documented how --exclude arguments are matched against filenames [ ] and Jelle van der Waa updated the LLVM test fixture difference for LLVM version 10 [ ] as well as adding a reference to the name of the h5dump tool in Arch Linux [ ]. Lastly, Mattia Rizzolo also fixed in incorrect build dependency [ ] and Vagrant Cascadian enabled diffoscope to locate the openssl and h5dump packages on GNU Guix [ ][ ], and updated diffoscope in GNU Guix to version 141 [ ] and 143 [ ].

strip-nondeterminism strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. In April, Chris Lamb made the following changes:

• Add deprecation plans to all handlers documenting how or if they could be disabled and eventually removed, etc. (#3)
• Normalise *.sym files as Java archives. (#15)
• Add support for custom .zip filename filtering and exclude two patterns of files generated by Maven projects in fork mode. (#13)

disorderfs disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues. This month, Chris Lamb fixed a long-standing issue by not drop UNIX groups in FUSE multi-user mode when we are not root (#1) and uploaded version 0.5.9-1 to Debian unstable. Vagrant Cascadian subsequently refreshed disorderfs in GNU Guix to version 0.5.9 [ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

• Bernhard M. Wiedemann:
  • elixir (parallelism)
  • gnutls (build failure)
  • moonjit/bcc (compile-time CPU-detection)
  • openstack (backport of patch to drop unreproducible sphinx .pickle files)
  • x3270 (merged, update date patch)
• Chris Lamb:
  • #958301 filed against dh-cargo.
  • #956549 filed against gmap.
  • #956591 filed against gpick.
  • #956477 filed against herbstluftwm.
  • #956304 filed against libcamera.
  • #956589 filed against libctl.
  • #956408 filed against minetest-mod-xdecor.
  • #955783 filed against netgen-lvs.
  • #958110 filed against nickle.
  • #958381 filed against nmrpflash.
  • #958382 filed against node-mqtt.
  • #956473 filed against sprai.
  • #955501 filed against yaz.
  • #956583 filed against xxhash.

In addition, Bernhard informed the following projects that their packages are not reproducible:

• acoular (report unknown non-determinism)
• cri-o (report a date issue)
• gnutls (report certtool being unable to extend certificates beyond 2049)
• gnutls (report copyright year variation)
• libxslt (report a bug about non-deterministic output from data corruption)
• python-astropy (report a future build failure in 2021)

Project documentation This month, Chris Lamb made a large number of changes to our website and documentation in the following categories:

• Community engagement improvements:
  • Update instructions to register for Salsa on our Contribute page now that the signup process has been overhauled. [ ]
  • Make it clearer that joining the rb-general mailing list is probably a first step for contributors to take. [ ]
  • Make our full contact information easier to find in the footer (#19) and improve text layout using bullets to separate sections [ ].
• Accessibility:
  • To improve accessibility, make all links underlined. (#12)
  • Use an enhanced foreground/background contrast ratio of 7.04:1. (#11)
• General improvements:
  • Add a new Academic publications page. (#22)
  • Add Trezor to our list of affiliated projects. (#26)
  • Add the JVM page to the documentation index (#17) and tidy the page itself a little [ ].
  • Add a GNU Libtool pointer to the Archive metadata documentation page. [ ]
• Internals:
  • Move to using jekyll-redirect-from over manual redirect pages [ ][ ] and add a redirect from /docs/buildinfo/ to /docs/recording/. (#23)
  • Limit the website self-check to not scan generated files [ ] and remove the old layout checker now that I have migrated all them [ ].
  • Move the news archive under the /news/ namespace [ ] and improve formatting of archived news links [ ].
  • Various improvements to the draft template generation. [ ][ ][ ][ ]

In addition, Holger Levsen clarified exactly which month we ceased to do weekly reports [ ] and Mattia Rizzolo adjusted the title style of an event page [ ]. Marcus Hoffman also started a discussion on our website s issue tracker asking for clarification on embedded signatures and Chris Lamb subsequently replied and asked Marcus to go ahead and propose a concrete change.

Testing framework We operate a large and many-featured Jenkins-based testing framework that powers tests.reproducible-builds.org that, amongst many other tasks, tracks the status of our reproducibility efforts as well as identifies any regressions that have been introduced.

• Chris Lamb:
  • Print the build environment prior to executing a build. [ ]
  • Drop a misleading disorderfs-debug prefix in log output when we change non-disorderfs things in the file and, as it happens, do not run disorderfs at all. [ ]
  • The CSS for the package report pages added a margin to all <a> HTML elements under <li> ones, which was causing a comma/bullet spacing issue. [ ]
  • Tidy the copy in the project links sidebar. [ ]
• Holger Levsen:
  • General:
    • Install jekyll-redirect-from as it now needed by the reproducible-builds.org website. [ ]
    • Improve/correct log parsing rules. [ ][ ]
  • Debian:
    • Reduce scheduling frequency of the buster distribution on the arm64 architecture, etc.. [ ][ ]
    • Show builds per day on a per-architecture basis for the last year on the Debian dashboard. [ ]
    • Drop the Subgraph OS package set as development halted in 2017 or 2018. [ ]
    • Update debrebuild to version from the latest version of devscripts. [ ][ ]
    • Add or improve various parts of the documentation. [ ][ ][ ]
  • Work on a Debian rebuilder:
    • Integrate sbuild. [ ][ ][ ][ ][ ]
    • Select a random .buildinfo file and attempt to build and compare the result. [ ][ ][ ][ ]
    • Improve output and related output formatting. [ ][ ][ ][ ][ ]
    • Outline next steps for the development of the tool. [ ][ ][ ]
    • Various refactoring and code improvements. [ ][ ][ ]

Lastly, Mattia Rizzolo fixed some log parsing code regarding potentially-harmless warnings from package installation [ ][ ] and the usual build node maintenance was performed by Holger Levsen [ ][ ][ ] and Mattia Rizzolo [ ][ ][ ].

---

Misc news On our mailing list this month, Santiago Torres asked whether we were still publishing releases of our tools to our website and Chris Lamb replied that this was not the case and fixed the issue. Later in the month Santiago also reported that the signature for the disorderfs package did not pass its GPG verification which was also fixed by Chris Lamb. Hans-Christoph Steiner of the Guardian Project asked whether there would be interest in making our website translatable which resulted in a WIP merge request being filed against the website and a discussion on how to track translation updates.
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Twitter: @ReproBuilds @reproducible_builds@fosstodon.org
• Reddit: /r/ReproducibleBuilds
• Mailing list: rb-general@lists.reproducible-builds.org

This month s report was written by Bernhard M. Wiedemann, Chris Lamb, Daniel Shahaf, Holger Levsen, Jelle van der Waa, kpcyrd, Mattia Rizzolo and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

Continuing on the path towards all my stuff being managed by Ansible, I ve figured out a method of managing the reverse DNS entries for subnets on the Hetzner Dedicated Server. There s a bunch of Ansible modules for handling Hetzner Cloud, but these servers are managed in Robot which the Cloud API doesn t cover. Instead, you need to use the Robot Webservice. Ansible does have a module for doing pretty arbitrary things with web APIs though, so using that I ve got the following playbook figured out to keep the reverse DNS entries in sync: The host groups this runs on are currently hardcoded as the VMs that live in the Hetzner Dedicated Server. A future iteration of this might use some inventory plugin to look up the subnets that are managed on Hetzner and create a group for all of those. Right now it won t be setting the reverse DNS for the router interface on that subnet, and won t automatically include new server s subnets. Gathering facts is disabled because all of these run locally. There is at least one VM running on this server that I can t log in to because I host it for someone else, so running locally is a necessity. The webservice credentials are stored in an Ansible Vault encrypted YAML file and loaded explicitly. An important note: the webservice username and password is not the same as your regular Robot username and password. You need to create a webservice user in Robot under Settings; Webservice and app settings . If you attempt to authenticate with an incorrect username and password 3 times in a row, your IP address will be blocked for 10 minutes. There are 6 hosts in this group, so I did this a few times before I realised there was a different user account I needed to create. I d suggest limiting to a single host while you re testing to get the authentication figured out. The actual calls to the webservice take place in a block just to avoid having to specify the delegate_to: localhost twice. The first step looks up the current PTR record, and accepts success if it gives either a 200 or 404 status code (it will be 404 if there is no existing pointer record). The result of this is used to conditionally create/update the PTR record in the next task. If nothing needs to be done, nothing will be changed and the second task will be skipped. If a change is needed, the second step is successful if either the PTR record is updated (status 200) or created (status 201). This was actually a lot easier than I thought it would be, and the uri module looks to be really flexible, so I bet there are other things that I could easily integrate into my Ansible playbooks.

Changes

• apt-offline: fix typos
• myrepos: cleanups (1 2), better error messages (1 2)
• Linux kernel: enable dm raid discard support when any devices support discard
• gh: fix .deb install info
• equivs: fix dir message
• webdl: fix iview episode order (sent privately)
• spelling dictionaries: codespell (1 2 3 4 5), lintian (1 2 3 4)
• Debian blends website: fix DDTP URLs
• Debian developers reference: fix DDTP URLs
• Debian bits blog: fix DDTP URLs
• Debian screenshots: deleted scm (manual page)
• Debian package tracking system: add options to disable oldoldstable pockets (1 2)
• Debian website: fix DDTP (1 2) and DebConf sponsors URLs
• Debian userdir-ldap: document #debian-private
• Debian package uploads: purple-discord (1 2 3)
• Debian sysadmin wiki pages: input/howto/postgres
• Debian website: document #debian-private
• Debian wiki pages: apcupsd, Blends/TasksPages, BOINC/Development, configure, DDTP (Migration), DebianAMD64, DebianEdu/Status/Buster, DebianHosting, DebianInstaller/Preseed/EditIso, DebianKernel/UserspaceTools, DebianNetDomains, DebianPartyLine/Mumble, DebianScience/Classification, DebianWannaBuildInfrastructureOnOneServer, DebianWiki/I18n, Derivatives/CensusQA, DevelopmentQuestions, EmbeddedCopies, EtcUnderRevisionControl, Exploits, Firmware/Updates, FlashBIOS, Fonts, Games/Team, Hardware/Wanted, HelpDebian/FAQ, HowToUpgradeKernel, InstallingDebianOn/CompuLab, IRC, Javascript/Policy, KdeMultimedia, L10n/DDTSS, LinuxMagazines, MemberBenefits (1 2), MiscellaneousQuestions, Mobile, OpenStack/OpenStack/todo, PackageArchitectureAlmostAny, PackagesArchSpecific, PageFragments/HowOrganisationsCanHelpDebian, qa.debian.org, ReleaseParty (Bullseye, Buster, SuiteTemplate), Services, Simple-CDD/Bzr, SparcPorting, SSH, SSH, StaticLinking, Teams (DebConf/Bursaries, DebianBuildd, DebianWiki, FrontDesk/DcSiteDevel, I18n), VerifyPAS
• vcs-home wiki pages: index (1 2)
• FOSSjobs wiki pages: resources

Issues

• Crashes in tracker-miner-fs, Xwayland
• Upgrade failure with unattended-upgrades
• Conffile removal needed in rubocop, parcimonie
• Patch backport needed in purple-discord
• Underlinking in bind9-libs
• Features in needrestart, buildd.debian.org, procps (1 2 3), cron, Debian code search, Debian package tracker, pass
• Broken HTML in evolution (sent privately, fixed by upstream)
• Warnings in urwid, webdl (sent privately)
• Removal needed for flasm (1 2)

Review

• Spam: reported 1 Debian bug reports and 178 Debian mailing list posts
• Debian wiki: RecentChanges for the month
• Debian screenshots:
  • approved dablin displaycal enfuse evolvotron flameshot flpsed font-viewer gimp-plugin-registry glue-sprite gpick gpscorrelate gpscorrelate-gui gtkmorph guetzli gummi hdrmerge icc-profiles-free inkscape-tutorials jhead jp2a jpeginfo jpegjudge kde-spectacle libjpeg-turbo-progs magic metacam pianobooster povray-examples printer-driver-cups-pdf propaganda-debian rawtherapee recoverjpeg renrot scm screengrab scrot shanty showfoto simple-image-reducer simple-scan sxiv teeworlds tuxpaint valentina viewnior x4d-icons xcftools
  • rejected musescore (ancient map), fastboot (logo)
  • approved deletion (despite bogus reason) of winwrangler (help output)
  • rejected deletion of blockattack (not spam as claimed), chromium (not deceptive as claimed), file-roller (screenshots of newest interface), jwm/onboard/aajm (spam), xfce4 (not a place to request wallpapers), viewvc (HTML jibberish), fonts-noto-color-emoji/sdl-ball/sddm-theme-breeze/hardinfo (jibberish), gufw (non-English screenshots are allowed), wifite (unsubstantiated claim of false information), mysql-workbench (single word)

Administration

• myrepos: fix the forum
• Debian: restart non-responsive tor daemon, restart processes due to OOM, apply debian.net changes for DD with expired key
• Debian wiki: approve accounts
• Debian QA services: deploy changes, auto-disable oldoldstable pockets

Communication

• Initiate discussion about removing purple-discord from Debian stable
• Respond to queries from Debian users and developers on the mailing lists and IRC

Sponsors The purple-discord work was sponsored by my employer. All other work was done on a volunteer basis.

Here s my (seventh) monthly update about the activities I ve done in the F/L/OSS world.

Debian

It s been 14 months since I ve started contributing to Debian. And 4 months since I ve been a Debian Developer. And in this beautiful time, I had this opprotunity to do and learn lots of new and interesting things. And most importantly, meet and interact with lots of lovely people!
Debian is $home.

Uploads:

• libgit2 (0.28.5+dfsg.1-1) - new upstream version.
• ruby-ffi-compiler (1.0.1-1) - NEW (#955497).
• rake (13.0.1-3/4) - using --gem-install layout and fixing autopkgtest.
• mcollective (2.12.5+dfsg-1) - new upstream version.
• ruby-guard (2.16.2-1) - fix regression caused by pry s upload (#954724).
• ruby-pry-byebug (3.9.0-1) - fix regression caused by pry s upload (#954572).
• ruby-ahoy-matey (3.0.2-1) - new upstream version.
• ruby-http-parser (1.2.1-1) - NEW (#955589).
• ruby-http-parser.rb (0.6.0-5) - Drop Conflicts field.
• golang-github-awalterschulze-gographviz (2.0.1-1) - new upstream version.
• ruby-ffi-yajl (2.3.1-3) - fix build in ARM $arch.
• ruby-http (4.4.1-1) - new upstream version ((#890075 and #858140).
• ruby-twitter (7.0.0-1) - new upstream version.
• ruby-rack (2.1.1-2) - migration to unstable.
• ruby-rack-oauth2 (1.11.0-1) - fix FTBFS.
• ruby-crb-blast (0.6.9-4) - fix regression caused by ruby-bio (#954536).
• ruby-sassc-rails (2.1.2-5) - Add Breaks+Replaces for ruby-sass-rails (#952682 and #954544).
• libdbd-firebird-perl (1.32-1) - new upstream version.
• ruby-minitest-global-expectations (1.0.1-1) - NEW (#956051).
• golang-github-cheekybits-genny (1.0.0-1) - NEW (#956128).
• node-clipboard (2.0.6+ds-1~bpo10+1) - backporting to buster.
• micro (2.0.2-3) - use cut -d'-' -f1 to just show upstream version.
• golang-github-go-errors-errors (1.0.1-4) - fix build and autopkgtest (#954521).
• micro (2.0.2-3~bpo10+1) - backporting to buster.
• libgit2 (1.0.0+dfsg.1-1) - new upstream version.
• micro (2.0.3-1) - add support for +LINE:COL flag syntax for cursor position (#953427).

Other $things:

• Attended Ruby team meeting. Logs here.
• Attended Perl team LHF. Report here.
• Sponsored a lot of uploads for William Desportes and Adam Cecile.
• Mentoring for newcomers.
• FTP Trainee reviewing.
• Moderation of -project mailing list.
• Applied for DUCI project for Google Summer of Code 2020.

---

Ruby2.7 Migration:

Ruby2.7 was recently released on 25th December, 2019. Santa s gift. Believe it or not. We, the Debian Ruby team, have been trying hard to make it migrate to testing. And it finally happened. The default version in testing is ruby2.7. Here s the news! \o/
Here s what I worked on this month for this transition.

Upstream: Opened several issues and proposed patches (in the form of PRs):

• Issue #35 against encryptor for Ruby2.7 test failures.
• Issue #28 against image_science for removing relative paths.
• Issue #106 against ffi-yajl for Ruby2.7 test failures.
• PR #5 against aggregate for simply using require.
• PR #6 against aggregate for modernizing CI and adding Ruby 2.5 and 2.7 support.
• Issue #13 against espeak-ruby for Ruby2.7 test failures.
• Issue #4 against tty-which for test failures in general.
• Issue #11 against packable for Ruby2.7 test failures. PR #12 has been proposed.
• Issue #10 against growl for test failures and proposed an initial patch.

Downstream: I fixed and uploaded the following packages in Debian:

• puppet-beaker (4.21.0-1) - new upstream version and fix FTBFS (#956595 and #954614).
• ruby-fakeweb (1.3.0+git20170806+dfsg1-2) - fix autopkgtest (#952042).
• puppet-lint (2.4.2-2) - fix FTBFS for Ruby2.7 migration.
• ruby-hoe (3.22.1+dfsg1-1) - new upstream version and fix FTBFS (#952041).
• rake-compiler (1.0.5-2) - fix FTBFS.
• ruby-aggregate (0.2.2-3) - fix autopkgtest.
• facter (3.11.0-4) - fix autopkgtest (#955582).

---

Debian LTS

Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success.
This was my seventh month as a Debian LTS paid contributor. I was assigned 24.00 hours and worked on the following things:

CVE Fixes and Announcements:

• Issued DLA 2178-1, fixing CVE-2020-11728 and CVE-2020-11729, for awl.
  For Debian 8 Jessie , these problems have been fixed in version 0.55-1+deb8u1.
• Issued DLA 2179-1, fixing CVE-2020-10968, CVE-2020-10969, CVE-2020-11111, CVE-2020-11112, CVE-2020-11113, CVE-2020-11619, and CVE-2020-11620, for jackson-databind.
  For Debian 8 Jessie , these problems have been fixed in version 2.4.2-2+deb8u14.
• Issued DLA 2180-1, fixing CVE-2020-11736, for file-roller.
  For Debian 8 Jessie , this problem has been fixed in version 3.14.1-1+deb8u2.
• Issued DLA 2190-1, fixing CVE-2020-10663, for ruby-json.
  For Debian 8 Jessie , this problem has been fixed in version 1.8.1-1+deb8u1.

Other LTS Work:

• Triaged jackson-databind, libconvert-asn1-perl, file-roller, awl, dom4j, and openvpn.
• Mark CVE-2013-7488/libconvert-asn1-perl as no-dsa for Jessie.
• Mark CVE-2020-11810/openvpn as no-dsa for Jessie.
• Ping ntp s upstream for relevant commits.
• Mark CVE-2019-16782/ruby-rack as no-dsa for Jessie.
• Attended first LTS meeting. Logs here.
• General discussion on LTS mailing list.

---

Other(s)

Sometimes it gets hard to categorize work/things into a particular category.
That s why I am writing all of those things inside this category.
This includes two sub-categories and they are as follows.

Personal: This month I could get the following things done:

• Most importantly, I finally migrated to a new website. Huge UI imporvement! \o/
  From Jekyll to Hugo, it was not easy. But it was worth it! Many thanks to Luiz for writing hugo-coder, Clement, and Samyak!
  If you find any flaws, issues and pull requests are welcomed at utkarsh2102/utkarsh2102.com
• Wrote battery-alert, a mini-project of my own to show battery alerts at <10% and >90%.
  Written in shell, it brings me all the satisfaction as it has saved my life on many occasions.
  And guess what? It has more users than just myself!
  Reviews and patches are welcomed \o/
• Mentored in HackOn Hackathon. Thanks to Manvi for reaching out!
  It was fun to see people developing some really nice projects.
• Thanks to Ray and John, I became a GitLab Hero!
  (I am yet to figure out my role and responibility though)
• Atteneded Intro Sec Con and had the most fun!
  Heard Ian s keynote and attended other talks and learned how to use WireShark!

Open Source: Again, this contains all the things that I couldn t categorize earlier.
Opened several issues and pull requests:

• Issue #297 against hugo-coder, asking to enable RSS feed for blogs.
• PR #316 for hugo-coder for fixing the above issue myself.
• Issue #173 against arbre for requesting a release.
• Issue #104 against combustion, asking to relax dependency on rubocop. Fixed in this commit.
• Issue #16 against ffi-compiler for requesting to fix homepage and license.
• Issue #57 against gographviz for requesting a release.
• Issue #14 against crb-blast, suggesting compatability with bio 2.0.x.
• Issue #58 against uniform_notifier for asking to drop the use of ruby-growl.
• PR #2072 for polybar, adding installation instructions on Debian systems.

---

Until next time.
:wq for today.

Here is my monthly update covering what I have been doing in the free software world during April 2020 (previous month's report). Looking it over prior to publishing, I am surprised how much I got done this month I felt that I was not only failing to do all the extra things I had planned, but I was doing far less than normal. But let us go easy on ourselves; nobody is nailing this.

• Made some small changes to my tickle-me-email library which implements Gettings Things Done (GTD)-like behaviours in IMAP inboxes in order to decode various headers correctly [...] and correct the counting logic in the send-later command's message limit. [...]
• Worked with @dormando with a architecture-specific problem in the Memcached caching system to fix grossly incorrect behaviour on big-endian architectures. [...]
• As part of my duties of being on the board of directors of the Open Source Initiative and Software in the Public Interest I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet, as well as the usual internal discussions regarding logistics, licensing, policy, liaising with the ClearlyDefined project and so on. In particular, I on-boarded the Ganeti project to SPI.
• Reviewed and merged a contribution to my django-cache-toolbox caching library for Django web applications to fix a nested traceback issue. (#20)

• Attended a number of OpenUK's Teabreak Tuesday and Future Leaders' Training events as well did more work as part of being on the judging panel of the OpenUK Awards. Nominations are open until 15th June in five different categories. I also attended a virtual open source 'hallway track' organised by Stormy Peters and plan to attend the next meeting in early May.

In addition, I did more hacking on the Lintian static analysis tool for Debian packages:

• New features and improvements:
  • Check for debian/rules files that specify -Wl,--as-needed as this is now the default from bullseye onwards. (#956146)
  • Warn about automatically-generated debug packages that ship files other than .debug. (#958945)
  • Warn about packages that specify --with=systemd with a Debhelper compatibility level of 10 or higher. (#949844)
  • Detect $* as using the Debhelper sequencer. (#930679)
  • Check for override dh_install (ie. with a space) in debian/rules; in 99% of cases this will be an omission of the underscore. [...]
• Bug fixes:
  • Allow python3-all-dev and python3-all-dbg to satisfy the check for packages that use py3versions with the -s argument. (#955799, #956134)
  • Build-Depends-Arch and Build-Depends-Indep do not imply each other, so don't warn about "duplicate" dependencies in this case. (#956368)
  • Ignore build profiles when checking packages for py3versions -s without the corresponding Build-Depends. (#958794)
  • Remove the pre-depends-directly-on-multiarch-support tag; any package pre-depending on the multiarch-support will not be installable in the bullseye distribution. (#798762)
  • Mark mailing-list-obsolete-in-debian-infrastructure as being experimental. (#958182, #958666)
  • Do not warn about empty dh_dwz-generated "multi-files". (#955752)
  • Don't warn about package-relation-with-self if we have specified a required architecture. (#956227)
• Misc:
  • Move Lintian itself to compatibility level 13 so it does not emit package-uses-old-debhelper-compat-version when run against itself. [...]
  • Drop the .travis.yml file as we are using Salsa now. [...]

Reproducible builds One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

• I submitted 14 patches to fix specific reproducibility issues in gmap, gpick, herbstluftwm, libcamera, libctl, libctl, minetest-mod-xdecor, netgen-lvs, nickle, nmrpflash, node-mqtt, sprai, xxhash & yaz.
• Submitted the following patches to fix reproducibility-related toolchain issues within Debian:
  • dh-cargo: Please make the output reproducible. (#958301)
  • rspamd: Please make paths.lua files reproducible. (#956120)
• Wrote a 20-page funding report to the Open Technology Fund -- whilst the Reproducible Builds project has submitted monthly reports to the otf-active mailing list this final report described in detail the status of each objective, our overall lessons and our future plans.
• The Reproducible Builds project also operates a Jenkins-based testing framework that powers tests.reproducible-builds.org. I made the following changes:
  • Print the build environment prior to executing a build. [...]
  • Drop a misleading disorderfs-debug prefix in log output when we change non-disorderfs things in the file and, as it happens, do not run disorderfs at all. [...]
  • The CSS for the package report pages added a margin to all a HTML elements under li ones, which was causing a comma/bullet spacing issue. [...]
  • Tidy the copy in the project links sidebar. [...]
• Continued collaborative work on an academic paper to be published within the next few months.
• Kept isdebianreproducibleyet.com up to date. [...]
• strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. In April, I made the following changes:
  • Add deprecation plans to all handlers documenting how (or if) they could be disabled and hopefully eventually removed, etc. (#3)
  • Normalise *.sym files as Java archives. (#15)
  • Add support for custom .zip filename filtering and exclude two patterns of files generated by Maven projects in "fork" mode. (#13)
• disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues. This month I fixed a long-standing issue by not drop UNIX groups in FUSE multi-user mode when we are not root. (#1)
• Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
• Drafted, published and publicised our monthly report.

Elsewhere in our tooling, I made the following changes to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, including preparing and uploading versions 139, 140, 141 and 142 to Debian:

• Comparison improvements:
  • Dalvik .dex files can also serve as APK containers so restrict the narrower identification of .dex files to files ending with this extension and widen the identification of APK files to when file(1) discovers a Dalvik file. (#28)
  • Add support for Hierarchical Data Format (HD5) files. (#95)
  • Add support for .p7c and .p7b certificates. (#94)
  • Strip paths from the output of zipinfo(1) warnings. (#97)
  • Don't uselessly include the JSON "similarity" percentage if it is "0.0%". [...]
  • Render multi-line difference comments in a way to show indentation. (#101)
• Testsuite improvements:
  • Add pdftotext as a requirement to run the PDF test_metadata text. (#99)
  • apktool 2.5.0 changed the handling of output of XML schemas so update and restrict the corresponding test to match. (#96)
  • Explicitly list python3-h5py in debian/tests/control.in to ensure that we have this module installed during a test run to generate the fixtures in these tests. [...]
  • Correct parsing of ./setup.py test --pytest-args arguments. [...]
• Misc:
  • Capitalise "Ordering differences only" in text comparison comments. [...]
  • Improve documentation of FILE_TYPE_HEADER_PREFIX and FALLBACK_FILE_TYPE_HEADER_PREFIX to highlight that only the first 16 bytes are used. [...]

Lastly, I made a large number of changes to our website and documentation in the following categories:

• Community engagement improvements:
  • Update instructions to register for Salsa on our Contribute page now that the signup process has been overhauled. [...]
  • Make it clearer that joining the rb-general mailing list is probably a first step for contributors to take. [...]
  • Make our full contact information easier to find in the footer (#19) and improve text layout using bullets to separate sections [...].
• Accessibility:
  • To improve accessibility, make all links underlined. (#12)
  • Use an enhanced foreground/background contrast ratio of 7.04:1. (#11)
• General improvements:
  • Add a new Academic publications page. (#22)
  • Add Trezor to our list of affiliated projects. (#26)
  • Add the JVM page to the documentation index (#17) and tidy the page itself a little [...].
  • Add a GNU Libtool pointer to the Archive metadata documentation page. [...]
• Internals:
  • Move to using jekyll-redirect-from over manual redirect pages [...][...] and add a redirect from /docs/buildinfo/ to /docs/recording/. (#23)
  • Limit the website self-check to not scan generated files [...] and remove the "old layout" checker now that I have migrated all them [...].
  • Move the news archive under the /news/ namespace [...] and improve formatting of archived news links [...].
  • Various improvements to the draft template generation. [...][...][...][...]

Debian LTS This month I have contributed 18 hours to Debian Long Term Support (LTS) and 7 hours on its sister Extended LTS project.

• Frontdesk duties, responding to user/developer questions, reviewing others' packages, participating in mailing list discussions, etc.
• Issued DLA 2171-1 for the Ceph distributed storage and file system to prevent a header-splitting vulnerability (CVE-2020-1760).
• It was discovered that there was a path-traversal issue in the Apache Shiro Java security framework (CVE-2020-1957) where a specially-crafted request could cause an authentication bypass. I therefore issued DLA 2181-1 to address this.
• Issued ELA-224-1 for the ntp Network Time Protocol server to prevent a denial of service vulnerability (CVE-2020-11868).
• Issued ELA-225-1 for dom4j, a library for working with various XML formats on the Java platform to address an XML external external entity vulnerability (CVE-2020-10683). This type of attack occurs when XML input containing a reference to an internet-faced entity is processed by a weakly configured XML parser. This attack may lead to the disclosure of confidential data, denial of service, server side request forgery as well as other system impacts.
• Issued ELA-224-2 to update the fix to ntp in ELA-224-2 to add further protection that was not present in the previous update.
• Investigated and triaged bluez [...], dom4 [...], inetutils [...], openldap [...], otrs2 [...], qemu [...][...][...], samba [...][...], varnish [...], xcftools [...], etc.
• Attended our first regular IRC contributor meeting.

You can find out more about the project via the following video:
Debian I only filed three bugs in April, including one against snapshot.debian.org to report that a Content-Type HTTP header is missing when downloading .deb files (#956471) and to report build failures in the macs & ruby-enumerable-statistics packages:

• Redis:
  • 6.0~rc3-1 New upstream beta release.
  • 6.0~rc4-1 New upstream beta release and use the newly-packaged liblzf-dev package over the local version. (#958321)
  • 5.0.7-3 Fix build failures with GCC 10. (#957751)
  • 5.0.7-4 Use the newly-packaged liblzf-dev package over the bundled version. (#958321)
  • 5.0.7-5 Ensure that the daemon is running prior to running the autopkgtests.
  • 5.0.7-6 Perform a "no change" sourceful upload to permit migration to the testing distribution.
  • 5.0.7-7 Add a sleep to ensure that the server has started before running the tests.
• Django:
  • 2.2.12-1 New upstream release.
  • 3.0.5-1 New upstream release.
• Memcached:
  • 1.6.3-1 New upstream release.
  • 1.6.5-1 New upstream release.
  • 1.6.5-2 Fix a failing autopkgtest that assumes a particular string is in the first kilobyte of the response which was no longer the case due to the addition of new configuration options.
• Redisearch:
  • 1.2.2-1 New upstream release.
  • 1.2.2-2 Ensure that the server is started before running autopkgtests.
  • 1.2.2-3 Sleep to ensure the server has really started before running the autopkgtests.
• OnionShare (2.2-2) Replace python-nautilus in the build-dependencies with python3-nautilus. (#943137)
• installation-birthday (15) Correct the (confusing) internal logic of --force from being visible to users. (#895686)
• txtorcon (20.0.0-1) New upstream release and refresh packaging.
• python-hiredis (1.0.1-1) New upstream release and refresh packaging.
• onioncircuits (0.6-3) Mark a non-deterministic autopkgtest as "flaky" for now to ease migration (#930448) and use DEB_VERSION_UPSTREAM_REVISION over manually parsing dpkg-parsechangelog in debian/rules.

A short while ago Daniel, Lars and I met to discuss Russ Cox s excellent essay Our Software Dependency Problem. This essay looks at software reuse in general, especially in the context of modern distribution methods like PyPI and NPM which make the whole process much more frictionless than traditional distribution methods used with languages like C. Possibly our biggest conclusion was that the essay is so eminently sensible that we mostly just talked about how much we agreed with it and how comprehensive it was, we particularly admired the clarity with which it explores how to evaluate the quality of free software projects. Next time we ll have to pick something more controversial to discuss!

Debian.org added support for DKIM in 2020. To configure this on my side, I had to do the following, on top of my email configuration.

1. add this line to /etc/opendkim/signing.table:

   *@debian.org marcos-debian.anarcat.user
   

2. add this line to /etc/opendkim/key.table:

   marcos-debian.anarcat.user debian.org:marcos-debian.anarcat.user:/etc/opendkim/keys/marcos-debian.anarcat.user.private
   

   Yes, that's quite a mouthful! That magic selector is long in that way because it needs a special syntax (specifically the .anarcat.user suffix) for Debian to be happy. The -debian string is to tell me where the key is published. The marcos prefix is to remind me where the private is used.
3. generate the key with:

   opendkim-genkey --directory=/etc/opendkim/keys/ --selector=marcos-debian.anarcat.user --domain=debian.org --verbose
   

   This creates the DNS record in /etc/opendkim/keys/marcos-debian.anarcat.user.txt (alongside the private key in .key).
4. restart OpenDKIM:

   service opendkim restart
   

   The DNS record will look something like this:

   marcos-debian.anarcat.user._domainkey   IN  TXT ( "v=DKIM1; h=sha256; k=rsa; "
   "p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtKzBK2f8vg5yV307WAOatOhypQt3ANQ95iDaewkVehmx42lZ6b4PzA1k5DkIarxjkk+7m6oSpx5H3egrUSLMirUiMGsIb5XVGBPFmKZhDVmC7F5G1SV7SRqqKZYrXTufRRSne1eEtA31xpMP0B32f6v6lkoIZwS07yQ7DDbwA9MHfyb6MkgAvDwNJ45H4cOcdlCt0AnTSVndcl"
   "pci5/2o/oKD05J9hxFTtlEblrhDXWRQR7pmthN8qg4WaNI4WszbB3Or4eBCxhUdvAt2NF9c9eYLQGf0jfRsbOcjSfeus0e2fpsKW7JMvFzX8+O5pWfSpRpdPatOt80yy0eqpm1uQIDAQAB" )  ; ----- DKIM key marcos-debian.anarcat.user for debian.org
   

5. The "p=MIIB..." string needs to be joined together, without the quotes and the p=, and sent in a signed email to changes@db.debian.org:

   -----BEGIN PGP SIGNED MESSAGE-----
   dkimPubKey: marcos.anarcat.user MIIB[...]
   -----BEGIN PGP SIGNATURE-----
   [...]
   

6. Wait a few minutes for DNS to propagate. You can check if they have with:

   host -t TXT marcos-debian.anarcat.user._domainkey.debian.org nsp.dnsnode.net
   

   (nsp.dnsnode.net being one of the NS records of the debian.org zone.)

If all goes well, the tests should pass when sending from your server as anarcat@debian.org.

Testing Test messages can be sent to dkimvalidator, mail-tester.com or check-auth@verifier.port25.com. Those tools will run Spamassassin on the received emails and report the results. What you are looking for is:

• -0.1 DKIM_VALID: Message has at least one valid DKIM or DK signature
• -0.1 DKIM_VALID_AU: Message has a valid DKIM or DK signature from author's domain
• -0.1 DKIM_VALID_EF: Message has a valid DKIM or DK signature from envelope-from domain

If one of those is missing, then you are doing something wrong and your "spamminess" score will be worse. The latter is especially tricky as it validates the "Envelope From", which is the MAIL FROM: header as sent by the originating MTA, which you see as from=<> in the postfix lost. The following will happen anyways, as soon as you have a signature, that's normal:

• 0.1 DKIM_SIGNED: Message has a DKIM or DK signature, not necessarily valid

And this might happen if you have a ADSP record but do not correctly sign the message with a domain field that matches the record:

• 1.1 DKIM_ADSP_ALL No valid author signature, domain signs all mail

That's bad and will affect your spam core badly. I fixed that issue by using a wildcard key in the key table:

--- a/opendkim/key.table
+++ b/opendkim/key.table
@@ -1 +1 @@
-marcos anarc.at:marcos:/etc/opendkim/keys/marcos.private
+marcos %:marcos:/etc/opendkim/keys/marcos.private

References

• Ubuntu documentation
• Debian wiki
• linode tutorial, also recommends rotating keys every 6 months
• jak-linux: uses rspamd instead of opendkim, and PostSRSd
• Gio's tutorial

This is a copy of a subset of my more complete email configuration.

Welcome to gambaru.de. Here is my monthly report (+ the first week in April) that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. I am sure I am not the only one who will remember March 2020 in the future as a month nobody was really fond of. I was mostly occupied with non-Debian work and managed to get ill in the same week I wanted to celebrate my birthday but it didn t matter anyway because of ehm quarantine and social distancing. Maybe next year March will be great again.
Debian Games

• I was notified by Minetest upstream that Debian did not build the game with CMAKE_BUILD_TYPE=Release or that we simply missed the -DNDEBUG compiler flag which in turn could cause some unexepected runtime errors. I quickly fixed the problem and backported the change to buster-backports. There is another pending merge request about build-depending on libspatialindex-dev. I was told it will speed up some processes on the server side, so I wanted to give it a try.
• I fixed RC bug #954722 in spring, the RTS game engine. A change in GCC caused yet another FTBFS but was rather easy to fix.
• I sponsored jag and runescape for Carlos Donizete Froes. Despite being such a trivial helper script for downloading the runescape launcher, the latter caused some controversy. Now it looks like all problems can be resolved and I expect another upload with bug fixes in April.
• Last but not least I packaged a new upstream release of extremetuxracer, the racing game with Tux for all the family.

Debian Java

• I worked on new releases of wildfly-common, undertow, jboss-threads, jboss-xnio, libsmali-java and apktool.
• I uploaded a security update of checkstyle to Stretch and Buster and prepared another point update for Buster to fix a bug in el-api, websocket-api and jsp-api when libservlet3.1-java was upgraded from Stretch to Buster.
• A missing jar file on the CLASSPATH in commons-configuration2 made mediathekview and other packages FTBFS (#955755) but it also motivated me to remove the unnecessary update check in MediathekView on every startup because it may take a while until I can upgrade this program again.
• I also applied a patch by Bas Couwenberg for OpenJFX to fix a FTBFS bug due to the -Werror=deprecated-declarations flag.

Misc

• While I am still waiting for ublock-origin being processed in the NEW queue, I packaged the latest version of another browser addon, https-everywhere.

Debian LTS This was my 49. month as a paid contributor and I have been paid to work 10 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following:

• I worked on Tomcat 8 in Jessie and prepared patches for CVE-2020-1938, CVE-2020-1935 and CVE-2019-17563. I am working together with Abhihith PA who is currently reviewing them. Especially CVE-2020-1938 requires careful attention because of new options to secure the AJP port and protocol. In contrast to Wheezy, Tomcat in Jessie will be supported at least for another year, so it makes sense to apply the upstream changes for hardening the setup.
• I prepared another Tomcat 8 update for Stretch which will be released this month.

ELTS Extended Long Term Support (ELTS) is a project led by Freexian to further extend the lifetime of Debian releases. It is not an official Debian project but all Debian users benefit from it without cost. The current ELTS release is Debian 7 Wheezy . This was my 22. month and I have been paid to work 9 hours on ELTS.

• I investigated all identified source packages from last month. They are supported but embed external software, sometimes affected by unfixed security vulnerabilities. After a closer inspection I discovered that most packages were either affected only by minor issues, which did not warrant an extra update, or they were not even affected at all because they linked against system libraries. However zlib, apache2 and php5 contained embedded and unfixed code copies of expat and file and zlib s miniunzip program was still prone to a directory traversal attack. I fixed the latter in ELA-222-1. The apache2 update will follow shortly and there is ongoing work for PHP5 anyway which allows us to fix the latest reported vulnerabilites and address the embedded code copy issues together in one update.

Thanks for reading and see you next time.

In a previous job I wrote some simple utilities which were helpful for continuous-integration pipelines. Most of these were very very simple, for example:

• Find all *.json files, and validate them.
• Find all *.yaml files, and validate them.
• Run all the tests in a given directory, execute them one by one.
  • But stop if any single test failed.
  • run-parts does this on Debian GNU/Linux systems, but on CentOS there is no -exit-on-error flag.
  • Which is the sole reason I had to re-implement that tool. Sigh.
• Parse PHP-files and look for comments that were bogus.
  • I mentioned this in the past.

Each of these simple tools were run of the mill shell-scripts, or simple binaries. I've been putting together a simple deployable tool which lets you run them easily, so here it is:

• https://github.com/skx/sysbox/

The idea is that you install the tool, then run the commands by specify the appropriate sub-command:

sysbox validate-yaml [path/to/find/files/beneath]

Of course if you symlink validate-yaml to sysbox you don't need to prefix with the name - so this is just like busybox in this regard. Might be interesting to some.

What is DKIM? DKIM (DomainKeys Identified Mail), as Wikipedia puts it, "is an email authentication method designed to detect forged sender addresses in emails (email spoofing), a technique often used in phishing and email spam". More prosaically, one of the reasons email spam is so abundant is that, given a certain email message, there is no simple way to know for certain who sent it and how reputable they are. So even if people having addresses @debian.org are very nice and well-behaving, any random spammer can easily send emails from whatever@debian.org, and even if you trust people from @debian.org you cannot easily configure your antispam filter to just accept all emails from @debian.org, because spammers would get in too. Since nearly ten years DKIM is there to help you. If you send an email from @debian.org with DKIM, it will have a header like this: The field d=debian.org is the domain this email claims to be from and the fields bh= and b= are a cryptographic public key signature certifying this fact. How do I check that the email is actually from @debian.org? I use the selector s=vps.gio.user to fetch the public key via DNS, and then use the public key to verify the signature. There it is! Debian declares in its DNS record that that key is authorized to sign outbound email from @debian.org. The spammer hopefully does not have access to Debian's DKIM keys, and they cannot sign emails. Many large and small email services have already deployed DKIM since years, while most @debian.org emails still do not use it. Why not? Because people send @debian.org emails from many different servers. Basically, every DD used their @debian.org address sends email from their own mail server, and those mail servers (fortunately) do not have access to Debian's DNS record to install their DKIM keys. Well, that was true until yesterday! :-) A few weeks ago I poked DSA asking to allow any Debian Developer to install their DKIM keys, so that DDs could use DKIM to sign their emails and hopefully reduce the amount of spam sent from @debian.org. They have done it (thank you DSA very much, especially adsb), and now it is possible to use it! How do I configure it? I will not write here a full DKIM tutorial, there are many around. You have to use opendkim-genkey to generate a key and then configure your mail server to use opendkim to digitally sign outbound email. There are a few Debian-specific things you have to care about, though. First the have to choose a selector, which is a string used to distinguish many DKIM keys belonging to the same domain. Debian allows you to installa a key whose selector is <something>.<uid>.user, where <uid> is your Debian uid (this is done both for namespacing reasons and for exposing who might be abusing the system). So check carefully that your selector has this form. Then you cannot edit directly Debian's DNS record. But you can use the email-LDAP gateway on db.debian.org to install your key in a way similar to how entries in debian.net are handled (see the updated documentation). Specifically, suppose that opendkim-genkey generated the following thing for selector vps.gio.user and domain debian.org: Then you have to carefully copy the content of the p= field (without being fooled by it being split between different strings) and construct a request of the form: and then send it GPG-signed to changes@db.debian.org: Then use host -t TXT vps.gio.user._domainkey.debian.org to chech the key gets published (it will probably take some minutes/hours, I don't know). Once it is published, you can enable DKIM in you mail server and your email will be signed. Congratulations, you will not look like a spammer any more! You can send an email to check-auth@verifier.port25.com to check that your setup is correct. They will reply with a report, including the success of DKIM test. Notice that currently Debian's setup only allows you to use RSA DKIM keys and doesn't allow you to set other DKIM fields (but you probably won't need to set them). EDIT DSA made an official announcement about DKIM support, which you might want to check out as well, together with its links. EDIT 2 Now ed25519 keys are supported, the syntax for specifying keys on LDAP is a little bit more flexible and you can also insert CNAME records. See the official documentation for the updated details. So we have solved our problems with spam? Ha, no! DKIM is only a small step. Useful, also because it enable other steps to be taken in the future, but small. In particular, DKIM enables you to say: "This particular email actually comes from @debian.org", but doesn't tell anybody what to do with emails that are not signed. A third-party mail server might wonder whether @debian.org emails are actually supposed to be signed or not. There is another standard for dealing with that, which is called DMARD, and I believe that Debian should eventually use it, but not now: the problem is that currently virtually no email from @debian.org is signed with DKIM, so if DMARC was enabled other mail servers would start to nuke all @debian.org emails, except those which are already signed, a minority. If people and services sending emails from @debian.org will start configuring DKIM on their servers, which is now possible, it will eventually come a time when DMARC can be enabled, and spammers will find themselves unable to send forged @debian.org emails. We are not there yet, but todays we are a little step closer than yesterday. Also, notice that having DKIM on @debian.org only counters spam pretending to be from @debian.org, but there is much more. The policy on what to accept is mostly independent on that on what you send. However, knowing that @debian.org emails have DKIM and DMARC would mean that we can set our spam filters to be more aggressive in general, but whitelist official Debian Developers and services. And the same can be done for other domains using DKIM and DMARC. Finally, notice that some incompatibilities between DKIM and mailing lists are known, and do not have a definitive answer yet. Basically, most mailing list engines modify either the body of the headers in forwarded emails, which means that DKIM does not validate any more. There are many proposed solutions, possibly none completely satisfying, but since spam is not very satisfying as well, something will have to be worked out. I wrote a lot already, though, so I wont't discuss this here.

Over the past few weeks things have been pretty hectic. Since I'm not working at the moment I'm mostly doing childcare instead. I need a break, now and again, so I've been sending our child to p iv koti two days a week with him home the rest of the time. I love taking care of the child, because he's seriously awesome, but it's a hell of a lot of work when most of our usual escapes are unavailable. For example we can't go to the (awesome) Helsinki Central Library as that is closed. Instead of doing things outdoors we've been baking bread together, painting, listening to music and similar. He's a big fan of any music with drums and shouting, so we've been listening to Rammstein, The Prodigy, and as much Queen as I can slip in without him complaining ("more bang bang!"). I've also signed up for some courses at the Helsinki open university, including Devops with Docker so perhaps I have a future career working with computers? I'm hazy. Finally I saw a fun post the other day on reddit asking about the creation of a DSL for server-setup. I wrote a reply which basically said two things:

• First of all you need to define the minimum set of primitives you can execute.
  • (Creating a file, fetching a package, reloading services when a configuration file changes, etc.)
• Then you need to define a syntax for expressing those rules.
  • Not using YAML. Because Ansible fucked up bigtime with that.
  • It needs to be easy to explain, it needs to be consistent, and you need to decide before you begin if you want "toy syntax" or "programming syntax".
  • Because adding on conditionals, loops, and similar, will ruin everything if you add it once you've started with the wrong syntax. Again, see Ansible.

Anyway I had an idea of just expressing things in a simple fashion, borrowing Puppet syntax (which I guess is just Ruby hash literals). So a module to do stuff with files would just look like this:

file   name   => "This is my rule",
       target => "/tmp/blah",
       ensure => "absent"  

The next thing to do is to allow that to notify another rule, when it results in a change. So you add in:

notify => "Name of rule"
# or
notify => [ "Name of rule", "Name of another rule" ]

You could also express dependencies the other way round:

shell   name => "Do stuff",
        command => "wc -l /etc/passwd > /tmp/foo",
        requires => [ "Rule 1", "Rule 2"]  

Anyway the end result is a simple syntax which allows you to do things; I wrote a file to allow me to take a clean system and configure it to run a simple golang application in an hour or so. The downside? Well the obvious one is that there's no support for setting up cron jobs, setting up docker images, MySQL usernames/passwords, etc. Just a core set of primitives. Adding new things is easy, but also an endless job. So I added the ability to run external/binary plugins stored outside the project. To support that is simple with the syntax we have:

• We pass the parameters, as JSON, to STDIN of the binary.
• We read the result from STDOUT
  • Did the rule result in a change to the system?
  • Or was it a NOP?

All good. People can write modules, if they like, and they can do that in any language they like. Fun times. We'll call it marionette since it's all puppet-inspired:

• https://github.com/skx/marionette

And that concludes this irregular update.

I just had a weekend full of very successful serious geekery. On a whim I thought: Wouldn't it be nice if people could interact with my game Kaleidogen also via a telegram bot? This led me to learn about how I write a Telegram bot in Haskell and how I can deploy such a Haskell program to Amazon Lambda. In particular the latter bit might be interesting to some of my readers, so here is how went about it.

Kaleidogen Kaleidogen is a little contemplative game (or toy game where, starting from just unicolored disks, you combine abstract circular patterns to breed more interesting patterns. See my FARM 2019 talk for more details, or check out the source repository. BTW, I am looking for help turning it into an Android app!

KaleidogenBot in action

Amazon Lambda Amazon Lambda is the Function as a service offering of Amazon Web Services. The idea is that you don t rent a server, where you have to deal with managing the whole system and that you are paying for constantly, but you just upload the code that responds to outside requests, and AWS takes care of the rest: Starting and stopping instances, providing a secure base system etc. When nobody is using the service, no cost occurs. This sounds ideal for hosting a toy Telegram bot: Most of the time nobody will be using it, and I really don't want to have to baby sit yet another service on my server. On Amazon Lambda, I can probably just forget about it. But Haskell is not one of the officially supported languages on Amazon Lambda. So to run Haskell on Lambda, one has to solve two problems:

• how to invoke the Haskell code on the server, and
• how to build Haskell so that it runs on the Amazon Linux distribution

A Haskell runtime for Lambda For the first we need a custom runtime. While this sounds complicated, it is actually a pretty simple concept: A runtime is an executable called bootstrap that queries the Lambda Runtime Interface for the next request to handle. The Lambda documentation is phrased as if this runtime has to be a dispatcher that calls the separate function s handler. But it could just do all the things directly. I found the Haskell package aws-lambda-haskell-runtime which provides precisely that: A function

runLambda :: (LambdaOptions -> IO (Either String LambdaResult)) -> IO ()

that talks to the Lambda Runtime API and invokes its argument on each message. The package also provides Template Haskell magic to collect handlers of any JSON-able type and generates a dispatcher, like you might expect from other, more dynamic languages. But that was too much magic for me, so I ignored that and just wrote the handler manually:

main :: IO ()
main = runLambda run
  where
   run ::  LambdaOptions -> IO (Either String LambdaResult)
   run opts = do
    result <- handler (decodeObj (eventObject opts)) (decodeObj (contextObject opts))
    either (pure . Left . encodeObj) (pure . Right . LambdaResult . encodeObj) result
data Event = Event
    path :: T.Text
  , body :: Maybe T.Text
    deriving (Generic, FromJSON)
data Response = Response
    statusCode :: Int
  , headers :: Value
  , body :: T.Text
  , isBase64Encoded :: Bool
    deriving (Generic, ToJSON)
handler :: Event -> Context -> IO (Either String Response)
handler Event body, path  context =
 

I expose my Lambda function to the world via Amazon s API Gateway, configured to just proxy the HTTP requests. This means that my code receives a JSON data structure describing the HTTP request (here called Event, listing only the fields I care about), and it will respond with a Response, again as JSON. The handler can then simply pattern-match on the path to decide what to do. For example this code handles URLs like /img/CAFFEEFACE.png, and responds with an image.

handler :: TC -> Event -> Context -> IO (Either String Response)
handler Event body, path  context
      Just bytes <- isImgPath path >>= T.decodeHex = do
        let pngData = genPurePNG bytes
        pure $ Right Response
              statusCode = 200
            , headers = object [ "Content-Type" .= ("image/png" :: String) ]
            , isBase64Encoded = True
            , body = T.decodeUtf8 $ LBS.toStrict $ Base64.encode pngData
             
     
isImgPath :: T.Text -> Maybe T.Text
isImgPath  = T.stripPrefix "/img/" >=> T.stripSuffix ".png"

If this program would grow more, then one should probably use something more structured for routing here; maybe servant, or bridging towards wai apps (amost like wai-lamda, but that still assumes an existing runtime, instead of simply being the runtime). But for my purposes, no extra layers of indirection or abstraction are needed!

Deploying Haskell to Lambda Building Haskell locally and deploying to different machiens is notoriously tricky; you often end up depending on a shared library that is not available on the other platform. The aws-lambda-haskell-runtime package, and similar projects like serverless-haskell, solve this using stack and Docker two technologies that are probably great, but I never warmed up to them. So instead adding layers and complexities, can I solve this instead my making things simpler? If I compiler my bootstrap into a static Linux binary, it should run on any Linux, including Amazon Linux. Unfortunately, building Haskell programs statically is also notoriously tricky. But it is made much simpler by the work of Niklas Hamb chen and others in the context of the Nix package manager, coordinated in the static-haskell-nix project. The promise here is that once you have set up building your project with Nix, then getting a static version is just one flag away. The support is not completely upstreamed into nixpkgs proper yet, but their repository has a nix file that contains a nixpkgs set with their patches:

let pkgs = (import (sources.nixpkgs-static + "/survey/default.nix")  ).pkgs; in

This, plus a fairly standard nix setup to build the package, yields what I was hoping for:

$ nix-build -A kaleidogen
/nix/store/ppwyq4d964ahd6k56wsklh93vzw07ln0-kaleidogen-0.1.0.0
$ file result/bin/kaleidogen-amazon-lambda
result/bin/kaleidogen-amazon-lambda: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped
$ ls -sh result/bin/kaleidogen-amazon-lambda
6,7M result/bin/kaleidogen-amazon-lambda

If we put this file, named bootstrap, into a zip file and upload it to Amazon Lambda, then it just works! Creating the zip file is easily scripted using nix:

  function-zip = pkgs.runCommandNoCC "kaleidogen-lambda"  
    buildInputs = [ pkgs.zip ];
    ''
    mkdir -p $out
    cp $ kaleidogen /bin/kaleidogen-amazon-lambda bootstrap
    zip $out/function.zip bootstrap
  '';

So to upload this, I use this one-liner (line-wrapped for your convenience):

nix-build -A function-zip &&
aws lambda update-function-code --function-name kaleidogen \
  --zip-file fileb://result/function.zip

Thanks to how Nix pins all dependencies, I am fairly confident that I can return to this project in 4 months and still be able to build it. Of course, I want continuous integration and deployment. So I build the project with GitHub Actions, using a cachix nix cache to significantly speed up the build, and auto-deploy to Lambda using aws-lambda-deploy; see my workflow file for details.

The Telegram part The above allows me to run basically any stateless service, and a Telegram bot is nothing else: When configured to act as a WebHook, Telegram will send a request with a message to our Lambda function, where we can react on it. The telegram-api package provides bindigs for the Telegram Bot API (although I had to use the repository version, as the version on Hackage has some bitrot). Slightly simplified I can write a handler for an Update:

handleUpdate :: Update -> TelegramClient ()
handleUpdate Update  message = Just m   = do
  let c = ChatId (chat_id (chat m))
  liftIO $ printf "message from %s: %s\n" (maybe "?" user_first_name (from m)) (maybe "" T.unpack (text m))
  if "/start"  T.isPrefixOf  fromMaybe "" (text m)
  then do
    rm <- sendMessageM $ sendMessageRequest c "Hi! I am @KaleidogenBot.  "
    return ()
  else do
    m1 <- sendMessageM $ sendMessageRequest c "One moment "
    withPNGFile  $ \pngFN -> do
      m2 <- uploadPhotoM $ uploadPhotoRequest c
        (FileUpload (Just "image/png") (FileUploadFile pngFN))
      return ()
handleUpdate _ u =
  liftIO $ putStrLn $ "Unhandled message: " ++ show u

and call this from the handler that I wrote above:

     
      path == "/telegram" =
      case eitherDecode (LBS.fromStrict (T.encodeUtf8 (fromMaybe "" body))) of
        Left err ->  
        Right update -> do
          runTelegramClient token manager $ handleUpdate Nothing update
          pure $ Right Response
              statusCode = 200
            , headers = object [ "Content-Type" .= ("text/plain" :: String) ]
            , isBase64Encoded = False
            , body = "Done"
             
     

Note that the Lambda code receives the request as JSON data structure with a body that contains the original HTTP request body. Which, in this case, is itself JSON, so we have to decode that. All that is left to do is to tell Telegram where this code lives:

curl --request POST \
  --url https://api.telegram.org/bot<token>/setWebhook
  --header 'content-type: application/json'
  --data ' "url": "https://api.kaleidogen.nomeata.de/telegram" '

As a little add on, I also created a Telegram game for Kaleidogen. A Telegram game is nothing but a webpage that runs inside Telegram, so it wasn t much work to wrap the Web version of Kaleidogen that way, but the resulting Telegram game (which you can access via https://core.telegram.org/bots/games) still looks pretty neat.

No /dev/dri/renderD128 I am mostly happy with this setup: My game is now available to more people in more ways. I don t have to maintain any infrastructure. When nobody is using this bot no resources are wasted, and the costs of the service are neglectible -- this is unlikely to go beyond the free tier, and even if it would, the cost per generated image is roughly USD 0.000021. There is one slight disappointment, though. What I find most intersting about Kaleidogen from a technical point of view is that when you play it in the browser, the images are not generated by my code. Instead, my code creates a WebGL shader program on the fly, and that program generates the image on your graphics card. I even managed to make the GL rendering code work headlessly, i.e. from a command line program, using EGL and libgbm and a helper written in C. But it needs access to a graphics card via /dev/dri/renderD128. Amazon does not provide that to Lambda code, and neither do the other big Function-as-a-service providers. So I had to swallow my pride and reimplement the rendering in pure Haskell. So if you think the bot is kinda slow, then that s why. Despite properly optimizing the pure implementation (the inner loop does not do allocations and deals only with unboxed Double# values), the GL shader version is still three times as fast. Maybe in a few years GPU access will be so ubiquitous that it s even on Amazon Lambda; then I can easily use that.

FTP master This month I accepted 156 packages and rejected 26. The overall number of packages that got accepted was 203. Debian LTS This was my sixty ninth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 30h. During that time I did LTS uploads of:

• [DLA 2156-1] e2fsprogs security update for one CVE
• [DLA 2157-1] weechat security update for three CVEs
• [DLA 2160-1] php5 security update for two CVEs
• [DLA 2164-1] gst-plugins-bad0.10 security update for four CVEs
• [DLA 2165-1] apng2gif security update for one CVE

Also my work on graphicsmagic was accepted which resulted in:

• [DSA 4640-1] graphicsmagick security update in Buster and Strech for 16 CVEs

Further I sent debdiffs of weechat/stretch, weechat/buster, e2fsprogs/stretch to the corresponding maintainers but got no feedback yet. As there have been lots of no-dsa-CVEs accumulated for wireshark, I started to work on them but could not upload yet. Last but not least I did some days of frontdesk duties. Debian ELTS This month was the twenty first ELTS month. During my really allocated time I uploaded:

• ELA-218-1 for e2fsprogs
• ELA-220-1 for php5
• ELA-221-1 for nss

I also did some days of frontdesk duties. Other stuff Unfortunately this month again strange things happened outside Debian and the discussions within Debian did not stop. Nonetheless I got some stuff done. I improved packaging of

• duktape
• gengetopt and also fixed an FTBFS
• pollen

I sponsored uploads of

• ocf-spec-core
• theme-d-gnome

Sorry to all people who also requested sponsoring, but sometimes things happen and your upload might be delayed. I uploaded new upstream versions of

• libctl
• libosmocore
• mpb
• pyicloud
• pywws

On my Go challenge I uploaded:
golang-github-dreamitgetit-statuscake, golang-github-ensighten-udnssdk, golang-github-apparentlymart-go-dump, golang-github-suapapa-go-eddystone, golang-github-joyent-gosdc, golang-github-nrdcg-goinwx, golang-github-bmatcuk-doublestar, golang-github-go-xorm-core, golang-github-svanharmelen-jsonapi, golang-github-goji-httpauth, golang-github-phpdave11-gofpdi

A new (upstream) simdjson release was announced by Daniel Lemire earlier this week, and my Twitter mentions have been running red-hot ever since as he was kind enough to tag me. Do look at that blog post, there is some impressive work in there. We wrapped up the (still very simple) rcppsimdjson around it last night and shipped it this morning. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire. Via some very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. For illustration, I highly recommend the video of the recent talk by Daniel Lemire at QCon (which was also voted best talk). The best-case performance is faster than CPU speed as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle use per byte parsed. This release brings upstream 0.3 (and 0.3.1) plus a minor tweak (also shipped back upstream). Our full NEWS entry follows.

Changes in version 0.0.4 (2020-04-03)

• Upgraded to new upstream releases 0.3 and 0.3.1 (Dirk in #9 closing #8)
• Updated example validateJSON to API changes.

But because Daniel is such a fantastic upstream developer to collaborate with, he even filed a full feature-request maybe you can consider upgrading as issue #8 at our repo containing the fully detailed list of changes. As it is so impressive I will simple quote the upper half of just the major changes:

Highlights

• Multi-Document Parsing: Read a bundle of JSON documents (ndjson) 2-4x faster than doing it individually. API docs / Design Details
• Simplified API: The API has been completely revamped for ease of use, including a new JSON navigation API and fluent support for error code and exception styles of error handling with a single API. Docs
• Exact Float Parsing: Now simdjson parses floats flawlessly without any performance loss (https://github.com/simdjson/simdjson/pull/558). Blog Post
• Even Faster: The fastest parser got faster! With a shiny new UTF-8 validator and meticulously refactored SIMD core, simdjson 0.3 is 15% faster than before, running at 2.5 GB/s (where 0.2 ran at 2.2 GB/s).

For questions, suggestions, or issues please use the issue tracker at the GitHub repo. Courtesy of CRANberries, there is also a diffstat report for this release. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

A new release 0.4.17 of RProtoBuf is now on CRAN. RProtoBuf provides R with bindings for the Google Protocol Buffers ( ProtoBuf ) data encoding and serialization library used and released by Google, and deployed very widely in numerous projects as a language and operating-system agnostic protocol. This release contains small polishes related to the release 0.4.16 which added JSON support for messages, and switched to ByteSizeLong. This release now makes sure JSON functionality is only tested where available (on version 3 of the Protocol Buffers library), and that ByteSizeLong is only called where available (version 3.6.0 or later). Of course, older versions build as before and remain fully supported.

Changes in RProtoBuf version 0.4.17 (2020-03-xx)

• Condition use of ByteSizeLong() on building with ProtoBuf 3.6.0 or later (Dirk in #71 fixing #70).
• The JSON unit tests are skipped if ProtoBuf 2.* is used (Dirk, also #71).
• The configure script now extracts the version from the DESCRIPTION file ( (Dirk, also #71).

CRANberries provides the usual diff to the previous release. The RProtoBuf page has copies of the (older) package vignette, the quick overview vignette, and the pre-print of our JSS paper. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

A new release 0.4.16 of RProtoBuf is now on CRAN. RProtoBuf provides R with bindings for the Google Protocol Buffers ( ProtoBuf ) data encoding and serialization library used and released by Google, and deployed very widely in numerous projects as a language and operating-system agnostic protocol. This release contains a PR contributed by Siddhartha Bagaria which adds JSON support for messages, which had been an open wishlist item. I also appeased a clang deprecation warning that had come up on one of the CRAN test machines.

Changes in RProtoBuf version 0.4.16 (2020-03-19)

• Added support for parsing and printing JSON (Siddhartha Bagaria in #68 closing wishlist #61).
• Switched ByteSize() to ByteSizeLong() to appease clang (Dirk).

CRANberries provides the usual diff to the previous release. The RProtoBuf page has copies of the (older) package vignette, the quick overview vignette, and the pre-print of our JSS paper. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Join me in the rabbit hole of git repository verification, and how we could improve it.

Problem statement As part of my work on automating install procedures at Tor, I ended up doing things like:

git clone REPO
./REPO/bootstrap.sh

... something eerily similar to the infamous curl pipe bash method which I often decry. As a short-term workaround, I relied on the SHA-1 checksum of the repository to make sure I have the right code, by running this both on a "trusted" (ie. "local") repository and the remote, then visually comparing the output:

$ git show-ref master
9f9a9d70dd1f1e84dec69a12ebc536c1f05aed1c refs/heads/master

One problem with this approach is that SHA-1 is now considered as flawed as MD5 so it can't be used as an authentication mechanism anymore. It's also fundamentally difficult to compare hashes for humans. The other flaw with comparing local and remote checksums is that we assume we trust the local repository. But how can I trust that repository? I can either:

1. audit all the code present and all the changes done to it after
2. or trust someone else to do so

The first option here is not practical in most cases. In this specific use case, I have audited the source code -- I'm the author, even -- what I need is to transfer that code over to another server. (Note that I am replacing those procedures with Fabric, which makes this use case moot for now as the trust path narrows to "trust the SSH server" which I already had anyways. But it's still important for my fellow Tor developers who worry about trusting the git server, especially now that we're moving to GitLab.) But anyways, in most cases, I do need to trust some other fellow developer I collaborate with. To do this, I would need to trust the entire chain between me and them:

1. the git client
2. the operating system
3. the hardware
4. the network (HTTPS and the CA cartel, specifically)
5. then the hosting provider (and that hardware/software stack)
6. and then backwards all the way back to that other person's computer

I want to shorten that chain as much as possible, make it "peer to peer", so to speak. Concretely, it would eliminate the hosting provider and the network, as attackers.

OpenPGP verification My first reaction is (perhaps perversely) to "use OpenPGP" for this. I figured that if I sign every commit, then I can just check the latest commit and see if the signature is good. The first problem here is that this is surprisingly hard. Let's pick some arbitrary commit I did recently:

commit b3c538898b0ed4e31da27fc9ca22cb55e1de0000
Author: Antoine Beaupr  <anarcat@debian.org>
Date:   Mon Mar 16 14:37:28 2020 -0400
    fix test autoloading
    pytest only looks for file names matching  test  by default. We inline
    tests inside the source code directly, so hijack that.
diff --git a/fabric_tpa/pytest.ini b/fabric_tpa/pytest.ini
new file mode 100644
index 0000000..71004ea
--- /dev/null
+++ b/fabric_tpa/pytest.ini
@@ -0,0 +1,3 @@
+[pytest]
+# we inline tests directly in the source code
+python_files = *.py

That's the output of git log -p in my local repository. I signed that commit, yet git log is not telling me anything special. To check the signature, I need something special: --show-signature, which looks like this:

commit b3c538898b0ed4e31da27fc9ca22cb55e1de0000
gpg: Signature faite le lun 16 mar 2020 14:37:53 EDT
gpg:                avec la clef RSA 7B164204D096723B019635AB3EA1DDDDB261D97B
gpg: Bonne signature de  Antoine Beaupr  <anarcat@orangeseeds.org>  [ultime]
gpg:                 alias  Antoine Beaupr  <anarcat@torproject.org>  [ultime]
gpg:                 alias  Antoine Beaupr  <anarcat@anarc.at>  [ultime]
gpg:                 alias  Antoine Beaupr  <anarcat@koumbit.org>  [ultime]
gpg:                 alias  Antoine Beaupr  <anarcat@debian.org>  [ultime]
Author: Antoine Beaupr  <anarcat@debian.org>
Date:   Mon Mar 16 14:37:28 2020 -0400
    fix test autoloading
    pytest only looks for file names matching  test  by default. We inline
    tests inside the source code directly, so hijack that.

Can you tell if this is a valid signature? If you speak a little french, maybe you can! But even if you would, you are unlikely to see that output on your own computer. What you would see instead is:

commit b3c538898b0ed4e31da27fc9ca22cb55e1de0000
gpg: Signature made Mon Mar 16 14:37:53 2020 EDT
gpg:                using RSA key 7B164204D096723B019635AB3EA1DDDDB261D97B
gpg: Can't check signature: No public key
Author: Antoine Beaupr  <anarcat@debian.org>
Date:   Mon Mar 16 14:37:28 2020 -0400
    fix test autoloading
    pytest only looks for file names matching  test  by default. We inline
    tests inside the source code directly, so hijack that.

Important part: Can't check signature: No public key. No public key. Because of course you would see that. Why would you have my key lying around, unless you're me. Or, to put it another way, why would that server I'm installing from scratch have a copy of my OpenPGP certificate? Because I'm a Debian developer, my key is actually part of the 800 keys in the debian-keyring package, signed by the APT repositories. So I have a trust path. But that won't work for someone who is not a Debian developer. It will also stop working when my key expires in that repository, as it already has on Debian buster (current stable). So I can't assume I have a trust path there either. One could work with a trusted keyring like we do in the Tor and Debian project, and only work inside that project, that said. But I still feel uncomfortable with those commands. Both git log and git show will happily succeed (return code 0 in the shell) even though the signature verification failed on the commits. Same with git pull and git merge, which will happily push your branch ahead even if the remote has unsigned or badly signed commits. To actually verify commits (or tags), you need the git verify-commit (or git verify-tag) command, which seems to do the right thing:

$ LANG=C.UTF-8 git verify-commit b3c538898b0ed4e31da27fc9ca22cb55e1de0000
gpg: Signature made Mon Mar 16 14:37:53 2020 EDT
gpg:                using RSA key 7B164204D096723B019635AB3EA1DDDDB261D97B
gpg: Can't check signature: No public key
[1]$

At least it fails with some error code (1, above). But it's not flexible: I can't use it to verify that a "trusted" developer (say one that is in a trusted keyring) signed a given commit. Also, it is not clear what a failure means. Is a signature by an expired certificate okay? What if the key is signed by some random key in my personal keyring? Why should that be trusted?

Worrying about git and GnuPG In general, I'm worried about git's implementation of OpenPGP signatures. There has been numerous cases of interoperability problems with GnuPG specifically that led to security, like EFAIL or SigSpoof. It would be surprising if such a vulnerability did not exist in git. Even if git did everything "just right" (which I have myself found impossible to do when writing code that talks with GnuPG), what does it actually verify? The commit's SHA-1 checksum? The tree's checksum? The entire archive as a zip file? I would bet it signs the commit's SHA-1 sum, but I just don't know, on the top of my head, and neither do git-commit or git-verify-commit say exactly what is happening. I had an interesting conversation with a fellow Debian developer (dkg) about this and we had to admit those limitations:

<anarcat> i'd like to integrate pgp signing into tor's coding practices more, but so far, my approach has been "sign commits" and the verify step was "TBD" <dkg> that's the main reason i've been reluctant to sign git commits. i haven't heard anyone offer a better subsequent step. if torproject could outline something useful, then i'd be less averse to the practice. i'm also pretty sad that git remains stuck on sha1, esp. given the recent demonstrations. all the fancy strong signatures you can make in git won't matter if the underlying git repo gets changed out from under the signature due to sha1's weakness

In other words, even if git implements the arcane GnuPG dialect just so, and would allow us to setup the trust chain just right, and would give us meaningful and workable error messages, it still would fail because it's still stuck in SHA-1. There is work underway to fix that, but in February 2020, Jonathan Corbet described that work as being in a "relatively unstable state", which is hardly something I would like to trust to verify code. Also, when you clone a fresh new repository, you might get an entirely different repository, with a different root and set of commits. The concept of "validity" of a commit, in itself, is hard to establish in this case, because an hostile server could put you backwards in time, on a different branch, or even on an entirely different repository. Git will warn you about a different repository root with warning: no common commits but that's easy to miss. And complete branch switches, rebases and resets from upstream are hardly more noticeable: only a tiny plus sign (+) instead of a star (*) will tell you that a reset happened, along with a warning (forced update) on the same line. Miss those and your git history can be compromised.

Possible ways forward I don't consider the current implementation of OpenPGP signatures in git to be sufficient. Maybe, eventually, it will mature away from SHA-1 and the interface will be more reasonable, but I don't see that happening in the short term. So what do we do?

git evtag The git-evtag extension is a replacement for git tag -s. It's not designed to sign commits (it only verifies tags) but at least it uses a stronger algorithm (SHA-512) to checksum the tree, and will include everything in that tree, including blobs. If that sounds expensive to you, don't worry too much: it takes about 5 seconds to tag the Linux kernel, according to the author. Unfortunately, that checksum is then signed with GnuPG, in a manner similar to git itself, in that it exposes GnuPG output (which can be confusing) and is likely similarly vulnerable to mis-implementation of the GnuPG dialect as git itself. It also does not allow you to specify a keyring to verify against, so you need to trust GnuPG to make sense of the garbage that lives in your personal keyring (and, trust me, it doesn't). And besides, git-evtag is fundamentally the same as signed git tags: checksum everything and sign with GnuPG. The difference is it uses SHA-512 instead of SHA-1, but that's something git will eventually fix itself anyways.

kernel patch attestations The kernel also faces this problem. Linus Torvalds signs the releases with GnuPG, but patches fly all over mailing list without any form of verification apart from clear-text email. So Konstantin Ryabitsev has proposed a new protocol to sign git patches which uses SHA256 to checksum the patch metadata, commit message and the patch itself, and then sign that with GnuPG. It's unclear to me what this solves, if anything, at all. As dkg argues, it would seem better to add OpenPGP support to git-send-email and teach git tools to recognize that (e.g. git-am) at least if you're going to keep using OpenPGP anyways. And furthermore, it doesn't resolve the problems associated with verifying a full archive either, as it only attests "patches".

jcat Unhappy with the current state of affairs, the author of fwupd (Richard Hughes) wrote his own protocol as well, called jcat, which provides signed "catalog files" similar to the ones provided in Microsoft windows. It consists of a "gzip-compressed JSON catalog files, which can be used to store GPG, PKCS-7 and SHA-256 checksums for each file". So yes, it is yet again another wrapper to GnuPG, probably with all the flaws detailed above, on top of being a niche implementation, disconnected from git.

The Update Framework One more thing dkg correctly identified is:

<dkg> anarcat: even if you could do exactly what you describe, there are still some interesting wrinkles that i think would be problems for you. the big one: "git repo's latest commits" is a loophole big enough to drive a truck through. if your adversary controls that repo, then they get to decide which commits to include in the repo. (since every git repo is a view into the same git repo, just some have more commits than others)

In other words, unless you have a repository that has frequent commits (either because of activity or by a bot generating fake commits), you have to rely on the central server to decide what "the latest version" is. This is the kind of problems that binary package distribution systems like APT and TUF solve correctly. Unfortunately, those don't apply to source code distribution, at least not in git form: TUF only deals with "repositories" and binary packages, and APT only deals with binary packages and source tarballs. That said, there's actually no reason why git could not support the TUF specification. Maybe TUF could be the solution to ensure end-to-end cryptographic integrity of the source code itself. OpenPGP-signed tarballs are nice, and signed git tags can be useful, but from my experience, a lot of OpenPGP (or, more accurately, GnuPG) derived tools are brittle and do not offer clear guarantees, and definitely not to the level that TUF tries to address. This would require changes on the git servers and clients, but I think it would be worth it.

Other Projects

OpenBSD There are other tools trying to do parts of what GnuPG is doing, for example minisign and OpenBSD's signify. But they do not integrate with git at all right now. Although I did find a hack] to use signify with git, it's kind of gross...

Golang Unsurprisingly, this is a problem everyone is trying to solve. Golang is planning on hosting a notary which would leverage a "certificate-transparency-style tamper-proof log" which would be ran by Google (see the spec for details). But that doesn't resolve the "evil server" attack, if we treat Google as an adversary (and we should).

Python Python had OpenPGP going for a while on PyPI, but it's unclear if it ever did anything at all. Now the plan seems to be to use TUF but my hunch is that the complexity of the specification is keeping that from moving ahead.

Docker Docker and the container ecosystem has, in theory, moved to TUF in the form of Notary, "a project that allows anyone to have trust over arbitrary collections of data". In practice however, in my somewhat limited experience, setting up TUF and image verification in Docker is far from trivial.

Android and iOS Even in what is possibly one of the strongest models (at least in terms of user friendliness), mobile phones are surprisingly unclear about those kind of questions. I had to ask if Android had end-to-end authentication and I am still not clear on the answer. I have no idea of what iOS does.

Conclusion One of the core problems with everything here is the common usability aspect of cryptography, and specifically the usability of verification procedures. We have become pretty good at encryption. The harder part (and a requirement for proper encryption) is verification. It seems that problem still remains unsolved, in terms of usability. Even Signal, widely considered to be a success in terms of adoption and usability, doesn't properly solve that problem, as users regularly ignore "The security number has changed" warnings... So, even though they deserve a lot of credit in other areas, it seems unlikely that hardcore C hackers (e.g. git and kernel developers) will be able to resolve that problem without at least a little bit of help. And TUF seems like the state of the art specification around here, it would seem wise to start adopting it in the git community as well. Update: git 2.26 introduced a new gpg.minTrustLevel to "tell various signature verification codepaths the required minimum trust level", presumably to control how Git will treat keys in your keyrings, assuming the "trust database" is valid and up to date. For an interesting narrative of how "normal" (without PGP) git verification can fail, see also A Git Horror Story: Repository Integrity With Signed Commits.

This year s FOSDEM conference was a lot of fun one of the things I always enjoy most about this particular conference (besides having some of the outstanding food you can get in Brussels and meeting with friends from the free software world) is the ability to meet a large range of new people who I wouldn t usually have interacted with, or getting people from different communities together who otherwise would not meet in person as each bigger project has their own conference (for example, the amount of VideoLAN people is much lower at GUADEC and Akademy compared to FOSDEM). It s also really neat to have GNOME and KDE developers within reach at the same place, as I care about both desktops a lot. An unexpected issue This blog post however is not about that. It s about what I learned when talking to people there about AppStream, and the outcome of that. Especially when talking to application authors but also to people who deal with larger software repositories, it became apparent that many app authors don t really want to deal with the extra effort of writing metadata at all. This was a bit of a surprise to me, as I thought that there would be a strong interest for application authors to make their apps look as good as possible in software catalogs. A bit less surprising was the fact that people apparently don t enjoy reading a large specification, reading a long-ish intro guide with lots of dos and don ts or basically reading any longer text at all before being able to create an AppStream MetaInfo/AppData file describing their software. Another common problem seems to be that people don t immediately know what a reverse-DNS ID is, the format AppStream uses for uniquely identifying each software component. So naturally, people either have to read about it again (bah, reading! ) or make something up, which occasionally is wrong and not the actual component-ID their software component should have. The MetaInfo Creator It was actually suggested to me twice that what people really would like to have is a simple tool to put together a MetaInfo file for their software. Basically a simple form with a few questions which produces the final file. I always considered this a nice to have, but not essential feature, but now I was convinced that this actually has a priority attached to it. So, instead of jumping into my favourite editor and writing a bunch of C code to create this make MetaInfo file form as part of appstreamcli, this time I decided to try what the cool kids are doing and make a web application that runs in your browser and creates all metadata there. So, behold the MetaInfo Creator! If you click this link, you will end up at an Angular-based web application that will let you generate MetaInfo/AppData files for a few component-types simply by answering a set of questions. The intent was to make this tool as easy to use as possible for someone who basically doesn t know anything about AppStream at all. Therefore, the tool will:

• Generate a rDNS component-ID suggestion automatically based on the software s homepage and name
• Fill out default values for anything it thinks it has enough data for
• Show short hints for what values we expect for certain fields
• Interactively validate the entered value, so people know immediately when they have entered something invalid
• Produce a .desktop file as well for GUI applications, if people select the option for it
• Show additional hints about how to do more with the metadata
• Create some Meson snippets as pointers how people can integrate the MetaInfo files into projects using the Meson build system

For the Meson feature, the tool simply can not generate a use this and be done script, as each Meson snippet needs to be adjusted for the individual project. So this option is disabled by default, but when enabled, a few simple Meson snippets will be produced which can be easily adjusted to the project they should be part of. The tool currently does not generate any release information for a MetaInfo file at all, This may be added in future. The initial goal was to have people create any MetaInfo file in the first place, having projects also ship release details would be the icing on the cake. I hope people find this project useful and use it to create better MetaInfo files, so distribution repositories and Flatpak repos look better in software centers. Also, since MetaInfo files can be used to create an inventory of software and to install missing stuff as-needed, having more of them will help to build smarter software managers, create smaller OS base installations and introspect what software bundles are made of easily. I welcome contributions to the MetaInfo Creator! You can find its source code on GitHub. This is my first web application ever, the first time I wrote TypeScript and the first time I used Angular, so I d bet a veteran developer more familiar with these tools will cringe at what I produced. So, scratch that itch and submit a PR! Also, if you want to create a form for a new component type, please submit a patch as well. C developer s experience notes for Angular, TypeScript, NodeJS This section is just to ramble a bit about random things I found interesting as a developer who mostly works with C/C++ and Python and stepped into the web-application developer s world for the first time. For a project like this, I would usually have gone with my default way of developing something for the web: Creating a Flask-based application in Python. I really love Python and Flask, but of course using them would have meant that all processing would have had to be done on the server. One the one hand I could have used libappstream that way to create the XML, format it and validate it, but on the other hand I would have had to host the Python app on my own server, find a place at Purism/Debian/GNOME/KDE or get it housed at Freedesktop somehow (which would have taken a while to arrange) and I really wanted to have a permanent location for this application immediately. Additionally, I didn t want people to send the details of new unpublished software to my server. TypeScript I must say that I really like TypeScript as a language compared to JavaScript. It is not really revolutionary (I looked into Dart and other ways to compile $stuff to JavaScript first), but it removes just enough JavaScript weirdness to be pleasant to use. At the same time, since TS is a superset of JS, JavaScript code is valid TypeScript code, so you can integrate with existing JS code easily. Picking TS up took me much less than an hour, and most of its features you learn organically when working on a project. The optional type-safety is a blessing and actually helped me a few times to find an issue. It being so close to JS is both a strength and weakness: On the one hand you have all the JS oddities in the language (implicit type conversion is really weird sometimes) and have to basically refrain from using them or count on the linter to spot them, but on the other hand you can immediately use the massive amount of JavaScript code available on the web. Angular The Angular web framework took a few hours to pick up there are a lot of concepts to understand. But ultimately, it s manageable and pretty nice to use. When working at the system level, a lot of complexity is in understanding how the CPU is processing data, managing memory and using the low-level APIs the operating system provides. With the web application stuff, a lot of the complexity for me was in learning about all the moving parts the system is comprised of, what their names are, what they are, and what works with which. And that is not a flat learning curve at all. As C developer, you need to know how the computer works to be efficient, as web developer you need to know a bunch of different tools really well to be productive. One thing I am still a bit puzzled about is the amount of duplicated HTML templates my project has. I haven t found a way to reuse template blocks in multiple components with Angular, like I would with Jinja2. The documentation suggests this feature does not exist, but maybe I simply can t find it or there is a completely different way to achieve the same result. NPM Ecosystem The MetaInfo Creator application ultimately doesn t do much. But according to GitHub, it has 985 (!!!) dependencies in NPM/NodeJS. And that is the bare minimum! I only added one dependency myself to it. I feel really uneasy about this, as I prefer the Python approach of having a rich standard library instead of billions of small modules scattered across the web. If there is a bug in one of the standard library functions, I can submit a patch to Python where some core developer is there to review it. In NodeJS, I imagine fixing some module is much harder. That being said though, using npm is actually pretty nice there is a module available for most things, and adding a new dependency is easy. NPM will also manage all the details of your dependency chain, GitHub will warn about security issues in modules you depend on, etc. So, from a usability perspective, there isn t much to complain about (unlike with Python, where creating or using a module ends up as a fight the system event way too often and the question which random file do I need to create now to achieve what I want? always exists. Fortunately, Poetry made this a bit more pleasant for me recently). So, tl;dr for this section: The web application development excursion was actually a lot of fun, and I may make more of those in future, now that I learned more about how to write web applications. Ultimately though, I enjoy the lower-level software development and backend development a bit more. Summary Check out the MetaInfo Creator and its source code, if you want to create MetaInfo files for a GUI application, console application, addon or service component quickly.

Did you know that the cloud team generates daily images for buster, bullseye, and sid? They re available for download from cdimage.debian.org and are published to Amazon EC2 and Microsoft Azure. This is done both to exercise our image generation infrastructure, and also to facilitate testing of the actual images and distribution in general. I ve often found it convenient to have easy access to a clean, up-to-date, disposable virtual machine, and you might too. Please note that these images are intended for testing purposes, and older ones may be removed at any time in order to free up various resources. You should not hardcode references to specific images in any tools or configuration. If you re downloading an image for local use, you ll probably want one of the nocloud images. They have an empty root password (the security ramifications of this should be obvious, so please be careful!), and don t rely on any cloud service for configuration. You can use the qcow2 images with QEMU on any Linux system, or retrieve the raw images for use with another VMM. If you want to use the images on Amazon EC2, you can identify the latest nightly build using the AWS CLI as follows:

# Select the most recent bullseye image for arm64 instance types:
$ aws ec2 describe-images --owner 903794441882 \
--region us-east-1 --output json \
--query "Images[?Architecture=='arm64']   [?starts_with(Name, 'debian-11-')]   max_by([], &Name)"
 
"Architecture": "arm64",
"CreationDate": "2020-03-04T05:31:12.000Z",
"ImageId": "ami-056a2fe946ef98607",
"ImageLocation": "903794441882/debian-11-arm64-daily-20200304-189",
"ImageType": "machine",
"Public": true,
"OwnerId": "903794441882",
"State": "available",
"BlockDeviceMappings": [
 
"DeviceName": "/dev/xvda",
"Ebs":  
"Encrypted": false,
"DeleteOnTermination": true,
"SnapshotId": "snap-0d7a569b159964d87",
"VolumeSize": 8,
"VolumeType": "gp2"
 
 
],
"Description": "Debian 11 (daily build 20200304-189)",
"EnaSupport": true,
"Hypervisor": "xen",
"Name": "debian-11-arm64-daily-20200304-189",
"RootDeviceName": "/dev/xvda",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
 
# Similarly, select the most recent sid amd64 AMI:
$ aws ec2 describe-images --owner 903794441882 \
--region us-east-1 --output json \
--query "Images[?Architecture=='x86_64']   [?starts_with(Name, 'debian-sid-')]   max_by([], &Name)"
 
"Architecture": "x86_64",
"CreationDate": "2020-03-04T05:13:58.000Z",
"ImageId": "ami-00ec9272298ca9059",
"ImageLocation": "903794441882/debian-sid-amd64-daily-20200304-189",
"ImageType": "machine",
"Public": true,
"OwnerId": "903794441882",
"State": "available",
"BlockDeviceMappings": [
 
"DeviceName": "/dev/xvda",
"Ebs":  
"Encrypted": false,
"DeleteOnTermination": true,
"SnapshotId": "snap-07c3fad3ff835248a",
"VolumeSize": 8,
"VolumeType": "gp2"
 
 
],
"Description": "Debian sid (daily build 20200304-189)",
"EnaSupport": true,
"Hypervisor": "xen",
"Name": "debian-sid-amd64-daily-20200304-189",
"RootDeviceName": "/dev/xvda",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
 

If you re using Microsoft Azure images, you can inspect the images with az vm image list and az vm image show, as follows:

$ az vm image list -o table --publisher debian --offer debian-sid-daily --location westeurope --all   sort -k 5   tail
debian-sid-daily Debian sid-gen2 Debian:debian-sid-daily:sid-gen2:0.20200228.184 0.20200228.184
debian-sid-daily Debian sid Debian:debian-sid-daily:sid:0.20200229.185 0.20200229.185
debian-sid-daily Debian sid-gen2 Debian:debian-sid-daily:sid-gen2:0.20200229.185 0.20200229.185
debian-sid-daily Debian sid Debian:debian-sid-daily:sid:0.20200301.186 0.20200301.186
debian-sid-daily Debian sid-gen2 Debian:debian-sid-daily:sid-gen2:0.20200301.186 0.20200301.186
debian-sid-daily Debian sid Debian:debian-sid-daily:sid:0.20200302.187 0.20200302.187
debian-sid-daily Debian sid-gen2 Debian:debian-sid-daily:sid-gen2:0.20200302.187 0.20200302.187
debian-sid-daily Debian sid Debian:debian-sid-daily:sid:0.20200303.188 0.20200303.188
debian-sid-daily Debian sid-gen2 Debian:debian-sid-daily:sid-gen2:0.20200303.188 0.20200303.188
Offer Publisher Sku Urn Version
$ az vm image show --location westeurope --urn debian:debian-sid-daily:sid:latest
 
"automaticOsUpgradeProperties":  
"automaticOsUpgradeSupported": false
 ,
"dataDiskImages": [],
"hyperVgeneration": "V1",
"id": "/Subscriptions/428325bd-cc87-41f1-b0d8-8caf8bb80b6b/Providers/Microsoft.Compute/Locations/westeurope/Publishers/debian/ArtifactTypes/VMImage/Offers/debian-sid-daily/Skus/sid/Versions/0.20200303.188",
"location": "westeurope",
"name": "0.20200303.188",
"osDiskImage":  
"operatingSystem": "Linux",
"sizeInBytes": 32212255232,
"sizeInGb": 30
 ,
"plan": null,
"tags": null
 

More information about cloud computing with Debian is available on the wiki.