Linux kernel getting a livepatch whilst running a marathon. Generated with AI.

Livepatch service eliminates the need for unplanned maintenance windows for high and critical severity kernel vulnerabilities by patching the Linux kernel while the system runs. Originally the service launched in 2016 with just a single kernel flavour supported.Over the years, additional kernels were added: new LTS releases, ESM kernels, Public Cloud kernels, and most recently HWE kernels too.Recently livepatch support was expanded for FIPS compliant kernels, Public cloud FIPS compliant kernels, and as well IBM Z (mainframe) kernels. Bringing the total of kernel flavours support to over 60 distinct kernel flavours supported in parallel. The table of supported kernels in the documentation lists the supported kernel flavours ABIs, the duration of individual build's support window, supported architectures, and the Ubuntu release. This work was only possible thanks to the collaboration with the Ubuntu Certified Public Cloud team, engineers at IBM for IBM Z (s390x) support, Ubuntu Pro team, Livepatch server & client teams.It is a great milestone, and I personally enjoy seeing the non-intrusive popup on my Ubuntu Desktop that a kernel livepatch was applied to my running system. I do enable Ubuntu Pro on my personal laptop thanks to the free Ubuntu Pro subscription for individuals.What's next? The next frontier is supporting ARM64 kernels. The Canonical kernel team has completed the gap analysis to start supporting Livepatch Service for ARM64. Upstream Linux requires development work on the consistency model to fully support livepatch on ARM64 processors. Livepatch code changes are applied on a per-task basis, when the task is deemed safe to switch over. This safety check depends mostly on kernel stacktraces. For these checks, CONFIG_HAVE_RELIABLE_STACKTRACE needs to be available in the upstream ARM64 kernel. (see The Linux Kernel Documentation). There are preliminary patches that enable reliable stacktraces on ARM64, however these turned out to be problematic as there are lots of fix revisions that came after the initial patchset that AWS ships with 5.10. This is a call for help from any interested parties. If you have engineering resources and are interested in bringing Livepatch Service to your ARM64 platforms, please reach out to the Canonical Kernel team on the public Ubuntu Matrix, Discourse, and mailing list. If you want to chat in person, see you at FOSDEM next weekend.

Focus This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

• visdom: fix license metadata
• webdl: ABC iView livestream support
• check-all-the-things: update AFL dep
• duck: more redirect/deletion phrases (1 2)
• devscripts: typo
• spelling dictionaries: codespell (1 2), lintian (1 2)
• Debian QA services: more usertags fixing (1 2 3)
• Debian package uploads: apt-listchanges read-edid
• Debian screenshots: removed wsjtx (Windows screenshot)
• Debian usertags: fix some more typos and mistakes
• Debian wiki pages: ConfigPackages, DebianBookworm, DebianEvents/internet/2020/MiniDebConfOnline2/DebCamp, DebianKernel/GitBisect, DebianMentorsFaq, DebianTrixie, Derivatives/CensusTemplate, Glossary, Lintian (Spec/LintianHarness, Tags (build-depends-on-python-dev-with-no-arch-any, Contributing, DB, Template, test)), Mobile, PulseAudio, qa.debian.org, Sahil%20Dhiman, Teams/ReleaseTeam/ReleaseCheckList, timeshift, TroubleShooting, VMware
• FOSSjobs wiki pages: resources

Issues

• Crashes in lintian-brush, evince
• Non-distributable files in nmslib
• Silent failure in tweeper
• Usability issues in systemd
• Features in dctrl-tools, lintian, gedit translate plugin
• New versions of youtube-dl
• Incorrect case conversion in webext-debianbuttons
• Missing documentation in synaptic, samba
• Syntax error in yubico-piv-tool
• Warnings in samba-common-bin
• Packaging needed for darkreader

Review

• Spam: reported 175 Debian mailing list posts
• Debian packages: sponsored acr
• Debian wiki: RecentChanges for the month
• Debian screenshots:
  • approved 2048 apparmor-notify apt-listdifferences apt-show-versions cataclysm-dda-curses ckb-next el-ixir growlight lazpaint-gtk2 lazpaint-qt5 nethack-console notcurses-bin pcsx2 performous-composer piu-piu projecteur puzzle-jigsaw ricochet setzer shotcut shotwell systray-mdstat tagainijisho tikzit vonsh wesnoth-1.14 wsjtx xsecurelock xserver-xorg-core
  • rejected mathgl (macOS), fceux (running proprietary game), photoflow (proprietary and problematic image), xserver-xorg-core (another package), xfce4-terminal (another XFCE package)
  • approved deletion of rpcbind (manual page)
  • rejected deletion of notcurses-bin (screenshots are versioned), gnome (Chromium is open source, the logo could be either), hashcat/onboard/knavalbattle/jack-keyboard/kcharselect (spam)

Administration

• Debian wiki: disable attachments due to security issue, approve accounts

Communication

• Respond to queries from Debian users and contributors on the mailing lists and IRC

Sponsors The visdom, apt-listchanges work and lintian-brush bug report were sponsored by my employer. All other work was done on a volunteer basis.

Here is another monthly update covering what I have been doing in the free software world during August 2020 (previous month):

• Filed a pull request against django-enumfield, a library that provides an enumeration-like model field for the Django web development framework. The classproperty helper has been moved to django.utils.functional in newer versions of Django. [...]
• Transferred the maintainership of my Strava Enhancement Suite Chrome extension to improve the user experience on the Strava athletic tracker to Pavel Dolecek.

• As part of my role of being the assistant Secretary of the Open Source Initiative and a board director of Software in the Public Interest, I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet, as well as the usual internal discussions, etc.
• Filed a pull request for JSON-C, a reference counting library to allow you to easily manipulate JSON objects from C in order to make the documentation build reproducibly. [...]
• Reviewed and merged some changes to my django-auto-one-to-one library for Django from Dan Palmer (which automatically creates and destroys associated model instances) to not configure signals for models that aren't installed and to honour INSTALLED_APPS during model setup. [...]
• Merged a pull request from Michael K. to cleanup the codebase after dropping support for Python 2 and Django 1.x [...] in my django-slack library which provides a convenient wrapper between projects using the Django and the Slack chat platform.
• Updated my django-staticfiles-dotd utility that concatenates Debian .d-style directories containing Javascript and CSS to drop unquote usage from the six compatibility library. [...]

I uploaded Lintian versions 2.86.0, 2.87.0, 2.88.0, 2.89.0, 2.90.0, 2.91.0 and 2.92.0, as well as made the following changes:

• New features:
  • Check for StandardOutput= and StandardError= fields that use the deprecated syslog or syslog-console systemd facilities. (#966617)
  • Add support for clzip as an alternative for lzip. (#967083)
  • Check for User=nobody and Group=nogroup in systemd .service files. (#966623)
• Bug fixes:
  • Don't emit the patch-not-forwarded-upstream tag for README files under debian/patches. (#968845)
  • Fix a false positive with the no-dh-sequencer tag due to indirect target dependences. (#968108)
• Reporting/interface:
  • Clarify the grammar of the package-uses-old-debhelper-compat-version tag. [...]
  • Update link to the detagtive web interface code as it has been moved to the lintian salsa namespace. [...]
  • Drop the misleading "Beta testing" from the webpage header. [...]
• Misc:
  • Add justification for the use of the lzip dependency in a previous debian/changelog entry. (#966817)
  • Update the generate-tag-summary release script to reflect change of tag definition filename extension change from .desc .tag. [...]
  • Revert a change to the spelling-error-in-rules-requires-root tag's severity; this is not a "spelling" check in the sense that it does not use our dictionary. [...]
  • Drop an unused $skip_tag argument in the extract_service_file_values routine. [...]

Reproducible Builds One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. The project is proud to be a member project of the Software Freedom Conservancy. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

• Further refined my merge request against the debian-installer component to allow all arguments from sources.list files (such as [check-valid-until=no]) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure and sent a gentle ping to the team that maintains that code.
• Responded at length to a discussion about the status of the reproducibility of ISO images [...], wrote back to the maintainer of WalletScrutiny asking why Android developers appear to be "allergic" to their application not being reproducible [...] and passed on an issue about the diffoscope website being down [...]. I also added a few short remarks on a proposal to bundle .buildinfo files within .deb archives [...].

This month, I:

• Filed a pull request for JSON-C, a reference counting library to allow you to easily construct JSON objects from C in order to make the documentation build reproducibly. [...]
• In Debian, I:
  • Kept isdebianreproducibleyet.com up to date. [...][...][...][...]
  • Submitted the following patches to fix reproducibility-related toolchain issues within Debian:
    • debhelper: Please make the examples from dh_missing reproducible. (#968187)
    • evolution: Please make the shipped .h files reproducible. (#968700)
  • I also submitted 9 patches to fix specific reproducibility issues in aflplusplus, chirp, golang-gonum-v1-plot, json-c, nmh, pencil2d, pixelmed-codec, serd & tpot.
• Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
• Filed a build-failure bug against the muroar package that was discovered while doing reproducibility testing. (#968189)
• We operate a large and many-featured Jenkins-based testing framework that powers tests.reproducible-builds.org. This month, I updated the self-serve package rescheduler to use HTML <pre> tags when dumping any debugging data. [...]

• Updated the main Reproducible Builds website and documentation to:
  • Clarify & fix a few entries on the "who" page [...][...] and ensure that images do not get to large on some viewports [...].
  • Clarify use of a pronoun re. Conservancy. [...]
  • Use "View all our monthly reports" over "View all monthly reports". [...]
  • Move a "is a" suffix out of the link target on the SOURCE_DATE_EPOCH age. [...]
• Drafted, published and publicised our monthly report.

diffoscope I made the following changes to diffoscope, including preparing and uploading versions 155, 156, 157 and 158 to Debian:

• New features:
  • Support extracting data of PGP signed data. (#214)
  • Try files named .pgp against pgpdump(1) to determine whether they are Pretty Good Privacy (PGP) files. (#211)
  • Support multiple options for all file extension matching. [...]
• Bug fixes:
  • Don't raise an exception when we encounter XML files with <!ENTITY> declarations inside the Document Type Definition (DTD), or when a DTD or entity references an external resource. (#212)
  • pgpdump(1) can successfully parse some binary files, so check that the parsed output contains something sensible before accepting it. [...]
  • Temporarily drop gnumeric from the Debian build-dependies as it has been removed from the testing distribution. (#968742)
  • Correctly use fallback_recognises to prevent matching .xsb binary XML files.
  • Correct identify signed PGP files as file(1) returns "data". (#211)
• Logging improvements:
  • Emit a message when ppudump version does not match our file header. [...]
  • Don't use Python's repr(object) output in "Calling external command" messages. [...]
  • Include the filename in the "... not identified by any comparator" message. [...]
• Codebase improvements:
  • Bump Python requirement from 3.6 to 3.7. Most distributions are either shipping with Python 3.5 or 3.7, so supporting 3.6 is not only somewhat unnecessary but also cumbersome to test locally. [...]
  • Drop some unused imports [...], drop an unnecessary dictionary comprehensions [...] and some unnecessary control flow [...].
  • Correct typo of "output" in a comment. [...]
• Release process:
  • Move generation of debian/tests/control to an external script. [...]
  • Add some URLs for the site that will appear on PyPI.org. [...]
  • Update "author" and "author email" in setup.py for PyPI.org and similar. [...]
• Testsuite improvements:
  • Update PPU tests for compatibility with Free Pascal versions 3.2.0 or greater. (#968124) [...][...][...]
  • Mark that our identification test for .ppu files requires ppudump version 3.2.0 or higher. [...]
  • Add an assert_diff helper that loads and compares a fixture output. [...][...][...][...]
• Misc:
  • Duplicate docker instructions in the "Get diffoscope" section of the diffoscope website. [...]

Debian Debian LTS This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 hours on its sister Extended LTS project.

• Investigated and triaged chrony [...], golang-1.8 [...], golang-go.crypto [...], golang-golang-x-net-dev [...], icingaweb2 [...], lua5.3 [...], mongodb [...], net-snmp, php7.0 [...], qt4-x11, qtbase-opensource-src, ruby-actionpack-page-caching [...], ruby-doorkeeper [...], ruby-json-jwt [...], ruby-kaminari [...], ruby-kaminari [...], ruby-rack-cors [...], shiro [...] & squirrelmail.
• Frontdesk duties, responding to user/developer questions, reviewing others' packages, participating in mailing list discussions, attending the Debian LTS BoF at DebConf20 etc.
• Issued DLA 2313-1 and ELA-257-1 to fix a privilege escalation vulnerability in Net-SNMP.
• Issued ELA-263-1 for qtbase-opensource-src and ELA-261-1 for qt4-x11, two components of cross-platform C++ application framework. A specially-crafted XBM image file could have caused a buffer overread.
• Issued ELA-268-1 to address unsafe serialisation vulnerabilities that were discovered in the PHP-based squirrelmail webmail client.
• Issued DLA 2311-1 for zabbix, the PHP-based monitoring system to fix a potential cross-site scripting vulnerability via <iframe> HTML elements.
• Issued DLA 2334-1 to fix a denial of service vulnerability in ruby-websocket-extensions, a library for managing long-lived HTTP 'WebSocket' connections.
• Issued DLA 2345-1 for PHP 7.0 as it was discovered that there was a use-after-free vulnerability when parsing PHAR files, a method of putting entire PHP applications into a single file.
• I also updated the Extended LTS website to pluralise the "Related CVEs" text in announcement emails [...] and dropped some trailing whitespace [...].

You can find out more about the project via the following video:

Uploads to Debian

• memcached:
  • 1.6.6-2 Enable TLS capabilities by default. (#968603)
  • 1.6.6-3 Add libio-socket-ssl-perl to test TLS support and perform a general package refresh.
• python-django:
  • 2.2.15-1 (unstable) New upstream bugfix release
  • 3.1-1 (experimental) New upstream release.
  • 2.2.15-2 (unstable) & 3.1-2 (experimental) Set the same PYTHONPATH when executing the runtime tests as we do in the package build. (#968577)
• docbook-to-man:
  • 2.0.0-43 Refresh packaging, and upload some changes from the Debian Janitor.
  • 2.0.0-44 Fix compatibility with GCC 10, restoring the missing /usr/bin/instant binary. (#968900)
• hiredis (1.0.0-1) New upstream release to experimental.

Here is my monthly update covering what I have been doing in the free and open source software world during July 2020 (previous month):

• Opened a pull request to make the build reproducible in PyERFA, a set of Python bindings for various astronomy-related utilities (#45), as well as one for PeachPy assembler to make the output of codecode/x86_64.py reproducible (#108).

• As part of being on the board of directors of the Open Source Initiative and Software in the Public Interest I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet, as well as the usual internal discussions regarding logistics and policy etc. This month, it was SPI's Annual General Meeting and the OSI has been running a number of remote strategy sessions for the board.
• Fixed an issue in my tickle-me-email library that implements Getting Things Done (GTD)-like behaviours in IMAP inboxes to ensure that all messages have a unique Message-Id header. [...]
• Reviewed and merged even more changes by Pavel Dolecek into my Strava Enhancement Suite, a Chrome extension to improve the user experience on the Strava athletic tracker.
• Updated travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub, to use the Travis CI continuous integration platform) to fix a compatibility issue with the latest version of mk-build-deps. [...][...]

For Lintian, the static analysis tool for Debian packages:

• Update the regular expression to search for all the released versions in a .changes file. [...]
• Avoid false-positives when matching sensible-utils utilities such as i3-sensible-pager. (#966022)
• Rename the send-patch tag to patch-not-forwarded-upstream. [...]
• Drop reminders from 26 tags that false-positives should be reported to Lintian as this is implicit in all our tags. [...]

Reproducible Builds One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. The project is proud to be a member project of the Software Freedom Conservancy. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter. This month, I:

• Opened pull requests to make the build reproducible in pyerfa (#45) as well as one for PeachPy to make the output of codecode/x86_64.py reproducible (#108).
• In Debian:
  • Submitted the following patches to fix reproducibility-related toolchain issues within Debian:
    • dh-fortran-mod: Please make the output reproducible. (#965255)
    • gem2deb: should not install mkmf.log files. (#964772)
    • nmap: Please make the obfuscated nmap_service.exe file reproducible. (#964369)
  • I also submitted 13 patches to fix specific reproducibility issues in flit, gmap, jskeus, libtpms, logilab-common, mrbayes, numpydoc, pyerfa, python-cooler, python-peachpy, python-pyxs, sratom & weather-util.
  • When investigating packages, I discovered and reported that dolfinx was running its test suite even when the nocheck build profile was set. (#965946)
  • Kept isdebianreproducibleyet.com up to date. [...]
• Categorised a large number of packages and issues in the Reproducible Builds 'notes' repository.
• Drafted, published and publicised our monthly report.

• Updated the main Reproducible Builds website and documentation to drop a number of unused Javascript files [...][...][...] and added unminified versions of Bootstrap and jQuery [...] and fixed a number of broken URLs [...][...].
• strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. This month, I made sure that we did not install the internal handler documentation generated from Perl POD documents [...] and fixed a trivial typo [...].

diffoscope Elsewhere in our tooling, I made the following changes to diffoscope, including preparing and uploading versions 150, 151, 152, 153 & 154 to Debian:

• New features:
  • Add support for flash-optimised F2FS filesystems. (#207)
  • Don't require zipnote(1) to determine differences in a .zip file as we can use libarchive. [...]
  • Allow --profile as a synonym for --profile=-, ie. write profiling data to standard output. [...]
  • Increase the minimum length of the output of strings(1) to eight characters to avoid unnecessary diff noise. [...]
  • Drop some legacy argument styles: --exclude-directory-metadata and --no-exclude-directory-metadata have been replaced with --exclude-directory-metadata= yes,no . [...]
• Bug fixes:
  • Pass the absolute path when extracting members from SquashFS images as we run the command with working directory in a temporary directory. (#189)
  • Correct adding a comment when we cannot extract a filesystem due to missing libguestfs module. [...]
  • Don't crash when listing entries in archives if they don't have a listed size such as hardlinks in ISO images. (#188)
• Output improvements:
  • Strip off the file offset prefix from xxd(1) and show bytes in groups of 4. [...]
  • Don't emit javap not found in path if it is available in the path but it did not result in an actual difference. [...]
  • Fix ... not available in path messages when looking for Java decompilers that used the Python class name instead of the command. [...]
• Logging improvements:
  • Add a bit more debugging info when launching libguestfs. [...]
  • Reduce the --debug log noise by truncating the has_some_content messages. [...]
  • Fix the compare_files log message when the file does not have a literal name. [...]
• Codebase improvements:
  • Rewrite and rename exit_if_paths_do_not_exist to not check files multiple times. [...][...]
  • Add an add_comment helper method; don't mess with our internal list directly. [...]
  • Replace some simple usages of str.format with Python 'f-strings' [...] and make it easier to navigate to the main.py entry point [...].
  • In the RData comparator, always explicitly return None in the failure case as we return a non-None value in the success one. [...]
  • Tidy some imports [...][...][...] and don't alias a variable when we don't end up using. [...]
  • Clarify the use of a separate NullChanges quasi-file to represent missing data in the Debian package comparator [...] and clarify use of a 'null' diff in order to remember an exit code. [...]
• Misc:
  • Profile the launch of libguestfs filesystems. [...]
  • Clarify and correct our contributing info. [...][...][...][...][...][...]

Debian In Debian, I made the following uploads this month:

• Redis (6.0.6-1) New upstream release.
• Django:
  • 2.2.14-1 New upstream bugfix release.
  • 3.1~rc1-1 New upstream release candidate.
• lastpass-cli (1.3.3-4) Fix a build failure under GCC 10. (#957416)

Debian LTS This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 for the Extended LTS project. This included:

• Frontdesk duties, including responding to user/developer questions, reviewing packages, participating in mailing list discussions, offering comments on our survey and attended our IRC contributor meeting. I also made some changes to tools after the move from the Debian version supported [...].
• Investigated and triaged atril, ceph, cimg, ffmpeg, golang-github-seccomp-libseccomp-golang, gosa, http-parser, jruby, json-c, jupyter-notebook, keystone, ksh, libjpeg-turbo, libopenmpt, libpam-radius-auth, log4net, mailman, milkytracker, mupdf, pillow, poppler, puma, python-rtslib-fb, roundcube, ruby-zip, salt, slirp, sqlite3, transmission & wordpress.
• Issued DLA 2273-1 for shiro, first to fix a path-traversal issue where a specially-crafted request could cause an authentication bypass (CVE-2020-1957) and then to fix an encoding issue introduced in the handling of the previous CVE-2020-1957 path-traversal issue which itself could have also caused an authentication bypass (CVE-2020-11989).
• Issued DLA 2274-1 to fix a possible signature verification issue in the fwupd firmware update tool as the return value of gpgme_op_verify_result was not being checked.
• Issued DLA 2299-1 and ELA 252-1 in order to prevent a privilege escalation vulnerability discovered in Net-SNMP, a set of tools for collecting and organising information about devices on computer networks.
• Issued DLA 2300-1 as it was discovered that there was an issue where kdepim-runtime would default to using unencrypted POP3 communication despite the user interface indicating that encryption was in use.

You can find out more about the project via the following video:

Here is my monthly update covering what I have been doing in the free software world during June 2020 (previous month):

• Opened two pull requests against the Ghostwriter distraction-free Markdown editor to:
  • Persist whether "focus mode" is enabled across between sessions. (#522)
  • Correct the ordering of the MarkdownAST::toString() debugging output. (#520)
• Will McGugan's "Rich" is a Python library to output formatted text, tables, syntax etc. to the terminal. I filed a pull request in order to allow for easy enabling and disabling of displaying the file path in Rich's logging handler. (#115)
• As part of my duties of being on the board of directors of the Open Source Initiative and Software in the Public Interest I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet, as well as the usual internal discussions regarding logistics and policy etc.
• Filed a pull request against the PyQtGraph Scientific Graphics and graphical user interface library to make the documentation build reproducibly. (#1265)
• Reviewed and merged a large number of changes by Pavel Dolecek to my Strava Enhancement Suite, a Chrome extension to improve the user experience on the Strava athletic tracker.

For Lintian, the static analysis tool for Debian packages:

• Don't emit breakout-link for architecture-independent .jar files under /usr/lib. (#963939)
• Correct a reference to override_dh_ in the long description of the excessive-debhelper-overrides tag. [...]
• Update data/fields/perl-provides for Perl 5.030003. [...]
• Check for execute_after and execute_before spelling mistakes just like override_*. [...]

Reproducible Builds One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. The project is proud to be a member project of the Software Freedom Conservancy. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter. This month, I:

• Filed a pull request against the PyQtGraph to make the documentation build reproducibly. (#1265)
• Addressed a regression in the octave-queueing package in the handling of absolute paths in Texinfo files. (#962187)
• Submitted the following patches to fix reproducibility-related toolchain issues within Debian:
  • critcl: Please make the teapot.txt files reproducible. (#963600)
  • sasmodels: Regression in debian/patches/reproducible-c-models.patch. (#962050)
• Contributed to a discussion regarding improving the reproducibility of packages that use the CMake build system. [...]
• Added to a conversation regarding the nondeterministic execution of order of Debian maintainer scripts that results in the arbitrary allocation of UNIX group IDs referencing the Tails operating system. [...]
• I also submitted 14 patches to fix specific reproducibility issues in fonts-anonymous-pro, gftl, golang-github-viant-toolbox, golang-v2ray-core, libmbim, libqmi, neovim-qt, netcdf-fortran, petitboot, python-pauvre, python-pyqtgraph, python-stem, seqtools & tkabber-plugins.
• Kept isdebianreproducibleyet.com up to date. [...][...]
• Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
• Updated the main Reproducible Builds website and documentation to limit the number of "news" posts to, for example, avoid showing items from 2017. [...]
• Drafted, published and publicised our monthly report.

Elsewhere in our tooling, I made the following changes to diffoscope including preparing and uploading versions 147, 148 and 149 to Debian:

• New features:
  • Add output from strings(1) to ELF binaries. (#148)
  • Allow user to mask/filter diff output via --diff-mask=REGEX. (!51)
  • Dump PE32+ executables (such as EFI applications) using objdump(1). (#181)
  • Add support for Zsh shell completion. (#158)
• Bug fixes:
  • Prevent a traceback when comparing PDF documents that did not contain metadata (ie. a PDF /Info stanza). (#150)
  • Fix compatibility with jsondiff version 1.2.0. (#159)
  • Fix an issue in GnuPG keybox file handling that left filenames in the diff. [...]
  • Correct detection of JSON files due to missing call to File.recognizes that checks candidates against file(1). [...]
• Output improvements:
  • Use the CSS word-break property over manually adding U+200B zero-width spaces as these were making copy-pasting cumbersome. (!53)
  • Downgrade the tlsh warning message to an "info" level warning. (#29)
• Logging improvements:
  • Log calls to subprocess.check_output by using a wrapper. (#151)
  • Clarify that we are generating presenter formats in a debug-level message. [...]
  • Log the version of jsondiff used. [...]
• Testsuite improvements:
  • Update tests for file(1) version 5.39. (#179)
  • Drop accidentally-duplicated copy of the --diff-mask tests. [...]
  • Don't mask an existing test. [...]
• Codebase improvements:
  • Replace obscure references to "WF" with "Wagner-Fischer" for clarity. [...]
  • Use a semantic AbstractMissingType type instead of remembering to check for both types of "missing" files. [...]
  • Add a comment regarding potential security issue in the .changes, .dsc and .buildinfo comparators. [...]
  • Drop a large number of unused imports. [...][...][...][...][...]
  • Make many code sections more Pythonic. [...][...][...][...]
  • Prevent some variable aliasing issues. [...][...][...]
  • Use some tactical f-strings to tidy up code [...][...] and remove explicit u"unicode" strings [...].
  • Refactor a large number of routines for clarity. [...][...][...][...]

trydiffoscope is the web-based version of diffoscope. This month, I specified a location for the celerybeat scheduler to ensure that the clean/tidy tasks are actually called which had caused an accidental resource exhaustion. (#12)

Debian I filed three bugs against:

• cmark: Please update the homepage URI. (#962576)
• petitboot: Please update Vcs-Git urls. (#963123)
• python-pauvre: FTBFS if the DISPLAY environment variable is exported. (#962698)

Debian LTS This month I have worked 18 hours on Debian Long Term Support (LTS) and 5 hours on its sister Extended LTS project.

• Investigated and triaged angular.js [...],icinga2 [...], intel-microcode [...], jquery [...], pdns-recursor [...], unbound [...] & wordpress [...].
• Frontdesk duties, including responding to user/developer questions, reviewing others' packages, participating in mailing list discussions as well as attending our contributor meeting.
• Issued DLA 2233-1 to fix two issues in the Django web development framework in order to fix a potential data leakage via malformed memcached keys (CVE-2020-13254) and to prevent a cross-site scripting attack in the Django administration system (CVE-2020-13596). This was followed by DLA 2233-2 to address a regression as well as uploads to Debian stretch (1.10.7-2+deb9u9) and buster (1.11.29-1~deb10u1). (More info)
• Issued DLA 2235-1 to prevent a file descriptor leak in the D-Bus message bus (CVE-2020-12049).
• Issued DLA 2239-1 for a security module for using the TACACS+ authentication service to prevent an issue where shared secrets such as private server keys were being added in plaintext to various logs.
• Issued DLA 2244-1 to address an escaping issue in PHPMailer, an email generation utility class for the PHP programming language.
• Issued DLA 2252-1 for the ngircd IRC server as it was discovered that there was an out-of-bounds access vulnerability in the server-to-server protocol.
• Issued DLA 2253-1 to resolve a vulnerability in the Lynis a security auditing tool because a shared secret could be obtained by simple observation of the process list when a data upload is being performed.

You can find out more about the project via the following video:

Uploads

• redis (6.0.4-1 & 6.0.5-1) New upstream releases.
• python-django:
  • 2.2.13-1 New upstream security release followed by a fix for an upstream regression 2.2.13-2
  • 3.0.7-1 New upstream security release followed by a fix for an upstream regression in 3.0.7-2
  • 3.1~beta1-1 New upstream beta release.

The Garmin 910XT is the premier GPS watch for triathlon training and racing.

1. You'll probably need a newer version of pyusb:

   $ git clone git://github.com/walac/pyusb
   

   There is no need to install it systemwide - you can simply leave it checked out somewhere.

2. Get the Garmin-Forerunner-610-Extractor project from GitHub:

   $ git clone git://github.com/Tigge/Garmin-Forerunner-610-Extractor.git
   

3. Turn your watch on and plug in the USB adaptor.

4. You can now sync your activities:

   $ sudo env PYTHONPATH=/path/to/pyusb ./garmin.py
   Request basic information...
     ANT version:   AJK1.04RAF
     Capabilities:  array('B', [8, 3, 0, 186, 54, 0])
     Serial number: 3480039
   Starting system...
   Searching...
   Authenticating with Forerunner 910XT (3863343203)
    - Passkey: OK
   Downloading 1 file(s)
   Downloading 2013-09-03_21-09-32-80-19364.fit [..........]
   

   The first time you sync, you will have to authenticate on the watch itself. Downloading the activities will be much slower than you expect.

5. After the syncing has finished, don't forget to turn the watch off. If you are used to the Garmin Edge 800 (which turns itself off when you insert the USB cable) you will probably not have the habit of turning off the unit.

6. Rather confusingly, your .FIT files are available in the Xdg configuration folder. For example:

   ls -l ~/.config/garmin-extractor/3863343203/activities/
   -rw-r--r-- 1 root root   2688 Jun  9 15:23 2013-06-08_12-33-24-80-2688.fit
   -rw-r--r-- 1 root root   1171 Jun  9 15:23 2013-06-08_12-37-14-80-1171.fit
   -rw-r--r-- 1 root root   6111 Jun  9 15:23 2013-06-08_22-19-28-80-6111.fit
   -rw-r--r-- 1 root root 170327 Jun  9 15:26 2013-06-09_11-09-44-80-170327.fit
   

7. Strava supports these .fit files through the From File upload option.

The Garmin 910XT is the premier GPS watch for triathlon training and racing.

1. You'll probably need a newer version of pyusb:

   $ git clone git://github.com/walac/pyusb
   

   There is no need to install it systemwide - you can simply leave it checked out somewhere.

2. Get the Garmin-Forerunner-610-Extractor project from GitHub:

   $ git clone git://github.com/Tigge/Garmin-Forerunner-610-Extractor.git
   

3. Turn your watch on and plug in the USB adaptor.

4. You can now sync your activities:

   $ sudo env PYTHONPATH=/path/to/pyusb ./garmin.py
   Request basic information...
     ANT version:   AJK1.04RAF
     Capabilities:  array('B', [8, 3, 0, 186, 54, 0])
     Serial number: 3480039
   Starting system...
   Searching...
   Authenticating with Forerunner 910XT (3863343203)
    - Passkey: OK
   Downloading 1 file(s)
   Downloading 2013-09-03_21-09-32-80-19364.fit [..........]
   

   The first time you sync, you will have to authenticate on the watch itself. Downloading the activities will be much slower than you expect.

5. After the syncing has finished, don't forget to turn the watch off. If you are used to the Garmin Edge 800 (which turns itself off when you insert the USB cable) you will probably not have the habit of turning off the unit.

6. Rather confusingly, your .FIT files are available in the Xdg configuration folder. For example:

   ls -l ~/.config/garmin-extractor/3863343203/activities/
   -rw-r--r-- 1 root root   2688 Jun  9 15:23 2013-06-08_12-33-24-80-2688.fit
   -rw-r--r-- 1 root root   1171 Jun  9 15:23 2013-06-08_12-37-14-80-1171.fit
   -rw-r--r-- 1 root root   6111 Jun  9 15:23 2013-06-08_22-19-28-80-6111.fit
   -rw-r--r-- 1 root root 170327 Jun  9 15:26 2013-06-09_11-09-44-80-170327.fit
   

7. Strava supports these .fit files through the From File upload option.

For some time now, we ve been gearing up to begin development on Ubuntu 11.04. While some folks have been putting the finishing touches on the 10.10 release, and bootstrapping the infrastructure for 11.04, others have been meeting with Canonical stakeholders, coordinating community brainstorm sessions, and otherwise collecting information about what our priorities should be in the next cycle. We re using what we ve learned to plan the Ubuntu Developer Summit next week in Orlando, where we ll refine these ideas into a plan for the cycle. We re organizing UDS a little bit differently this time, with the main program divided into the following tracks to reflect the key considerations for Ubuntu today:

• Application Developers Making it faster, easier, and more enjoyable to develop and distribute new applications on (and for) Ubuntu
• Cloud Delivering the best experience of cloud computing, whether hosting in a public cloud or building your own private cloud
• Hardware Compatibility Measuring and improving compatibility with a wide range of laptops, netbooks, servers and desktops
• Multimedia Formulating the best software stacks for graphics, audio and video in Ubuntu
• Package Selection and System Defaults Choosing the right components to keep Ubuntu lean, flexible and ready-to-run, while ensuring that the pieces fit and work together cleanly
• Performance Squeezing the best performance out of today s free software stack, from the Linux kernel and GNU toolchain through user interfaces
• Ubuntu the Project Continuously improving the way we work together to produce Ubuntu, both within the project and with our upstream and downstream partners

You can click on the links above for a preview of the schedule for the week, with links to more detailed blueprints which will develop during and following UDS. If you ll be joining us in person, then I ll see you there! If not, be sure to review Laura s guide on how to participate remotely.

10 days ago, I uploaded syslinux-themes-debian. It is sitting in the NEW queue ever since, and aparently I cannot rely on it being accepted anytime soon (it can be installed from the repository mentioned on the maintainers homepage, though). However, while being still a first working version only that needs some improvements (especially until it could be used as a generic ressource for any other tool to make use of it, like live-helper or debian-cd, eventually), here's a screenshot from the squeeze theme, credits and a big thanks for the awesome graphics to Agnieszka Czajkowska.

I just wrote this in my user page in the English wikipedia:

Antti-Juhani Kaijanaho is no longer actively working on Wikipedia. While I approve, in principle, all the new and stricted editor policies, it is emotionally very demanding to be challenged under new policies for edits that were made in good faith under older policies. There just aren’t enough positive aspects to doing Wikipedia to offset that very big negative aspect.

Some of the challenges toward my edits have happened in the English Wikipedia, and some in the Finnish Wikipedia. I have felt like this for about a year now – it’s probably time to make it official. I may make minor edits, or write in talk pages occasionally, but any substantial edits I won’t bother with any more.

Just to prevent people from thinking I’ve gotten old and stodgy, I’ve joined the Twitter bandwagon. Here’s my feed, which also appears in my sidebar. I’m reminded of a Utah Phillips quote, from a different context:

Well, and I’m open to all those things. If you live in California, you’ve got to be open; if you’re not they pry you open.

I’m not sure whether I really agree with Twitter–it reminds me of this interesting collection of postmodern thoughts from n+1 magazine:

The Decivilizing Process As the specific addressee of any set of remarks becomes less important, in the midst of more and more babble, it will become more and more difficult to remember the special status of listening human beings, in the confusion of shouted orders.

But, hey, it’s worth a try.

Share This

If you use grep-dctrl, sort-dctrl or tbl-dctrl, I’d like to see your use cases! I’m building a system test automaton for dctrl-tools, and I’d like to include as much real-world examples as possible so that any regressions get found before uploading. You can send example command lines (and preferably also a test file, unless it’s one of the standard Debian files like Packages) to me by email at <ajk@debian.org>, as a wishlist bug report against dctrl-tools or by a Darcs patch against the experimental repository (see instructions).

Insipired by Daniel Burrows’ tests, I decided to do mine. The object is to come up with a Haskell program that performs the job of wc -l as efficiently as possible. My test material is a randomly generated file with all lines less than 80 characters long. Summary: my final idiomatic version is twice as slow as wc -l, and my final nonidiomatic version is as fast as wc -l, or at least within the margin of error. First, the baseline:

$ time wc -l testfile
7798137 testfile
real    0m12.127s
user    0m0.765s
sys     0m0.399s

Then the most idiomatic program. Honestly, I expect it to suck performance-wise:

main = getContents >>= print . length . lines

And it does:

$ time ./wcl1.hs < testfile
7798137
real    1m14.993s
user    0m54.182s
sys     0m0.573s

But only by a factor of six. My first hypothesis is that garbage collection is the culprit. And, indeed:

$ time ./wcl1.hs +RTS -sstderr -RTS < testfile
./wcl1.hs +RTS -sstderr
7798137
25,236,174,340 bytes allocated in the heap
5,070,193,924 bytes copied during GC
117,484 bytes maximum residency (3750 sample(s))
96274 collections in generation 0 ( 26.36s)
3750 collections in generation 1 (  0.74s)
2 Mb total memory in use
INIT  time    0.00s  (  0.00s elapsed)
MUT   time   27.17s  ( 42.72s elapsed)
GC    time   27.10s  ( 33.72s elapsed)
EXIT  time    0.00s  (  0.00s elapsed)
Total time   54.27s  ( 76.44s elapsed)
%GC time      49.9%  (44.1% elapsed)
Alloc rate    928,824,966 bytes per MUT second
Productivity  50.1% of total user, 35.5% of total elapsed
real    1m16.616s
user    0m54.278s
sys     0m0.700s

As you can see, GC takes half of the processing time. Unfortunately, minimizing GC time does not have as big an effect as I hoped:

$ time ./wcl1.hs +RTS -H100M  -RTS  < testfile
7798137
real    1m1.354s
user    0m44.371s
sys     0m1.171s

Here, I’m effectively telling it to use a 100-megabyte heap. A separate GC profiling run tells me that GC time has dropped from 50 % to 1 %. The memory problem seems to be a red herring. What we’re doing here is creating four separately allocated memory cells for each character in the input. Of course, the cells are generated lazily, that is, a cell pair is allocated when the lines function demands another character; it then creates another cell pair to denote the same character in a list of lines. As soon as each line has been counted, the four cells become garbage. This sounds like a lot of work! But apparently it isn’t; I tried a few variations that did away with the stream of input altogether, not much unlike what Daniel did in his tests. Surprisingly, the performance worsened a lot. The lesson: functional programming language implementations, unlike typical imperative ones, are built on the assumption that you’ll be doing a lot of allocation. What I suspect is taking a lot of the time is arithmetic. Haskell’s Int is boxed, and lazy by default, adding a lot of overhead. I hand-fused length and line, yielding a tail-recursive countLines function, and I added the command-line switch -funbox-strict-fields, which enables a few significant optimizations of the laziness of Int. The results were nice:

ajk@kukkamaljakko:~/scratch$ cat wcl4.hs
module Main (main) where
main :: IO ()
main = getContents >>= print . countLines 0
countLines :: Int -> String -> Int
countLines n ('n':r) = countLines (n + 1) r
countLines n (_:r) = countLines n r
countLines n [] = n
ajk@kukkamaljakko:~/scratch$ ghc -O -Wall -funbox-strict-fields --make wcl4.hs -o wcl4
Chasing modules from: wcl4.hs
Compiling Main             ( wcl4.hs, wcl4.o )
Linking ...
ajk@kukkamaljakko:~/scratch$ time ./wcl4 < testfile
7798137
real    0m37.961s
user    0m29.853s
sys     0m0.495s
ajk@kukkamaljakko:~/scratch$ time ./wcl4 +RTS -H100M < testfile
7798137
real    0m25.153s
user    0m13.497s
sys     0m0.803s
ajk@kukkamaljakko:~/scratch$

With a large heap, the program is only twice as slow as the baseline. This is pretty good, considering that the code is still idiomatically functional. The following rather ugly, imperative program does away with the lazy stream and carefully avoids any laziness without doing any too dirty tricks:

ajk@kukkamaljakko:~/scratch$ cat wcl7.hs
module Main (main) where
import Data.Word
import Data.Array.IO
import IO (stdin)
count :: Int -> (Int, Int) -> IOUArray Int Word8 -> IO Int
count 0 _ _   False = undefined -- force ac evaluation
count ac (l,u) _   l > u = return ac
count ac (l,u) ar
    = do a < - readArray ar l
         let ac' = case a of 10 -> ac + 1
                             _  -> ac
         count ac’ (l+1, u) ar
maxn :: Int
maxn = 4096
main :: IO ()
main = do
  arr < - newArray (0,maxn) 0
  let loop :: Int -> IO Int
      loop acc = do n < - hGetArray stdin arr maxn
                    if n == 0
                       then return acc
                       else do acc' <- count acc (0,n) arr
                               loop acc'
  res <- loop 0
  print res
ajk@kukkamaljakko:~/scratch$ ghc -O  -Wall -funbox-strict-fields --make wcl7.hs -o wcl7
Chasing modules from: wcl7.hs
Compiling Main             ( wcl7.hs, wcl7.o )
Linking ...
ajk@kukkamaljakko:~/scratch$ time ./wcl7 < testfile
7798137
real    0m13.431s
user    0m10.342s
sys     0m0.467s

Of course, this program is not Haskell 98, but the nonstandard things it uses are fairly common. As can be seen, this version’s performance is, considering the margin of error, equal to the baseline. The compiler used in this test is The Glorious Glasgow Haskell Compilation System, version 6.4.1, as packaged in Debian sid.

I won. I completed 50,000 words in time. Of course, only about 13,000 words are directly usable if and when I end up editing the story, the rest will have to be removed – they were imposed by the strict timetable and the explicit leave to lower one’s standards. But, bottom line: I WON! Ps. Please don’t ask to see the manuscript. It’s not that good Pps. My mother won, too. Conga rats!

Date: Mon, 17 Apr 2000 03:07:13 +0300
From: Antti-Juhani Kaijanaho <gaia@iki.fi>
To: "Debian developers' misc list" <debian-devel@lists.debian.org>
Subject: RFC/ITP: Constitution of Finland

On IRC I proposed packaging the Constitution of Finland.  Among other
things, it would be an example of a good Constitution :-)  The response
on IRC was positive; what do people think here?

Since Finnish laws are specifically not protected by copyright (see
Finnish copyright law), the law is free and can go to main.  There is
an English translation at http://www.om.fi/perustuslaki/3313.htm.
I am researching its copyright status.

Assuming people are okay with this, and the copyright issues around the
English translation are resolved, I will package Constitution of Finland
to woody.

-- 
%%% Antti-Juhani Kaijanaho % gaia@iki.fi % http://www.iki.fi/gaia/ %%%

                    I'm moving IRL on May 2, 2000.
               New contact information on the home page

Do you remember this infamous thread? :) For some reason, it has almost completely been censored off the Debian mailing list archives. I digged my own archive and put the thread online in a mailbox. Have fun with nostalgia and AJK-bashing :) Maybe this will help to reestablish the thread in the archives; who knows?

Can we agree on not calling Anthony Towns AJ? Every time someone refers to AJ in the lists or in Planet Debian, it takes me a few moments to realize it's not me they are talking about. Let's face it, AJ is a fairly obvious nickname for anybody with the (partial) initials A. J. And there are bound to be many of those. I personally discourage people calling me A(-)J, but there still are many people who do so. Since I have been called that by many people since eigth grade (around 1991), I have learned to respond to that name. I personally use it only in the three-letter signature AJK. It took me some time to realize that Joe Wreschnig wasn't calling my reading of the old SC very strange, and even now I cannot be 100 % sure. Update: Joe has updated his post to no longer refer to Anthony as AJ.

Here is my hackergotchi:

I see from my logs that more than one Planet is republishing my Debian bits. It seems there is no way for me to figure out what those planets are (I know about Debian Planet since I submitted my blog there myself) without doing something like this. I have two requests for the maintainers of those planets:

1. Let me know that you are republishing my bits; if your planet is publically available, let me know the address; if you ask, I'll keep it to myself. (Debian Planet need not bother, I already know you.)
2. Please, consider syndicating en/ instead of en/debian/.

My mail address is <ajk@debian.org>. Or leave a comment to this entry :)

Just uploaded to experimental a new version of grep-dctrl, 2.3. It can do numeric comparisons:

ajk@kukkaruukku[01:02:34]:dctrl-tools$ grep-available -FInstalled-Size --gt 100000 -sPackage
Package: openoffice.org-bin
Package: atlc-examples
Package: xlibmesa-dri-dbg
Package: vtk-doc
Package: gcc-snapshot
Package: fgfs-base
Package: gcc-experimental
Package: libopenh323-dbg
Package: vegastrike-data