error: symbol grub_is_lockdown not found

update-grub2

dpkg-reconfigure -plow grub-efi-amd64

Force extra installation to the EFI removable media path?

Some EFI-based systems are buggy and do not handle new bootloaders correctly. If you force an extra installation of GRUB to the EFI removable media path, this should ensure that this system will boot Debian correctly despite such a problem. However, it may remove the ability to boot any other operating systems that also depend on this path. If so, you will need to make sure that GRUB is configured successfully to be able to boot any other OS installations correctly.

Update NVRAM variables to automatically boot into Debian?

GRUB can configure your platform's NVRAM variables so that it boots into Debian automatically when powered on. However, you may prefer to disable this behavior and avoid changes to your boot configuration. For example, if your NVRAM variables have been set up such that your system contacts a PXE server on every boot, this would preserve that behavior.

eddy@aptonia:/ $ cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
r8169

DRV_NAME=rtl8821ce
DRV_VERSION=v5.5.2_34066.20200325

cp -r . /usr/src/$ DRV_NAME -$ DRV_VERSION

dkms add -m $ DRV_NAME -v $ DRV_VERSION

dkms build -m $ DRV_NAME -v $ DRV_VERSION -k 5.10.0-0.bpo.8-amd64
dkms install -m $ DRV_NAME -v $ DRV_VERSION -k 5.10.0-0.bpo.8-amd64

dkms build rtl8821ce/v5.5.2_34066.20200325 -k 4.19.0-17-amd64
dkms install rtl8821ce/v5.5.2_34066.20200325 -k 4.19.0-17-amd64

1. Turn off the VM
2. Make sure you are using VirtIO for the disk, select Discard mode: unmap
3. Boot into Windows, and from an elevated prompt run: Optimize-Volume -DriveLetter C -ReTrim -Verbose
4. Shut down the VM
5. Run a dummy conversion which sparsifies the image: qemu-img convert -O qcow2 orig.qcow2 sparse.qcow2
6. Rename/Backup the images and boot back into Windows.

• https://wiki.debian.org/DebianEvents/de/2021/DebianReunionHamburg#Dates_and_format_of_the_event
• https://wiki.debian.org/DebianEvents/de/2021/DebianReunionHamburg#Covid_things
• https://wiki.debian.org/DebianEvents/de/2021/DebianReunionHamburg#Call_for_papers
• https://wiki.debian.org/DebianEvents/de/2021/DebianReunionHamburg#Registration - registration is mandatory for this event!

Debian

This was my 32nd month of actively contributing to Debian. I became a DM in late March 2019 and a DD on Christmas 19! \o/ Tough month but I mostly spent on it churning through the immense backlog. But that somewhat backfired and I have even more backlog than ever. :D Anyway, I did the following stuff in Debian:

Uploads and bug fixes:

• ruby3.0 (3.0.0-2) - Upload to unstable! \o/

Other $things:

• Mentoring for newcomers.
• Moderation of -project mailing list.

---

Ubuntu

This was my 7th month of actively contributing to Ubuntu. Now that I ve joined Canonical to work on Ubuntu full-time, there s a bunch of things I do! \o/ I mostly worked on different things, I guess. But mostly on packaging keylime and some Google Agents upload(s) and SRU(s). Also did a lot of reviewing, et al. I was too lazy to maintain a list of things I worked on so there s no concrete list atm. Maybe I ll get back to this section later or will start to list stuff from next month onward, as I ve been doing before. :D

---

Debian (E)LTS

Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success. And Debian Extended LTS (ELTS) is its sister project, extending support to the Jessie release (+2 years after LTS support). This was my twenty-third month as a Debian LTS and eleventh month as a Debian ELTS paid contributor.
I was assigned 23.75 hours for LTS and 40.00 hours for ELTS and worked on the following things:
(however, I only worked for 23.75h on ELTS work, thereby, carrying the rest to next month)

LTS CVE Fixes and Announcements:

• Issued DLA 2743-1, fixing CVE-2017-5715, for amd64-microcode.
  For Debian 9 stretch, these problems have been fixed in version 3.20181128.1~deb9u1.
• Issued DLA 2744-1, fixing the versioning issue, for usermode.
  For Debian 9 stretch, these problems have been fixed in version 1.109-1+deb9u1.
• Issued DLA 2750-1, fixing CVE-2019-20421, CVE-2021-3482, CVE-2021-29457, CVE-2021-29473, CVE-2021-31291, and CVE-2021-31292, for exiv2.
  For Debian 9 stretch, these problems have been fixed in version 0.25-3.1+deb9u3.

ELTS CVE Fixes and Announcements:

• Issued ELA 479-1, fixing CVE-2019-20421, CVE-2021-3482, CVE-2021-29457, CVE-2021-29473, CVE-2021-31291, and CVE-2021-31292, for exiv2.
  For Debian 8 jessie, these problems have been fixed in version 0.24-4.1+deb8u6.
• Noticed that there s a fallout of CVE-2021-3185, where an update was issued for gst-plugins-bad1.0, however, not for gst-plugins-bad0.10.
  Thanks to Sylvain s script, this came up and I prepped an update for that.
• Started to work on libjdom1-java s regression.

Other (E)LTS Work:

• Front-desk duty from 26-07 until 01-08 and from 30-08 until 05-09 for both LTS and ELTS.
• Triaged haproxy, ntfs-3g, and cyrus-imapd, and exiv2, ffmpeg, git, gpac, inetutils, mc, modsecurity-crs, node-object-path, php-pear, systemd-cron, node-tar, ruby2.3, gst-plugins-bad0.10, jsoup, libxstream-java, qemu, tomcat7, ruby2.1, prototypejs, pillow, cpio, and qtbase-opensource-src, and amd64-microcode.
• Mark CVE-2021-39240/haproxy as not-affected for stretch and jessie.
• Mark CVE-2021-39241/haproxy as not-affected for stretch and jessie.
• Mark CVE-2021-39242/haproxy as not-affected for stretch and jessie.
• Mark CVE-2021-33582/cyrus-imapd as no-dsa for stretch.
• Mark CVE-2020-18771/exiv2 as no-dsa for exiv2 for stretch.
• Mark CVE-2020-18899/exiv2 as no-dsa for exiv2 for stretch.
• Mark CVE-2021-38171/ffmpeg as postponed for stretch.
• Mark CVE-2021-40330/git as no-dsa for stretch and jessie.
• Mark CVE-2020-19481/gpac as ignored for stretch.
• Mark CVE-2021-40491/inetutils as no-dsa for stretch.
• Mark CVE-2021-36370/mc as no-dsa for stretch and jessie.
• Mark CVE-2021-35368/modsecurity-crs as no-dsa for stretch.
• Mark CVE-2021-23434/node-object-path as end-of-life for stretch.
• Mark CVE-2021-32610/php-pear as no-dsa for stretch.
• Mark CVE-2017-9525/systemd-cron as no-dsa for stretch.
• Mark CVE-2021-37701/node-tar as end-of-life for stretch.
• Mark CVE-2021-37712/node-tar as end-of-life in stretch.
• Mark CVE-2021-3750/qemu as postponsed for jessie.
• Mark CVE-2021-27511/prototypejs as postponsed for jessie.
• Mark CVE-2021-23437/pillow as postponed for stretch and jessie.
• Auto EOL ed gpac, cacti, openscad, cgal, cyrus-imapd-2.4, libsolv, mosquitto, atomicparsley, gtkpod, node-tar, libapache2-mod-auth-openidc, neutron, inetutils and linux for jessie.
• Drop cpio from ela-needed; open issues don t warrant an ELA.
• Attended monthly Debian LTS meeting.
• Answered questions (& discussions) on IRC (#debian-lts and #debian-elts).
• General and other discussions on LTS private and public mailing list.

---

Until next time.
:wq for today.

Focus This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

• zxing-cpp: use system deps, fix Python tests
• git-remote-hg: gitignore
• hexedit: add test
• libpst: fix header detection
• webdl: fix SBS video params
• libusbgx: fix alignment
• tracegraph: API porting
• pyemd: cleanup code, packaging
• pytest-rerunfailures: cleanup code, build, packaging
• librecaptcha: cleanup Python packaging, links
• check-all-the-things: update MIME types, command verbosity (1 2), TODO items (1 2 3)
• devscripts: multiple packages for grep-excuses --wipnity
• debhelper: typo, cmake ctest argument order
• Debian website: sync langs
• Debian /updates to -security transition: 1 2 3 4 5 6
• Debian release info hardcoding: 1 2
• Debian QA services: cleanup
• Debian package uploads: libpst, foxtrotgps, sptag (1 2 3), pytest-rerunfailures, git-remote-hg (1 2), librecaptcha
• Debian wiki pages: 900000thBugContest, ALSA, AlyssaRosenzweig, BugTriage, DebianDay/2021/India/Online, DebianEdu/Status/Bullseye, FrontPage, Fvwm, HostingControlPanels, InstallingDebianOn (Microsoft/Windows/SubsystemForLinux, Thinkpad/L13 Gen 2/buster), ReleaseParty, SourcesList, Status/Stable, SuitesAndReposExtension, Teams (Apt, ReleaseTeam/ReleaseCheckList), ungoogled-chromium
• FOSSjobs wiki pages: Resources
• Wikipedia: Display_Data_Channel

Issues

• Crash in eog
• Web request mishandling in fossjobs
• AppArmor issues in torbrowser-launcher
• Broken links in Debian backports NEW, GMime
• Missing link in GMime
• Outdated links in crouton, breezy
• Broken pages in GNOME Bugzilla archive
• Features in debiman (1 2), gokart, debhelper, shellcheck (1 2)
• New versions of umatrix, doc8, git-remote-hg
• New repo needed for misspelling research
• Hardcoded Debian release info updates for debiman (1 2)
• Syntax error in freefem++
• Adoption needed for veles
• Underlinking in libuemf0
• Conffile removal needed in wireplumber
• Build issues in SPTAG (1 2), plocate, zxing-cpp
• Git tag conflict in git-remote-hg
• Dependency updates for catfish, education-common, cruft-ng
• Cruft deposited by qgis-providers
• More binary packages needed in zxing-cpp (1 2)
• Malicious Python module on PyPI

Review

• Spam: reported 1 sourceware bug report, 2 Debian bug reports and 86 Debian mailing list posts
• Debian packages: reviewed iotop-c and approved DM upload
• Debian wiki: RecentChanges for the month
• Debian BTS usertags: changes for the month
• Debian screenshots:
  • approved aptitude faenza-icon-theme gnome-icon-theme-yasis gnome-illustrious-icon-theme gnome-power-manager juffed mm3d mystiq shiki-brave-theme
  • rejected unetbootin (random web page), tuxmath (photograph not screenshot)

Administration

• Debian servers: expand LV, fix debbugs config
• Debian wiki: unblock IP addresses, approve accounts
• Debian QA services: deploy changes

Communication

• Initiate discussion about Debian source tarball choices

Sponsors The pyemd, pytest-rerunfailures, libpst, sptag, librecaptcha work was sponsored by my employer. All other work was done on a volunteer basis.

Safe upgrades A central idea behind Candid is that services evolve over time, and so also their interfaces evolve. As they do, it is desirable to keep the interface usable by clients who have not been updated. In particular on a blockchainy platform like the Internet Computer, where some programs are immutable and cannot be changed to accommodate changes in the interface of the services they use, this is of importance. Therefore, Candid defines which changes to an interface are guaranteed to be backward compatible. Of course it s compatible to add new methods to a service, but some changes to a method signature can also be ok. For example, changing

service A1 :  
  get_value : (variant   current; previous : nat  )
    -> (record   value : int; last_change : nat  )
 

to

service A2 :  
  get_value : (variant   current; previous : nat; default  )
    -> (record   value : int; last_change : nat; committed : bool  )
 

is fine: It doesn t matter that clients that still use the old interface don t know about the new constructor of the argument variant. And the extra field in the result record will silently be ignored by old clients. In the Candid spec, this relation is written as A2 <: A1 (note the order!), and the formal footing this builds on is subtyping. We thus say that it is safe to upgrade a service to a subtype , and that A2 s interface is a subtype of A1 s. In small examples, I often use nat and int, because every nat is also a int, but some int values are not not nat values, namely the negative ones. We say nat is a subtype of int, nat <: int. So a function that in the first returns a int can in the new version return a nat without breaking old clients. But the other direction doesn t work: If the old version s method had a return type of nat, then changing that to int would be a breaking change, as old clients would not be prepared to handle negative numbers. Note that arguments of function types follow different rules. In fact, the rules are simply turned around, and in argument position (also called negative position ), we can evolve the interface to accept supertypes. Concretely, a function that originally expected an nat can be changed to expect an int, but the other direction doesn t work, because there might still be old clients around that send negative numbers. This reversing-of-the-rules is called contravariance. The vision is that the developer s tooling will warn the developer before they upgrade the code of a running service to a type that is not a subtype of the old interface. Let me stress, though, that all this is about not breaking existing clients that use the old interface. It does not mean that a client developer who fetches the new interface for your service won t have to change their code to make his programs compile again.

Higher order composition So far, we considered the simple case of one service with a bunch of clients, i.e. the first order situation. Things get much more interesting if we have multiple services that are composed, and that pass around references to each other, and we still want everything to be nicely typed and never fail even as we upgrade services. Therefore, Candid s type system can express that a service s method expects or a returns a reference to another service or method, and the type that this service or method should have. For example

service B :   add_listener : (text, func (int) -> ()) -> ()  

says that the service with interface B has a method called add_listener which takes two arguments, a plain value of type text and a reference to a function that itself expects a int-typed argument. The contravariance of the subtyping rules explained above also applies to the types of these function reference. And because the int in the above type is on the left of two function arrows, the subtyping rule direction flips twice, and it is therefore safe to upgrade the service to accept the following interface:

service B :   add_listener : (text, func (nat) -> ()) -> ()  

Soundness theorem and Coq proof The combination of these higher order features and the safe upgrade mechanism is maybe the unique selling point of Candid, but also what made its design quite tricky sometimes. And although the conventional subtyping rules work well, we wanted to do some less conventional things (more on that below), and more than once thought we had a good solution, only to notice that it did not hold water in complicated higher-order cases. But what does it mean to hold water? I felt the need to precisely define a soundness criteria for higher order interface description languages, which you can find in the document An IDL Soundness Proposition. This is a general framework which you can instantiate with your concrete IDL language and rules, and then it tells you what you have to prove to consider your language to be sound. Intuitively, it simulates all possible ways in which services can be defined and upgraded, and in which they can pass around references to each other, and the soundness property is that then that for all messages sent between services, they can always be understood. The document also shows, in general, that any system that builds on canonical subtyping in sound, as expected. That is still a generic theorem that you can instantiate with a concrete system, but now there is less to do. Because these proofs can get tricky in corner cases, it is valuable to mechanize them in an interactive theorem prover and have the computer check the proofs. So I have created a Coq formalization that defines the soundness criteria, including the reduction to canonical subtyping. It also defines MiniCandid, a greatly simplified variant of Candid, proves various properties about it (transitivity etc.) and finally instantiates the soundness theorem. I am particularly fond of the use of coinduction to model the equirecursive types, and the use of named cases, as we know them from Isabelle, to make the proofs a bit more readable and maintainable. I am less fond of how much work it seems to be to extend MiniCandid with more of Candid s types. Already adding vec was more work than it s worth it, and I defensively blame Coq s not-so-great support for nested recursion. The soundness relies on subtyping, and that is all fine and well as long as we stick to canonical rules. Unfortunately, practical considerations force us to deviate from these a bit, as we see in the next post of this series.

• [1] https://tinyurl.com/yhrq5887
• [2] https://tinyurl.com/yhmog5ex
• [3] https://tinyurl.com/ygnmjm9j
• [4] https://www.bitsnbites.eu/three-fundamental-flaws-of-simd/
• [5] https://www.dailydot.com/debug/ghostezra-telegram-fued-qanon/
• [6] https://tinyurl.com/yhvganlc
• [7] https://tinyurl.com/yfljnlq5
• [8] https://tinyurl.com/yft5qwbc
• [9] https://tinyurl.com/yjjnnyfu
• [10] https://tinyurl.com/yjxq8c2v
• [11] https://tinyurl.com/y7hhrxuz
• [12] https://tinyurl.com/yj33txpj

• a language to describe the interface of a service, with named methods, arguments and results,
• a type system for these arguments and results that maps to many possible implementation languages, thus enabling interoperability between these languages,
• a wire format (encoding) of values of these types,
• a notion of safe upgrade that allows service developers to evolve their interface with the assurance that no clients break, even in the presence of higher-order composition,
• a meta-theory of safe interface description languages and a formal proof that Candid has these properties, and
• a bunch of tools and libraries.

Blog post series overview

1. The rough idea (this post) Announcing the blog post series, and very briefly outlining what an Interface Description Language even is.
2. Safe higher-order upgrades Every can do first-order interface description languages. What makes Candid special is its support for higher-order service composition. This post also contains some general meta-theory and a Coq proof!
3. Opt is special Extending records in argument position is surprisingly tricky. Lean why, and how we solved it.
4. Language integration The point of Candid is inter-op, so let s look the various patterns of connecting Candid to your favorite programming language.
5. Quirks The maybe most interesting post in this series: Candid has a bunch of quirks (hashed field names, no tuples but sequnces, references that nobody used, and more). I ll explain some of them, why they are there, and what else we could have done. Beware: This post will be opinionated.

The rough idea The idea of an interface description language is easy to begin with. Something like

service A :  
  add : (int) -> ();
  get : () -> (int);
 

clearly communicates that a service called A provides two methods add and get, that add takes an integer as an argument, and that get returns such a number. Of course, this only captures the outer shape of the service, but not what it actually does we might assume it implements a counter of sorts, but that is not part of the Candid interface description. In order to now use the service, we also need a (low-level) transport mechanism. Candid itself does not specify that, but merely assumes that it exists, and is able to transport the name of the invoked method and a raw sequence of bytes to the service, and a response (again a raw sequence of bytes) back. Already on the Internet Computer we have two distinct transport mechanisms used are external calls via an HTTP-based RPC interface and on-chain inter-canister calls. Both are handled by the service in the same way. Here we can see that Candid succeeds in abstracting over differences in the low-level transport mechanism. Because of this abstraction, it is possible to use Candid over other transportation mechanisms as well (conventional HTTPS, E-Mail, avian carrier). I think Candid has potential there as well, so even if you are not interested in the Internet Computer, this post may be interesting to you. The translation of argument and result values (e.g. numbers of type int) to the raw sequence of bytes is specified by Candid, defining a wire format. For example, the type int denotes whole numbers of arbitrary size, and they are encoded using the LEB128 scheme. The Candid wire format also contains a magic number (DIDL, which is 0x4449444c in hex) and a type description (more on that later). So when passing the value 2342 to the add, the raw bytes transported will be 0x4449444c00017da612. Of course we want to transfer more data than just integers, and thus Candid supports a fairly complete set of common basic types (nat, int, nat8, nat16, nat32, nat64, int8, int16, int32, int64, float32, float64, bool, text, blob) and composite types (vectors, optional values , records and variants). The design goal is to provide a canonical set of types enough to express most data that you might want to pass, but no more than needed, so that different host languages can support these types easily. The Candid type system is structural. This means that two types are the same when they are defined the same way. Imagine two different services defining the same

type User = record   name : text; user_id : nat  

then although User is defined in two places, it s still the same type. In other words, these name type definitions are always just simple aliases, and what matters is their right-hand side. Because Candid types can be recursive (e.g. type Peano = opt Peano), this means we have an equirecursive type system, which makes some things relatively hard. So far, nothing too special about Candid compared to other interface definition languages (although a structural equirecursive type system is already something). In the next post we look at reference types, and how such higher order features make Candid an interesting technology.

• Domain migration to debian.net subdomain - fabre.debian.net
• Sponsorship for fabre.debian.net
• Useful use cases of fabre.debian.net

LTS This is my sixth month of working for LTS. I was assigned 12 hrs and worked all of them.

Released DLAs

1. DLA 2742-1 ffmpeg_7:3.2.15-0+deb9u3
   • CVE-2020-22036: A heap-based Buffer Overflow vulnerability in filter_intra at libavfilter/vf_bwdif.c, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22032: A heap-based Buffer Overflow vulnerability in gaussian_blur, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22031: A Heap-based Buffer Overflow vulnerability in filter16_complex_low, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22028: Buffer Overflow vulnerability in filter_vertically_8 at libavfilter/vf_avgblur.c, which could cause a remote Denial of Service.
   • CVE-2020-22026: Buffer Overflow vulnerability exists in the config_input function at libavfilter/af_tremolo.c, which could let a remote malicious user cause a Denial of Service.
   • CVE-2020-22025: A heap-based Buffer Overflow vulnerability exists in gaussian_blur at libavfilter/vf_edgedetect.c, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22023: A heap-based Buffer Overflow vulnerabililty exists in filter_frame at libavfilter/vf_bitplanenoise.c, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22022: A heap-based Buffer Overflow vulnerability exists in filter_frame at libavfilter/vf_fieldorder.c, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22021: Buffer Overflow vulnerability at filter_edges function in libavfilter/vf_yadif.c, which could let a remote malicious user cause a Denial of Service.
   • CVE-2020-22020: Buffer Overflow vulnerability in the build_diff_map function in libavfilter/vf_fieldmatch.c, which could let a remote malicious user cause a Denial of Service.
   • CVE-2020-22016: A heap-based Buffer Overflow vulnerability at libavcodec/get_bits.h when writing .mov files, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22015: Buffer Overflow vulnerability in mov_write_video_tag due to the out of bounds in libavformat/movenc.c, which could let a remote malicious user obtain sensitive information, cause a Denial of Service, or execute arbitrary code.
   • CVE-2020-21041: Buffer Overflow vulnerability exists via apng_do_inverse_blend in libavcodec/pngenc.c, which could let a remote malicious user cause a Denial of Service
   • CVE-2021-3566: The tty demuxer did not have a read_probe function assigned to it. By crafting a legitimate ffconcat file that references an image, followed by a file the triggers the tty demuxer, the contents of the second file will be copied into the output file verbatim (as long as the -vcodec copy option is passed to ffmpeg).
   • CVE-2021-38114: libavcodec/dnxhddec.c does not check the return value of the init_vlc function. Crafted DNxHD data can cause unspecified impact.
2. DLA 2742-2 ffmpeg_7:3.2.15-0+deb9u4 During the backporting of one of patches in CVE-2020-22021 one line was wrongly interpreted and it caused the regression during the deinterlacing process. Thanks to Jari Ruusu for the reporting the issue and for the testing of prepared update.

Other LTS-related work

• Analyzed CVE-2020-22027 and marked as ignored for stretch. Original patch is not appliable. The issue is reproducible though.
• Analyzed CVE-2020-22019 and marked as ignored for stretch.
• Analyzed CVE-2020-22017 and marked as ignored for stretch.
• Updated tomcat8-LTS-salsa-repo to tne newer 8.5.54-0+deb9u7 version.
• Minor WIKI update.
• Frontdesk duties CW33/2021. It was my fist attempt to do such kind of task. I triaged apache2 and gitit.
• Analyzed CVEs in firmware-nonfree.
• Started to work on rustc.

LTS-Meeting

• I attended the Debian LTS team Jitsi-meeting (though the connection was extremely bad).
• Partly participated in preparation of Debconf21 BoF Funding Projects to Improve Debian .

Debian Science Team

• Partly participated in Debconf21 Debian Science BoF.

Other FLOSS activities

• Reviewed many merge requests in Yade open source project, merge some of them.

A song for Debconf21 ["What shall we do with the drunken sailor"]

What shall we do with the online Debconf?What shall we do with the online Debconf?What shall we do with the online Debconf?

Earl-y in the morning

Close it up as we agreed it

Save each script in case we need it

Work out how we best live-feed itNext year s Debconf s dawning

Next year s Kosovo and Pristina

This virtual Debconf needs no cleaner

Hope when COVID s gone we re keener

To meet up every morning

Thanks to the Debconf orga team

Thanks to those who Loopy meme

Things are not always as they seeem

In virtual Debconf s morning

Bullseye s out its share is rising

Debconf s fun and quite surprising

Linux 30 :)

Yours and my thing - Debian s 28

Thanks to all who video d sessions

Debconf T-shirts - prized possessions

Debcamp bug-fixed some regressions

Now onto next year!

A closing song for Debconf 21 [Frere Jacques/Brueder Martin]

DebConf21

Virtual DebConf s

Now all through

Closed for you

Kosovo is next year

See you in Pristina!!

DebConf 22

Twenty twenty-two

The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. Linux distributions like Debian fulfill an important function in the FOSS ecosystem - they are system integrators that take existing free and open source software projects and adapt them where necessary to work well together. They also make it possible for users to install more software in an easy and consistent way and with some degree of quality control and review. One of the consequences of this model is that the distribution package often lags behind upstream releases. This is especially true for distributions that have tighter integration and standardization (such as Debian), and often new upstream code is only imported irregularly because it is a manual process - both updating the package, but also making sure that it still works together well with the rest of the system. The process of importing a new upstream used to be (well, back when I started working on Debian packages) fairly manual and something like this:

• Go to the upstream s homepage, find the tarball and signature and verify the tarball
• Make modifications so the tarball matches Debian s format
• Diff the original and new upstream tarballs and figure out whether changes are reasonable and which require packaging changes
• Update the packaging, changelog, build and manually test the package
• Upload

Ecosystem Improvements However, there have been developments over the last decade that make it easier to import new upstream releases into Debian packages.

Uscan and debian QA watch Uscan and debian/watch have been around for a while and make it possible to find upstream tarballs. A debian watch file usually looks something like this:

1 2	version=4 http://somesite.com/dir/filenamewithversion.tar.gz

The QA watch service regularly polls all watch locations in the archive and makes the information available, so it s possible to know which packages have changed without downloading each one of them.

Git Git is fairly ubiquitous nowadays, and most upstream projects and packages in Debian use it. There are still exceptions that do not use any version control system or that use a different control system, but they are becoming increasingly rare. [1]

debian/upstream/metadata DEP-12 specifies a file format with metadata about the upstream project that a package was based on. In particular relevant for our case is the fact it has fields for the location of the upstream version control location. debian/upstream/metadata files look something like this:

1 2 3

--- Repository: https://www.dulwich.io/code/dulwich/ Repository-Browse: https://www.dulwich.io/code/dulwich/

While DEP-12 is still a draft, it has already been widely adopted - there are about 10000 packages in Debian that ship a debian/upstream/metadata file with Repository information.

Autopkgtest The Autopkgtest standard and associated tooling provide a way to run a defined set of tests against an installed package. This makes it possible to verify that a package is working correctly as part of the system as a whole. ci.debian.net regularly runs these tests against Debian packages to detect regressions.

Vcs-Git headers The Vcs-Git headers in debian/control are the equivalent of the Repository field in debian/upstream/metadata, but for the packaging repositories (as opposed to the upstream ones). They ve been around for a while and are widely adopted, as can be seen from zack s stats: The vcswatch service that regularly polls packaging repositories to see whether they have changed makes it a lot easier to consume this information in usable way.

Debhelper adoption Over the last couple of years, Debian has slowly been converging on a single build tool - debhelper s dh interface. Being able to rely on a single build tool makes it easier to write code to update packaging when upstream changes require it.

Debhelper DWIM Debhelper (and its helpers) increasingly can figure out how to do the Right Thing in many cases without being explicitly configured. This makes packaging less effort, but also means that it s less likely that importing a new upstream version will require updates to the packaging. With all of these improvements in place, it actually becomes feasible in a lot of situations to update a Debian package to a new upstream version automatically. Of course, this requires that all of this information is available, so it won t work for all packages. In some cases, the packaging for the older upstream version might not apply to the newer upstream version. The Janitor has attempted to import a new upstream Git snapshot and a new upstream release for every package in the archive where a debian/watch file or debian/upstream/metadata file are present. These are the steps it uses:

• Find new upstream version

  • If release, use debian/watch - or maybe tagged in upstream repository
  • If snapshot, use debian/upstream/metadata s Repository field
  • If neither is available, use guess-upstream-metadata from upstream-ontologist to guess the upstream Repository

• Merge upstream version into packaging repository, possibly importing tarballs using pristine-tar
• Update the changelog file to mention the new upstream version

• Run some checks to ensure there are no unintentional changes, e.g.:

  • Scan diff between old and new for surprising license changes

    • Today, abort if there are any - in the future, maybe update debian/copyright

  • Check for obvious compatibility breaks - e.g. sonames changing

• Attempt to update the packaging to reflect upstream changes

  • Refresh patches

• Attempt to build the package with deb-fix-build, to deal with any missing dependencies
• Run the autopkgtests with deb-fix-build to deal with missing dependencies, and abort if any tests fail

Results When run over all packages in unstable (sid), this process works for a surprising number of them.

Fresh Releases For fresh-releases (aka imports of upstream releases), processing all packages maintained in Git for which QA watch reports new releases (about 11,000): That means about 2300 packages updated, and about 4000 unchanged.

Fresh Snapshots For fresh-snapshots (aka imports of latest Git commit from upstream), processing all packages maintained in Git (about 26,000): Or 5100 packages updated and 2100 for which there was nothing to do, i.e. no upstream commits since the last Debian upload. As can be seen, this works for a surprising fraction of packages. It s possible to get the numbers up even higher, by both improving the tooling, the autopkgtests and the metadata that is provided by packages.

1
2
3
4
5
6

echo deb "[arch=amd64 signed-by=/usr/share/keyrings/debian-janitor-archive-keyring.gpg]" \
    https://janitor.debian.net/ fresh-snapshots main   sudo tee /etc/apt/sources.list.d/fresh-snapshots.list
echo deb "[arch=amd64 signed-by=/usr/share/keyrings/debian-janitor-archive-keyring.gpg]" \
    https://janitor.debian.net/ fresh-releases main   sudo tee /etc/apt/sources.list.d/fresh-releases.list
sudo curl -o /usr/share/keyrings/debian-janitor-archive-keyring.gpg https://janitor.debian.net/pgp_keys
apt update

1

apt install -t fresh-snapshots r-cran-roxygen2