>>> from genie.conf.base import Device
>>> device = Device("router", os="iosxr")
>>> # Hack to parse outputs without connecting to a device
>>> device.custom.setdefault("abstraction",  )["order"] = ["os", "platform"]
>>> cmd = "show route ipv4 unicast"
>>> output = """
... Tue Oct 29 21:29:10.924 UTC
...
... O    10.13.110.0/24 [110/2] via 10.12.110.1, 5d23h, GigabitEthernet0/0/0/0.110
... """
>>> device.parse(cmd, output=output)
 'vrf':  'default':  'address_family':  'ipv4':  'routes':  '10.13.110.0/24':  'route': '10.13.110.0/24',
       'active': True,
       'route_preference': 110,
       'metric': 2,
       'source_protocol': 'ospf',
       'source_protocol_codes': 'O',
       'next_hop':  'next_hop_list':  1:  'index': 1,
          'next_hop': '10.12.110.1',
          'outgoing_interface': 'GigabitEthernet0/0/0/0.110',
          'updated': '5d23h' 

1. The data models used are dependent on the vendor and OS, despite the documentation saying otherwise. For example, the data model used for IPv4 interfaces is different between NX-OS and IOS-XR.
2. The parsers rely on line-by-line regular expressions to extract data and some Python code as glue. This is fragile and may break silently.

  Loopback10 is Up, ipv4 protocol is Up
    Vrf is default (vrfid 0x60000000)
    Internet protocol processing disabled
  Loopback30 is Up, ipv4 protocol is Down [VRF in FWD reference]
    Vrf is ran (vrfid 0x0)
    Internet address is 203.0.113.17/32
    MTU is 1500 (1500 is available to IP)
    Helper address is not set
    Directed broadcast forwarding is disabled
    Outgoing access list is not set
    Inbound  common access list is not set, access list is not set
    Proxy ARP is disabled
    ICMP redirects are never sent
    ICMP unreachables are always sent
    ICMP mask replies are never sent
    Table Id is 0x0

 
  "Loopback10":  
    "int_status": "up",
    "oper_status": "up",
    "vrf": "ran",
    "vrf_id": "0x0",
    "ipv4":  
      "203.0.113.17/32":  
        "ip": "203.0.113.17",
        "prefix_length": "32"
       ,
      "mtu": 1500,
      "mtu_available": 1500,
      "broadcast_forwarding": "disabled",
      "proxy_arp": "disabled",
      "icmp_redirects": "never sent",
      "icmp_unreachables": "always sent",
      "icmp_replies": "never sent",
      "table_id": "0x0"
     
   
 

• Remove support for parsing CVS $Id$ strings and emitting \id commands.
• Stop wrapping output text.
• Fix internal section links.
• Fix conversion of nested =over blocks.
• Fix conversion of =over blocks without an =item.
• Fix links to headings containing markup.
• Fix metacharacter escaping in headings, contents, and navbar.
• Fix handling of =for thread and =begin thread.
• Fix non-breaking spaces in navbar section names.
• Fix internal links whose anchors are wrapped.
• Output the document before erroring out if POD errors were seen.
• Suppress \signature if the input document was empty.
• Add bugtracker, homepage, and repository package metadata.

Changes in RcppSMC version 0.2.5 (2021-09-09)

• Compilation under Solaris is aided via std::pow use (Dirk in #65 fixing #64)

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

• Introduction to Free Software and Debian,
• Packaging, policy, and Debian infrastructure,
• Systems administration, automation and orchestration,
• Cloud and containers,
• Security,
• Community, diversity, local outreach and social context,
• Internationalization, Localization and Accessibility,
• Embedded and Kernel,
• Debian Blends and Debian derived distributions,
• Debian in Arts and Science
• and other.

Changes in version 0.1.6 (2021-09-07)

• The C++17 dependency was stated more clearly in the DESCRIPTION file (Dirk)
• The simdjson version was updated to release 1.0.0 (Daniel Lemire in #70)

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

The record extension problem Initially, the upgrade rules of Candid were a straight-forward application of the canonical subtyping rules. This worked and was sound, but it forbid one very commonly requested use case: Extending records in argument position. The subtyping rule for records say that

   record   old_field : nat; new_field : int  
<: record   old_field : nat  

or, in words, that subtypes can have more field than supertypes. This makes intuitive sense: A user of a such a record value that only expects old_field to be there is not bothered if suddenly a new_field appears. But a user who expects new_field to be there is thrown off if it suddenly isn t anymore. Therefore it is ok to extend the records returned by a service with new fields, but not to extend the record in your methods s argument types. But users want to extend their record types over time, also in argument position! In fact, they often want to extend them in both positions. Consider a service with the following interface, where the CUser record appears both in argument and result position:

type CUser = record   login : text; name : text  ;
service C :  
  register_user : (CUser) -> ();
  get_user_data : (text) -> (CUser);
 

It seems quite natural to want to extend the record with a new field (say, the age of the user). But simply changing the definition of CUser to

type CUser = record   login : text; name : text; age : nat  

is not ok, because now register_user requires the age field, but old clients don t provide it. So how can we allow developers to make changes like this (because they really really want that), while keeping the soundness guarantees made by Candid (because we really really want that)? This question has bothered us for over two years, and we even created a meta issue that records the dozen approached we considered, discussed and eventually ditched.

Dynamic subtyping checks in opt I will spare you the history lesson, though, and explain the solution we eventually came up with. In the example above it seems unreasonable to let the developer add a field age of type nat. Since there may be old clients around, the service unavoidably has to deal with records that don t have an age field. If the code expects an age value, what should it do in that case? But one can argue that changing the record type to

type CUser = record   login : text; name : text; age : opt nat  

could work: If the age field is present, use the value. If the field is absent, inject a null value during decoding. In the first-order case, this rather obvious solution would work just fine, and we d be done. But Candid aims to solve the higher order case, and I said things get tricky here, didn t I? Consider another, independent service D with the following interface:

type DUser = record   login : text; name : text  ;
service D :  
  on_user_added : (func (DUser) -> ()) -> ()
 

This service has a method that takes a method reference, presumably with the intention of calling it whenever a new user was added to service D. And because the types line up nicely, we can compose these two services, by once calling D.on_user_added(C.register_user), so that from now on D calls C.register_user( ). These kind of service mesh-ups are central to the vision of the Internet Computer! But what happens if the services now evolve their types in different ways, e.g. towards

type CUser = record   login : text; name : text; age : opt nat  

and

type DUser = record   login : text; name : text; age : opt variant   under_age; adult  

Individually, these are type changes that we want to allow. But now the call from D to C may transmit a value of record ; age = opt variant under_age when C expects a natural number! This is precisely what we want to prevent by introducing types, and it seems we have lost soundness. The best solution we could find is to make the opt type somewhat special, and apply these extra rules when decoding at an expected type of opt t.

• If this was a record field, and the provided record value doesn t even have a field of that name, don t fail but instead treat this as null. This handles the first-order example above.
• If there is a value, look at its type (which, in Candid, is part of the message).
  • If it is opt t' and t' <: t, then decode as normal.
  • If it is opt t' but t' <: t does not hold, then treat that as null. This should only happen in these relatively obscure higher-order cases where services evolved differently and incompatibly, and makes sure that the calls that worked before the upgrades will continue to work afterwards. It is not advisable to actually use that rule when changing your service s interface. Tools that assist the developer with an upgrade check should prevent or warn about the use of this rule.
• Not strictly required here, but since we are making opt special anyways: If its type t' is not of the form opt , pretend it was opt t', and also pretend that the given value was wrapped in opt. This allows services to make record field in arguments that were required in the old version optional in a new version, e.g. as a way to deprecated them. So it is mildly useful, although I can report that it makes the meta-theory and implementation rather complex, in particular together with equirecursion and beasts like type O = opt O. See this discussion for a glimpse of that.

In the above I stress that we look at the type of the provided value, and not just the value. For example, if the sender sends the value opt vec at type opt vec nat, and the recipient expects opt vec bool, then this will decode as null, even though one could argue that the value opt vec could easily be understood at type opt vec bool. We initially had that, but then noticed that this still breaks soundness when there are references inside, and we have to do a full subtyping check in the decoder. This is very unfortunate, because it makes writing a Candid decoder a noticeably harder task that requires complicated graph algorithms with memoization (which I must admit Motoko has not implemented yet), but it s the least bad solution we could find so far.

Isn t that just dynamic typing? You might have noticed that with these rules, t <: opt t' holds for all types t and t'. In other words, every opt is a top type (like reserved), thus all optional types are equivalent. One could argue that we are throwing all of the benefits of static typing over board this way. But it s not quite as bad: It s true that decoding Candid values now involves a dynamic check inserting null values in certain situations, but as long as everyone plays by the rules (i.e. upgrades their services according to the Candid safe upgrade rules, and heeds the warning alluded to above), these certain situations will only happen in the obscurest of cases. It is, by the way, not material to this solution that the special subtyping behavior is applied to the opt type, and a neighboring point in the design space would be a dedicated upgraded t type constructor. That would allow developers to use the canonical opt type with the usual subtyping rules and communicate their intent ( clients should consider this field optional vs. old clients don t know about this field, but new clients should use it ) more cleanly at the expense of having more non-canonical types in the type system. To see why it is convenient if Candid has mostly just the normal types, read the next post, which will describe how Candid can be integrated into a host language.

The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. As covered in my post from last week, the Janitor now regularly tries to import new upstream git snapshots or upstream releases into packages in Sid.

webhooks for upstream repositories At the moment there is no service in Debian (yet - perhaps this is something that vcswatch or a sibling service could also do?) that scans upstream repositories for changes. However, if you maintain an upstream package, you can use a webhook to notify the janitor that commits have been made to your repository, and it will create a new package in fresh-snapshots. Webhooks from the following hosting site software are currently supported:

• GitHub
• Gitlab
• Launchpad
• Gitea
• Gogs

You can simply use the URL https://janitor.debian.net/ as the target for hooks. There is no need to specify a secret, and the hook can either use a JSON or form encoding payload. The endpoint should tell you whether it understood a webhook request, and whether it took any action. It s fine to submit webhooks for repositories that the janitor does not (yet) know about.

GitHub For GitHub, you can do so in the Webhooks section of the Settings tab. Fill the form as shown below and click on Add webhook:

GitLab On GitLab instances, you can find the Webhooks tab under the Settings menu for each repository (under the gear symbol). Fill the form in as shown below and click Add Webhook:

Launchpad For Launchpad, go to the repository (for Git) web view and click Manage Webhooks. From there, you can add a new webhook; fill the form in as shown below and click Add Webhook:

Changes in tidyCpp version 0.0.4 (2021-09-05)

• Minor updates to DESCRIPTION
• New snippet rollminmaxExample with simple vector use
• New class NumVec motivated from rolling min/max example
• Expand the vignette with C++ example based on NumVec
• Add a table of contents to the vignette

Changes in tidyCpp version 0.0.3 (2021-03-07)

• Switch CI use to r-ci
• Protect nil object definition

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Changes in RQuantLib version 0.4.13 (2021-09-02)

• Changes in RQuantLib code:
  • Small updates to DESCRIPTION and README.md
  • Switch CI use to r-ci
  • An eleven-year old typo was correct in a manual page (Dirk in #156 fixing #155 with thanks to @klin133)
  • Daycount convention calls deprecated in QuantLib 1.23 were updated (Dirk in #157)
  • Updated remaining http:// URLs in documentation to https://

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Changes in RcppSMC version 0.2.4 (2021-09-01)

• Multiple Sequential Monte Carlo extensions (Ilya Zarubin as part of Google Summer of Code 2021)
  • Provide informative user output (convergence diagnostics) for PMMH example #50 (Ilya in #50 and #52 addressing #25, bullet point 5)
  • Support for tracking of ancestral lines for base sampler class (Ilya in #56)
  • Support for conditional SMC via derived conditionalSampler class (Ilya in #60)
• Add URL and BugReports to DESCRIPTION (Dirk in #53)

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

error: symbol grub_is_lockdown not found

update-grub2

dpkg-reconfigure -plow grub-efi-amd64

Force extra installation to the EFI removable media path?

Some EFI-based systems are buggy and do not handle new bootloaders correctly. If you force an extra installation of GRUB to the EFI removable media path, this should ensure that this system will boot Debian correctly despite such a problem. However, it may remove the ability to boot any other operating systems that also depend on this path. If so, you will need to make sure that GRUB is configured successfully to be able to boot any other OS installations correctly.

Update NVRAM variables to automatically boot into Debian?

GRUB can configure your platform's NVRAM variables so that it boots into Debian automatically when powered on. However, you may prefer to disable this behavior and avoid changes to your boot configuration. For example, if your NVRAM variables have been set up such that your system contacts a PXE server on every boot, this would preserve that behavior.

eddy@aptonia:/ $ cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
r8169

DRV_NAME=rtl8821ce
DRV_VERSION=v5.5.2_34066.20200325

cp -r . /usr/src/$ DRV_NAME -$ DRV_VERSION

dkms add -m $ DRV_NAME -v $ DRV_VERSION

dkms build -m $ DRV_NAME -v $ DRV_VERSION -k 5.10.0-0.bpo.8-amd64
dkms install -m $ DRV_NAME -v $ DRV_VERSION -k 5.10.0-0.bpo.8-amd64

dkms build rtl8821ce/v5.5.2_34066.20200325 -k 4.19.0-17-amd64
dkms install rtl8821ce/v5.5.2_34066.20200325 -k 4.19.0-17-amd64

• https://wiki.debian.org/DebianEvents/de/2021/DebianReunionHamburg#Dates_and_format_of_the_event
• https://wiki.debian.org/DebianEvents/de/2021/DebianReunionHamburg#Covid_things
• https://wiki.debian.org/DebianEvents/de/2021/DebianReunionHamburg#Call_for_papers
• https://wiki.debian.org/DebianEvents/de/2021/DebianReunionHamburg#Registration - registration is mandatory for this event!

Debian

This was my 32nd month of actively contributing to Debian. I became a DM in late March 2019 and a DD on Christmas 19! \o/ Tough month but I mostly spent on it churning through the immense backlog. But that somewhat backfired and I have even more backlog than ever. :D Anyway, I did the following stuff in Debian:

Uploads and bug fixes:

• ruby3.0 (3.0.0-2) - Upload to unstable! \o/

Other $things:

• Mentoring for newcomers.
• Moderation of -project mailing list.

---

Ubuntu

This was my 7th month of actively contributing to Ubuntu. Now that I ve joined Canonical to work on Ubuntu full-time, there s a bunch of things I do! \o/ I mostly worked on different things, I guess. But mostly on packaging keylime and some Google Agents upload(s) and SRU(s). Also did a lot of reviewing, et al. I was too lazy to maintain a list of things I worked on so there s no concrete list atm. Maybe I ll get back to this section later or will start to list stuff from next month onward, as I ve been doing before. :D

---

Debian (E)LTS

Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success. And Debian Extended LTS (ELTS) is its sister project, extending support to the Jessie release (+2 years after LTS support). This was my twenty-third month as a Debian LTS and eleventh month as a Debian ELTS paid contributor.
I was assigned 23.75 hours for LTS and 40.00 hours for ELTS and worked on the following things:
(however, I only worked for 23.75h on ELTS work, thereby, carrying the rest to next month)

LTS CVE Fixes and Announcements:

• Issued DLA 2743-1, fixing CVE-2017-5715, for amd64-microcode.
  For Debian 9 stretch, these problems have been fixed in version 3.20181128.1~deb9u1.
• Issued DLA 2744-1, fixing the versioning issue, for usermode.
  For Debian 9 stretch, these problems have been fixed in version 1.109-1+deb9u1.
• Issued DLA 2750-1, fixing CVE-2019-20421, CVE-2021-3482, CVE-2021-29457, CVE-2021-29473, CVE-2021-31291, and CVE-2021-31292, for exiv2.
  For Debian 9 stretch, these problems have been fixed in version 0.25-3.1+deb9u3.

ELTS CVE Fixes and Announcements:

• Issued ELA 479-1, fixing CVE-2019-20421, CVE-2021-3482, CVE-2021-29457, CVE-2021-29473, CVE-2021-31291, and CVE-2021-31292, for exiv2.
  For Debian 8 jessie, these problems have been fixed in version 0.24-4.1+deb8u6.
• Noticed that there s a fallout of CVE-2021-3185, where an update was issued for gst-plugins-bad1.0, however, not for gst-plugins-bad0.10.
  Thanks to Sylvain s script, this came up and I prepped an update for that.
• Started to work on libjdom1-java s regression.

Other (E)LTS Work:

• Front-desk duty from 26-07 until 01-08 and from 30-08 until 05-09 for both LTS and ELTS.
• Triaged haproxy, ntfs-3g, and cyrus-imapd, and exiv2, ffmpeg, git, gpac, inetutils, mc, modsecurity-crs, node-object-path, php-pear, systemd-cron, node-tar, ruby2.3, gst-plugins-bad0.10, jsoup, libxstream-java, qemu, tomcat7, ruby2.1, prototypejs, pillow, cpio, and qtbase-opensource-src, and amd64-microcode.
• Mark CVE-2021-39240/haproxy as not-affected for stretch and jessie.
• Mark CVE-2021-39241/haproxy as not-affected for stretch and jessie.
• Mark CVE-2021-39242/haproxy as not-affected for stretch and jessie.
• Mark CVE-2021-33582/cyrus-imapd as no-dsa for stretch.
• Mark CVE-2020-18771/exiv2 as no-dsa for exiv2 for stretch.
• Mark CVE-2020-18899/exiv2 as no-dsa for exiv2 for stretch.
• Mark CVE-2021-38171/ffmpeg as postponed for stretch.
• Mark CVE-2021-40330/git as no-dsa for stretch and jessie.
• Mark CVE-2020-19481/gpac as ignored for stretch.
• Mark CVE-2021-40491/inetutils as no-dsa for stretch.
• Mark CVE-2021-36370/mc as no-dsa for stretch and jessie.
• Mark CVE-2021-35368/modsecurity-crs as no-dsa for stretch.
• Mark CVE-2021-23434/node-object-path as end-of-life for stretch.
• Mark CVE-2021-32610/php-pear as no-dsa for stretch.
• Mark CVE-2017-9525/systemd-cron as no-dsa for stretch.
• Mark CVE-2021-37701/node-tar as end-of-life for stretch.
• Mark CVE-2021-37712/node-tar as end-of-life in stretch.
• Mark CVE-2021-3750/qemu as postponsed for jessie.
• Mark CVE-2021-27511/prototypejs as postponsed for jessie.
• Mark CVE-2021-23437/pillow as postponed for stretch and jessie.
• Auto EOL ed gpac, cacti, openscad, cgal, cyrus-imapd-2.4, libsolv, mosquitto, atomicparsley, gtkpod, node-tar, libapache2-mod-auth-openidc, neutron, inetutils and linux for jessie.
• Drop cpio from ela-needed; open issues don t warrant an ELA.
• Attended monthly Debian LTS meeting.
• Answered questions (& discussions) on IRC (#debian-lts and #debian-elts).
• General and other discussions on LTS private and public mailing list.

---

Until next time.
:wq for today.

LTS This is my sixth month of working for LTS. I was assigned 12 hrs and worked all of them.

Released DLAs

1. DLA 2742-1 ffmpeg_7:3.2.15-0+deb9u3
   • CVE-2020-22036: A heap-based Buffer Overflow vulnerability in filter_intra at libavfilter/vf_bwdif.c, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22032: A heap-based Buffer Overflow vulnerability in gaussian_blur, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22031: A Heap-based Buffer Overflow vulnerability in filter16_complex_low, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22028: Buffer Overflow vulnerability in filter_vertically_8 at libavfilter/vf_avgblur.c, which could cause a remote Denial of Service.
   • CVE-2020-22026: Buffer Overflow vulnerability exists in the config_input function at libavfilter/af_tremolo.c, which could let a remote malicious user cause a Denial of Service.
   • CVE-2020-22025: A heap-based Buffer Overflow vulnerability exists in gaussian_blur at libavfilter/vf_edgedetect.c, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22023: A heap-based Buffer Overflow vulnerabililty exists in filter_frame at libavfilter/vf_bitplanenoise.c, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22022: A heap-based Buffer Overflow vulnerability exists in filter_frame at libavfilter/vf_fieldorder.c, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22021: Buffer Overflow vulnerability at filter_edges function in libavfilter/vf_yadif.c, which could let a remote malicious user cause a Denial of Service.
   • CVE-2020-22020: Buffer Overflow vulnerability in the build_diff_map function in libavfilter/vf_fieldmatch.c, which could let a remote malicious user cause a Denial of Service.
   • CVE-2020-22016: A heap-based Buffer Overflow vulnerability at libavcodec/get_bits.h when writing .mov files, which might lead to memory corruption and other potential consequences.
   • CVE-2020-22015: Buffer Overflow vulnerability in mov_write_video_tag due to the out of bounds in libavformat/movenc.c, which could let a remote malicious user obtain sensitive information, cause a Denial of Service, or execute arbitrary code.
   • CVE-2020-21041: Buffer Overflow vulnerability exists via apng_do_inverse_blend in libavcodec/pngenc.c, which could let a remote malicious user cause a Denial of Service
   • CVE-2021-3566: The tty demuxer did not have a read_probe function assigned to it. By crafting a legitimate ffconcat file that references an image, followed by a file the triggers the tty demuxer, the contents of the second file will be copied into the output file verbatim (as long as the -vcodec copy option is passed to ffmpeg).
   • CVE-2021-38114: libavcodec/dnxhddec.c does not check the return value of the init_vlc function. Crafted DNxHD data can cause unspecified impact.
2. DLA 2742-2 ffmpeg_7:3.2.15-0+deb9u4 During the backporting of one of patches in CVE-2020-22021 one line was wrongly interpreted and it caused the regression during the deinterlacing process. Thanks to Jari Ruusu for the reporting the issue and for the testing of prepared update.

Other LTS-related work

• Analyzed CVE-2020-22027 and marked as ignored for stretch. Original patch is not appliable. The issue is reproducible though.
• Analyzed CVE-2020-22019 and marked as ignored for stretch.
• Analyzed CVE-2020-22017 and marked as ignored for stretch.
• Updated tomcat8-LTS-salsa-repo to tne newer 8.5.54-0+deb9u7 version.
• Minor WIKI update.
• Frontdesk duties CW33/2021. It was my fist attempt to do such kind of task. I triaged apache2 and gitit.
• Analyzed CVEs in firmware-nonfree.
• Started to work on rustc.

LTS-Meeting

• I attended the Debian LTS team Jitsi-meeting (though the connection was extremely bad).
• Partly participated in preparation of Debconf21 BoF Funding Projects to Improve Debian .

Debian Science Team

• Partly participated in Debconf21 Debian Science BoF.

Other FLOSS activities

• Reviewed many merge requests in Yade open source project, merge some of them.

A song for Debconf21 ["What shall we do with the drunken sailor"]

What shall we do with the online Debconf?What shall we do with the online Debconf?What shall we do with the online Debconf?

Earl-y in the morning

Close it up as we agreed it

Save each script in case we need it

Work out how we best live-feed itNext year s Debconf s dawning

Next year s Kosovo and Pristina

This virtual Debconf needs no cleaner

Hope when COVID s gone we re keener

To meet up every morning

Thanks to the Debconf orga team

Thanks to those who Loopy meme

Things are not always as they seeem

In virtual Debconf s morning

Bullseye s out its share is rising

Debconf s fun and quite surprising

Linux 30 :)

Yours and my thing - Debian s 28

Thanks to all who video d sessions

Debconf T-shirts - prized possessions

Debcamp bug-fixed some regressions

Now onto next year!

A closing song for Debconf 21 [Frere Jacques/Brueder Martin]

DebConf21

Virtual DebConf s

Now all through

Closed for you

Kosovo is next year

See you in Pristina!!

DebConf 22

Twenty twenty-two

The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. Linux distributions like Debian fulfill an important function in the FOSS ecosystem - they are system integrators that take existing free and open source software projects and adapt them where necessary to work well together. They also make it possible for users to install more software in an easy and consistent way and with some degree of quality control and review. One of the consequences of this model is that the distribution package often lags behind upstream releases. This is especially true for distributions that have tighter integration and standardization (such as Debian), and often new upstream code is only imported irregularly because it is a manual process - both updating the package, but also making sure that it still works together well with the rest of the system. The process of importing a new upstream used to be (well, back when I started working on Debian packages) fairly manual and something like this:

• Go to the upstream s homepage, find the tarball and signature and verify the tarball
• Make modifications so the tarball matches Debian s format
• Diff the original and new upstream tarballs and figure out whether changes are reasonable and which require packaging changes
• Update the packaging, changelog, build and manually test the package
• Upload

Ecosystem Improvements However, there have been developments over the last decade that make it easier to import new upstream releases into Debian packages.

Uscan and debian QA watch Uscan and debian/watch have been around for a while and make it possible to find upstream tarballs. A debian watch file usually looks something like this:

1 2	version=4 http://somesite.com/dir/filenamewithversion.tar.gz

The QA watch service regularly polls all watch locations in the archive and makes the information available, so it s possible to know which packages have changed without downloading each one of them.

Git Git is fairly ubiquitous nowadays, and most upstream projects and packages in Debian use it. There are still exceptions that do not use any version control system or that use a different control system, but they are becoming increasingly rare. [1]

debian/upstream/metadata DEP-12 specifies a file format with metadata about the upstream project that a package was based on. In particular relevant for our case is the fact it has fields for the location of the upstream version control location. debian/upstream/metadata files look something like this:

1 2 3

--- Repository: https://www.dulwich.io/code/dulwich/ Repository-Browse: https://www.dulwich.io/code/dulwich/

While DEP-12 is still a draft, it has already been widely adopted - there are about 10000 packages in Debian that ship a debian/upstream/metadata file with Repository information.

Autopkgtest The Autopkgtest standard and associated tooling provide a way to run a defined set of tests against an installed package. This makes it possible to verify that a package is working correctly as part of the system as a whole. ci.debian.net regularly runs these tests against Debian packages to detect regressions.

Vcs-Git headers The Vcs-Git headers in debian/control are the equivalent of the Repository field in debian/upstream/metadata, but for the packaging repositories (as opposed to the upstream ones). They ve been around for a while and are widely adopted, as can be seen from zack s stats: The vcswatch service that regularly polls packaging repositories to see whether they have changed makes it a lot easier to consume this information in usable way.

Debhelper adoption Over the last couple of years, Debian has slowly been converging on a single build tool - debhelper s dh interface. Being able to rely on a single build tool makes it easier to write code to update packaging when upstream changes require it.

Debhelper DWIM Debhelper (and its helpers) increasingly can figure out how to do the Right Thing in many cases without being explicitly configured. This makes packaging less effort, but also means that it s less likely that importing a new upstream version will require updates to the packaging. With all of these improvements in place, it actually becomes feasible in a lot of situations to update a Debian package to a new upstream version automatically. Of course, this requires that all of this information is available, so it won t work for all packages. In some cases, the packaging for the older upstream version might not apply to the newer upstream version. The Janitor has attempted to import a new upstream Git snapshot and a new upstream release for every package in the archive where a debian/watch file or debian/upstream/metadata file are present. These are the steps it uses:

• Find new upstream version

  • If release, use debian/watch - or maybe tagged in upstream repository
  • If snapshot, use debian/upstream/metadata s Repository field
  • If neither is available, use guess-upstream-metadata from upstream-ontologist to guess the upstream Repository

• Merge upstream version into packaging repository, possibly importing tarballs using pristine-tar
• Update the changelog file to mention the new upstream version

• Run some checks to ensure there are no unintentional changes, e.g.:

  • Scan diff between old and new for surprising license changes

    • Today, abort if there are any - in the future, maybe update debian/copyright

  • Check for obvious compatibility breaks - e.g. sonames changing

• Attempt to update the packaging to reflect upstream changes

  • Refresh patches

• Attempt to build the package with deb-fix-build, to deal with any missing dependencies
• Run the autopkgtests with deb-fix-build to deal with missing dependencies, and abort if any tests fail

Results When run over all packages in unstable (sid), this process works for a surprising number of them.

Fresh Releases For fresh-releases (aka imports of upstream releases), processing all packages maintained in Git for which QA watch reports new releases (about 11,000): That means about 2300 packages updated, and about 4000 unchanged.

Fresh Snapshots For fresh-snapshots (aka imports of latest Git commit from upstream), processing all packages maintained in Git (about 26,000): Or 5100 packages updated and 2100 for which there was nothing to do, i.e. no upstream commits since the last Debian upload. As can be seen, this works for a surprising fraction of packages. It s possible to get the numbers up even higher, by both improving the tooling, the autopkgtests and the metadata that is provided by packages.

1
2
3
4
5
6

echo deb "[arch=amd64 signed-by=/usr/share/keyrings/debian-janitor-archive-keyring.gpg]" \
    https://janitor.debian.net/ fresh-snapshots main   sudo tee /etc/apt/sources.list.d/fresh-snapshots.list
echo deb "[arch=amd64 signed-by=/usr/share/keyrings/debian-janitor-archive-keyring.gpg]" \
    https://janitor.debian.net/ fresh-releases main   sudo tee /etc/apt/sources.list.d/fresh-releases.list
sudo curl -o /usr/share/keyrings/debian-janitor-archive-keyring.gpg https://janitor.debian.net/pgp_keys
apt update

1

apt install -t fresh-snapshots r-cran-roxygen2

ThinkPad X1 Carbon with its lid closed

Keyboard with an ANSI QWERTY layout (aka English EU for Lenovo). The LEDs on the speaker and microphone keys work out of the box on Linux.

ThinkPad X1 Carbon with its lid opened

ThinkPad USB-C Dock Gen 2 with network, keyboard, mouse, power and two screens plugged

• Lenovo ThinkPad P14s with an AMD Ryzen CPU currently unavailable with a HiDPI screen,
• Dell XPS 13 no USB-A, no HDMI port,
• HP Spectre x360 14 no HDMI port, or
• Tuxedo InfinityBook Pro 14 Gen6.