Freexian Collaborators: Debian Contributions: DebConf Bursaries, /usr-move, sbuild, and more! (by Stefano Rivera)
Contributing to Debian
is part of Freexian s mission. This article
covers the latest achievements of Freexian and their collaborators. All of this
is made possible by organizations subscribing to our
Long Term Support contracts and
consulting services.
DebConf Bursary updates, by Utkarsh Gupta
Utkarsh is the bursaries team lead for DebConf 24. Bursary
requests are dispatched to a
team of volunteers to review. The results are collated, adjusted and merged to
produce priority lists of requests to fund. Utkarsh raised the team,
coordinated the review, and issued bursaries to attendees.
DebConf Bursary updates, by Utkarsh Gupta
Utkarsh is the bursaries team lead for DebConf 24. Bursary
requests are dispatched to a
team of volunteers to review. The results are collated, adjusted and merged to
produce priority lists of requests to fund. Utkarsh raised the team,
coordinated the review, and issued bursaries to attendees.
/usr-move, by Helmut Grohne
More and more, the /usr
-move transition is being carried out by multiple
contributors and many performed around a hundred of the requested uploads. Of
these, Helmut contributed five patches and two uploads. As a result, there are
less than 350 packages left to be converted, and all of the non-trivial cases
have patches. We started with three times that number. Thanks to everyone
involved for supporting this effort.
For people interested in background information of this transition,
Helmut gave a presentation at MiniDebConf Berlin
2024
(slides).
sbuild, by Helmut Grohne
While unshare
mode of sbuild
has existed for quite a while, it is
now getting significant use in Debian, and new problems are popping up.
Helmut looked into an apparmor-related
failure and provided a diagnosis.
While relevant code would detect the chroot
nature of a schroot
backend and skip apparmor
tests, the unshare
environment would be
just good enough to run and fail the test. As sbuild
exposes fewer
special kernel filesystems, the tests will be skipped again.
Another problem popped up when gobject-introspection
added a
dependency on the host architecture Python interpreter in a cross build
environment. sbuild
would prefer installing (and failing) a host
architecture Python to installing the qemu
alternative. Attempts to
fix this would result in systemd killing
sbuild. ischroot
as used by
libc6.postinst
would not classify the unshare
environment as a
chroot
. Therefore libc6.postinst
would run telinit
which would kill
the build process. This is a complex interaction problem that shall
eventually be solved by providing triggers from libc6
to be
implemented by affected init systems.
Salsa CI updates, by Santiago Ruano Rinc n
Several issues arose about Salsa CI last month, and it is probably worth
mentioning part of the challenges of defining its framework in YAML.
With the upcoming end-of-support of Debian 10 buster as LTS, armel was
removed from deb.debian.org
, making the jobs that build images for
buster/armel to fail. While the removal of buster/armel from the
repositories is a natural change, it put some light on the flaws in
the Salsa CI design regarding the support of the different Debian
releases. Currently, the images are defined like these (from
.images-debian.yml
):
.all-supported-releases: &all-supported-releases
- stretch
- stretch-backports
- buster
- bullseye
- bullseye-backports
- bookworm
- bookworm-backports
- trixie
- sid
- experimental
And from them, different images are built according to the different
jobs and how they are supported, for example:
images-prod-arm:
stage: build
extends: .build_template
tags:
- $SALSA_CI_ARM_RUNNER_TAG
parallel:
matrix:
# Base image, all releases, all arches
- IMAGE_NAME: base
ARCH:
- arm32v5
- arm32v7
- arm64v8
RELEASE: *all-supported-releases
The removal of buster/armel could be easily reflected as:
images-prod-arm:
stage: build
extends: .build_template
tags:
- $SALSA_CI_ARM_RUNNER_TAG
parallel:
matrix:
# Base image, fully supported releases, all arches
- IMAGE_NAME: base
ARCH:
- arm32v5
- arm32v7
- arm64v8
RELEASE:
- stretch
- buster
- bullseye
- bullseye-backports
- bookworm
- bookworm-backports
- trixie
- sid
- experimental
# buster only supports armhf and arm64
- IMAGE_NAME: base
ARCH:
- arm32v7
- arm64v8
RELEASE: buster
Evidently, this increases duplication of the release support data, which
is of course not optimal and it is error prone when changing the data
about supported releases. A better approach would be to have two
different YAML lists, such as:
# releases that have partial support. E.g.: buster is transitioning to
# Debian LTS, and buster armel is no longer found in deb.debian.org
.old-releases: &old-releases
- stretch
- buster
.currently-supported-releases: ¤tly-supported-releases
- bullseye
- bullseye-backports
- bookworm
- bookworm-backports
- trixie
- sid
- experimental
and then a unified list:
.all-supported-releases: &all-supported-releases
- *old-releases
- *currently-supported-releases
that could be used in the matrix of the jobs that build all the images
available in the pipeline container registry.
However, due to limitations in
GitLab,
it is not possible to expand the variables or mapping values in a
parallel:matrix
context. At least not in an elegant fashion.
This is the kind of issue that recently arose and that Santiago is
currently working to solve, in the simplest possible way.
Astute readers would notice that stretch is listed in the fully
supported releases. And there is no problem with stretch, because it is
built from archive.debian.org
. Otto actually has tried to
fix
the broken image build job doing the same, but it is still incorrect,
because the security repository is not (yet) available in archive.debian.org
.
Additionally, Santiago has also worked on other merge requests, such as:
- support branch/tags as target head in the test projects,
- build autopkgtest image on top of stable
- Add .yamllint and make it happy in the autopkgtest-lxc project
- enable FF_SCRIPT_SECTIONS to log multiline commands, among others.
Archiving DebConf Websites, by Stefano Rivera
DebConf, the annual Debian conference, has its own new website every
year. These are typically complex dynamic web applications (featuring
registration, call for papers, scheduling, etc.) Once the conference is
over, there is no need to keep maintaining these applications, so we
archive the sites off as static HTML, and serve them from Debian s
static CDN.
Stefano archived the websites for the last two DebConfs.
The schedule system behind DebConf 14
and 15 s websites was a derivative of
Canonical s summit
system. This was only used for a couple of years before migrating to
wafer, the current system. Archiving
summit content has been on the nice to have list for years, but nobody
has ever tackled it. The machine that served the sites went away a
couple of years ago. After much digging, a backup of the database was
found, and Stefano got this code running on an ancient Python 2.7.
Recently Stefano put this all together and hooked in an archive export
to finally get this content preserved.
Python 3.x and pypy3 security bug triage, by Stefano Rivera
Stefano Rivera triaged all the open security bugs against the Python 3.x
and PyPy3 packages for Debian s stable and LTS releases. Several had
been fixed but this wasn t recorded in the security tracker.
Linux livepatching support for Debian, by Santiago Ruano Rinc n
In collaboration with Emmanuel Arias, Santiago filed ITP bug
#1070494.
As stated in the bug, more than an Intent to Package, it is an Intent to
Design and Implement live patching support for the Linux kernel in
Debian. For now, Emmanuel and Santiago have done exploratory work and
they are working to understand the different possibilities to implement
livepatching. One possible direction is to rely on
kpatch, and the other is to
package the modules using regular packaging tools. Also, it is needed
to evaluate if it is possible to rely on distributing the modules via
packages, or instead as a service, as it is done by some commercial
distributions.
Miscellaneous contributions
- Thorsten Alteholz uploaded
cups-bjnp
to improve packaging.
- Colin Watson tracked down a baffling CI issue in openssh to unblock several merge requests, removed the user_readenv=1 option from its PAM configuration, and started on the first stage of his plan to split out GSS-API key exchange support to separate packages.
- Colin did his usual routine work on the Python team, upgrading 26 packages to new upstream versions, and cherry-picking an upstream PR to fix a pytest 8 incompatibility in ipywidgets.
- Colin NMUed a couple of packages to reduce the need for explicit overrides in Packages-arch-specific, and removed some other obsolete entries from there.
- Emilio managed various library transitions, and helped finish a few of the remaining t64 transitions.
- Helmut sent a patch for enabling piuparts to work as a regular user building on earlier work.
- Helmut sent patches for 7 cross build failures, 6 other debian bugs and fixed an infrastructure problem in crossqa.debian.net.
- Nicholas worked on a sponsored package upload, and discovered the blhc tool for diagnosing build hardening.
- Stefano Rivera started and completed the re2 transition. The release team suggested moving to a virtual package scheme that includes the absl ABI (as re2 now depends on it). Adopted this.
- Stefano continued to work on DebConf 24 planning.
- Santiago continued to work on DebConf24 Content tasks as well as Debconf25 organisation.
/usr
-move transition is being carried out by multiple
contributors and many performed around a hundred of the requested uploads. Of
these, Helmut contributed five patches and two uploads. As a result, there are
less than 350 packages left to be converted, and all of the non-trivial cases
have patches. We started with three times that number. Thanks to everyone
involved for supporting this effort.
For people interested in background information of this transition,
Helmut gave a presentation at MiniDebConf Berlin
2024
(slides).
sbuild, by Helmut Grohne
While unshare
mode of sbuild
has existed for quite a while, it is
now getting significant use in Debian, and new problems are popping up.
Helmut looked into an apparmor-related
failure and provided a diagnosis.
While relevant code would detect the chroot
nature of a schroot
backend and skip apparmor
tests, the unshare
environment would be
just good enough to run and fail the test. As sbuild
exposes fewer
special kernel filesystems, the tests will be skipped again.
Another problem popped up when gobject-introspection
added a
dependency on the host architecture Python interpreter in a cross build
environment. sbuild
would prefer installing (and failing) a host
architecture Python to installing the qemu
alternative. Attempts to
fix this would result in systemd killing
sbuild. ischroot
as used by
libc6.postinst
would not classify the unshare
environment as a
chroot
. Therefore libc6.postinst
would run telinit
which would kill
the build process. This is a complex interaction problem that shall
eventually be solved by providing triggers from libc6
to be
implemented by affected init systems.
Salsa CI updates, by Santiago Ruano Rinc n
Several issues arose about Salsa CI last month, and it is probably worth
mentioning part of the challenges of defining its framework in YAML.
With the upcoming end-of-support of Debian 10 buster as LTS, armel was
removed from deb.debian.org
, making the jobs that build images for
buster/armel to fail. While the removal of buster/armel from the
repositories is a natural change, it put some light on the flaws in
the Salsa CI design regarding the support of the different Debian
releases. Currently, the images are defined like these (from
.images-debian.yml
):
.all-supported-releases: &all-supported-releases
- stretch
- stretch-backports
- buster
- bullseye
- bullseye-backports
- bookworm
- bookworm-backports
- trixie
- sid
- experimental
And from them, different images are built according to the different
jobs and how they are supported, for example:
images-prod-arm:
stage: build
extends: .build_template
tags:
- $SALSA_CI_ARM_RUNNER_TAG
parallel:
matrix:
# Base image, all releases, all arches
- IMAGE_NAME: base
ARCH:
- arm32v5
- arm32v7
- arm64v8
RELEASE: *all-supported-releases
The removal of buster/armel could be easily reflected as:
images-prod-arm:
stage: build
extends: .build_template
tags:
- $SALSA_CI_ARM_RUNNER_TAG
parallel:
matrix:
# Base image, fully supported releases, all arches
- IMAGE_NAME: base
ARCH:
- arm32v5
- arm32v7
- arm64v8
RELEASE:
- stretch
- buster
- bullseye
- bullseye-backports
- bookworm
- bookworm-backports
- trixie
- sid
- experimental
# buster only supports armhf and arm64
- IMAGE_NAME: base
ARCH:
- arm32v7
- arm64v8
RELEASE: buster
Evidently, this increases duplication of the release support data, which
is of course not optimal and it is error prone when changing the data
about supported releases. A better approach would be to have two
different YAML lists, such as:
# releases that have partial support. E.g.: buster is transitioning to
# Debian LTS, and buster armel is no longer found in deb.debian.org
.old-releases: &old-releases
- stretch
- buster
.currently-supported-releases: ¤tly-supported-releases
- bullseye
- bullseye-backports
- bookworm
- bookworm-backports
- trixie
- sid
- experimental
and then a unified list:
.all-supported-releases: &all-supported-releases
- *old-releases
- *currently-supported-releases
that could be used in the matrix of the jobs that build all the images
available in the pipeline container registry.
However, due to limitations in
GitLab,
it is not possible to expand the variables or mapping values in a
parallel:matrix
context. At least not in an elegant fashion.
This is the kind of issue that recently arose and that Santiago is
currently working to solve, in the simplest possible way.
Astute readers would notice that stretch is listed in the fully
supported releases. And there is no problem with stretch, because it is
built from archive.debian.org
. Otto actually has tried to
fix
the broken image build job doing the same, but it is still incorrect,
because the security repository is not (yet) available in archive.debian.org
.
Additionally, Santiago has also worked on other merge requests, such as:
- support branch/tags as target head in the test projects,
- build autopkgtest image on top of stable
- Add .yamllint and make it happy in the autopkgtest-lxc project
- enable FF_SCRIPT_SECTIONS to log multiline commands, among others.
Archiving DebConf Websites, by Stefano Rivera
DebConf, the annual Debian conference, has its own new website every
year. These are typically complex dynamic web applications (featuring
registration, call for papers, scheduling, etc.) Once the conference is
over, there is no need to keep maintaining these applications, so we
archive the sites off as static HTML, and serve them from Debian s
static CDN.
Stefano archived the websites for the last two DebConfs.
The schedule system behind DebConf 14
and 15 s websites was a derivative of
Canonical s summit
system. This was only used for a couple of years before migrating to
wafer, the current system. Archiving
summit content has been on the nice to have list for years, but nobody
has ever tackled it. The machine that served the sites went away a
couple of years ago. After much digging, a backup of the database was
found, and Stefano got this code running on an ancient Python 2.7.
Recently Stefano put this all together and hooked in an archive export
to finally get this content preserved.
Python 3.x and pypy3 security bug triage, by Stefano Rivera
Stefano Rivera triaged all the open security bugs against the Python 3.x
and PyPy3 packages for Debian s stable and LTS releases. Several had
been fixed but this wasn t recorded in the security tracker.
Linux livepatching support for Debian, by Santiago Ruano Rinc n
In collaboration with Emmanuel Arias, Santiago filed ITP bug
#1070494.
As stated in the bug, more than an Intent to Package, it is an Intent to
Design and Implement live patching support for the Linux kernel in
Debian. For now, Emmanuel and Santiago have done exploratory work and
they are working to understand the different possibilities to implement
livepatching. One possible direction is to rely on
kpatch, and the other is to
package the modules using regular packaging tools. Also, it is needed
to evaluate if it is possible to rely on distributing the modules via
packages, or instead as a service, as it is done by some commercial
distributions.
Miscellaneous contributions
- Thorsten Alteholz uploaded
cups-bjnp
to improve packaging.
- Colin Watson tracked down a baffling CI issue in openssh to unblock several merge requests, removed the user_readenv=1 option from its PAM configuration, and started on the first stage of his plan to split out GSS-API key exchange support to separate packages.
- Colin did his usual routine work on the Python team, upgrading 26 packages to new upstream versions, and cherry-picking an upstream PR to fix a pytest 8 incompatibility in ipywidgets.
- Colin NMUed a couple of packages to reduce the need for explicit overrides in Packages-arch-specific, and removed some other obsolete entries from there.
- Emilio managed various library transitions, and helped finish a few of the remaining t64 transitions.
- Helmut sent a patch for enabling piuparts to work as a regular user building on earlier work.
- Helmut sent patches for 7 cross build failures, 6 other debian bugs and fixed an infrastructure problem in crossqa.debian.net.
- Nicholas worked on a sponsored package upload, and discovered the blhc tool for diagnosing build hardening.
- Stefano Rivera started and completed the re2 transition. The release team suggested moving to a virtual package scheme that includes the absl ABI (as re2 now depends on it). Adopted this.
- Stefano continued to work on DebConf 24 planning.
- Santiago continued to work on DebConf24 Content tasks as well as Debconf25 organisation.
deb.debian.org
, making the jobs that build images for
buster/armel to fail. While the removal of buster/armel from the
repositories is a natural change, it put some light on the flaws in
the Salsa CI design regarding the support of the different Debian
releases. Currently, the images are defined like these (from
.images-debian.yml
):
.all-supported-releases: &all-supported-releases
- stretch
- stretch-backports
- buster
- bullseye
- bullseye-backports
- bookworm
- bookworm-backports
- trixie
- sid
- experimental
images-prod-arm:
stage: build
extends: .build_template
tags:
- $SALSA_CI_ARM_RUNNER_TAG
parallel:
matrix:
# Base image, all releases, all arches
- IMAGE_NAME: base
ARCH:
- arm32v5
- arm32v7
- arm64v8
RELEASE: *all-supported-releases
images-prod-arm:
stage: build
extends: .build_template
tags:
- $SALSA_CI_ARM_RUNNER_TAG
parallel:
matrix:
# Base image, fully supported releases, all arches
- IMAGE_NAME: base
ARCH:
- arm32v5
- arm32v7
- arm64v8
RELEASE:
- stretch
- buster
- bullseye
- bullseye-backports
- bookworm
- bookworm-backports
- trixie
- sid
- experimental
# buster only supports armhf and arm64
- IMAGE_NAME: base
ARCH:
- arm32v7
- arm64v8
RELEASE: buster
# releases that have partial support. E.g.: buster is transitioning to
# Debian LTS, and buster armel is no longer found in deb.debian.org
.old-releases: &old-releases
- stretch
- buster
.currently-supported-releases: ¤tly-supported-releases
- bullseye
- bullseye-backports
- bookworm
- bookworm-backports
- trixie
- sid
- experimental
.all-supported-releases: &all-supported-releases
- *old-releases
- *currently-supported-releases
parallel:matrix
context. At least not in an elegant fashion.
This is the kind of issue that recently arose and that Santiago is
currently working to solve, in the simplest possible way.
Astute readers would notice that stretch is listed in the fully
supported releases. And there is no problem with stretch, because it is
built from archive.debian.org
. Otto actually has tried to
fix
the broken image build job doing the same, but it is still incorrect,
because the security repository is not (yet) available in archive.debian.org
.
Additionally, Santiago has also worked on other merge requests, such as:
- support branch/tags as target head in the test projects,
- build autopkgtest image on top of stable
- Add .yamllint and make it happy in the autopkgtest-lxc project
- enable FF_SCRIPT_SECTIONS to log multiline commands, among others.
Archiving DebConf Websites, by Stefano Rivera
DebConf, the annual Debian conference, has its own new website every
year. These are typically complex dynamic web applications (featuring
registration, call for papers, scheduling, etc.) Once the conference is
over, there is no need to keep maintaining these applications, so we
archive the sites off as static HTML, and serve them from Debian s
static CDN.
Stefano archived the websites for the last two DebConfs.
The schedule system behind DebConf 14
and 15 s websites was a derivative of
Canonical s summit
system. This was only used for a couple of years before migrating to
wafer, the current system. Archiving
summit content has been on the nice to have list for years, but nobody
has ever tackled it. The machine that served the sites went away a
couple of years ago. After much digging, a backup of the database was
found, and Stefano got this code running on an ancient Python 2.7.
Recently Stefano put this all together and hooked in an archive export
to finally get this content preserved.
Python 3.x and pypy3 security bug triage, by Stefano Rivera
Stefano Rivera triaged all the open security bugs against the Python 3.x
and PyPy3 packages for Debian s stable and LTS releases. Several had
been fixed but this wasn t recorded in the security tracker.
Linux livepatching support for Debian, by Santiago Ruano Rinc n
In collaboration with Emmanuel Arias, Santiago filed ITP bug
#1070494.
As stated in the bug, more than an Intent to Package, it is an Intent to
Design and Implement live patching support for the Linux kernel in
Debian. For now, Emmanuel and Santiago have done exploratory work and
they are working to understand the different possibilities to implement
livepatching. One possible direction is to rely on
kpatch, and the other is to
package the modules using regular packaging tools. Also, it is needed
to evaluate if it is possible to rely on distributing the modules via
packages, or instead as a service, as it is done by some commercial
distributions.
Miscellaneous contributions
- Thorsten Alteholz uploaded
cups-bjnp
to improve packaging.
- Colin Watson tracked down a baffling CI issue in openssh to unblock several merge requests, removed the user_readenv=1 option from its PAM configuration, and started on the first stage of his plan to split out GSS-API key exchange support to separate packages.
- Colin did his usual routine work on the Python team, upgrading 26 packages to new upstream versions, and cherry-picking an upstream PR to fix a pytest 8 incompatibility in ipywidgets.
- Colin NMUed a couple of packages to reduce the need for explicit overrides in Packages-arch-specific, and removed some other obsolete entries from there.
- Emilio managed various library transitions, and helped finish a few of the remaining t64 transitions.
- Helmut sent a patch for enabling piuparts to work as a regular user building on earlier work.
- Helmut sent patches for 7 cross build failures, 6 other debian bugs and fixed an infrastructure problem in crossqa.debian.net.
- Nicholas worked on a sponsored package upload, and discovered the blhc tool for diagnosing build hardening.
- Stefano Rivera started and completed the re2 transition. The release team suggested moving to a virtual package scheme that includes the absl ABI (as re2 now depends on it). Adopted this.
- Stefano continued to work on DebConf 24 planning.
- Santiago continued to work on DebConf24 Content tasks as well as Debconf25 organisation.
Linux livepatching support for Debian, by Santiago Ruano Rinc n
In collaboration with Emmanuel Arias, Santiago filed ITP bug
#1070494.
As stated in the bug, more than an Intent to Package, it is an Intent to
Design and Implement live patching support for the Linux kernel in
Debian. For now, Emmanuel and Santiago have done exploratory work and
they are working to understand the different possibilities to implement
livepatching. One possible direction is to rely on
kpatch, and the other is to
package the modules using regular packaging tools. Also, it is needed
to evaluate if it is possible to rely on distributing the modules via
packages, or instead as a service, as it is done by some commercial
distributions.
Miscellaneous contributions
- Thorsten Alteholz uploaded
cups-bjnp
to improve packaging.
- Colin Watson tracked down a baffling CI issue in openssh to unblock several merge requests, removed the user_readenv=1 option from its PAM configuration, and started on the first stage of his plan to split out GSS-API key exchange support to separate packages.
- Colin did his usual routine work on the Python team, upgrading 26 packages to new upstream versions, and cherry-picking an upstream PR to fix a pytest 8 incompatibility in ipywidgets.
- Colin NMUed a couple of packages to reduce the need for explicit overrides in Packages-arch-specific, and removed some other obsolete entries from there.
- Emilio managed various library transitions, and helped finish a few of the remaining t64 transitions.
- Helmut sent a patch for enabling piuparts to work as a regular user building on earlier work.
- Helmut sent patches for 7 cross build failures, 6 other debian bugs and fixed an infrastructure problem in crossqa.debian.net.
- Nicholas worked on a sponsored package upload, and discovered the blhc tool for diagnosing build hardening.
- Stefano Rivera started and completed the re2 transition. The release team suggested moving to a virtual package scheme that includes the absl ABI (as re2 now depends on it). Adopted this.
- Stefano continued to work on DebConf 24 planning.
- Santiago continued to work on DebConf24 Content tasks as well as Debconf25 organisation.
- Thorsten Alteholz uploaded
cups-bjnp
to improve packaging. - Colin Watson tracked down a baffling CI issue in openssh to unblock several merge requests, removed the user_readenv=1 option from its PAM configuration, and started on the first stage of his plan to split out GSS-API key exchange support to separate packages.
- Colin did his usual routine work on the Python team, upgrading 26 packages to new upstream versions, and cherry-picking an upstream PR to fix a pytest 8 incompatibility in ipywidgets.
- Colin NMUed a couple of packages to reduce the need for explicit overrides in Packages-arch-specific, and removed some other obsolete entries from there.
- Emilio managed various library transitions, and helped finish a few of the remaining t64 transitions.
- Helmut sent a patch for enabling piuparts to work as a regular user building on earlier work.
- Helmut sent patches for 7 cross build failures, 6 other debian bugs and fixed an infrastructure problem in crossqa.debian.net.
- Nicholas worked on a sponsored package upload, and discovered the blhc tool for diagnosing build hardening.
- Stefano Rivera started and completed the re2 transition. The release team suggested moving to a virtual package scheme that includes the absl ABI (as re2 now depends on it). Adopted this.
- Stefano continued to work on DebConf 24 planning.
- Santiago continued to work on DebConf24 Content tasks as well as Debconf25 organisation.