1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | def parse_plain_text_body(text): lines = text.splitlines() for i, line in enumerate(lines): if line == 'Reply to this email directly or view it on GitHub:': return lines[i + 1].split('#')[0] if (line == 'For more details, see:' and lines[i + 1].startswith('https://code.launchpad.net/')): return lines[i + 1] try: (field, value) = line.split(':', 1) except ValueError: continue if field.lower() == 'merge request url': return value.strip() return None |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | pub fn parse_plain_text_body(text: &str) -> Option<String> let lines: Vec<&str> = text.lines().collect(); for (i, line) in lines.iter().enumerate() if line == &"Reply to this email directly or view it on GitHub:" return Some(lines[i + 1].split('#').next().unwrap().to_string()); if line == &"For more details, see:" && lines[i + 1].starts_with("https://code.launchpad.net/") return Some(lines[i + 1].to_string()); if let Some((field, value)) = line.split_once(':') if field.to_lowercase() == "merge request url" return Some(value.trim().to_string()); None |
1 2 3 4 5 6 7 8 9 10 11 12 13 | use pyo3::prelude::*; #[pyfunction] fn parse_plain_text_body(text: &str) -> Option<String> janitor_mail_filter::parse_plain_text_body(text) #[pymodule] pub fn _mail_filter(py: Python, m: &PyModule) -> PyResult<()> m.add_function(wrap_pyfunction!(parse_plain_text_body, m)?)?; Ok(()) |
1 2 3 4 5 6 7 8 9 10 11 12 | [package] name = "mailfilter-py" version = "0.0.0" authors = ["Jelmer Vernoo <jelmer@jelmer.uk>"] edition = "2018" [lib] crate-type = ["cdylib"] [dependencies] janitor-mail-filter = path = "../mailfilter" pyo3 = version = ">=0.14", features = ["extension-module"] |
1 2 3 4 5 6 7 8 9 | #!/usr/bin/python3 from setuptools import setup from setuptools_rust import RustExtension, Binding setup( rust_extensions=[RustExtension( "janitor._mailfilter", "crates/mailfilter-py/Cargo.toml", binding=Binding.PyO3)], ) |
1 | from ._mailfilter import parse_plain_text_body |
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. Kali Linux have been running their own instance of the Janitor for the last year, under the kali-bot user on GitLab. Their web site has some excellent documentation explaining how the bot works. Both projects share some common components - the core janitor codebase, Silver-Platter and the various codemods (lintian-brush and deb-new-upstream). The site and some of the review logic is different for Kali. The Kali bot has several campaigns:
The last campaign doesn t exist in the Debian janitor, and pulls in new changes from packages that have been imported from other distributions.For more information about the Janitor s lintian-fixes efforts, see the landing page.
---
name: pyupgrade
command: 'pyupgrade --exit-zero-even-if-changed $(find -name "test_*.py")'
mode: propose
merge-request:
commit-message: Upgrade Python code to a modern version
---
- url: https://github.com/jelmer/dulwich
- url: https://github.com/jelmer/xandikos
$ svp batch generate --recipe=pyupgrade.yaml --candidates=candidate.syml pyupgrade
$ ls pyupgrade
batch.yaml dulwich xandikos
$ cd pyupgrade/dulwich
$ git log
commit 931f9ffb26e9143c56f20e0b85e6ddb0a8eee2eb (HEAD -> master)
Author: Jelmer Vernoo <jelmer@jelmer.uk>
Date: Sat Feb 25 22:28:12 2023 +0000
Run pyupgrade
diff --git a/dulwich/tests/compat/test_client.py b/dulwich/tests/compat/test_client.py
index 02ab6c0a..9b0661ed 100644
--- a/dulwich/tests/compat/test_client.py
+++ b/dulwich/tests/compat/test_client.py
@@ -628,7 +628,7 @@ class HTTPGitServer(http.server.HTTPServer):
self.server_name = "localhost"
def get_url(self):
- return "http:// : /".format(self.server_name, self.server_port)
+ return f"http:// self.server_name : self.server_port /"
class DulwichHttpClientTest(CompatTestCase, DulwichClientTestBase):
...
1 2 3 4 5 6 7 8 9 10 11 12 13 | name: pyupgrade work: - url: https://github.com/dulwich/dulwich name: dulwich description: Upgrade to modern Python statements commit-message: Run pyupgrade mode: propose - url: https://github.com/jelmer/xandikos name: xandikos description: Upgrade to modern Python statements commit-message: Run pyupgrade mode: propose recipe: ../pyupgrade.yaml |
$ svp batch publish pyupgrade
$ svp batch status pyupgrade
svp batch publish pyupgrade
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | #!/usr/bin/python3 from urllib.request import urlopen import sys from debian.deb822 import Deb822 import yaml with open('debian/control', 'r') as f: package = Deb822(f)['Source'] with urlopen("https://release.debian.org/transitions/export/packages.yaml") as f: data = yaml.safe_load(f) def find_transitions(data, package): for entry in data: if entry['name'] != package: continue return dict(entry['list']) return transitions = find_transitions(data, package) print(transitions) sys.exit(1 if 'ongoing' in transitions.values() else 0) |
$ debcheckout bctoolbox
git clone https://salsa.debian.org/pkg-voip-team/linphone-stack/bctoolbox.git bctoolbox ...
Cloning into 'bctoolbox'...
...
$ cd bctoolbox
$ in-transition.py
'auto-upperlimit-libbctoolbox1': 'ongoing'
Eventually, I ve settled on Navidrome. It s got a couple of things going for it:
- supysonic; Python. Slow. Ran into some issues with subsonic clients. No real web UI.
- gonic; Go. Works well & fast enough. Minimal web UI, i.e. no ability to play music from a browser.
- airsonic; Java. Last in a chain of (abandoned) forks. More effort to get to work, and resource intensive.
I run Navidrome in Kubernetes. It s surprisingly easy to get going. Here s the deployment I m using:
- Good subsonic implementation that worked with all the Android apps I used it with.
- Great Web UI for use in a browser
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | apiVersion: apps/v1
kind: Deployment
metadata:
name: navidrome
spec:
replicas: 1
selector:
matchLabels:
app: navidrome
template:
metadata:
labels:
app: navidrome
spec:
containers:
- name: navidrome
image: deluan/navidrome:latest
imagePullPolicy: Always
resources:
limits:
cpu: ".5"
memory: "2Gi"
requests:
cpu: "0.1"
memory: "10M"
ports:
- containerPort: 4533
volumeMounts:
- name: navidrome-data-volume
mountPath: /data
- name: navidrome-music-volume
mountPath: /music
env:
- name: ND_SCANSCHEDULE
value: 1h
- name: ND_LOGLEVEL
value: info
- name: ND_SESSIONTIMEOUT
value: 24h
- name: ND_BASEURL
value: /navidrome
livenessProbe:
httpGet:
path: /navidrome/app
port: 4533
initialDelaySeconds: 30
periodSeconds: 3
timeoutSeconds: 90
volumes:
- name: navidrome-data-volume
hostPath:
path: /srv/navidrome
type: Directory
- name: navidrome-music-volume
hostPath:
path: /srv/media/music
type: Directory
---
apiVersion: v1
kind: Service
metadata:
name: navidrome
spec:
ports:
- port: 4533
name: web
selector:
app: navidrome
type: ClusterIP
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: navidrome
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- backend:
service:
name: navidrome
port:
name: web
path: /navidrome(/ $)(.*)
pathType: Prefix
|
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. As covered in my post from last week, the Janitor now regularly tries to import new upstream git snapshots or upstream releases into packages in Sid.
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. Linux distributions like Debian fulfill an important function in the FOSS ecosystem - they are system integrators that take existing free and open source software projects and adapt them where necessary to work well together. They also make it possible for users to install more software in an easy and consistent way and with some degree of quality control and review. One of the consequences of this model is that the distribution package often lags behind upstream releases. This is especially true for distributions that have tighter integration and standardization (such as Debian), and often new upstream code is only imported irregularly because it is a manual process - both updating the package, but also making sure that it still works together well with the rest of the system. The process of importing a new upstream used to be (well, back when I started working on Debian packages) fairly manual and something like this:
1 2 | version=4
http://somesite.com/dir/filenamewithversion.tar.gz
|
1 2 3 | ---
Repository: https://www.dulwich.io/code/dulwich/
Repository-Browse: https://www.dulwich.io/code/dulwich/
|
1 2 3 4 5 6 | echo deb "[arch=amd64 signed-by=/usr/share/keyrings/debian-janitor-archive-keyring.gpg]" \
https://janitor.debian.net/ fresh-snapshots main sudo tee /etc/apt/sources.list.d/fresh-snapshots.list
echo deb "[arch=amd64 signed-by=/usr/share/keyrings/debian-janitor-archive-keyring.gpg]" \
https://janitor.debian.net/ fresh-releases main sudo tee /etc/apt/sources.list.d/fresh-releases.list
sudo curl -o /usr/share/keyrings/debian-janitor-archive-keyring.gpg https://janitor.debian.net/pgp_keys
apt update
|
1 | apt install -t fresh-snapshots r-cran-roxygen2
|
[1] | I m not saying that a monoculture is great here, but it does help distributions. |
silver-platter
, ognibuild
, lintian-brush
, ...) and if
you want to use it to fix a bug, you first need to make sure there's a Lintian
tag that flags the issue you're working on. Then you need to write a
lintian-brush fixer to fix said issue. Sadly, sometimes writing a new
Lintian tag to flag a trivial changes is not the appropriate course of action
and only creates clutter.
All this to say until now, I was a missing a "quick and somewhat dirty2"
way to make simple one-off changes to a bunch of packages. 200 lines of Python
later, I'm happy to report I have a simple way to replace
the old Clojure Team email in d/control
by the new one for all of our
packages. Even better, although this script doesn't aim to be a versatile tool
like the Janitor is, most of the functions can be reused for other similar
one-off scripts.
Many thanks to Felix Lechner showing me the very handy Lintian Query JSON
interface!
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. The FOSS world uses a wide variety of different build tools; given a git repository or tarball, it can be hard to figure out how to build and install a piece of software. Humans will generally know what build tool a project is using when they check out a project from git, or they can read the README. And even then, the answer may not always be straightforward to everybody. For automation, there is no obvious place to figure out how to build or install a project.
1 2 3 4 5 6 7 8 9 | % git clone https://github.com/dulwich/dulwich
% cd dulwich
% ogni --schroot=unstable-amd64-sbuild dist
Writing dulwich-0.20.21/setup.cfg
creating dist
Creating tar archive
removing 'dulwich-0.20.21' (and everything under it)
Found new tarball dulwich-0.20.21.tar.gz in /var/run/schroot/mount/unstable-amd64-sbuild-974d32d7-6f10-4e77-8622-b6a091857e85/build/tmpucazj7j7/package/dist.
|
1 2 3 4 5 6 7 8 9 | % wget https://download.samba.org/pub/ldb/ldb-2.3.0.tar.gz
% tar xvfz ldb-2.3.0.tar.gz
% cd ldb-2.3.0
% ogni install --prefix=/tmp/ldb
+ install /tmp/ldb/include/ldb.h (from include/ldb.h)
Waf: Leaving directory /tmp/ldb-2.3.0/bin/default'
'install' finished successfully (11.395s)
|
1 2 3 4 5 6 | % wget https://cpan.metacpan.org/authors/id/T/TO/TORU/XML-LibXML-LazyBuilder-0.08.tar.gz _ <https://cpan.metacpan.org/authors/id/T/TO/TORU/XML-LibXML-LazyBuilder-0.08.tar.gz> _
% tar xvfz XML-LibXML-LazyBuilder-0.08.tar.gz
Cd XML-LibXML-LazyBuilder-0.08
% ogni test
|
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. The upstream ontologist is a project that extracts metadata about upstream projects in a consistent format. It does this with a combination of heuristics and reading ecosystem-specific metadata files, such as Python s setup.py, rust s Cargo.toml as well as e.g. scanning README files.
% guess-upstream-metadata
<string>:2: (INFO/1) Duplicate implicit target name: "contributing".
Name: dulwich
Repository: https://www.dulwich.io/code/
X-Security-MD: https://github.com/dulwich/dulwich/tree/HEAD/SECURITY.md
X-Version: 0.20.21
Bug-Database: https://github.com/dulwich/dulwich/issues
X-Summary: Python Git Library
X-Description:
This is the Dulwich project.
It aims to provide an interface to git repos (both local and remote) that
doesn't call out to git directly but instead uses pure Python.
X-License: Apache License, version 2 or GNU General Public License, version 2 or later.
Bug-Submit: https://github.com/dulwich/dulwich/issues/new
[1] | Obviously this won t be able to describe the full licensing situation for many projects. Projects like scancode-toolkit are more appropriate for that. |
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. In my last blogpost, I introduced the buildlog consultant - a tool that can identify many reasons why a Debian build failed. For example, here s a fragment of a build log where the Build-Depends lack python3-setuptools:
849 850 851 852 853 854 855 856 857 858 | dpkg-buildpackage: info: host architecture amd64
fakeroot debian/rules clean
dh clean --with python3,sphinxdoc --buildsystem=pybuild
dh_auto_clean -O--buildsystem=pybuild
I: pybuild base:232: python3.9 setup.py clean
Traceback (most recent call last):
File "/<<PKGBUILDDIR>>/setup.py", line 2, in <module>
from setuptools import setup
ModuleNotFoundError: No module named 'setuptools'
E: pybuild pybuild:353: clean: plugin distutils failed with: exit code=1: python3.9 setup.py clean
|
% analyse-sbuild-log --json ~/build.log
"stage": "build",
"section": "Build",
"lineno": 857,
"kind": "missing-python-module",
"details": "module": "setuptools", "python_version": 3, "minimum_version": null
% apt-file search /usr/lib/python3/dist-packages/setuptools/__init__.py
python3-setuptools: /usr/lib/python3/dist-packages/setuptools/__init__.py
% deb-fix-build
Using output directory /tmp/tmpyz0nkgqq
Using sbuild chroot unstable-amd64-sbuild
Using fixers:
Building debian packages, running 'sbuild --no-clean-source -A -s -v'.
Attempting to use fixer upstream requirement fixer(apt) to address MissingPythonDistribution('setuptools_scm', python_version=3, minimum_version='4')
Using apt-file to search apt contents
Adding build dependency: python3-setuptools-scm (>= 4)
Building debian packages, running 'sbuild --no-clean-source -A -s -v'.
Attempting to use fixer upstream requirement fixer(apt) to address MissingPythonDistribution('toml', python_version=3, minimum_version=None)
Adding build dependency: python3-toml
Building debian packages, running 'sbuild --no-clean-source -A -s -v'.
Built 0.5.2-1- changes files at [ saneyaml_0.5.2-1_amd64.changes ].
% git log -p
commit 5a1715f4c7273b042818fc75702f2284034c7277 (HEAD -> master)
Author: Jelmer Vernoo <jelmer@jelmer.uk>
Date: Sun Apr 4 02:35:56 2021 +0100
Add missing build dependency on python3-toml.
diff --git a/debian/control b/debian/control
index 5b854dc..3b27b73 100644
--- a/debian/control
+++ b/debian/control
@@ -1,6 +1,6 @@
Rules-Requires-Root: no
Standards-Version: 4.5.1
-Build-Depends: debhelper-compat (= 12), dh-sequence-python3, python3-all, python3-setuptools (>= 50), python3-wheel, python3-setuptools-scm (>= 4)
+Build-Depends: debhelper-compat (= 12), dh-sequence-python3, python3-all, python3-setuptools (>= 50), python3-wheel, python3-setuptools-scm (>= 4), python3-toml
Testsuite: autopkgtest-pkg-python
Source: python-saneyaml
Priority: optional
commit f03047da80fcd8468ee231fbc4cf8488d7a0acd1
Author: Jelmer Vernoo <jelmer@jelmer.uk>
Date: Sun Apr 4 02:35:34 2021 +0100
Add missing build dependency on python3-setuptools-scm (>= 4).
diff --git a/debian/control b/debian/control
index a476cc2..5b854dc 100644
--- a/debian/control
+++ b/debian/control
@@ -1,6 +1,6 @@
Rules-Requires-Root: no
Standards-Version: 4.5.1
-Build-Depends: debhelper-compat (= 12), dh-sequence-python3, python3-all, python3-setuptools (>= 50), python3-wheel
+Build-Depends: debhelper-compat (= 12), dh-sequence-python3, python3-all, python3-setuptools (>= 50), python3-wheel, python3-setuptools-scm (>= 4)
Testsuite: autopkgtest-pkg-python
Source: python-saneyaml
Priority: optional
634 635 636 637 638 639 640 641 642 643 644 645 646 647 | -- Required GNU Radio Component: ANALOG missing!
-- Could NOT find GNURADIO (missing: GNURADIO_RUNTIME_FOUND)
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
-- Could NOT find LOG4CPP (missing: LOG4CPP_INCLUDE_DIRS
LOG4CPP_LIBRARIES)
CMake Error at CMakeLists.txt:593 (message):
*** Log4cpp is required to build gnss-sdr
-- Configuring incomplete, errors occurred!
See also "/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/CMakeFiles/
CMakeOutput.log".
See also "/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/CMakeFiles/
CMakeError.log".
|
% analyse-sbuild-log build.log
Failed stage: build
Section: build
Failed line: 641:
*** Log4cpp is required to build gnss-sdr
Error: Missing dependency: Log4cpp
% analyse-sbuild-log --json build.log
"stage": "build", "section": "Build", "lineno": 641, "kind": "missing-dependency", "details": "name": "Log4cpp""
Most of these can be fixed with simple sed or perl one-liners. Some of these scripts are publically available, for example:
- Updating the Vcs-Git and Vcs-Browser URLs after migrating from alioth to salsa
- Stripping trailing whitespace in various control files
- Updating e.g. homepage URLs to use https rather than http
- The R packaging team's routine-update,
- Ond ej Nov 's onovy-mass repository.
The tool also provides some basic infrastructure for testing that these scripts do what they should, and e.g. don't have unintended side-effects. The idea is that it should be safe, quick and unobtrusive to run lintian-brush, and get it to opportunistically fix lintian issues and to leave the source tree alone when it can't.
- Reverts the changes made by the script if it failed with an error
- Commits the changes to the VCS with an appropriate commit message
- Adds a changelog entry (if desired)
% debcheckout talloc
declared git repository at https://salsa.debian.org/samba-team/talloc.git
git clone https://salsa.debian.org/samba-team/talloc.git talloc ...
Cloning into 'talloc'...
remote: Enumerating objects: 2702, done.
remote: Counting objects: 100% (2702/2702), done.
remote: Compressing objects: 100% (996/996), done.
remote: Total 2702 (delta 1627), reused 2601 (delta 1550)
Receiving objects: 100% (2702/2702), 1.70 MiB 565.00 KiB/s, done.
Resolving deltas: 100% (1627/1627), done.
% cd talloc
talloc% lintian-brush
Lintian tags fixed: 'insecure-copyright-format-uri', 'public-upstream-key-not-minimal'
% git log
commit 0ea35f4bb76f6bca3132a9506189ef7531e5c680 (HEAD -> master)
Author: Jelmer Vernoo <jelmer@debian.org>
Date: Tue Dec 4 16:42:35 2018 +0000
Re-export upstream signing key without extra signatures.
Fixes lintian: public-upstream-key-not-minimal
See https://lintian.debian.org/tags/public-upstream-key-not-minimal.html for more details.
debian/changelog 1 +
debian/upstream/signing-key.asc 102 +++++++++++++++---------------------------------------------------------------------------------------
2 files changed, 16 insertions(+), 87 deletions(-)
commit feebce3147df561aa51a385c53d8759b4520c67f
Author: Jelmer Vernoo <jelmer@debian.org>
Date: Tue Dec 4 16:42:28 2018 +0000
Use secure copyright file specification URI.
Fixes lintian: insecure-copyright-format-uri
See https://lintian.debian.org/tags/insecure-copyright-format-uri.html for more details.
debian/changelog 3 +++
debian/copyright 2 +-
2 files changed, 4 insertions(+), 1 deletion(-)
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. The Janitor knows how to talk to different hosting platforms. For each hosting platform, it needs to support the platform- specific API for creating and managing merge proposals. For each hoster it also needs to have credentials. At the moment, it supports the GitHub API, Launchpad API and GitLab API. Both GitHub and Launchpad have only a single instance; the GitLab instances it supports are gitlab.com and salsa.debian.org. This provides coverage for the vast majority of Debian packages that can be accessed using Git. More than 75% of all packages are available on salsa - although in some cases, the Vcs-Git header has not yet been updated. Of the other 25%, the majority either does not declare where it is hosted using a Vcs-* header (10.5%), or have not yet migrated from alioth to another hosting platform (9.7%). A further 2.3% are hosted somewhere on GitHub (2%), Launchpad (0.18%) or GitLab.com (0.15%), in many cases in the same repository as the upstream code. The remaining 1.6% are hosted on many other hosts, primarily people s personal servers (which usually don t have an API for creating pull requests).
Hoster | Open | Merged & Applied | Closed |
github.com | 92 | 168 | 5 |
gitlab.com | 12 | 3 | 0 |
code.launchpad.net | 24 | 51 | 1 |
salsa.debian.org | 1,360 | 5,657 | 126 |
For more information about the Janitor's lintian-fixes efforts, see the landing page.
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. lintian-brush can currently fix about 150 different issues that lintian can report, but that's still a small fraction of the more than thousand different types of issue that lintian can detect. If you're interested in contributing a fixer script to lintian-brush, there is now a guide that describes all steps of the process:
For more information about the Janitor's lintian-fixes efforts, see the landing page.
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. As of dpkg 1.16.2 and apt 0.8.13, Debian has full support for multi-arch. To quote from the multi-arch implementation page:
Multiarch lets you install library packages from multiple architectures on the same machine. This is useful in various ways, but the most common is installing both 64 and 32- bit software on the same machine and having dependencies correctly resolved automatically. In general you can have libraries of more than one architecture installed together and applications from one architecture or another installed as alternatives.The Multi-Arch specification describes a new Multi-Arch header which can be used to indicate how to resolve cross-architecture dependencies. The existing Debian Multi-Arch hinter is a version of dedup.debian.net that compares binary packages between architectures and suggests fixes to resolve multi-arch problems. It provides hints as to what Multi- Arch fields can be set, allowing the packages to be safely installed in a Multi-Arch world. The full list of almost 10,000 hints generated by the hinter is available at https://dedup.debian.net/static/multiarch-hints.yaml. Recent versions of lintian-brush now include a command called apply-multiarch-hints that downloads and locally caches the hints and can apply them to a package maintained in Git. For example, to apply multi-arch hints to autosize.js:
$ debcheckout autosize.js
declared git repository at https://salsa.debian.org/js-team/autosize.js.git
git clone https://salsa.debian.org/js-team/autosize.js.git autosize.js ...
Cloning into 'autosize.js'...
[...]
$ cd autosize.js
$ apply-multiarch-hints
Downloading new version of multi-arch hints.
libjs-autosize: Add Multi-Arch: foreign.
node-autosize: Add Multi-Arch: foreign.
$ git log -p
commit 3f8d1db5af4a87e6ebb08f46ddf79f6adf4e95ae (HEAD -> master)
Author: Jelmer Vernoo <jelmer@debian.org>
Date: Fri Sep 18 23:37:14 2020 +0000
Apply multi-arch hints.
+ libjs-autosize, node-autosize: Add Multi-Arch: foreign.
Changes-By: apply-multiarch-hints
diff --git a/debian/changelog b/debian/changelog
index e7fa120..09af4a7 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+autosize.js (4.0.2~dfsg1-5) UNRELEASED; urgency=medium
+
+ * Apply multi-arch hints.
+ + libjs-autosize, node-autosize: Add Multi-Arch: foreign.
+
+ -- Jelmer Vernoo <jelmer@debian.org> Fri, 18 Sep 2020 23:37:14 -0000
+
autosize.js (4.0.2~dfsg1-4) unstable; urgency=medium
* Team upload
diff --git a/debian/control b/debian/control
index 01ca968..fbba1ae 100644
--- a/debian/control
+++ b/debian/control
@@ -20,6 +20,7 @@ Architecture: all
Depends: $ misc:Depends
Recommends: javascript-common
Breaks: ruby-rails-assets-autosize (<< 4.0)
+Multi-Arch: foreign
Description: script to automatically adjust textarea height to fit text - NodeJS
Autosize is a small, stand-alone script to automatically adjust textarea
height to fit text. The autosize function accepts a single textarea element,
@@ -32,6 +33,7 @@ Package: node-autosize
Architecture: all
Depends: $ misc:Depends
, nodejs
+Multi-Arch: foreign
Description: script to automatically adjust textarea height to fit text - Javascript
Autosize is a small, stand-alone script to automatically adjust textarea
height to fit text. The autosize function accepts a single textarea element,
For more information about the Janitor's lintian-fixes efforts, see the landing page.
The Debian Janitor is an automated system that commits fixes for (minor) issues in Debian packages that can be fixed by software. It gradually started proposing merges in early December. The first set of changes sent out ran lintian-brush on sid packages maintained in Git. This post is part of a series about the progress of the Janitor. On 12 July 2019, the Janitor started fixing lintian issues in packages in the Debian archive. Now, a year and a half later, it has processed every one of the almost 28,000 packages at least once. As discussed two weeks ago, this has resulted in roughly 65,000 total changes. These 65,000 changes were made to a total of almost 17,000 packages. Of the remaining packages, for about 4,500 lintian-brush could not make any improvements. The rest (about 6,500) failed to be processed for one of many reasons they are e.g. not yet migrated off alioth, use uncommon formatting that can't be preserved or failed to build for one reason or another. Now that the entire archive has been processed, packages are prioritized based on the likelihood of a change being made to them successfully. Over the course of its existence, the Janitor has slowly gained support for a wider variety of packaging methods. For example, it can now edit the templates for some of the generated control files. Many of the packages that the janitor was unable to propose changes for the first time around are expected to be correctly handled when they are reprocessed. If you re a Debian developer, you can find the list of improvements made by the janitor in your packages by going to https://janitor.debian.net/m/.
For more information about the Janitor's lintian-fixes efforts, see the landing page.
One hiccup we ve encountered in SecureDrop development is that not all Python wheels can be built reproducibly. We ship multiple (Python) projects in Debian packages, with Python dependencies included in those packages as wheels. In order for our Debian packages to be reproducible, we need that wheel build process to also be reproducibleParallel to this, transparencylog.com was also launched, a service that verifies the contents of URLs against a publicly recorded cryptographic log. It keeps an append-only log of the cryptographic digests of all URLs it has seen. (GitHub repo) On 18th September, Bernhard M. Wiedemann will give a presentation in German, titled Wie reproducible builds Software sicherer machen ( How reproducible builds make software more secure ) at the Internet Security Digital Days 2020 conference.
ftp.debian.org
are made from their claimed sources. It also served as a general update on the status of reproducible builds within Debian. The video (145 MB) and slides are available.
There were also a number of other talks that involved Reproducible Builds too. For example, the Malayalam language mini-conference had a talk titled , ? ( I want to join Debian, what should I do? ) presented by Praveen Arimbrathodiyil, the Clojure Packaging Team BoF session led by Elana Hashman, as well as Where is Salsa CI right now? that was on the topic of Salsa, the collaborative development server that Debian uses to provide the necessary tools for package maintainers, packaging teams and so on.
Jonathan Bustillos (Jathan) also gave a talk in Spanish titled Un camino verificable desde el origen hasta el binario ( A verifiable path from source to binary ). (Video, 88MB)
openwrt-devel
mailing list asking for clarification on when to raise the PKG_RELEASE
identifier of a package. This is needed in order to successfully perform rebuilds in a reproducible builds context.
In openSUSE, Bernhard M. Wiedemann published his monthly Reproducible Builds status update.
Chris Lamb provided some comments and pointers on an upstream issue regarding the reproducibility of a Snap / SquashFS archive file. [ ]
.buildinfo
build certificates have been tainted on the official Debian build servers, as these environments have files underneath the /usr/local/sbin
directory [ ]. He also filed against bug for debrebuild
after spotting that it can fail to download packages from snapshot.debian.org
[ ].
This month, several issues were uncovered (or assisted) due to the efforts of reproducible builds.
For instance, Debian bug #968710 was filed by Simon McVittie, which describes a problem with detached debug symbol files (required to generate a traceback) that is unlikely to have been discovered without reproducible builds. In addition, Jelmer Vernooij called attention that the new Debian Janitor tool is using the property of reproducibility (as well as diffoscope when applying archive-wide changes to Debian:
New merge proposals also include a link to the diffoscope diff between a vanilla build and the build with changes. Unfortunately these can be a bit noisy for packages that are not reproducible yet, due to the difference in build environment between the two builds. [ ]56 reviews of Debian packages were added, 38 were updated and 24 were removed this month adding to our knowledge about identified issues. Specifically, Chris Lamb added and categorised the
nondeterministic_version_generated_by_python_param
and the lessc_nondeterministic_keys
toolchain issues. [ ][ ]
Holger Levsen sponsored Lukas Puehringer s upload of the python-securesystemslib pacage, which is a dependency of in-toto, a framework to secure the integrity of software supply chains. [ ]
Lastly, Chris Lamb further refined his merge request against the debian-installer
component to allow all arguments from sources.list
files (such as [check-valid-until=no]
) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure and sent a ping to the team that maintains that code.
asymptote
(shell/Perl date)getfem
(embeds datetime and user, submitted via email)getdp
(hostname and user)getdp
(user)guix
(disable parallelism)httpcomponents-client
(Java documentation generator readdir
order)kuberlr
(date)lal
(date and time issue, submitted via email)libmesh
(host)OBS
(discuss how to track old build prjconf
metadata in buildinfo)openblas
(disable CPU detection)openfoam-selector
(date)perl
(toolchain, date)python-blosc
(CPU detection)python-eventlet
(fails to build far in the future)rna-star
(date and hostname)trilinos
(date)xz/b4
(workaround CPU count influencing output, reported upstream)json-c
(forwarded upstream).nmh
.golang-gonum-v1-plot
.chirp
.pixelmed-codec
.debhelper
.muroar
.serd
.pencil2d
.tpot
.evolution
.aflplusplus
.plexus-archiver
(timezone/DST issue)libjpeg-turbo
.jack-audio-connection-kit
.glusterfs
.155
, 156
, 157
and 158
to Debian:
<!ENTITY>
declarations inside the Document Type Definition (DTD), or when a DTD or entity references an external resource. (#212)pgpdump(1)
can successfully parse some binary files, so check that the parsed output contains something sensible before accepting it. [ ]gnumeric
from the Debian build-dependencies as it has been removed from the testing distribution. (#968742)fallback_recognises
to prevent matching .xsb
binary XML files.file(1)
returns data
. (#211)ppudump
version does not match our file header. [ ]repr(object)
output in Calling external command messages. [ ].ppu
files requires ppudump
version 3.2.0 or higher. [ ]setup.py
that diffoscope works with Python version 3.8 [ ] and Frazer Clews applied some Pylint suggestions [ ] and removed some deprecated methods [ ].
SOURCE_DATE_EPOCH
age. [ ]tests.reproducible-builds.org
. This month, Holger Levsen made the following changes:
arm64
architecture. [ ]armhf
. [ ][ ][ ]ruby-jekyll-polyglot
package to needed for the recently-added internationalisation and translation support on the Reproducible Builds website. [ ]jenkins.debian.net
, www.reproducible-builds.org
, diffoscope.org
, buildinfos.debian.net
, etc.). [ ][ ][ ][ ][ ]arm64
architecture anymore. [ ]If you think you know how to spread the word about reproducibility in the context of Bitcoin wallets through WalletScrutiny, your contributions are highly welcome on this PR [ ]Julien Lepiller posted to the list linking to a blog post by Tavis Ormandy titled You don t need reproducible builds. Morten Linderud (foxboron) responded with a clear rebuttal that Tavis was only considering the narrow use-case of proprietary vendors and closed-source software. He additionally noted that the criticism that reproducible builds cannot prevent against backdoors being deliberately introduced into the upstream source ( bugdoors ) are decidedly (and deliberately) outside the scope of reproducible builds to begin with. Chris Lamb included the Reproducible Builds mailing list in a wider discussion regarding a tentative proposal to include
.buildinfo
files in .deb
packages, adding his remarks regarding requiring a custom tool in order to determine whether generated build artifacts are identical in a reproducible context. [ ]
Jonathan Bustillos (Jathan) posted a quick email to the list requesting whether there was a list of To do tasks in Reproducible Builds.
Lastly, Chris Lamb responded at length to a query regarding the status of reproducible builds for Debian ISO or installation images. He noted that most of the technical work has been performed but there are at least four issues until they can be generally advertised as such . He pointed that the privacy-oriented Tails operation system, which is based directly on Debian, has had reproducible builds for a number of years now. [ ]
#reproducible-builds
on irc.oftc.net
.
rb-general@lists.reproducible-builds.org
Next.