Context

• See hibernate-pocket, hibernate-pocket-2, hibernate-pocket-3, hibernate-pocket-4,hibernate-pocket-5, hibernate-pocket-6, hibernate-pocket-7, hibernate-pocket-8, hibernate-pocket-9 hibernate-pocket-10 hibernate-pocket-11

Update to latest rockchip-devel For some reason I decided to try re-applying the PCI series. Good news: the pci series finally applies cleanly.

$ git fetch collabora && git switch -c tmp collabora  # [1]
$ b4 am 20250715-pci-port-reset-v6-0-6f9cce94e7bb@oss.qualcomm.com
$ git switch reform-patches  # [2]
$ git rebase -i tmp

1. https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux.git#rockchip-devel
2. https://salsa.debian.org/bremner/collabora-rockchip-3588#reform-patches

Rebuild the kernel

$ cp /boot/config-6.17.0-rc7+ .config
$ make olddefconfig
$ yes ''   make localmodconfig
$ make KBUILD_IMAGE=arch/arm64/boot/Image bindeb-pkg -j$(nproc)

try the hibernation test, again Running the following test script

set -x
echo platform >  /sys/power/pm_test
echo reboot > /sys/power/disk
sleep 2
rmmod mt76x2u
sleep 2
echo disk >  /sys/power/state
sleep 2
modprobe mt76x2u

Initially there is some output like this

[  151.752683] rockchip-dw-pcie a40c00000.pcie: Failed to receive PME_TO_Ack
[  151.754035] PM: hibernation: hibernation debug: Waiting for 5 second(s).
[  157.821584] rockchip-dw-pcie a40c00000.pcie: Phy link never came up
[  157.822139] rockchip-dw-pcie a40c00000.pcie: fail to resume
[  157.822636] rockchip-dw-pcie a40c00000.pcie: PM: dpm_run_callback(): genpd_restore_noirq returns -110
[  157.823442] rockchip-dw-pcie a40c00000.pcie: PM: failed to restore noirq: error -110

A small amount of detective work suggests that a40c00000.pcie corresponds to the first PCI bridge on the rk3588 SOC.

$ ls -l /sys/bus/pci/devices
total 0
lrwxrwxrwx 1 root root 0 Sep 23 10:32 0003:30:00.0 -> ../../../devices/platform/a40c00000.pcie/pci0003:30/0003:30:00.0
lrwxrwxrwx 1 root root 0 Sep 23 10:32 0004:40:00.0 -> ../../../devices/platform/a41000000.pcie/pci0004:40/0004:40:00.0
lrwxrwxrwx 1 root root 0 Sep 23 10:32 0004:41:00.0 -> ../../../devices/platform/a41000000.pcie/pci0004:40/0004:40:00.0/0004:41:00.0

Then after a pause,

[ 1032.039237] watchdog: CPU5: Watchdog detected hard LOCKUP on cpu 6
[ 1032.039778] Modules linked in: xt_CHECKSUM xt_tcpudp nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat x_tables bridge stp llc nf_tables aes_neon_bs aes_neon_blk ccm dwmac_rk binfmt_misc mt76x2_common mt76x02_usb mt76_usb mt76x02_lib mt76 rk805_pwrkey snd_soc_tlv320aic31xx snd_soc_simple_card mac80211 rockchip_saradc reform2_lpc(OE) industrialio_triggered_buffer libarc4 kfifo_buf cfg80211 industrialio rockchip_thermal rockchip_rng cdc_acm rfkill snd_soc_rockchip_i2s_tdm hantro_vpu rockchip_rga panthor v4l2_vp9 v4l2_jpeg snd_soc_audio_graph_card videobuf2_dma_sg v4l2_h264 drm_gpuvm snd_soc_simple_card_utils drm_exec evdev joydev dm_mod nvme_fabrics efi_pstore configfs nfnetlink autofs4 ext4 crc16 mbcache jbd2 btrfs blake2b_generic xor xor_neon raid6_pq mali_dp snd_soc_meson_axg_toddr snd_soc_meson_axg_fifo snd_soc_meson_codec_glue panfrost drm_shmem_helper gpu_sched ao_cec_g12a meson_vdec(C) videobuf2_dma_contig videobuf2_memops v4l2_mem2mem videobuf2_v4l2 videodev
[ 1032.039834]  videobuf2_common mc dw_hdmi_i2s_audio meson_drm meson_canvas meson_dw_mipi_dsi meson_dw_hdmi mxsfb mux_mmio panel_edp imx_dcss ti_sn65dsi86 nwl_dsi mux_core pwm_imx27 hid_generic usbhid hid onboard_usb_dev nvme nvme_core nvme_keyring nvme_auth snd_soc_hdmi_codec snd_soc_core xhci_plat_hcd xhci_hcd snd_pcm_dmaengine snd_pcm snd_timer snd soundcore rtc_pcf8523 fan53555 micrel phy_package stmmac_platform stmmac pcs_xpcs phylink mdio_devres rk808_regulator of_mdio sdhci_of_dwcmshc fixed_phy sdhci_pltfm fwnode_mdio libphy sdhci phy_rockchip_usbdp dw_mmc_rockchip dw_mmc_pltfm typec phy_rockchip_naneng_combphy pwm_rockchip dw_wdt phy_rockchip_samsung_hdptx dwc3 cqhci dw_mmc mdio_bus rockchip_dfi ehci_platform rockchipdrm ulpi ehci_hcd dw_hdmi_qp ohci_platform udc_core ohci_hcd analogix_dp dw_mipi_dsi i2c_rk3x cpufreq_dt usbcore phy_rockchip_inno_usb2 dw_mipi_dsi2 drm_dp_aux_bus usb_common [last unloaded: mt76x2u]
[ 1032.039886] Sending NMI from CPU 5 to CPUs 6:

previous episode

$ git diff --shortstat v1.11.5
 493 files changed, 25015 insertions(+), 21135 deletions(-)

New outlet service The major change in Akvorado 2.0 is splitting the inlet service into two parts: the inlet and the outlet. Previously, the inlet handled all flow processing: receiving, decoding, and enrichment. Flows were then sent to Kafka for storage in ClickHouse:

Akvorado flow processing before the introduction of the outlet service

Network flows reach the inlet service using UDP, an unreliable protocol. The inlet must process them fast enough to avoid losing packets. To handle a high number of flows, the inlet spawns several sets of workers to receive flows, fetch metadata, and assemble enriched flows for Kafka. Many configuration options existed for scaling, which increased complexity for users. The code needed to avoid blocking at any cost, making the processing pipeline complex and sometimes unreliable, particularly the BMP receiver.¹ Adding new features became difficult without making the problem worse.² In Akvorado 2.0, the inlet receives flows and pushes them to Kafka without decoding them. The new outlet service handles the remaining tasks:

Akvorado flow processing after the introduction of the outlet service

This change goes beyond a simple split:³ the outlet now reads flows from Kafka and pushes them to ClickHouse, two tasks that Akvorado did not handle before. Flows are heavily batched to increase efficiency and reduce the load on ClickHouse using ch-go, a low-level Go client for ClickHouse. When batches are too small, asynchronous inserts are used (e20645). The number of outlet workers scales dynamically (e5a625) based on the target batch size and latency (50,000 flows and 5 seconds by default). This new architecture also allows us to simplify and optimize the code. The outlet fetches metadata synchronously (e20645). The BMP component becomes simpler by removing cooperative multitasking (3b9486). Reusing the same RawFlow object to decode protobuf-encoded flows from Kafka reduces pressure on the garbage collector (8b580f). The effect on Akvorado s overall performance was somewhat uncertain, but a user reported 35% lower CPU usage after migrating from the previous version, plus resolution of the long-standing BMP component issue.

Other changes This new version includes many miscellaneous changes, such as completion for source and destination ports (f92d2e), and automatic restart of the orchestrator service (0f72ff) when configuration changes to avoid a common pitfall for newcomers. Let s focus on some key areas for this release: observability, documentation, CI, Docker, Go, and JavaScript.

Observability Akvorado exposes metrics to provide visibility into the processing pipeline and help troubleshoot issues. These are available through Prometheus HTTP metrics endpoints, such as /api/v0/inlet/metrics. With the introduction of the outlet, many metrics moved. Some were also renamed (4c0b15) to match Prometheus best practices. Kafka consumer lag was added as a new metric (e3a778). If you do not have your own observability stack, the Docker Compose setup shipped with Akvorado provides one. You can enable it by activating the profiles introduced for this purpose (529a8f). The prometheus profile ships Prometheus to store metrics and Alloy to collect them (2b3c46, f81299, and 8eb7cd). Redis and Kafka metrics are collected through the exporter bundled with Alloy (560113). Other metrics are exposed using Prometheus metrics endpoints and are automatically fetched by Alloy with the help of some Docker labels, similar to what is done to configure Traefik. cAdvisor was also added (83d855) to provide some container-related metrics. The loki profile ships Loki to store logs (45c684). While Alloy can collect and ship logs to Loki, its parsing abilities are limited: I could not find a way to preserve all metadata associated with structured logs produced by many applications, including Akvorado. Vector replaces Alloy (95e201) and features a domain-specific language, VRL, to transform logs. Annoyingly, Vector currently cannot retrieve Docker logs from before it was started. Finally, the grafana profile ships Grafana, but the shipped dashboards are broken. This is planned for a future version.

Documentation The Docker Compose setup provided by Akvorado makes it easy to get the web interface up and running quickly. However, Akvorado requires a few mandatory steps to be functional. It ships with comprehensive documentation, including a chapter about troubleshooting problems. I hoped this documentation would reduce the support burden. It is difficult to know if it works. Happy users rarely report their success, while some users open discussions asking for help without reading much of the documentation. In this release, the documentation was significantly improved.

$ git diff --shortstat v1.11.5 -- console/data/docs
 10 files changed, 1873 insertions(+), 1203 deletions(-)

The documentation was updated (fc1028) to match Akvorado s new architecture. The troubleshooting section was rewritten (17a272). Instructions on how to improve ClickHouse performance when upgrading from versions earlier than 1.10.0 was added (5f1e9a). An LLM proofread the entire content (06e3f3). Developer-focused documentation was also improved (548bbb, e41bae, and 871fc5). From a usability perspective, table of content sections are now collapsable (c142e5). Admonitions help draw user attention to important points (8ac894).

Example of use of admonitions in Akvorado's documentation

Continuous integration This release includes efforts to speed up continuous integration on GitHub. Coverage and race tests run in parallel (6af216 and fa9e48). The Docker image builds during the tests but gets tagged only after they succeed (8b0dce).

GitHub workflow to test and build Akvorado

End-to-end tests (883e19) ensure the shipped Docker Compose setup works as expected. Hurl runs tests on various HTTP endpoints, particularly to verify metrics (42679b and 169fa9). For example:

## Test inlet has received NetFlow flows
GET http://127.0.0.1:8080/prometheus/api/v1/query
[Query]
query: sum(akvorado_inlet_flow_input_udp_packets_total job="akvorado-inlet",listener=":2055" )
HTTP 200
[Captures]
inlet_receivedflows: jsonpath "$.data.result[0].value[1]" toInt
[Asserts]
variable "inlet_receivedflows" > 10
## Test inlet has sent them to Kafka
GET http://127.0.0.1:8080/prometheus/api/v1/query
[Query]
query: sum(akvorado_inlet_kafka_sent_messages_total job="akvorado-inlet" )
HTTP 200
[Captures]
inlet_sentflows: jsonpath "$.data.result[0].value[1]" toInt
[Asserts]
variable "inlet_sentflows" >=   inlet_receivedflows  

Docker Akvorado ships with a comprehensive Docker Compose setup to help users get started quickly. It ensures a consistent deployment, eliminating many configuration-related issues. It also serves as a living documentation of the complete architecture. This release brings some small enhancements around Docker:

• an example to configure Traefik for TLS with Let s Encrypt (1a27bb),
• HTTP compression (bee9a5), and
• support of IPv6 (a74a41).

Previously, many Docker images were pulled from the Bitnami Containers library. However, VMWare acquired Bitnami in 2019 and Broadcom acquired VMWare in 2023. As a result, Bitnami images were deprecated in less than a month. This was not really a surprise⁴. Previous versions of Akvorado had already started moving away from them. In this release, the Apache project s Kafka image replaces the Bitnami one (1eb382). Thanks to the switch to KRaft mode, Zookeeper is no longer needed (0a2ea1, 8a49ca, and f65d20). Akvorado s Docker images were previously compiled with Nix. However, building AArch64 images on x86-64 is slow because it relies on QEMU userland emulation. The updated Dockerfile uses multi-stage and multi-platform builds: one stage builds the JavaScript part on the host platform, one stage builds the Go part cross-compiled on the host platform, and the final stage assembles the image on top of a slim distroless image (268e95 and d526ca).

# This is a simplified version
FROM --platform=$BUILDPLATFORM node:20-alpine AS build-js
RUN apk add --no-cache make
WORKDIR /build
COPY console/frontend console/frontend
COPY Makefile .
RUN make console/data/frontend
FROM --platform=$BUILDPLATFORM golang:alpine AS build-go
RUN apk add --no-cache make curl zip
WORKDIR /build
COPY . .
COPY --from=build-js /build/console/data/frontend console/data/frontend
RUN go mod download
RUN make all-indep
ARG TARGETOS TARGETARCH TARGETVARIANT VERSION
RUN make
FROM gcr.io/distroless/static:latest
COPY --from=build-go /build/bin/akvorado /usr/local/bin/akvorado
ENTRYPOINT [ "/usr/local/bin/akvorado" ]

When building for multiple platforms with --platform linux/amd64,linux/arm64,linux/arm/v7, the build steps until the highlighted line execute only once for all platforms. This significantly speeds up the build. Akvorado now ships Docker images for these platforms: linux/amd64, linux/amd64/v3, linux/arm64, and linux/arm/v7. When requesting ghcr.io/akvorado/akvorado, Docker selects the best image for the current CPU. On x86-64, there are two choices. If your CPU is recent enough, Docker downloads linux/amd64/v3. This version contains additional optimizations and should run faster than the linux/amd64 version. It would be interesting to ship an image for linux/arm64/v8.2, but Docker does not support the same mechanism for AArch64 yet (792808).

Go This release includes many changes related to Go but not visible to the users.

Toolchain In the past, Akvorado supported the two latest Go versions, preventing immediate use of the latest enhancements. The goal was to allow users of stable distributions to use Go versions shipped with their distribution to compile Akvorado. However, this became frustrating when interesting features, like go tool, were released. Akvorado 2.0 requires Go 1.25 (77306d) but can be compiled with older toolchains by automatically downloading a newer one (94fb1c).⁵ Users can still override GOTOOLCHAIN to revert this decision. The recommended toolchain updates weekly through CI to ensure we get the latest minor release (5b11ec). This change also simplifies updates to newer versions: only go.mod needs updating. Thanks to this change, Akvorado now uses wg.Go() (77306d) and I have started converting some unit tests to the new test/synctest package (bd787e, 7016d8, and 159085).

Testing When testing equality, I use a helper function Diff() to display the differences when it fails:

got := input.Keys()
expected := []int 1, 2, 3 
if diff := helpers.Diff(got, expected); diff != ""  
    t.Fatalf("Keys() (-got, +want):\n%s", diff)
 

This function uses kylelemons/godebug. This package is no longer maintained and has some shortcomings: for example, by default, it does not compare struct private fields, which may cause unexpectedly successful tests. I replaced it with google/go-cmp, which is stricter and has better output (e2f1df).

Another package for Kafka Another change is the switch from Sarama to franz-go to interact with Kafka (756e4a and 2d26c5). The main motivation for this change is to get a better concurrency model. Sarama heavily relies on channels and it is difficult to understand the lifecycle of an object handed to this package. franz-go uses a more modern approach with callbacks⁶ that is both more performant and easier to understand. It also ships with a package to spawn fake Kafka broker clusters, which is more convenient than the mocking functions provided by Sarama.

Improved routing table for BMP To store its routing table, the BMP component used kentik/patricia, an implementation of a patricia tree focused on reducing garbage collection pressure. gaissmai/bart is a more recent alternative using an adaptation of [Donald Knuth s ART algorithm][] that promises better performance and delivers it: 90% faster lookups and 27% faster insertions (92ee2e and fdb65c). Unlike kentik/patricia, gaissmai/bart does not help efficiently store values attached to each prefix. I adapted the same approach as kentik/patricia to store route lists for each prefix: store a 32-bit index for each prefix, and use it to build a 64-bit index for looking up routes in a map. This leverages Go s efficient map structure. gaissmai/bart also supports a lockless routing table version, but this is not simple because we would need to extend this to the map storing the routes and to the interning mechanism. I also attempted to use Go s new unique package to replace the intern package included in Akvorado, but performance was worse.⁷

Miscellaneous Previous versions of Akvorado were using a custom Protobuf encoder for performance and flexibility. With the introduction of the outlet service, Akvorado only needs a simple static schema, so this code was removed. However, it is possible to enhance performance with planetscale/vtprotobuf (e49a74, and 8b580f). Moreover, the dependency on protoc, a C++ program, was somewhat annoying. Therefore, Akvorado now uses buf, written in Go, to convert a Protobuf schema into Go code (f4c879). Another small optimization to reduce the size of the Akvorado binary by 10 MB was to compress the static assets embedded in Akvorado in a ZIP file. It includes the ASN database, as well as the SVG images for the documentation. A small layer of code makes this change transparent (b1d638 and e69b91).

JavaScript Recently, two large supply-chain attacks hit the JavaScript ecosystem: one affecting the popular packages chalk and debug and another impacting the popular package @ctrl/tinycolor. These attacks also exist in other ecosystems, but JavaScript is a prime target due to heavy use of small third-party dependencies. The previous version of Akvorado relied on 653 dependencies. npm-run-all was removed (3424e8, 132 dependencies). patch-package was removed (625805 and e85ff0, 69 dependencies) by moving missing TypeScript definitions to env.d.ts. eslint was replaced with oxlint, a linter written in Rust (97fd8c, 125 dependencies, including the plugins). I switched from npm to Pnpm, an alternative package manager (fce383). Pnpm does not run install scripts by default⁸ and prevents installing packages that are too recent. It is also significantly faster.⁹ Node.js does not ship Pnpm but it ships Corepack, which allows us to use Pnpm without installing it. Pnpm can also list licenses used by each dependency, removing the need for license-compliance (a35ca8, 42 dependencies). For additional speed improvements, beyond switching to Pnpm and Oxlint, Vite was replaced with its faster Rolldown version (463827). After these changes, Akvorado only pulls 225 dependencies.

Next steps I would like to land three features in the next version of Akvorado:

• Add Grafana dashboards to complete the observability stack. See issue #1906 for details.
• Integrate OVH s Grafana plugin by providing a stable API for such integrations. Akvorado s web console would still be useful for browsing results, but if you want to build and share dashboards, you should switch to Grafana. See issue #1895.
• Move some work currently done in ClickHouse (custom dictionaries, GeoIP and IP enrichment) back into the outlet service. This should give more flexibility for adding features like the one requested in issue #1030. See issue #2006.

---

I started working on splitting the inlet into two parts more than one year ago. I found more motivation in recent months, partly thanks to Claude Code, which I used as a rubber duck. Almost none of the produced code was kept:¹⁰ it is like an intern who does not learn.

---

1. Many attempts were made to make the BMP component both performant and not blocking. See for example PR #254, PR #255, and PR #278. Despite these efforts, this component remained problematic for most users. See issue #1461 as an example.
2. Some features have been pushed to ClickHouse to avoid the processing cost in the inlet. See for example PR #1059.
3. This is the biggest commit:

   $ git show --shortstat ac68c5970e2c   tail -1
   231 files changed, 6474 insertions(+), 3877 deletions(-)
   

4. Broadcom is known for its user-hostile moves. Look at what happened with VMWare.
5. As a Debian developer, I dislike these mechanisms that circumvent the distribution package manager. The final straw came when Go 1.25 spent one month in the Debian NEW queue, an arbitrary mechanism I don t like at all.
6. In the early years of Go, channels were heavily promoted. Sarama was designed during this period. A few years later, a more nuanced approach emerged. See notably Go channels are bad and you should feel bad.
7. This should be investigated further, but my theory is that the intern package uses 32-bit integers, while unique uses 64-bit pointers. See commit 74e5ac.
8. This is also possible with npm. See commit dab2f7.
9. An even faster alternative is Bun, but it is less available.
10. The exceptions are part of the code for the admonition blocks, the code for collapsing the table of content, and part of the documentation.

This post is a review for Computing Reviews for We, Programmers A Chronicle of Coders from Ada to AI , a book published in Addison-Wesley

• It ran on a cron job, not on demand
• It processed all OpenStack releases at once, making it slow
• The rsync from Jenkins hosts to osbpo.debian.net was also cron-driven
• No immediate feedback after a package build

• Jenkins would build the backport
• Store it in a local repository
• Wait up to 30 minutes (or more) for the cron job to run rsync + status update
• Only then would the status page reflect the new version

• Developer git push triggers Jenkins
• Jenkins builds the package publishes to local repo
• Jenkins immediately triggers a webhook on osbpo.debian.net

• rsyncs the new package to the central Debian repo
• Pulls the latest OpenStack releases from git and use its YAML data (instead of parsing the release HTML pages)
• Regenerates the status page, comparing what upstream released and what s in Debian

• webhook to receive secure, HMAC-verified triggers that it processes in an async way
• Apache to serve the status pages and the Debian OpenStack repositories
• Custom scripts to rsync packages, validate, and generate reports

• Add status for Debian stable (ie: without the osbpo.debian.net add-on repository), which we used to have with os-version-checker.
• Add a per-release config file option to be able to mask not packaged project on a per OpenStack release granularity

Changes in RcppArmadillo version 15.0.2-2 (2025-09-18)

• Minor update to skeleton Makevars,Makevars.win
• Update README.md to mention ldlasb2 repository
• Minor documentation update (#487)
• Synchronized with Armadillo upstream (#488)
• Refine Armadillo version selection in coordination with CRAN maintainers to support transition towards Armadillo 15.0.*

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

1. Separate application-specific users with granular permissions allowing only necessary access and no more.
2. Distributing and storing passwords and credentials securely
3. Ensuring all remote connections are properly encrypted

How encrypting database connections with TLS differs from web server HTTP(S) Even though MariaDB (and other databases) use the same SSL/TLS protocol for encrypting remote connections as web servers and HTTPS, the way it is implemented is significantly different, and the different security assumptions are important for a database administrator to grasp. Firstly, most HTTP requests to a web server are unauthenticated, meaning the web server serves public web pages and does not require users to log in. Traditionally, when a user logs in over a HTTP connection, the username and password were transmitted in plaintext as a HTTP POST request. Modern TLS, which was previously called SSL, does not change how HTTP works but simply encapsulates it. When using HTTPS, a web browser and a web server will start an encrypted TLS connection as the very first thing, and only once established, do they send HTTP requests and responses inside it. There are no passwords or other shared secrets needed to form the TLS connection. Instead, the web server relies on a trusted third party, a Certificate Authority (CA), to vet that the TLS certificate offered by the web server can be trusted by the web browser. For a database server like MariaDB, the situation is quite different. All users need to first authenticate and log in to the server before getting being allowed to run any SQL and getting any data out of the server. The database server and client programs have built-in authentication methods, and passwords are not, and have never been, sent in plaintext. Over the years, MySQL and its successor, MariaDB, have had multiple password authentication methods: the original SHA-1-based hashing, later double SHA-1-based mysql_native_password, followed by sha256_password and caching_sha256_password in MySQL and ed25519 in MariaDB. The MariaDB.org blog post by Sergei Golubchik recaps the history of these well. Even though most modern MariaDB installations should be using TLS to encrypt all remote connections in 2025, having the authentication method be as secure as possible still matters, because authentication is done before the TLS connection is fully established. To further harden the authentication agains man-in-the-middle attacks, a new password the authentication method PARSEC was introduced in MariaDB 11.8, which builds upon the previous ed25519 public-key-based verification (similar to how modern SSH does), and also combines key derivation with PBKDF2 with hash functions (SHA512,SHA256) and a high iteration count. At first it may seem like a disadvantage to not wrap all connections in a TLS tunnel like HTTPS does, but actually not having the authentication done in a MitM resistant way regardless of the connection encryption status allows a clever extra capability that is now available in MariaDB: as the database server and client already have a shared secret that is being used by the server to authenticate the user, it can also be used by the client to validate the server s TLS certificate and no third parties like CAs or root certificates are needed. MariaDB 11.8 was the first LTS version to ship with this capability for zero-configuration TLS. Note that the zero-configuration TLS also works on older password authentication methods and does not require users to have PARSEC enabled. As PARSEC is not yet the default authentication method in MariaDB, it is recommended to enable it in installations that use zero-configuration TLS encryption to maximize the security of the TLS certificate validation.

Why the root user in MariaDB has no password and how it makes the database more secure Relying on passwords for security is problematic, as there is always a risk that they could leak, and a malicious user could access the system using the leaked password. It is unfortunately far too common for database passwords to be stored in plaintext in configuration files that are accidentally committed into version control and published on GitHub and similar platforms. Every application or administrative password that exists should be tracked to ensure only people who need it know it, and rotated at regular intervals to ensure old employees etc won t be able to use old passwords. This password management is complex and error-prone. Replacing passwords with other authentication methods is always advisable when possible. On a database server, whoever installed the database by running e.g. apt install mariadb-server, and configured it with e.g. nano /etc/mysql/mariadb.cnf, already has full root access to the operating system, and asking them for a password to access the MariaDB database shell is moot, since they could circumvent any checks by directly accessing the files on the system anyway. Therefore, MariaDB, since version 10.4 stopped requiring the root user to enter a password when connecting locally, and instead checks using socket authentication whether the user is the operating-system root user or equivalent (e.g. running sudo). This is an elegant way to get rid of a password that was actually unnecessary to begin with. As there is no root password anymore, the risk of an external user accessing the database as root with a leaked password is fully eliminated. Note that socket authentication only works for local connections on the same server. If you want to access a MariaDB server remotely as the root user, you would need to configure a password for it first. This is not generally recommended, as explained in the next section.

Create separate database users for normal use and keep root for administrative use only Out of the box a MariaDB installation is already secure by default, and only the local root user can connect to it. This account is intended for administrative use only, and for regular daily use you should create separate database users with access limited to the databases they need and the permissions required. The most typical commands needed to create a new database for an app and a user the app can use to connect to the database would be the following:

sql Copy

CREATE DATABASE app_db; CREATE USER 'app_user'@'%' IDENTIFIED BY 'your_secure_password'; GRANT ALL PRIVILEGES ON app_db.* TO 'app_user'@'%'; FLUSH PRIVILEGES;

CREATE DATABASE app_db;
CREATE USER 'app_user'@'%' IDENTIFIED BY 'your_secure_password';
GRANT ALL PRIVILEGES ON app_db.* TO 'app_user'@'%';
FLUSH PRIVILEGES;

Alternatively, if you want to use the parsec authentication method, run this to create the user:

sql Copy

CREATE OR REPLACE USER 'app_user'@'%' IDENTIFIED VIA parsec USING PASSWORD('your_secure_password');

CREATE OR REPLACE USER 'app_user'@'%'
 IDENTIFIED VIA parsec
 USING PASSWORD('your_secure_password');

Note that the plugin auth_parsec is not enabled by default. If you see the error message ERROR 1524 (HY000): Plugin 'parsec' is not loaded fix this by running INSTALL SONAME 'auth_parsec';. In the CREATE USER statements, the @'%' means that the user is allowed to connect from any host. This needs to be defined, as MariaDB always checks permissions based on both the username and the remote IP address or hostname of the user, combined with the authentication method. Note that it is possible to have multiple user@remote combinations, and they can have different authentication methods. A user could, for example, be allowed to log in locally using the socket authentication and over the network using a password. If you are running a custom application and you know exactly what permissions are sufficient for the database users, replace the ALL PRIVILEGES with a subset of privileges listed in the MariaDB documentation. For new permissions to take effect, restart the database or run FLUSH PRIVILEGES.

Allow MariaDB to accept remote connections and enforce TLS Using the above 'app_user'@'%' is not enough on its own to allow remote connections to MariaDB. The MariaDB server also needs to be configured to listen on a network interface to accept remote connections. As MariaDB is secure by default, it only accepts connections from localhost until the administrator updates its configuration. On a typical Debian/Ubuntu system, the recommended way is to drop a new custom config in e.g. /etc/mysql/mariadb.conf.d/99-server-customizations.cnf, with the contents:

Copy

[mariadbd] # Listen for connections from anywhere bind-address = 0.0.0.0 # Only allow TLS encrypted connections require-secure-transport = on

[mariadbd]
# Listen for connections from anywhere
bind-address = 0.0.0.0
# Only allow TLS encrypted connections
require-secure-transport = on

For settings to take effect, restart the server with systemctl restart mariadb. After this, the server will accept connections on any network interface. If the system is using a firewall, the port 3306 would additionally need to be allow-listed. To confirm that the settings took effect, run e.g. mariadb -e "SHOW VARIABLES LIKE 'bind_address';" , which should now show 0.0.0.0. When allowing remote connections, it is important to also always define require-secure-transport = on to enforce that only TLS-encrypted connections are allowed. If the server is running MariaDB 11.8 and the clients are also MariaDB 11.8 or newer, no additional configuration is needed thanks to MariaDB automatically providing TLS certificates and appropriate certificate validation in recent versions. On older long-term-supported versions of the MariaDB server one would have had to manually create the certificates and configure the ssl_key, ssl_cert and ssl_ca values on the server, and distribute the certificate to the clients as well, which was cumbersome, so good it is not required anymore. In MariaDB 11.8 the only additional related config that might still be worth setting is tls_version = TLSv1.3 to ensure only the latest TLS protocol version is used. Finally, test connections to ensure they work and to confirm that TLS is used by running e.g.:

shell Copy

mariadb --user=app_user --password=your_secure_password \ --host=192.168.1.66 -e '\s'

mariadb --user=app_user --password=your_secure_password \
 --host=192.168.1.66 -e '\s'

The result should show something along:

Copy

-------------- mariadb from 11.8.3-MariaDB, client 15.2 for debian-linux-gnu (x86_64) ... Current user: app_user@192.168.1.66 SSL: Cipher in use is TLS_AES_256_GCM_SHA384, cert is OK ...

--------------
mariadb from 11.8.3-MariaDB, client 15.2 for debian-linux-gnu (x86_64)
...
Current user: app_user@192.168.1.66
SSL: Cipher in use is TLS_AES_256_GCM_SHA384, cert is OK
...

If running a Debian/Ubuntu system, see the bundled README with zcat /usr/share/doc/mariadb-server/README.Debian.gz to read more configuration tips.

Should TLS encryption be used also on internal networks? If a database server and app are running on the same private network, the chances that the connection gets eavesdropped on or man-in-the-middle attacked by a malicious user are low. However, it is not zero, and if it happens, it can be difficult to detect or prove that it didn t happen. The benefit of using end-to-end encryption is that both the database server and the client can validate the certificates and keys used, log it, and later have logs audited to prove that connections were indeed encrypted and show how they were encrypted. If all the computers on an internal network already have centralized user account management and centralized log collection that includes all database sessions, reusing existing SSH connections, SOCKS proxies, dedicated HTTPS tunnels, point-to-point VPNs, or similar solutions might also be a practical option. Note that the zero-configuration TLS only works with password validation methods. This means that systems configured to use PAM or Kerberos/GSSAPI can t use it, but again those systems are typically part of a centrally configured network anyway and are likely to have certificate authorities and key distribution or network encryption facilities already set up. In a typical software app stac however, the simplest solution is often the best and I recommend DBAs use the end-to-end TLS encryption in MariaDB 11.8 in most cases. Hopefully with these tips you can enjoy having your MariaDB deployments both simpler and more secure than before!

Once the proper measurement points are known, we want to constrain the system such that what it does is simple enough to understand and easy to repeat. It is quite telling that the push for software that enables reproducible builds only really took off after an embarrassing widespread security issue ended up affecting the entire Internet. That there had already been 50 years of software development before anyone thought that introducing a few constraints might be a good idea is, well, let s just say it generates many emotions, none of them happy, fuzzy ones.

Debian Contributions: 2025-08 Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

Preparing for setup.py install deprecation, by Colin Watson setuptools upstream will be removing the setup.py install command on 31 October. While this may not trickle down immediately into Debian, it does mean that in the near future nearly all Python packages will have to use pybuild-plugin-pyproject (though they don t necessarily have to use pyproject.toml; this is just a question of how the packaging runs the build system). Some of the Python team talked about this a bit at DebConf, and Colin volunteered to write up some notes on cases where this isn t straightforward. This page will likely grow as the team works on this problem.

Salsa CI, by Santiago Ruano Rinc n Santiago fixed some pending issues in the MR that moves the pipeline to sbuild+unshare, and after several months, Santiago was able to mark the MR as ready. Part of the recent fixes include handling external repositories, honoring the RELEASE autodetection from d/changelog (thanks to Ahmed Siam for spotting the main reason of the issue), and fixing a regression about the apt resolver for *-backports releases. Santiago is currently waiting for a final review and approval from other members of the Salsa CI team, and being able to merge it. Thanks to all the folks who have helped testing the changes or provided feedback so far. If you want to test the current MR, you need to include the following pipeline definition in your project s CI config file:

---
include:
  - https://salsa.debian.org/santiago/pipeline/raw/sbuild-unshare-02-salsa-ci/salsa-ci.yml
  - https://salsa.debian.org/santiago/pipeline/raw/sbuild-unshare-02-salsa-ci/pipeline-jobs.yml

As a reminder, this MR will make the Salsa CI pipeline build the packages more similar to how it s built by the Debian official builders. This will also save some resources, since the default pipeline will have one stage less (the provisioning) stage, and will make it possible for more projects to be built on salsa.debian.org (including large projects and those from the OCaml ecosystem), etc. See the different issues being fixed in the MR description.

Debian 13 trixie release, by Emilio Pozuelo Monfort On August 9th, Debian 13 trixie was released, building on two years worth of updates and bug fixes from hundreds of developers. Emilio helped coordinate the release, communicating with several teams involved in the process.

DebConf 26 Site Visit, by Stefano Rivera Stefano visited Santa Fe, Argentina, the site for DebConf 26 next year. The aim of the visit was to help build a local team and see the conference venue first-hand. Stefano and Nattie represented the DebConf Committee. The local team organized Debian meetups in Buenos Aires and Santa Fe, where Stefano presented a talk on Debian and DebConf. Venues were scouted and the team met with the university management and local authorities.

Miscellaneous contributions

• Rapha l updated tracker.debian.org after the trixie release to add the new forky release in the set of monitored distributions. He also reviewed and deployed the work of Scott Talbert showing open merge requests from salsa in the action needed panel.
• Rapha l reviewed some DEP-3 changes to modernize the embedded examples in light of the broad git adoption.
• Rapha l configured new workflows on debusine.debian.net to upload to trixie and trixie-security, and officially announced the service on debian-devel-announce, inviting Debian developers to try the service for their next upload to unstable.
• Carles created a merge request for django-compressor upstream to fix an error when concurrent node processing happened. This will allow removing a workaround added in openstack-dashboard and avoid the same bug in other projects that use django-compressor.
• Carles prepared a system to detect packages that Recommends packages which don t exist in unstable. Processed (either reported or ignored due to mis-detected problems or temporary problems) 16% of the reports. Will continue next month.
• Carles got familiar and gave feedback for the freedict-wikdict package. Planned contributions with the maintainer to improve the package.
• Helmut responded to queries related to /usr-move.
• Helmut adapted crossqa.d.n to the release of trixie .
• Helmut diagnosed sufficient failures in rebootstrap to make it work with gcc-15.
• Helmut fixed the CI pipeline of debvm.
• Helmut sent patches for 19 cross build problems.
• Faidon discovered that the Multi-Arch hinter would emit confusing hints about :any annotations. Helmut identified the root cause to be the handling of virtual packages and fixed it.
• Enrico took some dust off python-debiancontributors and prototyped a receiving end for salsa webpings, to start followup work to contributors.debian.org discussions at DebConf25.
• Colin upgraded about 70 Python packages to new upstream versions, which is around 10% of the backlog; this included a complicated Pydantic upgrade in collaboration with the Rust team.
• Colin fixed a bug in debbugs that caused incoming emails to bugs.debian.org with certain header contents to go missing.
• Thorsten uploaded sane-airscan, which was already in experimental, to unstable.
• Thorsten created a script to automate the upload of new upstream versions of foomatic-db. The database contains information about printers and regularly gets an update. Now it is possible to keep the package more up to date in Debian.
• Stefano prepared updates to almost all of his packages that had new versions waiting to upload to unstable. (beautifulsoup4, hatch-vcs, mkdocs-macros-plugin, pypy3, python-authlib, python-cffi, python-mitogen, python-pip, python-pipx, python-progress, python-truststore, python-virtualenv, re2, snowball, soupsieve).
• Stefano uploaded two new python3.13 point releases to unstable.
• Stefano updated distro-info-data in stable releases, to document the trixie release and expected EoL dates.
• Stefano did some debian.social sysadmin work (keeping up quotas with growing databases and filesystems).
• Stefano supported the Debian treasurers in processing some of the DebConf 25 reimbursements.
• Lucas uploaded ruby3.4 to experimental. It was already approved by FTP masters.
• Lucas uploaded ruby-defaults to experimental to add support for ruby3.4. It will allow us to start triggering test rebuilds and catch any FTBFS with ruby3.4.
• Lucas did some administrative work for Google Summer of Code (GSoC) and replied to some queries from mentors and students.
• Anupa helped to organize release parties for Debian 13 and Debian Day events.
• Anupa did the live coverage for the Debian 13 release and prepared the Bits post for the release announcement and 32nd Debian Day as part of the Debian Publicity team.
• Anupa attended a Debian Day event organized by FOSS club SSET as a speaker.

Debian LTS contributors In August, 21 contributors have been paid to work on Debian LTS, their reports are available:

• Abhijith PA did 10.0h (out of 0.0h assigned and 14.0h from previous period), thus carrying over 4.0h to the next month.
• Andrej Shadura did 12.0h (out of 9.0h assigned and 3.0h from previous period).
• Bastien Roucari s did 20.0h (out of 19.75h assigned and 0.25h from previous period).
• Ben Hutchings did 22.75h (out of 16.5h assigned and 6.25h from previous period).
• Carlos Henrique Lima Melara did 10.0h (out of 10.0h assigned).
• Chris Lamb did 18.0h (out of 18.0h assigned).
• Daniel Leidert did 23.25h (out of 23.25h assigned).
• Emilio Pozuelo Monfort did 23.25h (out of 23.25h assigned).
• Guilhem Moulin did 15.0h (out of 15.0h assigned).
• Jochen Sprickerhof did 11.0h (out of 6.0h assigned and 16.75h from previous period), thus carrying over 11.75h to the next month.
• Lee Garrett did 16.25h (out of 0.0h assigned and 16.25h from previous period).
• Lucas Kanashiro did 20.0h (out of 1.25h assigned and 18.75h from previous period).
• Markus Koschany did 5.0h (out of 13.0h assigned and 9.75h from previous period), thus carrying over 17.75h to the next month.
• Paride Legovini did 8.0h (out of 0.0h assigned and 8.0h from previous period).
• Roberto C. S nchez did 7.5h (out of 11.75h assigned and 11.0h from previous period), thus carrying over 15.25h to the next month.
• Santiago Ruano Rinc n did 13.5h (out of 7.25h assigned and 7.75h from previous period), thus carrying over 1.5h to the next month.
• Stefano Rivera did 0.5h (out of 0.0h assigned and 3.0h from previous period), thus carrying over 2.5h to the next month.
• Sylvain Beucler did 10.0h (out of 23.25h assigned), thus carrying over 13.25h to the next month.
• Thorsten Alteholz did 22.75h (out of 22.75h assigned).
• Tobias Frost did 4.0h (out of 0.0h assigned and 12.0h from previous period), thus carrying over 8.0h to the next month.
• Utkarsh Gupta did 16.0h (out of 22.75h assigned), thus carrying over 6.75h to the next month.

Evolution of the situation In August, we released 27 DLAs. The month of August marked the release of Debian 13 (codename trixie ). This is worth noting because it brought with it the return of the customary fast development pace of Debian unstable, which included several contributions from LTS Team members. More on that below. Of the many security updates which were published (and a few non-security updates as well), some notable ones are highlighted here.

• Notable security updates:
  • gnutls28 prepared by Adrian Bunk, fixes several potential denial of service vulnerabilities
  • apache2, prepared by Bastien Roucari s, fixes several vulnerabilities including a potential denial of service and SSL/TLS-related access control
  • mbedtls (original update, regression update) prepared by Andrej Shadura, fixes several potential denial of service and information disclosure vulnerabilities
  • openjdk-17, prepared by Emilio Pozuelo Monfort, fixes several vulnerabilities which could result in denial of service, information disclosure or weakened TLS connections
• Notable non-security updates:
  • distro-info-data, prepared by Stefano Rivera, adds information concerning future Debian and Ubuntu releases
  • ca-certificates-java, prepared by Bastien Roucari s, fixes some bugs which could disrupt future updates

The LTS Team continues to welcome the collaboration of maintainers from across the Debian community. The contributions of maintainers from outside the LTS Team include: postgresql-13 (Christoph Berg), sope (Jordi Mallach), thunderbird (Carsten Schoenert), and iperf3 (Roberto Lumbreras). Finally, LTS Team members also contributed updates of the following packages:

• redis (to stable), prepared by Chris Lamb
• firebird3.0 (to oldstable and stable), prepared by Adrian Bunk
• node-tmp (to oldstable, stable, and unstable), prepared by Adrian Bunk
• openjpeg2 (to oldstable, stable, and unstable), prepared by Adrian Bunk
• apache2 (to oldstable), prepared by Bastien Roucari s
• unbound (to oldstable), prepared by Guilhem Moulin
• luajit (to oldstable), prepared by Guilhem Moulin
• golang-github-gin-contrib-cors (to oldstable and stable), prepared by Thorsten Alteholz
• libcoap3 (to stable), prepared by Thorsten Alteholz
• libcommons-lang-java and libcommons-lang3-java (both to unstable), prepared by Daniel Leidert
• python-flask-cors (to oldstable), prepared by Daniel Leidert

The LTS Team would especially like to thank our many longtime friends and sponsors for their support and collaboration.

Thanks to our sponsors Sponsors that joined recently are in bold.

• Platinum sponsors:
  • Toshiba Corporation (for 119 months)
  • Civil Infrastructure Platform (CIP) (for 87 months)
  • VyOS Inc (for 51 months)
• Gold sponsors:
  • F. Hoffmann-La Roche AG (for 129 months)
  • Akamai - Linode (for 123 months)
  • Babiel GmbH (for 113 months)
  • Plat Home (for 112 months)
  • University of Oxford (for 69 months)
  • Deveryware (for 56 months)
  • EDF SA (for 41 months)
  • Dataport A R (for 16 months)
  • CERN (for 14 months)
• Silver sponsors:
  • Domeneshop AS (for 134 months)
  • Nantes M tropole (for 128 months)
  • Univention GmbH (for 120 months)
  • Universit Jean Monnet de St Etienne (for 120 months)
  • Ribbon Communications, Inc. (for 114 months)
  • Exonet B.V. (for 103 months)
  • Leibniz Rechenzentrum (for 98 months)
  • Minist re de l Europe et des Affaires trang res (for 81 months)
  • Cloudways by DigitalOcean (for 71 months)
  • Dinahosting SL (for 69 months)
  • Platform.sh SAS (for 63 months)
  • Moxa Inc. (for 57 months)
  • sipgate GmbH (for 55 months)
  • OVH US LLC (for 53 months)
  • Tilburg University (for 53 months)
  • GSI Helmholtzzentrum f r Schwerionenforschung GmbH (for 44 months)
  • THINline s.r.o. (for 17 months)
  • Copenhagen Airports A/S (for 11 months)
• Bronze sponsors:
  • Evolix (for 134 months)
  • Seznam.cz, a.s. (for 134 months)
  • Intevation GmbH (for 131 months)
  • Linuxhotel GmbH (for 131 months)
  • Daevel SARL (for 130 months)
  • Megaspace Internet Services GmbH (for 129 months)
  • Greenbone AG (for 128 months)
  • NUMLOG (for 128 months)
  • WinGo AG (for 127 months)
  • Entr ouvert (for 118 months)
  • Adfinis AG (for 116 months)
  • Tesorion (for 111 months)
  • Laboratoire LEGI - UMR 5519 / CNRS (for 110 months)
  • Bearstech (for 102 months)
  • LiHAS (for 102 months)
  • Catalyst IT Ltd (for 97 months)
  • Demarcq SAS (for 91 months)
  • Universit Grenoble Alpes (for 77 months)
  • TouchWeb SAS (for 69 months)
  • SPiN AG (for 66 months)
  • CoreFiling (for 62 months)
  • Institut des sciences cognitives Marc Jeannerod (for 57 months)
  • Observatoire des Sciences de l Univers de Grenoble (for 53 months)
  • Tem Innovations GmbH (for 48 months)
  • WordFinder.pro (for 47 months)
  • CNRS DT INSU R sif (for 46 months)
  • Soliton Systems K.K. (for 41 months)
  • Alter Way (for 39 months)
  • Institut Camille Jordan (for 29 months)
  • SOBIS Software GmbH (for 14 months)
  • Tuxera Inc. (for 5 months)

Changes in RcppArmadillo version 15.0.2-1 (2025-09-08)

• Upgraded to Armadillo release 15.0.2-1 (Medium Roast)
  • Optionally use OpenMP parallelisation for fp16 matrix multiplication
  • Faster vectorisation of cube tubes
• Provide a top-level include file armadillo as fallback (#480)
• Retire the no-longer-needed check for insufficient LAPACK as R now should supply a sufficient libRlapack (when used) (#483)
• Two potential narrowing warnings are avoided via cast

Changes in RcppArmadillo version 15.0.1-1 (2025-09-01)

• Upgraded to Armadillo release 15.0.1-1 (Medium Roast)
  • Workaround for GCC compiler bug involving misoptimisation of complex number multiplication
• This version contains both 'legacy' and 'current' version of Armadillo (see also below). Package authors should set a '#define' to select the 'current' version, or select the 'legacy' version (also chosen as default) if they must. See GitHub issue #475 for more details.
• Updated DESCRIPTION and README.md

Changes in RcppArmadillo version 15.0.0-1 (2025-08-21) (GitHub Only)

• Upgraded to Armadillo release 15.0.0-1 (Medium Roast)
  • C++14 is now the minimum required C++ standard
  • Aded preliminary support for matrices with half-precision fp16 element type
  • Added second form of cond() to allow detection of failures
  • Added repcube()
  • Added .freeze() and .unfreeze() member functions to wall_clock
  • Extended conv() and conv2() to accept the "valid" shape argument
• Also includes Armadillo 14.6.3 as fallback for C++11 compilations
• This new 'dual' setup had been rigorously tested with five interim pre-releases of which several received full reverse-dependency checks

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

• [DLA 4272-1] aide security update of two CVEs where a local attacker can take advantage of these flaws to hide the addition or removal of a file from the the report, tamper with the log output, or cause aide to crash during report printing or database listing.
• [DLA 4284-1] udisks2 security update to fix one CVE related to a possible local privilege escalation.
• [DLA 4285-1] golang-github-gin-contrib-cors security update to fix one CVE related to circumvention of restrictions.
• [#1112054] trixie-pu of golang-github-gin-contrib-cors, prepared and uploaded
• [#1112335] trixie-pu of libcoap3, prepared and uploaded
• [#1112053] bookworm-pu of golang-github-gin-contrib-cors, prepared and uploaded

• [ELA-1499-1] aide security update to fix one embargoed CVE in Stretch, related to a crash. The other CVE mentioned above was not affecting Stretch.
• [ELA-1508-1] udisks2 security update to fix one embargoed CVE in Stretch and Buster.

• sane-airscan to unstable.
• foomatic-db to unstable.

• boinor to unstable.
• openvlbi to unstable.
• c-munipack to unstable.
• supernovas to unstable (sponsored upload).

• radlib to unstable.
• pyicloud to unstable.
• libcoap3 to unstable and adopted it.

• osmo-cbc to unstable.
• libsmpp34 to unstable.

• mpb to unstable.
• puppet-module-cirrax-gitolite to unstable.
• tntdb to unstable.
• mailio to unstable.
• sockstat to unstable.
• mdns-scan to unstable.
• gcal to unstable.

Here on a summer night in the grass and lilac smell
Drunk on the crickets and the starry sky,
Oh what fine stories we could tell
With this moonlight to tell them by. A summer night, and you, and paradise,
So lovely and so filled with grace,
Above your head, the universe has hung its lights,
And I reach out my hand and touch your face.

I believe in impulse, in all that is green,
In the foolish vision that comes out true.
I believe that all that is essential is unseen,
And for this lifetime, I believe in you. All of the lovers and the love they made:
Nothing that was between them was a mistake.
All that we did for love s sake,
Was not wasted and will never fade. All who have loved will be forever young
And walk in grandeur on a summer night
Along the avenue. They live in every song that is sung,
In every painting of pure light,
In every pas de deux. O love that shines from every star,
Love reflected in the silver moon;
It is not here, but it is not far.
Not yet, but it will be here soon.

Welcome to the August 2025 report from the Reproducible Builds project! Welcome to the latest report from the Reproducible Builds project for August 2025. These monthly reports outline what we ve been up to over the past month, and highlight items of news from elsewhere in the increasingly-important area of software supply-chain security. If you are interested in contributing to the Reproducible Builds project, please see the Contribute page on our website. In this report:

1. Reproducible Builds Summit 2025
2. Reproducible Builds and live-bootstrap at WHY2025
3. DALEQ Explainable Equivalence for Java Bytecode
4. Reproducibility regression identifies issue with AppArmor security policies
5. Rust toolchain fixes
6. Distribution work
7. diffoscope
8. Website updates
9. Reproducibility testing framework
10. Upstream patches

Reproducible Builds Summit 2025 Please join us at the upcoming Reproducible Builds Summit, set to take place from October 28th 30th 2025 in Vienna, Austria!** We are thrilled to host the eighth edition of this exciting event, following the success of previous summits in various iconic locations around the world, including Venice, Marrakesh, Paris, Berlin, Hamburg and Athens. Our summits are a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort. During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field. Our aim is to create an inclusive space that fosters collaboration, innovation and problem-solving. If you re interesting in joining us this year, please make sure to read the event page which has more details about the event and location. Registration is open until 20th September 2025, and we are very much looking forward to seeing many readers of these reports there!

Reproducible Builds and live-bootstrap at WHY2025 WHY2025 (What Hackers Yearn) is a nonprofit outdoors hacker camp that takes place in Geestmerambacht in the Netherlands (approximately 40km north of Amsterdam). The event is organised for and by volunteers from the worldwide hacker community, and knowledge sharing, technological advancement, experimentation, connecting with your hacker peers, forging friendships and hacking are at the core of this event . At this year s event, Frans Faase gave a talk on live-bootstrap, an attempt to provide a reproducible, automatic, complete end-to-end bootstrap from a minimal number of binary seeds to a supported fully functioning operating system . Frans talk is available to watch on video and his slides are available as well.

DALEQ Explainable Equivalence for Java Bytecode Jens Dietrich of the Victoria University of Wellington, New Zealand and Behnaz Hassanshahi of Oracle Labs, Australia published an article this month entitled DALEQ Explainable Equivalence for Java Bytecode which explores the options and difficulties when Java binaries are not identical despite being from the same sources, and what avenues are available for proving equivalence despite the lack of bitwise correlation:

[Java] binaries are often not bitwise identical; however, in most cases, the differences can be attributed to variations in the build environment, and the binaries can still be considered equivalent. Establishing such equivalence, however, is a labor-intensive and error-prone process.

Jens and Behnaz therefore propose a tool called DALEQ, which:

disassembles Java byte code into a relational database, and can normalise this database by applying Datalog rules. Those databases can then be used to infer equivalence between two classes. Notably, equivalence statements are accompanied with Datalog proofs recording the normalisation process. We demonstrate the impact of DALEQ in an industrial context through a large-scale evaluation involving 2,714 pairs of jars, comprising 265,690 class pairs. In this evaluation, DALEQ is compared to two existing bytecode transformation tools. Our findings reveal a significant reduction in the manual effort required to assess non-bitwise equivalent artifacts, which would otherwise demand intensive human inspection. Furthermore, the results show that DALEQ outperforms existing tools by identifying more artifacts rebuilt from the same code as equivalent, even when no behavioral differences are present.

Jens also posted this news to our mailing list.

Reproducibility regression identifies issue with AppArmor security policies Tails developer intrigeri has tracked and followed a reproducibility regression in the generation of AppArmor policy caches, and has identified an issue with the 4.1.0 version of AppArmor. Although initially tracked on the Tails issue tracker, intrigeri filed an issue on the upstream bug tracker. AppArmor developer John Johansen replied, confirming that they can reproduce the issue and went to work on a draft patch. Through this, John revealed that it was caused by an actual underlying security bug in AppArmor that is to say, it resulted in permissions not (always) matching what the policy intends and, crucially, not merely a cache reproducibility issue. Work on the fix is ongoing at time of writing.

Rust toolchain fixes Rust Clippy is a linting tool for the Rust programming language. It provides a collection of lints (rules) designed to identify common mistakes, stylistic issues, potential performance problems and unidiomatic code patterns in Rust projects. This month, however, Sosth ne Gu don filed a new issue in the GitHub requesting a new check that would lint against non deterministic operations in proc-macros, such as iterating over a HashMap .

Distribution work In Debian this month:

• Holger made extensive updates to Debian package reproducibility testing infrastructure this month, including:
  • Upgrading all of the nodes to Debian trixie.
  • Adding tests for the new Debian forky release.
  • Dropping tests for Debian bookworm.
  • Dropping support for the armhf architecture. From July 2015, Vagrant Cascadian has been hosting a zoo of approximately 35 armhf systems which were used for building Debian packages for that architecture.
• Holger Levsen also uploaded strip-nondeterminism, our program that improves reproducibility by stripping out non-deterministic information such as timestamps or other elements introduced during packaging. This new version, 1.14.2-1, adds some metadata to aid the deputy tool. ( #1111947)
• 8 reviews of Debian packages were added, 5 were updated and 5 were removed this month adding to our knowledge about identified issues.
• Marc Haber posted to our mailing list this month asking for assistance with the duperemove package in Debian, which appears to be an issue where the order the object files are linked together is dependent on the underlying filesystem . Chris Lamb provided a detailed analysis, including a suggestion that this can be resolved by adding a locale-agnostic sort.

Lastly, Bernhard M. Wiedemann posted another openSUSE monthly update for their work there.

diffoscope diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made the following changes, including preparing and uploading versions, 303, 304 and 305 to Debian:

• Improvements:
  • Use sed(1) backreferences when generating debian/tests/control to avoid duplicating ourselves. [ ]
  • Move from a mono-utils dependency to versioned mono-devel mono-utils dependency, taking care to maintain the [!riscv64] architecture restriction. [ ]
  • Use sed over awk to avoid mangling dependency lines containing = (equals) symbols such as version restrictions. [ ]
• Bug fixes:
  • Fix a test after the upload of systemd-ukify version 258~rc3. [ ]
  • Ensure that Java class files are named .class on the filesystem before passing them to javap(1). [ ]
  • Do not run jsondiff on files over 100KiB as the algorithm runs in O(n^2) time. [ ]
  • Don t check for PyPDF version 3 specifically; check for >= 3. [ ]
• Misc:
  • Update copyright years. [ ][ ]

In addition, Martin Joerg fixed an issue with the HTML presenter to avoid crash when page limit is None [ ] and Zbigniew J drzejewski-Szmek fixed compatibility with RPM 6 [ ]. Lastly, John Sirois fixed a missing requests dependency in the trydiffoscope tool. [ ]

Website updates Once again, there were a number of improvements made to our website this month including:

• Chris Lamb:
  • Write and publish a news entry for the upcoming summit. [ ]
  • Add some assets used at FOSSY, such as the badges and the paper handouts. [ ]
• Holger Levsen:
  • Restructure the new project history pages pages [ ] and add some recent news entries [ ].
  • Mark the OpenWrt tests as disabled on the Who is Involved. [ ]
  • Various changes to the upcoming Reproducible Builds Summit page. [ ]
• Jochen Sprickerhof made various improvements to the Vienna summit page. [ ][ ]
• Mattia Rizzolo also made various improvements to the Vienna summit page. [ ][ ][ ][ ][ ][ ][ ]
• kpcyrd made a number of changes to the new project history pages [ ][ ]

Reproducibility testing framework The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In August, however, a number of changes were made by Holger Levsen, including:

• reproduce.debian.net-related:
  • Run 4 workers on the o4 node again in order to speed up testing. [ ][ ][ ][ ]
  • Also test trixie-proposed-updates and trixie-updates etc. [ ][ ]
  • Gather seperate statistics for each tested release. [ ]
  • Support sources from all Debian suites. [ ]
  • Run new code from the prototype database rework branch for the amd64-pull184 pseudo-architecture. [ ][ ]
  • Add a number of helpful links. [ ][ ][ ][ ][ ][ ][ ][ ][ ]
  • Temporarily call debrebuild without the --cache argument to experiment with a new version of devscripts. [ ][ ][ ]
  • Update public TODO. [ ]
• Installation tests:
  • Add comments to explain structure. [ ]
  • Mark more old jobs as old or dead . [ ][ ][ ]
  • Turn the maintenance job into a no-op. [ ]
• Jenkins node maintenance:
  • Increase penalties if the osuosl5 or ionos7 nodes are down. [ ]
  • Stop trying to fix network automatically. [ ]
  • Correctly mark ppc64el architecture nodes when down. [ ]
  • Upgrade the remaining arm64 nodes to Debian trixie in anticipation of the release. [ ][ ]
  • Allow higher SSD temperatures on the riscv64 architecture. [ ]
• Debian-related:
  • Drop the armhf architecture; many thanks to Vagrant for physically hosting the nodes for ten years. [ ][ ]
  • Add Debian forky, and archive bullseye. [ ][ ][ ][ ][ ][ ][ ]
  • Document the filesystem space savings from dropping the armhf architecture. [ ]
  • Exclude i386 and armhfr from JSON results. [ ]
  • Update TODOs for when Debian trixie and forky have been released. [ ][ ]
• tests.reproducible-builds.org-related:
  • Add a link to reproduce.debian.net. [ ]
  • Improve the dashboard graphs. [ ][ ][ ]
• Misc:
  • Detect errors with openQA erroring out. [ ]
  • Drop the long-disabled openwrt_rebuilder jobs. [ ]
  • Use qa-jenkins-dev@alioth-lists.debian.net as the contact for jenkins.debian.net. [ ]
  • Redirect reproducible-builds.org/vienna25 to reproducible-builds.org/vienna2025. [ ]
  • Disable all OpenWrt reproducible CI jobs, in coordination with the OpenWrt community. [ ][ ]
  • Make reproduce.debian.net accessable via IPv6. [ ]
  • Ignore that the megacli RAID controller requires packages from Debian bookworm. [ ]

In addition,

• James Addison migrated away from deprecated toplevel deb822 Python module in favour of debian.deb822 in the bin/reproducible_scheduler.py script [ ] and removed a note on reproduce.debian.net note after the release of Debian trixie [ ].
• Jochen Sprickerhof made a huge number of improvements to the reproduce.debian.net statistics calculation [ ][ ][ ][ ][ ][ ] as well as to the reproduce.debian.net service more generally [ ][ ][ ][ ][ ][ ][ ][ ].
• Mattia Rizzolo performed a lot of work migrating scripts to SQLAlchemy version 2.0 [ ][ ][ ][ ][ ][ ] in addition to making some changes to the way openSUSE reproducibility tests are handled internally. [ ]
• Lastly, Roland Clobus updated the Debian Live packages after the release of Debian trixie. [ ][ ]

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

• Bernhard M. Wiedemann:
  • cpython
  • hashcat
  • neochat
  • pocketbase
• Chris Lamb:
  • #1111497 filed against neon27.
• Jochen Sprickerhof:
  • #1111620 filed against pcbasic.
  • #1111624 filed against shellia.
  • #1111625 filed against bup.
• Mark Johnston:
  • FreeBSD
  • FreeBSD
  • FreeBSD
  • FreeBSD
• Robin Candau:
  • syd

Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Mastodon: @reproducible_builds@fosstodon.org
• Mailing list: rb-general@lists.reproducible-builds.org

$ grep-dctrl -v -F Testsuite --regex -s Package -n . trixie   cut -d - -f 1   uniq -c   sort -n  tail -20
     50 apertium
     50 kodi
     51 lomiri
     53 maven
     55 libjs
     57 globus
     66 cl
     67 pd
     72 lua
     79 php
     88 puppet
     91 r
    111 gnome
    124 ruby
    140 ocaml
    152 rust
    178 golang
    341 fonts
    557 python
   1072 haskell

#!/bin/sh
set -eu
extract()  
  local release
  local url
  release="$1"
  url="$2"
  if [ ! -f "$ release " ]; then
    rm -f "$ release .gz"
    curl --silent -o $ release .gz "$ url "
    gunzip "$ release .gz"
  fi
  local with_tests
  local total
  with_tests="$(grep-dctrl -c -F Testsuite --regex . $release)"
  total="$(grep-dctrl -c -F Package --regex . $release)"
  echo "  $ release    $ with_tests    $ total    $((100*with_tests/total))%  "
 
echo "  **Release**   **Packages with tests**   **Total number of packages**   **% of packages with tests**  "
echo " ------------- ------------------------- ------------------------------ ------------------------------ "
for release in wheezy jessie stretch buster; do
  extract "$ release " "http://archive.debian.org/debian/dists/$ release /main/source/Sources.gz"
done
for release in bullseye bookworm trixie; do
  extract "$ release " "http://ftp.br.debian.org/debian/dists/$ release /main/source/Sources.gz"
done

apt install opensc

pkcs11-tool --list-objects [-l]

apt install pkcs11-provider

• In [provider_sect] add pkcs11 = pkcs11_sect
• In [default_sect], uncomment activate = 1
• Add this new section:

[pkcs11_sect]
module = /usr/lib/x86_64-linux-gnu/ossl-modules/pkcs11.so
pkcs11-module-path = /usr/lib/x86_64-linux-gnu/pkcs11/opensc-pkcs11.so
default_algorithms = ALL
activate = 1

openssl pkey -in 'pkcs11:id=%01' -pubin -pubout -text

apt install libengine-pkcs11-openssl

openssl cms -nodetach -binary -cades -outform DER -in filename -out filename.p7m -sign -signer 'pkcs11:id=%01' -keyform engine -engine pkcs11

• Debian packages:
  • chrony:
    • Bugs:
      • opened #1111222: 099-scfilter system test is flaky
  • debian-policy:
    • Bugs:
      • opened and replied to #1111126: Copyright format does not explain how to describe a license text itself
  • devscripts:
    • Bugs:
      • opened #1112065: uscan: Regression of handling @PACKAGE@ in upstream URL
  • firmware-free:
    • Merge requests:
      • merged !10: Build CIS files and keyspan_pda firmware from source
  • firmware-nonfree:
    • Bugs:
      • closed #920937: firmware-atheros: Firmware QCA6174 crashes on wakeup from suspend
      • closed #934781: firmware-iwlwifi: iwl4965: Microcode SW error detected
      • closed #988335: RTL8125B Failure of key exchange and association
      • closed #1023631: firmware-misc-nonfree: Possibly missing firmware for Creative Labs Sound Blaster Z PCIe sound card
      • closed #1050852: firmware-misc-nonfree: System crashes and reboots when i915/kbl_dmc_ver1_04.bin is present
      • closed #1061430: firmware-realtek: Realtek 10ec:8136 wrongly identified as Fast Ethernet instead of Gigabit
      • closed #1062817: firmware-amd-graphics: Invalid UVD handle causes the graphics driver to crash
      • closed #1085387: iwlwifi: wifi does not work at startup, starts working after 2-3 reboots and then works fine
      • closed #1093167: firmware-nonfree: AMD NPU firmware is not packaged
      • replied to #1105064: firmware-realtek: Please add rtl_bt/rtl8723ds_fw.bin
      • replied to #1109015: Wifi not working anymore with new atheros firmware from experimental
    • Merge requests:
      • opened and merged !126: Refactor build for next upstream version and forky release cycle
      • opened and merged !127: Update to 20250808
    • Uploads:
      • uploaded version 20250808-1 to unstable
  • initramfs-tools:
    • Bugs:
      • closed #1108204: initramfs-tools: update-initramfs trigger does not update the initramfs in some cases
      • closed #1108924: initramfs-tools: Cannot boot Trixie d-i rc2 USB storage target riscv64 MODULES=most (missing: cdns3 cdns3_starfive)
    • Merge requests:
      • reviewed and closed !182: unmkinitramfs: handle both uppercase and lowercase hex numbers
      • merged !183: hook-functions: add modules for non-default checksums on btrfs
      • opened and merged !184: unmkinitramfs: Accept lower-case hex digits in cpio headers
    • Uploads:
      • uploaded version 0.150 to unstable
  • kernel-handbook:
    • Merge requests:
      • closed !7: Add disabling SECURITY_LOCKDOWN_LSM to chapter-common-tasks
      • reviewed and merged !10: chapter-common-tasks: Update linux headers common binary parameter
  • kernel-team:
    • Merge requests:
      • merged !9: Improvements to find-recent-urgent for meeting agenda
  • ktls-utils:
    • Uploads:
      • uploaded version 1.2.1-2 to unstable
      • uploaded version 1.2.1-3 to unstable
  • kup:
    • Uploads:
      • uploaded version 0.3.6-6 to unstable
  • linux:
    • Bugs:
      • closed #1106386: Bluetooth: btusb: Add RTL8851BE device 13d3:3601
      • replied to and closed #1108271: e2fsprogs: orphan_file prevents ro LVM snapshots from being mounted
      • replied to #1110193: r8169 Rx very slow; r8168 works fine
      • closed #1110780: linux-image-6.12.38+deb13-amd64: HID BPF support is not enabled by default
      • replied to #1111455: linux-image-6.12.41+deb13-amd64: kernel BUG at fs/netfs/read_collect.c:316 netfs: Can t donate prior to front
      • opened and closed #1111670: ext4 regression in 6.16.1
    • Merge requests:
      • reviewed !1382: Store build time signing key encrypted
      • closed !1507: Drop yama: Disable by default and use upstream default to restrict ptrace
      • reviewed !1577: [arm64] udeb: Add USB TYPE-C modules for usb-modules. Then enable MT6359_AUXADC and panel KD070FHFID015 for MediaTek devices
      • merged !1588: linux-misc-tools: Fix FTBFS on sparc64
      • merged !1597: [armhf] Add phy-gmii-sel module to nic-shared-modules udeb
      • merged !1598: d/c/defines.toml: Bump abi_suffix to +deb14 for unstable releases and define forky and trixie-backports releases
      • (LTS) opened !1600: Sign modules using an ephemeral key; bump ABI for every upload
      • merged !1601: Enable DRM_NOUVEAU_GSP_DEFAULT for Nvidia
      • reviewed !1602: Backport Store build time signing key encrypted
      • (LTS) opened and merged !1603: Sign modules using an ephemeral key; bump ABI for every upload
      • opened and merged !1613: bootconfig: Fix negative seeks on 32-bit with LFS enabled
      • opened !1615: Draft: Support for drivers in Rust
      • merged !1622: Sign modules and support lockdown always
    • Uploads:
      • uploaded version 6.16.3-1 to unstable
    • (LTS) Added some regression fixes to the bullseye-security branch, but did not upload it
  • (LTS) linux-6.1:
    • Uploads:
      • uploaded version 6.1.147-1~deb11u1 to bullseye-security
  • linux-base:
    • Bugs:
      • replied to #1111052: Please increase default UDP receive buffer size (rmem_max)
    • Uploads:
      • uploaded version 4.14 to unstable
  • wireless-regdb:
    • Uploads:
      • (LTS) uploaded version 2025.07.10-1~deb12u1 to bookworm
• Debian non-package bugs:
  • general:
    • closed #1111472: general: Big issues with ThinkPad s Dual Mouse (Trackpad/Trackpoint) setup
  • installation-reports:
    • replied to #1111507: firmware-realtek-udeb missing rtl8168e-3.fw (needed for d-i)
  • release.debian.org:
    • (LTS) opened #1110340: bookworm-pu: package wireless-regdb/2025.07.10-1~deb12u1
• Mailing lists:
  • debian-devel:
    • replied to How do I fully test a watch file
    • replied to Please check open Merge Requests before your next upload
    • replied to Request to reconsider i386 (x86) port for Debian 13
  • debian-events-eu:
    • posted Debian Day/release party near Leuven, 16 August
  • debian-kernel:
    • replied to Agenda items for kernel-team meeting on 2025-08-06
    • posted Agenda items for kernel-team meeting on 2025-08-27
    • replied to Kernel Handbook - Out of date ABI maintenance info
  • debian-lts-announce:
    • posted [SECURITY] [DLA 4271-1] linux-6.1 security update
  • debian-www:
    • posted Stale copy of Debian Kernel Handbook
  • initramfs:
    • posted initramfs-tools 0.150 release
  • linux-trace-kernel:
    • posted [PATCH] bootconfig: Fix negative seeks on 32-bit with LFS enabled
  • stable:
    • replied to [PATCH 6.12 164/444] bootconfig: Fix unaligned access when building footer