What did not work Vegan food. I might be spoiled by other conferences. Both at EuroPycon last year (definitely bigger, a lot more commercial) and at PyCon CZ 23 (similar in size, a lot more DIY) there was catering with explicitly vegan options. As I ve mentioned in the beginning, we went to a supermarket before we went to the conference and we had to go there one more time during the conference. I think there was a mixture between a total lack of awareness and a LOT of miscommunication. The breakfasts at the conference consisted of pastries and baguettes - I asked at the first day what the vegan options were and the answer was I don t know, maybe the baguette? and we were asked to only take as much baguette as the people who also got pastries. The lunch was prepared by the Restaurant associatif de Kern vent which is a canteen at the university campus. When we asked if there is vegan food, the people there said that there was only a vegetarian option so we only ate salad. Only later we heard via word of mouth that one has to explicitly ask for a vegan meal which was apparently prepared separatly and you had to find the right person that knows about it (I think thats very Debian-like ). But even then a person once got a vegetarian option offered as vegan food. One problem was also the missing / confusing labeling of the food. At the conference dinner there was apparently vegan food, but it was mixed with all the other food. There were some labels but with hundreds of hungry people around and caterers removing empty plates and dropping off plates with other stuff, everything gets mixed up. In the end we ate bread soaked in olive oil, until the olive oil got taken away by the catering people literally while we were dipping the bread in it. And when these issues were raised, some of the reactions can be summarized as You re holding it wrong which was really frustrating. The dinners at the conference hall were similar. At some point I had the impression that vegan and vegetarian was simply seen as the same thing. If the menus would be written like a debian/copyright file it would probably have looked like this:

Food: *
Diet: Vegan or Vegetarian

But the thing is that Vegan and Vegetarian cannot be mixed. Its similar to non compatible licenses. Once you mix vegan food with vegan food with vegetarian food it s not vegan anymore. Don t get me wrong, I know its hard to organize food for hundreds of people. But if you don t know what it means to provide a vegan option, just communicate the fact so people can look alternatives in advance. During the week some of the vegan people shared food, which was really nice and there were also a lot of non-vegan people who tried to help, organized extra food or simply listened to the hangry rants. Thanks for that!

Paris Saturday was the last day of DebConf and it was a rainy day. On Sunday morning we took the TGV back to Paris and then stayed there for one night because the next night train back to Vienna was on Monday. Luckily the weather was better in Paris. The first thing we did was to look up a vegan burger place. In the evening we strolled along the Seine and had a couple of beers at the Jardins du Trocad ro. Monday the rain also arrived in Paris and we mostly went from one cafe to the next, but also managed to visit Notre Dame.

Conclusio The next DebConf will be in Argentina and I think its likely that DebConf 27 will also not happen anywhere in trainvelling distance. But even if, I think the Mini DebConfs are more my style of happening (there is one planned in Hamburg next spring, and a couple of days ago I learned that there will be a Back to the Future musical show in Hamburg during that time). Nonetheless I had a nice time and I stumbled over some projects I might get more involved in. Thanks also to my travel buddies who put up with me

tl;dr: there is an attack in the wild which is triggering dangerous-but-seemingly-intended behaviour in the Oj JSON parser when used in the default and recommended manner, which can lead to everyone s favourite kind of security problem: object deserialization bugs! If you have the oj gem anywhere in your Gemfile.lock, the quickest mitigation is to make sure you have Oj.default_options = mode: :strict somewhere, and that no library is overwriting that setting to something else.

Prologue As a sensible sysadmin, all the sites I run send me a notification if any unhandled exception gets raised. Mostly, what I get sent is error-handling corner cases I missed, but now and then things get more interesting. In this case, it was a PG::UndefinedColumn exception, which looked something like this:

PG::UndefinedColumn: ERROR:  column "xyzzydeadbeef" does not exist

This is weird on two fronts: firstly, this application has been running for a while, and if there was a schema problem, I d expect it to have made itself apparent long before now. And secondly, while I don t profess to perfection in my programming, I m usually better at naming my database columns than that. Something is definitely hinky here, so let s jump into the mystery mobile!

The column name is coming from outside the building! The exception notifications I get sent include a whole lot of information about the request that caused the exception, including the request body. In this case, the request body was JSON, and looked like this:

 "name":":xyzzydeadbeef", ... 

The leading colon looks an awful lot like the syntax for a Ruby symbol, but it s in a JSON string. Surely there s no way a JSON parser would be turning that into a symbol, right? Right?!? Immediately, I thought that that possibly was what was happening, because I use Sequel for my SQL database access needs, and Sequel treats symbols as database column names. It seemed like too much of a coincidence that a vaguely symbol-shaped string was being sent in, and the exact same name was showing up as a column name. But how the flying fudgepickles was a JSON string being turned into a Ruby symbol, anyway? Enter Oj.

Oj? I barely know aj A long, long time ago, the standard Ruby JSON library had a reputation for being slow. Thus did many competitors flourish, claiming more features and better performance. Strong amongst the contenders was oj (for Optimized JSON ), touted as The fastest JSON parser and object serializer . Given the history, it s not surprising that people who wanted the best possible performance turned to Oj, leading to it being found in a great many projects, often as a sub-dependency of a dependency of a dependency (which is how it ended up in my project). You might have noticed in Oj s description that, in addition to claiming fastest , it also describes itself as an object serializer . Anyone who has kept an eye on the security bug landscape will recall that object deserialization is a rich vein of vulnerabilities to mine. Libraries that do object deserialization, especially ones with a history that goes back to before the vulnerability class was well-understood, are likely to be trouble magnets. And thus, it turns out to be with Oj. By default, Oj will happily turn any string that starts with a colon into a symbol:

>> require "oj"
>> Oj.load(' "name":":xyzzydeadbeef","username":"bob","answer":42 ')
=>  "name"=>:xyzzydeadbeef, "username"=>"bob", "answer"=>42 

How that gets exploited is only limited by the creativity of an attacker. Which I ll talk about more shortly but first, a word from my rant cortex.

Insecure By Default is a Cancer While the object of my ire today is Oj and its fast-and-loose approach to deserialization, it is just one example of a pervasive problem in software: insecurity by default. Whether it s a database listening on 0.0.0.0 with no password as soon as its installed, or a library whose default behaviour is to permit arbitrary code execution, it all contributes to a software ecosystem that is an appalling security nightmare. When a user (in this case, a developer who wants to parse JSON) comes across a new piece of software, they have by definition no idea what they re doing with that software. They re going to use the defaults, and follow the most easily-available documentation, to achieve their goal. It is unrealistic to assume that a new user of a piece of software is going to do things the right way , unless that right way is the only way, or at least the by-far-the-easiest way. Conversely, the developer(s) of the software is/are the domain experts. They have knowledge of the problem domain, through their exploration while building the software, and unrivalled expertise in the codebase. Given this disparity in knowledge, it is tantamount to malpractice for the experts the developer(s) to off-load the responsibility for the safe and secure use of the software to the party that has the least knowledge of how to do that (the new user). To apply this general principle to the specific case, take the Using section of the Oj README. The example code there calls Oj.load, with no indication that this code will, in fact, parse specially-crafted JSON documents into Ruby objects. The brand-user user of the library, no doubt being under pressure to Get Things Done, is almost certainly going to look at this Using example, get the apparent result they were after (a parsed JSON document), and call it a day. It is unlikely that a brand-new user will, for instance, scroll down to the Further Reading section, find the second last (of ten) listed documents, Security.md , and carefully peruse it. If they do, they ll find an oblique suggestion that parsing untrusted input is never a good idea . While that s true, it s also rather unhelpful, because I d wager that by far the majority of JSON parsed in the world is untrusted , in one way or another, given the predominance of JSON as a format for serializing data passing over the Internet. This guidance is roughly akin to putting a label on a car s airbags that driving at speed can be hazardous to your health : true, but unhelpful under the circumstances. The solution is for default behaviours to be secure, and any deviation from that default that has the potential to degrade security must, at the very least, be clearly labelled as such. For example, the Oj.load function should be named Oj.unsafe_load, and the Oj.load function should behave as the Oj.safe_load function does presently. By naming the unsafe function as explicitly unsafe, developers (and reviewers) have at least a fighting chance of recognising they re doing something risky. We put warning labels on just about everything in the real world; the same should be true of dangerous function calls. OK, rant over. Back to the story.

But how is this exploitable? So far, I ve hopefully made it clear that Oj does some Weird Stuff with parsing certain JSON strings. It caused an unhandled exception in a web application I run, which isn t cool, but apart from bombing me with exception notifications, what s the harm? For starters, let s look at our original example: when presented with a symbol, Sequel will interpret that as a column name, rather than a string value. Thus, if our save an update to the user code looked like this:

# request_body has the JSON representation of the form being submitted
body = Oj.load(request_body)
DB[:users].where(id: user_id).update(name: body["name"])

In normal operation, this will issue an SQL query along the lines of UPDATE users SET name='Jaime' WHERE id=42. If the name given is Jaime O Dowd , all is still good, because Sequel quotes string values, etc etc. All s well so far. But, imagine there is a column in the users table that normally users cannot read, perhaps admin_notes. Or perhaps an attacker has gotten temporary access to an account, and wants to dump the user s password hash for offline cracking. So, they send an update claiming that their name is :admin_notes (or :password_hash). In JSON, that ll look like "name":":admin_notes" , and Oj.load will happily turn that into a Ruby object of "name"=>:admin_notes . When run through the above update the user code fragment, it ll produce the SQL UPDATE users SET name=admin_notes WHERE id=42. In other words, it ll copy the contents of the admin_notes column into the name column which the attacker can then read out just by refreshing their profile page.

But Wait, There s More! That an attacker can read other fields in the same table isn t great, but that s barely scratching the surface. Remember before I said that Oj does object serialization ? That means that, in general, you can create arbitrary Ruby objects from JSON. Since objects contain code, it s entirely possible to trigger arbitrary code execution by instantiating an appropriate Ruby object. I m not going to go into details about how to do this, because it s not really my area of expertise, and many others have covered it in detail. But rest assured, if an attacker can feed input of their choosing into a default call to Oj.load, they ve been handed remote code execution on a platter.

Mitigations As Oj s object deserialization is intended and documented behaviour, don t expect a future release to make any of this any safer. Instead, we need to mitigate the risks. Here are my recommended steps:

1. Look in your Gemfile.lock (or SBOM, if that s your thing) to see if the oj gem is anywhere in your codebase. Remember that even if you don t use it directly, it s popular enough that it is used in a lot of places. If you find it in your transitive dependency tree anywhere, there s a chance you re vulnerable, limited only by the ingenuity of attackers to feed crafted JSON into a deeply-hidden Oj.load call.
2. If you depend on oj directly and use it in your project, consider not doing that. The json gem is acceptably fast, and JSON.parse won t create arbitrary Ruby objects.
3. If you really, really need to squeeze the last erg of performance out of your JSON parsing, and decide to use oj to do so, find all calls to Oj.load in your code and switch them to call Oj.safe_load.
4. It is a really, really bad idea to ever use Oj to deserialize JSON into objects, as it lacks the safety features needed to mitigate the worst of the risks of doing so (for example, restricting which classes can be instantiated, as is provided by the permitted_classes argument to Psych.load). I d make it a priority to move away from using Oj for that, and switch to something somewhat safer (such as the aforementioned Psych). At the very least, audit and comment heavily to minimise the risk of user-provided input sneaking into those calls somehow, and pass mode: :object as the second argument to Oj.load, to make it explicit that you are opting-in to this far more dangerous behaviour only when it s absolutely necessary.
5. To secure any unsafe uses of Oj.load in your dependencies, consider setting the default Oj parsing mode to :strict, by putting Oj.default_options = mode: :strict somewhere in your initialization code (and make sure no dependencies are setting it to something else later!). There is a small chance that this change of default might break something, if a dependency is using Oj to deliberately create Ruby objects from JSON, but the overwhelming likelihood is that Oj s just being used to parse ordinary JSON, and these calls are just RCE vulnerabilities waiting to give you a bad time.

Is Your Bacon Saved? If I ve helped you identify and fix potential RCE vulnerabilities in your software, or even just opened your eyes to the risks of object deserialization, please help me out by buying me a refreshing beverage. I would really appreciate any support you can give. Alternately, if you d like my help in fixing these (and many other) sorts of problems, I m looking for work, so email me.

Changes in version 0.0.16 (2025-07-23)

• Synchronized with QuantLib 1.39 released today
• Calendar updates for Israel, minor utility functions update
• Minor package maintenance updates

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

• The asterisk package once again didn t make it into trixie (see #1031046)
• The new debian-repro-status package provides the identically named command-line tool debian-repro-status to query the reproducibility status of your installed Debian packages
• The Grml live system project provided further of their packages into Debian. Available as of trixie are now also grml-keyring (OpenPGP certificates used for signing the Grml repositories), grml-hwinfo (a tool which collects information of the hardware ) + grml-paste (command line interface for paste.debian.net)
• If you use pacemaker, be aware that its fence-agents package is now a transitional package. All the fence-agents got split into separate packages (fence-agents-$whatever). If you want to have all the fence-agents available, make sure to install the fence-agents-all package. If you have Recommends disabled, you definitely should be aware of this.
• usrmerge is finalized (also see dpkg warning issue in release notes)
• For an overview of the XMPP/Jabber situation in trixie see xmpp-team s blog post
• The curl package now includes the wcurl command line tool, being a simple wrapper around curl to easily download files

• support for colors (f.e. green for installs/upgrades, yellow for downgrades, red for removals, can be disabled via &dash&dashno-color, APT_NO_COLOR=1 or NO_COLOR=1 and customized via e.g. APT::Color::Action::Install cyan )
• organizes output in more readable sections and shows removals more prominently
• uses sequoia to verify signatures
• includes a new solver
• the new apt modernize-sources command converts /etc/apt/sources.list.d/*.list files into the new .sources format (AKA DEB822)
• the new apt distclean command removes all files under $statedir/lists except Release, Release.gpg, and InRelease (it can be used for example, when finalizing images distributed to users)
• new configuration option APT::NeverAutoRemove::KernelCount for keeping a configurable amount of kernels, f.e. setting APT::NeverAutoRemove::KernelCount 3 will keep 3 kernels (including the running, and most recent)
• new command line option &dash&dashsnapshot, and configuration option APT::Snapshot, controlling the snapshot chosen for archives with Snapshot: enable
• new command line option &dash&dashupdate to run the update command before the specified command, like apt &dash&dashupdate install zsh,
  apt &dash&dashupdate remove foobar or apt &dash&dashupdate safe-upgrade
• apt-key is gone, and there s no replacement for it available (if you need an interface for listing present keys)

• run0: temporarily and interactively acquire elevated or different privileges (serves a similar purpose as sudo)
• systemd-ac-power: Report whether we are connected to an external power source
• systemd-confext: Activates System Extension Images
• systemd-vpick: Resolve paths to .v/ versioned directories
• varlinkctl: Introspect with and invoke Varlink services

• busctl: new option &dash&dashlimit&dashmessages=NUMBER (Stop monitoring after receiving the specified number of message)
• hostnamectl: new option &dashj (same as &dash&dashjson=pretty on tty, &dash&dashjson=short otherwise)
• journalctl: new options &dash&dashimage&dashpolicy=POLICY (Specify disk image dissection policy), &dash&dashinvocation=ID (Show logs from the matching invocation ID), &dashI (Show logs from the latest invocation of unit), &dash&dashexclude-identifier=STRING (Hide entries with the specified syslog identifier),&dash&dashtruncate-newline (Truncate entries by first newline character), &dash&dashlist-invocations (Show invocation IDs of specified unit), &dash&dashlist-namespaces (Show list of journal namespaces)
• kernel-install: new commands add&dashall + list and plenty of new command line options
• localectl: new option &dash&dashfull (Do not ellipsize output)
• loginctl: new options &dash&dashjson=MODE (Generate JSON output for list-sessions/users/seats) + &dashj (Same as &dash&dashjson=pretty on tty, &dash&dashjson=short otherwise)
• networkctl: new commands edit FILES DEVICES (Edit network configuration files), cat [FILES DEVICES ] (Show network configuration files), mask FILES (Mask network configuration files) + unmask FILES (Unmask network configuration files) + persistent-storage BOOL (Notify systemd-networkd if persistent storage is ready), and new options &dash&dashno-ask-password (Do not prompt for password), &dash&dashno-reload (Do not reload systemd-networkd or systemd-udevd after editing network config), &dash&dashdrop-in=NAME (Edit specified drop-in instead of main config file), &dash&dashruntime (Edit runtime config files) + &dash&dashstdin (Read new contents of edited file from stdin)
• systemctl new commands list-paths [PATTERN] (List path units currently in memory, ordered by path), whoami [PID ] (Return unit caller or specified PIDs are part of), soft-reboot (Shut down and reboot userspace) + sleep (Put the system to sleep), and new options &dash&dashcapsule=NAME (Connect to service manager of specified capsule), &dash&dashbefore (Show units ordered before with list-dependencies ), &dash&dashafter (Show units ordered after with list-dependencies ), &dash&dashkill-value=INT (Signal value to enqueue), &dash&dashno-warn (Suppress several warnings shown by default), &dash&dashmessage=MESSAGE (Specify human readable reason for system shutdown), &dash&dashimage&dashpolicy=POLICY (Specify disk image dissection policy), &dash&dashreboot&dashargument=ARG (Specify argument string to pass to reboot()), &dash&dashdrop-in=NAME (Edit unit files using the specified drop-in file name), &dash&dashwhen=TIME (Schedule halt/power-off/reboot/kexec action after a certain timestamp) + &dash&dashstdin (Read
  new contents of edited file from stdin)
• systemd-analyze new commands architectures [NAME ] (List known architectures), smbios11 (List strings passed via SMBIOS Type #11), image-policy POLICY (Analyze image policy string), fdstore SERVICE (Show file descriptor store contents of service), malloc [D-BUS SERVICE ] (Dump malloc stats of a D-Bus service), has-tpm2 (Report whether TPM2 support is available), pcrs [PCR ] (Show TPM2 PCRs and their names) + srk [>FILE] (Write TPM2 SRK (to FILE)) and new options &dash&dashno-legend (Disable column headers and hints in plot with either &dash&dashtable or &dash&dashjson=), &dash&dashinstance=NAME (Specify fallback instance name for template units), &dash&dashunit=UNIT (Evaluate conditions and asserts of unit), &dash&dashtable (Output plot s raw time data as a table), &dash&dashscale-svg=FACTOR (Stretch x-axis of plot by FACTOR (default: 1.0)), &dash&dashdetailed (Add more details to SVG plot), &dash&dashtldr (Skip comments and empty lines), &dash&dashimage
  -policy=POLICY (Specify disk image dissection policy) + &dash&dashmask (Parse parameter as numeric capability mask)
• systemd-ask-password: new options &dash&dashuser (Ask only our own user s agents) + &dash&dashsystem (Ask agents of the system and of all users)
• systemd-cat: new option &dash&dashnamespace=NAMESPACE (Connect to specified journal namespace)
• systemd-creds: new options &dash&dashuser (Select user-scoped credential encryption), &dash&dashuid=UID (Select user for scoped credentials) + &dash&dashallow-null (Allow decrypting credentials with empty key)
• systemd-detect-virt: new options &dash&dashcvm (Only detect whether we are run in a confidential VM) + &dash&dashlist-cvm (List all known and detectable types of confidential virtualization)
• systemd-firstboot: new options &dash&dashimage-policy=POLICY (Specify disk image dissection policy), &dash&dashkernel-command-line=CMDLINE (Set kernel command line) + &dash&dashreset (Remove existing files)
• systemd-id128: new commands var-partition-uuid (Print the UUID for the /var/ partition) + show [NAME UUID] (Print one or more UUIDs), and new options &dash&dashno-pager (Do not pipe output into a pager), &dash&dashno-legend (Do not show the headers and footers), &dash&dashjson=FORMAT (Output inspection data in JSON), &dashj (Equivalent to &dash&dashjson=pretty (on TTY) or &dash&dashjson=short (otherwise)) + &dashP &dash&dashvalue (Only print the value)
• systemd-inhibit: new option &dash&dashno-ask-password (Do not attempt interactive authorization)
• systemd-machine-id-setup: new option &dash&dashimage-policy=POLICY (Specify disk image dissection policy)
• systemd-mount: new options &dash&dashjson=pretty short off (Generate JSON output) + &dash&dashtmpfs (Create a new tmpfs on the mount point)
• systemd-notify: new options &dash&dashreloading (Inform the service manager about configuration reloading), &dash&dashstopping (Inform the service manager about service shutdown), &dash&dashexec (Execute command line separated by ; once done), &dash&dashfd=FD (Pass specified file descriptor with along with message) + &dash&dashfdname=NAME (Name to assign to passed file descriptor(s))
• systemd-path: new option &dash&dashno-pager (Do not pipe output into a pager)
• systemd-run: new options &dash&dashexpand-environment=BOOL (Control expansion of environment variables), &dash&dashjson=pretty short off (Print unit name and invocation id as JSON), &dash&dashignore-failure (Ignore the exit status of the invoked process) + &dash&dashbackground=COLOR (Set ANSI color for background)
• systemd-sysext: new options &dash&dashmutable=yes no auto import ephemeral ephemeral-import (Specify a mutability mode of the merged hierarchy), &dash&dashno-reload (Do not reload the service manager), &dash&dashimage-policy=POLICY (Specify disk image dissection policy) + &dash&dashnoexec=BOOL (Whether to mount extension overlay with noexec)
• systemd-sysusers: new options &dash&dashtldr (Show non-comment parts of configuration) + &dash&dashimage-policy=POLICY (Specify disk image dissection policy)
• systemd-tmpfiles: new command &dash&dashpurge(Delete files and directories marked for creation in specified configuration files (careful!)), and new options &dash&dashuser (Execute user configuration), &dash&dashtldr (Show non-comment parts of configuration files), &dash&dashgraceful (Quietly ignore unknown users or groups), &dash&dashimage-policy=POLICY (Specify disk image dissection policy) + &dash&dashdry-run (Just print what would be done)
• systemd-umount: new options &dash&dashjson=pretty short off (Generate JSON output) + &dash&dashtmpfs (Create a new tmpfs on the mount point)
• timedatectl: new commands ntp-servers INTERFACE SERVER (Set the interface specific NTP servers) + revert INTERFACE (Revert the interface specific NTP servers) and new option &dashP NAME (Equivalent to &dash&dashvalue &dash&dashproperty=NAME)

• systemd-boot-efi-amd64-signed (Tools to manage UEFI firmware updates (signed))
• systemd-boot-tools (simple UEFI boot manager tools)
• systemd-cryptsetup (Provides cryptsetup, integritysetup and veritysetup utilities)
• systemd-netlogd (journal message forwarder)
• systemd-repart (Provides the systemd-repart and systemd-sbsign utilities)
• systemd-standalone-shutdown (standalone shutdown binary for use in exitrds)
• systemd-ukify (tool to build Unified Kernel Images)

• New Debian package linux-bpf-dev, providing the header file for BPF CO-RE builds
• New Debian package intel-sdsi, Intel On Demand (SDSi) provisioning tool
• New Debian package virtme-ng, providing helper scripts to easily test a Linux kernel on a QEMU VM
• The kernel modules are installed xz compressed
• Several new syscalls, like cachestat, fchmodat2, futex_wake, futex_wait, futex_requeue, listmount, statmount, lsm_get_self_attr/lsm_set_self_attr/lsm_list_modules/, mseal + setxattrat/getxattrat/listxattrat/removexattrat
• New Integrity Policy Enforcement (IPE) LSM
• Plenty of changes in io_uring, including support for bind/listen
• support devices with a block size larger than system page sizes in the XFS filesystem + VFS
• ntfs driver was replaced by ntfs3
• Device Memory TCP for faster network device transfers
• Support for perf ftrace profile
• support for atomic write operations in the block layer
• FUSE pass-through for file I/O
• ability to prevent writes to block devices containing mounted filesystems
• support for data-type profiling in the perf tool
• Guest-first memory for KVM
• TCP Authentication Option Linux implementation [RFC5925]
• mount beneath support for filesystems
• new noswap mount option for tmpfs filesystems
• Linux NFS server gained support for RPC-with-TLS [RFC 9289 (Towards Remote Procedure Call Encryption by Default)]
• PLB (Protective Load Balancing) for IPv6 (PLB is a host based mechanism for load balancing across switch links)

• prometheus-dnsmasq-exporter
• prometheus-mysqlrouter-exporter
• prometheus-pgbackrest-exporter
• prometheus-pgbouncer-exporter
• prometheus-phpfpm-exporter

• blkdiscard: new option &dash&dashquiet (suppress warning messages)
• blockdev: new options &dash&dashgetdiskseq (get disk sequence number) + &dash&dashgetzonesz (get zone size)
• dmesg: new option &dash&dashkmsg-file (use the file in kmsg format), new &dash&dashtime-format argument raw
• findmnt: new options &dash&dashlist-columns (list the available columns), &dash&dashdfi (imitate the output of df(1) with -i option), &dash&dashid (filter by mount node ID), &dash&dashfilter (apply display filter) + &dash&dashuniq-id (filter by
  mount node 64-bit ID)
• fstrim: new option -types . (limit the set of filesystem types)
• hardlink: new options &dash&dashrespect-dir (directory names have to be identical), &dash&dashexclude-subtree (regular expression to exclude directories), &dash&dashprioritize-trees (files found in the earliest specified top-level directory have higher priority), &dash&dashlist-duplicates (print every group of duplicate files), &dash&dashmount (stay within the same filesystem) + &dash&dashzero (delimit output with NULs instead of newlines)
• ipcmk: new options &dash&dashposix-shmem (create POSIX shared memory segment of size), &dash&dashposix-semaphore (create POSIX semaphore), &dash&dashposix-mqueue (create POSIX message queue) + &dash&dashname (name of the POSIX resource)
• ipcrm: new options &dash&dashposix-shmem (remove POSIX shared memory segment by name), &dash&dashposix-mqueue (remove POSIX message queue by name), &dash&dashposix-semaphore (remove POSIX semaphore by name) + &dash&dashall= (remove all in specified category)
• lsblk: new options &dash&dashct-filter (restrict the next counter), &dash&dashct (define a custom counter), &dash&dashhighlight (colorize lines matching the expression), &dash&dashlist-columns (list the available columns), &dash&dashnvme (output info about NVMe devices), &dash&dashproperties-by (methods used to gather data), &dash&dashfilter (print only lines matching the expression), &dash&dashvirtio (output info about virtio devices)
• lscpu: new options &dash&dashraw (use raw output format (for -e, -p and -C)) + &dash&dashhierarchic= (use subsections in summary (auto, never, always))
• lsipc: new options &dash&dashposix-shmems (POSIX shared memory segments), &dash&dashposix-mqueues (POSIX message queues), &dash&dashposix-semaphores (POSIX semaphores), &dash&dashname (POSIX resource identified by name)
• lslocks: new option &dash&dashlist-columns (list the available columns)
• lslogins: new option &dash&dashlastlog2 (set an alternate path for lastlog2)
• lsns: new options &dash&dashpersistent (namespaces without processes), &dash&dashfilter (apply display filter) + &dash&dashlist-columns (list the available columns)
• mkswap: new options &dash&dashendianness= (specify the endianness to use (native, little or big)), &dash&dashoffset (specify the offset in the device), &dash&dashsize (specify the size of a swap file in bytes) + &dash&dashfile (create a swap file)
• namei: &dash&dashcontext (print any security context of each file)
• nsenter: new options &dash&dashnet-socket (enter socket s network namespace), &dash&dashuser-parent (enter parent user namespace), &dash&dashkeep-caps (retain capabilities granted in user namespaces), &dash&dashenv (inherit environment variables from tar get process) + &dash&dashjoin-cgroup (join the cgroup of the target process)
• runuser: new option &dash&dashno-pty (do not create a new pseudo-terminal)
• setarch: new option &dash&dashshow= (show current or specific personality and exit)
• setpriv: new options &dash&dashptracer (allow ptracing from the given process), &dash&dashlandlock-access (add Landlock access), &dash&dashlandlock-rule (add Landlock rule) + &dash&dashseccomp-filter (load seccomp filter from file)
• su: new option &dash&dashno-pty (do not create a new pseudo-terminal)
• unshare: new option &dash&dashload-interp ( load binfmt definition in the namespace)
• whereis: new option -g (interpret name as glob (pathnames pattern))
• wipefs: new argument option feature for &dash&dashbackup= option to specify directory (instead of default $HOME)
• zramctl: new option &dash&dashalgorithm-params (algorithm parameters to use)

• addpart (tell the kernel about the existence of a specified partition): use partx instead
• delpart (tell the kernel to forget about a specified partition): use partx instead
• last (show a listing of last logged in users, binary got moved to wtmpdb), lastb (show a listing of last logged in users), mesg (control write access of other users to your terminal), utmpdump (dump UTMP and WTMP files in raw format): see Debian release notes for details

• ctrlaltdel (set the function of the Ctrl-Alt-Del combination)
• mkfs.bfs (make an SCO bfs filesystem)
• fsck.cramfs + mkfs.cramfs (compressed ROM file system)
• fsck.minix + mkfs.minix (Minix filesystem)
• resizepart (tell the kernel about the new size of a partition)

• bits: convert bit masks from/to various formats
• blkpr: manage persistent reservations on a device
• coresched: manage core scheduling cookies for tasks
• enosys: utility to make syscalls fail with ENOSYS
• exch: atomically exchanges paths between two files
• fadvise: utility to use the posix_fadvise system call
• pipesz: set or examine pipe buffer sizes and optionally execute command.
• waitpid: utility to wait for arbitrary processes

• OpenSSH no longer supports DSA keys: see Debian s release notes for further details
• openssh-server no longer reads ~/.pam_environment: see Debian s release notes for further details

• allow forwarding Unix Domain sockets via ssh -W
• OpenSSH penalty behavior: visit my separate blog post for more details
• add support for reading ED25519 private keys in PEM PKCS8 format. Previously only the OpenSSH private key format was supported.
• the new hybrid post-quantum algorithm mlkem768x25519-sha256 (based on the FIPS 203 Module-Lattice Key Encapsulation mechanism (ML-KEM) combined with X25519 ECDH) is now used by default for key agreement. This algorithm is considered to be safe against attack by quantum computers, is guaranteed to be no less strong than the popular curve25519-sha256 algorithm, has been standardised by NIST and is considerably faster than the previous default.
• the ssh-agent will now delete all loaded keys when signaled with SIGUSR1. This allows deletion of keys without having access to $SSH_AUTH_SOCK.
• support systemd-style socket activation in ssh-agent using the LISTEN_PID/LISTEN_FDS mechanism. Activated when these environment variables are set, the agent is started with the -d or -D option and no socket path is set.
• add a sshd -G option that parses and prints the effective configuration without attempting to load private keys and perform other checks. (This allows usage of the option before keys have been generated and for configuration evaluation and verification by unprivileged users.)
• add support for configuration tags to ssh(1). This adds a ssh_config(5) Tag directive and corresponding Match tag predicate that may be used to select blocks of configuration.
• add a match localnetwork predicate. This allows matching on the addresses of available network interfaces and may be used to vary the effective client configuration based on network location.
• add a %j token that expands to the configured ProxyJump hostname
• add support for Match sessiontype to ssh_config. Allows matching on the type of session initially requested, either shell for interactive sessions, exec for command execution sessions, subsystem for subsystem requests, such as sftp, or none for transport/forwarding-only sessions.
• allow glob(3) patterns to be used in sshd_config AuthorizedKeysFile and AuthorizedPrincipalsFile directives.

$ wget -qO- http://deb.debian.org/debian/dists/sid/Release   grep -i ^Acquire-By-Hash:
Acquire-By-Hash: yes
$ wget -qO- http://archive.ubuntu.com/ubuntu/dists/devel/Release   grep -i ^Acquire-By-Hash:
Acquire-By-Hash: yes

$ wget -qO- http://http.kali.org/kali/dists/kali-rolling/Release   grep -i ^Acquire-By-Hash:   echo not supported
not supported
$ wget -qO- https://archive.raspberrypi.com/debian/dists/trixie/Release   grep -i ^Acquire-By-Hash:   echo not supported
not supported
$ wget -qO- http://packages.linuxmint.com/dists/faye/Release   grep -i ^Acquire-By-Hash:   echo not supported
not supported
$ wget -qO- https://apt.pop-os.org/release/dists/noble/Release   grep -i ^Acquire-By-Hash:   echo not supported
not supported

• Packages files: list the binary packages that are available in the repository
• Sources files: list the source packages that are available in the repository
• Contents files: list files provided by each package (used by the command apt-file)
• and even more, such as PDiff, Translations, DEP-11 metadata, etc etc...

apt update -> Release -> Packages (main/amd64)
                      -> Packages (contrib/amd64)
                      -> Packages (non-free/amd64)
                      -> Packages (non-free-firmware/amd64)

$ podman run --rm kali-rolling apt -q -o Debug::Acquire::http=true update 2>&1   grep ^GET
GET /kali/dists/kali-rolling/InRelease HTTP/1.1    # <- returns a redirect, that is why the file is requested twice
GET /kali/dists/kali-rolling/InRelease HTTP/1.1
GET /kali/dists/kali-rolling/non-free/binary-amd64/Packages.gz HTTP/1.1
GET /kali/dists/kali-rolling/main/binary-amd64/Packages.gz HTTP/1.1
GET /kali/dists/kali-rolling/non-free-firmware/binary-amd64/Packages.gz HTTP/1.1
GET /kali/dists/kali-rolling/contrib/binary-amd64/Packages.gz HTTP/1.1

$ podman run --rm debian:sid apt -q -o Debug::Acquire::http=true update 2>&1   grep ^GET
GET /debian/dists/sid/InRelease HTTP/1.1
GET /debian/dists/sid/main/binary-amd64/by-hash/SHA256/22709f0ce67e5e0a33a6e6e64d96a83805903a3376e042c83d64886bb555a9c3 HTTP/1.1

$ podman run --rm kali-rolling apt -q -o Debug::Acquire::http=true update 2>&1   grep -e ^Answer -e ^HTTP
Answer for: http://http.kali.org/kali/dists/kali-rolling/InRelease
HTTP/1.1 302 Found
Answer for: http://mirror.freedif.org/kali/dists/kali-rolling/InRelease
HTTP/1.1 200 OK
Answer for: http://mirror.freedif.org/kali/dists/kali-rolling/non-free-firmware/binary-amd64/Packages.gz
HTTP/1.1 200 OK
Answer for: http://mirror.freedif.org/kali/dists/kali-rolling/contrib/binary-amd64/Packages.gz
HTTP/1.1 200 OK
Answer for: http://mirror.freedif.org/kali/dists/kali-rolling/main/binary-amd64/Packages.gz
HTTP/1.1 200 OK
Answer for: http://mirror.freedif.org/kali/dists/kali-rolling/non-free/binary-amd64/Packages.gz
HTTP/1.1 200 OK

$ sudo rm /var/cache/approx/kali/dists/ -fr
$ gbp buildpackage --git-builder=sbuild
+------------------------------------------------------------------------------+
  Update chroot                                Wed, 11 Jun 2025 10:33:32 +0000  
+------------------------------------------------------------------------------+
Get:1 http://http.kali.org/kali kali-dev InRelease [41.4 kB]
Get:2 http://http.kali.org/kali kali-dev/contrib Sources [81.6 kB]
Get:3 http://http.kali.org/kali kali-dev/main Sources [17.3 MB]
Get:4 http://http.kali.org/kali kali-dev/non-free Sources [122 kB]
Get:5 http://http.kali.org/kali kali-dev/non-free-firmware Sources [8297 B]
Get:6 http://http.kali.org/kali kali-dev/non-free amd64 Packages [197 kB]
Get:7 http://http.kali.org/kali kali-dev/non-free-firmware amd64 Packages [10.6 kB]
Get:8 http://http.kali.org/kali kali-dev/contrib amd64 Packages [120 kB]
Get:9 http://http.kali.org/kali kali-dev/main amd64 Packages [21.0 MB]
Err:9 http://http.kali.org/kali kali-dev/main amd64 Packages
  File has unexpected size (20984689 != 20984861). Mirror sync in progress? [IP: ::1 9999]
  Hashes of expected file:
   - Filesize:20984861 [weak]
   - SHA256:6cbbee5838849ffb24a800bdcd1477e2f4adf5838a844f3838b8b66b7493879e
   - SHA1:a5c7e557a506013bd0cf938ab575fc084ed57dba [weak]
   - MD5Sum:1433ce57419414ffb348fca14ca1b00f [weak]
  Release file created at: Wed, 11 Jun 2025 07:15:10 +0000
Fetched 17.9 MB in 9s (1893 kB/s)
Reading package lists...
E: Failed to fetch http://http.kali.org/kali/dists/kali-dev/main/binary-amd64/Packages.gz  File has unexpected size (20984689 != 20984861). Mirror sync in progress? [IP: ::1 9999]
   Hashes of expected file:
    - Filesize:20984861 [weak]
    - SHA256:6cbbee5838849ffb24a800bdcd1477e2f4adf5838a844f3838b8b66b7493879e
    - SHA1:a5c7e557a506013bd0cf938ab575fc084ed57dba [weak]
    - MD5Sum:1433ce57419414ffb348fca14ca1b00f [weak]
   Release file created at: Wed, 11 Jun 2025 07:15:10 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.
E: apt-get update failed

$ grep kali /etc/approx/approx.conf 
#kali http://http.kali.org/kali <- do not use the redirector!
kali  http://kali.download/kali

$ sudo debootstrap kali-dev kali-dev http://http.kali.org/kali
[...]
I: Target architecture can be executed
I: Retrieving InRelease 
I: Checking Release signature
I: Valid Release signature (key id 827C8569F2518CC677FECA1AED65462EC8D5E4C5)
I: Retrieving Packages 
I: Validating Packages 
W: Retrying failed download of http://http.kali.org/kali/dists/kali-dev/main/binary-amd64/Packages.gz
I: Retrieving Packages 
I: Validating Packages 
W: Retrying failed download of http://http.kali.org/kali/dists/kali-dev/main/binary-amd64/Packages.gz
I: Retrieving Packages 
I: Validating Packages 
W: Retrying failed download of http://http.kali.org/kali/dists/kali-dev/main/binary-amd64/Packages.gz
I: Retrieving Packages 
I: Validating Packages 
W: Retrying failed download of http://http.kali.org/kali/dists/kali-dev/main/binary-amd64/Packages.gz
I: Retrieving Packages 
I: Validating Packages 
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
I: Checking component main on http://http.kali.org/kali...
I: Retrieving adduser 3.152
[...]

1. debootstrap requests the Release file, gets redirected to a mirror, and retrieves it from there
2. then it requests the Packages file, gets redirected to another mirror that is not in sync with the first one, and retrieves it from there
3. validation fails, since the checksum is not as expected
4. try again and again

$ sudo debootstrap kali-dev kali-dev http://http.kali.org/kali
[...]
I: Base system installed successfully.
$ cat kali-dev/etc/apt/sources.list
deb http://http.kali.org/kali kali-dev main
$ ls -l kali-dev/var/lib/apt/lists/
total 80468
-rw-r--r-- 1 root root    41445 Jun 19 07:02 http.kali.org_kali_dists_kali-dev_InRelease
-rw-r--r-- 1 root root 82299122 Jun 19 07:01 http.kali.org_kali_dists_kali-dev_main_binary-amd64_Packages
-rw-r--r-- 1 root root    40562 Jun 19 11:54 http.kali.org_kali_dists_kali-dev_Release
-rw-r--r-- 1 root root      833 Jun 19 11:54 http.kali.org_kali_dists_kali-dev_Release.gpg
drwxr-xr-x 2 root root     4096 Jun 19 11:54 partial

$ sudo sed -i 's/main$/main contrib non-free non-free-firmware/' kali-dev/etc/apt/sources.list
$ cat kali-dev/etc/apt/sources.list
deb http://http.kali.org/kali kali-dev main contrib non-free non-free-firmware
$ sudo chroot kali-dev apt update
Hit:1 http://http.kali.org/kali kali-dev InRelease
Get:2 http://kali.download/kali kali-dev/contrib amd64 Packages [121 kB]
Get:4 http://mirror.sg.gs/kali kali-dev/non-free-firmware amd64 Packages [10.6 kB]
Get:3 http://mirror.freedif.org/kali kali-dev/non-free amd64 Packages [198 kB]
Err:3 http://mirror.freedif.org/kali kali-dev/non-free amd64 Packages
  File has unexpected size (10442 != 10584). Mirror sync in progress? [IP: 66.96.199.63 80]
  Hashes of expected file:
   - Filesize:10584 [weak]
   - SHA256:71a83d895f3488d8ebf63ccd3216923a7196f06f088461f8770cee3645376abb
   - SHA1:c4ff126b151f5150d6a8464bc6ed3c768627a197 [weak]
   - MD5Sum:a49f46a85febb275346c51ba0aa8c110 [weak]
  Release file created at: Fri, 23 May 2025 06:48:41 +0000
Fetched 336 kB in 4s (77.5 kB/s)  
Reading package lists... Done
E: Failed to fetch http://mirror.freedif.org/kali/dists/kali-dev/non-free/binary-amd64/Packages.gz  File has unexpected size (10442 != 10584). Mirror sync in progress? [IP: 66.96.199.63 80]
   Hashes of expected file:
    - Filesize:10584 [weak]
    - SHA256:71a83d895f3488d8ebf63ccd3216923a7196f06f088461f8770cee3645376abb
    - SHA1:c4ff126b151f5150d6a8464bc6ed3c768627a197 [weak]
    - MD5Sum:a49f46a85febb275346c51ba0aa8c110 [weak]
   Release file created at: Fri, 23 May 2025 06:48:41 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.

$ sudo chroot kali-dev apt -q -o Debug::Acquire::http=true update 2>&1   grep -e ^Answer -e ^HTTP
Answer for: http://http.kali.org/kali/dists/kali-dev/InRelease
HTTP/1.1 304 Not Modified
Answer for: http://http.kali.org/kali/dists/kali-dev/contrib/binary-amd64/Packages.gz
HTTP/1.1 302 Found
Answer for: http://http.kali.org/kali/dists/kali-dev/non-free/binary-amd64/Packages.gz
HTTP/1.1 302 Found
Answer for: http://http.kali.org/kali/dists/kali-dev/non-free-firmware/binary-amd64/Packages.gz
HTTP/1.1 302 Found
Answer for: http://kali.download/kali/dists/kali-dev/contrib/binary-amd64/Packages.gz
HTTP/1.1 200 OK
Answer for: http://mirror.sg.gs/kali/dists/kali-dev/non-free-firmware/binary-amd64/Packages.gz
HTTP/1.1 200 OK
Answer for: http://mirror.freedif.org/kali/dists/kali-dev/non-free/binary-amd64/Packages.gz
HTTP/1.1 200 OK

• Kali uses a redirector + a network of mirrors
• Kali repo doesn't support Acquire-By-Hash
• The fact that the redirector honors If-Modified-Since makes the matter a bit worse

• Regular users that just use APT to update their system or install packages are not affected by those issues
• Power users (that setup a caching proxy) or developers (that use debootstrap) are the most likely to hit those issues every now and then
• It only happens during those specific windows of time, when mirrors might not be in sync with each others, 4 times a day
• It's rather easy to workaround on the user side, by NOT using the redirector
• However, unless you're deep into these things, you're unlikely to understand what caused the issues, and to magically guess the workarounds

• Our first Platinum sponsor is Infomaniak. Infomaniak is Switzerland s leading developer of Web technologies. With operations all over Europe and based exclusively in Switzerland, the company designs and manages its own data centers powered by 100% renewable energy, and develops all its solutions locally, without outsourcing. With millions of users and the trust of public and private organizations across Europe - such as RTBF, the United Nations, central banks, over 3,000 radio and TV stations, as well as numerous cities and security bodies - Infomaniak stands for sovereign, sustainable and independent digital technology. The company offers a complete suite of collaborative tools, cloud hosting, streaming, marketing and events solutions, while being owned by its employees and self-financed exclusively by its customers.
• Proxmox is the second Platinum sponsor. Proxmox develops powerful, yet easy-to-use Open Source server software. The product portfolio from Proxmox, including server virtualization, backup, and email security, helps companies of any size, sector, or industry to simplify their IT infrastructures. The Proxmox solutions are built on Debian, we are happy that they give back to the community by sponsoring DebConf25.
• Viridien is our third Platinum sponsor. Viridien is an advanced technology, digital and Earth data company that pushes the boundaries of science for a more prosperous and sustainable future. Viridien has been using Debian-based systems to power most of its HPC infrastructure and its cloud platform since 2009 and currently employs two active Debian Project Members.
• EDF is our fourth Platinum sponsor. EDF is a leading global utility company focused on low-carbon power generation. The group uses advanced engineering and scientific computing tools to drive innovation and efficiency in its operations, especially in nuclear power plant design and safety assessment. Since 2003, the EDF Group has been using Debian as its main scientific computing environment. Debian's focus on stability and reproducibility ensures that EDF's calculations and simulations produce consistent and accurate results.
• AMD is our fifth Platinum sponsor. The AMD ROCm platform includes programming models, tools, compilers, libraries, and runtimes for AI and HPC solution development on AMD GPUs. Debian is an officially supported platform for AMD ROCm and a growing number of components are now included directly in the Debian distribution. For more than 55 years AMD has driven innovation in high-performance computing, graphics and visualization technologies. AMD is deeply committed to supporting and contributing to open-source projects, foundations, and open-standards organizations, taking pride in fostering innovation and collaboration within the open-source community.

• Ubuntu, the Operating System delivered by Canonical.
• Freexian, a services company specialized in Free Software and in particular Debian GNU/Linux, covering consulting, custom developments, support and training.
• Lenovo, a global technology leader manufacturing a wide portfolio of connected products including smartphones, tablets, PCs and workstations as well as AR/VR devices, smart home/office and data center solutions.
• Collabora, a global consultancy delivering Open Source software solutions to the commercial world.
• Vyos Networks provides a free routing platform that competes directly with other commercially available solutions and VyOS is an open source network operating system based on Debian.

• Google, one of the largest technology companies in the world, providing a wide range of Internet-related services and products such as online advertising technologies, search, cloud computing, software, and hardware.
• Arm: leading technology provider of processor IP, Arm powered solutions have been supporting innovation for more than 30 years and are deployed in over 280 billion chips to date.
• The Bern University of Applied Sciences with around 7,925 students enrolled, located in the Swiss capital.
• Siemens is a technology company focused on industry, infrastructure and transport.
• Linagora develops ethical Free and Open Source Software, supports Open Source products that helps humans and boosts digital progress. They promote Free Software in France.
• Pexip brings the ease of commercial video platforms to secure and sovereign environments without compromising control or performance.
• Sipgate offers cloud telephony that automates routine tasks, recognizes customer needs, and enables deep CRM integrations.
• evolix is a french company specialized in Hosting and Managed Services Provider (MSP) working only with Open Source softwares.
• Civil Infrastructure Platform, a collaborative project hosted by the Linux Foundation, establishing an open source base layer of industrial grade software.
• credativ: a consulting and services company offering comprehensive services and technical support for the implementation and operation of Open Source Software in business applications.
• Wind River is a leader in delivering the highest levels of secure, safe, and reliable solutions for mission-critical intelligent systems.
• NovaCustom is a company that lets you configure your own laptop with various hardware and software options, focusing on privacy and security.
• Microsoft who enables digital transformation for the era of an intelligent cloud and an intelligent edge. Its mission is to empower every person and every organization on the planet to achieve more.
• McKay Brothers is the acknowledged leader in providing extreme low latency microwave private bandwidth for firms trading in financial markets.
• Matanel Foundation, whose first concern is to preserve the cohesion of a society and a nation plagued by divisions.
• Qualcomm Technologies one of the world's leading companies in field of mobile technology, sponsors and contributes to Open Source developer communities that drive collaboration.

• Loongson,
• Altus Metrum,
• Worteks,
• Savoir-faire Linux,
• OS-Sci,
• Framework.

• ISG,
• Lab STICC,
• Logilab,
• Univention,
• CNRS Sciences Informatiques,
• P le open source et communs num riques,
• Soci t Informatique de France,
• CYUniversit ,
• BEARSTECH.

1. Reproducible Builds at FOSSY 2025
2. Distribution work
3. diffoscope
4. OSS Rebuild updates
5. Website updates
6. Upstream patches
7. Reproducibility testing framework

Reproducible Builds at FOSSY 2025 On Saturday 2nd August, Vagrant Cascadian and Chris Lamb will be presenting at this year s FOSSY 2025. Their talk, titled Never Mind the Checkboxes, Here s Reproducible Builds!, is being introduced as follows:

There are numerous policy compliance and regulatory processes being developed that target software development but do they solve actual problems? Does it improve the quality of software? Do Software Bill of Materials (SBOMs) actually give you the information necessary to verify how a given software artifact was built? What is the goal of all these compliance checklists anyways or more importantly, what should the goals be? If a software object is signed, who should be trusted to sign it, and can they be trusted forever?

The talk will introduce the audience to Reproducible Builds as a set of best practices which allow users and developers to verify that software artifacts were built from the source code, but also allows auditing for license compliance, providing security benefits, and removes the need to trust arbitrary software vendors. Hosted by the Software Freedom Conservancy and taking place in Portland, Oregon, USA, FOSSY aims to be a community-focused event: Whether you are a long time contributing member of a free software project, a recent graduate of a coding bootcamp or university, or just have an interest in the possibilities that free and open source software bring, FOSSY will have something for you . More information on the event is available on the FOSSY 2025 website, including the full programme schedule. Vagrant and Chris will also be staffing a table this year, where they will be available to answer any questions about Reproducible Builds and discuss collaborations with other projects.

---

Distribution work In Debian this month:

• Holger Levsen has discovered that it is now possible to bootstrap a minimal Debian trixie using 100% reproducible packages. This result can itself be reproduced, using the debian-repro-status tool and mmdebstrap s support for hooks:

    $ mmdebstrap --variant=apt --include=debian-repro-status \
         --chrooted-customize-hook=debian-repro-status \
         trixie /dev/null 2>&1   grep "Your system has"
     INFO  debian-repro-status > Your system has 100.00% been reproduced.
  

• On our mailing list this month, Helmut Grohne wrote an extensive message raising an issue related to Uploads with conflicting buildinfo filenames:

  Having several .buildinfo files for the same architecture is something that we plausibly want to have eventually. Imagine running two sets of buildds and assembling a single upload containing buildinfo files from both buildds in the same upload. In a similar vein, as a developer I may want to supply several .buildinfo files with my source upload (e.g. for multiple architectures). Doing any of this is incompatible with current incoming processing and with reprepro.

• 5 reviews of Debian packages were added, 4 were updated and 8 were removed this month adding to our ever-growing knowledge about identified issues.

In GNU Guix, Timothee Mathieu reported that a long-standing issue with reproducibility of shell containers across different host operating systems has been solved. In their message, Timothee mentions:

I discovered that pytorch (and maybe other dependencies) has a reproducibility problem of order 1e-5 when on AVX512 compared to AVX2. I first tried to solve the problem by disabling AVX512 at the level of pytorch, but it did not work. The dev of pytorch said that it may be because some components dispatch computation to MKL-DNN, I tried to disable AVX512 on MKL, and still the results were not reproducible, I also tried to deactivate in openmpi without success. I finally concluded that there was a problem with AVX512 somewhere in the dependencies graph but I gave up identifying where, as this seems very complicated.

The IzzyOnDroid Android APK repository made more progress in June. Not only have they just passed 48% reproducibility coverage, Ben started making their reproducible builds more visible, by offering rbtlog shields, a kind of badge that has been quickly picked up by many developers who are proud to present their applications reproducibility status.
Lastly, in openSUSE news, Bernhard M. Wiedemann posted another monthly update for their work there.

diffoscope diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made the following changes, including preparing and uploading versions 298, 299 and 300 to Debian:

• Add python3-defusedxml to the Build-Depends in order to include it in the Docker image. [ ]
• Handle the RPM format s HEADERSIGNATURES and HEADERIMMUTABLE as a special-case to avoid unnecessarily large diffs. Thanks to Daniel Duan for the report and suggestion. [ ][ ]
• Update copyright years. [ ]

In addition, @puer-robustus fixed a regression introduced in an earlier commit which resulted in some differences being lost. [ ][ ] Lastly, Vagrant Cascadian updated diffoscope in GNU Guix to version 299 [ ][ ] and 300 [ ][ ].

OSS Rebuild updates OSS Rebuild has added a new network analyzer that provides transparent HTTP(S) interception during builds, capturing all network traffic to monitor external dependencies and identify suspicious behavior, even in unmodified maintainer-controlled build processes. The text-based user interface now features automated failure clustering that can group similar rebuild failures and provides natural language failure summaries, making it easier to identify and understand patterns across large numbers of build failures. OSS Rebuild has also improved the local development experience with a unified interface for build execution strategies, allowing for more extensible environment setup for build execution. The team also designed a new website and logo.

Website updates Once again, there were a number of improvements made to our website this month including:

• Arnaud Brousseau added Stage , a new Linux distribution, to our Tools page.
• Chris Lamb improved the docker instructions on the diffoscope website. [ ]

---

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

• Chris Riches:
  • rpm
• Bernhard M. Wiedemann:
  • arandr, curl, dpdk, dpdk, eww, gnucash, gramps, latex2html, llvm20, mp, nvidia-open-driver-G06-signed, obs, ovmf, pcre2, perl-XML-LibXML, perl, python-reportlab, python313, qt6-datavis3d, qt6-declarative, qt6-sensors, qt6-virtualkeyboard, rage-encryption, scummvm, timescaledb & zoxide.
  • Plus several issues with how rpmbuild expands the %jobs variable in the .src.rpm header, including: chromium, cmake, edk2, firefox-esr, gnome-keyring-sharp, gtk2-engine-aurora, gtk2-engine-cleanice, gtk2-engines, libqt5-qtlocation, libreoffice, luabind, lxmenu-data, mozc, MozillaThunderbird, perl-DateTime-Calendar-Mayan, perl-Getopt-ArgvFile, perl-MooseX-Meta-TypeConstraint-ForceCoercion, perl-XML-Entities, python-convertdate, suitesparse & webkit2gtk3
• Robin Candau:
  • gramps (use SOURCE_DATE_EPOCH when compressing man pages)
• Chris Lamb:
  • #1108273 filed against tree-puzzle.
  • #1108281 filed against cctools.
  • #1108532 filed against python-django-import-export.

---

Reproducibility testing framework The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In June, however, a number of changes were made by Holger Levsen, including:

• reproduce.debian.net-related:
  • Installed and deployed rebuilderd version 0.24 from Debian unstable in order to make use of the new compression feature added by Jarl Gullberg for the database. This resulted in massive decrease of the SQLite databases:
    • 79G 2.8G (all)
    • 84G 3.2G (amd64)
    • 75G 2.9G (arm64)
    • 45G 2.1G (armel)
    • 48G 2.2G (armhf)
    • 73G 2.8G (i386)
    • 72G 2.7G (ppc64el)
    • 45G 2.1G (riscv64)
    for a combined saving from 521G 20.8G. This naturally reduces the requirements to run an independent rebuilderd instance and will permit us to add more Debian suites as well.
  • During migration to the latest version of rebuilderd, make sure several services are not started. [ ]
  • Actually run rebuilderd from /usr/bin. [ ]
  • Raise temperatures for NVME devices on some riscv64 nodes that should be ignored. [ ][ ]
  • Use a 64KB kernel page size on the ppc64el architecture (see #1106757). [ ]
  • Improve ordering of some failed to reproduce statistics. [ ]
  • Detect a number of potential causes of build failures within the statistics. [ ][ ]
  • Add support for manually scheduling for the any architecture. [ ]
• Misc:
  • Update the Codethink nodes as there are now many kernels installed. [ ][ ]
  • Install linux-sysctl-defaults on Debian trixie systems as we need ping functionality. [ ]
  • Limit the fs.nr_open kernel turnable. [ ]
  • Stop submitting results to deprecated buildinfo.debian.net service. [ ][ ]

In addition, Jochen Sprickerhof greatly improved the statistics and the logging functionality, including adopting to the new database format of rebuilderd version 0.24.0 [ ] and temporarily increasing maximum log size in order to debug a nettlesome build [ ]. Jochen also dropped the CPUSchedulingPolicy=idle systemd flag on the workers. [ ]

Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Mastodon: @reproducible_builds@fosstodon.org
• Mailing list: rb-general@lists.reproducible-builds.org

Debian LTS contributors In June, 20 contributors have been paid to work on Debian LTS, their reports are available:

• Abhijith PA did 14.0h (out of 14.0h assigned).
• Adrian Bunk did 23.5h (out of 23.5h assigned).
• Andreas Henriksson did 3.0h (out of 3.0h assigned and 17.0h from previous period), thus carrying over 17.0h to the next month.
• Andrej Shadura did 2.0h (out of 3.0h assigned and 7.0h from previous period), thus carrying over 8.0h to the next month.
• Bastien Roucari s did 20.0h (out of 20.0h assigned).
• Ben Hutchings did 8.0h (out of 7.5h assigned and 16.0h from previous period), thus carrying over 15.5h to the next month.
• Carlos Henrique Lima Melara did 12.0h (out of 12.0h assigned).
• Chris Lamb did 18.0h (out of 18.0h assigned).
• Daniel Leidert did 22.0h (out of 22.5h assigned and 1.0h from previous period), thus carrying over 1.5h to the next month.
• Emilio Pozuelo Monfort did 23.5h (out of 16.75h assigned and 6.75h from previous period).
• Guilhem Moulin did 14.0h (out of 11.5h assigned and 3.5h from previous period), thus carrying over 1.0h to the next month.
• Jochen Sprickerhof did 21.0h (out of 0.5h assigned and 22.75h from previous period), thus carrying over 2.25h to the next month.
• Lucas Kanashiro did 20.0h (out of 20.0h assigned).
• Markus Koschany did 23.25h (out of 17.0h assigned and 6.25h from previous period).
• Roberto C. S nchez did 21.25h (out of 20.75h assigned and 3.25h from previous period), thus carrying over 2.75h to the next month.
• Santiago Ruano Rinc n did 12.75h (out of 15.0h assigned), thus carrying over 2.25h to the next month.
• Sean Whitton did 1.0h (out of 4.25h assigned and 1.75h from previous period), thus carrying over 5.0h to the next month.
• Sylvain Beucler did 23.5h (out of 23.5h assigned).
• Thorsten Alteholz did 15.0h (out of 15.0h assigned).
• Tobias Frost did 2.5h (out of 12.0h assigned), thus carrying over 9.5h to the next month.

Evolution of the situation In June, we released 35 DLAs.

• Notable security updates:
  • mariadb-10.5, prepared by Otto Kek l inen, fixes vulnerabilities which could result in denial of service, information disclosure, or unauthorized data modification
  • python-django, prepared by Chris Lamb, fixes vulnerabilities which would result in log injection or denial of service
  • webkit2gtk, prepared by Emilio Pozuelo Monfort, fixes many vulnerabilities which could results in a wide range of issues
  • xorg-server, prepared by Emilio Pozuelo Monfort, fixes multiple vulnerabilities which may result in privilege escalation
  • sudo, prepared by Thorsten Alteholz, fixes a vulnerability which could result in privilege escalation
• Notable non-security updates:
  • debian-security-support, prepared by Santiago Ruano Rinc n, updates status of packages which receive limited security support or which have reached the end of security support
  • dns-root-data, prepared by Sylvain Beucler, updates the DNSSEC trust anchors

This month s contributions from outside the regular team include the mariadb-10.5 update mentioned above, prepared by Otto Kek l inen (the package maintainer); an update to libfile-find-rule-perl, prepared by Salvatore Bonaccorso (a member of the Debian Security Team); an update to activemq, prepared by Emmanuel Arias (a maintainer of the package). Additionally, LTS Team members contributed stable updates of the following packages:

• curl, prepared by Carlos Henrique Lima Melara
• python-tornado, prepared by Daniel Leidert
• python-flask-cors, prepared by Daniel Leidert
• common-vfs, prepared by Daniel Leidert
• cjson, prepared by Adrian Bunk
• icu, prepared by Adrian Bunk
• node-tar-fs, prepared by Adrian Bunk
• rar, prepared by Adrian Bunk

Something of particular noteworthiness is that LTS contributor Carlos Henrique Lima Melara discovered a regression in the upstream fix for CVE-2023-2753 in curl. The corrective action which he took included providing a patch to upstream, uploading a stable update of curl, and further updating the version of curl in LTS. DebConf, the annual Debian Conference, is coming up in July and, as is customary each year, the week preceding the conference will feature an event called DebCamp. The DebCamp week provides an opportunity for teams and other interested groups/individuals to meet together in person in the same venue as the conference itself, with the purpose of doing focused work, often called sprints . LTS coordinator Roberto C. S nchez has announced that the LTS Team is planning to hold a sprint primarily focused on the Debian security tracker and the associated tooling used by the LTS Team and the Debian Security Team.

Thanks to our sponsors Sponsors that joined recently are in bold.

• Platinum sponsors:
  • Toshiba Corporation (for 117 months)
  • Civil Infrastructure Platform (CIP) (for 85 months)
  • VyOS Inc (for 49 months)
• Gold sponsors:
  • Roche Diagnostics International AG (for 127 months)
  • Akamai - Linode (for 121 months)
  • Babiel GmbH (for 111 months)
  • Plat Home (for 110 months)
  • University of Oxford (for 67 months)
  • Deveryware (for 54 months)
  • EDF SA (for 39 months)
  • Dataport A R (for 14 months)
  • CERN (for 12 months)
• Silver sponsors:
  • Domeneshop AS (for 132 months)
  • Nantes M tropole (for 126 months)
  • Univention GmbH (for 118 months)
  • Universit Jean Monnet de St Etienne (for 118 months)
  • Ribbon Communications, Inc. (for 112 months)
  • Exonet B.V. (for 102 months)
  • Leibniz Rechenzentrum (for 96 months)
  • Minist re de l Europe et des Affaires trang res (for 80 months)
  • Cloudways by DigitalOcean (for 69 months)
  • Dinahosting SL (for 67 months)
  • Platform.sh SAS (for 61 months)
  • Moxa Inc. (for 55 months)
  • sipgate GmbH (for 53 months)
  • OVH US LLC (for 51 months)
  • Tilburg University (for 51 months)
  • GSI Helmholtzzentrum f r Schwerionenforschung GmbH (for 42 months)
  • THINline s.r.o. (for 15 months)
  • Copenhagen Airports A/S (for 9 months)
• Bronze sponsors:
  • Evolix (for 132 months)
  • Seznam.cz, a.s. (for 132 months)
  • Intevation GmbH (for 129 months)
  • Linuxhotel GmbH (for 129 months)
  • Daevel SARL (for 128 months)
  • Megaspace Internet Services GmbH (for 127 months)
  • Greenbone AG (for 126 months)
  • NUMLOG (for 126 months)
  • WinGo AG (for 125 months)
  • Entr ouvert (for 116 months)
  • Adfinis AG (for 114 months)
  • Tesorion (for 109 months)
  • Laboratoire LEGI - UMR 5519 / CNRS (for 108 months)
  • Bearstech (for 100 months)
  • LiHAS (for 100 months)
  • Catalyst IT Ltd (for 95 months)
  • Demarcq SAS (for 89 months)
  • Universit Grenoble Alpes (for 75 months)
  • TouchWeb SAS (for 67 months)
  • SPiN AG (for 64 months)
  • CoreFiling (for 60 months)
  • Institut des sciences cognitives Marc Jeannerod (for 55 months)
  • Observatoire des Sciences de l Univers de Grenoble (for 51 months)
  • Tem Innovations GmbH (for 46 months)
  • WordFinder.pro (for 45 months)
  • CNRS DT INSU R sif (for 44 months)
  • Soliton Systems K.K. (for 39 months)
  • Alter Way (for 37 months)
  • Institut Camille Jordan (for 27 months)
  • SOBIS Software GmbH (for 12 months)
  • Tuxera Inc. (for 3 months)

AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

For threaded and hybrid servers (e.g. event or worker), MaxRequestWorkers restricts the total number of threads that will be available to serve clients. For hybrid MPMs, the default value is 16 (ServerLimit) multiplied by the value of 25 (ThreadsPerChild). Therefore, to increase MaxRequestWorkers to a value that requires more than 16 processes, you must also raise ServerLimit.

For the prefork MPM, this directive sets the maximum configured value for MaxRequestWorkers for the lifetime of the Apache httpd process. For the worker and event MPMs, this directive in combination with ThreadLimit sets the maximum configured value for MaxRequestWorkers for the lifetime of the Apache httpd process. For the event MPM, this directive also defines how many old server processes may keep running and finish processing open connections. Any attempts to change this directive during a restart will be ignored, but MaxRequestWorkers can be modified during a restart. Special care must be taken when using this directive. If ServerLimit is set to a value much higher than necessary, extra, unused shared memory will be allocated. If both ServerLimit and MaxRequestWorkers are set to values higher than the system can handle, Apache httpd may not start or the system may become unstable. With the prefork MPM, use this directive only if you need to set MaxRequestWorkers higher than 256 (default). Do not set the value of this directive any higher than what you might want to set MaxRequestWorkers to. With worker, use this directive only if your MaxRequestWorkers and ThreadsPerChild settings require more than 16 server processes (default). Do not set the value of this directive any higher than the number of server processes required by what you may want for MaxRequestWorkers and ThreadsPerChild. With event, increase this directive if the process number defined by your MaxRequestWorkers and ThreadsPerChild settings, plus the number of gracefully shutting down processes, is more than 16 server processes (default).

user = site342999writer
group = site342999writer
listen = /run/php/8.1-site342999.sock
listen.owner = www-data
listen.group = www-data
pm = ondemand
pm.max_children = 12
pm.max_requests = 500
php_admin_value[memory_limit] = 256M

<FilesMatch \.php$>
        SetHandler "proxy:unix:/var/run/php/8.1-site342999.sock fcgi://localhost"
</FilesMatch>

By default, mod_proxy will allow and retain the maximum number of connections that could be used simultaneously by that web server child process. Use the max parameter to reduce the number from the default. The pool of connections is maintained per web server child process, and max and other settings are not coordinated among all child processes, except when only one child process is allowed by configuration or MPM design.

If set, this will be the maximum time to wait for a free connection in the connection pool, in milliseconds. If there are no free connections in the pool, the Apache httpd will return SERVER_BUSY status to the client.

 <Proxy "fcgi://localhost">
    ProxySet acquire=1 max=12
  </proxy>

Context

• See hibernate-pocket, hibernate-pocket-2

Serial console hardware

• Manual is unclear about name of connector (J16 in schematics, J17 in manual).
• Also numbering of pins is not given afaict.
• Clone https://source.mnt.re/reform/pocket-reform.git
• Look at pocket-reform-motherboard.kicad_pcb
• From the PCB I can confirm J16 and pins numbered left (sysctl) to right.
• attach "dtech" prolific PL2303 based serial to usb cable per serial console section of PR manual
• lsusb shows ID 067b:23a3 Prolific Technology, Inc. ATEN Serial Bridge
• install tio
• add my user to group dialout
• newgrp dialout
• tio /dev/ttyUSB0 -b 1500000
• A closer look at the PCB in kicad makes me realize the pin labels in the manual are wrong. 4 = GND, 5 = UART1_RX, 6= UART1_TX. With that change I have U-boot output on boot.

Serial console software With some help from minute on ircs://irc.libera.chat:6697/#mnt-reform, I got the kernel boot arguments right to have not just u-boot output but linux kernel output on the serial console. In consfigurator notation

(on-change
      (file:has-content "/etc/flash-kernel/ubootenv.d/00reform2_serial_console"
        "setenv bootargs \"$ bootargs  console=ttyS2,1500000 keep_bootcon\"")
    (cmd:single "flash-kernel"))

The filename should sort before "00reform2_ubootenv" so that the existing "console=tty1" still ends up at the end. previous episode next episode

Typesetters did not like the laser printer. Wedding photographers still hate the iphone. And some musicians are outraged that AI is now making mediocre pop music.