Debusine aims to be an
integrated solution to build, distribute and maintain a Debian-based
distribution. At Debconf 25, we talked about using it to pre-test uploads
to Debian
unstable,
and also touched on how Freexian is using it to help maintain the Debian
LTS and ELTS projects.
When Debian 10 (buster) moved to ELTS status in 2024, this came with a new
difficulty that hadn t existed for earlier releases. Debian 10 added UEFI
Secure Boot support, meaning that there are now signed variants of the boot
loader and Linux kernel packages. Debian has a
system where certain
packages are configured as needing to be signed, and those packages include
a template for a source package along with the unsigned objects themselves.
The signing service generates detached signatures for all those objects, and
then uses the template to build a source package that it uploads back to the
archive for building in the usual way.
Once buster moved to ELTS, it could no longer rely on Debian s signing
service for all this. Freexian operates parallel infrastructure for the
archive, and now needed to operate a parallel signing service as well. By
early 2024 we were already planning to move ELTS infrastructure towards
Debusine, and so it made sense to build a signing service there as well.
Separately, we were able to obtain a Microsoft signature for Freexian s shim
build, allowing us to chain this into the trust path for most deployed x86
machines.
Freexian can help other organizations running Debian derivatives through the
same process, and can provide secure signing infrastructure to the standards
required for UEFI Secure Boot.
Prior art
We considered both
code-signing (Debian s
current implementation) and
lp-signing (Ubuntu s current
implementation) as prior art. Neither was quite suitable for various
reasons.
code-signing relies on polling a configured URL for each archive to
fetch a GPG-signed list of signing requests, which would have been awkward
for us to set up, and it assumes that unsigned packages are sufficiently
trusted for it to be able to run dpkg -x and dpkg-source -b on them
outside any containment. dpkg -x has had the occasional security
vulnerability, so this seemed unwise for a service that might need to deal
with signing packages for multiple customers.
lp-signing is a microservice accepting authenticated requests, and is
careful to avoid needing to manipulate packages itself. However, this
relies on a different and incompatible mechanism for indicating that
packages should be signed, which wasn t something we wanted to introduce
in ELTS.
Workers
Debusine already had an established system of external workers that run
tasks under various kinds of containment. This seems like a good fit: after
all, what s a request to sign a package but a particular kind of task? But
there are some problems here: workers can run essentially arbitrary code
(such as build scripts in source packages), and even though that s under
containment, we don t want to give such machines access to highly-sensitive
data such as private keys.
Fortunately, we d already introduced the idea of different kinds of workers
a few months beforehand, in order to be able to run privileged server
tasks that have direct access to the Debusine database. We built on that
and added signing workers , which are much like external workers except
that they only run signing tasks, no other types of tasks run on them, and
they have access to a private database with information about the keys
managed by their Debusine instance. (Django s support for multiple
databases
made this quite easy to arrange: we were able to keep everything in the same
codebase.)
Key management
It s obviously bad practice to store private key material in the clear, but
at the same time the signing workers are essentially oracles that will
return signatures on request while ensuring that the rest of Debusine has no
access to private key material, so they need to be able to get hold of it
themselves. Hardware security
modules (HSMs) are
designed for this kind of thing, but they can be inconvenient to manage when
large numbers of keys are involved.
Some keys are more valuable than others. If the signing key used for an
experimental archive leaks, the harm is unlikely to be particularly serious;
but if the ELTS signing key leaks, many customers will be affected. To
match this, we implemented two key protection arrangements for the time
being: one suitable for low-value keys encrypts the key in software with a
configured key and stores the public key and ciphertext in the database,
while one suitable for high-value keys stores keys as PKCS
#11 URIs that can be set up
manually by an instance
administrator.
We packaged some YubiHSM
tools
to make this easier for our sysadmins.
The signing worker calls back to the Debusine server to check whether a
given work request is authorized to use a given signing key. All operations
related to private keys also produce an audit log entry in the private
signing database, so we can track down any misuse.
Tasks
Getting Debusine to do anything new usually requires figuring out how to
model the operation as a
task.
In this case, that was complicated by wanting to run as little code as
possible on the signing workers: in particular, we didn t want to do all the
complicated package manipulations there.
The approach we landed on was a chain of three tasks:
ExtractForSigning
runs on a normal external worker. It takes the result of a package build
and picks out the individual files from it that need to be signed, storing
them as separate artifacts.
Sign
runs on a signing worker, and (of course) makes the actual signatures,
storing them as artifacts.
AssembleSignedSource
runs on a normal external worker. It takes the signed artifacts and
produces a source package containing them, based on the template found in
the unsigned binary package.
Workflows
Of course, we don t want people to have to create all those tasks directly
and figure out how to connect everything together for themselves, and that s
what
workflows
are good at. The
make_signed_source
workflow does all the heavy lifting of creating the right tasks with the
right input data and making them depend on each other in the right ways,
including fanning out multiple copies of all this if there are multiple
architectures or multiple template packages involved. Since you probably
don t want to stop at just having the signed source packages, it also kicks
off builds to produce signed binary packages.
Even this is too low-level for most people to use directly, so we wrapped it
all up in our
debian_pipeline
workflow, which just needs to be given a few options to enable signing
support (and those options can be locked down by workspace owners).
What s next?
In most cases this work has been enough to allow ELTS to carry on issuing
kernel security updates without too much disruption, which was the main
goal; but there are other uses for a signing system. We included OpenPGP
support from early on, which allows Debusine to sign its own builds, and
we ll soon be extending that to sign APT repositories hosted by Debusine.
The current key protection arrangements could use some work. Supporting
automatically-generated software-encrypted keys and manually-generated keys
in an HSM is fine as far as it goes, but it would be good to be able to have
the best of both worlds by being able to automatically generate keys
protected by an HSM. This needs some care, as HSMs often have quite small
limits on the number of objects they can store at any one time, and the
usual workaround is to export keys from the HSM under wrap (encrypted by a
key known only to the HSM) so that they can be imported only when needed.
We have a general idea of how to do this, but doing it efficiently will need
care.
We d be very interested in hearing from organizations that need this sort of
thing, especially for Debian derivatives. Debusine provides lots of other
features that can help you.
Please get in touch with us at
sales@freexian.com if any of this sounds useful
to you.
Debconf 25 photos
Debconf 25 came to the end in Brest, France this year a couple weeks ago.
This has been a very different and unusually interesting Debconf. For me it was for two, related reasons: for one the conference
was close enough in Western Europe that I could simply drive there with a car (which reminds me that I should make a blog post
about the BMW i5, before I am done with it at the end of this year) and for the other - the conference is close enough to Western
Europe that many other Debian developers could join this year who have not been seen at the event for many years. Being able to arrive early, decompress and spend extra time looking around the place made the event itself even more enjoyable than usual.
The French cuisine, especially in its Breton expression, has been a very welcome treat. Even if there were some rough patches
with the food selection, amount, or waiting, it was still a great experience.
I specifically want to say a big thank you to the
organisers for everything, but very explicitly for planning all the talk/BOF rooms in the same building and almost on the same
floor. It saved me a lot of footwork, but also for other participants the short walks between the talks made it possible to always
have a few minutes to talk to people or grab a croissant before running to the next talk.
IMHO we should come back to a tradition of organising Debconf in Europe every 2-3 years. This maximises one of the main goals of
Debconf - bringing as many Debian Developers as possible together in one physical location. This works best when the location is
actually close to large concentrations of existing developers. In other years, the other goal of Debconf can then take priority -
recruiting new developers in new locations. However, these goals could both be achieved at the same time - there are plenty of
locations in Europe and even in Western Europe that still have good potential for attracting new developers. Especially if we focus
on organising the event on the campuses of some larger technically-oriented universities.
This year was also very productive for me a lot of conversations with various people about all kinds of topics, especially technical packaging questions. It has been a long time since the very basic foundations of Debian packaging work have been so fundamentally refactored and modernized as in the past year. Tag2upload has become a catalyst for git-based packaging and for automated workflows via Salsa, and all of that feeds back into focusing on a few best-supported packaging workflows. There is still a bit of a documentation gap
of a new contributor getting to these modern packaging workflows from the point where the New Maintainers Guide stops.
In any case, next year Debconf will be happening in Santa Fe, Argentina. And the year after that it is all still open and in
a close competition
between Japan, Spain, Portugal, Brazil and .. El Salvador? Personally, I would love to travel to Japan (again), but Spain or
Portugal would also be great locations to meet more European developers again.
As for Santa Fe ... it is quite likely that I will not be able to make it there next year, for (planned) health reasons. I guess
I should also write a new blog post about what it means to be a Debconf Photographer, so that someone else could do this as well,
and also reduce the "bus factor" along the way.
But before that - here is the main group photo from this year:
You can also see it on:
Additionally, check out photos from other people on GIT LFS and consider adding your own photos there as well.
Other places I have updated with up-to-date information are these wiki pages:
Handling malicious
requests with fail2ban
I've been receiving a lot of malicious requests for a while now, so I
decided to try out fail2ban as a possible solution.
I see fail2ban as nice to have tool that is useful to keep down the
"noise", but I wouldn't rely on it for security. If you need a tool to
block unauthorized attempts or monitor log files excessively, you are
probably doing something wrong.
I'm currently using fail2ban 1.0.2-2 from Debian Bookworm.
Unfortunatly, I quickly ran into a problem, fail2ban doesn't work out of
the box with this version:
systemd[1]: Started fail2ban.service - Fail2Ban Service.
fail2ban-server[2840]: 2025-07-28 14:40:13,450 fail2ban.configreader [2840]: WARNING 'allowipv6' not defined in 'Definition'. Using default one: 'auto'
fail2ban-server[2840]: 2025-07-28 14:40:13,456 fail2ban [2840]: ERROR Failed during configuration: Have not found an y log file for sshd jail
fail2ban-server[2840]: 2025-07-28 14:40:13,456 fail2ban [2840]: ERROR Async configuration of server failed
systemd[1]: fail2ban.service: Main process exited, code=exited, status=255/EXCEPTION
systemd[1]: fail2ban.service: Failed with result 'exit-code'.
The good news is that this issue has already been addressed for
Debian Trixie.
Since I prefer to manage my own configuration, I removed the default
file at /etc/fail2ban/jail.d/defaults-debian.conf and
replaced it with a custom setup. To fix the earlier issue, I also added
a systemd backend to the sshd jail so it would stop expecting a
logpath.
Here's the configuration I'm using:
I like to make things explicit, so I did repeat some lines from the
default jail.conf file. In the end, I'm quite happy with it so far. Soon
after I set it up, fail2ban was already banning a few hosts.
$ sudo fail2ban-client status nginx-bad-request
Status for the jail: nginx-bad-request
- Filter
- Currently failed: 42
- Total failed: 454
- Actions
- Currently banned: 12
- Total banned: 39
I am always looking for ways of increasing the compute power I have at a reasonable price. I am very happy with my HP z840 dual CPU workstation [1] that I m using as a server and my HP z640 single CPU workstation [2]. Both of them were available second hand at quite reasonable prices and could be cheaply upgraded to faster CPUs. But if I can get something a lot faster for a reasonable price then I ll definitely get it.
Socket LGA2011-v3
The home server and home workstation I currently use have socket LGA2011-v3 [3] which supports the E5-2699A v4 CPU which gives a rating of 26,939 according to Passmark [4]. That Passmark score is quite decent, you can get CPUs using DDR4 RAM that go up to almost double that but it s a reasonable speed and it works in systems that are readily available at low prices. The z640 is regularly on sale for less than $400AU and the z840 is occasionally below $600.
The Dell PowerEdge T430 is an ok dual-CPU tower server using the same socket. One thing that s not well known is that is it limited to something like 135W per CPU when run with two CPUs. So it will work correctly with a single E5-2697A v4 with 145W TDP (I ve tested that) but will refuse to boot with two of them. In my test system I tried replacing the 495W PSUs with 750W PSUs and it made no difference, the motherboard has the limit. With only a single CPU you only get 8/12 DIMM sockets and not all PCIe slots work. There are many second hand T430s on sale with only a single CPU presumably because the T330 sucks. My T430 works fine with a pair of E5-2683 v4 CPUs.
The Dell PowerEdge T630 also takes the same CPUs but supports higher TDP than the T430. They also support 18*3.5 disks or 32*2.5 but they are noisy. I wouldn t buy one for home use.
AMD
There are some nice AMD CPUs manufactured around the same time and AMD has done a better job of making multiple CPUs that fit the same socket. The reason I don t generally use AMD CPUs is that they are used in a minority of the server grade systems so as I want ECC RAM and other server features I generally can t find AMD systems at a reasonable price on ebay etc. There are people who really want second hand server grade systems with AMD CPUs and outbid me. This is probably a region dependent issue, maybe if I was buying in the US I could get some nice workstations with AMD CPUs at low prices.
Socket LGA1151
Socket LGA1151 [5] is used in the Dell PowerEdge T330. It only supports 2 memory channels and 4 DIMMs compared to the 4 channels and 8 DIMMs in LGA2011, and it also has a limit of 64G total RAM for most systems and 128G for some systems. By today s standards even 128G is a real limit for server use, DDR4 RDIMMs are about $1/GB and when spending $600+ on system and CPU upgrade you wouldn t want to spend less than $130 on RAM. The CPUs with decent performance for that socket like the i9-9900K aren t supported by the T330 (possibly they don t support ECC RAM). The CPUs that Dell supports perform very poorly. I suspect that Dell deliberately nerfed the T330 to drive sales of the T430.
The Lenovo P330 uses socket LGA1151-2 but has the same issues of taking slow CPUs in addition to using UDIMMs which are significantly more expensive on the second hand market.
Socket LGA2066
The next Intel socket after LGA2011-v3 is LGA2066 [6]. That is in The Dell Precision 5820 and HP Z4 G4. It takes an i9-10980XE for 32,404 on Passmark or a W-2295 for 30,906. The variant of the Dell 5820 that supports the i9 CPUs doesn t seem to support ECC RAM so it s not a proper workstation. The single thread performance difference between the W-2295 and the E5-2699A v4 is 2640 to 2055, a 28% increase for the W-2295. There are High Frequency Optimized cpus for socket LGA2011-v3 but they all deliver less than 2,300 on the Passmark single-thread tests which is much less than what you can get from socket LGA2066. The W-2295 costs $1000 on ebay and the E5-2699A v4 is readily available for under $400 and a few months ago I got a matched pair for a bit over $400. Note that getting a matched pair of Intel CPUs is a major pain [7].
Comparing sockets LGA2011-v3 and LGA2066 for a single-CPU system is a $300 system (HP x640) + $400 CPU (E5-2699A v4) vs $500 system (Dell Precision 5820) + $1000 CPU (W-2295), so more than twice the price for a 30% performance benefit on some tasks. The LGA2011-v3 and USB-C both launched in 2014 so LGA2011-v3 systems don t have USB-C sockets, a $20 USB-C PCIe card doesn t change the economics.
Socket LGA3647
Socket LGA3647 [8] is used in the Dell PowerEdge T440. It supports 6 channels of DDR4 RAM which is a very nice feature for bigger systems. According to one Dell web page the best CPU Dell officially supports for this is the Xeon Gold 5120 which gives performance only slightly better than the E5-2683 v4 which has a low enough TDP that a T430 can run two of them. But according to another Dell web page they support 16 core CPUs which means performance better than a T430 but less than a HP z840. The T440 doesn t seem like a great system, if I got one cheap I could find a use for it but I wouldn t pay the prices that they go for on ebay. The Dell PowerEdge T640 has the same socket and is described as supporting up to 28 core CPUs. But I anticipate that it would be as loud as the T630 and it s also expensive.
This socket is also used in the HP Z6 G4 which takes a W-3265 or Xeon Gold 6258R CPU for the high end options. The HP Z6 G4 systems on ebay are all above $1500 and the Xeon Gold 6258R is also over $1000 so while the Xeon Gold 6258R in a Z6 G4 will give 50% better performance on multithreaded operations than the systems I currently have it s costing almost 3* as much. It has 6 DIMM sockets which is a nice improvement over the 4 in the z640. The Z6 G4 takes a maximum of 768G of RAM with the optional extra CPU board (which is very expensive both new and on ebay) compared to my z840 which has 512G and half it s DIMM slots empty. The HP Z8 G4 has the same socket and takes up to 3TB of RAM if used with CPUs that support it (most CPUs only support 768G and you need a M variant to support more). The higher performance CPUs supported in the Z6 G4 and Z8 G4 don t have enough entries in the Passmark database to be accurate, but going from 22 cores in the E5-2699A v4 to 28 in the Xeon Platinum 8180 when using the same RAM technology doesn t seem like a huge benefit. The Z6 and Z8 G4 systems run DDR4 RAM at up to 2666 speed while the z640 and z840 only to 2400, a 10% increase in RAM speed is nice but not a huge difference.
I don t think that any socket LGA3647 systems will ever be ones I want to buy. They don t offer much over LGA2011-v3 but are in newer and fancier systems that will go for significantly higher prices.
DDR5
I think that DDR5 systems will be my next step up in tower server and workstation performance after the socket LGA2011-v3 systems. I don t think anything less will offer me enough of a benefit to justify a change. I also don t think that they will be in the price range I am willing to pay until well after DDR6 is released, some people are hoping for DDR6 to be released late this year but next year seems more likely. So maybe in 2027 there will be some nice DDR5 systems going cheap.
CPU Benchmark Results
Here are the benchmark results of CPUs I mentioned in this post according to passmark.com [9]. I didn t reference results of CPUs that only had 1 or 2 results posted as they aren t likely to be accurate.
I ve participated in this year s Google Summer of Code (GSoC)
program and have been working on the small (90h) autopkgtests for the rsync
package project at Debian.
Writing my proposal
Before you can start writing a proposal, you need to select an organization
you want to work with. Since many organizations participate in GSoC, I ve
used the following criteria to narrow things down for me:
Programming language familiarity: For me only Python (preferably) as well
as shell and Go projects would have made sense. While learning another
programming language is cool, I wouldn t be as effective and helpful to
the project as someone who is proficient in the language already.
Standing of the organization: Some of the organizations participating in
GSoC are well-known for the outstanding quality of the software they
produce. Debian is one of them, but so is e.g. the Django Foundation or
PostgreSQL. And my thinking was that the higher the quality of the
organization, the more there is to learn for me as a GSoC student.
Mentor interactions: Apart from the advantage you get from mentor
feedback when writing your proposal (more on that further below), it is
also helpful to gauge how responsive/helpful your potential mentor is
during the application phase. This is important since you will be working
together for a period of at least 2 months; if the mentor-student
communication doesn t work, the GSoC project is going to be difficult.
Free and Open-Source Software (FOSS) communication platforms: I
generally believe that FOSS projects should be built on FOSS
infrastructure. I personally won t run proprietary software
when I want to contribute to FOSS in my spare time.
Be a user of the project: As Eric S. Raymond has pointed out
in his seminal The Cathedral and the Bazaar 25 years ago
Every good work of software starts by scratching a developer s personal
itch.
Once I had some organizations in mind whose projects I d be interested in
working on, I started writing proposals for them. Turns out, I started
writing my proposals way too late: In the end I only managed to hand in a
single one which is risky. Competition for the GSoC projects is fierce
and the more quality (!) proposals you send out, the better your chances are
at getting one. However, don t write proposals for the sake of it: Reviewers
get way too many AI slop proposals already and you will not do yourself a
favor with a low-quality proposal. Take the time to read the
instructions/ideas/problem descriptions the project mentors have provided
and follow their guidelines. Don t hesitate to reach out to project mentors:
In my case, I ve asked Samuel Henrique a few clarification questions whereby
the following (email) discussion has helped me greatly in improving my
proposal. Once I ve finalized my proposal draft, I ve sent it to Samuel for
a review, which again led to some improvements to the final proposal
which I ve uploaded to the GSoC program webpage.
Community bonding period
Once you get the information that you ve been accepted into the GSoC program
(don t take it personally if you don t make it; this was my second attempt
after not making the cut in 2024), get in touch with your prospective mentor
ASAP. Agree upon a communication channel and some response times. Put
yourself in the loop for project news and discussions whatever that means in
the context of your organization: In Debian s case this boiled down to
subscribing to a bunch of mailing lists and IRC channels. Also make sure to
setup a functioning development environment if you haven t done so for
writing the proposal already.
Payoneer setup
The by far most annoying part of GSoC for me. But since you don t have a
choice if you want to get the stipend, you will need to signup for an
account at Payoneer.
In this iteration of GSoC all participants got a personalized link to open a
Payoneer account. When I tried to open an account by following this link, I
got an email after the registration and email verification that my
account is being blocked because Payoneer deems the email adress I gave a
temporary one. Well, the email in question is most certainly anything but
temporary, so I tried to get in touch with the Payoneer support - and ended
up in an LLM-infused kafkaesque support hell. Emails are answered by an LLM
which for me meant utterly off-topic replies and no help whatsoever. The
Payoneer website offers a real-time chat, but it is yet another instance of
a bullshit-spewing LLM bot. When I at last tried to call them (the
support lines are not listed on the Payoneer website but were provided by
the GSoC program), I kid you not, I was being told that their platform is
currently suffering from technical problems and was hung up on. Only thanks
to the swift and helpful support of the GSoC administrators (who get
priority support from Payoneer) I was able to setup a Payoneer account in
the end.
Apart from showing no respect to customers, Payoneer is also ripping them
off big time with fees (unless you get paid in USD). They charge you 2% for
currency conversions to EUR on top of the FX spread they take. What
worked for me to avoid all of those fees, was to open a USD account at Wise
and have Payoneer transfer my GSoC stipend in USD to that account. Then I
exchanged the USD to my local currency at Wise for significantly less than
Payoneer would have charged me. Also make sure to close your Payoneer
account after the end of GSoC to avoid their annual fee.
Project work
With all this prelude out of the way, I can finally get to the actual work
I ve been doing over the course of my GSoC project.
Background
The upstream rsync project generally sees little development. Nonetheless,
they released version 3.4.0 including some CVE fixes earlier
this year. Unfortunately, their changes broke the -H
flag. Now, Debian package maintainers need to apply those security fixes to
the package versions in the Debian repositories; and those are typically a
bit older. Which usually means that the patches cannot be applied as is but
will need some amendments by the Debian maintainers. For these cases it is
helpful to have autopkgtests defined, which check the package s
functionality in an automated way upon every build.
The question then is, why should the tests not be written upstream such that
regressions are caught in the development rather than the distribution
process? There s a lot to say on this question and it probably depends a lot
on the package at hand, but for rsync the main benefits are twofold:
The upstream project mocks the ssh connection over which rsync is most
typically used. Mocking is better than nothing but not the real thing. In
addition to being a more realisitic test scenario for the typical rsync
use case, involving an ssh server in the test would automatically extend
the overall resilience of Debian packages as now new versions of the
openssh-server package in Debian benefit from the test cases in the
rsync reverse dependency.
The upstream rsync test framework is somewhat idiosyncratic and
difficult to port to reimplementations of rsync. Given that the
original rsync upstream sees little development, an extensive test suit
further downstream can serve as a threshold for drop-in replacements for
rsync.
Goal(s)
At the start of the project, the Debian rsync package was just running (a
part of) the upstream tests as autopkgtests. The relevant snippet from the
build log for the rsync_3.4.1+ds1-3 package reads:
Samuel and I agreed that it would be a good first milestone to make the
skipped tests run. Afterwards, I should write some rsync test cases for
local calls, i.e. without an ssh connection, effectively using rsync as
a more powerful cp. And once that was done, I should extend the tests such
that they run over an active ssh connection.
With these milestones, I went to work.
Upstream tests
Running the seven skipped upstream tests turned out to be fairly
straightforward:
Two upstream tests concern access control lists and extended
filesystem attributes. For these tests to run they rely on
functionality provided by the acl and xattr Debian packages. Adding
those to the Build-Depends list in the debian/control file of the
rsync Debian package repo made them run.
Four upstream tests required root privileges to run. The autopkgtest
tool knows the needs-root restriction for that reason. However, Samuel
and I agreed that the tests should not exclusively run with root
privileges. So, instead of just adding the restiction to the existing
autopkgtest test, we created a new one which has the needs-root
restriction and runs the upstream-tests-as-root script - which is
nothing else than a symlink to the existing upstream-tests script.
The commits to implement these changes can be found in this merge
request.
The careful reader will have noticed that I only made 2 + 4 = 6 upstream
test cases run out of 7: The leftover upstream test is checking the
functionality of the --ctimes rsync option. In the context of Debian, the
problem is that the Linux kernel doesn t have a syscall to set the creation
time of a file. As long as that is the case, this test will always be
skipped for the Debian package.
Local tests
When it came to writing Debian specific test cases I started of a completely
clean slate. Which is a blessing and a curse at the same time: You have full
flexibility but also full responsibility.
There were a few things to consider at this point in time:
Which language to write the tests in?
The programming language I am most proficient in is Python. But testing a
CLI tool in Python would have been weird: it would have meant that I d
have to make repeated subprocess calls to run rsync and then read from
the filesystem to get the file statistics I want to check.
Samuel suggested I stick with shell scripts and make use of diffoscope -
one of the main tools used and maintained by the Reproducible Builds
project - to check whether the file contents and
file metadata are as expected after rsync calls. Since I did not have
good reasons to use bash, I ve decided to write the scripts to be POSIX
compliant.
How to avoid boilerplate? If one makes use of a testing framework, which
one?
Writing the tests would involve quite a bit of boilerplate, mostly related
to giving informative output on and during the test run, preparing the
file structure we want to run rsync on, and cleaning the files up after
the test has run. It would be very repetitive and in violation of DRY
to have the code for this appear in every test. Good testing frameworks
should provide convenience functions for these tasks. shunit2 comes with
those functions, is packaged for Debian, and given that it is already
being used in the curl project, I decided to go with it.
Do we use the same directory structure and files for every test or should
every test have an individual setup?
The tradeoff in this question being test isolation vs. idiosyncratic
code. If every test has its own setup, it takes a) more work to write the
test and b) more work to understand the differences between
tests. However, one can be sure that changes to the setup in one test will
have no side effects on other tests. In my opinion, this guarantee was
worth the additional effort in writing/reading the tests.
Having made these decisions, I simply started writing
tests and ran into issues very quickly.
rsync and subsecond mtime diffs
When testing the rsync --times option, I observed a weird phenomenon: If
the source and destination file have modification times which differ only in
the nanoseconds, an rsync --times call will not synchronize the
modification times. More details about this behavior and examples can be
found in the upstream issue I raised. In the Debian tests we
had to occasionally work around this by setting the timestamps explicitly
with touch -d.
diffoscope regression
In one test case, I was expecting a difference in the modification times but
diffoscope would not report a diff. After a good amount of time spent
on debugging the problem (my default, and usually correct, assumption is
that something about my code is seriously broken if I run into issues like
that), I was able to show that diffoscope only displayed this behavior in
the version in the unstable suite, not on Debian stable (which I am running
on my development machine).
Since everything pointed to a regression in the diffoscope project and
with diffoscope being written in Python, a language I am familiar with, I
wanted to spend some time investigating (and hopefully fixing) the problem.
Running git bisect on the diffoscope repo helped me in identifying the
commit which introduced the regression: The commit contained an optimization
via an early return for bit-by-bit identical files. Unfortunately, the early
return also caused an explicitly requested metadata comparison (which could
be different between the files) to be skipped.
With a nicely diagnosed issue like that, I was able to
go to a local hackerspace event, where people work on FOSS together for an
evening every month. In a group, we were able to first, write a test which
showcases the broken behavior in the latest diffoscope version, and
second, make a fix to the code such that the same test passes going
forward. All details can be found in this merge request.
shunit2 failures
At some point I had a few autopkgtests setup and passing, but adding a new
one would throw me totally inexplicable errors. After trying to isolate the
problem as much as possible, it turns out that shunit2 doesn t play well
together we the -e shell option. The project mentions this in the release
notes for the 2.1.8 version1, but in my opinion a
constraint this severe should be featured much more prominently, e.g. in the
README.
Tests over an ssh connection
The centrepiece of this project; everything else has in a way only been
preparation for this.
Obviously, the goal was to reuse the previously written local tests in some
way. Not only because lazy me would have less work to do this way, but also
because of a reduced long-term maintenance burden of one rather than two
test sets.
As it turns out, it is actually possible to accomplish that: The
remote-tests script doesn t do much apart from starting an ssh server on
localhost and running the local-tests script with the REMOTE environment
variable set.
The REMOTE environment variable changes the behavior of the local-tests
script in such a way that it prepends "$REMOTE": to the destination of the
rsync invocations. And given that we set REMOTE=rsync@localhost in the
remote-tests script, local-tests copies the files to the exact same
locations as before, just over ssh.
The implementational details for this can be found in this merge
request.
proposed-updates
Most of my development work on the Debian rsync package took place during
the Debian freeze as the release of Debian Trixie is just
around the corner. This means that uploading by Debian Developers (DD) and
Debian Maintainers (DM) to the unstable suite is discouraged as it makes
migrating the packages to testing more difficult for the Debian release
team. If DDs/DMs want to have the package version in unstable migrated to
testing during the freeze they have to file an unblock request.
Samuel has done this twice (1, 2) for my work for Trixie but has
asked me to file the proposed-updates request for current
stable (i.e. Debian Bookworm) myself after I ve backported my
tests to bookworm.
Unfinished business
To run the upstream tests which check access control list and extended file
system attributes functionality, I ve added the acl and xattr packages
to Build-Depends in debian/control. This, however, will only make the
packages available at build time: If Debian users install the rsync
package, the acl and xattr packages will not be installed alongside
it. For that, the dependencies would have to be added to Depends or
Suggests in debian/control. Depends is probably to strong of a relation
since rsync clearly works well in practice without, but adding them to
Suggests might be worthwhile. A decision on this would involve checking,
what happens if rsync is called with the relevant options on a host
machine which has those packages installed, but where the destination
machine lacks them.
Apart from the issue described above, the 15 tests I managed to write are
are a drop in the water in light of the infinitude of rsync
optionsand their combinations. Most glaringly, not all
options of the --archive option are covered separately (which would help
indicating what code path of rsync broke in a regression). To increase the
likelihood of catching regressions with the autopkgtests, the test
coverage should be extended in the future.
Conclusion
Generally, I am happy with my contributions to Debian over the course of my
small GSoC project: I ve created an extensible, easy to understand, and
working autopkgtest setup for the Debian rsync package. There are two
things which bother me, however:
In hindsight, I probably shouldn t have gone with shunit2 as a testing
framework. The fact that it behaves erratically with the -e flag is a
serious drawback for a shell testing framework: You really don t want a
shell command to fail silently and the test to continue running.
As alluded to in the previous section, I m not particularly proud of the
number of tests I managed to write.
On the other hand, finding and fixing the regression in diffoscope - while
derailing me from the GSoC project itself - might have a redeeming quality.
DebConf25
By sheer luck I happened to work on a GSoC project at Debian over a time
period during which the annual Debian conference would take place
close enough to my place of residence. Samuel pointed the
opportunity to attend DebConf out to me during the community bonding period
and since I could make time for the event in my schedule, I signed up.
DebConf was a great experience which - aside from gaining more knowledge
about Debian development - allowed me to meet the actual people usually
hidden behind email adresses and IRC nicks. I can wholeheartedly recommend
attending a DebConf to every interested Debian user!
For those who have missed this year s iteration of the conference, I can
recommend the following recorded talks:
While not featuring as a keynote speaker (understandably so as the newcomer
to Debian community that I am), I could still contribute a bit to the
conference program.
Debian install workshop
Additionally, with so many Debian experts gathering in one place while KDE s
End of 10 campaign is ongoing, I felt it natural to
organize a Debian install workhop. In hindsight I can say
that I underestimated how much work it would be, especially for me who does
not speak a word of French. But although the turnout of people who wanted us
to install Linux on their machines was disappointingly low, it was still
worth it: Not only because the material in the repo can be
helpful to others planning install workshops but also because it was nice to
meet a) the person behind the Debian installer images and b) the
local Brest/Finist re Linux user group as well as the motivated and helpful
people at Infini.
Credits
I want to thank the Open Source team at Google for organizing GSoC: The
highly structured program with a one-to-one mentorship is a great avenue to
start contributing to well established and at times intimidating FOSS
projects. And as much as I disagree with Google s surveillance
capitalist business model, I have to give it to them that the company at
least takes its responsibility for FOSS (somewhat) seriously - unlike many
other businesses which rely on FOSS and choose to freeride of it.
Big thanks to the Debian community! I ve experienced nothing but
friendliness in my interactions with the community.
And lastly, the biggest thanks to my GSoC mentor Samuel Henrique. He has
dealt patiently and competently with all my stupid newbie questions. His
support enabled me to make - albeit small - contributions to Debian. It has
been a pleasure to work with him during GSoC and I m looking forward to
working together with him in the future.
Obviously, I ve only read them after experiencing the problem.
It's Sunday and I'm now sitting in the train from Brest to Paris where I will be changing to Germany, on the way back from the annual Debian conference. A full week of presentations, discussions, talks and socializing is laying behind me and my head is still spinning from the intensity.
Pollito and the gang of DebConf mascots wearing their conference badges (photo: Christoph Berg)
Table of Contents
Sunday, July 13th
It started last Sunday with traveling to the conference. I got on the Eurostar in Duisburg and we left on time, but even before reaching Cologne, the train was already one hour delayed for external reasons, collecting yet another hour between Aachen and Liege for its own technical problems. "The train driver is working on trying to fix the problem." My original schedule had well over two hours for changing train stations in Paris, but being that late, I missed the connection to Brest in Montparnasse. At least in the end, the total delay was only one hour when finally arriving at the destination. Due to the French julliet quatorze fireworks approaching, buses in Brest were rerouted, but I managed to catch the right bus to the conference venue, already meeting a few Debian people on the way.
The conference was hosted at the IMT Atlantique Brest campus, giving the event a nice university touch. I arrived shortly after 10 in the evening and after settling down a bit, got on one of the "magic" buses for transportation to the camping site where half of the attendees where stationed. I shared a mobile home with three other Debianites, where I got a small room for myself.
Monday, July 14th
Next morning, we took the bus back to the venue with a small breakfast and the opening session where Enrico Zini invited me to come to his and Nicolas Dandrimont's session about Debian community governance and curation, which I gladly did. Many ideas about conflict moderation and community steering were floated around. I hope some of that can be put into effect to make flamewars on the mailing lists less heated and more directed. After that, I attended Olly Betts' "Stemming with Snowball" session, which is the stemmer used also in PostgreSQL. Text search is one of the areas in PostgreSQL that I never really looked closely at, including the integration into the postgresql-common package, so it was nice to get more information about that.
In preparation for the conference, a few of us Ham radio operators in Debian had decided to bring some radio gear to DebConf this year in order to perhaps spark more interest for our hobby among the fellow geeks. In the afternoon after the talks, I found a quieter spot just outside of the main hall and set up a shortwave antenna by attaching a 10m mast to one of the park benches there. The 40m band was still pretty much closed, but I could work a few stations from England, just across the channel from Bretagne, answering questions from interested passing-by Debian people between the contacts. Over time, the band opened and more European stations got into the log.
F/DF7CB in Brest (photo: Evangelos Ribeiro Tzaras)
Tuesday, July 15th
Tuesday started with Helmut Grohne's session about "Reviving (un)schroot". The schroot program has been Debian's standard way of managing build chroots for a long time, but it is more and more being regarded as obsolete with all kinds of newer containerization and virtualization technologies taking over. Since many bits of Debian infrastructure depend on schroot, and its user interface is still very useful, Helmut reimplemented it using Linux namespaces and the "unshare" systemcall. I had already worked with him at the Hamburg Minidebconf to replace the apt.postgresql.org buildd machinery with the new system, but we were not quite there yet (network isolation is nice, but we still sometimes need proper networking), so it was nice to see the effort is still progressing and I will give his new scripts a try when I'm back home.
Next, Stefano Rivera and Colin Watson presented Debusine, a new package repository and workflow management system. It looks very promising for anyone running their own repository, so perhaps yet another bit of apt.postgresql.org infrastructure to replace in the future. After that, I went to the Debian LTS BoF session by Santiago Ruano Rinc n and Bastien Roucari s - Debian releases plus LTS is what we are covering with apt.postgresql.org. Then there were bits from the DPL (Debian Project Leader), and a session moderated by Stefano Rivera interesting to me as a member of the Debian Technical Committee on the future structure of the packages required for cross-building in Debian, a topic which had been brought to TC a while ago. I am happy that we could resolve the issue without having to issue a formal TC ruling as the involved parties (kernel, glibc, gcc and the cross-build people) found a promising way forward themselves. DebConf is really a good way to get such issues unstuck.
Ten years ago at the 2015 Heidelberg DebConf, Enrico had given a seminal "Semi-serious stand-up comedy" talk, drawing parallels between the Debian Open Source community and the BDSM community - "People doing things consensually together". (Back then, the talk was announced as "probably unsuitable for people of all ages".) With his unique presentation style and witty insights, the session made a lasting impression on everyone attending. Now, ten years later (and he and many in the audience being ten years older), he gave an updated version of it. We are now looking forward to the sequel in 2035. The evening closed with the famous DebConf tradition of the Cheese & Wine party in a old fort next to the coast, just below the conference venue. Even when he's a fellow Debian Developer, Ham and also TC member, I had never met Paul Tagliamonte in person before, but we spent most of the evening together geeking out on all things Debian and Ham radio.
The northern coast of Ushant (photo: Christoph Berg)
Wednesday, July 16th
Wednesday already marked the end of the first half of the week, the day of the day trips. I had chosen to go to Ouessant island (Ushant in English) which marks the Western end of French mainland and hosts one of the lighthouses yielding the way into the English channel. The ferry trip included surprisingly big waves which left some participants seasick, but everyone recovered fast. After around one and a half hours we arrived, picked up the bicycles, and spent the rest of the day roaming the island. The weather forecast was originally very cloudy and 18 C, but over noon this turned into sunny and warm, so many got an unplanned sunburn. I enjoyed the trip very much - it made up for not having time visiting the city during the week. After returning, we spent the rest of the evening playing DebConf's standard game, Mao (spoiler alert: don't follow the link if you ever intend to play).
Having a nice day (photo: Christoph Berg)
Thursday, July 17th
The next day started with the traditional "Meet the Technical Committee" session. This year, we trimmed the usual slide deck down to remove the boring boilerplate parts, so after a very short introduction to the work of the committee by our chairman Matthew Vernon, we opened up the discussion with the audience, with seven (out of 8) TC members on stage. I think the format worked very well, with good input from attendees. Next up was "Don't fear the TPM" by Jonathan McDowell. A common misconception in the Free Software community is that the TPM is evil DRM hardware working against the user, but while it could be used in theory that way, the necessary TPM attestations seem to impossible to attain in practice, so that wouldn't happen anyway. Instead, it is a crypto coprocessor present in almost all modern computers that can be used to hold keys, for example to be used for SSH. It will also be interesting to research if we can make use of it for holding the Transparent Data Encryption keys for CYBERTEC's PostgreSQL Enterprise Edition.
Aigars Mahinovs then directed everyone in place for the DebConf group picture, and Lucas Nussbaum started a discussion about archive-wide QA tasks in Debian, an area where I did a lot of work in the past and that still interests me. Antonio Terceiro and Paul Gevers followed up with techniques to track archive-wide rebuilding and testing of packages and in turn filing a lot of bugs to track the problems. The evening ended with the conference dinner, again in the fort close by the coast. DebConf is good for meeting new people, and I incidentally ran into another Chris, who happened to be one of the original maintainers of pgaccess, the pre-predecessor of today's pgadmin. I admit still missing this PostgreSQL frontend for its simplicity and ability to easily edit table data, but it disappeared around 2004.
Friday, July 18th
On Friday, I participated in discussion sessions around contributors.debian.org (PostgreSQL is planning to set up something similar) and the New Member process which I had helped to run and reform a decade or two ago. Agathe Porte (also a Ham radio operator, like so many others at the conference I had no idea of) then shared her work on rust-rewriting the slower parts of Lintian, the Debian package linter. Craig Small talked about "Free as in Bytes", the evolution of the Linux procps free command. Over the time and many kernel versions, the summary numbers printed became better and better, but there will probably never be a version that suits all use cases alike. Later over dinner, Craig (who is also a TC member) and I shared our experiences with these numbers and customers (not) understanding them. He pointed out that for PostgreSQL and looking at used memory in the presence of large shared memory buffers, USS (unique set size) and PSS (proportional set size) should be more realistic numbers than the standard RSS (resident set size) that the top utility is showing by default.
Antonio Terceiro and Paul Gevers again joined to lead a session, now on ci.debian.net and autopkgtest, the test driver used for running tests on packages after then have been installed on a system. The PostgreSQL packages are heavily using this to make sure no regressions creep in even after builds have successfully completed and test re-runs are rescheduled periodically. The day ended with Bdale Garbee's electronics team BoF and Paul Tagliamonte and me setting up the radio station in the courtyard, again answering countless questions about ionospheric conditions and operating practice.
Saturday, July 19th
Saturday was the last conference day. In the first session, Nikos Tsipinakis and Federico Vaga from CERN announced that the LHC will be moving to Debian for the accelerator's frontend computers in their next "long shutdown" maintenance period in the next year. CentOS broke compatibility too often, and Debian trixie together with the extended LTS support will cover the time until the next long shutdown window in 2035, until when the computers should have all been replaced with newer processors covering higher x86_64 baseline versions. The audience was very delighted to hear that Debian is now also being used in this prestige project.
Ben Hutchings then presented new Linux kernel features. Particularly interesting for me was the support for atomic writes spanning more than one filesystem block. When configured correctly, this would mean PostgreSQL didn't have to record full-page images in the WAL anymore, increasing throughput and performance. After that, the Debian ftp team discussed ways to improve review of new packages in the archive, and which of their processes could be relaxed with new US laws around Open Source and cryptography algorithms export. Emmanuel Arias led a session on Salsa CI, Debian's Gitlab instance and standard CI pipeline. (I think it's too slow, but the runners are not under their control.) Julian Klode then presented new features in APT, Debian's package manager. I like the new display format (and a tiny bit of that is also from me sending in wishlist bugs).
In the last round of sessions this week, I then led the Ham radio BoF with an introduction into the hobby and how Debian can be used. Bdale mentioned that the sBitx family of SDR radios is natively running Debian, so stock packages can be used from the radio's touch display. We also briefly discussed his involvement in ARDC and the possibility to get grants from them for Ham radio projects. Finally, DebConf wrapped up with everyone gathering in the main auditorium and cheering the organizers for making the conference possible and passing Pollito, the DebConf mascot, to the next organizer team.
Pollito on stage (photo: Christoph Berg)
Sunday, July 20th
Zoom back to the train: I made it through the Paris metro and I'm now on the Eurostar back to Germany. It has been an intense week with all the conference sessions and meeting all the people I had not seen so long. There are a lot of new ideas to follow up on both for my Debian and PostgreSQL work. Next year's DebConf will take place in Santa Fe, Argentina. I haven't yet decided if I will be going, but I can recommend the experience to everyone!
The post The Debian Conference 2025 in Brest appeared first on CYBERTEC PostgreSQL Services & Support.
On Saturday 19 July 2025, the annual Debian Developers and Contributors
Conference came to a close.
Over 443 attendees representing 50 countries from around the world came
together for a combined 169 events (including some which took place during
the DebCamp) including more than 50 Talks, 39 Short Talks, 5 Discussions,
59 Birds of a Feather sessions ("BoF" informal meeting between developers
and users), 10 workshops, and activities in support of furthering our
distribution and free software,
learning from our mentors and peers, building our community, and having a bit
of fun.
The conference was preceded by the annual
DebCamp hacking session held 7
through 13 July where Debian Developers and Contributors convened to
focus on their individual Debian-related projects or work in team sprints
geared toward in-person collaboration in developing Debian.
This year, a session was dedicated to prepare the BoF "Dealing with
Dormant Packages: Ensuring Debian's High Standards"; another, at the
initiative of our DPL, to prepare suggestions for the BoF Package
Acceptance in Debian: Challenges and Opportunities"; and an afternoon around
Salsa-CI.
As has been the case for several years, a special effort has been made to
welcome newcomers and help them become familiar with Debian and DebConf
by organizing a sprint "New Contributors Onboarding" every day of Debcamp,
followed more informally by mentorship during DebConf.
The actual Debian Developers Conference started on Monday 14 July 2025.
In addition to the traditional "Bits from the DPL" talk, the continuous
key-signing party, lightning talks, and the announcement of next year's
DebConf26, there were several update sessions shared by internal projects
and teams.
Many of the hosted discussion sessions were presented by our technical
core teams with the usual and useful "Meet the Technical Committee", the
"What's New in the Linux Kernel" session, and a set of BoFs about Debian
packaging policy and Debian infrastructure. Thus, more than a quarter of the
discussions dealt with this theme, including talks about our tools and
Debian's archive processes. Internationalization and Localization have been
the subject of several talks. The Python, Perl, Ruby, Go, and Rust programming
language teams also shared updates on their work and efforts. Several talks
have covered Debian Blends and Debian-derived distributions and other talks
addressed the issue of Debian and AI.
More than 17 BoFs and talks about community, diversity, and local outreach
highlighted the work of various teams involved in not just the technical but
also the social aspect of our community; four women who have made
contributions to Debian through their artwork in recent years presented
their work.
The one-day session "DebConf 2025 Academic Track!", organized in collaboration
with the IRISA laboratory was the first session welcoming fellow academics
at DebConf, bringing together around ten presentations.
The schedule
was updated each day with planned and ad hoc activities introduced by
attendees over the course of the conference. Several traditional activities
took place: a job fair, a poetry performance, the traditional
Cheese and Wine party (this year with cider as well), the Group Photos, and the
Day Trips.
For those who were not able to attend, most of the talks and sessions were
broadcasted live and recorded; currently the videos are made available
through this
link.
Almost all of the sessions facilitated remote participation via IRC and Matrix
messaging apps or online collaborative text documents which allowed remote
attendees to "be in the room" to ask questions or share comments with the
speaker or assembled audience.
DebConf25 saw over 441 T-shirts, 3 day trips, and up to 315 meals planned per
day.
All of these events, activities, conversations, and streams coupled with our
love, interest, and participation in Debian and F/OSS certainly made this
conference an overall success both here in Brest, France and online
around the world.
The DebConf25 website
will remain active for archival purposes and will continue to offer
links to the presentations and videos of talks and events.
Next year, DebConf26 will be held
in Santa Fe, Argentina, likely in July. As tradition follows before
the next DebConf the local organizers in Argentina will start the
conference activities with DebCamp with a particular focus on
individual and team work towards improving the distribution.
DebConf is committed to a safe and welcome environment for all
participants. See the
web page about the Code of Conduct on the DebConf25 website
for more details on this.
Debian thanks the commitment of numerous
sponsors
to support DebConf25, particularly our Platinum Sponsors:
AMD,
EDF,
Infomaniak,
Proxmox, and
Viridien.
We also wish to thank our Video and Infrastructure teams, the DebConf25
and DebConf committees, our host nation of France, and each and every
person who helped contribute to this event and to Debian overall.
Thank you all for your work in helping Debian continue to be "The Universal
Operating System".
See you next year!
About Debian
The Debian Project was founded in 1993 by Ian Murdock to be a truly free
community project. Since then the project has grown to be one of the
largest and most influential Open Source projects. Thousands of
volunteers from all over the world work together to create and maintain
Debian software. Available in 70 languages, and supporting a huge range
of computer types, Debian calls itself the universal operating system.
About DebConf
DebConf is the Debian Project's developer conference. In addition to a
full schedule of technical, social and policy talks, DebConf provides an
opportunity for developers, contributors and other interested people to
meet in person and work together more closely. It has taken place
annually since 2000 in locations as varied as Scotland, Bosnia and Herzegovina,
India, Korea. More information about DebConf is available from
https://debconf.org/.
About AMD
The AMD ROCm platform includes programming models,
tools, compilers, libraries, and runtimes for AI and HPC solution development
on AMD GPUs. Debian is an officially supported platform for AMD ROCm and a
growing number of components are now included directly in the Debian
distribution. For more than 55 years AMD has driven innovation in
high-performance computing, graphics and visualization technologies. AMD is
deeply committed to supporting and contributing to open-source projects,
foundations, and open-standards organizations, taking pride in fostering
innovation and collaboration within the open-source community.
About EDF
EDF is a leading global utility company focused on
low-carbon power generation. The group uses advanced engineering and scientific
computing tools to drive innovation and efficiency in its operations,
especially in nuclear power plant design and safety assessment. Since 2003,
the EDF Group has been using Debian as its main scientific computing
environment. Debian's focus on stability and reproducibility ensures that
EDF's calculations and simulations produce consistent and accurate results.
About Infomaniak
Infomaniak is Switzerland's leading developer
of Web technologies. With operations all over Europe and based exclusively in
Switzerland, the company designs and manages its own data centers powered by
100% renewable energy, and develops all its solutions locally, without
outsourcing. With millions of users and the trust of public and private
organizations across Europe - such as RTBF, the United Nations, central banks,
over 3,000 radio and TV stations, as well as numerous cities and security
bodies - Infomaniak stands for sovereign, sustainable and independent digital
technology. The company offers a complete suite of collaborative tools, cloud
hosting, streaming, marketing and events solutions, while being owned by its
employees and self-financed exclusively by its customers.
About Proxmox
Proxmox develops powerful, yet easy-to-use Open
Source server software. The product portfolio from Proxmox, including server
virtualization, backup, and email security, helps companies of any size,
sector, or industry to simplify their IT infrastructures. The Proxmox solutions
are built on Debian, we are happy that they give back to the community by
sponsoring DebConf25.
About Viridien
Viridien is an advanced technology,
digital and Earth data company that pushes the boundaries of science for a more
prosperous and sustainable future. Viridien has been using Debian-based systems
to power most of its HPC infrastructure and its cloud platform since 2009 and
currently employs two active Debian Project Members.
Contact Information
For further information, please visit the DebConf25 web page at
https://debconf25.debconf.org/ or send
mail to press@debian.org.
DebConf 25 happened between 14th July and
19th July and I was there. It was my first DebConf (the big one, I was at a
Mini DebConf in Hamburg a
couple of years ago) and it was interesting. DebConf 25 happened at a Campus
University at the outskirts of Brest and I was rather reluctant to go at first
(EuroPython 25 was happening at the same time
in Prague), but I decided to use the chance of DebConf happening in Europe,
reachable by train from Vienna. We took the nighttrain to Paris, then found our
way through the maze that is the Paris underground system and then got to Brest
with the TGV. On our way to the Conference site we made a detour to a
supermarket, which wasn t that easy because is was a national holiday in France
and most of the shops were closed. But we weren t sure about the food situation
at DebConf and we also wanted to get some beer.
At the conference we were greeted by very friendly people at the badge station
and the front desk and got our badges, swag and most important the keys to
pretty nice rooms on the campus. Our rooms had a small private bathroom with a
toilet and a shower and between the two rooms was a shared kitchen with a
refrigerator and a microwave. All in all, the accommodation was simple but
provided everything we needed and especially a space to have some privacy.
During the next days I watched a lot of talks, met new people, caught up with
old friends and also had a nice time with my travel buddies. There was a beach
near the campus which I used nearly every day. It was mostly sunny except for
the last day of the conference, which apparently was not common for the Brest
area, so we got lucky regarding the weather.
Given that we only arrived in the evening of the first day of DebConf, I missed
the talk When Free Software Communities Unite: Tails, Tor, and the Fight for
Privacy
(recording),
but I watched it on the way home and it was also covered by
LWN.
On Tuesday I started the day by visiting a talk about
tag2upload
(recording).
The same day there was also an academic track and I watched the talk titled
Integrating Knowledge Graphs into the Debian
Ecosystem
(recording) which presented
a property graph showing relationships between various entities like packages,
maintainers or bugs (there is a
repository with parts of a paper,
but not much other information). The speaker also mentioned the graphcast
framework and the ontocast
framework which sound interesting - we
might have use for something liked this at $dayjob.
In the afternoon there was a talk about the
ArchWiki
(recording)
which gave a comprehensive insight in how the
ArchWiki and the community behind it works. Right
after that was a Debian Wiki BoF. There are various technical limitations with
the current wiki software and there are not enough helping hands to maintain
the service and do content curation. But the BoF had some nice results: there
is now a new debian-wiki mailinglist,
an IRC channel, a MediaWiki installation has been set up during DebConf, there
are efforts to migrate the data and most importantly: and handful of people who
want to maintain the service and organize the content of the wiki. I think the
input from the ArchWiki folks gave some ideas how that team could operate.
Wednesday was the day of the daytrip. I did not sign up for any of the trips
and used the time to try out
tag2upload, uploaded
the latest labwc release to
experimental
and spent the rest of the day at the beach.
Other noteworthy session I ve attended were the Don t fear the
TPM talk
(recording),
which showed me a lot of stuff to try out, the session about
lintian-ng (no
recording), which is an experimental approach to make lintian faster, the
review of the first year of wcurls
existence (no
recording yet) and the summary of Rust packaging in
Debian (no
recording yet). In between the sessions I started working on packaging
wlr-sunclock
(#1109230).
What did not work
Vegan food.
I might be spoiled by other conferences. Both at EuroPycon last year (definitely
bigger, a lot more commercial) and at PyCon CZ
23 (similar
in size, a lot more DIY) there was catering with explicitly vegan options.
As I ve mentioned in the beginning, we went to a supermarket before we went to
the conference and we had to go there one more time during the conference. I
think there was a mixture between a total lack of awareness and a LOT of
miscommunication. The breakfasts at the conference consisted of pastries and
baguettes - I asked at the first day what the vegan options were and the answer
was I don t know, maybe the baguette? and we were asked to only take as
much baguette as the people who also got pastries.
The lunch was prepared by the Restaurant associatif de Kern vent which is
a canteen at the university campus. When we asked if there is vegan food, the
people there said that there was only a vegetarian option so we only ate salad.
Only later we heard via word of mouth that one has to explicitly ask for a
vegan meal which was apparently prepared separatly and you had to find the
right person that knows about it (I think thats very Debian-like ). But even
then a person once got a vegetarian option offered as vegan food.
One problem was also the missing / confusing labeling of the food. At the
conference dinner there was apparently vegan food, but it was mixed with all
the other food. There were some labels but with hundreds of hungry people around
and caterers removing empty plates and dropping off plates with other stuff,
everything gets mixed up. In the end we ate bread soaked in olive oil, until the
olive oil got taken away by the catering people literally while we were dipping
the bread in it.
And when these issues were raised, some of the reactions can be summarized as
You re holding it wrong which was really frustrating.
The dinners at the conference hall were similar. At some point I had the
impression that vegan and vegetarian was simply seen as the same thing.
If the menus would be written like a debian/copyright file it would probably
have looked like this:
Food: *
Diet: Vegan or Vegetarian
But the thing is that Vegan and Vegetarian cannot be mixed. Its similar to non
compatible licenses. Once you mix vegan food with vegan food with vegetarian
food it s not vegan anymore.
Don t get me wrong, I know its hard to organize food for hundreds of people.
But if you don t know what it means to provide a vegan option, just communicate
the fact so people can look alternatives in advance. During the week some of
the vegan people shared food, which was really nice and there were also a lot
of non-vegan people who tried to help, organized extra food or simply listened
to the hangry rants. Thanks for that!
Paris
Saturday was the last day of DebConf and it was a rainy day. On Sunday morning
we took the TGV back to Paris and then stayed there for one night because the
next night train back to Vienna was on Monday. Luckily the weather was
better in Paris. The first thing we did was to look up a vegan burger place. In
the evening we strolled along the Seine and had a couple of beers at the
Jardins du Trocad ro. Monday the rain also arrived in Paris and we mostly went
from one cafe to the next, but also managed to visit Notre Dame.
Conclusio
The next DebConf will be in Argentina and I think its likely that DebConf 27
will also not happen anywhere in trainvelling distance. But even if, I think
the Mini DebConfs are more my style of happening (there is one planned in
Hamburg next spring, and a couple of days ago I learned that there will be a
Back to the Future musical show in Hamburg during that time). Nonetheless I had
a nice time and I stumbled over some projects I might get more involved in.
Thanks also to my travel buddies who put up with me
tl;dr: there is an attack in the wild which is triggering dangerous-but-seemingly-intended behaviour in the Oj JSON parser when used in the default and recommended manner, which can lead to everyone s favourite kind of security problem: object deserialization bugs!
If you have the oj gem anywhere in your Gemfile.lock, the quickest mitigation is to make sure you have Oj.default_options = mode: :strict somewhere, and that no library is overwriting that setting to something else.
Prologue
As a sensible sysadmin, all the sites I run send me a notification if any unhandled exception gets raised.
Mostly, what I get sent is error-handling corner cases I missed, but now and then things get more interesting.
In this case, it was a PG::UndefinedColumn exception, which looked something like this:
PG::UndefinedColumn: ERROR: column "xyzzydeadbeef" does not exist
This is weird on two fronts: firstly, this application has been running for a while, and if there was a schema problem, I d expect it to have made itself apparent long before now.
And secondly, while I don t profess to perfection in my programming, I m usually better at naming my database columns than that.
Something is definitely hinky here, so let s jump into the mystery mobile!
The column name is coming from outside the building!
The exception notifications I get sent include a whole lot of information about the request that caused the exception, including the request body.
In this case, the request body was JSON, and looked like this:
"name":":xyzzydeadbeef", ...
The leading colon looks an awful lot like the syntax for a Ruby symbol, but it s in a JSON string.
Surely there s no way a JSON parser would be turning that into a symbol, right?
Right?!?
Immediately, I thought that that possibly was what was happening, because I use Sequel for my SQL database access needs, and Sequel treats symbols as database column names.
It seemed like too much of a coincidence that a vaguely symbol-shaped string was being sent in, and the exact same name was showing up as a column name.
But how the flying fudgepickles was a JSON string being turned into a Ruby symbol, anyway?
Enter Oj.
Oj? I barely know aj
A long, long time ago, the standard Ruby JSON library had a reputation for being slow.
Thus did many competitors flourish, claiming more features and better performance.
Strong amongst the contenders was oj (for Optimized JSON ), touted as The fastest JSON parser and object serializer .
Given the history, it s not surprising that people who wanted the best possible performance turned to Oj, leading to it being found in a great many projects, often as a sub-dependency of a dependency of a dependency (which is how it ended up in my project).
You might have noticed in Oj s description that, in addition to claiming fastest , it also describes itself as an object serializer .
Anyone who has kept an eye on the security bug landscape will recall that object deserialization is a rich vein of vulnerabilities to mine.
Libraries that do object deserialization, especially ones with a history that goes back to before the vulnerability class was well-understood, are likely to be trouble magnets.
And thus, it turns out to be with Oj.
By default, Oj will happily turn any string that starts with a colon into a symbol:
How that gets exploited is only limited by the creativity of an attacker.
Which I ll talk about more shortly but first, a word from my rant cortex.
Insecure By Default is a Cancer
While the object of my ire today is Oj and its fast-and-loose approach to deserialization, it is just one example of a pervasive problem in software: insecurity by default.
Whether it s a database listening on 0.0.0.0 with no password as soon as its installed, or a library whose default behaviour is to permit arbitrary code execution, it all contributes to a software ecosystem that is an appalling security nightmare.
When a user (in this case, a developer who wants to parse JSON) comes across a new piece of software, they have by definition no idea what they re doing with that software.
They re going to use the defaults, and follow the most easily-available documentation, to achieve their goal.
It is unrealistic to assume that a new user of a piece of software is going to do things the right way , unless that right way is the only way, or at least the by-far-the-easiest way.
Conversely, the developer(s) of the software is/are the domain experts.
They have knowledge of the problem domain, through their exploration while building the software, and unrivalled expertise in the codebase.
Given this disparity in knowledge, it is tantamount to malpractice for the experts the developer(s) to off-load the responsibility for the safe and secure use of the software to the party that has the least knowledge of how to do that (the new user).
To apply this general principle to the specific case, take the Using section of the Oj README.
The example code there calls Oj.load, with no indication that this code will, in fact, parse specially-crafted JSON documents into Ruby objects.
The brand-user user of the library, no doubt being under pressure to Get Things Done, is almost certainly going to look at this Using example, get the apparent result they were after (a parsed JSON document), and call it a day.
It is unlikely that a brand-new user will, for instance, scroll down to the Further Reading section, find the second last (of ten) listed documents, Security.md , and carefully peruse it.
If they do, they ll find an oblique suggestion that parsing untrusted input is never a good idea .
While that s true, it s also rather unhelpful, because I d wager that by far the majority of JSON parsed in the world is untrusted , in one way or another, given the predominance of JSON as a format for serializing data passing over the Internet.
This guidance is roughly akin to putting a label on a car s airbags that driving at speed can be hazardous to your health : true, but unhelpful under the circumstances.
The solution is for default behaviours to be secure, and any deviation from that default that has the potential to degrade security must, at the very least, be clearly labelled as such.
For example, the Oj.load function should be named Oj.unsafe_load, and the Oj.load function should behave as the Oj.safe_load function does presently.
By naming the unsafe function as explicitly unsafe, developers (and reviewers) have at least a fighting chance of recognising they re doing something risky.
We put warning labels on just about everything in the real world; the same should be true of dangerous function calls.
OK, rant over.
Back to the story.
But how is this exploitable?
So far, I ve hopefully made it clear that Oj does some Weird Stuff with parsing certain JSON strings.
It caused an unhandled exception in a web application I run, which isn t cool, but apart from bombing me with exception notifications, what s the harm?
For starters, let s look at our original example: when presented with a symbol, Sequel will interpret that as a column name, rather than a string value.
Thus, if our save an update to the user code looked like this:
# request_body has the JSON representation of the form being submitted
body = Oj.load(request_body)
DB[:users].where(id: user_id).update(name: body["name"])
In normal operation, this will issue an SQL query along the lines of UPDATE users SET name='Jaime' WHERE id=42.
If the name given is Jaime O Dowd , all is still good, because Sequel quotes string values, etc etc.
All s well so far.
But, imagine there is a column in the users table that normally users cannot read, perhaps admin_notes.
Or perhaps an attacker has gotten temporary access to an account, and wants to dump the user s password hash for offline cracking.
So, they send an update claiming that their name is :admin_notes (or :password_hash).
In JSON, that ll look like "name":":admin_notes" , and Oj.load will happily turn that into a Ruby object of "name"=>:admin_notes .
When run through the above update the user code fragment, it ll produce the SQL UPDATE users SET name=admin_notes WHERE id=42.
In other words, it ll copy the contents of the admin_notes column into the name column which the attacker can then read out just by refreshing their profile page.
But Wait, There s More!
That an attacker can read other fields in the same table isn t great, but that s barely scratching the surface.
Remember before I said that Oj does object serialization ?
That means that, in general, you can create arbitrary Ruby objects from JSON.
Since objects contain code, it s entirely possible to trigger arbitrary code execution by instantiating an appropriate Ruby object.
I m not going to go into details about how to do this, because it s not really my area of expertise, and many others have covered it in detail.
But rest assured, if an attacker can feed input of their choosing into a default call to Oj.load, they ve been handed remote code execution on a platter.
Mitigations
As Oj s object deserialization is intended and documented behaviour, don t expect a future release to make any of this any safer.
Instead, we need to mitigate the risks.
Here are my recommended steps:
Look in your Gemfile.lock (or SBOM, if that s your thing) to see if the oj gem is anywhere in your codebase.
Remember that even if you don t use it directly, it s popular enough that it is used in a lot of places.
If you find it in your transitive dependency tree anywhere, there s a chance you re vulnerable, limited only by the ingenuity of attackers to feed crafted JSON into a deeply-hidden Oj.load call.
If you depend on oj directly and use it in your project, consider not doing that.
The json gem is acceptably fast, and JSON.parse won t create arbitrary Ruby objects.
If you really, really need to squeeze the last erg of performance out of your JSON parsing, and decide to use oj to do so, find all calls to Oj.load in your code and switch them to call Oj.safe_load.
It is a really, really bad idea to ever use Oj to deserialize JSON into objects, as it lacks the safety features needed to mitigate the worst of the risks of doing so (for example, restricting which classes can be instantiated, as is provided by the permitted_classes argument to Psych.load).
I d make it a priority to move away from using Oj for that, and switch to something somewhat safer (such as the aforementioned Psych).
At the very least, audit and comment heavily to minimise the risk of user-provided input sneaking into those calls somehow, and pass mode: :object as the second argument to Oj.load, to make it explicit that you are opting-in to this far more dangerous behaviour only when it s absolutely necessary.
To secure any unsafe uses of Oj.load in your dependencies, consider setting the default Oj parsing mode to :strict, by putting Oj.default_options = mode: :strict somewhere in your initialization code (and make sure no dependencies are setting it to something else later!).
There is a small chance that this change of default might break something, if a dependency is using Oj to deliberately create Ruby objects from JSON, but the overwhelming likelihood is that Oj s just being used to parse ordinary JSON, and these calls are just RCE vulnerabilities waiting to give you a bad time.
Is Your Bacon Saved?
If I ve helped you identify and fix potential RCE vulnerabilities in your software, or even just opened your eyes to the risks of object deserialization, please help me out by buying me a refreshing beverage.
I would really appreciate any support you can give.
Alternately, if you d like my help in fixing these (and many other) sorts of problems, I m looking for work, so email me.
The sixteenth release of the qlcal package
arrivied at CRAN today, once
again following the QuantLib 1.39
release this morning.
qlcal
delivers the calendaring parts of QuantLib. It is provided (for the R
package) as a set of included files, so the package is self-contained
and does not depend on an external QuantLib library (which can be
demanding to build). qlcal covers
over sixty country / market calendars and can compute holiday lists, its
complement (i.e. business day lists) and much more. Examples
are in the README at the repository, the package page,
and course at the CRAN package
page.
This releases mainly synchronizes qlcal with
the QuantLib release 1.39.
Changes in version 0.0.16
(2025-07-23)
Synchronized with QuantLib 1.39 released today
Calendar updates for Israel, minor utility functions
update
Minor package maintenance updates
Courtesy of my CRANberries, there
is a diffstat report for this
release. See the project page
and package documentation for more details, and more examples.
Sidequest: Install u-boot-menu
Don't be like me and reboot without configuring u-boot-menu. Although the defaults make sense for most people, in my case I lost access to the serial console (because that custom config needed to be re-done), and the default delay was not enough to choose a backup kernel. In consfigurator notation:
Thanks to a hint from joschc (and a bit of luck) I realized the issue
I filed was nonsense. Yay?
The panel driver is not added by the rk3588 patches (since the build
process applies all the patches, this is not a problem for building
from reform-debian-packages).
After applying the two patches in
reform-debian-packages/linux/patches6.15/imx8mp-mnt-pocket-reform/pocket-panel,
the patched 6.16 kernel boots, and seems to work, including the
panel.
The updated source is on branch reform-patches at
https://salsa.debian.org/bremner/collabora-rockchip-3588
Unsurprisingly hibernate is not working out of the box with 6.16. My
next mission is to apply the recommended
pci-reset patches on top of 6.16.
Update on 2025-07-28: added note about Debian 13/trixie support for OpenVox (thanks, Ben Ford!)
Debian v13 with codename trixie is scheduled to be published as new stable release on 9th of August 2025.
I was the driving force at several of my customers to be well prepared for the upcoming stable release (my efforts for trixie started in August 2024). On the one hand, to make sure packages we care about are available and actually make it into the release. On the other hand, to ensure there are no severe issues that make it into the release and to get proper and working upgrades. So far everything is looking pretty well and working fine, the efforts seemed to have payed off. :)
As usual with major upgrades, there are some things to be aware of, and hereby I m starting my public notes on trixie that might be worth for other folks. My focus is primarily on server systems and looking at things from a sysadmin perspective.
Further readings
As usual start at the official Debian release notes, make sure to especially go through What s new in Debian 13 + issues to be aware of for trixie (strongly recommended read!).
Package versions
As a starting point, let s look at some selected packages and their versions in bookworm vs. trixie as of 2025-07-20 (mainly having amd64 in mind):
Package
bookworm/v12
trixie/v13
ansible
2.14.3
2.19.0
apache
2.4.62
2.4.64
apt
2.6.1
3.0.3
bash
5.2.15
5.2.37
ceph
16.2.11
18.2.7
docker
20.10.24
26.1.5
dovecot
2.3.19
2.4.1
dpkg
1.21.22
1.22.21
emacs
28.2
30.1
gcc
12.2.0
14.2.0
git
2.39.5
2.47.2
golang
1.19
1.24
libc
2.36
2.41
linux kernel
6.1
6.12
llvm
14.0
19.0
lxc
5.0.2
6.0.4
mariadb
10.11
11.8
nginx
1.22.1
1.26.3
nodejs
18.13
20.19
openjdk
17.0
21.0
openssh
9.2p1
10.0p1
openssl
3.0
3.5
perl
5.36.0
5.40.1
php
8.2+93
8.4+96
podman
4.3.1
5.4.2
postfix
3.7.11
3.10.3
postgres
15
17
puppet
7.23.0
8.10.0
python3
3.11.2
3.13.5
qemu/kvm
7.2
10.0
rsync
3.2.7
3.4.1
ruby
3.1
3.3
rust
1.63.0
1.85.0
samba
4.17.12
4.22.3
systemd
252.36
257.7-1
unattended-upgrades
2.9.1
2.12
util-linux
2.38.1
2.41
vagrant
2.3.4
2.3.7
vim
9.0.1378
9.1.1230
zsh
5.9
5.9
Misc unsorted
The asterisk package once again didn t make it into trixie (see #1031046)
The new debian-repro-status package provides the identically named command-line tool debian-repro-status to query the reproducibility status of your installed Debian packages
The Grml live system project provided further of their packages into Debian. Available as of trixie are now also grml-keyring (OpenPGP certificates used for signing the Grml repositories), grml-hwinfo (a tool which collects information of the hardware ) + grml-paste (command line interface for paste.debian.net)
If you use pacemaker, be aware that its fence-agents package is now a transitional package. All the fence-agents got split into separate packages (fence-agents-$whatever). If you want to have all the fence-agents available, make sure to install the fence-agents-all package. If you have Recommends disabled, you definitely should be aware of this.
support for colors (f.e. green for installs/upgrades, yellow for downgrades, red for removals, can be disabled via &dash&dashno-color, APT_NO_COLOR=1 or NO_COLOR=1 and customized via e.g. APT::Color::Action::Install cyan )
organizes output in more readable sections and shows removals more prominently
the new apt modernize-sources command converts /etc/apt/sources.list.d/*.list files into the new .sources format (AKA DEB822)
the new apt distclean command removes all files under $statedir/lists except Release, Release.gpg, and InRelease (it can be used for example, when finalizing images distributed to users)
new configuration option APT::NeverAutoRemove::KernelCount for keeping a configurable amount of kernels, f.e. setting APT::NeverAutoRemove::KernelCount 3 will keep 3 kernels (including the running, and most recent)
new command line option &dash&dashsnapshot, and configuration option APT::Snapshot, controlling the snapshot chosen for archives with Snapshot: enable
new command line option &dash&dashupdate to run the update command before the specified command, like apt &dash&dashupdate install zsh, apt &dash&dashupdate remove foobar or apt &dash&dashupdate safe-upgrade
apt-key is gone, and there s no replacement for it available (if you need an interface for listing present keys)
systemd
systemd got upgraded from v252.36-1~deb12u1 to 257.7-1 and there are lots of changes.
Be aware that systemd v257 has a new net.naming_scheme, v257 being PCI slot number is now read from firmware_node/sun sysfs file. The naming scheme based on devicetree aliases was extended to support aliases for individual interfaces of controllers with multiple ports. This might affect you, see e.g. #1092176 and #1107187, the Debian Wiki provides further useful information.
There are new systemd tools available:
run0: temporarily and interactively acquire elevated or different privileges (serves a similar purpose as sudo)
systemd-ac-power: Report whether we are connected to an external power source
systemd-vpick: Resolve paths to .v/ versioned directories
varlinkctl: Introspect with and invoke Varlink services
The tools provided by systemd gained several new options:
busctl: new option &dash&dashlimit&dashmessages=NUMBER (Stop monitoring after receiving the specified number of message)
hostnamectl: new option &dashj (same as &dash&dashjson=pretty on tty, &dash&dashjson=short otherwise)
journalctl: new options &dash&dashimage&dashpolicy=POLICY (Specify disk image dissection policy), &dash&dashinvocation=ID (Show logs from the matching invocation ID), &dashI (Show logs from the latest invocation of unit), &dash&dashexclude-identifier=STRING (Hide entries with the specified syslog identifier),&dash&dashtruncate-newline (Truncate entries by first newline character), &dash&dashlist-invocations (Show invocation IDs of specified unit), &dash&dashlist-namespaces (Show list of journal namespaces)
localectl: new option &dash&dashfull (Do not ellipsize output)
loginctl: new options &dash&dashjson=MODE (Generate JSON output for list-sessions/users/seats) + &dashj (Same as &dash&dashjson=pretty on tty, &dash&dashjson=short otherwise)
networkctl: new commands edit FILES DEVICES (Edit network configuration files), cat [FILES DEVICES ] (Show network configuration files), mask FILES (Mask network configuration files) + unmask FILES (Unmask network configuration files) + persistent-storage BOOL (Notify systemd-networkd if persistent storage is ready), and new options &dash&dashno-ask-password (Do not prompt for password), &dash&dashno-reload (Do not reload systemd-networkd or systemd-udevd after editing network config), &dash&dashdrop-in=NAME (Edit specified drop-in instead of main config file), &dash&dashruntime (Edit runtime config files) + &dash&dashstdin (Read new contents of edited file from stdin)
systemctl new commands list-paths [PATTERN] (List path units currently in memory, ordered by path), whoami [PID ] (Return unit caller or specified PIDs are part of), soft-reboot (Shut down and reboot userspace) + sleep (Put the system to sleep), and new options &dash&dashcapsule=NAME (Connect to service manager of specified capsule), &dash&dashbefore (Show units ordered before with list-dependencies ), &dash&dashafter (Show units ordered after with list-dependencies ), &dash&dashkill-value=INT (Signal value to enqueue), &dash&dashno-warn (Suppress several warnings shown by default), &dash&dashmessage=MESSAGE (Specify human readable reason for system shutdown), &dash&dashimage&dashpolicy=POLICY (Specify disk image dissection policy), &dash&dashreboot&dashargument=ARG (Specify argument string to pass to reboot()), &dash&dashdrop-in=NAME (Edit unit files using the specified drop-in file name), &dash&dashwhen=TIME (Schedule halt/power-off/reboot/kexec action after a certain timestamp) + &dash&dashstdin (Read
new contents of edited file from stdin)
systemd-analyze new commands architectures [NAME ] (List known architectures), smbios11 (List strings passed via SMBIOS Type #11), image-policy POLICY (Analyze image policy string), fdstore SERVICE (Show file descriptor store contents of service), malloc [D-BUS SERVICE ] (Dump malloc stats of a D-Bus service), has-tpm2 (Report whether TPM2 support is available), pcrs [PCR ] (Show TPM2 PCRs and their names) + srk [>FILE] (Write TPM2 SRK (to FILE)) and new options &dash&dashno-legend (Disable column headers and hints in plot with either &dash&dashtable or &dash&dashjson=), &dash&dashinstance=NAME (Specify fallback instance name for template units), &dash&dashunit=UNIT (Evaluate conditions and asserts of unit), &dash&dashtable (Output plot s raw time data as a table), &dash&dashscale-svg=FACTOR (Stretch x-axis of plot by FACTOR (default: 1.0)), &dash&dashdetailed (Add more details to SVG plot), &dash&dashtldr (Skip comments and empty lines), &dash&dashimage
-policy=POLICY (Specify disk image dissection policy) + &dash&dashmask (Parse parameter as numeric capability mask)
systemd-ask-password: new options &dash&dashuser (Ask only our own user s agents) + &dash&dashsystem (Ask agents of the system and of all users)
systemd-cat: new option &dash&dashnamespace=NAMESPACE (Connect to specified journal namespace)
systemd-creds: new options &dash&dashuser (Select user-scoped credential encryption), &dash&dashuid=UID (Select user for scoped credentials) + &dash&dashallow-null (Allow decrypting credentials with empty key)
systemd-detect-virt: new options &dash&dashcvm (Only detect whether we are run in a confidential VM) + &dash&dashlist-cvm (List all known and detectable types of confidential virtualization)
systemd-firstboot: new options &dash&dashimage-policy=POLICY (Specify disk image dissection policy), &dash&dashkernel-command-line=CMDLINE (Set kernel command line) + &dash&dashreset (Remove existing files)
systemd-id128: new commands var-partition-uuid (Print the UUID for the /var/ partition) + show [NAME UUID] (Print one or more UUIDs), and new options &dash&dashno-pager (Do not pipe output into a pager), &dash&dashno-legend (Do not show the headers and footers), &dash&dashjson=FORMAT (Output inspection data in JSON), &dashj (Equivalent to &dash&dashjson=pretty (on TTY) or &dash&dashjson=short (otherwise)) + &dashP &dash&dashvalue (Only print the value)
systemd-inhibit: new option &dash&dashno-ask-password (Do not attempt interactive authorization)
systemd-machine-id-setup: new option &dash&dashimage-policy=POLICY (Specify disk image dissection policy)
systemd-mount: new options &dash&dashjson=pretty short off (Generate JSON output) + &dash&dashtmpfs (Create a new tmpfs on the mount point)
systemd-notify: new options &dash&dashreloading (Inform the service manager about configuration reloading), &dash&dashstopping (Inform the service manager about service shutdown), &dash&dashexec (Execute command line separated by ; once done), &dash&dashfd=FD (Pass specified file descriptor with along with message) + &dash&dashfdname=NAME (Name to assign to passed file descriptor(s))
systemd-path: new option &dash&dashno-pager (Do not pipe output into a pager)
systemd-run: new options &dash&dashexpand-environment=BOOL (Control expansion of environment variables), &dash&dashjson=pretty short off (Print unit name and invocation id as JSON), &dash&dashignore-failure (Ignore the exit status of the invoked process) + &dash&dashbackground=COLOR (Set ANSI color for background)
systemd-sysext: new options &dash&dashmutable=yes no auto import ephemeral ephemeral-import (Specify a mutability mode of the merged hierarchy), &dash&dashno-reload (Do not reload the service manager), &dash&dashimage-policy=POLICY (Specify disk image dissection policy) + &dash&dashnoexec=BOOL (Whether to mount extension overlay with noexec)
systemd-sysusers: new options &dash&dashtldr (Show non-comment parts of configuration) + &dash&dashimage-policy=POLICY (Specify disk image dissection policy)
systemd-tmpfiles: new command &dash&dashpurge(Delete files and directories marked for creation in specified configuration files (careful!)), and new options &dash&dashuser (Execute user configuration), &dash&dashtldr (Show non-comment parts of configuration files), &dash&dashgraceful (Quietly ignore unknown users or groups), &dash&dashimage-policy=POLICY (Specify disk image dissection policy) + &dash&dashdry-run (Just print what would be done)
systemd-umount: new options &dash&dashjson=pretty short off (Generate JSON output) + &dash&dashtmpfs (Create a new tmpfs on the mount point)
timedatectl: new commands ntp-servers INTERFACE SERVER (Set the interface specific NTP servers) + revert INTERFACE (Revert the interface specific NTP servers) and new option &dashP NAME (Equivalent to &dash&dashvalue &dash&dashproperty=NAME)
Debian s systemd ships new binary packages:
systemd-boot-efi-amd64-signed (Tools to manage UEFI firmware updates (signed))
systemd-cryptsetup (Provides cryptsetup, integritysetup and veritysetup utilities)
systemd-netlogd (journal message forwarder)
systemd-repart (Provides the systemd-repart and systemd-sbsign utilities)
systemd-standalone-shutdown (standalone shutdown binary for use in exitrds)
systemd-ukify (tool to build Unified Kernel Images)
Linux Kernel
The trixie release ships a Linux kernel based on latest longterm version 6.12. As usual there are lots of changes in the kernel area, including better hardware support, and this might warrant a separate blog entry. To highlight some changes with Debian trixie:
New Debian package linux-bpf-dev, providing the header file for BPF CO-RE builds
New Debian package intel-sdsi, Intel On Demand (SDSi) provisioning tool
New Debian package virtme-ng, providing helper scripts to easily test a Linux kernel on a QEMU VM
See Kernelnewbies.org for further changes between kernel versions.
Configuration management
For puppet users, Debian provides the puppet-agent (v8.10.0), puppetserver (v8.7.0) and puppetdb (v8.4.1) packages. Puppet s upstream does not provide packages for trixie, yet. Given how long it took them for Debian bookworm, and with their recent Plans for Open Source Puppet in 2025, it s unclear when (and whether at all) we might get something. As a result of upstream behavior, also the OpenVox project evolved, and they already provide Debian 13/trixie support (https://apt.voxpupuli.org/openvox8-release-debian13.deb). FYI: the AIO puppet-agent package for bookworm (v7.34.0-1bookworm) so far works fine for me on Debian/trixie. Be aware that due to the apt-key removal you need a recent version of the puppetlabs-apt for usage with trixie. The puppetlabs-ntp module isn t yet ready for trixie (regarding ntp/ntpsec), if you should depend on that.
ansible is available and made it with version 2.19 into trixie.
Prometheus stack
Prometheus server was updated from v2.42.0 to v2.53, and all the exporters that got shipped with bookworm are still around (in more recent versions of course). Trixie gained some new exporters:
Virtualization
docker (v26.1.5), ganeti (v3.1.0), libvirt (v11.3.0, be aware of significant changes to libvirt packaging), lxc (v6.0.4), podman (v5.4.2), openstack (see openstack-team on Salsa), qemu/kvm (v10.0.2), xen (v4.20.0) are all still around.
Proxmox already announced their PVE 9.0 BETA, being based on trixie and providing 6.14.8-1 kernel, QEMU 10.0.2, LXC 6.0.4, OpenZFS 2.3.3.
Vagrant is available in version 2.3.7, but Vagrant upstream does not provide packages for trixie yet. Given that HashiCorp adopted the BSL, the future of vagrant in Debian is unclear.
If you re relying on VirtualBox, be aware that upstream doesn t provide packages for trixie, yet. VirtualBox is available from Debian/unstable (version 7.1.12-dfsg-1 as of 2025-07-20), but not shipped with stable release since quite some time (due to lack of cooperation from upstream on security support for older releases, see #794466). Be aware that starting with Linux kernel 6.12, KVM initializes virtualization on module loading by default. This prevents VirtualBox VMs from starting. In order to avoid this, either add kvm.enable_virt_at_load=0 parameter into kernel command line or unload the corresponding kvm_intel / kvm_amd module.
If you want to use Vagrant with VirtualBox on trixie, be aware that Debian s vagrant package as present in trixie doesn t support the VirtualBox package version 7.1 as present in Debian/unstable (manually patching vagrant s meta.rb and rebuilding the package without Breaks: virtualbox (>= 7.1) is known to be working).
util-linux
The are plenty of new options available in the tools provided by util-linux:
blkdiscard: new option &dash&dashquiet (suppress warning messages)
blockdev: new options &dash&dashgetdiskseq (get disk sequence number) + &dash&dashgetzonesz (get zone size)
dmesg: new option &dash&dashkmsg-file (use the file in kmsg format), new &dash&dashtime-format argument raw
findmnt: new options &dash&dashlist-columns (list the available columns), &dash&dashdfi (imitate the output of df(1) with -i option), &dash&dashid (filter by mount node ID), &dash&dashfilter (apply display filter) + &dash&dashuniq-id (filter by
mount node 64-bit ID)
fstrim: new option -types . (limit the set of filesystem types)
hardlink: new options &dash&dashrespect-dir (directory names have to be identical), &dash&dashexclude-subtree (regular expression to exclude directories), &dash&dashprioritize-trees (files found in the earliest specified top-level directory have higher priority), &dash&dashlist-duplicates (print every group of duplicate files), &dash&dashmount (stay within the same filesystem) + &dash&dashzero (delimit output with NULs instead of newlines)
ipcmk: new options &dash&dashposix-shmem (create POSIX shared memory segment of size), &dash&dashposix-semaphore (create POSIX semaphore), &dash&dashposix-mqueue (create POSIX message queue) + &dash&dashname (name of the POSIX resource)
ipcrm: new options &dash&dashposix-shmem (remove POSIX shared memory segment by name), &dash&dashposix-mqueue (remove POSIX message queue by name), &dash&dashposix-semaphore (remove POSIX semaphore by name) + &dash&dashall= (remove all in specified category)
lsblk: new options &dash&dashct-filter (restrict the next counter), &dash&dashct (define a custom counter), &dash&dashhighlight (colorize lines matching the expression), &dash&dashlist-columns (list the available columns), &dash&dashnvme (output info about NVMe devices), &dash&dashproperties-by (methods used to gather data), &dash&dashfilter (print only lines matching the expression), &dash&dashvirtio (output info about virtio devices)
lscpu: new options &dash&dashraw (use raw output format (for -e, -p and -C)) + &dash&dashhierarchic= (use subsections in summary (auto, never, always))
lslocks: new option &dash&dashlist-columns (list the available columns)
lslogins: new option &dash&dashlastlog2 (set an alternate path for lastlog2)
lsns: new options &dash&dashpersistent (namespaces without processes), &dash&dashfilter (apply display filter) + &dash&dashlist-columns (list the available columns)
mkswap: new options &dash&dashendianness= (specify the endianness to use (native, little or big)), &dash&dashoffset (specify the offset in the device), &dash&dashsize (specify the size of a swap file in bytes) + &dash&dashfile (create a swap file)
namei: &dash&dashcontext (print any security context of each file)
nsenter: new options &dash&dashnet-socket (enter socket s network namespace), &dash&dashuser-parent (enter parent user namespace), &dash&dashkeep-caps (retain capabilities granted in user namespaces), &dash&dashenv (inherit environment variables from tar get process) + &dash&dashjoin-cgroup (join the cgroup of the target process)
runuser: new option &dash&dashno-pty (do not create a new pseudo-terminal)
setarch: new option &dash&dashshow= (show current or specific personality and exit)
setpriv: new options &dash&dashptracer (allow ptracing from the given process), &dash&dashlandlock-access (add Landlock access), &dash&dashlandlock-rule (add Landlock rule) + &dash&dashseccomp-filter (load seccomp filter from file)
su: new option &dash&dashno-pty (do not create a new pseudo-terminal)
unshare: new option &dash&dashload-interp ( load binfmt definition in the namespace)
whereis: new option -g (interpret name as glob (pathnames pattern))
wipefs: new argument option feature for &dash&dashbackup= option to specify directory (instead of default $HOME)
zramctl: new option &dash&dashalgorithm-params (algorithm parameters to use)
Now no longer present in util-linux as of trixie:
addpart (tell the kernel about the existence of a specified partition): use partx instead
delpart (tell the kernel to forget about a specified partition): use partx instead
last (show a listing of last logged in users, binary got moved to wtmpdb), lastb (show a listing of last logged in users), mesg (control write access of other users to your terminal), utmpdump (dump UTMP and WTMP files in raw format): see Debian release notes for details
coresched: manage core scheduling cookies for tasks
enosys: utility to make syscalls fail with ENOSYS
exch: atomically exchanges paths between two files
fadvise: utility to use the posix_fadvise system call
pipesz: set or examine pipe buffer sizes and optionally execute command.
waitpid: utility to wait for arbitrary processes
OpenSSH
OpenSSH was updated from v9.2p1 to 10.0p1-5, so if you re interested in all the changes, check out the release notes between those versions (9.3, 9.4, 9.5, 9.6, 9.7, 9.8, 9.9 + 10.0).
Let s highlight some notable behavior changes in Debian:
add support for reading ED25519 private keys in PEM PKCS8 format. Previously only the OpenSSH private key format was supported.
the new hybrid post-quantum algorithm mlkem768x25519-sha256 (based on the FIPS 203 Module-Lattice Key Encapsulation mechanism (ML-KEM) combined with X25519 ECDH) is now used by default for key agreement. This algorithm is considered to be safe against attack by quantum computers, is guaranteed to be no less strong than the popular curve25519-sha256 algorithm, has been standardised by NIST and is considerably faster than the previous default.
the ssh-agent will now delete all loaded keys when signaled with SIGUSR1. This allows deletion of keys without having access to $SSH_AUTH_SOCK.
support systemd-style socket activation in ssh-agent using the LISTEN_PID/LISTEN_FDS mechanism. Activated when these environment variables are set, the agent is started with the -d or -D option and no socket path is set.
add a sshd -G option that parses and prints the effective configuration without attempting to load private keys and perform other checks. (This allows usage of the option before keys have been generated and for configuration evaluation and verification by unprivileged users.)
add support for configuration tags to ssh(1). This adds a ssh_config(5) Tag directive and corresponding Match tag predicate that may be used to select blocks of configuration.
add a match localnetwork predicate. This allows matching on the addresses of available network interfaces and may be used to vary the effective client configuration based on network location.
add a %j token that expands to the configured ProxyJump hostname
add support for Match sessiontype to ssh_config. Allows matching on the type of session initially requested, either shell for interactive sessions, exec for command execution sessions, subsystem for subsystem requests, such as sftp, or none for transport/forwarding-only sessions.
allow glob(3) patterns to be used in sshd_config AuthorizedKeysFile and AuthorizedPrincipalsFile directives.
Thanks to everyone involved in the release, looking forward to trixie + and happy upgrading!
Let s continue with working towards Debian/forky. :)
So, for the first year, this year s DebConf had the DebConf Academic Track ,
that is, content for a one-day-long set of short sessions, for which of them
there was a written article presenting the content often with a very academic
format, but not necessarily. I hope that for future DebConfs we will continue to
hold this track, and that we can help bridge the gap: to get people that are
not usually from the academic / universitary prepare content that will be
formally expressed and included in a long-lasting, indexed book of
proceedings. We did have (informal) proceedings in many of the early DebConfs,
and I m very happy to regain that, but with 20 years of better practices.
Anyway, of course, I took the opportunity to join this experiment, and together
with my Mexican colleague Tzolkin Gardu o who is finishing her PhD here in
France (or should I say, second to her, as she is the true leading author of our
work). And here you can see the paper we submitted to the DebConf Academic
Track, which will be published soon:
A retrieval-augmented-generation pipeline to help users query system-provided
documentation
The corresponding source code is all available at Tzolkin s
repository in GitHub.
So, what is it about, in shorter words and layman terms?
Debian has lots of documentation, but lacks in discoverability. We targetted our
venerable manpages: It is well known that manpages are relevant, well-organized
documentation describing how to use each of the binaries we ship in Debian. Eric
Raymond wrote in his well-known essay The Art of Unix Programming
(2003)
that the Unix cultural style is telegraphic but complete. It does not hold you
by the hand, but it usualy points in the right direction.
Our original intent was to digest all of the information in our manpages, but we
narrowed it to only the first section of the manual due to the limitations of
the hardware a good friend lent us to play with LLMs. We took four different
base, redistributable (although, yes, non-DFSG-free) Large Language Models
downloaded from HuggingFace (T5-small, MiniLM-L6-v2, BGE-small-en and
Multilingual-e5-small), and trained them with the 34579 pages found inside
/usr/share/man/man1 of all of the existing Debian packages. We did some
interesting fine-tuning (for further details, please refer to the paper itself
or to our GitHub repository.
The idea is to present an interactive tool that udnerstand natural language
queries, and answers with the specific manpage to which they better relate
(I d like to say they answer best , but Tzolkin has repeatedly tried to correct
my understanding of the resulting vectorial distances).
I had prepared some images to present as interaction examples, but I ll wrap up
this post with something even better So, this is what you would get with the
following queries:
Of course, garbage-in means garbage-out. LLMs cannot tell you I don t have an
answer for you , so you get hallucinations (or at least, irrelevant answers):
I want my lunch tickets for
Brest. Guess I was
hungry when working my examples
But I am a vegan, going
on with the silliness Anyway, do take note that, although it is not
directly relatable, the reported distance is very large, so the model does
tell me in a way these are my best results, but I wouldn t trust them too
much .
We were happy to present like this. During DebCamp, however, I was able to
devote some time to translating my client into a Web-accessible system. Do note
that it s a bit brittle, and might break now and then. But you are welcome to
play with it!
Play with the Web interface for our RAG for Debian
manuals!
I find it worth sharing that, while we used quite a bit of GPU for the training
(not too much a couple of nights on an old-although-powerful nVidia 1070 card
lent to us by the Felipe Esquivel Fund for Science and Cultural Support), all
querying takes place in the CPU, and I usually get between 0.1 and 0.3 seconds
per query. My server s architecture is far from rock-solid, and it can break now
and then but at least it should respawn upon failure And it should at
least be available there for a couple of weeks into August.
Anyway, this has been a very interesting journey getting my toes wet in the
waters of LLM, and while current results are a bit crude, I think this can be
made into an interesting tool for Debian exploration.
Debian packaging is notoriously hard. Far too many new contributors give up while trying, and many long-time contributors leave due to burnout from having to do too many thankless maintenance tasks. Some just skip testing their changes properly because it feels like too much toil.
Debcraft is my attempt to solve this by automating all the boring stuff, and making it easier to learn the correct practices and helping new and old packagers better track changes in both source code and build artifacts.
The challenge of declarative packaging code
Unlike how rpm or apk packages are done, the deb package sources by design avoid having one massive procedural packaging recipe. Instead, the packaging is defined in multiple declarative files in the debian/ subdirectory. For example, instead of a script running install -m 755 bin/btop /usr/bin/btop there is a file debian/btop.install containing the line usr/bin/btop.
This makes the overall system more robust and reliable, and allows, for example, extensive static analysis to find problems without having to build the package. The notable exception is the debian/rules file, which contains procedural code that can modify any aspect of the package build. Almost all other files are declarative.
Benefits include, among others, that the effect of a Debian-wide policy change can be relatively easily predicted by scanning what attributes and configurations all packages have declared.
The drawback is that to understand the syntax and meaning of each file, one must understand which build tools read which files and traverse potentially multiple layers of abstraction. In my view, this is the root cause for most of the perceived complexity.
Common complaints about .deb packaging
Related to the above, people learning Debian packaging frequently voice the following complaints:
Debian has too many tools to learn, often with overlapping or duplicate functionality.
Too much outdated and inconsistent documentation that makes learning the numerous tools needlessly hard.
Lack of documentation of the generally agreed best practices, mainly due to Debian s reluctance as a project to pick one tool and deprecate the alternatives.
Multiple layers of abstraction and lack of clarity on what any single change in the debian/ subdirectory leads to in the final package.
Requirement of Debian packages to be developed on a Debian system.
How Debcraft solves (some of) this
Debcraft is intentionally opinionated for the sake of simplicity, and makes heavy use of git, git-buildpackage, and most importantly Linux containers, supporting both Docker and Podman.
By using containers, Debcraft frees the user from the requirement of having to run Debian. This makes .deb packaging more accessible to developers running some other Linux distro or even Mac or Windows (with WSL). Of course we want developers to run Debian (or a derivative like Ubuntu) but we want them even more to build, test and ship their software as .deb. Even for Debian/Ubuntu users having everything done inside clean hermetic containers of the latest target distribution version will yield more robust, secure and reproducible builds and tests. All containers are built automatically on-the-fly using best practices for layer caching, making everything easy and fast.
Debcraft has simple commands to make it easy to build, rebuild, test and update packages. The most fundamental command is debcraft build, which will not only build the package but also fetch the sources if not already present, and with flags such as --distribution or --source-only build for any requested Debian or Ubuntu release, or generate source packages only for Debian or PPA upload purposes.
For ease of use, the output is colored and includes helpful explanations on what is being done, and suggests relevant Debian documentation for more information.
Most importantly, the build artifacts, along with various logs, are stored in separate directories, making it easy to compare before and after to see what changed as a result of the code or dependency updates (utilizing diffoscope among others).
While the above helps to debug successful builds, there is also the debcraft shell command to make debugging failed builds significantly easier by dropping into a shell where one can run various dh commands one-by-one.
Once the build works, testing autopkgtests is as easy as running debcraft test. As with all other commands, Debcraft is smart enough to read information like the target distribution from the debian/changelog entry.
When the package is ready to be released, there is the debcraft release command that will create the Debian source package in the correct format and facilitate uploading it either to your Personal Package Archive (PPA) or if you are a Debian Developer to the official Debian archive.
Automatically improve and update packages
Additionally, the command debcraft improve will try to fix all issues that are possible to address automatically. It utilizes, among others, lintian-brush, codespell and debputy. This makes repetitive Debian maintenance tasks easier, such as updating the package to follow the latest Debian policies.
To update the package to the latest upstream version there is also debcraft update. It will read the package configuration files such as debian/gbp.conf and debian/watch and attempts to import the latest upstream version, refresh patches, build and run autopkgtests. If everything passes, the new version is committed. This helps automate the process of updating to new upstream versions.
Try out Debcraft now!
On a recent version of Debian and Ubuntu, Debcraft can be installed simply by running apt install debcraft. To use Debcraft on some other distribution or to get the latest features available in the development version install it using:
git clone https://salsa.debian.org/debian/debcraft.git
cd debcraft
make install-local
To see exact usage instructions run debcraft --help.
Contributions welcome
Current Debcraft version 0.5 still has some rough edges and missing features, but I have personally been using it for over a year to maintain all my packages in Debian. If you come across some issue, feel free to file a report at https://salsa.debian.org/debian/debcraft/-/issues or submit an improvement at https://salsa.debian.org/debian/debcraft/-/merge_requests. The code is intentionally written entirely in shell script to keep the barrier to code contribution as low as possible.
By the way, if you aspire to become a Debian Developer, and want to follow my examples in using state-of-the-art tooling and collaborate using salsa.debian.org, feel free to reach out for mentorship. I am glad to see more people contribute to Debian!
Introduction
Some months ago, my colleague Madeeha Javed and I wrote a tool to convert QEMU disk images into qcow2, writing the result directly to stdout.
This tool is called qcow2-to-stdout.py and can be used for example to create a new image and pipe it through gzip and/or send it directly over the network without having to write it to disk first.
This program is included in the QEMU repository: https://github.com/qemu/qemu/blob/master/scripts/qcow2-to-stdout.py
If you simply want to use it then all you need to do is have a look at these examples:
$ qcow2-to-stdout.py source.raw > dest.qcow2 $ qcow2-to-stdout.py -f dmg source.dmg gzip > dest.qcow2.gz
If you re interested in the technical details, read on.
A closer look under the hood
QEMU uses disk images to store the contents of the VM s hard drive. Images are often in qcow2, QEMU s native format, although a variety of other formats and protocols are also supported.
I have written in detail about the qcow2 format in the past (for example, here and here), but the general idea is very easy to understand: the virtual drive is divided into clusters of a certain size (64 KB by default), and only the clusters containing non-zero data need to be physically present in the qcow2 image. So what we have is essentially a collection of data clusters and a set of tables that map guest clusters (what the VM sees) to host clusters (what the qcow2 file actually stores).
qemu-img is a powerful and versatile tool that can be used to create, modify and convert disk images. It has many different options, but one question that sometimes arises is whether it can use stdin or stdout instead of regular files when converting images.
The short answer is that this is not possible in general. qemu-img convert works by checking the (virtual) size of the source image, creating a destination image of that same size and finally copying all the data from start to finish.
Reading a qcow2 image from stdin doesn t work because data and metadata blocks can come in any arbitrary order, so it s perfectly possible that the information that we need in order to start writing the destination image is at the end of the input data .
Writing a qcow2 image to stdout doesn t work either because we need to know in advance the complete list of clusters from the source image that contain non-zero data (this is essential because it affects the destination file s metadata). However, if we do have that information then writing a new image directly to stdout is technically possible.
The bad news is that qemu-img won t help us here: it uses the same I/O code as the rest of QEMU. This generic approach makes total sense because it s simple, versatile and is valid for any kind of source and destination image that QEMU supports. However, it needs random access to both images.
If we want to write a qcow2 file directly to stdout we need new code written specifically for this purpose, and since it cannot reuse the logic present in the QEMU code this was written as a separate tool (a Python script).
The process itself goes like this:
Read the source image from start to finish in order to determine which clusters contain non-zero data. These are the only clusters that need to be present in the new image.
Write to stdout all the metadata structures of the new image. This is now possible because after the previous step we know how much data we have and where it is located.
Read the source image again and copy the clusters with non-zero data to stdout.
Images created with this program always have the same layout: header, refcount tables and blocks, L1 and L2 tables, and finally all data clusters.
One problem here is that, while QEMU can read many different image formats, qcow2-to-stdout.py is an independent tool that does not share any of the code and therefore can only read raw files. The solution here is to use qemu-storage-daemon. This program is part of QEMU and it can use FUSE to export any file that QEMU can read as a raw file. The usage of qemu-storage-daemon is handled automatically and the user only needs to specify the format of the source file:
$ qcow2-to-stdout.py -f dmg source.dmg > dest.qcow2qcow2-to-stdout.py can only create basic qcow2 files and does not support features like compression or encryption. However, a few parameters can be adjusted, like the cluster size (-c), the width of the reference count entries (-r) and whether the new image is created with the input as an external data file (-d and -R).
And this is all, I hope that you find this tool useful and this post informative. Enjoy!
Acknowledgments
This work has been developed by Igalia and sponsored by Outscale, a Dassault Syst mes brand.
This problem would not happen if the input data was in raw format but in this case we would not know the size in advance.
Posted on July 15, 2025
Tags: madeof:bits, topic:xmpp, topic:debian
This is an approximation of what I told at my talk Federated instant
messaging, 100% debianized
at DebConf 25, for people who prefer reading text. There will also be a
video recording, as soon as it s ready :) at the link above.
Communicating is a basic human need, and today some kind of
computer-mediated communication is a requirement for most people,
especially those in this room.
With everything that is happening, it s now more important than ever
that these means of communication aren t controlled by entities that
can t be trusted, whether because they can stop providing the service at
any given time or worse because they are going to abuse it in order to
extract more profit.
If only there was a well established chat system based on some standard
developed in an open way, with all of the features one expects
from a chat system but federated so that one can choose between many
different and independent providers, or even self-hosting.
But wait, it does exist!
I m not talking about IRC, I m talking about XMPP!
While it has been around since the last millennium, it has not remained
still, with hundred of XMPP Extension Protocols, or XEPs that have been
developed to add all of the features that nobody in 1999 imagined we
could need in Instant Messaging today, and more, such as IoT devices or
even social networks.
There is a myth that this makes XMPP a mess of incompatible software,
but there is an XEP for that: XEP-0479: XMPP Compliance Suites 2023,
which is a list of XEPs that needs to be supported by Instant Messaging
servers and clients, including mobile ones, and all of the recommended
ones will mostly just work.
These include conversations.im on android, dino on linux, which also
works pretty nicely on linux phones, gajim for a more fully featured
option that includes the kitchen sink, profanity for text interface
fanatics like me, and I ve heard that monal works decently enough on the
iThings.
One thing that sets XMPP apart from other federated protocols, is that
it has already gone through the phase where everybody was on one very
big server, which then cut out federation, and we ve learned from the
experience. These days there are still a few places that cater to
newcomers, like https://account.conversations.im/, https://snikket.org/
(which also includes tools to make it easier to host your own instance)
and https://quicksy.im/, but most people are actually on servers of a
manageable size.
My strong recommendation is for community hosting: not just self-hosting
for yourself, but finding a community you feel part of and trust, and
share a server with them, whether managed by volunteers from the
community itself, or by a paid provider.
If you are a Debian Developer, you already have one: you can go to
https://db.debian.org/ , select Change rtc password to set your own
password, wait an hour or so and you re good to go, as described at the
bottom of https://wiki.debian.org/Teams/DebianSocial.
A few years ago it had remained a bit behind, but these days it s
managed by an active team, and if you re missing some features, or just
want to know what s happening with it, you can join their BoF on Friday
afternoon (and
also thank them for their work).
But for most people in this room, I d also recommend finding a friend or
two who can help as a backup, and run a server for your own families or
community: as a certified lazy person who doesn t like doing sysadmin
jobs, I can guarantee it s perfectly feasible, about in the same range
of difficulty as running your own web server for a static site.
The two most popular servers for this, prosody and ejabberd, are well
maintained in Debian, and these days there isn t a lot more to do than
installing them, telling them your hostname, setting up a few DNS
entries, and then you mostly need to keep the machine updated and very
little else.
After that, it s just applying system security updates, upgrading
everything every couple years (some configuration updates may be needed,
but nothing major) and maybe helping some non-technical users, if you
are hosting your non-technical friends (the kind who would
need support on any other platform).
Question time (including IRC questions) included which server would be
recommended for very few users (I use prosody and I m very happy with
it, but I believe ejabberd works also just fine), then somebody reminded
me that I had forgotten to mention https://www.chatons.org/ , which
lists free, ethical and decentralized services, including xmpp ones.
I was also asked a comparison with matrix, which does cover a very
similar target as XMPP, but I am quite biased against it, and I d prefer
to talk well of my favourite platform than badly of its competitor.
DebConf25, the 26th edition of the Debian
conference is taking place in Brest Campus of
IMT Atlantique Bretagne-Pays de la Loire,
France. We appreciate the organizers for their hard work, and hope this event
will be highly beneficial for those who attend in person as well as online.
This event would not be possible without the help from our generous sponsors.
We would like to warmly welcome the sponsors of DebConf 25, and introduce them to
you.
We have five Platinum sponsors.
Our first Platinum sponsor is Infomaniak.
Infomaniak is Switzerland s leading developer of Web technologies. With
operations all over Europe and based exclusively in Switzerland, the company
designs and manages its own data centers powered by 100% renewable energy,
and develops all its solutions locally, without outsourcing. With millions of
users and the trust of public and private organizations across Europe - such
as RTBF, the United Nations, central banks, over 3,000 radio and TV stations,
as well as numerous cities and security bodies - Infomaniak stands for
sovereign, sustainable and independent digital technology. The company offers
a complete suite of collaborative tools, cloud hosting, streaming, marketing
and events solutions, while being owned by its employees and self-financed
exclusively by its customers.
Proxmox is the second Platinum sponsor.
Proxmox develops powerful, yet easy-to-use Open Source server software. The
product portfolio from Proxmox, including server virtualization, backup,
and email security, helps companies of any size, sector, or industry to
simplify their IT infrastructures. The Proxmox solutions are built on Debian,
we are happy that they give back to the community by sponsoring DebConf25.
Viridien is our third Platinum sponsor.
Viridien is an advanced technology, digital and Earth data company that
pushes the boundaries of science for a more prosperous and sustainable future.
Viridien has been using Debian-based systems to power most of its HPC
infrastructure and its cloud platform since 2009 and currently employs two
active Debian Project Members.
EDF is our fourth Platinum sponsor.
EDF is a leading global utility company focused on low-carbon power
generation. The group uses advanced engineering and scientific computing
tools to drive innovation and efficiency in its operations, especially in
nuclear power plant design and safety assessment.
Since 2003, the EDF Group has been using Debian as its main scientific
computing environment. Debian's focus on stability and reproducibility
ensures that EDF's calculations and simulations produce consistent and
accurate results.
AMD is our fifth Platinum sponsor.
The AMD ROCm platform includes programming models, tools, compilers,
libraries, and runtimes for AI and HPC solution development on AMD GPUs.
Debian is an officially supported platform for AMD ROCm and a growing
number of components are now included directly in the Debian distribution.
For more than 55 years AMD has driven innovation in high-performance
computing, graphics and visualization technologies.
AMD is deeply committed to supporting and contributing to open-source
projects, foundations, and open-standards organizations, taking pride in
fostering innovation and collaboration within the open-source community.
Our Gold sponsors are:
Ubuntu,
the Operating System delivered by Canonical.
Freexian,
a services company specialized in Free Software and in particular Debian
GNU/Linux, covering consulting, custom developments, support and training.
Lenovo,
a global technology leader manufacturing a wide portfolio of connected
products including smartphones, tablets, PCs and workstations as well as
AR/VR devices, smart home/office and data center solutions.
Collabora, a global consultancy delivering
Open Source software solutions to the commercial world.
Vyos Networks provides a free routing platform that
competes directly with other commercially available solutions and VyOS is an
open source network operating system based on Debian.
Our Silver sponsors are:
Google, one of the largest technology companies in
the world, providing a wide range of Internet-related services and products
such as online advertising technologies, search, cloud computing, software,
and hardware.
Arm: leading technology provider of processor
IP, Arm powered solutions have been supporting innovation for
more than 30 years and are deployed in over 280 billion chips to date.
Siemens is a technology company
focused on industry, infrastructure and transport.
Linagora develops ethical Free and Open Source
Software, supports Open Source products that helps humans and boosts digital
progress. They promote Free Software in France.
Pexip brings the ease of commercial video
platforms to secure and sovereign environments without compromising control
or performance.
Sipgate offers cloud telephony that automates
routine tasks, recognizes customer needs, and enables deep CRM
integrations.
evolix is a french company specialized in Hosting
and Managed Services Provider (MSP) working only with Open Source softwares.
Civil Infrastructure Platform,
a collaborative project hosted by the Linux Foundation, establishing an open
source base layer of industrial grade software.
credativ: a consulting and services company
offering comprehensive services and technical support for the implementation
and operation of Open Source Software in business applications.
Wind River is a leader in delivering the
highest levels of secure, safe, and reliable solutions for mission-critical
intelligent systems.
NovaCustom is a company that lets you
configure your own laptop with various hardware and software options,
focusing on privacy and security.
Microsoft who enables
digital transformation for the era of an intelligent cloud and an intelligent
edge. Its mission is to empower every person and every organization on the
planet to achieve more.
McKay Brothers is the acknowledged
leader in providing extreme low latency microwave private bandwidth for firms
trading in financial markets.
Matanel Foundation, whose first concern is to
preserve the cohesion of a society and a nation plagued by divisions.
Qualcomm Technologies
one of the world's leading companies in field of mobile technology, sponsors
and contributes to Open Source developer communities that drive collaboration.
A special thanks to the
IMT Atlantique Bretagne-Pays de la Loire,
our Venue Partner and our Network Partner ResEl!
Thanks to all our sponsors for their support!
Their contributions enable a diverse global community of Debian developers and
maintainers to collaborate, support one another, and share knowledge at
DebConf25.
Tobias Frost
did 2.5h (out of 12.0h assigned), thus carrying over 9.5h to the next month.
Evolution of the situation
In June, we released 35 DLAs.
Notable security updates:
mariadb-10.5, prepared by Otto Kek l inen, fixes vulnerabilities which could result in denial of service, information disclosure, or unauthorized data modification
python-django, prepared by Chris Lamb, fixes vulnerabilities which would result in log injection or denial of service
webkit2gtk, prepared by Emilio Pozuelo Monfort, fixes many vulnerabilities which could results in a wide range of issues
xorg-server, prepared by Emilio Pozuelo Monfort, fixes multiple vulnerabilities which may result in privilege escalation
sudo, prepared by Thorsten Alteholz, fixes a vulnerability which could result in privilege escalation
Notable non-security updates:
debian-security-support, prepared by Santiago Ruano Rinc n, updates status of packages which receive limited security support or which have reached the end of security support
dns-root-data, prepared by Sylvain Beucler, updates the DNSSEC trust anchors
This month s contributions from outside the regular team include the mariadb-10.5 update mentioned above, prepared by Otto Kek l inen (the package maintainer); an update to libfile-find-rule-perl, prepared by Salvatore Bonaccorso (a member of the Debian Security Team); an update to activemq, prepared by Emmanuel Arias (a maintainer of the package).
Additionally, LTS Team members contributed stable updates of the following packages:
curl, prepared by Carlos Henrique Lima Melara
python-tornado, prepared by Daniel Leidert
python-flask-cors, prepared by Daniel Leidert
common-vfs, prepared by Daniel Leidert
cjson, prepared by Adrian Bunk
icu, prepared by Adrian Bunk
node-tar-fs, prepared by Adrian Bunk
rar, prepared by Adrian Bunk
Something of particular noteworthiness is that LTS contributor Carlos Henrique Lima Melara discovered a regression in the upstream fix for CVE-2023-2753 in curl. The corrective action which he took included providing a patch to upstream, uploading a stable update of curl, and further updating the version of curl in LTS.
DebConf, the annual Debian Conference, is coming up in July and, as is customary each year, the week preceding the conference will feature an event called DebCamp. The DebCamp week provides an opportunity for teams and other interested groups/individuals to meet together in person in the same venue as the conference itself, with the purpose of doing focused work, often called sprints . LTS coordinator Roberto C. S nchez has announced that the LTS Team is planning to hold a sprint primarily focused on the Debian security tracker and the associated tooling used by the LTS Team and the Debian Security Team.
Thanks to our sponsors
Sponsors that joined recently are in bold.
Log from (failed) platform test
After some fun I got the serial console working and re-ran
the platform test.
After a bit of reading the serial console, I realized that rmmod dwc3 was causing
more problems than it solved, in particularly reliable hard lockup on one of the CPUs.
My revised test script is
set -xecho platform > /sys/power/pm_test
echo reboot > /sys/power/disk
sleep2
rmmod mt76x2u
sleep2echo disk > /sys/power/state
sleep2
modprobe mt76x2u
The current problem seems to be pcie not resuming properly.
[ 65.306842] usbcore: deregistering interface driver mt76x2u
[ 65.343606] wlx000a5205eb2d: deauthenticating from 20:05:b7:00:2d:89 by local choice (Reason: 3=DEAUTH_LEAVING)
[ 67.995239] PM: hibernation: hibernation entry
[ 68.048103] Filesystems sync: 0.022 seconds
[ 68.049005] Freezing user space processes
[ 68.051075] Freezing user space processes completed (elapsed 0.001 seconds)
[ 68.051760] OOM killer disabled.
[ 68.052597] PM: hibernation: Basic memory bitmaps created
[ 68.053108] PM: hibernation: Preallocating image memory
[ 69.719040] PM: hibernation: Allocated 366708 pages for snapshot
[ 69.719650] PM: hibernation: Allocated 1466832 kbytes in 1.66 seconds (883.63 MB/s)
[ 69.720370] Freezing remaining freezable tasks
[ 69.723558] Freezing remaining freezable tasks completed (elapsed 0.002 seconds)
[ 69.728002] rk_gmac-dwmac fe1b0000.ethernet end0: Link is Down
[ 69.992324] rockchip-dw-pcie a40c00000.pcie: Failed to receive PME_TO_Ack
[ 69.993405] PM: hibernation: debug: Waiting for 5 seconds.
[ 76.059484] rockchip-dw-pcie a40c00000.pcie: Phy link never came up
[ 76.060043] rockchip-dw-pcie a40c00000.pcie: fail to resume
[ 76.060546] rockchip-dw-pcie a40c00000.pcie: PM: dpm_run_callback(): genpd_restore_noirq returns -110
[ 76.061363] rockchip-dw-pcie a40c00000.pcie: PM: failed to restore noirq: error -110
For some time now I was looking for a device to replace my Thinkpad. Its a 14"
device, but thats to big for my taste. I am a big fan of small notebooks, so
when frame.work announced their 12" laptop, I took the chance and ordered one
right away.
I was in one of the very early batches and got my package a couple of days ago.
When ordering, I chose the DIY edition, but in the end there was not that much
of DIY to do: I had to plug in the storage and the memory, put the keyboard in
and tighten some screws. There are very detailed
instructions with a lot of
photos that tell you which part to put where, which is nice.
My first impressions of the device are good - it is heavier than I anticipated,
but very vell made. It is very easy to assemble and disassemble and it feels
like it can take a hit.
When I started it the first time it took some minutes to boot because of the
new memory module, but then it told me right away that it could not detect an
operating system. As usual when I want to install a new system, I created
a GRML live usb system and tried to boot from this
USB device. But the Framwork BIOS did not want to let me boot GRML, telling
me it is blocked by the current security policy. So I started to look in the
BIOS where I could find the SecureBoot configuration, but there was no such
setting anywhere. I then resorted to a Debian Live image, which was allowed
to boot.
I only learned later, that the SecureBoot setting is in a separate section
that is not part of the main BIOS configuration dialog. There is an Administer
Secure Boot icon which you can choose when starting the device, but
apparently only before you try to load an image that is not
allowed.
I always use my personal minimal install
script to
install my Debian systems, so it did not make that much of a difference to use
Debian Live instead of GRML. I only had to apt install debootstrap before
running the script.
I updated the install script to default to trixie and to also install
shim-signed and after successful installation booted into Debian 13 on the
Framwork 12. Everthing seems to work fine so far. WIFI works. For sway to
start I had to install firmware-intel-graphics. The touchscreen works without
me having to configure anything (though I don t have frame.work stylus, as they
are not yet available), also changing the brightness of the screen worked right
away. The keyboard feels very nice, likewise the touchpad, which I configured
to allow tap-to-click using the tap enabled option of
sway-input.
One small downside of the keyboard is that it does not have a backlight, which
was a surprise. But given that this is a frame.work laptop, there are chances
that a future generation of the keyboard will have backlight support.
The screen of the laptop can be turned all the way around to the back of the
laptops body, so it can be used as a tablet. In this mode the keyboard gets
disabled to prevent accidently pushing keys when using the device in tablet
mode.
For online meetings I still prefer using headphones with cables over bluetooth
once, so I m glad that the laptop has a headphone jack on the side.
Above the screen there are a camera and a microphone, which both have separate
physical switches to disable them.
I ordered a couple of expansion cards, in the current setup I use two USB-C,
one HDMI and one USB-A. I also ordered a 1TB expansion card and only used this
to transfer my /home, but I soon realized that the card got rather hot, so I
probably won t use it as a permanent expansion.
I can not yet say a lot about how long the battery lasts, but I will bring
the laptop to DebConf 25, I guess there I ll find out. There I might also
have a chance to test if the screen is bright enough to be usable outdoors ;)
Debusine aims to be an
integrated solution to build, distribute and maintain a Debian-based
distribution. At Debconf 25, we talked about using it to pre-test uploads
to Debian
unstable,
and also touched on how Freexian is using it to help maintain the Debian
LTS and ELTS projects.
When Debian 10 (
Debconf 25 photos
It's Sunday and I'm now sitting in the train from Brest to Paris where I will be changing to Germany, on the way back from the 
Pollito and the gang of DebConf mascots wearing their conference badges (photo: Christoph Berg)
F/DF7CB in Brest (photo: Evangelos Ribeiro Tzaras)
The northern coast of Ushant (photo: Christoph Berg)
Having a nice day (photo: Christoph Berg)
Pollito on stage (photo: Christoph Berg)
Given that we only arrived in the evening of the first day of DebConf, I missed
the talk 
Wednesday was the day of the daytrip. I did not sign up for any of the trips
and used the time to 
If the menus would be written like a 
The sixteenth release of the 
So, for the first year, this year s DebConf had the DebConf Academic Track ,
that is, content for a one-day-long set of short sessions, for which of them
there was a written article presenting the content often with a very academic
format, but not necessarily. I hope that for future DebConfs we will continue to
hold this track, and that we can help bridge the gap: to get people that are
not usually from the academic / universitary prepare content that will be
formally expressed and included in a long-lasting, indexed book of
proceedings. We did have (informal) proceedings in many of the early DebConfs,
and I m very happy to regain that, but with 20 years of better practices.
Anyway, of course, I took the opportunity to join this experiment, and together
with my Mexican colleague Tzolkin Gardu o who is finishing her PhD here in
France (or should I say, second to her, as she is the true leading author of our
work). And here you can see the paper we submitted to the DebConf Academic
Track, which will be published soon:
Debian packaging is notoriously hard. Far too many new contributors give up while trying, and many long-time contributors leave due to burnout from having to do too many thankless maintenance tasks. Some just skip testing their changes properly because it feels like too much toil.
Like each month, have a look at the work funded by 
My first impressions of the device are good - it is heavier than I anticipated,
but very vell made. It is very easy to assemble and disassemble and it feels
like it can take a hit.
When I started it the first time it took some minutes to boot because of the
new memory module, but then it told me right away that it could not detect an
operating system. As usual when I want to install a new system, I created
a 
I only learned later, that the SecureBoot setting is in a separate section
that is not part of the main BIOS configuration dialog. There is an Administer
Secure Boot icon which you can choose when starting the device, but
One small downside of the keyboard is that it does not have a backlight, which
was a surprise. But given that this is a frame.work laptop, there are chances
that a future generation of the keyboard will have backlight support.
The screen of the laptop can be turned all the way around to the back of the
laptops body, so it can be used as a tablet. In this mode the keyboard gets
disabled to prevent accidently pushing keys when using the device in tablet
mode.
For online meetings I still prefer using headphones with cables over bluetooth
once, so I m glad that the laptop has a headphone jack on the side.
Above the screen there are a camera and a microphone, which both have separate
physical switches to disable them.
I ordered a couple of expansion cards, in the current setup I use two USB-C,
one HDMI and one USB-A. I also ordered a 1TB expansion card and only used this
to transfer my 
