Search Results: "ed"

18 August 2026

Thomas Lange: LLM usage in Debian

After spending many hours on reading all the proposals and discussions the best choice for me is NOTA (None of the above). We do not need to create new rules for LLM usage, we already have our DFSG and our social contract. Keep it simple, stupid. Avoid more rules!

Andy Simpkins: My first go at tracking down a kernel bug

A couple of weekends back, I upgraded my home sever. It failed to restart after running apt dist-upgrade The only update that was performed was to the kernel, it went from 6.12.88+deb13-amd64 to 6.12.100+deb13-amd64. I had previously performed an apt-get upgrade, and rebooted the machine, so I was pretty sure that this was to blame. This blog entry (is a late) attempt to document how I went about finding a fix for this issue so that next time I don t need as much hand holding as I did this time around :-) (1)
Having my machine not boot following an upgrade is pretty rare, but has happened before. Usually it is because I have done something wrong so as always confirming I haven t broken something by accident is always my first step I plugged in a keyboard an monitor to the machine and watched it boot. Being a server this takes a long time (I guess because at this stage of system initialisation we want to test things sequentially) Watching the system boot I see the usual BIOS/UEFI stages for this machine, followed by the grub menu and the the local screen showed: 
            Loading Linux 6.12.100+deb13-amd64 ...
            Loading initial ramdisk ...
Nothing else. That was it. OK that looks like I have a broken system all right, and at very early stage of the boot process process. (2)
Breaking into the grub menu and removing the quiet option yields a little more information (but not much): 
            Loading Linux 6.12.100+deb13-amd64  
            Loading initial ramdisk ...
            
            	EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID d
            	evice path
            		EFI stub: Measured initrd data into PCR 9
and nothing else. (3) Initial debugging (4) Triage Start looking for where the fault first occurred. At this point I needed help, and given that Sledge was visiting I asked if he would sanity check what I was doing. His initial thoughts were that that /boot had run out of space, but replaying my step (3) with him acting as a rubber duck showed that this was something other than PBKAC Sledge had a quick look, then informed me that between kernel images 6.12.88+deb13 and 6.12.100+deb13 Debian stable has only had shipped .90 .94 .95 and .96 kernels. We could easily try them all: OK I now have the first kernel image that doesn t boot on my system, time to raise a bug

Up until now I have been walking to my garage where the server is located and standing in front of a rack
with a monitor and keyboard plugged into the machine. However this machine supports IPMI so I spent a little time getting that up and running so that I can continue from the relative comfort of my desk (with lights, a chair and not needing to hold the keyboard with one hand) Great I can now grab screen shots from the confort of my desk (unfortunatly they are only screen shots not text files, but at least we can seen the early stage of boot, Post, grub menu and then initramfs before system log happens) (5) Collating information for the initial bug report Sledge had mentioned my problem in irc/#debain-kernal where iam_tj suggested that we try appending
debug earlycon=efifb to the kernal command line. This yielded 15 seconds worth of messages before the system locked up the last few messages being (vmlinuz-6.12.96+deb13-amd64):

[ 14.663477] RCU Tasks: Setting shift to 5 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=20.
[ 14.750474] RCU Tasks Rude: Setting shift to 5 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=20.
[ 14.838024] RCU Tasks Trace: Setting shift to 5 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=20.
[ 14.929752] NR_IRQS: 524544, nr_irqs: 584, preallocated irqs: 16
[ 15.016814] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 15.104011] Console: colour dummy device 80 25
[ 15.191236] printk: legacy console [tty0] enabled
[ 15.278249] printk: legacy bootconsole [efifb0] disabled Booting the working kernel with the same kernel options yields the SAME messages with slightly differing times, but then continues to login prompt: 

 [   14.697466] RCU Tasks: Setting shift to 5 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=20.
 [   14.784936] RCU Tasks Rude: Setting shift to 5 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=20.
 [   14.872067] RCU Tasks Trace: Setting shift to 5 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=20.
 [   14.964000] NR_IRQS: 524544, nr_irqs: 584, preallocated irqs: 16
 [   15.051482] rcu: srcu_init: Setting srcu_struct sizes based on contention.
 [   15.226079] printk: legacy console [tty0] enabled
 [   15.313751] printk: legacy bootconsole [efifb0] disabled
 [   15.400831] ACPI: Core revision 20240827
 [   15.401415] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635855245 ns
 [   15.401464] APIC: Switch to symmetric I/O mode setup
 
... and so on
iam_tj also suggested adding keep_bootcon with debug earlycon=efifb keep_bootcon on vmlinuz-6.12.96+deb13-amd64:
We get a LOT further and we see a crash / trace-back: 
[ 34.285342] BUG: kernel NULL pointer dereference, address: 0000000000000000
I raised bug #1143721 and followed it up with screen captures of the boot sequence (captured from the IPMI client) and files containing the output of dmidecode, lscpu and lspci to kive the kernel team as much information as possible: 
[6.12.96+deb13-amd64 debug earlycon=efifb keep_bootcon.tar.gz (application/gzip, attachment)]
[dmidecode.txt (text/plain, attachment)]
[lscpu.txt (text/plain, attachment)]
[lspci.txt (text/plain, attachment)]
(6) Tracking down the bug Git Bisect The problem with this type of bug is that it is hardware (class) specific, whilst the kernel doesn t boot on my system, it clearly has worked on machines used by the kernel team, the Debian test and build infrastructure, (otherwise this kernel would never have been released) and everyone else who has upgraded to the newer kernel before I did (otherwise we would be drowning in fails to boot bug reports). Carnil s excellent response to my bug: Message #15 (and help in IRC) provided me with a detailed step by step guide in how to track down the individual git commit that fails on my system. I had already (with Sledge s suggestion) made a clone of the stable branch, but was struggling to follow the steps in the Debian Linux Kernel Handbook to re-build a duplicate kernel because I didn t understand how to obtain the same configuration that Debian used to build the kernel; Carnil s email provided me the missing steps (Highlighted). 
git clone --single-branch -b linux-6.12.y https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
cd linux-stable
git checkout v6.12.95
cp /boot/config-$(uname -r) .config
yes ''   make localmodconfig
make savedefconfig
mv defconfig arch/x86/configs/my_def
test 6.12.96 to ensure this is "bad"
git checkout v6.12.96
make my_defconfig
make -j $(nproc) bindeb-pkg
  install the resulting .deb package and confirm it fails to boot and triggers the NULL pointer dereference.
Right I can now start to Bisect the problem: 
git bisect start
git bisect good v6.12.95
git bisect bad v6.12.96
Rather than use the half step point s git bisect suggested I was advised in irc to jump straight to the a given commit that from the git log was suspected as the culprit:

git checkout 977855894bca4b87afa50d21e3f3e85a5a0e901f
build and install .
fails
git bisect bad

git checkout 977855894bca4b87afa50d21e3f3e85a5a0e901f~1 ## ~1 is the commit beforehand
build and install .
fails
git bisect good The entire test tree can shown with git bisect log and this was submitted as an email to the bug report, we have found our smoking gun :-) Finally I would like to thank Carnil, Iam_tj for their time patience and fantastic support in guiding me through finding this regression. Right now kernel bugs are coming in thick and fast with a lot of AI assisted bug hunting, the increased numbers of bugs mean that the kernel team are especially busy. Hopefully our paths will cross and I ll be able to buy you some beers (or whatever) soon. thank you. Sledge also deserves thanks for putting up with me and pointing me in the right direction (as ever). Lucky for me that he lives nearby so I can provide beers on a regular basis :-)

John Goerzen: AI in Debian: The Vote, Proposals, and Nuance

Let me start with a hypothesis:
For human developers, using coding LLMs magnifies their difference in skill levels.
I am one that rarely thinks things are always black and white. Back in March, I wrote Artifial Intelligence: Shades of Gray. Since then, I ve had more of a chance to experiment with LLMs myself. I also happen to work for an employer that is taking a very pragmatic approach to LLMs: teams and individuals use it as they see fit, but if they are causing considerable expense, they have to justify it. In various settings, I have seen the egregious examples of AI slop we all know about. As I wrote in March, I have seen it both waste more time than it saves, and save a ton of time. I have come to see that, as a tool, it is most valuable when it is running under the supervision of an experienced engineer. It is at its worst when it has no such supervision; the vibe coding and other low-quality slop we see. A coding agent is like a junior developer or research assistant. When properly supervised, they help projects move along more quickly by letting a senior developer focus on the more difficult, less mundane aspects of the project. But one couldn t expect a junior developer to consistently deliver high-quality code and architecture on their own. Let s put a pin in this idea and look at the story in Debian. LLM use in Debian There is a vote happening in Debian around the use of LLMs. In typical Debian fashion, there are 8 options to choose from, many of them similar. Most of these proposals acknowledge there are different types of tasks done in Debian, but the proposals don t differentiate between them well. Let me do so here. These are some of the LLM-relevant tasks people in Debian perform: I m going to focus my remarks here on packaging upstream software for Debian, since this is by far the most time-consuming developer task project-wide. It matters to our users that we get this right, and packaging quality is one of the things that sets Debian apart from other distros. Packaging things for Debian requires knowledge of some specific tools, such as debhelper, that aren t widely used anywhere else. In most cases, it is fairly rote time-consuming work. In other words, by its design, it requires people with senior-level skills to do grunt work. I can t overstate how massive a burden this grunt work is. I maintain some packages for Go and Rust. By Debian policy, all of those packages dependencies must also exist as Debian packages, and be used to build against. When upstream adopts a newer version of some library, it can unleash cascading dependencies that can take hours to sort out. Worse, the Rust team and the Go team use entirely different ways of managing packages (Go uses one Git repo per package, while Rust has a monorepo with specialized scripts to import Cargo packages and generate Debian ones). On top of that, we can t just modify things like usual; we have to use quilt. And on top of that, I m also a backports maintainer, so all the work (and usually even more) has to be done there also. Now let s pull on that pin from the earlier conversation. This is exactly the kind of scenario that a well-supervised coding LLM is most effective in. I could see a seasoned developer saving hours, maybe even days, by turning over the mundane tasks of managing trees of cascading dependencies over to a coding tool and verifying and directing the process. (Yes, I have been using em-dashes for years; LLMs have copied people like me, not the other way around! This post was not written with any AI assistance.) Actually, this is almost a dream scenario for a coding assistant. The result is time-consuming to formulate but easy to review, which is the opposite of the way these things often go. I can assure you with 100% certainty that humans aren t adding a lot of value in this process. It would be wrong to believe that a human is carefully reading every line of code in dozens of updated or new library packages. The problem set is too big, the time too short, and the code too varied and complex. Coding agents seem to be most effective when there are strong test suites that they can test changes against. Debian builds, especially of modern packages, tend to have this property. Many packages have test suites that are run during build. And, if the package builds in an isolated environment (and especially if its downstream dependencies do also), then there is a decent chance that it s fairly correct. Maybe needing some manual tweaking here and there, but generally a successful build is a reasonable indicator. You can argue that it would make more sense for Debian to just include dependencies in source packages, along with some version information to support security rebuilds, and I d tend to agree with you. But we are where we are. This would be one of the more significant leaps forward in developer productivity, but it complicates things like copyright reviews. Where are LLMs run? What is the environmental impact? Most of the proposals seem to make the assumption that LLMs must always run in some large, hosted datacenter. As I noted in my March article, I have had credible results on even an older GPU running on solar power. That said, it is undeniable that LLMs are fueling a datacenter boom, and this in turn is producing a significant new demand for resources. Most notably for the global scale: electricity, which is sometimes generated using carbon-emitting technologies. Bill McKibben, who has been a leading voice in the fight against climate change since the 1980s, has made some interesting points recently: he s noted that solar power is the fastest kind of generation we can build, and a number of large AI companies are investing heavily in solar, even to the point of fully offsetting new datacenter s needs. On the other hand, he s also noted that some companies are buying inefficient and dirty gas turbines. It is decidedly a mixed bag. The heavy investment in solar can have knock-on positive effects for infrastructure. Obviously, not every picture here is rosy. This analysis doesn t touch on the real land and water use situation, either. On the other hand, if an LLM allows me to do in an hour what I would have done in a day, that s a day of not heating or cooling the work area generally not sustaining a human for the purpose of writing code for Debian. HVAC energy consumption dwarfs my GPU, and I d imagine probably also the slice of LLM energy used. Holistically, I would have to conclude the picture is mixed. It is possible to use LLMs in a pretty green way, and also in a pretty dirty way. Assuming Conditions Never Change A flaw in most of these proposals is they assume that the conditions at this present moment will always hold. In fact, that the conditions at the present moment will not continue is something both AI cheerleaders and AI skeptics agree on. For instance: Ed Zitron has done a ton of research into the financing side of AI, and has concluded that the current model is unsustainable and headed for a significant bubble burst. I m not positioned to personally evaluate those claims, but if that happens, what is the result? Perhaps it is a steeply increasing cost of inference for the frontier models, slower pace of training/evolution for them, etc. In a recent episode of Oxide and Friends, Simon Willison discussed the open weight models that are now available. They have been making remarkable strides in efficiency and capabilities, to the point where $50,000 of hardware can now run high-end open weight models with capabilities that are at least in the same ballpark as the American frontier models. This puts running high-end models locally squarely within reach of universities and small- to medium-sized businesses, with power requirements that can be met with standard commercial solar and wind installations. The lack of nuance in the more restrictive proposals is particularly concerning. Proposal A doesn t allow the use or assitance of LLMs . So it bans my solar-powered GPU. It bans using LLMs to find security issues. It bans all sorts of things that don t seem to be ban-worthy, alongside the things that do. And it codifies it in the very hard-to-change social contract. That proposal, and some like it, seem to imply that all LLM output is bad. I grant you that AI slop is a real and legitimate concern, and many Open Source projects have to deal with it. On the other hand, we have all seen first-hand how the security of the Linux kernel has benefited dramatically from AI analysis. It is certain that black hats are using these tools. If we refuse to use modern security tools, our security will be compromised (and what is the environmental and social impact of THAT?) I find the statement Generative AI is characterized by producing output of a nature that would ordinarily be produced and consumed by humans to be particularly interesting. The same was once said of compilers. The Real Concerns You might think from reading this that I am some AI cheerleader. I m not. I share the ethics of the FLOSS movement, and have for decades. I abhor the power and lack of ethics that many big names in the field are running with at the moment. I ve had to put up Anubis on this blog, for instance. I have personally experienced the effects of AI slop, especially at review time. This is a real problem, though I don t think the more draconian policies are likely to help (the looser you must disclose stand a fighting chance, but I m not sure they would help, either.) Done poorly, AI threatens developer burnout by overwhelming them with poor code and verbose but useless explanations. Done well, AI can help prevent developer burnout by automating tedious and low-value tasks. Shouldn t our goal be that humans submit work to Debian, using tools they prefer, and take responsibility for it? Does it matter if someone uses ed, vim, emacs, or vscode? If they use LSP or just run gcc manually? I d say we benefit from the diversity. Wouldn t we be better off to benefit from the diversity here, and judge work as we always have: on its merits, not what tools were used to create it? Fundamentally, a GR is a long and arduous process. It s not easy to reverse later. Amending the Social Contract is even longer and more arduous (I should know; I may have been the first one to try). The LLM landscape is fast-moving. None of us can really predict where it will be in a year. Will the current market leading companies even still exist? Will it be at all credible to refuse to use AI-assisted security tools? What is the most effective way to deal with AI slop? What level of utility will we be able to achieve with models run locally? Some of these proposals would make sense if drafted in some way short of a GR, which would allow more maneuverability as the landscape changes. Brief analysis of the options Considering the proposals: In favor of nuance I find that black-and-white thinking is almost always something to be avoided. I see it too often. I see it in politics, I see it in our software, I see it in discussions around AI. Are there deeply unethical things happening in AI? Absolutely. Are they doing some impressive things? Also yes. We have accepted this nuance in other areas. For instance, almost all the hardware Debian runs on has closed-source hardware, and has components manufactured or assembled in countries with some of the worst human rights records on the planet. I m not saying this is a great state of affairs. It is something we should speak up about and act upon. But the worse state of affairs would be no Debian because the hardware is impure .

16 August 2026

Bits from Debian: Debian turns 33!

It has now been thirty-three years since the Debian project was announced to the world by Ian Murdock, on August 16, 1993. This anniversary is an opportunity to reaffirm the goals, characteristics, and qualities of the Debian project: it s an association of individuals who have made common cause to create a free operating system. Our distribution is characterized by a commitment to software freedom, as enshrined in the Debian Social Contract and the Debian Free Software Guidelines. It focuses on security and stability. This stability is crucial to Debian position in the free software ecosystem. With our users as our priority, Debian makes special efforts regarding accessibility with Debian-Accessibility and diversity with our Outreach Programs. Debian Day is a great opportunity to get together, whether for a local meetup, or simply to grab a coffee with other members of the Debian community. Check out the Debian Day wiki to see if there is a celebration near you. And if there isn't, maybe you can organize it next year! Today is also an opportunity for you to start or resume your contributions to Debian. For example, you can install the how-can-i-help package and see if there is a bug in any of the software that you use that you can help to fix, contribute small tips on how to install Debian on your machines to our wiki pages, or put a Debian live image in an USB memory and give it to some person near you, who still didn't discover Debian. Thanks to everybody who has contributed to develop our beloved operating system in these 33 years, and Happy birthday Debian!

Vasudev Kamath: Releasing debvulns-exporter and debvulns CLI 0.2.2

I made another minor release with several enhancements: handling non-Debian origin vulnerabilities, improving data caching, and sharing the cache between the debvulns CLI and the exporter. Additionally, there are a few improvements on the dashboard front. Here is a breakdown of what changed.
Handling Vulnerabilities in Non-Debian Origin Packages During the previous release, I noticed that the grafana package which is not in Debian and was installed via an upstream repository was reported as vulnerable with multiple issues. Looking into why this happened, I found that all the CVEs reported in the dashboard were indeed listed on security-tracker.debian.org, but without a fixed version or status description. The logic assumed no fix was available and marked the package as vulnerable on the dashboard.
How Did I Solve This? Google maintains a distributed vulnerability database for open-source projects called osv.dev. I checked the generic vulnerability data for those CVEs on OSV (unbound to any specific distribution) and found that the issues were already fixed in the upstream version I was running. What I needed was a way to differentiate native Debian packages from non-Debian packages, which corresponds to the Origin field in APT metadata.
Pitfall The AI-generated code initially attempted to differentiate package origin using apt_pkg.PackageRecords and its origin field. However, many native Debian packages were incorrectly flagged as non-Debian. On closer inspection, when an upgrade is available for a package, the installed version's origin field can be unset. I had to resolve this by detecting available upgrades and inspecting the candidate version's origin instead, which was implemented in this patch. This solution was proudly crafted by me ;-) (partly because I ran out of API limits and had to wait 6 hours for the next reset).
Caching OSV Data Initially, the AI implemented the exporter to re-download the entire OSV dataset on every run, which was unnecessary. Since vulnerability data does not change rapidly once published, caching it on disk for longer than the standard 24-hour Debian/EPSS cache makes sense. OSV vulnerability data is now cached for 7 days before a refresh is triggered. All cache expiration thresholds remain configurable via CLI flags.
Catch One caveat with this approach: I have not yet verified whether every upstream CVE is tracked on security-tracker.debian.org. In the case of grafana, the entries existed. This feature operates on the assumption that security-tracker.debian.org indexes CVE metadata regardless of whether the package is native to Debian. I plan to re-evaluate this and add fallback handling if that assumption fails.
Unified Cache Directory for CLI and Exporter Another issue was cache segregation: the debvulns CLI utility defaulted to /var/cache/debvulns, while the Prometheus exporter used /var/cache/debvulns-exporter. While harmless when running only one tool, installing both led to duplicated cache storage and redundant network requests. Since the core evaluation logic is identical across both tools, they now share a unified cache directory to eliminate duplicate downloads.
Dashboard Changes During the initial dashboard rollout, my test environment (my laptop alongside Debian 11 and Debian 12 VMs) reported a high aggregated vulnerability count. It was not immediately obvious whether these were distinct vulnerabilities or the same CVEs replicated across all three machines. This mirrors common questions raised during vulnerability reviews:
  • How many unique vulnerabilities are present across the fleet?
  • Which unique packages are affected?
The dashboard has been redesigned to surface unique vulnerability counts alongside affected package lists. The updated dashboard is shown below:
What's Next? A few planned items remain to make debvulns a comprehensive vulnerability reporting toolkit for Debian systems:
  1. Kernel Vulnerability Handling: Currently, installing a patched kernel marks the vulnerability as resolved, even if the system has not rebooted into it. The system remains exposed while the vulnerable kernel is executing in memory. Factoring in running kernel versions is crucial.
  2. Reboot and Service Restart Tracking: Similar to kernel upgrades requiring a reboot, userland library and binary fixes require running services to be restarted. This is typically detected via needrestart. Integrating this behavior directly into debvulns will provide complete visibility in a single dashboard metric.
  3. Debian Packaging: Once the above features are stable, the final step is packaging debvulns for Debian so it can be installed directly from the archive.
Until then, happy hacking.

Benjamin Mako Hill: Sad Story

Picture of a box in a fireplace saying:

Not a screenshot of despair. But only because it s not a screenshot.

15 August 2026

Sven Hoexter: FrOSCon 2026: TLS Talk

Info: German content only, sorry. I was pondering for the past three years if I should give some sort of TLS basics talk at FrOSCon. I finally stepped up this year and gave that talk today, with the title "TLS, mTLS, SNI, ECH, CAA, HTTPS, PKI, Zertifikate und ein bisschen PQC". I was too optimistic with my 50 slides, and had to drop the Post Quantum Cryptography part at the end. Still got positive feedback from Zugschlus and others - thanks a lot for that <3 - and was asked for the slides. It's not a piece of art, but maybe it helps to release the LibreOffice odp file as well, so others can use it as a base for other events or corp internal talks. So here is the froscon-tls-2026.pdf and froscon-tls-2026.odp, both released under the CC BY-NC license. The video is also available at media.ccc.de if you want to watch it. Thanks to everyone who made FrOSCon happen for the 21th time!

Russell Coker: Hacked by Chinafans

What Happened On 2026/08/10 at 2:11 am Australian eastern standard time (2026/08/09 16:11 UTC) someone created a post titled Hacked by Chinafans on my documents blog [1]. The person in question created an account named 67965e42a3c3 on that site with the email address 67965e42a3c3@google.com associated with it (I tried emailing that address and it bounced). At 04:28:41am Australian eastern standard time (18:28 UTC) I was sent an email titled Have you been hacked by a reader of my blogs who subscribed to the RSS feed of my documents blog (a blog that I never expected anyone to read by RSS). Along the lines of the wisdom of crowds should we have the unexpected observation and problem reporting of crowds ? I appreciate the notification, I might not have noticed until the next time I watched an unusually good movie otherwise. The account in question was apparently created on 2026-07-21 at 16:43:47 (presumably UTC) even though at the time I believe creating accounts was not permitted. As an aside the timestamp of account creation is stored in the user_registered column of the wp_users table in the database, there doesn t appear to be a way to access this in a standard WordPress installation other than doing a SQL query. 
2026-07-24 15:43:17 status triggers-pending wordpress:all 7.0+dfsg1-1
2026-07-24 15:43:19 upgrade wordpress:all 7.0+dfsg1-1 7.0.2+dfsg1-1
Above are the relevant sections of my dpkg log showing the WordPress versions in use. I was running version 7.0+dfsg1-1 at the time the account was apparently created. I am confident in the accuracy of the dpkg logs and believe that they did not compromise the OS, I am not sure whether they ran hostile SQL code to change fields in the MySQL database so had to consider the possibility that the account creation time could have been set to a deliberately misleading value. I checked backups of the MySQL database stored off-site and found that the account in question was not in the 2026-07-21 backup (which was done before 16:43) but in the 2026-07-22 backup. The WordPress release history [2] has version 7.0.1 released on 2026-07-09 and version 7.0.2 released on 2026-07-17. So presumably the attacker diffed the code on those releases, found an exploitable bug, and used it to create an account on my blog with admin privs. Then they waited a few weeks to see if I would notice and published a blog post when I didn t notice. WordPress Deficiencies
  1. WordPress doesn t seem to store the version it s running at the time of operations. So anyone who doesn t have a suitable external log of versions deployed (such as the dpkg.log file for a Debian managed installation) won t know for sure which version was running. It supports automatic updates but you can t be sure that they happened soon after the release.
  2. There is no log of IP addresses used for operations. There are apparently some 3rd party modules to log such things and web pages documenting how to modify the PHP to add it but nothing in the standard distribution.
  3. Software should have a standard distribution with some support for logging of security relevant data. The typical situation is that people don t plan for logging such things until after they have been attacked so the data should be recorded without users going out of their way to log it.
  4. A log of security relevant data should be stored in a database table with only insert access (no update, delete, or drop).
  5. Ideally a CMS would support different database accounts for different purposes. Someone from an internal network or VPN could talk to an instance of the web server which has a database username and password giving full access. Everyone from outside the trusted range gets an instance of the web server with database access only allowing to read the posts and appearance configuration and to enter comments. If the database didn t allow the account used for public access to create new admin users or create posts then it would be a lot harder for attackers.
  6. Ideally for everything that stores user account data there would be an easy way of getting a list of users in a plain text format to allow running diff. The design of WordPress has two tables, one for users and one for encoded metadata about users of which one will be the access level. The following SQL command will give a list of all users that aren t subscribers (everyone above the minimum level of access which is typical for new users) along with their encoded password and access level. This could be used in a monitoring system to alert about new privileged users. The TABLE_PREFIX variable is for the prefix for WordPress tables, which is wp_ by default but can be any legal value. 
    select $TABLE_PREFIXusers.user_login, $TABLE_PREFIXusers.user_pass, $TABLE_PREFIXusermeta.meta_value from  $TABLE_PREFIXusers join $TABLE_PREFIXusermeta on $TABLE_PREFIXusers.id = $TABLE_PREFIXusermeta.user_id and meta_key='$TABLE_PREFIXcapabilities' and meta_value != 'a:1: s:10:"subscriber";b:1; ';
What Next? The blog post they created had a couple of links to Telegram which could presumably be used to contact them. If anyone involved in computer security wants a copy of the original post to do so then they can contact me by any of the usual methods. I am interested in communication with the attacker if they wish, Telegram is not a service I use but I presume that anyone capable of doing this sort of attack is also capable of finding other ways of contacting me. I have idly considered changing to a static site generator, here is a good list of static site generators [3]. I have also idly considered other platforms for blogging such as Lemmy. I don t know if Lemmy is better than WordPress for security and updates, but there are plenty of free instances running where it wouldn t be an issue I have to work on. 15 Years It s been 15 years since my blog server was cracked by a trojaned ssh client [4]. At least this time it was only one service that was compromised.
Related posts:
  1. blogging software Previously I asked for advice about running an Intranet blog,...
  2. WordPress and Thumbnails I have just had a lot of trouble with Thumbnails...
  3. WordPress Maintainability For a while I ve been maintaining my own WordPress packages....

Russell Coker: AMD Video Drivers, LLMs, and Debian Kernels

The AMD GPU Problem For a while I ve been having issues with AMD GPUs, video locking up periodically. I blogged about this late last year but I first had noticeable problems early last year [1]. The problems hadn t only concerned my workstation but also my home server which is also used as a workstation. I ve recently upgraded my machines to Debian/Testing, my home server has been generally OK but my workstation has been crashing a lot. Every second day when on kernel 7.1.6 and then when on 7.1.7 it crashed at least once a day. The AMD GPUs I have are [AMD/ATI] Baffin [Radeon RX 460/560D / Pro 450/455/460/555/555X/560/560X] (rev e5) in my main desktop workstation, [AMD/ATI] Lexa [Radeon 540X/550X/630 / RX 640 / E9171 MCM] (rev c1) in my build server, and [AMD/ATI] Baffin [Radeon RX 460/560D / Pro 450/455/460/555/555X/560/560X] (rev cf) in my home server. They aren t new GPUs, but also aren t really old and they all support 4K and better resolution. Chat GPT Was Useful When I googled the errors I was seeing I found nothing useful. On the suggestion of a friend I tried asking ChatGPT. Generally I don t recommend asking LLMs about such things, but it can be a last resort as long as you know what you are doing. ChatGPT asked me to run a number of commands to get information for it to make more informed decisions. I know that the output of lspci and similar commands isn t a risk, but a novice could be tricked into running commands that expose sensitive data. ChatGPT did give me some useful information, not a solution but an indication that the problem was due to driver bugs. Upgrading to Experimental Debian/Experimental is for packages that are expected to have problems and generally aren t recommended even for the people who usually use Debian/Unstable. It s commonly used for packages that are needed to develop other packages, EG new libraries that aren t fully usable but which are needed to package newer versions of applications. I upgraded my workstation to the Debian/Experimental kernel 7.2~rc7-1~exp1 after having tried every other convenient option. Generally I wouldn t recommend that anyone run an Experimental kernel without a really good reason, but crashing more than once a day is a fairly good reason. That kernel has now given me over 4 days of uptime on a system that previously wouldn t last a day. I installed it on my dual-socket build server that has an old AMD GPU in it for test purposes and that also hasn t crashed since. I installed it on my ML test machine which has an Intel B580 Battlemage GPU with 16G of VRAM and was repeatedly getting a kernel panic related to the GPU a few seconds after boot and now it also works correctly. It seems that the 7.1.x kernels have bugs in the AMD video drivers and in some part of the code that affects Intel video drivers and that the bugs in question are fixed in the tree that will become 7.2. I would not recommend anyone who has a 7.1.x kernel working fine for them try 7.2 RC kernels at this time, but anyone who has GPU related problems (particularly Intel and AMD GPUs) should definitely test it out. I also don t recommend upgrading any system with an AMD GPU to Debian/Testing or Debian/Unstable at this time unless you are also prepared to install an Experimental kernel if it becomes necessary. There are a several kernel log dumps related to this after the break (which won t be in RSS feeds). This is mainly for Google so that other people who have such issues can get more useful results out of Google searches than I got. Future Support Options Separate from the issue of whether commercial LLMs like ChatGPT can be useful for solving technical problems there is the issue of whether they are desirable. I think that we really don t want people solving problems in FOSS systems with closed-source LLMs. This leads to loss of privacy, loss of the control users deserve to have over their own systems, and an implied promotion of non-fee software. I think that the ideal would be to have a cross distribution effort to generate training data for a support LLM system which can then be further trained by each distribution for a greater emphasis on distribution specific issues. Errors on AMD GPUs 
2026-08-09T23:03:06.004792+10:00 xev kernel: amdgpu 0000:02:00.0: GPU fault detected: 147 0x00024802
2026-08-09T23:03:06.004792+10:00 xev kernel: amdgpu 0000:02:00.0: Process kscreenlocker_g pid 42037 thread kscreenloc:cs0 pid 42044
2026-08-09T23:03:06.004793+10:00 xev kernel: amdgpu 0000:02:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x00000800
2026-08-09T23:03:06.004794+10:00 xev kernel: amdgpu 0000:02:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0F048002
2026-08-09T23:03:06.004795+10:00 xev kernel: amdgpu 0000:02:00.0: VM fault (0x02, vmid 7, pasid 130) at page 2048, write from 'TC0' (0x54433000) (72)
2026-08-09T23:03:06.008762+10:00 xev kernel: amdgpu 0000:02:00.0: GPU fault detected: 147 0x00004802
2026-08-09T23:03:06.008768+10:00 xev kernel: amdgpu 0000:02:00.0: Process kscreenlocker_g pid 42037 thread kscreenloc:cs0 pid 42044
2026-08-04T01:13:37.505839+10:00 xev kernel: ------------[ cut here ]------------ 
2026-08-04T01:13:37.505859+10:00 xev kernel: amdgpu 0000:02:00.0: [drm] drm_WARN_ON_ONCE(cur_vblank != vblank->last) 
2026-08-04T01:13:37.505862+10:00 xev kernel: WARNING: CPU: 6 PID: 210534 at drivers/gpu/drm/drm_vblank.c:362 drm_update_vblank_count+0x2f1/0x3c0 [drm] 
2026-08-04T01:13:37.505866+10:00 xev kernel: snd_intel_dspcfg wmi_bmof rc_core snd_intel_sdw_acpi drm_ttm_helper uas realtek snd_usbmidi_lib snd_hda_codec ttm mdio_devres snd_hda_core snd_seq_midi drm_kms_helper usb_storage mc snd_hwdep libphy snd_seq_midi_event intel_uncore snd_pcm_oss i2c_algo_bit serio_raw snd_rawmidi pcspkr snd_mixer_oss i2c_i801 video snd_seq snd_pcm i2c_smbus lpc_ich snd_seq_device mei_me e1000e snd_timer mei snd tpm_infineon soundcore joydev bnx2 wmi button nfsd auth_rpcgss nfs_acl lockd grace sunrpc coretemp br_netfilter bridge stp llc sg ghash_clmulni_intel loop msr i2c_dev drm efi_pstore configfs nfnetlink ip_tables x_tables autofs4 btrfs blake2b_generic dm_crypt dm_mod raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx libcrc32c xor raid6_pq raid1 raid0 md_mod ext4 crc16 mbcache jbd2 crc32c_generic virtio_blk evdev hid_generic usbhid hid sd_mod xhci_pci xhci_hcd ahci ehci_pci ehci_hcd libahci crc32c_intel libata usbcore aesni_intel nvme psmouse scsi_mod gf128mul crypto_simd nvme_core cryptd 
2026-08-04T01:13:37.505879+10:00 xev kernel: nvme_auth scsi_common usb_common efivarfs 
2026-08-04T01:13:37.505880+10:00 xev kernel: CPU: 6 UID: 1008 PID: 210534 Comm: sshd-session Tainted: G D 6.12.88+deb13-amd64 #1 Debian 6.12.88-1 
2026-08-04T01:13:37.505881+10:00 xev kernel: Tainted: [D]=DIE 
2026-08-04T01:13:37.505883+10:00 xev kernel: Hardware name: Hewlett-Packard HP Z640 Workstation/212A, BIOS M60 v02.61 03/23/2023 
2026-08-04T01:13:37.505884+10:00 xev kernel: RIP: 0010:drm_update_vblank_count+0x2f1/0x3c0 [drm] 
2026-08-04T01:13:37.505885+10:00 xev kernel: Code: 48 8b 5f 50 48 85 db 75 03 48 8b 1f e8 68 eb 2b cf 48 c7 c1 70 3e cb c0 48 89 da 48 c7 c7 f9 6f cb c0 48 89 c6 e8 af d7 a6 ce <0f> 0b e9 4b fe ff ff 48 8b 4c 24 18 e9 31 fe ff ff 31 f6 48 85 db 
2026-08-04T01:13:37.505887+10:00 xev kernel: RSP: 0000:ffffd3cc8681fca0 EFLAGS: 00010082 
2026-08-04T01:13:37.505888+10:00 xev kernel: RAX: 0000000000000000 RBX: ffff8c6b42b13710 RCX: 0000000000000027 
2026-08-04T01:13:37.505889+10:00 xev kernel: RDX: ffff8c89ef521788 RSI: 0000000000000001 RDI: ffff8c89ef521780 
2026-08-04T01:13:37.505890+10:00 xev kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: ffffd3cc8681fb20 
2026-08-04T01:13:37.505891+10:00 xev kernel: R10: ffff8c8a6fef3628 R11: 0000000000000003 R12: 0000000000000000 
2026-08-04T01:13:37.505892+10:00 xev kernel: R13: ffff8c6c07853828 R14: 0000000000000003 R15: 0000000000000000 
2026-08-04T01:13:37.505893+10:00 xev kernel: FS: 00007ffaf2fd5880(0000) GS:ffff8c89ef500000(0000) knlGS:0000000000000000 
2026-08-04T01:13:37.505895+10:00 xev kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
2026-08-04T01:13:37.505896+10:00 xev kernel: CR2: 00007fb1718c8000 CR3: 000000074521a004 CR4: 00000000003706f0 
2026-08-04T01:13:37.505897+10:00 xev kernel: Call Trace: 
2026-08-04T01:13:37.505898+10:00 xev kernel:  
2026-08-04T01:13:37.505899+10:00 xev kernel: drm_crtc_accurate_vblank_count+0x41/0xc0 [drm] 
2026-08-04T01:13:37.505900+10:00 xev kernel: dm_pflip_high_irq+0x155/0x330 [amdgpu] 
2026-08-04T01:13:37.505901+10:00 xev kernel: amdgpu_dm_irq_handler+0x85/0x1f0 [amdgpu] 
2026-08-04T01:13:37.505902+10:00 xev kernel: amdgpu_irq_dispatch+0xd2/0x230 [amdgpu] 
2026-08-04T01:13:37.505903+10:00 xev kernel: amdgpu_ih_process+0x84/0x100 [amdgpu] 
2026-08-04T01:13:37.505904+10:00 xev kernel: amdgpu_irq_handler+0x23/0x60 [amdgpu] 
2026-08-04T01:13:37.505905+10:00 xev kernel: __handle_irq_event_percpu+0x4a/0x190
2026-08-04T01:13:37.505907+10:00 xev kernel: handle_irq_event+0x38/0x80 
2026-08-04T01:13:37.505908+10:00 xev kernel: handle_edge_irq+0x8b/0x230 
2026-08-04T01:13:37.505909+10:00 xev kernel: __common_interrupt+0x45/0xe0 
2026-08-04T01:13:37.505910+10:00 xev kernel: common_interrupt+0x42/0xa0 
2026-08-04T01:13:37.505911+10:00 xev kernel: asm_common_interrupt+0x26/0x40 
2026-08-04T01:13:37.505912+10:00 xev kernel: RIP: 0033:0x7ffaf3c5fd7b 
2026-08-04T01:13:37.505913+10:00 xev kernel: Code: 70 c7 00 66 0f 6e f8 c1 ef 02 66 0f 70 f7 e0 83 c7 01 66 0f ef ff 66 0f fa f2 0f 1f 44 00 00 f3 0f 7e 01 66 0f 6f ce 83 c6 01 <48> 83 e9 08 f2 0f 70 c0 1b 66 0f 6f e0 66 0f 6f e8 66 41 0f f9 c0 
2026-08-04T01:13:37.505915+10:00 xev kernel: RSP: 002b:00007fff86a5e0e0 EFLAGS: 00000202 
2026-08-04T01:13:37.505916+10:00 xev kernel: RAX: 0000000000008000 RBX: 0000562614a04050 RCX: 0000562614982ed8 
2026-08-04T01:13:37.505946+10:00 xev kernel: RDX: 0000000000007fe2 RSI: 0000000000000fad RDI: 0000000000002000 
2026-08-04T01:13:37.505948+10:00 xev kernel: RBP: 0000000000000000 R08: 000056261498ac40 R09: 0000000000008000 
2026-08-04T01:13:37.505949+10:00 xev kernel: R10: 0000000000000066 R11: 0000000000007fe1 R12: 0000000000007efa
2026-08-04T01:13:37.505950+10:00 xev kernel: R13: 0000000000008000 R14: 0000000000008000 R15: 000000000000ffe0 
2026-08-04T01:13:37.505951+10:00 xev kernel:  
2026-08-04T01:13:37.505953+10:00 xev kernel: ---[ end trace 0000000000000000 ]--- 
2026-08-04T01:55:40.844110+10:00 xev kernel: pcieport 0000:00:03.3: AER: Multiple Correctable error message received from 0000:00:03.3 
2026-08-04T01:55:40.844130+10:00 xev kernel: pcieport 0000:00:03.3: PCIe Bus Error: severity=Correctable, type=Data Link Layer, (Receiver ID) 
2026-08-04T01:55:40.844132+10:00 xev kernel: pcieport 0000:00:03.3: device [8086:6f0b] error status/mask=00000040/00002000 
2026-08-04T01:55:40.844134+10:00 xev kernel: pcieport 0000:00:03.3: [ 6] BadTLP
2026-08-11T09:33:33.473855+10:00 xev kernel: amdgpu 0000:02:00.0: GPU fault detected: 147 0x00024802
2026-08-11T09:33:33.473871+10:00 xev kernel: amdgpu 0000:02:00.0: Process kscreenlocker_g pid 150905 thread kscreenloc:cs0 pid 150912
2026-08-11T09:33:33.473871+10:00 xev kernel: amdgpu 0000:02:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x00000800
2026-08-11T09:33:33.473873+10:00 xev kernel: amdgpu 0000:02:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0F048002
2026-08-11T09:33:33.473873+10:00 xev kernel: amdgpu 0000:02:00.0: VM fault (0x02, vmid 7, pasid 63) at page 2048, write from 'TC0' (0x54433000) (72)
2026-08-11T09:33:33.473874+10:00 xev kernel: amdgpu 0000:02:00.0: GPU fault detected: 147 0x00004802
2026-08-11T09:33:33.473874+10:00 xev kernel: amdgpu 0000:02:00.0: Process kscreenlocker_g pid 150905 thread kscreenloc:cs0 pid 150912
2026-08-11T09:33:33.473875+10:00 xev kernel: amdgpu 0000:02:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x00000800
2026-08-11T09:33:33.473876+10:00 xev kernel: amdgpu 0000:02:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0E048002
2026-08-11T09:33:33.473876+10:00 xev kernel: amdgpu 0000:02:00.0: VM fault (0x02, vmid 7, pasid 63) at page 2048, read from 'TC0' (0x54433000) (72)
2026-08-11T09:33:35.481863+10:00 xev kernel: amdgpu 0000:02:00.0: Dumping IP State
2026-08-11T09:33:35.481875+10:00 xev kernel: amdgpu 0000:02:00.0: Dumping IP State Completed
2026-08-11T09:33:35.481875+10:00 xev kernel: amdgpu 0000:02:00.0: [drm] AMDGPU device coredump file has been created
2026-08-11T09:33:35.481876+10:00 xev kernel: amdgpu 0000:02:00.0: [drm] Check your /sys/class/drm/card0/device/devcoredump/data
2026-08-11T09:33:35.481877+10:00 xev kernel: amdgpu 0000:02:00.0: GPU fault detected: 146 0x0110040c
2026-08-11T09:33:35.481877+10:00 xev kernel: amdgpu 0000:02:00.0: Process kscreenlocker_g pid 150905 thread kscreenloc:cs0 pid 150912
2026-08-11T09:33:35.481878+10:00 xev kernel: amdgpu 0000:02:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x00000022
2026-08-11T09:33:35.481879+10:00 xev kernel: amdgpu 0000:02:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0E00400C
2026-08-11T09:33:35.481879+10:00 xev kernel: amdgpu 0000:02:00.0: VM fault (0x0c, vmid 7, pasid 63) at page 34, read from 'TC3' (0x54433300) (4)
2026-08-11T09:33:35.489845+10:00 xev kernel: amdgpu 0000:02:00.0: ring gfx timeout, signaled seq=5123619, emitted seq=5123621
2026-08-11T09:33:35.489853+10:00 xev kernel: amdgpu 0000:02:00.0: Process kscreenlocker_g pid 150905 thread kscreenloc:cs0 pid 150912
2026-08-11T09:33:35.489854+10:00 xev kernel: amdgpu 0000:02:00.0: GPU reset begin!. Source: 1
2026-08-11T09:33:35.493839+10:00 xev kernel: amdgpu 0000:02:00.0: [drm] ERROR Failed to initialize parser -125!
2026-08-11T09:33:35.737848+10:00 xev kernel: amdgpu: cp is busy, skip halt cp
2026-08-11T09:33:35.897842+10:00 xev kernel: amdgpu: rlc is busy, skip halt rlc
2026-08-11T09:33:35.897852+10:00 xev kernel: amdgpu 0000:02:00.0: BACO reset
2026-08-11T09:33:36.485849+10:00 xev kernel: amdgpu 0000:02:00.0: GPU reset succeeded, trying to resume
2026-08-11T09:33:36.485859+10:00 xev kernel: amdgpu 0000:02:00.0: [drm] PCIE GART of 256M enabled (table at 0x000000F402000000).
2026-08-11T09:33:36.485860+10:00 xev kernel: amdgpu 0000:02:00.0: VRAM is lost due to GPU reset!
Errors on Battlemage 
Aug 11 17:01:47 ami kernel: ------------[ cut here ]------------
Aug 11 17:01:47 ami kernel: xe 0000:23:00.0: [drm] DMC 1 mmio[0]/0x5f074 incorrect (expected 0x96fc0, current 0x0)
Aug 11 17:01:47 ami kernel: WARNING: drivers/gpu/drm/i915/display/intel_dmc.c:696 at assert_dmc_loaded+0x275/0x430 [xe], CPU#0: kworker/0:3/215
Aug 11 17:01:47 ami kernel: Modules linked in: intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common xe(+) skx_edac snd_h>
Aug 11 17:01:47 ami kernel:  msr i2c_dev configfs efi_pstore efivarfs autofs4 btrfs libblake2b raid6_pq xor mpt3sas raid_class scsi_transport_sas megarai>
Aug 11 17:01:47 ami kernel: CPU: 0 UID: 0 PID: 215 Comm: kworker/0:3 Not tainted 7.1.7+deb14-amd64 #1 PREEMPT(lazy)  Debian 7.1.7-1 
Aug 11 17:01:47 ami kernel: Hardware name: HP HP Z4 G4 Workstation/81C5, BIOS P61 v03.00 04/15/2026
Aug 11 17:01:47 ami kernel: Workqueue: sync_wq local_pci_probe_callback
Aug 11 17:01:47 ami kernel: RIP: 0010:assert_dmc_loaded+0x291/0x430 [xe]
Aug 11 17:01:47 ami kernel: Code: 24 10 e8 f2 e5 a3 ce 48 8d 3d bb 85 0d 00 8b 54 24 0c 45 89 e9 45 89 e0 48 89 c6 52 8b 4c 24 2c 51 8b 4c 24 30 48 8b 54>
Aug 11 17:01:47 ami kernel: RSP: 0018:ffffd27ac0b87b80 EFLAGS: 00010282
Aug 11 17:01:47 ami kernel: RAX: ffffffffc1743dfd RBX: ffff8c5b80e54000 RCX: 0000000000000001
Aug 11 17:01:47 ami kernel: RDX: ffff8c5b81df5a10 RSI: ffffffffc1743dfd RDI: ffffffffc1605860
Aug 11 17:01:47 ami kernel: RBP: ffff8c5b86955000 R08: 0000000000000000 R09: 000000000005f074
Aug 11 17:01:47 ami kernel: R10: 0000000000000000 R11: 0000000000091050 R12: 0000000000000000
Aug 11 17:01:47 ami kernel: R13: 000000000005f074 R14: 0000000000000001 R15: 0000000000000000
Aug 11 17:01:47 ami kernel: FS:  0000000000000000(0000) GS:ffff8c673e172000(0000) knlGS:0000000000000000
Aug 11 17:01:47 ami kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug 11 17:01:47 ami kernel: CR2: 00007ffed1fdcd00 CR3: 0000000ae942a003 CR4: 00000000003706f0
Aug 11 17:01:47 ami kernel: Call Trace:
Aug 11 17:01:47 ami kernel:  
Aug 11 17:01:47 ami kernel:  intel_dmc_enable_pipe+0xe4/0x290 [xe]
Aug 11 17:01:47 ami kernel:  ? drm_crtc_vblank_reset+0x4d/0x120 [drm]
Aug 11 17:01:47 ami kernel:  intel_modeset_setup_hw_state+0xb50/0x1e10 [xe]
Aug 11 17:01:47 ami kernel:  ? intel_display_driver_probe_nogem+0x138/0x1a0 [xe]
Aug 11 17:01:47 ami kernel:  intel_display_driver_probe_nogem+0x138/0x1a0 [xe]
Aug 11 17:01:47 ami kernel:  xe_display_init_early+0xb2/0x140 [xe]
Aug 11 17:01:47 ami kernel:  xe_device_probe+0x3c8/0xb50 [xe]
Aug 11 17:01:47 ami kernel:  ? xe_pm_init_early+0x152/0x160 [xe]
Aug 11 17:01:47 ami kernel:  xe_pci_probe+0xc26/0x1150 [xe]
Aug 11 17:01:47 ami kernel:  local_pci_probe+0x3e/0x90
Aug 11 17:01:47 ami kernel:  local_pci_probe_callback+0x16/0x20
Aug 11 17:01:47 ami kernel:  process_one_work+0x19d/0x3a0
Aug 11 17:01:47 ami kernel:  worker_thread+0x1af/0x320
Aug 11 17:01:47 ami kernel:  ? __pfx_worker_thread+0x10/0x10
Aug 11 17:01:47 ami kernel:  kthread+0xe3/0x120
Aug 11 17:01:47 ami kernel:  ? __pfx_kthread+0x10/0x10
Aug 11 17:01:47 ami kernel:  ret_from_fork+0x2b2/0x340
Aug 11 17:01:47 ami kernel:  ? __pfx_kthread+0x10/0x10
Aug 11 17:01:47 ami kernel:  ret_from_fork_asm+0x1a/0x30
Aug 11 17:01:47 ami kernel:  
Aug 11 17:01:47 ami kernel: ---[ end trace 0000000000000000 ]---
Related posts:
  1. AMD Video Driver Issues I have had some graphics hangs on my HP z640...
  2. Kernel issues with Debian Xen and CentOS Kernels Last time I tried using a Debian 64bit Xen kernel...
  3. PCIe Problems HP z840 Dead Slot I just had an issue with...

14 August 2026

Reproducible Builds (diffoscope): diffoscope 328 released

The diffoscope maintainers are pleased to announce the release of diffoscope version 328. This version includes the following changes: 
[ Chris Lamb ]
* Don't require python3-guestfs in the autopkgtests on 32-bit architectures.
  (Closes: #1144372)
[ Jochen Sprickerhof ]
* Use the XML comparators for SVG vector image files. (Closes: #1144242)
You find out more by visiting the project homepage.

13 August 2026

Gunnar Wolf: File recovery in process...

Ohai, I have some pending, encrypted mails to answer. And some of my answers for the next few days (particularly to what pertains to the current in-discussion vote on LLM usage in Debian) will be unsigned, even though I d like otherwise. My desktop system at work is showing some data corruption, and I m slowly backing up my data. Fortunately, it seems I haven t lost any data, but still, given I m letting rsync run until it starts spewing I/O errors, then power down and let the machine cool a bit, and start again it is a potentially long process. And yes, this makes me somewhat angry. Why angry? Because I m working on a brand-new computer (well, have used it for slightly over six months), custom-built to specs requested by my workplace. Specs that I don t really need, this machine is an utter luxury (i.e. an AMD Ryzen 9 9950X processor with 16 real cores / 32 threads; 128GB RAM in this day and age of RAM shortage, quite recent 32GB GPU, and lots of shiny lights seen in its huge fishbowl cabinet, liquid-based cooling ). The specs came not from me, but from people who had no idea what we would use them for. And yes, I expect the little fortune spent on this machine to be good for my use for probably a decade, as my previous computer was, but the amount paid was exorbitant. But still, what I learned recently is that the 4TB nVME SSD it has (a T-Force TM8FFJX34T) is a very cheap brand, bought because it was close to half the price of other offerings similar in capacity. According to smartctl s output, th SSD operates with a Warning Comp. Temp. Threshold: 90 Celsius and Critical Comp. Temp. Threshold: 110 Celsius, which sounds sensible, even too high for my standards (my last two laptops have been fanless yes, an ARM system is very different from a high-end gaming machine). I m right now typing from my laptop, which shows 78 C and 82 C for warning/critical thresholds. And as expected, under heavy sustained reads (backing up to my NFS server), the desktop s smartctl shows Temperature: 83 Celsius and, further down, Temperature Sensor 1: 107 Celsius and Temperature Sensor 2: 82 Celsius (don t know which of these would make the threshold jump). The SSD has sustained Media and Data Integrity Errors: 20 and Warning Comp. Temperature Time: 21 (although Critical Comp. Temperature Time: 0). At least one of my colleagues have shrugged and installed a SATA SSD, laying the huge nVME basically to waste. Anyway I also learned I am not the first, but the fourth person to notice this kind of issues in this system (out of ten similar purchased systems AIUI). It is completely unacceptable, and I ll be pushing our Institute s authorities to demand the provider to provide either good component quality for this very expensive system that has many luxury items, or to fix the system s build in a way the nVME does not heat as much as it currently does. Anyway, sigh, I only wanted to say, please excuse me for not using my cryptographic keys for a couple of days PS- I m also currently not connected to IRC and Jabber (and some similar technologies), as my bouncer runs from my usual workstation.

Jonathan Dowland: DIY skate punch-out

The punch set-up
the punched boot
Since I wrote about my fly30 ice skates, I'd continued to battle pain around the navicular bone in my feet. The action that seems to have finally fixed it was to perform a "punch out": a very localized remoulding of the area of the boot that presses against the sore area. I basically followed the process from this helpful YouTube video. I narrowed down the exact spot by borrowing some lipstick and transferring it from my navicular bone to the boot lining, then making that more permanent with a sharpie. My punch was a spare part from a radiator valve which I packed with US cents (I couldn't fit any UK coins in). For the receiving-end, I tried another part from the radiator valve but I think it wasn't sufficiently larger than the punch to work well, so I swapped that out for a spoon.
take 2 take 2
I didn't have a temperature sensor I could use and I used a heat gun rather than a hairdryer, so I YOLO'd it a little. Some of the wrap on one of my boots is now distorted from where I didn't move the heat gun enough. It only took a minute or two to get the boot hot enough to be flexible. I set a 15 minute timer once the clamp was in place. I've only skated one session since I did this but the pain seems to have gone! It's remarkably freeing to be skating without constantly trying to manage pain. Now I can focus on technique.

12 August 2026

Reproducible Builds: Reproducible Builds summit 2026 to take place in Gothenburg

This event is happening soon see below for registration instructions!

We are extremely pleased to announce the upcoming Reproducible Builds summit, which will take place from September 22nd 24th 2026 in the city of Gothenburg, Sweden. This year, we are thrilled to host the tenth edition of this exciting event, following the success of previous summits in various iconic locations around the world, including Vienna (2025), Hamburg (2023 2024), Venice (2022), Marrakesh (2019), Paris (2018), Berlin (2017), Berlin (2016) and Athens (2015). If you re excited about joining us this year, please make sure to read the event page which has more details about the event and location. As in previous years, we will be sending invitations to all those who attended our previous summit events or expressed interest to do so. However, even if you do not receive a personal invitation, please do email the organizers and we will find a way to accommodate you.

About the event The Reproducible Builds Summit is a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort. During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field. Our aim is to create an inclusive space that fosters collaboration, innovation and problem-solving.

Schedule Although the exact content of the meeting will be shaped by the participants, the main goals will include:
  • Update & exchange about the status of reproducible builds in various projects.
  • Improve collaboration both between and inside projects.
  • Expand the scope and reach of reproducible builds to more projects.
  • Work together and hack on solutions.
  • Establish space for more strategic and long-term thinking than is possible in virtual channels.
  • Brainstorm designs on tools enabling users to get the most benefits from reproducible builds.
  • Discuss how reproducible builds will be usable and meaningful to users and developers alike.
Logs and minutes will be published after the meeting.

Location & date

Registration instructions Please reach out if you d like to participate in hopefully interesting, inspiring and intense technical sessions about reproducible builds and beyond! We look forward to what we anticipate to be yet another extraordinary event!

11 August 2026

Colin Watson: Free software activity in July 2026

About 95% of my Debian contributions this month were sponsored by Freexian. You can also support my work directly via Liberapay or GitHub Sponsors. OpenSSH Now that Ubuntu 26.04 LTS has been released, I ve been getting back to the GSS-API key exchange package split in our OpenSSH packaging. Once I started testing my draft openssh-gssapi source package, I realized that I needed to make some changes in the main openssh source package first in order to support it. The dependency from openssh-server to openssh-client was awkward, as was the (related) fact that openssh-client contained shared documentation for other OpenSSH binary packages. After some thought, I created a new openssh-common binary package, moved shared documentation and the ssh-keygen program to that, and dropped dependencies on openssh-client which were no longer necessary (fixing #699473 and #1070098 in the process). This caused a couple of regressions (#1141420 and #1141550) that I had to fix, and more subtly it also caused a number of autopkgtest regressions in other packages because openssh-client is no longer in base images as a result of a dependency from openssh-server. I believe I have fixes for all of these either pending review or merged (one of which I did in August rather than July): I upgraded from 10.3p1 to 10.4p1, and in the process contributed a GSS-API option handling fix upstream. I made openssh-ssh1 s package description more accurately describe the package, thanks to suggestions from Matthias Lang. Installer team With support from a Freexian customer, I reviewed, tested, edited, and merged a patch to add VLAN support. I described the details of what I did in a comment. This has been vaguely on my to-do list since, er, about 2014, so it was very satisfying to get it sorted out. Python packaging New upstream versions: Other build/test failures: I fixed some other bugs: I adopted transaction for the Python team. I attended the Python BoF at DebConf remotely, although a badly-timed fibre outage in the village I live in really didn t help. Code reviews Other bits and pieces Dan Poltawski pointed out in a Fediverse post that the project history didn t list Sruthi as the current DPL. I fixed that, although it doesn t look as though the fix is in the published version yet. I upgraded yubihsm-shell to 2.8.0.

Freexian Collaborators: Debian Contributions: DebConf 26 organization, d-i VLAN support and more! (by Anupa Ann Joseph)

Debian Contributions: 2026-07 Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

DebConf 26 organization, by Lucas Kanashiro, Santiago Ruano Rinc n, Stefano Rivera and Antonio Terceiro The 27th Annual Debian Conference was held in Santa Fe, Argentina, and several Freexian fellows were quite busy by being involved in the organization team.
  • Santiago continued helping with duties related to the local team, e.g. preparing or proof-reading some announcements, reviewing the proposed food and the menus for the different diet required.
  • During the conference, Kanashiro and Santiago also carried over tasks related to the content of the conference, including updating the schedule as it became necessary during the event.
  • Stefano worked within the core video team, setting up equipment in talk rooms and coordinating the live video streaming. Stefano also supported the front desk and local organisers as a website developer and conference book-keeper.
  • Antonio kept working on website maintenance, specially in support of the content team. During DebConf he also ran a hands-on workshop to help interested contributors get started with developing the DebConf websites.

d-i VLAN support, by Colin Watson In environments that use IEEE 802.1Q VLANs, some hosts (such as routers attached to trunk ports) may need to apply VLAN tags themselves rather than relying on switches to do so. There has been a long-running request to add support for these to the Debian installer with a proposed patch set put together by several people over the years, and a Freexian customer asked us to help get this over the line. Colin reviewed the latest version of the patch set, applied a number of corrections, added Netplan support, spent some time testing a variety of possible paths through the installer, and landed this. There s also now documentation for this in the next version of the installation guide.

Miscellaneous contributions
  • Carles wrote documentation for installing Mailman3 and migrating from Mailman2. Added it into Mailman3 upstream documentation.
  • Carles, using po-debconf-manager: reviewed 2 packages, submitted 2 packages
  • Carles organized Catalan translation update. Created a Debian Wiki page to have an overview of the work / coordination during next months. Reviewed and submitted some pages.
  • Carles improved the documentation for building the debian.org Web in MR 1154 and MR 1557. Fixed debian-reference documentation. Added sections on Mutt Wiki page (handling of mailto, viewing HTML parts web browser), update and improve bash-completion Wiki page. Added a troubleshooting section in Signal Wiki.
  • Thorsten did another upload of hplip to fix RC bugs. He also spent some time taking care of older bugs. Most of the time such bugs had been fixed in a previous upload but haven t been closed in the BTS. He also uploaded a new upstream version of foomatic-db. Last but not least, he gave some user support with the package epson-inkjet-printer-escpr. There seems to be a new software available for Epson printers. Unfortunately the license is not compatible with DFSG and so this software will never make it into Debian.
  • During DebCamp 26, the Golang team had a dedicated Sprint to transition the Golang toolchain (namely on dh-golang) to make builds aware of the module defined upstream with the aim of solving important issues. To help in these efforts, Santiago made changes in the Salsa CI pipeline and documented on how to use it to check if a package requires adjustments after the toolchain update.
  • Santiago continued co-mentoring Aryan Karamtoth on the Linux livepatching project, specifically providing feedback about the implementation of dlp-tools.
  • Stefano reviewed and merged a migration of Debian reimbursements from wkhtmltopdf to weasyprint, unblocking an upgrade to Debian trixie.
  • Stefano s cPython upstream merge request adding multiarch tags to stable ABI extensions was finally merged.
  • Stefano iterated on his upstream cPython merge request to add CI coverage for Debian s multi-arch expectations.
  • Stefano uploaded Python 3.15.0 beta 4 to Debian experimental.
  • Stefano uploaded Python 3.13 to trixie, fixing a regression in a previous trixie point update he made.
  • Helmut continued to report undeclared file conflicts.
  • Helmut sent patches for three cross build failures.
  • Helmut proposed a MR to port piuparts to pathlib.
  • Antonio has done quite some work on Debian CI, including rebuilding the Debian CI armhf/armel worker VMs, and releasing debci 4.2, implementing a backup scheme, and several improvements to the codebase such as improving the incus-lxc backend in preparation for switching to the upcoming switch to using it by default as announced in the latest bits from the ci.debian.net operators.
  • Emilio helped with transitions, particularly with Python 3.14 as default and Perl 5.42. During the Perl transition, an issue was identified with how britney schedules autopkgtests for binNMUs, and that testing was reverted for the time being.
  • Colin restructured openssh-* binary packages to better support the upcoming GSS-API package split. This caused several autopkgtest regressions in other packages because openssh-server no longer depends on openssh-client, all of which have fixes either pending review or merged now.
  • Lucas started a discussion around the creation of a Debian packaging video course for newcomers in the context of the Outreach team.
  • Lucas reviewed some contributions to ruby3.4 and provided feedback.
  • Anupa worked with Jean-Pierre Giraud on the point release announcements for Debian 13.6 and Debian 12.15.
  • Anupa joined Jean-Pierre Giraud to prepare the Micronews for DebConf 26 press coverage.

10 August 2026

Bits from Debian: DebConf26 Local Team says goodbye

On Saturday 25 July 2026, the annual Debian Developers and Contributors Conference came to a close. The Debian Press team would now like to share this personal and beautiful message from the Santa Fe Local Team. Words from DC26 Local Team DebConf26 is over, and those of us who were part of the Local Team are trying to return to normality , if such a thing exists after organizing a DebConf. This event changed our lives and would not have been possible without the help of many great people. We would especially like to thank everyone who became part of our extended local team. Our endless thanks go to Gunnar who also instigated this whole adventure , Santiago, Nattie, Stefano, and Olasd. Thank you for supporting and guiding us, sharing your experience, and helping us find solutions throughout the entire process. It was also made possible thanks to the great work, strong support and patience of international teams: Fundraising, Bursaries, Content, Video, Treasury, Visa, Website, Accommodation, Front Desk, Cheese and Wine, Publicity as well as all the other teams and individuals who contributed. We apologize if we have forgotten to mention anyone; many people helped make this event possible. Our deepest thanks also go to everyone who joined us in working on the event, especially Fer, Jos , and Juli n, who showed great commitment and took responsibility for several important tasks. We would also like to extend our gratitude to FICH, the Universidad Nacional del Litoral, the institutions, organizations, sponsors, suppliers, and everyone who contributed in one way or another to welcoming the Debian community to Santa Fe. And finally, a very special thank you to our families, to whom we dedicated little time these past few weeks, who supported us on this adventure, enduring the exhaustion, the calls and messages at all hours, and the occasional stressful situation. Always giving us that much-needed, encouraging hug with so much love. These were very intense weeks, during which we tried to give our best so that everyone could enjoy their stay and so that the Debian community had the necessary conditions to meet, work, share knowledge, and continue creating the magic that characterizes community life and the development of Debian. As happens at every DebConf, there were difficulties, unexpected situations, and challenges that required us to improvise, learn, and perform a few juggling acts. There were also moments that will certainly remain as memorable anecdotes: the antisocial room , some gas heaters worthy of a museum, and newly unlocked powers for negotiating with suppliers. We have no evidence, but also no doubt, that for many people the Conference Dinner was one of the best moments of the event. A few ingredients we had hoped would happen naturally were missing, such as more wine nights and at least one in-person football match. During the two weeks of DebConf, we experienced every kind of weather and a wide range of emotions. Above all, however, we saw people enjoying themselves and building friendships, which fills us with pride. Thank you very much to everyone who came and helped DebConf26 leave such a beautiful mark on our hearts. We hope our paths cross again somewhere in life. Best regards, Leonardo, Emmanuel, Mariano, Pablo, and Mart n DebConf26 Local Team About Debian The Debian Project was founded in 1993 by Ian Murdock to be a truly free community project. Since then the project has grown to be one of the largest and most influential Open Source projects. Thousands of volunteers from all over the world work together to create and maintain Debian software. Available in 70 languages, and supporting a huge range of computer types, Debian calls itself the universal operating system. About DebConf DebConf is the Debian Project's developer conference. In addition to a full schedule of technical, social and policy talks, DebConf provides an opportunity for developers, contributors and other interested people to meet in person and work together more closely. It has taken place annually since 2000 in locations as varied as Scotland, Bosnia and Herzegovina, India, Korea, France. More information about DebConf is available from https://debconf.org/. Contact Information For further information, please visit the DebConf26 web page at https://debconf26.debconf.org/ or send mail to press@debian.org.

Jonathan Dowland: time-delayed scifi roundup feed

I enjoy reading The Guardian's monthly round-up of new SF novels, which can be found in their Science Fiction Books section, and can also be read via feed. Since the round-up is of new books, at the time the round-up is published they're usually only available in hardback. When it comes to choosing a book to read, these days I am tending towards paperbacks: I've largely ran out of room for hardbacks. So I decided to apply a time delay to their feed. Six months is roughly enough that a book mentioned in a round-up should be shortly available in paperback. The first obstacle was that The Guardian only publish roughly the last six months of articles in their feed, and so the posts I want have disappeared. However, my Feed Reader (FreshRSS) had older copies stored in its database, and I am able to re-publish those using User Queries. (This also gives me an opportunity to filter out non-roundup articles from the Guardian's feed). It's then a nice short piece of scripting (this time, using Ruby) to filter the republished feed on the publication date. To make the most recent articles appear new, I also modify the metadata for filtered entries to appear 6 months newer than they are. 
#!/usr/bin/ruby
require 'rss'
# replace with the user query feed URI
uri       = 'https://www.theguardian.com/books/science-fiction/rss'
now       = Time.now
sixMonths = 6 * 30 * 24 * 60 * 60
feed      = RSS::Parser.parse(uri)
feed.items.select! do  item 
  item.date + sixMonths < now
end
feed.items.collect! do  item 
  item.date += sixMonths
  item
end
puts "Content-Type: text/xml\r\n\r"
puts feed
I stuck that up on my private web server, subscribed to it in my FreshRSS and voila, a time-delayed list of books to read, most likely available in paperback.

9 August 2026

Elana Hashman: Managing virtualenvs with a little bash

When you need to install something directly from PyPI, Python virtualenvs have been my go-to for over a decade. A quick virtualenv intro Most of my readers are probably already familiar with virtualenvs, but for completeness, I'll give you a brief introduction. A virtualenv (short for "virtual environment") is an isolated distribution of Python packages, where you can independently install packages without disturbing your system packages or other virtualenvs. You can set one up like this, assuming you are using Python 3.3 or higher: 
python3 -m venv ~/.venv/my-virtualenv
The directory specified here is just a convention. I keep all my virtualenvs in the .venv folder in my home directory, but you can pick whatever location you like. To use the virtualenv, you must activate it: 
source ~/.venv/my-virtualenv/bin/activate
This activation script is a special shell script that configures your current shell, pointing at all the right paths in order to use the virtual environment. source runs this script in your current shell session to set it up. You will notice that this adds (my-virtualenv) to the beginning of your shell prompt, reminding you that the "my-virtualenv" virtualenv is active. Now when you pip install amazing-package, the software will only be available in this virtual environment. When you're done, you can deactivate it like so: 
deactivate
Wonderful! Managing many virtualenvs gets annoying Over time, I end up accumulating many virtualenvs, which can become harder to manage. Maybe something like this: 
$ ls ~/.venv/
my-virtualenv cool-project snakes-ahoy
I also don't want to type source ~/.venv/my-virtualenv/bin/activate every time I use the virtualenv, because it gets very repetitive only the name of the venv is really needed. But luckily, we can write a little bit of bash to make managing this less annoying. (Or you can use one of many Python developer tools that are designed to manage this, like pipx, but when I merely want to consume Python software, I might not have a development environment set up. So that's beyond the scope of this post!) If you add the following shell function to your ~/.bashrc or ~/.bash_aliases file, it will nicely wrap our activation command: 
setup-venv()  
        source "$HOME/.venv/$1/bin/activate"
Now all we need to run is 
setup-venv my-virtualenv
So much quicker! Spicing it up with tab completion The first thing I noticed after writing this wrapper was that I started hitting tab on the virtual environment name, but... nothing happened. Wouldn't it be nice to know what virtualenvs I had available, and to not have to type out the whole long thing? Well, we can write it ourselves If for some reason you don't already have bash completion installed, on a Debian-based system, you will need to install it with 
apt install bash-completion
In order to configure our bash completion, we will create a new file, /etc/bash_completion.d/venv, with the following contents: 
_list_venvs()
 
    local cur prev opts
    COMPREPLY=()
    cur="$ COMP_WORDS[COMP_CWORD] "
    prev="$ COMP_WORDS[COMP_CWORD-1] "
    opts=$(find $HOME/.venv/ -mindepth 1 -maxdepth 1 -type d -printf "%f ")
    COMPREPLY=( $(compgen -W "$ opts " -- $ cur ) )
    return 0
 
complete -F _list_venvs setup-venv
This file defines another shell function order to determine how to autocomplete the options for our setup-venv function. $opts is where we define the options for our function. We generate it with a find command looking at the .venv folder in the current user's home directory, then only including child folders (excluding the current directory itself, .venv, in our results) by using the min/max depth and type arguments, and printing just the individual directory names, deliminated by spaces using our print formatter. Everything else is the standard scaffolding required to use bash completions. Once you save this file and reload your shell, you'll see that you are able to use completions as expected! 
setup-venv <tab>
my-virtualenv cool-project snakes-ahoy
setup-venv s<tab>
setup-venv snakes-ahoy
Complaints, comments, questions? Hope this was helpful! If it wasn't, that's too bad. But don't worry you can safely ignore this post.

Reproducible Builds: Reproducible Builds in July 2026

Welcome to the July 2026 report from the Reproducible Builds project! In our reports, we try to outline the most important things that we have been up to over the past month. As a quick recap about what problem our project intends to solve, whilst anyone may inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries. The motivation behind the reproducible builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised or not. If you are interested in contributing to the project, please visit the Contribute page on our website. In this month s report, we cover:

  1. Tool development
  2. Distribution work
  3. Three new scholarly papers
  4. Patches
  5. Misc news

Tool development diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made the following changes, including preparing and uploading versions 324, 325 and 326 to Debian:
  • Fix tests to work with zipdetails 4.0008. (#1141359)
  • Bump debhelper compatibility level to 13. [ ]
  • Update copyright years. [ ]
In addition, Paul Spooren made changes to allow trailing garbage in Gzip files [ ] and Vagrant Cascadian added an external tool reference for the pedump binary to use the mono package under GNU Guix. [ ]
disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into system calls to reliably flush out reproducibility issues. This month, Christelle Gloor added the option to sort by ctime as returned by the lstat(2) syscall. [ ], which Chris Lamb uploaded whilst bumping the Standards-Version to version 4.7.4 [ ]. Bernhard Wiedemann also updated disorderfs to version 0.7.0 in openSUSE.
Yet again, there were a number of improvements made to our website this month as well. For example, Chris Lamb, by request of Digital Ocean, changed the target of a referral link so that they can manage incoming referrers [ ] and pushed a number of changes to the Tools page [ ].

Distribution work In Debian this month, 32 reviews of Debian packages were added, 26 were updated and a total of 21 were removed this month, adding to our extensive knowledge about identified issues. A number of issue types were added by Chris Lamb, including:
  • python_towncrier_build_date [ ][ ]
  • log_files_installed_in_package [ ]
  • fontforge_varies_by_timezone [ ][ ]
Chris also added a further note for the build_date_in_manpage_generated_by_spf13_cobra issue. [ ]
In addition, there is a new page showing verification rebuilds of OpenWrt APK packages and firmware images, powered by rebuilderd:

Three new scholarly papers Yan Li, Nan Jiang, Qihang Zhou, Shaowen Xu, Yamin Xie and Xiaoqi Jia of the Chinese Academy of Sciences published a paper titled VCAligner: Aligning Source Distribution Versions with Upstream Git Commits to Secure Supply Chain:
We present VCAligner, a content-based alignment methodology that constructs inverted indexes over VCS histories to precisely map released artifacts to their originating commits, independent of fragile version tags. We evaluated VCAligner on a dataset of 2,984 verifiable PyPI packages derived from the 4,000 most-downloaded projects linked to public GitHub upstreams. Our results reveal a critical weakness in conventional tag-based heuristics: while they appear effective on 85% of the dataset, the residual 15% failure rate generates a catastrophic downstream audit workload of over 10.3 million commits. In contrast, VCAligner reduces this burden by two orders of magnitude ( 158 ), bounding the total workload to under 65,000 commits. Furthermore, we provide the large-scale characterization of Packaging Noise, classifying artifact divergence into structural additions (Path Phantoms) and content mutations (Blob Phantoms), thereby isolating the distinct attack surfaces of malicious injection and code tampering.

Jens Dietrich and Spencer Sun from the Victoria University of Wellington together with Tim W. White and Behnaz Hassanshahi from Oracle Inc pre-published their paper No Snake Oil: Verifying Python Package Builds (PDF):
Python has become the default language for interacting with AI, with packages being distributed through registries like the Python Package Index (PyPI). This creates a need to analyse supply chains comprising such packages. One such analysis is to rebuild packages in order to identify compromised builds injecting malware. Independent rebuilds in hardened environments have the added advantage that they can generate and record provenance in order to increase the trustworthiness of packages. Two tools that are designed to automate such rebuilds and run them at scale are macaron and oss-rebuild. We study 12,180 popular releases from PyPI and find that the byte-for-byte equivalence rate is generally low. We analyse the reasons why they produce different wheels, and find that equivalence between the original and rebuilt wheels can often still be established, preserving most of the guarantees users expect from rebuildable releases. We present and evaluate daleq4py, a tool to establish the equivalence of Python wheels through the kernel of a normalisation function that is based on provenance-preserving datalog rules. Experimental results show that daleq4py substantially expands the set of rebuilds that can be accepted as equivalent. Although only 15.4% of macaron rebuilds and 19.1% of oss-rebuild rebuilds are byte-for-byte identical to the published PyPI wheels, daleq4py establishes wheel equivalence for 60.2% and 78.9% of source-equivalent rebuilds, respectively.

Denise Nanni, Julien Malka, Stefano Zacchiroli and Th o Zimmermann from T l com Paris together with Gabriele D Angelo from the University of Bologna pre-published their paper Understanding Build Reproducibility in the F-Droid Ecosystem (PDF), which was accepted at the 2026 ACM Conference on Reproducibility and Replicability:
The security of open source applications benefits considerably from the possibility of rebuilding their source and verifying the output. F-Droid, a prominent distribution for open source Android applications, systematically rebuilds them from source and tests their bitwise reproducibility at app publishing time. However, F-Droid offers no guarantee that app reproducibility will continue to hold in the future. As software ecosystems evolve, reproducibility may degrade, with potential negative consequences for software preservation and security. We present the first empirical study of build reproducibility in the F-Droid app ecosystem. Analyzing historical reproducibility logs, we find that the overall bitwise reproducibility rate has been steadily increasing over time (as new versions of apps are published). We then evaluate how reproducibility holds in time for fixed app versions, by attempting to rebuild 18 904 app versions that F-Droid had previously confirmed bitwise reproducible, published between September 2018 and February 2026, achieving an 83% rebuild success rate, and identify missing dependencies as the dominant cause of failure, accounting for 76% of non-rebuildable cases. Among successfully rebuilt apps, 94% are also bitwise reproducible-i.e., they still yield bitwise identical artifacts upon rebuild. Together, these results show that while bitwise reproducibility largely holds for apps that can be rebuilt, rebuildability itself is highly sensitive to temporal decay.

Patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where applicable or possible. This month, we wrote a large number of such patches, including:

Misc news On our mailing list this month, Colin Winter of Markovian Protocol wrote to our mailing list on the topic of Reproducible verification for retained logs:
Reproducible builds remove trust in the builder: anyone re-derives the same artifact from the same source, byte for byte. The same shape applies one layer over, to a retained record. Most record-keeping regimes (the EU AI Act s Article 12 logging is the current example) require that events be recorded and logs retained, but not that a retained log be verifiable, by a party who was not present, as unaltered and existing when claimed. That leaves an integrity obligation resting on trusting the party being audited.
(Full thread)

Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

7 August 2026

Thorsten Alteholz: My Debian Activities in July 2026

Debian LTS/ELTS This was my hundred-forty-fifth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. During my allocated time I uploaded or worked on: Unfortunately the number of assigned hours was rather low this month. So besides doing some days of FD at the end of the month, where I also had to process a new package list for ELTS, and a review of the rsync package (prepared by Sylvain), not much happened here. Debian Printing This month I uploaded a new upstream versions: Besides the package upload, I also took care of some older bugs of hplip. This work is generously funded by Freexian! Debian Lomiri This month I continued the upload of lomiri packages with new upstream versions. Thanks to the help of my other colleagues, this project could be finished now. This work is generously funded by Fre(i)e Software GmbH! Debian Astro This month I uploaded a new upstream version or a bugfix version of: Debian IoT Unfortunately I had no time to work in this category this month. Debian Mobcom This month I uploaded a new upstream version or a bugfix version of: Next month I intend to upload new upstream versions of all Osmocom packages. As far as I can tell, these uploads will happen without soname changes. I like that :-). misc This month I uploaded a new upstream version or a bugfix version of:

Next.

Previous.