Search Results: "reg"

15 May 2026

Freexian Collaborators: Debian Contributions: Detecting undeclared file conflicts, contributors.debian.org mini-sprint, security-tracker performance and more! (by Anupa Ann Joseph)

Debian Contributions: 2026-04 Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

Undeclared file conflicts, by Helmut Grohne The duplication checker, the Multi-Arch hinter, and the /usr-move analyzer share significant parts of their code. While the /usr-move transition is complete, the other tools needed a bit of love. Helmut added Python type annotations, slightly improved the performance of the duplication website and shared more code between these tools. Building upon this Helmut looked into file conflicts of various kinds such as unrelated packages installing overlapping files, file type conflicts, mismatching directory metadata and shared files of Multi-Arch: same packages with varying content. Implementing reliable detection proved to be difficult due to the amount of corner cases. So Helmut semi-manually filed bugs. In that process, it became apparent that binNMUs do not reproduce SOURCE_DATE_EPOCH across architectures and therefore some shared files embedding the build date would vary in content. Additionally, a significant number of reports required further correspondence.

contributors.debian.org mini-sprint, by Enrico Zini Enrico Zini met with Mattia Rizzolo to continue the work started at DebConf 25 on crediting contributions done via salsa, and to catch up with accumulated site issues. Building on the same kind of infrastructure used to notify tag2upload, salsa.debian.org triggers a webping on pushes and merge request activity, which causes a small JSON payload to be queued in a private directory on contributors.debian.org. We worked on processing, filtering and aggregating the files in the queue into a private, staging database table. When configuring a data source on the site, it is now possible to configure automated submission of contributions from information in the staging table. This makes it significantly simpler to credit contributors for all teams that use Salsa as their code repository and coordination tool, as the site can take care of the data mining for you. See more details in the sprint report posted to debian-devel-announce.

MiniDebConf Campinas, by Lucas Kanashiro, Santiago Ruano Rinc n and Antonio Terceiro MiniDebConf Campinas was held between April 23rd and 25th, at the State University of Campinas, and was preceded by a MiniDebcamp between April 20th and 22nd. Freexian was Gold sponsor for the event, and Freexian collaborators were active contributors to the conference success. Lucas and Santiago delivered a talk about Debian LTS during MiniDebConf Campinas 2026, where they described how the LTS project benefits Debian users and developers, while strengthening Debian itself. Lucas and Antonio delivered a talk about internship programs in Debian during MiniDebConf Campinas 2026, with the goal of getting students interested in working in and with Debian. Lucas took part in the MiniDebConf Campinas content team, reviewing/accepting talks and building the schedule. Antonio led a session where he invited the audience to weigh in on current controversies in Debian. The session presented playful elements as colored signs to denote agree/disagree, and was not recorded, to help people feel more comfortable about speaking up. He might be convinced to lead a similar session at the next DebConf. Antonio also organized a debate to discuss the consequences of new Brazilian regulation for the protection of children and adolescents in digital spaces for Debian and other free operating systems, but also for the free software community in general. This session was very fruitful and will lead into further actions, as one of the main outcomes was the realization that the free software community must follow the discussion leading up to similar regulations more closely to avoid being caught by surprise when they come into effect.

security-tracker performance, by Helmut Grohne and Emilio Pozuelo Monfort Prompted by spontaneous influx of web requests on Freexian s security-tracker back in February, we considered the options for managing that demand. One of our mitigations was making it faster. To that end, Helmut sent two MRs towards improving the situation. There are four notable improvements. The use of Python s str.translate generally speeds up rendering of larger templates. Indexing the CVE names avoids a costly sequential table scan. Avoiding FFI calls while sorting and reducing the queryset speeds up the source package view. Emilio reviewed and deployed the changes on to the Debian instance. Together these changes provide a twofold speedup on both Freexian s and Debian s instance on average.

dput-ng data loss bug, by Colin Watson Ian Jackson (not affiliated with Freexian) reported that dput-ng could lose data when using the local install method, which could cause misleading results in tests of other packages; they also filed an initial merge request to fix it. Colin improved this to isolate its tests properly, and uploaded it.

Miscellaneous contributions
  • Lucas coordinated the src:valkey update to version 9 in unstable with a potential co-maintainer.
  • Lucas provided a security update for src:valkey targeting trixie .
  • Thorsten did two uploads of foo2zjs, one to fix a bug and one to improve packaging. As there have been several CVEs published for cups he also did an upload of a new upstream version. Unfortunately this introduces a regression and another upload was needed to take care of a crash. The patch for one CVE also broke a test script, which is used by lots of printing packages in Debian. As a result some autopkgtest runs failed. This could be fixed as well and the only remaining issue that needs some more investigation is related to cups-pdf. It is also worth mentioning that some issues related to the apparmor configuration of cups could be resolved.
  • Helmut sent patches for 11 cross build failures.
  • Helmut sent a MR for enabling the new mainline YT6801 ethernet Linux driver and it is now working fine with Debian s 7.x kernels.
  • Helmut upgraded a crossqa.debian.net autobuilder to trixie .
  • Carles using po-debconf-manager, improved Catalan translations: reviewed 2 packages, submitted 3 packages, deleted 5 packages.
  • Carles did further code developments for check-relations: steps towards making it production ready when the initial round of reports are analyzed. New show-package (information) command, improvements for report_missing cases, added support for ignoring packages for specific reasons, added unit tests, added CI. Used it to open 39 new bugs. Also followed up different open bugs
  • Rapha l completed the French translation of Zulip for the release of version 12.0. Zulip is a nice 100% free software threaded communication platform for distributed teams.
  • Stefano did routine uploads of python-pipx, python-mitogen, platformdirs, python-authlib, python-discovery, distro-info-data, python-virtualenv, python-certifi, python-wheel, pypy3.
  • Stefano uploaded distro-info-data updates to stable and oldstable proposed updates, with the latest Ubuntu release.
  • Stefano took part in DebConf 26 preparation meetings.
  • Stefano prepared DebConf s online video streaming infrastructure for MiniDebConf Campinas, and configured the Debian reimbursement system to handle their travel bursary claims.
  • Stefano helped MiniDebConf Hamburg prepare their website for 2027.
  • Stefano did some sysadmin work on debian.social infrastructure.
  • Stefano reviewed Matthias python3.15 packaging and rebased his work on top of it.
  • Antonio implemented several improvements to the Debian CI platform, including but not limited to adding support for dark mode, dropping compatibility with ActiveRecord < 7 which is no longer shipped in Debian stable, and generating content-based links to static assets, in two parts.
  • Antonio debugged a general slowness in salsa, caused by loss of IPv6 connectivity between the salsa host and the remote object storage in the cloud , which is a problem due to an open upstream bug in gitlab.
  • Santiago reviewed different changes to the Salsa CI pipeline, including the new uscan test job, prepared by Tha s Rebou as Araujo, and the final review to introduce faketime testing, made by quila Macedo.
  • Santiago continued helping the DebConf 26 local team to prepare the conference.
  • Emilio updated libxpm to address a security issue.
  • Colin finished upgrading groff to 1.24.1; 1.24.0 and 1.24.1 were the first upstream releases since 2023 and had extensive changes, so this took some time to get right.
  • Colin released bookworm and trixie fixes for CVE-2026-3497 in openssh, and issued the corresponding BSA-130 for trixie-backports.
  • Colin upgraded openssh to 10.3p1.
  • Anupa worked on the accounting tasks for MiniDebConf Kanpur and prepared and submitted a report to the fiscal host.

11 May 2026

Freexian Collaborators: Debusine workflow performance issues (by Colin Watson)

During March and April, we had a number of performance issues that made Debusine s core functions of running work requests and reflecting their results in workflows quite unreliable. Investigating and fixing this took up a lot of time from both the Debusine development team and Freexian s sysadmins. The central problems involved a series of database concurrency and worker communication issues that interacted in complex ways. On bad days, this caused between 10% and 25% of processed work requests to fail unnecessarily. We communicated some of the problems to users on IRC, but not consistently since we didn t entirely understand the scope of the problems at the time. Most of the problems are fixed now, but we had a retrospective meeting to make sure we understood what happened and that we learn from it. Here s a summary.

Data model Debusine s workflows consist of many individual work requests. Each work request has a database row representing its state, which means that the overall state of a workflow is distributed across many rows. Changes to one work request (for example, when it is completed) can cause changes to other work requests (perhaps unblocking it so that it can be scheduled to an idle worker). Those changes may happen concurrently, and in practice often do. Workers typically need to create artifacts containing the output of tasks: these include things like packages, build logs, and test output. Debusine records task history so that it can make better decisions about how to schedule work requests. Since this might otherwise grow without bound, the server expires older parts of that history after a while. The same is true for many other kinds of data.

Causes
  • Because workflows involve changes that propagate between work requests, there were historically some cases where different parts of the system could deadlock due to trying to take update locks on overlapping sets of work request rows in different orders. We mitigated that somewhere around 2025-11-05 by locking entire workflows in one go before making any change that might need to propagate between work requests like this; that dealt with the deadlocks, but it s quite a heavyweight locking strategy that sometimes caused significant delays.
  • We ve been working for some time to make Debusine useful to Debian developers, and regression tracking is an important part of that: it lets developers test uploads without being too badly misled by tests in related packages that were already failing before they started. On 2026-03-11 we enabled this by default on debusine.debian.net, after testing it for a while. Although this is useful, it put more load on the system as a whole, often approximately doubling the number of work requests in a given workflow with many additional dependencies between them.
  • Like much of the world, we re in an arms race with unethical scrapers desperately trying to feed everyone else s data into LLMs before they run out of money. We saw a substantial uptick here towards the end of March, which meant that we had to temporarily disable regression tracking and to put some other mitigations in front of our web interface.
  • We historically haven t had systematic internal timeouts. Prompted by ruff, a Google Summer of Code applicant went through and added timeouts in many places, including some calls between the worker and the server. This was fiddly work and the student did a solid job, so I m not putting them on blast or anything! However, it did mean that some things that came in under load balancer timeouts now timed out earlier on the client side of the request (and hence in Debusine workers), which made some problems show up in different ways and be more obvious. This was deployed on 2026-04-03.

Fixes

Workflow orchestration Figuring out what individual work requests need to be run as part of a workflow - the process we call orchestration - can be challenging. Unlike typical CI pipelines, these workflows often span substantial chunks of a distribution: a glibc update can involve retesting nearly everything! Nevertheless, it s not particularly helpful for it to take hours just to build the workflow graph. Fixing this involved many classic database optimizations such as adding indexes and CTEs, but probably the most effective fix was adding a cache for lookups within each orchestrator run or work request. Profiling showed that resolving lookups was a hot spot, and the way that task data is often passed down through a workflow meant that the same lookup could be resolved hundreds or thousands of times in a large workflow.

Expiry We knew for quite some time that our expiry job took very aggressive locks, effectively blocking most of the rest of the system. This was an early decision to make the expiry logic simpler by allowing it to follow graphs without worrying about concurrent activity, but it clearly couldn t stay that way forever. Row locks in PostgreSQL was very helpful in figuring out the correct approach here. Since we re mainly concerned about the possibility of new foreign key references being created to artifacts we re considering for expiry, and since that would involve taking FOR KEY SHARE locks on those rows, we can explicitly take FOR UPDATE locks (which conflict with FOR KEY SHARE), and then recompute the set of artifacts to expire with any locked artifacts marked to keep. This was delicate work, but it saved minutes of downtime every day.

Whole-workflow locking I mentioned earlier that we avoided some deadlock issues by taking locks on entire workflows. To ensure that these locks are effective even against code that isn t specifically aware of them, this is implemented by using SELECT FOR UPDATE on all the work request rows in the workflow. In some cases the search for which rows to lock itself tripped up the PostgreSQL planner.

Scheduling We run multiple Celery workers for various purposes. Some of them can do many things in parallel, but in some specific cases (notably the task scheduler) we only ever want a single instance to run at once. Unfortunately a bug in the systemd service meant that the scheduler often ran concurrently anyway! Once we fixed that, the scheduler logs became a lot less confusing. When Debusine was small, it was reasonable for it to perform scheduling very aggressively, typically as soon as any change occurred to a work request or a worker that might possibly influence it. This doesn t scale very well, though, and even though we tried to batch multiple scheduling triggers that occurred within a single transaction, it could still make debugging very confusing. We reduced the number of changes that would result in immediate scheduling, and deferred everything else to a regular tick . The scheduler may not be able to assign a work request to an idle worker due to the workflow being locked. That isn t a major problem in itself; it can just try again later. However, in very large workflows, we found that it often worked its way down all the pending work requests one by one finding that each of them was locked, which was slow and also produced a huge amount of log noise. It now assumes that if a work request is locked, then it might as well skip other work requests in the same workflow until the next scheduler run. Between them, these changes reduced the number of locks typically being held on debusine.debian.net by about 80%: Lock graph

Worker refactoring The Debusine worker has always been partially asynchronous, but while it was actually executing a task - in other words, most of the time, at least in busy periods - it didn t respond to inbound websocket messages, causing spurious disconnections. We restructured the whole worker to be fully event-based. We also had to put quite a bit of effort into improving the path by which workers report work request completion, because if that hits a timeout then it can mean throwing away hours of work. We have some further improvements in mind, but for now we defer most of this work to a Celery task so that whole-workflow locks aren t on the critical path.

Database write volume One of our sysadmins observed that our database write volume was consistently very high. This was a puzzle, but for a long time we left that unexplored. Eventually we thought to ask PostgreSQL s own statistics, and we found a surprise:
debusine=> SELECT relname AS table_name,
debusine->        n_tup_ins AS inserts,
debusine->        n_tup_upd AS updates,
debusine->        n_tup_del AS deletes,
debusine->        (n_tup_ins + n_tup_upd + n_tup_del) AS total_dml
debusine-> FROM pg_stat_user_tables
debusine-> WHERE (n_tup_ins + n_tup_upd + n_tup_del) > 0
debusine-> ORDER BY total_dml DESC
debusine-> LIMIT 20;
              table_name                inserts    updates     deletes   total_dml
--------------------------------------+---------+------------+---------+------------
 db_collectionitem                      1418251   3578202388   3630143   3583250782
 db_token                                 15143     11212106     11389     11238638
 db_workrequest                          386196      6399071   1820500      8605767
 db_fileinartifact                      2783021      1837929   1663887      6284837
 django_celery_results_taskresult       1819301      1501623   1791656      5112580
 db_artifact                             960077      3340859    663890      4964826
 db_collectionitemmatchconstraint       1550457            0   2207486      3757943
 db_artifactrelation                    2229382            0   1363825      3593207
 db_fileupload                          1023400      1057036   1023346      3103782
 db_file                                1673194            0    970252      2643446
 db_fileinstore                         1411995            0    970259      2382254
 db_filestore                                 0      2381578         0      2381578
 django_session                          645423      1519880       531      2165834
 db_workrequest_dependencies             365877            0    936537      1302414
 db_worker                                18317       949280      9487       977084
 db_collection                            10061           85    177741       187887
 db_workerpooltaskexecutionstatistics     28721            0         0        28721
 db_workerpoolstatistics                   1640            0         0         1640
 db_workflowtemplate                        130          158       649          937
 db_identity                                 76          661         0          737
(20 rows)
Oh my - that s a lot of db_collectionitem updates and must surely be out of proportion with what we really need. Can we narrow that down by asking about the most recently-updated tuples?
debusine=> SELECT DISTINCT category
debusine-> FROM db_collectionitem
debusine-> WHERE id IN (
debusine->     SELECT id FROM db_collectionitem
debusine->     ORDER BY xmin::text::integer DESC LIMIT 10000
debusine-> );
           category
------------------------------
 debusine:historical-task-run
(1 row)
That might not be absolutely reliable, but it was certainly a hint. As per PostgreSQL s documentation, by default UPDATE always performs physical updates to every matching row regardless of whether the data has changed, and our code to expire old task history entries wasn t doing that properly. Once we knew where to look, it was easy to add some extra constraints. This reduced our mean write volume on debusine.debian.net from about 23 MB/s to about 3 MB/s, which had an immediate knock-on effect on our request failure rate: Disk write graph HTTP errors

Current state Our metrics indicate that things are a lot better now. We still have a few things to deal with, such as:
  • Some more performance fixes are on their way to fix some remaining cases where views are very slow or where file uploads from workers fail due to locks.
  • We have some changes in the works to revamp how work request changes propagate through workflows in a way that doesn t require so many heavyweight locks.
  • We have a number of monitoring and alerting improvements we d like to make, both for outcomes (things like slow Celery tasks) and possible root causes (database performance). We d also like to deploy some more modern observability tools; hunting for things using journalctl isn t terrible, but it s not really the state of the art.
  • We need to improve how we communicate to users when we re having operational problems, both informally (IRC, etc.) and on the site.
  • Retries don t always behave the way you d expect in workflows.
I hope this has been an interesting tour through the sorts of things that can go wrong in this kind of distributed system!

9 May 2026

Jelmer Vernoo : Remove-after Annotations for Debian Files

deb-scrub-obsolete is a tool in the debian-codemods suite that tries to identify and remove cruft automatically. It knows about dummy transitional packages, superseded alternatives, and similar patterns it can detect by querying the archive. But some workarounds are too project-specific for a generic tool to recognise on its own. Developers can leave structured comments in their packaging files that tell deb-scrub-obsolete when a particular line or block can be removed. The Debian Janitor regularly runs various codemods like deb-scrub-obsolete on all vcs-accessible Debian packages. This means that if you leave a remove-after: trixie annotation in your package, you will automatically get a pull request to remove the annotated code once trixie has been released, without needing to remember to do it yourself.
The Comment Format The annotations take the form of specially-formatted comments. For shell files (and by extension most maintainer scripts), a line-level annotation looks like this:
install -m 755 compat-wrapper /usr/lib/foo/  # remove-after: trixie
When trixie has been released, deb-scrub-obsolete will remove that line entirely. The comment can appear anywhere on the line before or after other comments and additional explanatory text can follow:
blah  # Trixie comes with blah built in # remove-after: trixie
For larger sections, block-level annotations bracket the code to remove:
# begin-remove-after: trixie
alternatives --add foo bar
alternatives --add foo bar1
# end-remove-after
These blocks can be nested, which is useful when one outer condition wraps several inner ones with finer-grained timing.
Expressions The initial set of supported expressions is deliberately small. The main one is a Debian release name: remove-after: trixie means once trixie has been released . The condition is checked against distro-info <https://manpages.debian.org/trixie/distro-info/distro-info.1.en.html>_, the same data source that other Debian tooling uses to track release status. The expression language is designed to be monotonic conditions should only ever go from false to true, not back. A workaround that needs to be re-introduced after removal belongs in a new commit, not in an annotation. If deb-scrub-obsolete cannot parse an annotation it finds in a file, it leaves all annotations in that file untouched, to avoid a situation where related blocks are only partially removed. Annotations can also carry a marker name an arbitrary label with no spaces, commas, or the word after which can then be passed to deb-scrub-obsolete on the command line. This makes it possible to trigger removal of a named set of annotations together, useful for coordinated transitions where several packages need to be cleaned up at the same time.
Future Extensions The initial expression set is minimal; the design leaves room for richer conditions. Some candidates under consideration:
  • Whether a particular suite has a new enough version of a package (removing a Build-Depends version constraint once it is satisfied everywhere)
  • Whether a package has been removed from the archive
  • Whether all currently-supported releases contain a new enough version
  • Whether a Debian transition has completed
Compound expressions using and / or are also on the list, for cases where removal depends on multiple conditions being true simultaneously.
Status The annotation format is specified but not yet implemented in deb-scrub-obsolete - it is planned for a future release. If you maintain Debian packages and have opinions on the annotation format or the expression language, feedback is welcome. The specification lives in scrub-obsolete/doc/scrub-annotations.md in the lintian-brush repository. Many thanks to Helmut Grohne for the initial suggestion and feedback on the design.

Russell Coker: Bad Criticism of LLMs (not AI)

Discussion of AI systems seems to be dominated by fears of uncommon and unlikely threats. I think that we should be focusing more on real issues with LLMs and with society in general and put the most effort towards the biggest problems. It s Not AI True Artificial Intelligence [1] (IE a computer that has the mental capacity of a household pet) is something that I think can be developed, but it hasn t been developed and we don t have good plans for developing it. We seem to be a lot further away from achieving that goal than we were from landing on the moon in 1962 when JFK gave his historic speech. What we have is a variety of pattern recognition systems that can predict what fits into a pattern. The most well known type of Machine Learning (ML) is the Large Language Model (LLM) which means ChatGPT and similar systems which predict which text would be likely to come next and can make an essay from it. They can give interesting and useful output, but there is no thought behind it, it s just a better form of Eliza (the famous program from 1964 that simulates conversation by pattern matching) [2]. By analysing billions of documents, storing the data in a condensed mathematical way, and then using computation to extract from that record LLMs can produce output that is unfortunately considered by some people to be good enough to include in legal documents submitted to courts, university assignments, and many other documents. But they do so without even having the thinking ability of a mouse. To call current systems AIs without any significant qualifiers when criticising them is to concede the debate about the worth of such things. If we develop AIs that can actually think we will have to deal with the issues in the SciFi horror short story Lena by qntm [3]. The Bad Arguments Here is a list of some of the most unreasonable arguments I ve seen against AI which distract attention from real problems both related to AI and other problems in society. Suicide and Homicide Wikipedia has a page listing Deaths Linked to Chatbots [4] which right now has 16 entries from 2023 to Feb 2026. They are all tragedies and as a society we should try to prevent such things. But what I would like to see from the media is some analysis of overall trends, yes it gets people s attention when someone dies in an unusual way but we need to have attention paid to the more numerous deaths which are preventable. It has become a standard practice to give information on Lifeline in media referencing suicide, it would be good if they also developed a practice of mentioning the relative incidence of a problem when publishing an article about it. One of the many factors that cause more suicides than chatbots is school, Scientific American has an informative article from 2022 about the correlation between child suicide and school [5]. It is based on US statistics and shows that the lowest suicide rate is in July (a no-school month in the US) which has a rate of 2.3 per 100,000 person years. So if kids had a quality of life equivalent to July all year around then there would be 2.3 suicides per 100,000 kids every year while if they had a quality of life equivalent to a Monday in January or November it would be 3.9 suicides per 100,000 kids every year. The article states Any time I present these data to teachers, parents, principals or school administrators, they are shocked. This should be common knowledge. It is common knowledge to anyone who takes any notice of what happens in schools, but paying attention to serious problems is unpleasant, it s more fun to pretend that school is good for everyone. No parent wants to think that they sent their child to a place that was horrible, no teacher wants to think that they are part of a system that harms kids. The US CDC has an informative article about youth suicide [6] which documents it as the 3rd largest cause of death in the 14-18 age range fro 2021. This article was published in 2024 and based on statistics from 2023 and earlier. It notes significant differences in suicides, attempts, and persistent feelings of sadness or hopelessness which had girls at more than twice the rate of boys and LGBQ+ kids at more than twice the rate of heterosexual students. It seems obvious that misogyny and homophobia is correlated with suicide and that s something that could and should be addressed in schools. My state has a Safer Schools program [7] to try and alleviate the problems related to homophobia, but I expect that things are getting worse in the US in that regard. 39.7% of kids in US high schools had persistent feelings of sadness or hopelessness before LLMs became popular, school could and should be a happy time for the vast majority of kids but instead almost half of the kids don t enjoy it and a majority of girls and LGBQ+ kids don t. Having no mention of trans kids is a significant omission from that article, based on everything I ve heard from trans people I expect that their statistics would be even worse. One could argue that the small number of deaths inspired by use or misuse of LLMs is an indication of a larger number of people suffering in ways that don t result in death and don t get noticed. But I don t think that can compare to the fact that the majority of girls and LGBQ+ kids have persistent feelings of sadness or hopelessness in the current school system. Regarding homicide, the Australian Institute of Criminology has an article showing that in the 2003-2004 time period 49% of women who were killed were listed as a domestic argument [8], that s something that could and should be addressed. That article claimed 308 homicide victims in that time period which is larger than the world-wide death toll from LLMs but also less than 1/3 the death toll from car accidents in Australia. Australia has less than 0.4% of the world population, a fairly low homicide rate, and a number of homicides that vastly outnumbers all world homicides related to LLMs. I think it s great to address any cause of suicide or homicide, but devoting government resources and legislation towards very uncommon causes instead of things that happen every day is not a good strategy. It would be fine to address all factors leading to suicide, but problems with the school system have been a major factor for decades with little effort applied to fix it. Fraud and Other Crime There is evidence of criminals using LLMs to help prepare for crimes, the ability to generate large amounts of text quickly can be used for fraud and extortion. This is going to be a serious problem and we need structural changes to society to deal with it. There is an ongoing issue of scammers convincing older people that their child or other young relative is in trouble and a large amount of cash is required to address it. This sort of scam as well as the more well known Nigerian scams will probably become more common as the cost of running them decreases. This may be more of a problem for people in developing countries as currently a common scam business model is to have people in regions where wages are low (such as Pakistan for one who I spoke to) scamming people in relatively wealthy countries like Australia so an attack with a low probability of success is financially viable. Cheaper attacks will make less affluent victims financially viable to the scammers. While writing this post I received a financial scam phone call trying to get me to invest in SpaceX that was run by an AI chat system, I expect to receive more of them and this is something that needs to be dealt with via both technical measures and legislation. Do we have to accept less freedom and less anonymity in finances as a cost of reducing financial crime? Greater restrictions on the use of cash would make some crimes more difficult or less profitable for criminals. As a society I think we need to have a discussion about a balance between financial freedom and freedom from criminal exploitation, failing to have such a discussion is likely to lead to policies which don t work well. Also one thing that ML systems are good at is recognising patterns in data. Banks could scan all their transactions and look for patterns that correlate with fraud. They currently do this badly and do things like locking credit cards when someone goes to another country and spends money. They could do a better job of that and involve the police in cases of obvious fraud even when the customer doesn t realise that they are a victim. This isn t a reason to criticise AIs , it s a reason to plan defensive technology that matches the capabilities of attackers. As an aside I used to work for a company that was developing AI software to scan bank phone calls and allow banks to recognise employees who acted illegally. Unfortunately the Royal Commission into banking misconduct [9] didn t impose any penalties that gave the banks a financial reason to avoid criminal activity. Unemployment and Inequality There are many claims about AI systems making large numbers of jobs obsolete, some of them are outlandish such as the claims that all white-collar jobs will be obsolete in the near future. There are some reasonable claims like the ability to replace some mundane jobs. Replacing jobs that suck with computers, robots, and other machinery is a good thing! Very few people wish that they were working on a farm without a tractor. In 1900 it s estimated that between 60% and 70% of the world labour force worked in agriculture and 40% of the US labour force did so. Now it s something like 27% globally and between 1% and 3% in developed countries. Automated factories are also a good thing, it s best to avoid boring and dangerous work. The most plausible claims about job replacement from AI is jobs that involve analysing and summarising documents. One example that comes to mind is the worst kind of journalism where press releases from companies are massaged into the format of a feature article. I don t think anyone wants that sort of job and doing it with AI hopefully means no human has to sign their name to it. For work like programming few people will be directly replaced by AI but if people can do their work more efficiently while using it then less people are required. I don t think that any programmer likes the part of their job where they have to skim read long documents looking for a clue about how to solve a problem with a library or protocol. A LLM processing the document and finding the potentially useful things will take away the drudgery from the work and allow greater productivity. The trend in replacing people has been making people work longer. If you force all employees to work 60 hour weeks then that can theoretically allow hiring fewer people than having 40 hour weeks. For some work that applies but for skilled work it mostly doesn t as productivity and work quality on average drops when people work more than 40 hours in a week. Another trend for exploiting people is having a low minimum wage and making accommodation expensive so that many people need to work two jobs. What we need is legislation to restore the situation in the 70s where a single full time job was sufficient to provide for a family. The low minimum wage and high expenses for many things is a problem that s been slowly developing over the course of decades while being mostly ignored by journalists. If they could concentrate on the real issues that are hurting workers today they could incite political action to fix these problems. Academic Cheating There is no shortage of ways of cheating in school and university. There are people who are paid to write essays, mobile phones are used for cheating in exams, etc. Getting an AI to write essays makes it easier to cheat for the essay writing part but does so with lower quality and in a less stealthy way. What s the worst case scenario? That we have to change to oral exams for all university subjects? In the US the average annual price for tuition at a university is apparently $25,000, if each student had individually supervised assessment for their exams at a cost of $100 per hour it would make the degree cost 4% more. The cost of university in the US is unreasonably high and that s a problem that needs to be fixed, but a hypothetical case of increasing the price by 4% isn t going to be a major part of it. Weak Arguments Against AI Computer Security Attacks There have been many claims made that AI will break the security of all systems and cause the type of disruption that was previously predicted for year 2000. Bruce Schneier has written a good analysis of the issues including how AI can be used by both attackers and defenders [10], he doesn t have a strong conclusion on whether the net result will be good or bad but his article does make it clear that the result is not going to be a total disaster. While I was working on this post I read another post by Bruce Schneier that was significantly more negative about this issue [11]. While I still don t think this will destroy civilisation I found his other post convincing enough to move computer security from the bad argument section to the weak argument section. Spidering the Web to Death There are issues of bots from AI companies doing a bad job of trying to download all the Internet s content and using a lot of resources. When it was just the major search engines and the Wayback Machine doing it the load was small due to having a small number of organisations that were very good at the way they did it having evolved practices over many years. Now we have a lot of idiots doing it badly and repeatedly hitting generated content. This is really annoying but is something that we can deal with. Currently my blog and many other sites are hosted on a Hetzner server with a E3-1271 v3 CPU with 32G of RAM and there are occasions where more than half the CPU power is being used to service web requests from such systems. Even on the server bidding (renting servers previously used by other customers) Hetzner isn t offering systems so slow nowadays, the slowest they offer is about 20% faster than that. This is something that can be dealt with by spending a little more on hosting until the companies doing that go bankrupt. I m sure this is a serious problem for some people, but for most people it s not a big deal. Also hostile traffic on the Internet is something we have all had to deal with as a part of life since the mid to late 90s. RAM Prices The unreasonably high prices for RAM are annoying and hurt the development of useful computer projects. Big companies can afford it, even with current high prices and large quantities of RAM used for some servers it s still not significant. But it is a major issue for hobbyists and small projects. Things like setting up a dozen test VMs for FOSS development are now too expensive for many people who develop software in their spare time. But this is a temporary thing, if AI companies were to keep buying RAM at high rates for a few years companies would just manufacture more of it to meet demand. In some situations capitalism can work. Environmental Damage There are many people claiming that power used by data centers for AI will lead to environmental damage, using power and water when there isn t enough. The trend of computer hardware is to get smaller and faster. It hasn t been going as fast as it used to in many areas but it hasn t stopped either and it s an exponential trend. There has been an increase in data centers (DCs) for AI use as the use has been increasing faster than the hardware gets smaller. Eventually they will stop increasing faster than advances in hardware and software can match and the size of DCs will decrease. As the production of renewable energy is increases the environmental cost of energy hungry industries decreases. In a few years this won t be an issue anyone is bothered about. False Claims About Danger as PR Jamie McClelland makes an interesting claim that the AI companies are pushing dangers of AI as a method of PR [12]. That seems plausible and combined with the tendency of many journalists to just massage press releases from companies into articles could be the reason for a lot of the bad arguments against AI. Good Arguments Against AI Spam Everywhere I ve previously written about Communication and Hostile AIs [13]. I think that filling all communication channels with rubbish is a denial of service attack against society. In the past communication took some effort, even the simplest email that was directly targeted at the recipient took some human effort and that reduced it s frequency. I get a lot of spam saying something like I see your web site doesn t rank in the top for Google searches while my web site in fact rates well and the actor named Russell Coker is ranking below me, so I know that such spam hasn t had the minimum of human involvement. Now a spammer who wanted to do a better job could get an LLM written spam for every target so the message was specifically aimed at them and would take much longer to be recognised by a human as spam and would also avoid most anti-spam software. Searching for businesses used to be easy, the phone book had listings for them and there was a real cost to being in the book as well as humans actively trying to stop fraud. Creating fake web sites to get business isn t too difficult but it s also not trivial at the moment and such fake sites won t look complete. Now with LLMs it s possible to create hundreds of sites that have content and look reasonable without human involvement. Instead of the small number of suicides and homicides inspired by AI chat systems we should probably be concerned about people who need psychological or medical advice being misled by bogus web sites created as part of fraud campaigns. Imagine people searching for mental health assistance finding web sites run by cults who oppose psychology as a profession. Imagine people searching for basic medical advice such as how to cook a healthy meal getting sucked in to web sites that start sane and then lead people to Ivermectin as a universal medicine. LLMs have the potential to take spam from quick and simple attacks to large scale targeted fraud aimed at people and organisations that don t have the resources to defend against it. There have been many reports of CEO impersonation fraud against major corporations aiming to steal hundreds of thousands of dollars and fraud against individuals who are persuaded to get amounts like $50,000 to help a relative who is allegedly in a difficult situation. But if every corner store experienced the same type of attack that CEOs experience and if every child had someone trying to steal the pocket money in the same way that relatively wealthy people are being targeted now it would really change things. David Brin wrote an insightful and informative blog post about this focusing on how AI generated content is being allowed to destroy YouTube [14]. Deep Fakes There is some overlap between filling all communications channels with rubbish (fake news etc) and deep fake. Making a fake photo of a politician or celebrity to lobby for legislative changes is a real issue but it s not what most people think of when the term deep fake is used. Using photo and video faking targeting non-consenting people is a serious issue. It s not just fake porn (which is a major issue and will cause some suicides) as there are many other possibilities. Fake videos showing behaviour that justifies sacking people from their jobs is going to become an issue, for people in public facing positions even proof that the videos are fake won t necessarily help them. Will we find ourselves in a situation where every politician gets deep-fake porn made of them and the only people who run for public office are ones who are cool with that? Will positions of leadership in the technology industry be restricted to people who aren t bothered by having the most depraved fake porn made of them? The Justice System We have seen a lot of evidence of law enforcement and the court system based on bias leading to bad results. The Innocence Project attempts to correct that and it s web site documents some of the things that have gone wrong [15]. Using AI systems to do some of the work of law enforcement by training computers on the flawed results of current systems can entrench bias and also make it harder to spot. When determining whether someone should be considered a suspect or whether a prisoner should be eligible for parole the number of factors that a human can use is limited. But a computer can take many more factors into account so the issues of whether inappropriate factors are being used can be masked. Computers are also unable to explain decisions that they made and are also able to come up with better fake reasons. In the past there have been racist policies in the US about banks not lending to people living in suburbs where most houses were owned by non-white people, these policies were documented and the documents have become part of the historical record showing racist policies. If a LLM decides not to lend money to people based on mathematical correlations it determined based on historical banking practices it could assign negative weights to factors such as non-English names and implement the racism in a large array of numbers with no proof. The current cases of lawyers getting LLM systems to do some of their work and having their incompetence revealed when the computer generated work is shown to be ridiculously bad are amusing. But that is not the real problem. The real problems will start when the computers in police cars start flagging every car owned by a non-write person as having a probable cause for a drug stop. Technically Not Financial Fraud The majority of the ecosystem around AI is a financial scam [16]. There are companies and individuals doing good things with machine learning some of which is based on hardware and software developed as part of this ecosystem. But the majority of it has no plausible path to profits and a the future of it inevitably ends with some bankruptcies. There are circular flows of money that have the major cloud providers and NVidia looped in, when the values of these companies correct it will become apparent that they have all burned a lot of money keeping this running and all the senior people have got a share of it (the entire purpose of stock options is to allow senior people to suck money out of the company). Then every cloud provider will increase costs while under chapter 11 and all the companies that depend on them will pay whatever it takes. That includes all major companies and most governments. Unlike the dot-com boom and crash and the housing crash the coming financial crash will impact every company that we deal with and most governments. So the people in first-world countries will effectively be taxed to pay for this scam while the executives go party in Monaco. This may seem like an extreme claim but it all happened before with the dot com crash and the housing market crash. The CEO class has an ongoing practice of doing things that aren t crimes because they lobby (bribe) politicians to make them legal. So the current stock market shenanigans around AI don t seem to involve things that governments consider to be crimes. But any normal person might be surprised to learn that such things are legal and most people would vote for such things to be crimes if they had the opportunity. A global financial crisis is the least of the problems that seem likely to afflict society from AI systems. But it will be more immediately obvious when it happens which could be this year! Propaganda Creating art requires skills that the type of people who want to create propaganda tend to lack. AI technologies allow creating art that is based on mathematical models of actual art to the requirements of the person running the program. I have seen the term AI Fascism used to describe the use of AI to help authoritarian governments. I am dubious about whether it deserves that term and while every article I ve read about the topic has had some good points I thought that they were all weak points. But there are lots of ways that governments can abuse their populations without going full fascist. In the last century there were lots of truly terrible governments that didn t even make the top 10 of fascism. AI Sycophants Bruce Schneier wrote an informative blog post about AI Chatbots and Trust which focused on sycophantic chatbots [17]. We have seen a lot of evidence of terrible behaviour and stupid decisions from rich people due to having no negative consequences for bad choices. The vast majority of the history of kings concerns bad decisions made by such people. A future where middle class and poor people can make the same bad decisions as rich people wouldn t be good. Good Things About ML Machine Learning (abbreviated as ML) can do useful things. It s not just Large Language Models (LLMs) such as ChatGPT etc. There are also ML systems that can analyse images and other data sets. I have found ChatGPT to be very useful for making suggestions for improving blog posts. I don t get it to write anything just ask for suggestions. It has pointed out things that I missed such as when I didn t include the price when reviewing a car because the car in question was much more expensive than I will ever pay, the price wasn t relevant to me but would be to some readers. It has also made useful suggestions about structure of blog posts, repeating points, and having a good conclusion. It has some downsides which include trying to erase my voice from my writing, suggesting that the rhetorical question does email suck? is unprofessional. I have worked for a company that used ML systems to analyse driver performance and alert people if a driver is falling asleep, using a phone, or otherwise seems unable to drive safely. Their business model involved a human reviewing the images from the drivers the computer flagged and then determining who is actually doing the wrong thing. This seems a good use of the technology. I have also worked for a company that used ML systems to analyse the performance of bank employees and detect potentially fraudulent behaviour. Preventing crime seems to be clearly a good thing and in this case the manager of the employee in question would review the evidence to make sure that they weren t being falsely accused. Conclusion I don t think that the problems with managing the changes that so called AI is introducing are particularly new. An example of how society handles change that s worth considering is car safety. The seat belt first became mandatory for aeroplanes in some jurisdictions in 1928. The Model T Ford is widely regarded as the first vehicle to start a mass market for cars and it was released in 1925. So if society acted in a reasonable way then for the majority of mass market cars seat belts would have been a standard feature. However seat belts were first made compulsory in 1970 in Victoria Australia and there are still people who think that they are safer without seat belts! The delay in adoption of car seat belts is only one example of needless deaths caused by not taking reasonable measures for car safety but it s one that s easy to demonstrate and measure. The difference between past problems like car safety and the current problems of AI is that the AI problems will be more pervasive. Most of my history as a car driver and car passenger was in cars that are much less safe than cars made in the last 10 years. But partly through luck I ve never been in a serious crash so being in cars that wouldn t have given me a low probability of surviving a freeway speed crash didn t affect me. There is no possibility that through any combination of luck and skill someone could avoid the downsides of AI . If nothing else the results of elections will be affected and no-one can avoid that. As a society we really need to address the real issues related to AI which in some cases requires legislation.

7 May 2026

Reproducible Builds: Reproducible Builds in April 2026

Welcome to our April 2026 report from the Reproducible Builds project! Our reports outline what we ve been up to over the past month, highlighting items of news from elsewhere in the increasingly-important area of software supply-chain security. As ever, if you are interested in contributing to the Reproducible Builds project, please see the Contribute page on our website. In this month s report, we cover:

  1. Tor stateless relays and Reproducible Builds
  2. Civil Infrastructure Platform celebrates 10 years of supporting industrial grade Linux
  3. Reproducible Builds at LinuxFest NorthWest
  4. Reproducibility issues in Rust binaries that embed random bytes
  5. Distribution work
  6. Patches
  7. diffoscope development
  8. Documentation updates
  9. Misc news


Tor stateless relays and Reproducible Builds An interesting post was published on Tor Project blog by Osservatorio Nessuno OdV this month on stateless relays . These are stateless, diskless operating systems that are designed to be used as Tor exit relays. According to the post, which is titled A Server That Forgets: Exploring Stateless Relays:
For relay operators, this approach raises the security bar by enforcing better behaviors by design: [ ]
  1. Reproducibility. A system that doesn t change between reboots is easier to verify and, eventually, to reproduce and audit.
Furthermore, using a Trusted Platform Module (TPM), could allow for greater integrity in the future:
Transparency logs. Once you have a measured boot chain, you can publish it. A relay operator provides a recipe for a reproducible build; anyone can recompute the expected hash and verify it matches what the TPM reports. An append-only transparency log can make these attestations publicly auditable. The Tor community could run an independent monitor to track this across the relay fleet.

Civil Infrastructure Platform celebrates 10 years of supporting industrial grade Linux Congratulations to the Civil Infrastructure Platform (CIP) for reaching their 10-year anniversary last month. CIP has been a supporter of Reproducible Builds for many years, and we have collaborated on a number of technical issues that overlap. As Chris Lamb mentions in CIP s press release:
The collaboration between the Reproducible Builds project and CIP highlights a critical shift in how we approach industrial software. Through verifiability, CIP ensures that the open source foundation of our critical infrastructure is not only sustainable but also demonstrably secure. This commitment to transparency is vital for the trust and resilience required by critical systems over decades of operation.

Reproducible Builds at LinuxFest NorthWest Vagrant Cascadian and Chris Lamb hosted a table in the exposition hall at LinuxFest NorthWest 2026 this month in Bellingham, WA, USA, introducing many people to Reproducible Builds and answering questions both days of the conference. In addition, Vagrant presented Beyond Trusting Open Source Software on Sunday afternoon, exploring the intersection of Free/Open Source Software, Reproducible Builds and Bootstrappable builds, and how they all reinforce each other. Vagrant s slides are available online, including source code to build them reproducibly.

Reproducibility issues in Rust binaries that embed random bytes Reproducible Builds developer kpcyrd opened a ticket on the Rustsec issue tracker regarding binaries that deliberately inject random bytes into their binaries as a secret seed for a Hash Collision DoS mitigation. As kpcyrd notes in his message, this causes issues for reproducibility, and because the relevant end-user binaries are mostly distributed pre-compiled through package managers, those binaries (and by extension the secret seed) are public knowledge . kpcyrd goes on to note:
This is somewhat unique to Rust because Python/JavaScript doesn t compile binaries, and Go (to my knowledge) is too restrictive during build for any library to pull something like this.

Distribution work In Arch Linux this month, Robin Candau and Mark Hegreberg worked at adding a new repro tag/version to the Arch Linux Docker images providing a bit-for-bit reproducible image. Robin also shared a related announcement and implementation details on our mailing list. Arch Linux developer Robin Candau posted a blog post announcing that Arch Linux Now Has a Bit-for-Bit Reproducible Docker Image . Robin mentions one interesting caveat:
to ensure reproducibility, the pacman [package manager] keys have to be stripped from the image, meaning that pacman is not usable out of the box in this image. While waiting to find a suitable solution to this technical constraint, we are therefore providing this reproducible image under a dedicated tag as a first milestone. [ ]
The blog post was also discussed on Hacker News.
In Debian this month, 24 reviews of Debian packages were added, 7 were updated and 16 were removed this month adding to our knowledge about identified issues. Vagrant Cascadian performed Non-Maintainer Uploads (NMUs) in Debian for several packages with outstanding patches over a year old jakarta-jmeter, wxmplot, critcl, vcsh and magic-wormhole-transit-relay. In addition, Reproducible Builds developer Jochen Sprickerhof filed a bug against the APT package manager to request that APT should ignore [a] 0 epoch when downloading or installing with a version specifier . This is related to the special-case handling of the optional epoch prefix in Debian package version numbers.
In NixOS, Julien Malka presented Lila: Decentralized Build Reproducibility Monitoring for the Functional Package Management Model, a paper written together with Arnout Engelen at the Mining Software Repositories (MSR) ACM conference, where it was awarded the MSR 2026 FOSS Impact Award. Congratulations!
Lastly, in openSUSE, Michael Schroeder added reproducibility verification support in the Open Build Service [ ] and Bernhard M. Wiedemann posted another openSUSE monthly update for their reproducibility work there.

Patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where applicable or possible. This month, we wrote a large number of such patches, including:

diffoscope development diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes, including preparing and uploading versions, 316, 317 and 318 to Debian.
  • Chris Lamb:
    • Bump Standards-Version to 4.7.4. [ ]
    • Correct ordering of python3-guestfs architecture restrictions. [ ]
    • Limit python3-guestfs Build-Dependency to architectures that are not i386. [ ]
    • Try to fix PYPI_ID_TOKEN debugging. [ ]
  • Holger Levsen:
    • Add ppc64el to the list of python3-guestfs architecture whitelist. (Closes: #1132974). [ ]
In addition, Vagrant Cascadian updated diffoscope in GNU Guix to version 317.

Documentation updates Yet again, there were a number of improvements made to our website this month including:
  • Manuel Jacob:
    • Fix some minor wording issues on the Stable inputs page, and update information about the sorting behavior of GNU Make [ ].
    • On the Archives page, remove information about deterministic archives in historical Fedora versions [ ], add a note about .tar file portability [ ] and correct a section about .tar PAX headers [ ].
  • Mattia Rizzolo:
  • kpcyrd:
  • ktecho:

Misc news On our mailing list this month:
  • Timo Pohl posted our list inviting people to online group discussions with 4-6 participants each to talk about your perception of terms and requirements for reproducibility. As Timo notes:
    During our research of the existing literature, as well as my experience at the Reproducible Builds Summit 2025 in Vienna, we noticed that some of the terminology in the field is not used consistently across different groups of people, and that the precise meaning of some core terms like reproducibility of an artifact in itself is not uniform.
    As Timo mentions, the sessions will last roughly 90 minutes and will be rewarded with 50 per participant.
  • kpcyrd posted to the list asking for assistance with fixing an issue after updating the flake.lock file for their repro-env project.
  • Aman Sharma of the KTH Royal Institute of Technology, Sweden, posted to our list in order to share that Eric Cornelissen, a PhD student in KTH s CHAINS group, is maintaining an open-source project to monitor the reproducibility of GitHub Actions:
    The goal of the project is to assess whether GitHub Actions can be reproduced. Currently, it focuses on two types of Actions: JavaScript-based actions and Docker-based actions (composite actions are not considered). For JavaScript actions, the project rebuilds the distributed files and compares them bit-by-bit with the repository contents. For Docker actions, it rebuilds images from the Dockerfile and checks for semantic equivalence, using diffoci, across builds.


Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

5 May 2026

Thorsten Alteholz: My Debian Activities in April 2026

Debian LTS/ELTS This was my hundred-forty-second month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. During my allocated time I uploaded or worked on: I also did a week of front desk duties and started to work on backports of the cups CVEs. Debian Printing This month I uploaded a new upstream versions: Unfortunately the first upload of cups introduces a regression and another upload was needed to take care of a crash. The patch for one CVE also broke a test script, which is used by lots of printing packages in Debian. As a result some autopkgtest runs failed. This could be fixed as well and the only remaining issue that needs some more investigation is related to cups-pdf. This work is generously funded by Freexian! Debian Lomiri This month I continued to work on unifying packaging on Debian and Ubuntu. This makes it easier to work on those packages independent of the used platform. I also started working on two new packages: lomiri-radio-app and lomiri-fretboardtrainer-app This work is generously funded by Fre(i)e Software GmbH! Debian Astro This month I uploaded a new upstream version or a bugfix version of: Debian IoT This month I uploaded a new upstream version or a bugfix version of: Marcos Talau joined the Debian IoT group, welcome aboard. Debian Mobcom This month I uploaded a new upstream version or a bugfix version of: misc This month I uploaded a new upstream version or a bugfix version of:

4 May 2026

Russell Coker: Tower Servers and Resizable BAR

A feature on modern PCIe implementations is Resizable BAR AKA REBAR . This basically means that instead of allocating 256MB of address space for a PCIe device to have it s memory mapped the device can ask for more, the limit can be 4G with some hardware or the combination of motherboard and expansion card can support 64bit addressing to allow the entire memory space of a GPU to be mapped in one region. Directly mapping all the memory will be faster no matter how things work, but a combination of algorithms optimised for a flat memory layout and overheads from remapping can cause 90% of performance to be lost without REBAR support. Some GPUs (or maybe the software driving them) will even refuse to work without it. I believe that almost all hardware supporting DDR4 will support REBAR at a hardware level, but in many cases the BIOS doesn t support it. There are people who have reflashed a system BIOS to add REBAR support and there are options to use a modified UEFI boot loader to replace the code that is used for mapping the GPU memory. The systems I like to use are server grade tower systems with registered ECC RAM, after a few years they become quite cheap and still give decent performance while supporting large amounts of RAM. But many such systems that could support REBAR don t, presumably because the vendor doesn t have a great interest in supporting new uses of old hardware. Comparing the Name Brand Servers The HP Z640 and Z840 systems I m running date from 2014 and give good performance with replacement CPUs that are cheap on ebay, but they don t support REBAR without a flashed BIOS. The next release of those HP servers are the HP Z6 and Z8 Gen 4 systems from 2017 that have BIOS support for enabling REBAR. The Lenovo Thinkstation Px20 (P520, P920, etc) don t support REBAR which is especially disappointing as they were on sale from 2017 to 2022 and have decently fast CPUs. The replacement for the Px20 systems are the ones that are still on sale now and they seem likely to have REBAR support but won t be affordable on ebay. The Dell PowerEdge T440 and R740 systems (and presumably all their servers from 2017) don t support REBAR. There are no google hits for T550 and R750 systems from 2021, so presumably no complaints means that Dell servers from that era support it. But the T350 servers are junk and only take slow CPUs, and the T550 systems are brutally expensive. The Precision 5520 systems don t support it and newer Precision workstations will get expensive. It seems that HP is best for this. Which HP Workstation The Z2 G4 only supports 64G of RAM so isn t worth considering. The Z4 G4 is low end and comes in two variants. The one with i5/i7/i9 CPUs doesn t support ECC RAM so isn t suitable for me, and that probably means most Z4 G4 systems on the market. The upside is that apparently 2*6pin PCIe power cables is standard so any size GPU should work and there are 8 DIMM slots supporting up to 512G of RAM. There are 3 options for PSU, 490w for 0 GPUs, 750W for 2 (small) GPUs, and 1000W for up to 4 GPUs. The Z6 G4 has an option for a second CPU that almost no-one selects, that reduces the space for RAM so there s only 6 DIMM slots. But as there is no option for a Z6 without ECC RAM every one on offer will be good. The Z8 G4 is a nice dual socket system that I would not use for a serious GPU after my experience of my Z840 having a motherboard problem from a big GPU. The Z4 G4 is going for about $500 on ebay with the 750W PSU, that is more than I want to pay but not a lot more. In 6 months they could be going for $350 or so. There are hardly any Z6 G4 systems on offer and they are all well over $1000 so I m not considering them. Conclusion I need to poll the second hand sites for Z4 G4 systems and find one going cheap. One of those could be a good ML test machine for a while and then become a workstation once the faster CPUs (which are currently around $900) become cheap.

Russell Coker: Copy Fail on Debian and SE Linux

I have just learned of the Copy Fail kernel vulnerability [1] thanks to alexanderkjall@mastodon.social (who I have just followed on Mastodon and I recommend that you follow too). The question for me (after installing the patched kernel the systems of mine that are most exposed) is whether SE Linux would have stopped that. Basic Policy Analysis For the SE Linux policy analysis the alg_socket class is the one that is related to the exploit. So the following policy analysis command (run as non-root with policy copied to /tmp from a running system) shows what domains are allowed access on my current Debian development system:
$ sesearch -A -c alg_socket /tmp/policy.35 
allow NetworkManager_t NetworkManager_t:alg_socket   accept bind create read setopt write  ;
allow bluetooth_t bluetooth_t:alg_socket   accept append bind connect create getattr getopt ioctl listen read setattr setopt shutdown write  ;
allow daemon init_t:alg_socket   getattr getopt ioctl read setopt write  ;
allow devicekit_disk_t domain:alg_socket getattr;
allow lvm_t lvm_t:alg_socket   append bind connect create getattr getopt ioctl read setattr setopt shutdown write  ;
allow sosreport_t domain:alg_socket getattr;
allow sysadm_t domain:alg_socket getattr;
allow unconfined_domain_type domain:alg_socket   accept append bind connect create getattr getopt ioctl listen lock map name_bind read recvfrom relabelfrom relabelto sendto setattr setopt shutdown write  ;
The above is the same as on the Trixie release policy as these things aren t changed often. Below is from Debian/Bookworm which is the same apart from Bookworm not allowing lvm_t:
$ sesearch -A -c alg_socket /tmp/policy.33
allow NetworkManager_t NetworkManager_t:alg_socket   accept bind create read setopt write  ;
allow bluetooth_t bluetooth_t:alg_socket   accept append bind connect create getattr getopt ioctl listen read setattr setopt shutdown write  ;
allow daemon init_t:alg_socket   getattr getopt ioctl read setopt write  ;
allow devicekit_disk_t domain:alg_socket getattr;
allow sosreport_t domain:alg_socket getattr;
allow sysadm_t domain:alg_socket getattr;
allow unconfined_domain_type domain:alg_socket   accept append bind connect create getattr getopt ioctl listen lock map name_bind read recvfrom relabelfrom relabelto sendto setattr setopt shutdown write  ;
I checked every Debian policy back to when the alg_socket class was first added and found that the older versions had fewer domains granted access. The most recently added was bluetooth_t and the one before that was NetworkManager_t. The Risky Lines Of those allow statements the following are the risks: Unconfined Domains and the unconfined_domain_type Attribute When writing policy lines like the following line aren t generally considered a problem as unconfined domains are allowed full access to the system. However it can be an issue if you have a process in an unconfined domain without root access, which means a regular user login. Unfortunately this happens to be where this exploit and the default Debian SE Linux configuration intersect.
allow unconfined_domain_type domain:alg_socket   accept append bind connect create getattr getopt ioctl listen lock map name_bind read recvfrom relabelfrom relabelto sendto setattr setopt shutdown write  ;
The following shell code gets a list of unconfined domains which can be entered from user domains.
A=""
for n in $(seinfo -x -a unconfined_domain_type grep _t$) ; do
  A="$A ($n)"
done
A=$(echo $A sed -e s/^.//)
sesearch -T -s user_application_exec_domain -c process egrep "$A;"
Below is the output on a Debian/Trixie (Stable) system. So a confined user in the user_t domain could run an X server and try and get it to run the exploit code (which seems difficult) or running a Wine or Mono program from the Window manager in a Wayland environment.
type_transition user_t xserver_exec_t:process xserver_t;
type_transition user_wm_t mono_exec_t:process mono_t;
type_transition user_wm_t wine_exec_t:process wine_t;
type_transition user_wm_t xserver_exec_t:process xserver_t;
The issue of unconfined domains in SE Linux policy needs much more work. I ll write some blog posts about it later and the next release of Debian will be significantly better in this regard. Daemons that Have Access
allow NetworkManager_t NetworkManager_t:alg_socket   accept bind create read setopt write  ;
allow bluetooth_t bluetooth_t:alg_socket   accept append bind connect create getattr getopt ioctl listen read setattr setopt shutdown write  ;
Network Manager is something that can potentially be exploited by a desktop user as it has a large attack surface for the desktop interface. But as the vast majority of desktop user accounts are unconfined that s not an issue. This might be an issue for some restricted desktop PCs, maybe kiosk systems and those PCs that were being installed in prisons. The bluetooth_t domain is used by the bluetooth daemon that runs as root. While we generally are less concerned about a root process being exploited the daemon will handle some data from hostile sources and it could be used as an escalation attack by someone with a hostile Bluetooth device. These can t be exploited without another bug. The Lines that Aren t Problems The getattr Lines
allow devicekit_disk_t domain:alg_socket getattr;
allow sosreport_t domain:alg_socket getattr;
allow sysadm_t domain:alg_socket getattr;
The above getattr access isn t an issue as it just allows seeing process information, and it s also by privileged domains. The init_t Sockets
allow daemon init_t:alg_socket   getattr getopt ioctl read setopt write  ;
The daemon access to sockets inherited from init_t probably isn t a great idea, it s from the following section in init.te which is to allow socket activation for daemons, the comment is concerning in this context. Also socket_class_set is overly broad as without even inspecting the systemd source code I m pretty sure that far fewer than 1/3 of the 55 classes allowed by that rule are actually supported in systemd.
ifdef( init_systemd', 
        # Until systemd is fixed
        allow daemon init_t:socket_class_set   getattr getopt ioctl read setopt write  ;
But that s not really a problem as systemd has to just not create a socket of that type, if a hostile party can make systemd create such sockets then you have probably already lost. SE Linux Protection Overall SE Linux systems running confined users (kiosks and other confined GUI environments) will be protected barring a bug in Network Manager or the Bluetooth daemon as long as there is no Xserver installed (or the X server won t run scripts on startup), no Wine system installed, and no Mono. SE Linux servers and VMs will be protected against daemon issues as long as the daemon isn t unconfined. To convert the default login to user_t run the following commands:
semanage login -m -s user_u -r s0 __default__
restorecon -R -v -F /home
But it is still possible to access an unconfined domain from user_t (a topic I will address in detail in a future blog post). To remove unconfined entirely (not a task for novices or something to be done on in production without testing and planning) run the following commands:
semanage login -m -s root -r s0 root
# logout and login again
semodule -X 100 -r unconfined
Then a Debian/Trixie system running SE Linux will be safe against this attack even when running a vulnerable kernel. If you still want to use root as unconfined_t but still have untrusted shell users then run the following command to remove the easiest ways for users to run a program in an unconfined domain:
semodule -X 100 -r mono wine
Success and Failure Blocked by SE Linux Below is what happens on stdout/stderr when SE Linux blocks the exploit (tested with vulnerable Debian kernel 6.12.74+deb13+1-amd64):
test@testing1:~$ python3 ./copy_fail_exp.py 
Traceback (most recent call last):
  File "/home/test/./copy_fail_exp.py", line 9, in <module>
    while i<len(e):c(f,i,e[i:i+4]);i+=4
                   ~^^^^^^^^^^^^^^
  File "/home/test/./copy_fail_exp.py", line 5, in c
    a=s.socket(38,5,0);a.bind(("aead","authencesn(hmac(sha256),cbc(aes))"));h=279;v=a.setsockopt;v(h,1,d('0800010000000010'+'0'*64));v(h,5,None,4);u,_=a.accept();o=t+4;i=d('00');u.sendmsg([b"A"*4+c],[(h,3,i*4),(h,2,b'\x10'+i*19),(h,4,b'\x08'+i*3),],32768);r,w=g.pipe();n=g.splice;n(f,w,o,offset_src=0);n(r,u.fileno(),o)
  File "/usr/lib/python3.13/socket.py", line 233, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^PermissionError: [Errno 13] Permission denied
test@testing1:~$ su
Password:
When the attack is blocked by SE Linux there will be no messages in the kernel message log but the SE Linux audit log (typically stored in /var/log/audit/audit.log) will have lines like the following:
type=AVC msg=audit(1777803068.070:76): avc:  denied    create   for  pid=811 comm="python3" scontext=user_u:user_r:user_t:s0 tcontext=user_u:user_r:user_t:s0 tclass=alg_socket permissive=0
type=SYSCALL msg=audit(1777803068.070:76): arch=c000003e syscall=41 success=no exit=-13 a0=26 a1=80005 a2=0 a3=0 items=0 ppid=791 pid=811 auid=1000 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=1 comm="python3" exe="/usr/bin/python3.13" subj=user_u:user_r:user_t:s0 key=(null)ARCH=x86_64 SYSCALL=socket AUID="test" UID="test" GID="test" EUID="test" SUID="test" FSUID="test" EGID="test" SGID="test" FSGID="test"
type=PROCTITLE msg=audit(1777803068.070:76): proctitle=707974686F6E33002E2F636F70795F6661696C5F6578702E7079
For that the :76 is the audit log entry number, the command ausearch -i -a 76 will interpret that message with the following output:
type=PROCTITLE msg=audit(05/03/26 10:11:08.070:76) : proctitle=python3 ./copy_fail_exp.py 
type=SYSCALL msg=audit(05/03/26 10:11:08.070:76) : arch=x86_64 syscall=socket success=no exit=EACCES(Permission denied) a0=alg a1=SOCK_SEQPACKET a2=ip a3=0x0 items=0 ppid=791 pid=811 auid=test uid=test gid=test euid=test suid=test fsuid=test egid=test sgid=test fsgid=test tty=pts0 ses=1 comm=python3 exe=/usr/bin/python3.13 subj=user_u:user_r:user_t:s0 key=(null) 
type=AVC msg=audit(05/03/26 10:11:08.070:76) : avc:  denied    create   for  pid=811 comm=python3 scontext=user_u:user_r:user_t:s0 tcontext=user_u:user_r:user_t:s0 tclass=alg_socket permissive=0 
When it Works Below is what happens when it works (again tested with Debian kernel 6.12.74+deb13+1-amd64):
test@testing1:~$ python3 ./copy_fail_exp.py 
# 
Here is the kernel log when the attack works:
[   30.441830] alg: No test for authencesn(hmac(sha256),cbc(aes)) (authencesn(hmac(sha256-avx2),cbc-aes-aesni))
[   30.447466] process 'su' launched '/bin/sh' with NULL argv: empty string added
When the Kernel Isn t Vulnerable If the kernel isn t vulnerable and SE Linux permits the attack (EG run from an unconfined domain) the following is seen on stdout/stderr:
$ python3 ./copy_fail_exp.py 
Password: 
su: Authentication failure
In that situation the kernel will log something like the following:
[   36.647023] alg: No test for authencesn(hmac(sha256),cbc(aes)) (authencesn(hmac-sha256-lib,cbc-aes-aesni))
This was tested on the Debian/Unstable kernel 6.19.13+deb14-amd64. Conclusion Run the following commands and then force all users to logout to make a Debian SE Linux system offering shell access reasonably safe against this bug. But also upgrade your kernel as soon as convenient because having multiple layers of protection is always good.
semanage login -m -s user_u -r s0 __default__
restorecon -R -v -F /home
semodule -X 100 -r mono wine
The GrapheneOS people are doing really good work on securing phones, I am most interested in Mobian (Debian on phones) but for people who have made different choices GrapheneOS is a good option. Here is the GrapheneOS statement on Copy Fail (they are not vulnerable to it) [3]. For people interested in running a secure Android build GrapheneOS is the best option. Their supported devices list shows Pixel 6 to Pixel 10 supported and Pixel 8 to Pixel 10a recommended [4]. In Australia Kogan sells refurbished Pixel 6 phones starting at $251 including delivery and refurbished Pixel 8 phones starting at $499 with First membership, they seem to have the cheapest Pixel phones. I want to make Debian more like Android in terms of security, but that s a topic for other blog posts. Here is the Debian page listing kernels that have been fixed against this exploit [5].

3 May 2026

Jelmer Vernoo : Inquest, a test result repository in Rust

testrepository For a long time I ve used Robert Collins testrepository (testr) to run tests in many of the projects I work on. It s a small, focused tool built around a simple idea: decouple the running of tests from the recording and querying of their results. The way it works is straightforward. A test runner emits a subunit stream a compact binary protocol for test results and testrepository stores those streams in a per-project .testrepository/ directory. Once results are in the repository, you can ask questions like which tests failed in the last run? , re-run only the failures , what are the slowest tests? , or what changed between this run and the previous one? . The killer feature, for me, has always been the failing-test loop. When a big test suite breaks, you don t want to re-run the whole thing after every fix you want to iterate on just the failures, and only re-run the full suite once they re all green. testrepository made that workflow ergonomic long before most language-specific test runners had anything comparable, and many of them still don t have a good answer for it. testrepository has served me well for over a decade, but it has been largely unmaintained for a while, and I had some ideas of improvements that I wanted to try out. So I wrote a Rust port, which has since grown a number of features of its own.
Inquest Inquest is a Rust port of testrepository that has since grown a number of features of its own. The binary is called inq.
Goals The goals are deliberately modest:
  • a single static binary, no Python runtime required
  • no need to write a dedicated config file for most projects
  • compatible enough with testrepository s workflow that I can switch projects over without retraining my fingers
  • a richer on-disk format that captures more about each run (git commit, command line, duration, exit code, concurrency)
  • good support for the languages I actually use day-to-day: Rust, Python, Go, and Node.js
  • mostly Do What I Mean (DWIM), e.g. getting me to know as quickly as possible what tests are failing and why, and being clever about doing this
Inquest reads and writes subunit v2 streams, so anything that can produce subunit (directly or via one of the many converters) can feed into it.
Quick start Inquest can usually figure out how to run your tests on its own. In a Rust, Python, Go or Node.js project:
 $ cd my-project
 $ inq
Or if the auto-detection doesn t work, you can ask it to generate a config file and then run the tests:
 $ inq auto
 $ inq run
inq auto writes an inquest.toml describing how to invoke the test runner; inq run runs the tests, captures the subunit stream, and stores the results in a .inquest/ directory. For a Rust project the generated config looks like:
 test_command = "cargo subunit $IDOPTION"
 test_id_option = "--test $IDFILE"
 test_list_option = "--list"
After the first run, the usual queries work:
 $ inq stats             # repository-wide statistics
 $ inq last              # results of the most recent run
 $ inq failing           # only the failing tests
 $ inq slowest           # the slowest tests in the last run
 $ inq run --failing     # re-run only what failed last time
The last one is the workflow I use most often: run the full suite once, fix the obvious failures, then iterate on inq run --failing until the list is empty.
A few things that aren t in testrepository Some of the features that have grown in inquest beyond the original testrepository functionality:
  • Timeouts. --test-timeout, --max-duration, and --no-output-timeout will kill a test process that is hanging or has stopped producing output. --test-timeout auto derives a per-test timeout from the historical duration of that test, which is handy for catching tests that hang. Once the test runner is killed, the test is marked as failed and the next test is started, so a broken test doesn t hold up the whole suite.

  • Ordering --order can be used to run tests in a specific order, e.g. to run the slowest tests first, to run the tests that failed most recently first, or to run the widest variety of tests first to maximize the chance of finding a failure early on.

  • Live progress. inq running tails the in-progress subunit stream on disk and reports observed/expected test counts, percent complete, elapsed wall-clock time, and an ETA derived from each test s historical duration. Useful when a CI run is taking longer than you d like.

  • Flakiness ranking. inq flaky ranks tests by pass fail transitions in consecutive runs in which the test was recorded, so chronically broken tests rank low and genuinely flapping tests rank high.

  • Comparing runs. inq diff <A> <B> shows what changed between two test runs newly failing, newly passing, and tests that flipped state which makes it easy to see whether your last change actually fixed (or broke) anything.

  • Bisecting git history. inq bisect <TEST> drives git bisect to find the commit that broke a given test. It defaults the known-good and known-bad commits from the recorded run history (the most recent run where the test passed, and the most recent where it failed), so in the common case there is no need to remember either just point it at the test name and let it work.

  • Richer run metadata. inq info shows the git commit, command line, duration, exit code, and concurrency for a run, with a flag for whether the working tree was dirty when the run started. Combined with inq diff this makes it much easier to triangulate when a regression was introduced.

  • Rerun a previous run verbatim. inq rerun <ID> re-runs exactly the tests of a previous run, in the same order, forwarding the same -- arguments that the original run used. inq rerun -1 repeats the latest.

  • Web based view. inq web serves a web-based view of the repository, with a dashboard of recent runs and detailed views of individual runs and tests.

Web UI Most of the time I drive inquest from the command line, but for browsing historical results of a large suite spotting flapping tests, drilling into a single test s run history, or just getting a visual sense of which parts of the suite are hurting a web view is more pleasant. inq web starts a local server with exactly that:
 $ inq web
The repository overview shows totals and a per-test history grid where each cell is one run, coloured by outcome. Bands of red make it easy to pick out tests that have been broken for a long time, and isolated red cells in an otherwise green column point at flaky tests. Inquest web UI repository overview, with a grid of per-run results Drilling into an individual test gives you its full run history, a duration sparkline, and per-run pass/fail status: Inquest web UI per-test view with run history and duration sparkline
Migrating from testrepository If you already have a .testrepository/ directory full of historical runs, inq upgrade will migrate it into the new .inquest/ format, with a progress bar for the impatient. The legacy .testr.conf (INI) format is still understood, so existing projects don t have to be converted to inquest.toml immediately though the TOML format is preferred for new projects.
Trying it The source is on GitHub at jelmer/inquest. To install from source:
 $ cargo install inquest
In a project with a Rust, Python, Go or Node.js test suite:
 $ inq
Bug reports and patches are welcome.

2 May 2026

Ben Hutchings: FOSS activity in April 2026

1 May 2026

Dirk Eddelbuettel: binb 0.0.8 on CRAN: Maintenance

The eight release of the binb package, and first in two years, is now on CRAN and in r2u. binb regroups four rather nice themes for writing LaTeX Beamer presentations much more easily in (R)Markdown. As a teaser, a quick demo combining all four themes is available; documentation and examples are in the package. This release contains regular internal updates to continuous integration, URLs reference and switch to Authors@R. The trigger for the release, though, was a small updated need when very recent pandoc versions (as shipped with RStudio) are used which require a new variable declaration in the LaTeX template files in order to process uncaptioned tables. The summary of changes follows.

Changes in binb version 0.0.8 (2026-05-01)
  • Small updates to documentation URLs and continuous integration
  • The package now uses Authors@R in DESCRIPTION
  • Newer pandoc versions are accommodated by adding a required counter variable in the latex template file

CRANberries provides a summary of changes to the previous version. For questions or comments, please use the issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub. You can also sponsor my Tour de Shore 2026 ride in support of the Maywood Fine Arts Center.

30 April 2026

Otto Kek l inen: Mentoring Mondays for aspiring Debian contributors

Featured image of post Mentoring Mondays for aspiring Debian contributorsI mentor several people in Debian, and have been repeatedly asked to offer an opportunity to ask questions on a live call. I have now started a recurring video call for exactly that, which I call Mentoring Mondays, and it is open for anyone aspiring to contribute to Debian, one of the oldest and most widely used Linux distributions. Mentoring Mondays have already been happening for the past few Mondays, and this week we had a record 20 people on the call. During the calls so far we have had a demo of updating a package for a new upstream release using gbp, and of how to create a Merge Request on Salsa for a new upstream version. There is clearly a need for this, so I am announcing this now also on my blog, just as I have publicly announced that I offer mentoring for aspiring Debian contributors.

What is Mentoring Mondays? Mentoring Mondays is a recurring video call that lasts roughly 45 minutes with the agenda:
  • Weekly walk-through: demo of something in Debian packaging, explaining the what and why
  • Discussion on pros and cons to help participants develop their judgment
  • Questions & answers on Debian packaging, or open source contributing in general
This is ideal for you if you:
  • Have built a .deb package at least once and want to do it better
  • Are stuck on a specific Debian packaging bug and need guidance
  • Want to understand how Debian Developers actually work day-to-day
The call is mainly intended for those who want to contribute to Debian (or Debian derivatives, with Ubuntu being the most popular), but anyone can join to learn about things related to contributing to a Linux distribution. Please note that video chat uses Debian Social Jitsi. Joining the call requires authentication using a Salsa account, which anyone contributing to Debian should have anyway. Calls are not recorded, so participants can chat freely, and are also encouraged to be on-camera for an enhanced sense of community.

Next call: Monday May 4th, 2026 Make sure you are logged into Salsa first, before opening the call on Debian s Jitsi instance.

Matrix channel and future meeting time announcements Please join the Matrix channel #mentoringmondays:matrix.debian.social if you plan to attend Mentoring Mondays. All future meeting times will be announced there. It is also the channel to post questions about Debian packaging to be answered during the call. The current meeting time is friendly to people in Europe, Asia and Australian time zones, and will repeat at the same time slot on:
  • May 11th, 2026 at 12:30 UTC
  • May 18th, 2026 at 12:30 UTC
  • May 25th, 2026 at 12:30 UTC
  • June 1st, 2026 at 12:30 UTC
  • June 8th, 2026 at 12:30 UTC
Starting in mid-June the meeting time will change to accommodate participation in different time zones.

Spread the word Feel free to extend the invite to anyone you think might be interested in joining! If you mention this on social media, please post using tag #mentoringmondays, or simply boost the existing posts on the social media of your preference: Mastodon, Lemmy, Reddit, Bluesky, LinkedIn, Farcaster, X.

Thanks A big thanks to Jason Kregting for helping organize. I would also like to thank in advance all the Debian Developers who are able to join the call and be available to participate in discussions and help answer questions.

27 April 2026

Gunnar Wolf: Heads we win, tails you lose AI detectors in education

This post is an unpublished review for Heads we win, tails you lose AI detectors in education
Educators throughout the world are tasked with the difficult requirement of evaluating students works, making sure the grades meaningfully reflect the students understanding of the subject, and that a graded assignment maps to the relevant work invested in solving it. After the irruption of Large-Language Models in late 2023, this task became obviously much harder: if a widely available computer program is able to solve an assignment in a way that resembles a human-generated response, how can educators meaningfully grade their groups? As it has been the case with different innovations over time (such as with the appearance of electronic calculators or the mass availability of digital encyclopedias), the first reactions were of prohibition and denial: students who use the new tool in question are to be disqualified or somehow punished. It is only some time after the innovation in question settles that teachers find a way to properly weigh, integrate and accept its use. The authors of this position article present several arguments as to why it is impossible, unethical and unadvisable to use automated AI detection systems to process student assignments. The first argument is whether it is at all possible to reliably differentiate human-written essays from LLM-generated artifacts. The first criticism is that AI detectors are, themselves, LLMs trained on human-generated texts (negative) and LLM-generated texts (positive). However, the only way to assert the training material is not noisy is to use pre-2020 text as human-generated but natural ways of writing are influenced by what people read, and the authors quote studies pointing out that human language, particularly in the scholarly fields, has incorporated terms and constructions that were used as LLM markers. Quoting the authors, As exposure to AI-generated material becomes increasingly widespread, it is reasonable to expect that the linguistic patterns of human writing will shift, reflecting the influence of AI-assisted texts encountered across education, media, and everyday communication . Stylistic elements and other such markers are being adopted back into regular speech at a high rate. Then, the aspect of ethics comes into play as well. While it is expected that teachers should demand intellectual integrity from students, and plagiarism detectors have been widely accepted into the workflow of academics, the accusation of presenting LLM output as own work is necessarily an uphill battle: the accused party is tasked with providing proof of innocence based on nebulous, probabilistic accusations. The authors argue, once an accusation of turning in a LLM-generated text is made on a student, the onus on proving innocence lies with the accused. The authors review and argue against a series of techniques that have been presented in literature to aid teachers in detecting LLM abuse, such as linguistic markers, single or multiple AI detectors, the use of false references, hidden adversarial prompts, arguing in all cases the techniques fail to be trustable enough and highlighting the probability of both false positives and negatives. They also present AI detection as a false dichotomy: many works presented are not 100% human generated nor 100% LLM-generated, but some pertinent LLM-generated paragraphs are presented mixed with human-generated content, in a positive, critical AI use ( Students work is frequently created with, not by, generative AI ). The article closes by reiterating the authors position: AI detection in education is not merely flawed; it is conceptually unsound . they call upon institutions to accept the use of generative LLMs cannot be solved through surveillance and punishment , but has to be tackled by an assessment design that recognizes AI s role in learning . This article s position is very strong and well argued, and although it will surely meet with ample opposition, it surely poses an important, very current problematic. As a teacher, I found it a very enlightening read.

Russ Allbery: Review: What We Are Seeking

Review: What We Are Seeking, by Cameron Reed
Publisher: Tor
Copyright: 2026
ISBN: 1-250-36474-4
Format: Kindle
Pages: 339
What We Are Seeking is a bit hard to classify beyond science fiction. I think I would call it anthropological science fiction, but it's also a first contact story and a planetary colony story. It is a standalone novel (well, so far as I know; see later in the review for caveats). This is Cameron Reed's second novel after the excellent and memorable cyberpunk novel The Fortunate Fall, first published in 1996 under Reed's former name of Raphael Carter. John Maraintha is a doctor from the world of Essius. He took what he thought was a temporary job on the Free Ship Edgar's Folly, where he's endured considerable culture shock. As the novel opens, John learns that the colonists on Scythia have requested a translator to talk to one of the native life forms, and a doctor since they're down to only one. John will be that doctor. The captain has decided, and by the rules of the free ships, John does not get a choice in the matter. The Scythian colony is about four hundred people, now located in a desert climate since the complex native life forms destroyed their previous settlement. The colonists are a split between Ischnurans and Zandaheans, two other human civilizations from the scatter of colony worlds left after Earth embraced AIs (aiyis here) and turned inward. Both of those groups marry, something John considers a moral abomination. Neither of them seem likely to understand Essian sexual ethics. More devastatingly, John had intended to spend some time as a ship doctor and then return home to a new place in Essian society. Once he lands on Scythia, the chances of that are gone; it is highly unlikely any ship would pick him up again and take him home. I have been trying to find the right books to compare What We Are Seeking with ever since I read it. The best I've come up with are Ursula K. Le Guin (particularly The Dispossessed), Eleanor Arnason's A Woman of the Iron People, and Becky Chambers's To Be Taught, If Fortunate. The start of the book felt like an intentional revisiting of an earlier era of science fiction, with somewhat updated science and politics, but the last half of the book, where the action picks up considerably, is a meditation on gender, social systems, religion, and small-group politics. All of that is mixed with biological exploration and a first-contact story with some quite-alien aliens. This is the sort of novel where the protagonist's culture is as foreign to the reader as any of the other cultures he counters, so the reader is assembling several jigsaw puzzles at once. John is dropped into an established colony with its own social norms and established hierarchies. The one other outsider, the translator Sudharma Jain, is, as his name implies, a Jain who keeps very strict religious observances. Half of the colony is from something akin to a fundamentalist Christian religious sect that practices patriarchy and strict marriage codes. The other half is more gently sexist (but still sexist) and has its own tradition of a third gender that becomes central to the story. John, meanwhile, is a strong believer in the Essian approach to social organization: Any two partners of any gender freely have sex by mutual consent and without obligation, and family is based solely on blood relations. These beliefs do not fit comfortably together, even when people are trying (as they mostly do) to be welcoming. The first half of this book is very slow. This gives all of the characters space to breathe and become comfortable, and the characterization is superb, but it is a book to start when you're in the mood for something slow and observational. There is a plot that gradually becomes apparent, or rather there are several plots that are intertwined, but tension and urgency are mostly reserved for the second half of the book. Instead, the book opens with a lot of close observation of alien flora and fauna and the untangling of subtle social dynamics among the Scythians. There is also a visitor from earth, much to the distress of the Scythians. Earth presence means the ships will not return and the colony may be cut off from any sort of technological resupply. Despite speaking a common language, that visitor is as mutually alien to the other groups as they are to the native flora. Her life is fully integrated with aiyis, giving her essentially godlike powers and the ability to turn off inconvenient emotions and disregard anything she doesn't want to see. What she and the Earth aiyis are doing on the planet is one of the early mysteries. The dialogue in this book is truly excellent. Each characters has their own voice, there are fascinating digressions on different words that lead to tidbits of world-building, and some of the culture-specific idioms are delightful.
"I'm making a mess of this. None of that matters. Let me fall out the window and come in the door again. This is how my story ought to start:"
The challenges for the characters in this story are slow but deep ones: belonging and self-definition, the conflict between cultural tradition and personal circumstance, and the sacrifices required to live with small groups in situations where civil war is viscerally attractive. It has one of the most comprehensive and fascinating treatments of transgender issues that I've read in science fiction. Its commentary on current politics is subtle and estranged in the way that science fiction does best, but still pointed and satisfying. And, well, there are passages like this that I absolutely adore:
"I wouldn't go that far. It could be they are right, the universe we see exists because a mind like ours created it at least, a mind enough like ours that we can say it wants one thing and not another, and when it acts it does so with intent. That's as good an idea as any. But it is certainly not plausible that such a being believes that people everywhere should marry, or that men should never visit men, or no one should become a jess. Look at what they have created. The universe could have been nothing at all, or one atom of hydrogen floating in a void, or a diamond crystal infinite in all directions, if their mind cared for simplicity or tidiness. Instead we have stars and planets and black holes and nebulas. It could have all been cold and dead, but there is life. They could have made one species for each world, or just a few, which could have stayed the same forever, but instead we have millions and millions, all of which are changing every moment, varying among themselves and boiling off in all directions. Such a god is like an artist who fills up a library of sketchbooks with their drawings of strange creatures, and when every scrap of paper in the place is used up, goes back with a different color ink and scribbles over them again. They are obsessed with variation they gorge themselves with it and never grow full. Do you really think a mind like that could want us all to live in the same way?"
I had one problem with this book, though, and for me it was a big one: There is no ending. Reed effectively builds tension, gets me caring about all of the characters, sets up several problems, starts down a path towards resolution, and then the book just... ends. Long-time readers of my reviews will know that I'm a denouement fanatic. I want the scouring of the shire, I want the chapter set in the happily ever after, I want the catharsis of an ending. This made me so grumpy! To be clear, this is not sequel bait (at least so far as I can tell). I can write a philosophical defense of the ending. The types of problems and lives that Reed set up don't have clear endings; this is, to some extent, the point. We muddle through, and then those who come after us muddle through some more, and the cumulative effect is called human civilization. And there is some denouement; Reed doesn't leave the reader at a cliffhanger or anything that egregious. But still, I wanted the happy ending, even though that was unrealistic for the style of story this is, because I'm a happy ending reader. This is not an ending sort of book; it's the sort of book where I get a sinking feeling at the 95% mark because there aren't enough pages left for the number of remaining unresolved problems. I've gotten less annoyed in the days since I finished the book, and I can appreciate the thematic point made by how the book ends, but I still feel like it's worth an advance warning if you're a reader like I am. I would be delighted by a sequel, but it didn't feel like that was the intent. Apart from that, this was both excellent and rather unlike a lot of current science fiction. I think the closest comparison I can make among recent novels I've read is Sue Burke's Semiosis. What We Are Seeking has a similar sort of world-building, but I liked these characters so much more. It felt like a classic literary science fiction novel, but very much written in 2026. Highly recommended, just beware of the lack of closure. Content notes: Sexism, homophobia, stomach illness, and some religious abuse. Rating: 8 out of 10

24 April 2026

Freexian Collaborators: Monthly report about Debian Long Term Support, March 2026 (by Santiago Ruano Rinc n)

The Debian LTS Team, funded by [Freexian s Debian LTS offering] (https://www.freexian.com/lts/debian/), is pleased to report its activities for March.

Activity summary During the month of March, 20 contributors have been paid to work on Debian LTS (links to individual contributor reports are located below). The team released 24 DLAs fixing 250 CVEs. We also welcomed two new members: Lukas M rdian and Emmanuel Arias to the team, who actually started to contribute to the LTS project several months ago. The team continued preparing security updates in its usual rhythm. Beyond the updates targeting Debian 11 ( bullseye ), which is the current release under LTS, the team also proposed updates for more recent releases (Debian 12 ( bookworm ) and Debian 13 ( trixie )), including Debian unstable. We highlight several notable security updates here below.
  • ansible (DLA 4502-1), prepared by Lee Garret in collaboration with Jochen, fixing a vulnerability that allows attackers to bypass unsafe content protections
  • asterisk (DLA 4515-1), prepared by Lukas M rdian, fixing four CVEs that include possible privilege escalations.
  • gimp (DLA 4500-1), prepared by Thorsten, fixing four CVEs related to denial of service or execution of arbitrary code.
  • gst-plugins-base1.0 and gst-plugins-ugly1.0 (DLA-4514-1, DLA-4516-1, respectively), both prepared by Utkarsh, addressing vulnerabilities that may yield to arbitrary code execution.
  • imagemagick, released by Bastien Roucari s (DLA 4497-1) fixing multiple vulnerabilities that could lead to information leaks, bypass of security policies, denial of service or arbitrary code execution.
  • libpng1.6 (DLA 4521-1), prepared by Tobias Frost, fixing an arbitrary code execution vulnerability
  • linux: Ben Hutchings released DLA 4498-1 and DLA 4499-1 for linux 5.10 and linux 6.1, respectively. Those updates especially address the CrackArmor flaw.
  • ruby-rack (DLA 4505-1), prepared by Utkarsh Gupta, addressing two vulnerabilities
  • strongswan (DLA 4512-1), prepared by Thorsten Alteholz, fixing a Denial of Service vulnerability
  • roundcube (DLA 4517-1) prepared by Guilhem Moulin, who discovered that one of the fixes provided by upstream was incomplete.
Contributions from outside the LTS Team: As usual, the thunderbird update, released as DLA 4511-1, was prepared by its maintainer Christoph Goehre. Thanks a lot for his continuous contributions. The LTS Team has also contributed with updates to the latest Debian releases:
  • Andreas Henriksson completed the uploads of glib2.0 for both trixie and bookworm
  • Arnaud Rebillout: python-cryptography for trixie
  • Arnaud and Bastien worked together to prepare a ca-certificates-java release for unstable
  • Bastien completed the upload of gpsd for trixie that was proposed in January.
  • Bastien uploaded a regression update of apache2 for trixie
  • Bastien prepared a zabbix point update for trixie
  • Bastien in collaboration with Markus released netty updates for trixie and bookworm DSA 6160-1
  • Daniel Leidert proposed python-tornado releases for both trixie and bookworm.
  • Daniel also prepared a python-authlib update for trixie
  • Guilhem prepared a mapserver update for bookworm.
  • Lucas Kanashiro proposed merge requests to fix three CVEs in erlang for both trixie and bookworm
  • Sylvain Beucler continued the work to replace p7zip with 7zip in the different supported releases, and proposed a point update for bookworm
  • Tobias prepared trixie and bookworm security updates, released as DSA-6189-1
  • Utkarsh prepared trixie and bookworm security update for ruby-rack, released as DSA-6180-1

Individual Debian LTS contributor reports

Thanks to our sponsors Sponsors that joined recently are in bold.

23 April 2026

Sergio Talens-Oliag: Developing a Git Worktree Helper with Copilot

Over the past few weeks I ve been developing and using a personal command-line tool called gwt (Git Worktree) to manage Git repositories using worktrees. This article explains what the tool does, how it evolved, and how I used GitHub Copilot CLI to develop it (in fact the idea of building the script was also to test the tool).

The Problem: Managing Multiple BranchesI was working on a project with multiple active branches, including orphans; the regular branches are for fixes or features, while the orphans are used to keep copies of remote documents or store processed versions of those documents. The project also uses a special orphan branch that contains the scripts and the CI/CD configuration to store and process the external documents (it is on a separate branch to avoid mixing its operation with the main project code). The plan is trigger a pipeline against the special branch from remote projects to create or update the doc branch for it in our git repository, retrieving artifacts from the remote projects to get the files and put them on an orphan branch (initially I added new commits after each update, but I changed the system to use force pushes and keep only one commit, as the history is not really needed). The original documents have to be changed, so, after ingesting them, we run a script that modifies them and adds or updates another branch with the processed version; the contents of that branch are used by the main branch build process (there we use git fetch and git archive to retrieve its contents). When working on the scripts to manage the orphan branches I discovered the worktree feature of git, a functionality that allows me to keep multiple branches checked out in parallel using a single .git folder, removing the need to use git switch and git stash when changing between branches (until now I ve been a heavy user of those commands). Reading about it I found that a lot of people use worktrees with the help of a wrapper script to simplify the management. After looking at one or two posts and the related scripts I decided to create my own using a specific directory structure to simplify things. That s how I started to work on the gwt script; as I also wanted to test copilot I decided to build it using its help (I have a pro license at work and wanted to play with the cli version instead of integrated into an editor, as I didn t want to learn a lot of new keyboard shortcuts).

The gwt Philosophy: Opinionated and Transparentgwt enforces a simple, filesystem-visible model:
  • Exactly one bare repository named bare.git (treated as an implementation detail)
  • One worktree directory per branch where the directory name matches the branch name
  • Single responsibility: gwt doesn t try to be a general git wrapper; it only handles operations that map cleanly to this layout
The repository structure looks like this:
my-repo/
+-- bare.git/           # the Git repository (internal)
+-- main/               # worktree for branch "main"
+-- feature/api/        # worktree for branch "feature/api"
+-- fix/docs/           # worktree for branch "fix/docs"
+-- orphan-history/     # worktree for the "orphan-history" branch
The tool follows five core design principles:
  1. Explicit over clever: Git commands are not hidden or reinterpreted
  2. Transparent execution: Every operation is printed before it happens
  3. Safe, preview-first operations: Destructive commands default to preview, confirmation, then apply
  4. Shell-agnostic core: The script never changes the caller s working directory (shell wrappers handle that)
  5. Opinionated but minimal: Only commands that fit the layout model are included

Core CommandsThe script provides these essential commands:
  • gwt init <url> Clone a repository and set up the gwt layout
  • gwt convert <dir> Convert an existing Git checkout to the gwt layout
  • gwt add [--orphan] <branch> [<base>] Create a new worktree (optionally orphaned)
  • gwt remove <branch> Remove a worktree and unregister it (asks the user to remove the local branch too, useful when removing already merged branches)
  • gwt rename <old> <new> Rename a branch AND its worktree directory
  • gwt list List all worktrees
  • gwt default [<branch>] Get or set the default branch
  • gwt current Print the current worktree or branch name
Except init and convert all of the commands work inside a directory structure that follows the gwt layout, which looks for the bare.git folder to find the root folder of the structure. As I don t want to hide which commands are really used by the wrapper, all git and filesystem operations pass through a single run shell function that prints each command before executing it. This gives complete visibility into what the tool is doing. Also, destructive operations (remove, rename) default to preview mode:
$ gwt remove feature-old --dry-run
+ git -C bare.git branch -d feature-old
+ git -C bare.git worktree remove feature-old/
Apply these changes? [y/N]:
The user sees exactly what will happen, can verify it s correct, and only then confirm execution.

Incremental Development with CopilotThe gwt script has grown from 597 lines in its original version (git-wt) to 1,111 lines when writing the first draft of this post. This growth happened through incremental, test-driven development, with each feature being refined based on real usage patterns. What follows is a little history of the script evolution written with the help of git log.

Initial versionFirst I wrote a design document and asked copilot to create the initial version of the git-wt script with the original core commands. I started to use the tool with a remote repostory (I made copies of the branches in some cases to avoid missing work) and fixed bugs (trivial ones with neovim, larger ones asking copilot to fix the issues for me, so I had less typing to do).

First command updateOne of the first commands I had to enhance was rename:
  • as I normally use branches with / on their name and my tool checks out the worktrees using the branch name as the path inside the gwt root folder (i.e. a fix/rename branch creates the fix directory and checks the branch inside the fix/rename folder) the rename command had to clean up the empty parent directories
  • when renaming a worktree we move the folders and fix the references using the worktree repair command to make things work locally, but the rename also affects the remote branch reference, to avoid surprises the command unsets the remote branch reference so it can be pushed again using the new name (of course, the user is responsible of managing the old remote branch, as the gwt can t guess what it should do with it).

Integration with the shellAs I use zsh with the Powerlevel10k theme I asked copilot to help me add visual elements to the prompt when working with gwt folders, something that I would have never tried without help, as it would have required a lot of digging on my part on how to do it, as I never looked into it. The initial version of the code was on an independent file that I sourced from my .zshrc file and it prints on the right part of the prompt when we are inside a gwt folder (note that if the folder is a worktree we see the existing git integration text right before it, so we have the previous behavior and we see that it is a gwt friendly repo) and if we are on the root folder or the bare.git folder we see gwt or bare (I added the text because there are no git promts on those folders). I also asked copilot to create zsh autocompletion functions (I only use zsh, so I didn t add autocompletion for other shells). The good thing here is that I wouldn t have done that manually, as it would have required some reading to get it right, but the output of copilot worked and I can update things using it or manually if I need to. One thing I was missing from the script was the possibility of changing the working directory easily, so I wrote a gwt wrapper function for zsh that intercepts commands that require shell cooperation (changing the working directory) and delegates everything else to the core script. Currently the function supports the following enhanced commands:
  • cd [<branch>]: change into a worktree or the default one if missing
  • convert <dir>: convert a checkout, then cd into the initial worktree
  • add [--orphan] <branch> [<base>]: create a worktree, then cd into it on success
  • rename <old> <new>: rename a worktree, then cd into it if we were inside it
Note that the cd command will not work on other shells or if the user does not load my wrapper, but the rest will still work without the working directory changes.

Renaming the commandAs I felt that git-wt was a long name I renamed the tool to gwt, I could have done it by hand, but using copilot I didn t have to review all files by myself and it did it right (note that I have it configured to always ask me before doing changes, as it sometimes tries to do something I don t want and I like to check its changes as I have the files in git repos, I manually add the files when I like the status and if the cli output is not clear I allow it to apply it and check the effects with git diff so I can validate or revert what was done).

The convert commandAfter playing with one repo I added the convert subcommand for migrating existing checkouts, it seemed a simple task at first, but it took multiple iterations to get it right, as I found multiple issues while testing (in fact I did copies of the existing checkouts to be able to re-test each update, as some of the iterations broke them). The version of the function when this post was first edited had the following comment explaining what it does:
# ---------------------------------------------------------------------------
# convert - convert an existing checkout into the gwt layout
# ---------------------------------------------------------------------------
#
# Must be run from the parent directory of <dir>.
#
# Steps:
#   1. Read branch from the checkout's HEAD
#   2. Rename <dir> to <dir>.wt.tmp (sibling, same filesystem)
#   3. Create <dir>/ as the new gwt root
#   4. Move <dir>.wt.tmp/.git to <dir>/bare.git; set core.bare = true
#   5. Fix fetch refspec (bare clone default maps refs directly, no remotes/)
#   6. Add a --no-checkout worktree so git wires up the metadata and
#      creates <dir>/<branch>/.git (the only file in that dir)
#   7. Move that .git file into the real working tree (<dir>.wt.tmp)
#   8. Remove the now-empty placeholder directory
#   9. Move the real working tree into place as <dir>/<branch>
#  10. Reset the index to HEAD so git status is clean
#      (--no-checkout leaves the index empty)
#  11. Create <dir>/.git -> bare.git symlink so plain git commands work
#      from the root without --git-dir
#
# The .git file ends up at the same absolute path git recorded in step 5,
# so no worktree repair is needed. Working tree files are never modified.
The .git link was added when I noticed that I could run commands that don t need the checked out files on the root of the gwt structure, which is handy sometimes (i.e. a git fetch or a git log, that shows the log of the branch marked as default). After playing with commands that used the bare.git folder I updated the init and convert commands to keep the origin refs, ensuring that the remote tracking works correctly.

Improving the add commandWhile playing with the tool on more repos I noticed that I also had to enhance the add command to better handle worktree creation, depending on my needs. Right now the tool supports the following use cases:
  • if the branch exists locally or on origin, it just checks it out.
  • if the branch does not exist, we create it using the given base branch or, if no base is given, the current worktree (if we are in the root folder or bare.git the command fails).
  • as I needed it for my project, I added a --orphan option to be able to create orphan branches directly.

Moving to a single fileEventually I decided to make the tool self contained; I removed the design document (I moved the content to comments on the top of the script and details to comments on each function definition) and added a pair of commands to print the code to source for the p10k and zsh integration (autocompletion & functions), leaving everything in a single file. Now my .zshrc file adds the following to source both things:
# After loading the p10k configuration
if type gwt >/dev/null 2>&1; then
  source <(gwt p10k)
fi
[...]
# After loading autocompletion
if type gwt >/dev/null 2>&1; then
  source <(gwt zsh)
fi

VersioningAs I modified the script I found interesting to use CalVer-based versioning (the version variable has the format YYYY.mm.dd-r#) so I added a subcommand to show its value or bump it using the current date and computing the right revision number.

About the use of copilotAlthough I ve never been a fan of AI tools I have to admit that the copilot CLI has been very useful for building the tool:
  • Rapid prototyping: Each commit represented a small feature or fix that I could implement, test immediately in my actual workflow, and iterate on based on the result
  • Edge case handling: Rather than trying to anticipate every scenario upfront, I could ask Copilot how to handle edge cases as they appeared in real usage
  • Script refinement: Questions like "how do I clean up empty directories after a rename" or "how do I detect if I m inside a specific worktree" were quickly answered with working code
  • Shell integration: The Zsh wrapper and completion system grew from simple prototypes to sophisticated features, with each iteration informed by how I actually used the tool
For example, the convert command started as a simple rename operation, but evolved to also create a .git symlink and intelligently handle various migration scenarios all because I used it repeatedly and refined the implementation each time.

Self-Contained and Opinionatedgwt is deliberately opinionated:
  • Zsh & Powerlevel10k Integration: The tool includes built-in Zsh shell integration, accessed via source <(gwt zsh) and supports adding a prompt segment when using p10k, as described earlier.
  • Directory Structure: The bare.git directory name is non-negotiable. This is how gwt discovers the repository root from any subdirectory, and how the tool knows whether a directory is a gwt repository. The simplicity of this marker means the discovery mechanism is foolproof and requires no configuration.
  • No Configuration Files: gwt deliberately has no configuration. There are no .gwtrc files or config directories. This makes it portable; the tool works the same way everywhere, and repositories can be shared across systems without synchronizing configuration.

From Script to SystemWhat started as a small helper script for managing worktrees has become a complete system:
  1. Core script (gwt): 1,111 lines of pure shell, no external dependencies
  2. Shell integration: Zsh functions and completions
  3. Prompt integration: Powerlevel10k segment
  4. Documentation: Built-in help and design philosophy documentation
The script is self-contained, everything needed for the tool to work is in a single file. This makes it trivial to update (just replace the script) or audit (no hidden dependencies).

Development with AI supportDeveloping gwt with copilot taught me some things:
  • Incremental refinement works well for small tools: Each iteration informed the next, resulting in a tool that handles real use cases elegantly
  • Transparency is a feature: Making operations visible builds confidence and is easier to debug
  • Opinionated tools can be powerful: By constraining the problem space (one bare repo, one worktree per branch), the solution becomes simpler and more robust
  • Shell integration matters: The same core commands are easier to use when they can automatically change directories and provide completions
  • Real-world testing is essential: I wouldn t have discovered the need for automatic directory cleanup or context-aware cd behavior without actually using the tool daily

What was next?The tool is stable and handles my daily workflow well, so my guess is that I would keep using it and fixing issues if or when I found them, but I do not plan to include additional features unless I find a use case that justifies it (i.e. I never added support for some of the worktree subcommands, as it is easier to use the git versions if I ever needed them).

What really happenedWhile editing this post I discovered that I needed to add another command to it and fixed a bug (see below). With those changes and the inclusion of a license and copyright notice (just in case I distribute it at some point) now the script is 1,217 lines long instead of the 1,111 it had when I started to write this entry.

Submodule SupportWhen I converted this blog repository to the gwt format and tried to preview the post using docker compose, it failed because the worktree I was on didn t have the Git submodule initialized. My blog theme is included on the repository as a submodule, and when I used gwt to check out different branches in worktrees, the submodule was not initialized in the new worktrees. This led me to add new internal function and a gwt submodule command to handle submodule initialization; the internal function is called from convert and add (when converting a repo or adding a worktree) and the public command is useful to update the submodules on existing branches.

Path Handling with Branch Names Containing SlashesThe second discovery was a bug in how the tool handled branch names containing slashes (e.g., feature/new-api, docs/user-guide), the worktree directories are created with the branch name as the path, so a branch like feature/new-api would create two nested folders (feature and new-api inside it). However, there was a mismatch in how the zsh wrapper function resolved worktree paths (initially it used shell parameter expansion, i.e. rel="$ cwd#"$REPO_ROOT"/ "), versus how the core script calculated them, causing the cd command to fail or navigate to the wrong location when branch names contained slashes. The fix involved ensuring consistent path resolution throughout the script and wrapper (now it uses a function that processes the git worktree list output), so that gwt cd feature/new-api correctly navigates to the worktree directory regardless of path depth.

Conclusiongwt is a tool that solves a real problem: managing multiple Git branches simultaneously without context-switching overhead. I m sure I m going to keep using it for my projects, as it simplifies some workflows, although I ll also use switch and stash in some cases, but I like the use of multiple worktrees in parallel. In fact I converted this blog repository checkout to the gwt format to work on a separate branch as it felt the right approach even if I m the only one using the repo now, and it helped me improve the tool, as explained before. Also, it was a good example of how to use AI tools like copilot to develop a simple tool and keep it evolving while using it. In any case, although I find the copilot useful and has saved me time, I don t trust it to work without supervision, it worked well, but got stuck some times and didn t do the things as I wanted in multiple occasions. I also have an additional problem now I ve been reading about it, but I don t really know which models to use or how the premium requests are computed (I ve only been playing with it since last month and I ran out of requests the last day of the month on purpose, just to see what happened it stops working ;). On my work machine I ve been using a specific user account with a GitHub Copilot Business subscription and I only used the Anthropic Claude Sonnet 4.6 model and with my personal account I configured the Anthropic Claude Haiku 4.5 model, but I ve only used that to create the initial draft of this post (I ended up rewriting most of it manually anyway) and to review the final version (I m not a native speaker and it was useful for finding typos and improving the style in some parts). I guess I ll try other models with copilot in the future and check other command line tools like aider or claude-code, but probably only using free accounts unless I get a payed account at work, as I have with GitHub Copilot. To be fair, what I will love to be able to do is to use local models (aider can do it), but the machines I have are not powerful enough. I tried to run a simple test and it felt really slow, but when I have the time or the need I ll try again, just in case.

22 April 2026

Vincent Bernat: CSS & vertical rhythm for text, images, and tables

Vertical rhythm aligns lines to a consistent spacing cadence down the page. It creates a predictable flow for the eye to follow. Thanks to the rlh CSS unit, vertical rhythm is now easier to implement for text.1 But illustrations and tables can disrupt the layout. The amateur typographer in me wants to follow Bringhurst s wisdom:
Headings, subheads, block quotations, footnotes, illustrations, captions and other intrusions into the text create syncopations and variations against the base rhythm of regularly leaded lines. These variations can and should add life to the page, but the main text should also return after each variation precisely on beat and in phase. Robert Bringhurst, The Elements of Typographic Style

Text Three factors govern vertical rhythm: font size, line height and margin or padding. Let s set our baseline with an 18-pixel font and a 1.5 line height:
html  
  font-size: 112.5%;
  line-height: 1.5;
 
h1, h2, h3, h4  
  font-size: 100%;
 
html, body,
h1, h2, h3, h4,
p, blockquote,
dl, dt, dd, ol, ul, li  
  margin: 0;
  padding: 0;
 
CSS Values and Units Module Level 4 defines the rlh unit, equal to the computed line height of the root element. All browsers support it since 2023.2 Use it to insert vertical spaces or to fix the line height when altering font size:3
h1, h2, h3, h4  
  margin-top: 2rlh;
  margin-bottom: 1rlh;
 
h1  
  font-size: 2.5rem;
  line-height: 2rlh;
 
h2  
  font-size: 1.5rem;
  line-height: 1rlh;
 
h3  
  font-size: 1.25rem;
  line-height: 1rlh;
 
p, blockquote, pre  
  margin-top: 1rlh;
 
aside  
  font-size: 0.875rem;
  line-height: 1rlh;
 
We can check the result by overlaying a grid4 on the content:
Screenshot of my website with a grid as an overlay and each line of text fitting on the grid
Using CSS rlh unit to set vertical space works well for text. You can display the grid using Ctrl+Shift+G.
If a child element uses a font with taller intrinsic metrics, it may stretch the line s box beyond the configured line height.5 A workaround is to reduce the line height to 1. The glyphs overflow but don t push the line taller.
code, kbd  
  line-height: 1;
 

Responsive images Responsive images are difficult to align on the grid because we don t know their height. CSS Rhythmic Sizing Module Level 1 introduces the block-step property to adjust the height of an element to a multiple of a step unit. But most browsers don t support it yet. With JavaScript, we can add padding around the image so it does not disturb the vertical rhythm:
const targets = document.querySelectorAll(".lf-media-outer");
const adjust = (el, height) =>  
  const rlh = parseFloat(getComputedStyle(document.documentElement).lineHeight);
  const padding = Math.ceil(height / rlh) * rlh - height;
  el.style.padding =  $ padding / 2 px 0 ;
 ;
targets.forEach((el) => adjust(el, el.clientHeight));
Screenshot of my website with a grid as an overlay and an image not breaking the vertical rhythm. Additional padding is visible before and after the image. The height of the image with padding is 216.
The image is snapped to the grid thanks to the additional padding computed with JavaScript. 216 is divisible by 27, our line height in this example.
As the image is responsive, its height can change. We need to wrap a resize observer around the adjust() function:
const ro = new ResizeObserver((entries) =>  
  for (const entry of entries)  
    const height = entry.contentBoxSize[0].blockSize;
    adjust(entry.target, height);
   
 );
for (const target of targets)  
  ro.observe(target);
 

Tables Table cells could set 1rlh as their height but they would feel constricted. Using 2rlh wastes too much space. Instead, we use incremental leading: we align one in every five lines.
table  
  border-spacing: 2px 0;
  border-collapse: separate;
  th  
    padding: 0.4rlh 1em;
   
  td  
    padding: 0.2rlh 0.5em;
   
 
To align the elements after the table, we need to add some padding. We can either reuse the JavaScript code from images or use a few lines of CSS that count the regular rows and compute the missing vertical padding:
table:has(tbody tr:nth-child(5n):last-child)     padding-bottom: 0.2rlh;  
table:has(tbody tr:nth-child(5n+1):last-child)   padding-bottom: 0.8rlh;  
table:has(tbody tr:nth-child(5n+2):last-child)   padding-bottom: 0.4rlh;  
table:has(tbody tr:nth-child(5n+3):last-child)   padding-bottom: 0  
table:has(tbody tr:nth-child(5n+4):last-child)   padding-bottom: 0.6rlh;  
A header cell has twice the padding of a regular cell. With two regular rows, the total padding is 2 2 0.2+2 0.4=1.6. We need to add 0.4rlh to reach 2rlh of extra vertical padding across the table.
Screenshot of my website with a grid as an overlay and a table following the vertical rhythm. Additional padding is visible after the table. The height of the table with padding is 405.
One line out of five is aligned to the grid. Additional padding is added after the table to not break the vertical rhythm. 405 is divisible by 27, our line height in this example.

None of this is necessary. But once you start looking, you can t unsee it. Until browsers implement CSS Rhythmic Sizing, a bit of CSS wizardry and a touch of JavaScript is enough to pull it off. The main text now returns after each intrusion precisely on beat and in phase.

  1. See Vertical rhythm using CSS lh and rlh units by Pawe Grzybek.
  2. For broader compatibility, you can replace 2rlh with calc(var(--line-height) * 2rem) and set the --line-height custom property in the :root pseudo-class. I wrote a simple PostCSS plugin for this purpose.
  3. It would have been nicer to compute the line height with calc(round(up, calc(2.4rem / 1rlh), 0) * 1rlh). Unfortunately, typed arithmetic is not supported by Firefox yet. Moreover, browsers support round() only since 2024. Instead, I coded a PostCSS plugin for this as well.
  4. The following CSS code defines a grid tracking the line height:
    body  
      position: relative;
     
    body::after  
      content: "";
      position: absolute;
      inset: 0;
      z-index: 9999;
      background: linear-gradient(180deg, #c8e1ff99 1px, transparent 1px);
      background-size: 20px 1rlh;
      pointer-events: none;
     
    
  5. See Deep dive CSS: font metrics, line-height and vertical-align by Vincent De Oliveira.

19 April 2026

Russ Allbery: Review: Collision Course

Review: Collision Course, by Michelle Diener
Series: Class 5 #6
Publisher: Eclipse
Copyright: November 2024
ISBN: 1-7637844-0-1
Format: Kindle
Pages: 289
Collision Course is the sixth novel in the Class 5 science fiction series and the first that doesn't use the Dark X naming convention. There are lots of spoilers in this story for the earlier books, but you don't have to remember all the details of previous events. Like the novella, Dark Ambitions, this novel returns to Rose, Sazo, and Dav instead of introducing another Earth woman and Class 5 ship. In Dark Class, Ellie discovered an interesting artifact of a previously-unknown space-faring civilization. Rose, Sazo, and Dav are on their way to make first contact when, during a routine shuttle flight between the Class 5 and Dav's Grih military ship, Rose is abducted. The aliens they came to contact have an aggressive, leverage-based negotiating strategy. They're also in the middle of a complicated war with more sides than are readily apparent. What I liked most about Dark Horse, the first book of this series and our introduction to Rose, was the revealed ethical system and a tense plot that hinged primarily on establishing mutual trust when there were excellent reasons for the characters to not trust each other. As the series has continued, I think the plots have become more complicated but the ethical dilemmas and revealing moments of culture shock have become less common. That is certainly true of Collision Course; this is science fiction as thriller, with a complex factional conflict, a lot of events, more plot reversals than the earlier books, but also less ethics and philosophy. I'm not sure if this is a complaint. I kind of miss the ethics and philosophy, but Diener also hasn't had much new to say for the past few books. The plot of Collision Course is quite satisfyingly twisty for a popcorn-style science fiction series. I was kept guessing about the merits of some of the factions quite late into the book, although admittedly I was in the mood for light entertainment and was not trying too hard to figure out where the book was going. I did read nearly the entire book in one sitting and stayed up until 2am to finish it, which is a solid indication that something Diener was doing worked. I do have quibbles, though. One is that the ending is a bit unsatisfying. Like Sazo, I was getting quite annoyed at the people capturing (and recapturing) Rose and would have enjoyed somewhat more decisive consequences. Also, and here I have to be vague to avoid spoilers, I was expecting a bit more of a redemption arc for one of the players in the multi-sided conflict. The ending I did get was believable but rather sad, and I wish Diener had either chosen a different outcome (this is light happily-ever-after science fiction, after all) or wrestled more directly with the implications. There were a bit too many "wait, one more thing" ending reversals and not quite enough emotional payoff for me. The other quibble is that Collision Course was a bit too damsel in distress for this series. Rose is pregnant, which Diener uses throughout the book as a way to raise the stakes of the plot and also make Rose more annoyed but also less capable than she was in her earlier novel. Both Sazo and Dav are in full heroic rescue mode, and while Diener still ensures Rose is primarily responsible for her own fate, there is some "military men attempt to protect the vulnerable woman" here. One of the things I like about this series is that it does not use that plot, so while the balance between Rose rescuing herself and other people rescuing her is still tilted towards Rose, I would have liked this book more if Rose were in firmer control of events. I will mostly ignore the fact that a human and a Grih sexually reproducing makes little to no biological sense, since Star Trek did similar things routinely and it's an established genre trope. But I admit that it still annoys me a bit that the alien hunk is essentially human except that he's obsessed with Rose's singing and has pointy ears. Diener cares about Rose's pregnancy a lot more than I did, which added to my mild grumpiness at how often it came up. Overall, this was fine. I prefer a bit more of a protagonist discovering how powerful she is by making ingenious use of the ethical dilemmas her captors have trapped themselves in, and a bit less of Rose untangling a complicated political situation by getting abducted by every player serially, but it still kept the pages turning. Any book that is sufficiently engrossing for me to read straight through is working at some level. Collision Course was highly readable, undemanding, and distracting, which is what I was looking for when I read it. I would put it about middle of pack in the series. If Rose's pregnancy is more interesting to you than it was to me, that might push it a bit higher. If you have gotten this far in the series, you will probably enjoy this, although it does feel like Diener is running out of new things to say about this universe. That's unfortunate given the number of threads about AI sentience and rights that could still be followed, but I think tracing them properly would require more philosophical meat than Diener intends for these books. Which is why the next book I grabbed was a Culture novel. Currently this is the final book in the Class 5 series, but there is no inherent reason why Diener couldn't write more of them. Rating: 7 out of 10

18 April 2026

Valhalla's Things: Pizza!

Posted on April 18, 2026
Tags: madeof:atoms, craft:cooking
This post contains a bit of consumerism and is full of references to commercial products, none of which caused me to receive any money nor non-monetary compensation. This post has also been written after eating in one meal the amount of bread-like stuff that we usually have in more than 24 hours. I ve been baking bread since a long time ago. I don t know exactly when, but probably it was the early 2000s or so, and remained a regular-ish thing until 2020, when it became an extremely regular thing, as in I believe I bake bread on average every other day. In the before times, I ve had a chance to bake pizza in a wood fired oven a few times: a friend had one and would offer the house, my partner would mind the fire, and I would get there with the dough and prepare the pizza. Now that we have moved to a new house, we don t have a good and convenient place for a proper wood fired oven in masonry, but we can use one of the portable ones, and having dealt with more urgent expenses, I decided that just before the potential collapse of the global economy was a good time as any to buy the oven I had been looking at since we found this house. I decided to get an Ooni Karu 2, having heard good things about the brand, and since it looked like a good balance between size and portability. I also didn t consider their gas fired ovens (nor did I buy the gas burner) because I m trying to get rid of gas, not add stuff that uses it, and I didn t get an electric one because I m not at all unhappy with the bakery-style pizza we make in our regular oven, and I have to admit we also wanted to play with fire1. We also needed an outdoor table suitable to use the oven on and store it. Here I looked for inspiration at the Ooni tables (and for cheaper alternatives in the same style), but my mother who shares the outdoor area with us wasn t happy with the idea of steel2. And then I was browsing the modern viking shores, and found that there was a new piece in the N MMAR series my mother likes (and of which we already have some reclining chairs): a kitchen unit in wood with a steel top. At first I expected to just skip the back panel, since it would be in the way when using the oven, but then I realized that it could probably be assembled upside down, down from the top between the table legs, and we decided to try that option. This week everything had arrived, and we could try it. Yesterday evening, after dinner (around 21, I think) I prepared the dough with the flour I usually use for bakery-style pizza: Farina di Grano Tenero Tipo 0 PANE (320 - 340 W); since I wanted to make things easier for myself I only used 55% hydration, so the recipe was:
  • 1 kg flour
  • 550 g water
  • 2 g dry yeast
  • 12 g salt
The next time I think I ll try with one of my other staples: Molino Bogetto etichetta blu (260/280 W) Then this morning we assembled the N MMAR , then I divided the dough in eight balls, put them in a covered but not sealed container 3, well floured with rice flour and then we fired the oven (as in: my partner did, I looked for a short while and then set the table and stuff), using charcoal, because we already had some, and could conveniently get more at the supermarket. When the oven had reached temperatures in the orange range4 I stretched the smallest ball out, working on my wooden peel, sprayed it with water5, sprinkled it with coarse salt and put it in the oven. After 30 seconds I turned it around with the new metal peel, then again after 30 seconds, and then I lost count of how many times I repeated this6, but it was probably 2 or 3 minutes until it looked good. A flatbread on a regular plate: it's only a bit more than half the plate in diameter, puffed up near the borders and thin in the middle, and only lightly browned in places, not burnt. It's sitting on the lower shelf of a wooden table. And it was good. The kind of pizza that is quite soft, especially near the borders. We ate it with fresh mozzarella and tomatoes, and then made another one the same way, to finish the mozzarella. Another flatbread on the same plate, this time it's about 4 cm smaller than the plate on all sides, and it's covered with brownish-red chopped up vegetables. This was supposed to be our lunch, but we decided to try one with some leftover cooked radicchio, and that also worked quite nicely. And finally, we decided we needed to try a more classical pizza, with tomato sauce and cured meat, of which we forgot to take pictures. Up to here we had eaten about half of the dough, and we were getting full: I had prepared significantly more than what I expected to eat, to be able to accidentally burn some, but also with the idea to bake something else to be eaten later. So I made two more focaccias with just water and salt, and then I tried to cook some bread with what I expected to be residual heat. Another flatbread with coarse salt and two bread rolls, one of which is completely carbonized on one side. The other one has been cut, and while it has a carbonized spot, it is also well cooked in the middle, and perfectly edible. Except that the oven was getting a bit too cold, so my partner added some charcoal, and when I put the last two unflattened balls right at the back of the oven where it was still warmer, that side carbonized. After 5 minutes I moved them to the middle of the oven, and turned them, and then after another turn and 5 more minutes they were ready. And other than the burnt crust, they were pretty edible. So, the thoughts after our first experience. Everybody around the table (my SO, my mother and me) was quite happy with the results, and they are different enough from the ones I could get with the regular oven. As I should have expected, it s much faster than a masonry oven, both in getting to temperature and in cooling down: my plan for residual heat bread cooking will have to be adjusted with experience. We were able to get it hot enough, but not as hot as it s supposed to be able to get: we suspect that using just charcoal may have influenced it, and next week we ll try to get some wood, and try with a mix. As for the recipe, dividing the dough in eight parts worked quite well: maybe the pizzas are a bit on the smaller side, but since they come one at a time it s more convenient to cut and share them, and maybe make a couple more at the end. Of course, I ll want to try different recipes, for different styles of pizzas (including some almost-trademark-violating ones) and for other types of flatbread. I expect it won t be hard to find volunteers to help us with the experiments. :D

  1. any insinuation that there may have been considerations of having a way to have freshly baked bread in case of a prolonged blackout may or may not be based on reality. But it wasn t the only or even the main reason.
  2. come on! it s made of STEEL. how can it be not good? :D
  3. IKEA 365+ 3.1 glass, the one that is 32 cm 21 cm 9 cm; it was just big enough for the amount of dough, and then I covered it with a lid that is missing the seal.
  4. why did they put a thermometer on it, and not add labels with the actual temperature? WHY???
  5. if you don t have dietary restrictions a bit of olive oil would taste even better.
  6. numbers above 2 are all basically the same, right?

Yifei Zhan: CommBank hardware MFA token

A while ago, CommBank started asking for MFA confirmation on its mobile app for every NetBank login on a browser. Previously, there was an option to use SMS for MFA, which isn t as secure as I would like, but it was at least usable. Since I m switching away from Android to Mobian and won t be able to use the CommBank app for much longer, I applied for a physical NetCode token. The hardware is made by Digipass and looks disposable. It is a small, battery powered gadget with a screen and a button. When pressed, it shows a temporary NetCode for authentication. Such a NetCode is required both for NetBank logins and approving online transactions. The letter that came with it has the wrong link for activation, the correct link is under NetBank -> Settings -> NetCode (under the Security section) To apply for a physical token, call the NetBank team, mention you can t use the app and need a physical NetCode token, and make sure they actually submit your request for a token. It took me 2 calls to get them to ship me a token. The hardware is free of charge but can only be applied for via phone call; unfortunately staff members at my local branch are unable to do anything in relation to NetBank. I was told privately by a CommBank employee that they are deprecating the hardware token in favor of the mobile app, I hope that won t happen anytime soon, or that they add support for passkeys before they do. The last time I checked, the CommBank app was LineageOS-friendly, but I don t want to configure WayDroid just to do online banking. PayID, the thing that allows you to receive payment via a phone number or email address, is not compatible with the hardware token, and existing PayID will be silently deactivated if you use hardware token. This looks to be an artificial restriction; I don t see why it has to be this way. Regular CommBank mobile app sessions will also be de-activated once the hardware token is activated (I was told so but my sessions weren t deactivated until I wiped my Android phone), and you won t be able to sign into mobile app again until you manually disable the NetCode token. Online banking has been getting progressively more invasive and anti-user over the last decade, from demanding remote attestation to requiring real time location data, each time locking certain features when those demands are not satisfied; all based on the flawed assumptions that everyone owns a phone running a certain flavor of iOS or Android, and has it ready all the time. I m not sure what can be done to reverse this trend, but on the personal level I will use NetBank less and go back to cash.

Next.

Previous.