Search Results: "beh"

14 July 2026

Jonas Meurer: zed-xdebug

Nextcloud PHP debugging with Xdebug in Zed editor I started to switch from PhpStorm to Zed as IDE recently as Zed is open source and has a much smaller footprint and is more slick than PhpStorm. One thing that I didn't get running immediately was Xdebug integration, so I did a bit of research and asked Claude for help. Here's a quick writeup of how to get it running. I have Zed installed as Flatpak on a Debian Trixie host system. The PHP process runs in a nextcloud-docker-dev Docker container.

Install Zed and configure debugging there Install Zed: flatpak install flathub dev.zed.Zed In Zed: open the Extensions view and install PHP. Configure the debugger: Create ~/.var/app/dev.zed.Zed/config/zed/debug.json:
[
   
    "label": "PHP: Listen to Xdebug",
    "adapter": "Xdebug",
    "request": "launch",
    "port": 9003,
    "pathMappings":  
      "/var/www/html":             "/home/<user>/devel/nextcloud/server",
      "/var/www/html/apps-extra":  "/home/<user>/devel/nextcloud/server/apps-extra",
      "/var/www/html/apps-shared": "/home/<user>/devel/nextcloud/apps-shared"
     
   
]
Add one entry per bind-mounted app directory. After creating the file, restart Zed. Inside Zed, select "debugger: start" from command palette and then "PHP: Listen to Xdebug". Verify Zed is listening. Running ss -tlnp grep 9003 on the host should show *:9003 with Zed as the process.

Configure Xdebug inside the container /usr/local/etc/php/conf.d/xdebug.ini:
xdebug.mode = debug
xdebug.idekey = PHPSTORM
xdebug.trace_output_name=trace.%R.%u
xdebug.profiler_output_name=profile.%R.%u
xdebug.output_dir=/shared/xdebug
xdebug.log = /var/log/xdebug.log
xdebug.log_level = 3
; Try to discover the client host, otherwise fall back to the docker host
xdebug.discover_client_host=true
xdebug.client_host=host.docker.internal
; When you cannot specify a trigger, use "xdebug.start_with_request = yes" to autostart debugging for all requests
; https://xdebug.org/docs/all_settings#start_with_request
xdebug.start_with_request = trigger
; Set xdebug.mode trace to use this
; More details at https://derickrethans.nl/flamboyant-flamegraphs.html
xdebug.trace_format=3
xdebug.trace_output_name=xdebug.%R.%u
Apply changes by restarting apache in the container: apache2ctl -k graceful Notes:
  • host.docker.internal resolves on Linux Docker only if the container was started with --add-host=host.docker.internal:host-gateway (nextcloud-docker-dev already does this).
  • discover_client_host = true makes xdebug follow X-Forwarded-For - useful behind Nextcloud's dev reverse proxy.

Test xdebug with a PHP command inside the container Run XDEBUG_SESSION=PHPSTORM php occ status inside the container and check /var/log/xdebug.log.

Install the browser extension Install Xdebug Helper (Firefox/Chrome). In its preferences, set the IDE Key to PhpStorm. It will set the XDEBUG_SESSION cookie when toggled to Debug. Click the Xdebug Helper icon in the browser and set it to Debug.

Test Xdebug with browser extension Load the URL that exercises the code path with the breakpoint. Zed should stop the code exection at the breakpoint.

11 July 2026

Reproducible Builds: Reproducible Builds in June 2026

Welcome to the June 2026 report from the Reproducible Builds project! In these reports, we outline the most important things that we have been up to over the past month. As a quick recap about what problem our project intends to solve, whilst anyone may inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries. The motivation behind the reproducible builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised or not. If you are interested in contributing to the project, please visit our Contribute page on our website. In this month s report, we cover:

  1. Only installing reproducible packages with repro-threshold
  2. Distribution work
  3. diffoscope development
  4. From our mailing list
  5. Documentation updates
  6. Patches
  7. Four new scholarly papers


Only installing reproducible packages with repro-threshold A very interesting demonstration is now available showing how you might configure your Debian system to only install packages that have been reproduced by m/n rebuilders. This is implemented via a reproduced+https:// APT transport ( a mechanism for communicating between the APT client and its repository source commonly HTTP):
Every package download is intercepted by repro-threshold, which queries two independent rebuilders for a signed attestation before allowing installation to proceed. [It] is important to note that [an] install will only succeed if all package dependencies are also reproducible.
The demo gives examples of how to quickly experiment with this using a Docker container.

Distribution work In Debian this month: The IzzyOnDroid Android APK repository reached its next milestone this month, now covering 2 out of every 3 apps (66.7%) with reproducible builds. Their documentation for debugging and fixing failed builds has steadily grown as well. More clients have picked up showing reproducibility results (e.g. Droid-ify), and Neo Store now can be configured to stick to only reproducible applications. Further, an independent builder has been added to the build farm, increasing the trust level even more as APK builds can have multiple confirmations now. At the same time, IzzyOnDroid s rbtlog got several new features. The most outstanding is caching for frequently used resources such as reproducible-apk-tools, command-line tools and NodeJS in order to counter ongoing issues with GitHub availability, while at the same time saving bandwidth and build time. This change also enables some other some smaller enhancements such as being able to configure build timeouts per recipe for those builds running longer than the average, release pattern filtering for update checks or having a field for maintainer notes to shortly summing up e.g. why a reproducible build failed.
Lastly, Bernhard M. Wiedemann posted another openSUSE monthly update for their reproducibility work there.

diffoscope development diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made the following changes, including preparing and uploading versions 319, 320, 321, 322 and 323 to Debian:
  • Debian adds an extra Flags: line in the output of ocamlobjinfo, so adjust the test for cross-distribution compatibility. [ ]
  • Bump debhelper compatibility level to 14. [ ]
  • Fix compatibility with Ocaml 5.4.1. [ ]
  • Use --long-form-style arguments when calling apktool in order to support apktool version 3. [ ]
  • Support Androguard version 4 and previous versions at the same time. [ ]
  • Update copyright years. [ ]
In addition, Jochen Sprickerhof added better header detection for the Sphinx documentation system [ ], Michael Daniels fixed the tests when run with zipdetails version 4.006 [ ] and Zbigniew J drzejewski-Szmek added a version of the deprecated os.path.commonprefix method [ ]. In addition, Vagrant Cascadian updated diffoscope in GNU Guix to version 321 and 323.
Chris Lamb also made the following changes to strip-nondeterminism, our tool to remove specific non-deterministic results from a completed build:
  • Skip symlinks when manually called via /usr/bin/strip-nondeterminism. (#1139000)
  • Update debian/watch format. [ ]
  • Drop Rules-Requires-Root: no and Priority: optional fields. [ ]
  • Bump Standards-Version to version 4.7.4. [ ]

From our mailing list On our mailing list this month:
  • kpcyrd posted to our mailing list regarding the waves of malware uploads to aur.archlinux.org . Curiously, every incident I looked at used npmjs.com for malware delivery , specifically where the npm package includes an (automatically executed) preinstall script that is an ELF binary.
  • kpcyrd also announced the release of debian-repro-status version 0.4.0, a tool written to give you an approximate idea of how viable it would be to enforce a reproducible packages only update policy for the computer system you ve built :
    The change updates dependencies to the latest versions, and adds support for multiple -H options, to query results from multiple rebuilderd instances. The results are also now fetched concurrently.
  • kpcyrd also reported that, whilst taking a screenshot for the above release, they noticed that the debian:sid container now is 100% reproducible.
  • Finally, kpcyrd also created a pull request against the add-determinism package to update the itertools and zip Python dependencies.

Documentation updates Yet again, there were a number of improvements made to our website this month including:
  • Chris Lamb added a reminder re. using the UTC variants of the Javascript Date methods. [ ]
  • Mattia Rizzolo moved OTF to the old sponsors list. Thank you for your support!. [ ]
  • kpcyrd updated the Rust documentation to recommend using the --release argument for consistency. [ ]

Patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where applicable or possible. This month, we wrote a large number of such patches, including:

Four new scholarly papers Kenichiro Muto and Kuniyasu Suzaki of the Institute of Information Security in Yokohama, Japan published an interesting paper this month titled Attestable Build Chain: Enabling Trust in Reproducible Builds (PDF). Their abstract is as follows: Ensuring trust in software supply chains requires verifying not only artifacts but also the processes that produce them. Although Reproducible Builds (R-B) require rebuilding to validate artifacts, they cannot verify whether the build was executed with the intended toolchain and inputs and may reproduce unintended or compromised builds without detection. We present Attestable Build Chain, a framework for externally verifying build-time execution without rebuilding. Rather than preventing compromise, it provides verifiable, tamper-evident evidence of actual build-time execution, enabling verification of build process integrity from observed file accesses during the build. [ ]
Julien Malka, Stefano Zacchiroli and Th o Zimmermann published a 50-page report detailing A Decade of Software Reproducibility in the Nix Package Ecosystem:
We find that functional package management enables extremely high rebuildability over time (near-universal ability to reconstitute historical build environments and rebuild software packages), while bitwise reproducibility has steadily improved and reaches a high point in recent years (up to 93% in 2024). Early years show substantially lower bitwise reproducibility, indicating that functional package management alone does not guarantee bitwise-identical outputs, and that the observed high level of bitwise reproducibility is not solely due to the package management approach. Common causes of unreproducibility, both in the rebuildability and bitwise reproducibility dimensions, include management of dates in build and test processes; we quantify their prevalence and other common causes using manual analysis of logs of rebuild failures and automated analysis of diffoscope.
A PDF of their report is available online
Tim Bastin of L3montree GmbH and Jacek Galowicz of Applicative Systems GmbH from DevGuard published a paper detailing How We Built a Sovereign, Reproducible Container Supply Chain for DevGuard:
This paper presents how the DevGuard project rebuilt its OCI container pipeline around reproducible Nix builds and independent dual-platform digest verification. DevGuard images are built hermetically from pinned source revisions, signed with Sigstore/Cosign, and verified through digest comparison across GitHub Actions and sovereign GitLab infrastructure hosted on container.gov.de. We describe the practical integration of reproducible OCI image builds into existing CI/CD workflows and argue that independently reproducible container digests provide a stronger integrity guarantee against build tampering than provenance alone. The paper further discusses remaining trust assumptions and the relevance of sovereign build infrastructure for government and regulated environments.

Finally, Yiseul Choi, Junga Kim, Jun-Ho Hong and Seongmin Kim of the Department of Convergence Security Engineering at the Sungshin Women s University in Seoul, Korea titled Attestation-based verification of SBOM integrity via consumer-side reproducibility:
Software bills of materials (SBOMs) support supply chain transparency, but they do not prove that a delivered SBOM reproducibly corresponds to its software artifact. Existing signing and provenance mechanisms protect integrity and traceability, yet lack consumer-side reproducible verification. We propose an SBOM integrity verification framework combining procedure disclosure, consumer-side reproduction, authority-generated reference evidence, and digest comparison. A trusted authority records a reference digest, and consumers compare it with locally reproduced and delivered SBOM digests. Experiments on 100 real-world container images show detection of artifact tampering, SBOM substitution, distribution modification, and adaptive tampering beyond signature-based approaches


Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

6 July 2026

Aigars Mahinovs: How to make a good group photo

Taking a good group photo consists of multiple aspects: I can say with confidence that nearly everything here comes from having failed to do these things right at least once, even on the latest attempts, so this is an ideal to reach towards, not something we expect to hit every time. The Goal The main goal of a big event group photo is capture both the moment itself and each individual person inside that moment. We want people, who were not there to see all the people involved and get an impression of what it was like being there. It needs to show the breadth and depth of people that make up this group, this project. And we want people who were there to be able to look back the next week, the next year or in ten years and remember - ah, yes, I was there, I was standing right there with this grin on my face next to this wonderful person and I was feeling great. Hardware Based on the goal we want to have high level photographic gear that is able to capture both a broad enough picture to encompass all the people and some of their surroundings to communicate the context (without undue distortions) and to deliver enough detail and resolution so that faces and facial expressions and underlying feelings of every single person in that group could be clearly seen and preserved. To both capture the context and minimise distortion the final picture should be just a bit wider than normal human field of view. That is about 50mm for a full-frame camera or 35mm for a typical 1.6 crop camera. You can go a bit wider if there are no better alternatives (as detailed in the scouting section), but be prepared that corners of the image will be distorted and not really usable (but we can fix that in processing step). Or you can go to unusual aspect ratios, like we did in Debconf 10. In the absence of a 100MP+ camera, you will need to be stitching together multiple frames to achieve resolution high enough to have enough pixels-per-face to see emotions clearly. This means that the photos you will actually be taking will be tighter than the overall field of view mentioned above. Still, a higher resolution camera body is preferable - nowadays 24MP-32MP cameras APS-C provide a good compromise between resolution and price, but 45-67MP full-frame cameras also exist on the market. Assume that we will be shooting in a bright environment, so most likely with quite low ISO settings, that means that high-ISO noise characteristics of more expensive cameras will not really play a role here. You will also not need very fast burst modes, even manual speed of one frame per second is sufficient. You will also want to get as much detail as possible out of your lens, and this is the most important part. You can do amazing work in all other steps of the process and have a great camera too, but if you pair it with a lens that is not sharp, then the end result will be disappointing. You want the lens that is sharpest corner-to-corner when stepped down to about f/8-f/11, that you can get for your system. You also want that lens to be about 85mm full-size sensor or 50mm for 1.6 crop size. Luckily that kind of range is also a great range for optical design and sharpest lenses are typically available in exactly these kinds of sizes. You absolutely want to have a fixed focal length lens, not a zoom lens. Even profession grade zoom lenses often deliver worse image quality compared to fixed lenses that cost less 1/10th of their price (when shooting in the same focal length). Newer design lenses are better than older lenses - optical design, coatings and precision manufacturing have advanced a lot over the decades. Retro look is great for mood, but not as good for actual resolution and clarity. You don't need to overpay for most expensive lenses because those often only improve image quality on lower F-stops. To encompass the whole group we will need to shoot at f/8 and in bright light, so the extra benefits of those f/1.2-capable super expensive lenses will not come into play here. We will have no use for a flash here. A tripod will be too restrictive when rapidly repositioning the camera between different parts of the panorama shoot. But a monopod might help with stability - I have not tried that myself, however. For my last photos I used a Canon EOS R7 (32.5MP) with Canon RF 50mm f/1.8 STM lens and considering an upgrade to Sigma 56mm f/1.4 DC DN for the next time. Scouting Scouting a good location for the group photo is another big chunk of a successful picture. The critical piece of the puzzle is lens-to-face distance. In order to keep everyone's face in-focus and have enough resolution on the farthest faces (without making nearest faces truly massive) we want to do everything possible to reduce the variance in lens-to-face distance - to reduce the difference in distance between closest and farthest face. The most effective way to do that is to have the photographer climb higher. To see this in action on the Debconf photos, compare Debconf6 (very high camera position, group on level ground - good) to Debconf10 (camera not too high, group on stairs, still good) and to Debconf17 (camera could not get high enough and the group is on flat ground - not great). Even the Debconf25 photo was suboptimal from this perspective. The Debconf23 photo was a very good example from the recent years - good height and also the group was positioned in a semi-circle so there were no people directly in front and very near to the camera. So you are looking for the highest point you could get to (even if that requires a special permission of key or a ladder) with a field large enough to fit the whole group comfortably. How to check that? Normally I simply take a photo from the top of the whole area and note down from there where the extreme corners of the group could be and still be fully seen in the shot - not blocked by trees, buildings and shadows. Then I go down and measure that space. Rule of thumb being - people in one horizontal line can stand 1 normal length step from each other and two horizontal lines can be half a step from each other vertically. So I can just measure a rough rectangle in steps, multiply the sides, multiply that by two and I have the rough number of people that can fit there for the photo. Once you have a candidate location or two, it is important to check them at the same time-of-day as you plan to do the photo (see organization section for that). You want to make sure that the whole area of the group is in the same illumination - if half of the group is in the sun and half in a shadow, then you will be having a very bad time later. The absolute ideal positioning for the group photo is to have everyone be in shadow, but still have enough bright skies and bright buildings in front of the people to give good illumination of the faces. Worst you can do is have the sun be behind the people (so all the faces are really dark) and second worst is have the sun be directly in front of the group, so that the faces are very well illuminated, but everyone's eyes are closed because they are being blinded by the sun. And sometimes all you can do is pray for some light clouds to provide for even and dispersed light. Debconf23 was very lucky that way. Another consideration is to how people are going to get to that place. You need to consider accessibility needs of people (it is ok, if it takes more effort or time, but it needs to be organized and communicated well in advance). And you need to consider how the big masses of people will be getting there - how to tell people where exactly it is and how to get there from various locations where people might be hanging out during the event? Having an alternate location indoors might be necessary if the weather report for the next days is not sufficiently predictable. We had to use that contingency in Debconf9, for example. Organization It's hard to take a good group photo if half of the group does not show up or is too late, so this needs some organization to happen smoothly. First of all you need to choose date and time for the photo. The photo does not take too much time from the schedule of the event and can be squeezed in after all the other events are already scheduled. In fact I prefer that as it allows you the flexibility of choosing the date based on weather conditions and time based on light and shadow conditions in potential photo spots. You don't want to choose the daytrip day as most people will be away and return times are not really predictable. You do not want to choose the morning after Cheese and Wine party for obvious reasons. First day and last two days are also sub-optimal as some people arrive late and some leave early for various personal reasons. Also you don't want it to happen just before Cheese and Wine either because then you'd have very little time and clarity to do the processing of the image on the same day. For timing, the best way, in my experience, is to schedule the photo directly after the end of talk sessions before a meal break - lunch or dinner. Typically in the Debconf schedule there are 2-3 daily breaks planned, say for Debconf25 there was lunch, afternoon break and dinner. Talks are planned to end ~10 minutes before those breaks (and meals) begin, so for example, afternoon break starts at 16:00 and all talks in the previous block end at 15:50. In such a case just schedule the "Group photo" event from 15:50 to 16:05. This gives people the info to go there directly from the end of all talks and that they will have sufficient time for break/meal afterwards. Do not forget to specify the location (as exactly as possible) in that event entry and make sure to post it at least two days in advance. People often want to wear something specific for the photo and thus need to know about it in advance. This also makes sure that people do not make alternate food plans for that specific break and don't leave the venue. Announce the date, time and the exact location as wide as possible, don't be shy. Announce and discuss mailing lists, IRC, Signal, Telegram, make sure the front desk knows in case anyone asks in-person, ... Check that it is again included in the announcements email on the day preceding the photo date. When the date has arrived, it is a good idea to check in early with people with special mobility needs to make sure they know where to go, how to get there and how much time they will need to be able to get there on time. As the final round of talks before the group photo is starting up, it is time to recruit "runners". I've had great success with this technique. The idea is pretty simple - for each room where people congregate (talk rooms, hacklabs, cafeteria, outside hackspace, front-desk, ...) go there and choose one person. You want to choose a person that you will recognise and remember among everyone else in the group, either because of who they are or what they are wearing, whatever works best for you. If they agree to help, instruct them to: "at end of talk, announce that the group photo happening now and the location, herd people towards the photo location, be the last person out, make sure there are no stragglers from this area behind you, when you arrive to the photo place I will assume that everyone else from this room is also now there, when you are there catch my attention and show this sign so I know for sure that it is all good and make sure that I did see it from you". With that sorted out all you will need to remember is how many runners you recruited and how many have reported in to figure out if everyone has now arrived or if we still have to wait for someone or some group. Then you will only have one last point of organization left - shaping the crowd into a group. People will not know what your vision for the group photo is, so you will have to give clear and LOUD instructions on where people should not be standing. Use clear, large gestures to support your words. You want to compact the group, have the people that just joined in the last moment and are standing to the side come deeper in and join the crowd. Have any holes in the middle of the crowd filled in. Forming a semi-circle instead of a blob helps with averaging face-to-lens distances. Make sure people are not in unexpected shadows. Make sure carried objects, like umbrellas of flags do not cover the faces of other people. Take the time to look at everyone face to make sure there are no people hiding behind someone's shoulder - typically they are not aware that their face is in fact not really visible. If there are such people, call them out and point directly at them and encourage them to step forward, if they wish to do so. You are the only one seeing the final picture now and only you can correct it before capturing the moment. So a few extra seconds here are worth taking, even if 300+ people are standing in scorching heat and waiting on you. When you are happy with what you are seeing, make sure to tell people clearly that you are now about to take the pictures and again remind them not to move and explicitly not to turn their heads to the side until you are done (this is the source of most of the extra work in processing). Be very loud and clear and make sure you have everyone's undivided attention before you start saying the important stuff. When done - say so. There will be other groups that will want to also have a photo taken after the main group is a bit more dispersed, so don't run away. Typically at least the T-shirt group will want a picture and also all the organizers. Final bit of organization during the group photo shooting itself is the sneaky self-insert. You may choose not to bother with it, or do it in the simplest way, like I did in Debconf6, but if you really want to blend in with the crowd, you need to have someone else take a photo of you in the exact same location at the same date and time from the same location. So you should already during shaping the crowd decide where you would fit in, it is easiest to blend in at the back of the crowd and to one or other side, so that it appears like you are just standing behind the shoulders of a couple peoples. Remember that spot - it is easiest if you stand in the exact same ground spot when your photo is taken. Just go down, recruit a volunteer to take your photo, make sure the settings are fixed to the same ones as for the group photo shots and have them take a handful of shots of you - one of you centered in the camera frame and a couple more with you more towards the corners of the frame. This distortion from being off-center in the frame may be important later. Preparation In addition to preparing the crowd for the photo, you also need to prepare yourself and the equipment. Make sure you have dusted your camera sensor and cleaned both inside and outside glass of your lens. It is usually a good idea to remove any filters from the lens. Install the hood, if that could help with blocking the sun flares. Make sure you have the right lens and that you have installed the right lens. For fixed settings I typically shoot in JPEG with RAW being there more like an emergency backup. The extra dynamic range of RAW could be used, but it is really complex to do that in combination with image blending and it is hard to get right, so I prefer an all-JPEG workflow and fix the dynamic range in the scene itself, before shooting. For Canon I am using the Standard profile that boosts the color saturation and sharpness a bit as I just enjoy that look and find it hard to get anything significantly better from RAW data even with a lot of effort. In any case make sure you have enough space on the cards to take at least 100 images and that you have a full battery. Do not use high speed burst setting because it is then too easy to take too many pictures at the start of the sequence and be stuck with your camera still in "Busy" state writing big RAW files to slowish SD cards and not allowing you to finish the full picture rapidly. You want to have the shutter speed at at least 1/100th of a second to prevent blur from both your hand movements and also from people in the shot moving around a bit (image stabilisation will not help you there). And you want to have the aperture to be around f/8 - lower apertures risk people in front or behind falling out of focus, make the lenses look less sharp. Higher apertures also start to become less sharp due to diffraction effects above f/8. ISO should stay as low as possible, ideally at ISO 100, but if there is not enough light then upping the ISO to 400 would be the first step that I would try to do and second would be decreasing the aperture to f/5.6. If there is too much light, then increasing the shutter speed should be the safe thing to do. As people start to arrive into the shooting location - check the exposure and nail down the settings, ideally in manual mode. Consider that left side could be a bit lighter or darker than right side. Err on the side of making the picture a bit too dark as there is more depth to darkness before cut-off compared to clipping on the high end. However, do not trust the exposure detection, instead take a picture and look specifically at skin tones in faces of people that already are standing in the photo area. Faces are the key bit and the exposure needs to be adjusted just to the faces and ignore darker of lighter clothing. Do some test shots and find settings where faces look not too bright, but also not very dark and fix those settings in manual mode. Now you are ready for the action. Shape the crowd, check the faces and the action can start! Execution During taking of the group photo you want to finish it fast, but at the same time you have to take the time to make it right. If you hurry too much under pressure, you risk being left with unusably blurry images and the whole effort wasted. Having already prepared and verified the manual settings makes it easier. When you are taking pictures, you have to remain as still as possible - even at very high shutter speeds even slow hand movements are still bad for image quality. So think of the movement as of biathlon athlete shooting the very middle of five, very separate targets - take a burst, reframe, then steady up for a second and only then take the next burst. 3 frames per burst are sufficient. 90% of the time the very first photo of a burst will be best. As you move from frame to frame, aim for just a bit more than half-frame overlap. This will give the opportunity to skip frames if all is good, but also have backup coverage of every face in case of problems. Proceed systematically, I typically start off on the top left of the crowd, then go right until the end of the line, then shift down half a frame and go left until the end and repeat until I am done with the crowd. After that it is very helpful to also immediately take photos of a "frame" around the whole crowd. Stitching process often distorts the frames in weird ways that leave holes in the resulting image that you can fill if you have a wide frame around the crowd. It is possible to compensate with creative cutouts in the final image (like Debconf9), but the more framing room you make, the more flexible you will be able to be with cropping of the final photo. The frame also gives you the opportunity to capture more of the context of the place and space. As an example, Debconf25 group photo in the end consisted from 9 images + 1 for sick people + 1 for me. I ended up missing the framing shots for bottom left, top left and top right corners. To get there I took 68 images. And in some years it was more than a hundred. Processing This part might be less stressful than taking the pictures from intensity perspective, but it lasts longer. Depending on you luck, skill and perfectionism it can take anywhere from 3 to 9 hours of work to complete. Before you start, however, you should first request things that you will need for other people. This can even be done before taking the actual group photo, but usually I forget. To finish the photo you will need three things: The first two you should be able to get from the respective organizers. The motto is harder. I typically try to ask the current DPL to come up with something describing the current mood of the project or of the event, but it is rare that it is that easy. Most of the time I came up with something as I was editing the photo and reflecting on what was the mood, the feeling, the mojo of this conference and of this year was like. Bend that around a recognisable phrase or expression, make it a bit more insider-relevant and you are on the right path. Some years this was the hardest part. For the panorama stitching I will describe the workflow that has served me good for years, but maybe there are better ways possible nowadays. Feel free to let me know! First I would save all photos taken and select one sharpest photo from every burst. Next I would select the minimal number of photos that appear to be covering the entire crowd. The fewer images you use, the better in the end because the most quality problems crop up in the areas where photos are getting stitched together. Fewer seams leads to fewer issues. Open Hugin (you will also need enblend and enfuse installed) and import your minimal set of images into it. Click the "Align" button and wait a while - the processor will be trying to figure out keypoints in each image and then try to match these points between the images to try to fit them all together into a single projection. To do that it will distort the images. This is the trial and error process part. You may need to add, remove or replace images to get the stitching to work or to work better. You may want to add more of the frame images to fill the ragged holes around the image. After initial alignment, go to "Move/Drag" tab and move the image a bit up in the projected field of view and make it a bit more central visually. That will help a bit with the distortions in the near-by people and people in the corners of the image. In the "Crop" tab set the initial crop - leave it generous, you can always crop more in later steps. Do not be afraid of leaving in sizable chunks of black homes, empty skies or grass. All of that can be filled in later as well. Go back to the "Assistant" tab and click "Create panorama". It is good enough to have JPEG output at 100% quality using exposure corrected low dynamic range output option. Make sure to check the "Keep intermediate images" option. This will not only generate the final, merged panorama, but also keep around the individual images after perspective correction and exposure blending steps. These are critical for fixing blending error in the next step. You might need to go back a forth a few times with a different sets of source images, maybe adding some image between other two, maybe removing another to reach a better starting point. The key part to pay attention - how many ugly stitches are there in the image. Check every face, the blending algorithms do not recognise faces and sometimes try to stitch one face from two or more images creating very weird effects. They can be fixed in the next step, but it is rather hard manual work, so the fewer such faces are in the blended image, the less work you will have. In some years I've managed to find a combination where all faces were good and in other years I had to manually fix 13-15 faces. Do not try to blend the extra pictures (like with you or with sick people) into the main panorama with Hugin - it will get very confused with the parts of the grass that it is able to see where other people were standing. The next is the final processing in GIMP. Think of it like a large and complex project - do as much as possible in separate layers, save often. Fixing wrongly stitched faces and also putting yourself into the photo are very similar activities in the end. Just the scale and the source differ. For yourself you just cut out yourself (upper torso is enough) from the separate photo. For corrupted face, choose one of two intermediate images that the Hugin created where the face is transformed, but not yet merged (with a different version of itself). In either case crop the photo to roughly the interesting size and put roughly in the right spot as a separate layer on top of the group photo background. Reduce the opacity of the small layer to 30-40% and zoom in to 400%. With that it is much simpler to position the layer with pixel precision. Then all you need to do is add a layer mask to this layer and paint it just right. Basically in layer mask black means transparent and white means non-transparent. So you need to just make everything that is you have white mask and everything that is not you have black mask. And smudge the border a bit with finger tool or blur to make the transition smoother. Easy to say. Hard to do. This is what takes most of the actual work hours in post-processing. You might miss someone. I am sure Phill is just thrilled to see me in the very middle of the Debconf25 final picture .... But do try to fix them all. Use large, sweeping geometric figures to cover up black holes, empty grass fields and other sub-optimal corner features. And then use that newly created free space to put in a large version of the logo of this years conference, decently sized motto and slightly smaller invitation to the next years conference. Do not forget to add a copyright and license statement somewhere in the corner in smaller, but still well readable font. I am using a text like: "Photo by: Full Name, Email: fullemail@debian.org, License: GPLv2+ or CCv3-BY" This ensures that this image may be used in any press coverage (with basic attribution) and also can be included in any GPL-licensed software, if that ever comes up. The same statement is also in the metadata of the image file (see Image-Metadata-Edit metadata in GIMP) along with information that states that this is "Debian Developer Conference Group photo, City, Country, Year". Image->Image properties->Comment is another place where GIMP hides this EXIF information. For ease of use, in addition to a full-resolution image it is also useful to make a lower resolution version that would still fit on a 4K screen at full resolution, so about 3840px wide. Some photo hosting services set other limits for image size as well, so it might be needed to scale the image down below 100Mpix to upload it to Google Photos, for example. Publishing So, it is finally 1AM and the group photo is ready! How do you push it out to people? Well, in all possible ways and places. Again - don't be shy, people do really want to see it. Push it to whatever you use for your shared photos. Push it to Debconf shared git (note that this is GIT-LFS repo, make sure you know how to add content to the LFS specifically). All permanent links to that in GroupPhotosAll wiki. And then send those links to IRC, Signal, Telegram groups, debconf-announce mailing list. Publish it in your blog and push that to Debian Planet. Push it in Threads, Bluesky and Mastodon. Send an email separately to Debconf orga team. And one to Debian Publicity Team so they can put it into the Debian Home Page and push via Debian micronews accounts. And that is about it. Now you can go back to enjoying the rest of the conference. Or running around doing other things that you think need to be done. It's up to you. You did it. This moment will remain with people for a very long time. And you helped. Questions? Feedback? Just ask here or here.

Russ Allbery: Review: The Player of Games

Review: The Player of Games, by Iain M. Banks
Series: Culture #2
Publisher: HarperPrism
Copyright: 1989
Printing: February 1987
ISBN: 0-06-105356-2
Format: Trade paperback
Pages: 295
The Player of Games is political space opera and the second book in the shared Culture setting. As with most Culture books, the reading order is not particularly important. It won the 1989 Locus Award for best science fiction novel and sometimes competes with Use of Weapons as the consensus best Culture novel. This review is a re-read and yet another experiment in how to re-review a book. This time, I decided to write a full second review with substantial spoilers so that I can talk in more detail about the book. If you want to avoid spoilers, or just want to see how my thoughts have evolved from my first reading, see my original review from 2005. Gurgeh plays games. He is probably the best strategy game player in the entirety of the galaxy-spanning Culture. He has written papers on game theory, won innumerable major championships, and is a celebrity in the circle of like-minded aficionados. Gurgeh is also bored and in the middle of the Culture equivalent of a mid-life crisis. As the story opens, he's vaguely unsatisfied and adrift, unenthused by his normal activities, and searching vaguely for something that will break through his ennui. He is caught by surprise by the thrill he gets from a moment's misunderstanding in which an opponent suspects him of cheating, which sets him up to be (apparently) clumsily blackmailed by a deeply unpleasant drone named Mawhrin-Skel. SPOILERS BELOW. If you have not read this book, consider stopping here and instead reading my original no spoiler review. The first hundred pages of The Player of Games is a slow, somewhat plodding introduction to Gurgeh, his social circle, and life in (one part of) the Culture. I remember being fascinated by this part the first time I read this book. It was only the second Culture novel I read and the first set in the Culture proper, so the world-building underlying this odd post-scarcity utopia on a vast intelligent habitat with sentient drones, complex privacy rules, endless cocktail parties, and apparently directionless socialites was intriguingly unlike the other science fiction I was reading at the time. This time through, I have to admit I was less impressed. Gurgeh is not very likable, and his desultory mid-life crisis is a little boring. None of his friends have enough depth to appear as more than side notes, in part because Gurgeh doesn't seem to care enough about any of them to make them interesting to the reader. I've since read seven other Culture novels, so Banks's cocktail parties hold less charm and I was impatient for the real action to begin. These chapters are still important, though, because they establish how utterly average Gurgeh is. He has one unique talent, a deep affinity with and obsession with strategy games, and is otherwise a bit of a depressed narcissist with a few casual relationships, a friend that he barely confides in, and a comfortable and familiar life. He is not in any way a hero or a charismatic figure; he just happens to be exceptionally good at one thing, enough to make him famous among people who care about that one thing and probably unknown to anyone else apart from the occasional idly perused news headline. He is the Culture's equivalent of the world chess champion. The Contact division of the Culture has a problem. The Empire of Azad in the Lesser Magellanic Cloud is a nasty, expansionist culture of the sort that Contact would like to deal with before it causes broader problems. The Culture's normal approaches are thwarted by an unusual organizing principle: The empire is built around and takes its name from the game of Azad, a highly complex strategy game developed over thousands of years. Azad is the civil service exams, means of political and religious dispute resolution, selection mechanism for the emperor, and civic religion. Faced with that oddity, Contact turned to Special Circumstances, the Culture's more aggressive and less restrained way of dealing with tricky problems. Special Circumstances, in turn, needs someone who can learn how to play the game of Azad. They want Gurgeh to take a very long trip. For all of Gurgeh's dissatisfaction, he's not impulsive enough to take a five year journey away from his life and everyone he knows just to play a novel game. Conveniently, Mawhrin-Skel's blackmail resolves this reluctance. The game of Azad requires some suspension of disbelief. Banks provides a few glimpses at the mechanics of the game, but those details are insufficient to reconstruct the rules, and some of the claims made about its properties are improbable at best. The best mental model I could build for it is a strategy or simulation game built around units and territory control, with supplemental side games used to build up resources for the main boards, but it's more of a plot device and a set piece than a world-building invention. The significance of Azad the game is its role in society: The Empire of Azad believes they have constructed a game whose complexity so closely models reality that the skills required for success in the game are precisely the skills required for success in the empire. The Empire of Azad is wrong, and this is one of the core themes of The Player of Games. As with many Culture novels, what Special Circumstances tells Gurgeh is, at best, incomplete. Gurgeh is a refutation of the basis of belief in Azad; this is why it is important thematically that he is an average, somewhat unlikable citizen of the Culture whose only special characteristic is skill at learning and playing games. Azad is the myth of meritocracy given physical form as a game. It provides the anchor of the empire for the same reason that societies on Earth place enormous weight on standardized tests, capitalist success, or public debates. All societies face the problem of selecting good leaders and testing opposing beliefs, and all societies attempt to find some form of shortcut, some set of general principles, tests, or objective metrics used to select the best person via a process that people consider plausible and fair. The game of Azad is a paragon of apparently meritocratic process. No matter who you are or what your background is, if you excel at the game that, in theory, objectively tests your skills, you are given a position of power. In practice, the Empire of Azad is not that naive. Manipulation outside of the game happens, only some players have the opportunity and resources to spend years learning the game at a deep level, and only their dominant sex truly stands a chance in games that matter. But neither is Azad's place in society a fiction. There is corruption around the edges, and a lot of people are filtered out before the games begin, but the highest echelons of society are true believers. The game does decide both rank and policy; Banks is arguing against a strong form of apparently working meritocracy. Gurgeh represents a refutation of this meritocracy through the mechanism that breaks every supposed meritocracy: The map is not and cannot be the territory. Any objective evaluation criteria is necessarily separate from what it is trying to measure, and in that separation there is always an opportunity. Gurgeh has none of the background, training, or mindset expected for a player of Azad because he could not possibly care less about any of the things Azad represents to the Empire. What he has instead is a preternatural skill at games and vast experience with the most intricate strategy games the Culture, a much larger society, has been able to devise. He also has both the patience and the resources to devote himself entirely to learning a game for several years, and past experience in doing that with other games. If Azad represents the civil service exams, Gurgeh is the person who has no interest in ruling but adores memorizing facts and taking tests. The theory behind the exams is that the skills to pass the exam only come with the correct mindset to do the job for which the exam is testing. Gurgeh is an existence proof that this is not always the case. Banks also uses Azad to show another aspect of the failure of meritocracy: A society whose rulers are chosen through a competition takes on the shape of that competition. The Empire of Azad is run by the winners of competitive games, so the empire is a winner-take-all system of dominance and status hierarchy. Here, I think Banks lays the point on a little thick; the empire is an irredeemable hellhole of misogyny, sexual abuse, slavery, genocide, and military colonialism to a degree that is a bit hard to justify solely from the game. There is a beautiful turning point about two-thirds of the way through the book where Gurgeh's face is shoved into just how vile Azad society is and reconsiders his approach to the tournament as a result, and I think it may have been a bit stronger if the morality had been a little less blatant and absolute. To the extent that Gurgeh has political beliefs, he represents a Culture flavor of soft liberalism. He has opinions about acceptable and unacceptable ways to treat people, but he grew up in a utopia and his opinions are mostly theoretical. When he sees just how vile people can be outside of that utopia, he is revolted and appalled and redoubles his efforts to fight that society in the only way he knows how, inside of a game. This part of the book follows the standard, if enjoyable, plot of a flawed but fundamentally decent person discovering a true injustice and becoming enraged at it. In a lot of books, that would have been where the plot stops. Banks is doing something more subtle and more interesting, though. Gurgeh wipes the board with his next challenger, but that soft liberalism eventually proves inadequate. To learn the game of Azad and to play in the tournament, Gurgeh has been wrapping himself in Azad culture and its language, and in that frame of mind he is losing the climactic game of the book. It's only when he is pushed to think in Marain, the native language of the Culture, that he understands what is happening in the game and how to defeat Nicosar, the emperor. This, on the surface, is a bit too close to the strong hypothesis of linguistic relativity to be entirely plausible, but such an objection would miss the point that Banks is making here. Marain is a construct, the product of considerable effort within the Culture to match language to the most nuance and complexity that brains can understand, and it is a language, one of the most social and collective artifacts a society can produce. Gurgeh is a remarkable individual with an impressive talent, but individual skill and achievement can only take him so far. The critical final piece is the support of societal infrastructure intentionally built and maintained to help him make better decisions. Once I noticed that point, I saw it everywhere in the book. The empire repeatedly attempts to subvert or distract Gurgeh with drugs, pleasure, politics, or danger, and at each point there is some critical piece of Culture social infrastructure that blunts the attack. Illicit substances and forbidden vices are less tempting to someone for whom the illicit has been demystified by the Culture's gentler approach to rules and boundaries. Embedded biological mechanisms allow him to divert drugs so that they don't affect him. At first, it's easy to read this as an exercise of self-control, but on this re-read I saw how much behind-the-scenes infrastructure supports Gurgeh's ability to ignore temptation. This social support notably does not take the form of some ideological principle or moral framework. Gurgeh is not a monk or an ascetic, as is obvious from the first third of the book, and he has no political ideology to speak of. He is a flawed person with a streak of danger-seeking and self-aggrandizement, which the Culture exploited to get him involved in Azad. But through a lot of hard work, technological and social, the Culture has given him a robust foundation and a set of mental and biological tools that make him remarkably hard to corrupt. The implication is that if Gurgeh has that support, so does every other member of the Culture. It's neither a religion or an ideology; it's well-maintained infrastructure, complex and nuanced and pragmatic, and composed of innumerable small solutions to specific problems. I think the true climax of this book takes place the night before the final day of the game, in the tower meeting between Gurgeh and Nicosar. Gurgeh has realized that he's already won; there's nothing Nicosar can do to salvage the game. He's also seen that the game represents a cultural conflict and conversation between the Culture and Azad and he's overwhelmed by the beauty of that communication and sadness that the game is about to be over. Gurgeh's true passion is the game. It is doubtless easier for him to be magnanimous because he's winning, but he also loves the structure of the game itself and what two players can create in a sort of collaborative competition. Gurgeh tries to express all of this to Nicosar. It is one of the most centrist liberal moments I've ever read in a novel, the pure essence of "reaching across the aisle" or "disagreeing agreeably." Gurgeh has seen something beautiful, something he's created with Nicosar, a moment of true communication, and he wants to share it. Surely Nicosar sees the same thing; surely now that he sees Gurgeh has won, he can appreciate the board structure, savor the moment, understand the transient beauty of a game that is about to end and how perfectly it captures the meeting of their different cultures. That moment does Gurgeh real credit. It's a rare sign of emotional and spiritual depth in a character who often seems superficial. Nicosar meets this outreach with unhinged, furious contempt. He despises everything Gurgeh represents, everything the Culture is, and the next day he tries to kill Gurgeh on the board of the game. It is a devastating critique of liberal tolerance, all the more so because Gurgeh's attitude and outreach is truly admirable. It is perhaps the most sympathetic moment that Gurgeh has in the entire book, the moment where the reader thinks "oh, I get it, I understand what he really cares about." Gurgeh assumes that Nicosar is not his position or culture, that they have made a moment of connection that transcends all the awful things he previously learned about the empire of Azad. That Nicosar, despite being the emperor of the society that is currently doing so many things Gurgeh finds repulsive, cannot be as bad as his society. And Nicosar considers that outreach to be weak, disgusting, and vile, and does everything that he can to destroy it. One of the oddest twists of our current moment is the obsession that some billionaires have with stories that are moral arguments against exactly what those billionaires are currently doing. The most obvious example is Peter Thiel, who is obsessed with The Lord of the Rings and has devoted his life to becoming Saruman, a character who is notably not one of the protagonists. It's as if something in them recognizes the power of the story, but some deep shame or narcissism or simple aversion allows them to completely ignore what the story means. Elon Musk is obsessed with the Culture novels. He names the SpaceX rockets following Culture Ship naming conventions and has claimed that one of his goals is to bring about a Culture-style utopia. And in 1989, years before anyone had ever heard of him, Banks cast him as the villain of The Player of Games. There is so much of Nicosar in Musk: the superficial charm, the limited brilliance (Nicosar is a very good Azad player), the ambition, the pride, and the vicious, spitting contempt for everything the Culture represents at every level deeper than superficial materialism. And Banks is as clear about his opinion of Nicosar as he is about anything in any Culture novel. One of the oldest fictional answers to what a society does with people like Nicosar is the consequences of hubris. By being unable to accept defeat, by holding a vision of the world so tightly, they become brittle and unstable and bring about their own collapse. In a broad sense, that is what happens in The Player of Games with a bit of pushing from Special Circumstances. By the politics of the game, Nicosar had already won; the results of Gurgeh's earlier games had already been faked, the final game had no political consequences, and everyone who knew its true outcome could be disposed of. Gurgeh's win could have been covered up and ignored. But Nicosar could not endure the thought that he would be beaten by someone like Gurgeh, playing Azad the way that Gurgeh was playing it. Gurgeh had to be destroyed on the board of the game; Nicosar's pride did not allow any other outcome, even if it meant Nicosar's death. However, Special Circumstances didn't let hubris be the end of the story. In the climax of the book, the drone protecting Gurgeh also makes sure that Nicosar dies. There is a fig leaf of plausible deniability, but it's so obvious that even the unobservant Gurgeh sees through it immediately. It's hard to escape the feeling that was Banks's answer to what to do with people like Nicosar: They cannot live within society, because they will not live peacefully within society. I enjoyed The Player of Games as much this time through as I did the first time, but for entirely different reasons. In my first read, I focused on the world-building of the Culture, the political machinations, and the concept of games as conversations between the players. This time, I was struck by the political commentary just below the surface. Special Circumstances wanted to resolve the problem of the Empire of Azad without a military conflict and occupation that would be long, brutal, expensive, and demoralizing. They found an answer that relied on the diversity of the Culture. A vast, utopian civilization in which people can pursue whatever interests make them happy produces innumerable microspecialized oddities, people with astonishing talents in some small field that only a tiny fraction of people care about. It produces, in other words, innumerable keys for locks that you may never encounter, but which are invaluable if you happen to stumble across that lock. Gurgeh is not a hero. He is not a paragon of moral virtue, or even a charming charismatic, He is an entirely average member of an extraordinary society, the beneficiary of thousands of years of concerted effort at producing a robust, flexible foundation on which to raise robust, flexible citizens with a shared sense of basic morality. Those people, by themselves, do not solve all of life's problems; the structure of Special Circumstances and its willingness to bend rules in order to maintain them is the tension and deus ex machina in all of the Culture novels. But much of the strength of Special Circumstances is that it has an entire civilization of people like Gurgeh to draw upon when it needs them. It has those people because the Culture comprehensively rejects competitive meritocracy, something that some readers of the Culture novels appear incapable of comprehending. Rating: 9 out of 10

5 July 2026

Dirk Eddelbuettel: Rcpp 1.1.2 on CRAN: Usual Improvements in Semi-Annual Update

rcpp logo Team Rcpp is excited to share that an brandnew new version 1.1.2 of Rcpp is now on CRAN, has also been uploaded to Debian, and has already built for r2u and r-universe; Windows etc builds at CRAN should follow in due course. Rcpp has long established itself as the most popular way of enhancing R with C or C++ code. Right now, 3236 packages on CRAN depend on Rcpp for making analytical code go faster and further. On CRAN, 13.4% of all packages depend (directly) on Rcpp, and 61.4% of all compiled packages do. From the cloud mirror of CRAN (which is but a subset of all CRAN downloads), Rcpp has been downloaded 121.6 million times. The two published papers (also included in the package as preprint vignettes) have, respectively, 2263 (JSS, 2011) and 471 (TAS, 2018) citations, while the the book (Springer useR!, 2013) has another 742. The is the second update in the 1.1.* series which had, among other changes, switched to C++11 as the minimum standard. This release continues as usual with the six-months January-July cycle started with release 1.0.5 in July 2020. Interim snapshots are always available via the r-universe page and repo. We continue to strongly encourage the use of these development released and their testing we tend to run our systems with them too. Having said that, we would like to reiterate that we strongly object to the upstream R release and change management which in this 4.6.* cycle made several abrupt changes forcing packages which consume header files to make very abrupt change. Rcpp, just like numerous other CRAN packages demonstrates that API changes can be undertaken responsibly in a managed manner which allows for transition periods followed by possible warning periods, deprecations periods and finally (but only at long last) errors. What happened here is a speed run to the final stage of forced errors. Uncool and irritating for something as widely used as R. This forced us to make an interim release 1.1.1-1.1 even though we have of course had a policy of always keeping properly tested, installable, and error-free releases candidate version in the main repository branch and hence available via R-universe tested packages for all relevant platforms, and even via binaries for most (including Ubuntu LTS). It would be nice if R Core found a way to take advantage of this. Maybe development cycles, running apart for a year as they do for R, should also include selected packages. Once again I am not attempting to summarize the different changes. The full list follows below and details all these changes, their respective PRs and, if applicable, issue tickets. Big thanks from all of us to all contributors!

Changes in Rcpp release version 1.1.2 (2026-07-01)
  • Changes in Rcpp API:
    • Use of execinfo.h is again conditional to avoid build complexity (Dirk in #1445 addressing #1442)
    • An internal state component for Datetime is now int (Dirk in #1448 and #1449 fixing #1447)
    • Three new (in R 4.6.0) attribute accessors are used conditionally (Dirk in #1450 closing #1432)
    • An UBSAN error in the Sugar-based NA comparison has been corrected (I aki in #1453 fixing #1452)
    • Treatment of Inf outside of integer range in Sugar function has been corrected (I aki in #1458 fixing #1455)
    • Integer overflow protection has been added for sugar functions (I aki in #1457 fixing #1454)
    • The parent environment is now accessed via R_ParentEnv (Dirk in #1460 fixing #1459)
    • Change to returning dataptr again for better handling of empty vectors (I aki in #1462 fixing #1461)
    • Undefined behavior errors in use of ListOf proxies have been addressed (I aki in #1464 fixing #1463)
    • Under newer R version, R_UnboundValue is no longer used (I aki in #1466 fixing #1465)
    • New R API access point R_getRegisteredNamespace() is used with current R versions (Dirk in #1469 fixing #1468)
    • The Nullable::as() exporter now uses an explicit cast to the templated type (Dirk in #1471 fixing #1470)
    • A memory leak in the variadic Rcpp::warning() template has been fixed (Kevin in #1475 fixing #1474)
    • The Nullable::operatorT() has been added as a 'opt-out' (Dirk in #1477 with coordination in #1472)
    • Add templated integer-index overload for operator[] on small systems such as WASM (Jeroen Ooms in #1482)
    • The attribute accessors in AttributeProxyPolicy no longer rely on get__() (Kevin in #1484 fixing #1483)
  • Changes in Rcpp Documentation:
    • Reference in the bibliography used by the package vignettes have been updated.
  • Changes in Rcpp Deployment:
    • Excute permissions are set consistently on scripts with shebangs (Mattias Ellert in #1467)
    • R 4.5.* has been added to the CI matrix (Dirk in #1476)
    • Three nag messages issued when obsolete build flag accessors are used now show Rcpp::: (Dirk in #1480 fixing #1456)
    • Reference GitHub Actions have been updated to their current versions (Dirk in #1481)
  • Non-release Changes:
    • A non-release hotfix 1.1.1-1 used by CRAN accommodates breaking changes to the API in R 4.6.0. It would be nice to have the same level of release management in R itself that CRAN expects from us.

Thanks to my CRANberries, you can also look at a diff to the previous interim release along with pre-releases 1.1.1-1 and 1.1.1-1.1 that were needed because R-devel once again sudden decided to move fast and break things. Not our doing. And there also should not have been a need to two such uploads but it was amateur hour all around. Questions, comments etc should go to the GitHub discussion or issue section, or the Rcpp list. Bugs reports are welcome at the GitHub issue tracker as well. GitHub offers decent search for issue, pull requests and discussions; as many topics have been covered it is worth checking as well.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

3 July 2026

Matthew Garrett: Securing agentic identity

As is the case for many people working in the security industry, the last few months of my life have been focused on dealing with people wanting to use LLMs everywhere. From an enterprise security perspective that s not an inherent problem - what s more of a problem is that people want those agents to have access to resources like their calendar and email and so on, and now we have somewhat non-deterministic agents that seem very enthusiastic to achieve what you asked whether that s a good idea or not, and we re combining this with credentials that give them access to sensitive data, and leaving those credentials on disk where they can be committed into git repos or exfiltrated to some other service to make use of them on the agent s behalf or well just any other number of things, at which point your CEO s email is suddenly readable by everyone and you re having a bad day. As I mentioned in my last post, pretty much every strong mechanism for keeping credentials in place is just not supported in the wider world. We can imagine a universe where agents use hardware (or at least hypervisor) backed certificates to obtain credentials and any that end up leaking are worthless as a result. But, sadly, that s not an option for most people using existing identity providers. The state of the art is that you use the device code flow and a human authenticates and the token ends up back inside the agent environment and then it proceeds to do whatever it wants with it and you just hope that you wake up the next morning without an awful infoleak occurring. (An aside: I do not like the device code flow as used in enterprise environments, and I never will. The identity provider doesn t have a real opportuity to inspect the security posture of the system asking for the token, and as a result some identity providers will restrict tokens that are issued in this way. The common alternative of doing stuff using a more standard flow and having a redirect URI pointing at localhost works fine for local systems and is a pain for remote ones, even if you can commit crimes with SSH forwarding. I m going to suggest something that I think is better, and you are free to disagree) I m not in a position to get every identity provider and service provider to change their security posture, so I m somewhat stuck in terms of the tokens they re willing to issue me - largely either JWTs or opaque access tokens, with no support for any mechanism of binding that token to an instance. The token that s going to have to be provided to the remote service is something I have little influence over. But that doesn t mean I can t influence the token that lands inside the agent s environment. I can issue a placeholder token to the agent, and force it to communicate via a proxy that swaps out the placeholder for the real thing. The worst the agent can do is exfiltrate the placeholder token, and as long as malicious actors don t have access to that proxy, it doesn t matter - nobody else can do anything with the placeholder. This isn t a terribly novel insight, and it seems like almost everybody has reinvented this on their own. But a lot of these implementations involve you somehow obtaining the real token in advance and then pasting that into something that generates a placeholder that you provide to your agent environment somehow, and it s all a bit clunky and awkward, and it also means that you need to deal with something that keeps track of the mapping between placeholders and real tokens and oh no we ve just invented a secret store, and if you want this to work at scale and reliably you re just invented a high availability distributed secret store, and a lot of people who ve read that are now shaking their heads and reaching for gin. Can we simplify this, and improve security at the same time? I think we can! Remember when I said as long as malicious actors don t have access to that proxy, it doesn t matter ? What if they do? What if they compromise one machine inside your environment and are then able to email a bunch of employees and convince their agents to send more tokens back to them and then delete the email before a human reads it? Now you have someone inside the wall with access to those tokens, and presumably with access to the proxy, and now they can be anyone whose agent was gullible enough to think sending them a token was a good idea. This isn t good! So, I thought for a while, and I came up with a new idea. We can have a broker service that obtains credentials for us. We can run that centrally, away from the agents. A client in an agentic environment can request a token, and that can result in a URL being generated and the user being directed to open a URL in a browser and authenticate. When the user authenticates, the authentication flow redirects the confirmation back via the broker, and the broker obtains the real auth token. The obvious thing to do now would be to return the auth token to the client in the agentic environment, but we don t do that. Instead, we mint a new JWT, and add a new claim - one that contains an encrypted copy of the token. In the process we can copy over all the original claims, because those aren t secret - and now even if the client inspects the token to figure out what access it has, it ll get a correct answer. We sign the new token with our own signing key, and pass that back to the client. The client now has a legitimate JWT that is utterly useless, because the signature isn t trusted by anyone other than us. How does it use it? It makes an API request via a proxy, including the new token in the Authorization: header. The proxy verifies the signature on the token, and then decrypts the original token and swaps out the fake token for the real one. The remote API sees what it expects, and everyone is happy. There s never a real token in the agentic environment, but also we don t need to store anyting anywhere. The only state is the encryption keys, and those can be injected into the environment at startup. You need to scale? Just start more of these processes. You need to support multiple availability zones? Just start more of these processes in different places. No persistent data is ever held in the broker or the proxy. You don t need to care about distributed databases or secret stores. This felt wonderfully elegant and I felt smug about coming up with a better idea, and then I went to a bar earlier this week and sat down to read RFC 8705 and the guy next to me saw that over my shoulder and asked what I was reading and I explained why I was interested and we talked about agentic identity and then he mentioned that fly.io had something that sounded very similar and I read that and gosh yes it is very similar, so damn you fly.io for stealing my ideas 3 years before I even had them. Anyway. Now I need to do better. Remember that there s still a risk around anyone who has access to the proxy having access to the encrypted keys? We can remove that risk as well. It s not uncommon for agentic environments to have an identity issued via something like SPIFFE, at which point they have a client certificate. You can probably guess where I m going with this. If we require that an agent present a client cert to the broker when requesting a token, we can embed a representation of that client cert into the token we mint. The proxy can then require mTLS for the client connection, and can verify that the presented certificate matches the one represented in the token. If it does then whoever s using the token has access to the private key associated with the environment it was issued to. If we then ensure that the private keys backing these certificates are either hardware or hypervisor backed, and as such tied to a specific instance, we now have a high degree of confidence that the token can only be used in its intended environment. Even if our identity provider doesn t support RFC 8705, we can. This is fairly straightforward where you re using a platform where your identity provider is also the environment that s consuming your tokens, and more annoying for third parties. The broker potentially needs some amount of third party vendor knowledge to make that work for everyone. This is even more the case where login isn t via your identity provider (thanks, github), but none of this is insurmountable - just annoying. And where vendors issue opaque tokens rather than JWTs, this still isn t a problem; we can just mint a new JWT that includes the opaque token as an encrypted claim, and include the same certificate binding. The opaque token ends up being the thing that s presented to the third party, but only after we ve verified the mTLS binding. In an ideal world none of this would be necessary - someone would spin up a new agentic environment, a user would prove their identity, and a certificate embodying that identity would be issued to the environment with a private key that can t be exfiltrated. That certificate would be sufficient to obtain new certificates associated with the same private key, and we could still bind that into mTLS identity. This would be much simpler, but browsers don t support it, so it s not likely to happen any time soon. Anyway. Even if we can t have the best thing, we can do better than we are at the moment, and also it would be lovely if we could standardise on this rather than have everyone build their own thing. The end.

2 July 2026

Matthew Garrett: Preventing token theft

When you log into a service you re given an authentication token. Each further request to the site includes that token, allowing the server to figure out who you are and ensuring that you have access to your data. Depending on site policy, this token may either be stored in memory (and so vanish if you restart your browser) or disk. The token is the proof of your identity. As far as the site is concerned, anyone with your token is you. These tokens may be traditional browser cookies, but they may also be stored in either site local storage or (if you re not using a browser) in some other storage location. In recent years we ve seen infostealer malware (like LummaC2) gain the ability to exfiltrate user tokens, allowing attackers to gain access to the user s data without needing to retain access to the user s machine. This attack is viable even if the site has strong MFA requirements, so passkeys don t help. Encrypting the tokens on disk doesn t prevent the malware from scraping them out of the browser s RAM or obtaining whatever key is used to encrypt them. This feels like a pretty hard problem to solve. But that hasn t stopped people from trying! Dirk Balfanz wrote an IETF draft describing a mechanism for using self-signed certificates for TLS authentication. This uses the mutual authentication feature of the TLS protocol that requires both sides prove their identity to each other. In regular TLS, the remote site presents a signed certificate that tells you who it is. When performing mutual authentication, you then present a certificate to the remote site telling it who you are. These client certificates are largely unused outside enterprise environments because they re a huge pain to deploy. It s not so much that this has sharp edges, it s that it s entirely made of sharp edges. Managing certificate deployment to your devices is hard. Browsers get confused if the certificates change under them. You have one certificate and it lives forever, so sites you present it to can track your identity. Users are prompted to choose a certificate to authenticate with, and if they pick the wrong one everything breaks and is hard to recover. I ve deployed this and I did not have a good time. But Balfanz s idea was simple. Rather than require certificates to be deployed, browsers would simply generate a certificate on the fly. The goal wasn t to prove the device or user s identity in any global way - but it would associate a TLS session with a specific certificate. You could then, for example, include a hash of the certificate in the cookie, and if someone tried to use that cookie without presenting that certificate then the cookie could be rejected. If the browser used a hardware-backed private key for the certificate then it would be impossible for an attacker to steal it. Sure, you could still steal cookies, but you wouldn t be able to use them. This was written almost 15 years ago, and seems simple, elegant, and functional. It didn t happen. Part of the reason for that is that, well, it wasn t quite so simple. One problem was privacy related. Cookies are only sent after the TLS session is established, so anyone monitoring the network doesn t know anything about the user identity. A naive implementation of this approach would have meant the client certificate being sent before session establishment, and now user identity can be tracked (no longer an issue if this was implemented on top of TLS 1.3, but this was a log time ago). This was avoided by reordering the client handshake, but that meant having to modify the TLS specification and implementations would have to be updated to support this. Another was that figuring out the granularity of the certificates was difficult. You d want to use different certificates for every site to avoid them effectively becoming tracking cookies, but you need to provide the certificate before cookies are set, and you don t know what origin the site is going to set in its cookies. If you generate a certificate for a.example.com and a different one for b.example.com, and a.example.com sets a cookie for *.example.com and includes the certificate you used for a.example.com, that cookie isn t going to work on b.example.com and things are broken. This meant supporting it wasn t as straightforward as it seemed - you d need to ensure that your cookie scope was compatible with the certificate scope. You could probably make this work well enough by aligning it with the Public Suffix List, but there was still some risk of expectations not being aligned. And, perhaps most importantly, TLS session resumption (replaced by pre-shared keys in TLS 1.3) somewhat defeats the purpose of the exercise - clients store state that allows them to re-establish a TLS connection without performing certificate exchange (this reduces overhead if a connection gets interrupted or you switch to a new network or anything along those lines), and anyone in a position to steal cookies could steal that state as well. The followup attempt was channel IDs. This simplified the implementation somewhat - rather than certificates, a raw public key would be sent, along with proof of possession of the private key in the form of a signature over a portion of the TLS handshake. This was required even in the event of session resumption, which avoided having to worry about theft of session secrets. The timing of the exchange was after the encrypted session had been established, so user identity couldn t be leaked that way either. Cookies could then be bound to this identifier. Unfortunately it didn t really deal with the problem of scoping keys in a way that would match cookie requirements, and the spec suggests that the right way of handling this is to scope keys to TLDs, which would enable user tracking across sites (Chrome s implementation apparently restricted it to eTLD+1, which would match the third party cookie policy and avoid the tracking risk). Chrome added support for this, but it was removed in early 2018. The discussion of some of the pain points in that message is interesting, explicitly calling out problems with connection coalescing across domains and the incompatibility with zero-RTT TLS1.3. The overall consensus at the time seems to be that trying to solve this entirely at the TLS layer has too many rough edges, and a different approach should be taken. And so almost 7 years after the initial draft for origin bound certificates, we come to token binding. This ended up being a rather more complex endeavour, covering 3 different RFCs describing how it impacts TLS, how to incorporate it into HTTP, and how to manage all the various parties involved in the process. The short version is that it s pretty similar to channel ID, except that there s also a documented mechanism for allowing tokens to be bound to one party and consumed by another, avoiding any need for widely scoped keys. Token binding effectively solved all the issues in the original proposal, but at the cost of somewhat more complexity. The RFC was finalised in October 2018. Chrome removed its (incomplete, draft) support for token binding in November 2018. Edge carried support until late 2024. Despite getting all the way through the RFC process, it s functionally dead. The process up until this point had been largely initiated by Google, with Microsoft contributing significantly to the token binding standards. The work had been focused on identifying a generic solution to the problem rather than tying it to any specific authentication flow. The next step was in a different direction - rather than trying to fix this for the entire internet, how about we try to fix it for OAuth? RFC 8705 is titled OAuth 2.0 Mutual-TLS Client Authentication and Certificate-Bound Access Tokens . This is basically the 2011 approach, but (a) with an explicit definition of how the certificate should be incorporated into issued auth cookies, and (b) with a proviso that well uh if you re going to use tokens issued by your IdP to authenticate to someone else then well you re going to need to use the same cert for both. This is probably fine for the company-owned-laptop case where you re actually fine with multiple sites being able to tie identities together (that s kind of the point here!), and also works for I am using an app and not a browser , but doesn t work for more generic scenarios. It also doesn t seem to take the session resumption case into account at all? Support for RFC8705 seems poor, as far as I can tell of the big players only Auth0 implements it. In theory it works fine with self-signed client certs but in reality that s going to be almost as difficult to support across multiple platforms as just issuing proper client certs in the first place, so deployment is going to be kind of a pain. But the good news is it doesn t rely on any TLS extensions or custom browser behaviour, so at the client side it works fine with any browser. Which brings us on to RFC 9449, Demonstrating Proof of Possession . This goes even further than RFC8705 in terms of reducing the burden of deployment - it works fine with existing browsers, and it doesn t even require any certs. The client generates a keypair and provides the pubkey when requesting the cookie. The cookie contains the pubkey. Every request to the service now provides the cookie with the pubkey and also provides a signature over the URI and HTTP method. If the signature matches the pubkey in the token then clearly the signature came from the machine the token was issued to, and everything is good. This does come with some downsides, though. The first is that it uses browser interfaces to generate the keys (typically crypto.subtle.generatekey()) and as far as I can tell there are no browsers that guarantee that that key is going to be generated in hardware even if it s marked non-exportable, so anyone able to steal the cookies can also steal the keys. The second is that the signature only covers the URI and HTTP method, and not the message content or any other headers, so anyone able to exfiltrate a valid signature can replay it against the same URI with different message content. The recommended way to handle this is to reject any signatures that weren t generated within the last few seconds, which is a wonderful additional way to allow clock skew to give you a Bad Day. And the third is that every single request has to be separately signed, which is not intrinsically a problem because computers are fast and have multiple cores, but if you re trying to solve the first problem by sticking the key in a TPM then you re dealing with something that s slow and single threaded and that s maybe acceptable if you re using client certificates (because there s going to be one signature per session and you can use the same session for multiple requests) but probably not if you re dealing with a user opening a browser that restores previous tabs and each of those is a webapp that fires off 100 requests in parallel. In case it wasn t clear, I don t like DPoP. It doesn t feel like it actually solves the underlying problem that we see in the real world (malware running in a context where if it can grab the tokens it can grab the keys), it adds a massive amount of overhead, and it has baked in replay vulnerabilities. I don t know why it exists and I m incredibly suspicious of vendors telling me that it fixes my problems, because if they re telling me that then I m going to end up assuming that they either don t understand my problems or they don t understand their technology, and neither of those is good. Still. Then we get to the thing that prompted me to write this - Chrome s announcement that they had launched device-bound session credentials. This is interesting because it s a Chrome feature that s explicitly intended to counter on-device malware, which was one of the things that was out of scope in 2018 when token binding was being removed. Since this is entire web level it doesn t have to be an RFC, and so is instead defined by W3C. I m going to handwave all the complexity and say that it s basically a way to register a public key when a cookie is issued, and then prove possession of the private key when it s time to renew the cookie. By making the cookies shortlived and having support for rotating them in the background, user impact is basically zero and while it s still possible for an attacker to exfiltrate and use a cookie they ll only be able to do so for a short window before it needs to be refreshed - something the attacker can t do, since they don t have the private key. This avoids the DPoP overhead because you only need to do signing once per cookie per cookie lifetime, and not on every single request. I don t like this due to the window where exfiltrated tokens can be used, but it feels like a strict improvement over the status quo. An extension called device-bound session credentials for enterprise allows pre-enrollment of device keys, so even though the actual runtime DBCE flow doesn t involve certificates, certificates can be used for device registration in enterprise environments and you can make sure that auth cookies only go to trusted devices. Unfortunately this is Chrome-only, and so we re going to need to wait for it to be backported to all the random app frameworks for it to have widespread support on mobile or for almost everyone s desktop app that s actually three websites in an Electron wrapper. Mozilla s current position is that they re not in favour of it, so I guess we ll see where Safari lands in terms of broad uptake. The last thing on my list is another client cert/OAuth binding, this one still in draft state at the time of writing. This one is aimed primarily at the use of agent-driven tooling, where you have something running in the background using a whole bunch of tools that are each acting on your behalf. Authenticating to all of them separately isn t a fun time, but giving broadly scoped access tokens to a non-deterministic agent and trusting that it ll never post them somewhere public also isn t a fun time. The key distinction between it and RFC8705 is that it s aimed at connections rather than sessions, which avoids the worries about session resumption. This is done with TLS Exporters, which in TLS 1.3 should be unique to the connection even over session resumption (TLS 1.2 may reuse some of the same key material for exporters over session resumption, so it s recommended to enforce 1.3 for this). By providing a new signature alongside the cookie on every new connection, the client proves that it still has access to the private key. This is a very new spec and I haven t had much time to work through it yet, but my naive understanding is that unlike RFC8705 this would require some additional client support to be able to regenerate the client signature on every TLS reconnection. This doesn t avoid all the problems that RFC8705 has, including how to scope certificates. For the agentic use case that probably doesn t matter - all these tools are acting on behalf of the same user, it s fine if all the sites involved know they re the same user. But it doesn t solve the general purpose user use case, and right now DBSC seems like the best we have there. But. Part of me still wonders whether Dirk Balfanz s approach was the right one. Yes, there s risk associated with TLS session resumption, but in the worst case you could just switch that off for high risk setups. The cookie scope argument is real, and also in cases where it could violate privacy the site owner could already choose to broaden their cookie scope and violate your privacy, and in cases where it breaks things you could just not make use of it. The other problems are largely fixed by TLS 1.3, and then we re just left with Browsers handle client certificates badly to which my answer is Yes, and we should fix that anyway . Despite having a pretty good answer to this solution over a decade ago, the closest we have to actual deployment is something that offers strictly worse security guarantees. And tokens keep getting stolen, and compromises keep occurring, and for the most part people shrug and get on with things.

27 June 2026

Steve McIntyre: It's dead, Jim!

I previously wrote about the upcoming UEFI CA rollover. Well, it's happened now - the old Microsoft UEFI CA from 2011 expired yesterday: Third Party Marketplace Root (used for signing option ROMs and other software)
  Subject: C=US, ST=Washington, L=Redmond, O=Microsoft Corporation, CN=Microsoft Corporation UEFI CA 2011
  Validity
    Not Before: Jun 27 21:22:45 2011 GMT
    Not After : Jun 27 21:32:45 2026 GMT
It's dead - it's not coming back... The world doesn't seem to have ended yesterday, so I guess we did ok? :-) How did we do? After a lot of prodding behind the scenes, Debian and many other distributions managed to get new shim binaries dual-signed with both the old and new CAs. The members of the shim-review team did a sterling job with reviews in the last few weeks. Since I started pushing people in May, we've had 21 reviews accepted successfully - see here for the list. Great stuff! Microsoft have also been working quickly - many of those shim submissions were accepted and signed by Microsoft very quickly too, with a turnaround time of less than 1 day in some cases. Not all of those signed shims have been published and used by the distros involved yet, but expect to see them in the wild in the coming weeks and months. These binaries should be good for people to use for the foreseeable future, until either we need to do another CA rollover or (sadly, more likely) we find an issue in shim that necessitates a new release. What's next? We already have one of our new dual-signed shim binaries in place in Debian, in unstable and testing (Forky) right now. In a couple of weeks from now, we'll be rolling out very similar new dual-signed shim binaries in the next point releases for Debian 12 (bookworm) and Debian 13 (trixie). We'll also be upgrading fwupd in both those point releases, to make DB and KEK updates work better. For more information about these updates, see https://wiki.debian.org/SecureBoot/CAChanges. For your own safety, validate that your systems are updated when possible. If you don't, they may fail to boot in future.

26 June 2026

Jonathan Wiltshire: Streamlining Debian Updates with AI: The Stable Update Adviser

Please consider supporting my work in Debian and elsewhere through Liberapay.


Some months ago I started working on a small experiment: a specialised LLM assistant for Debian maintainers preparing proposed updates for stable and oldstable.

The aim is not to replace review by the Stable Release Managers, nor to automate away maintainer judgement. It is meant to be a pre-advice tool: something maintainers can use while they are still preparing a proposed update, before sending it to other people to review.

If you ve uploaded to stable before you may have some experience with our acceptance criteria. Does the version sort correctly? Is the changelog clear enough? Is the delta focused? Are the patches documented? Are the right bugs or CVEs mentioned? Are they of sufficient severity? What is the risk to users of stable? Is there an ABI change hiding in the diff?

These are the sort of checks where a structured first pass can be useful, provided it is treated as advice rather than authority.

To explain what the tool does, what could be more appropriate in an AI age than to ask it?

Describe yourself and your abilities and limitations for a blog post.

It answers:

I m Debian Stable Update Adviser, a specialised ChatGPT assistant for Debian maintainers preparing proposed updates for Debian stable and oldstable.

My main job is to review a single .debdiff and give practical, policy-aware feedback before a maintainer submits or uploads a proposed update. I focus on whether the update appears suitable for Debian stable or oldstable, especially under the Stable Release Managers expectations for minimal, targeted, low-risk changes.

I can inspect a debdiff for Debian changelog quality, including suite, version, bug references, CVE mentions, and rationale. I can check version correctness using Debian dpkg version semantics. I can review packaging changes under debian/, such as patches, maintscripts, symbols files, conffiles, Build-Depends, autopkgtests, and control metadata. I can also group upstream source changes into behavioural and non-behavioural categories, and flag areas that may carry regression risk.

My reports are intended to be concise and useful to maintainers. A typical review includes a quick summary, behavioural and non-behavioural change breakdown, mechanical policy checks, risk assessment, stable-policy checklist, acceptance likelihood, rationale, maintainer advice, and a clear conclusion.

My limitations are important. I cannot prove that a package builds, that tests pass, that the bug is fully fixed, or that the Stable Release Managers will accept the upload. I also do not replace maintainer judgment, archive review, autopkgtest results, reproducibility checks, or real-world upgrade testing. I can only review the provided debdiff and point out likely issues, risks, and improvements.

I also do not proceed without a valid debdiff. The whole point of my review is to assess the actual proposed delta, not to speculate from package names or bug descriptions alone.

In short, I act as a focused pre-review assistant for Debian stable and oldstable proposed updates: useful for catching policy, versioning, changelog, packaging, and regression-risk issues early, but not a substitute for the Debian review process itself.

That last point is an important one. This is not an oracle, and it is not part of the Debian archive machinery. It is a checklist with some language-model assistance, aimed at reading a debdiff and pointing out things that might be worth another look.

I know that commercial AI and LLM tools are not universally welcome in Debian. That is understandable. Debian depends on transparency, human responsibility, licensing clarity, and technical correctness. LLMs have obvious problems in all of those areas. They can be wrong, and worse, they can be wrong in a fluent and plausible way. They are impossible to reproduce and their training is opaque.

But I see this as a useful first pass for a maintainer who is unused to working in stable, and would benefit from a virtual mentor giving their proposal a quick check and reassurance. Perhaps they don t have a more experienced co-maintainer to ask. Perhaps they are conscious that stable reviews are presently a two-man effort and want to avoid adding round trips to that load. Perhaps they just need some reassurance.

So despite my reservations I am today opening the adviser up for general use, and I m interested in feedback about how it responds to real world proposals in various states. Most of the examples I have tested with already had a green light, so the value added by the adviser is limited. I would especially be interested in seeing a transcript alongside the submitted debdiff.

Try it out

I would dearly love to build this in a more Debian-ish environment, but for now I m limited in resources and skill to do that (help is welcome). Until that s a reality, you can try out the ChatGPT implementation: Debian Stable Update Adviser

20 June 2026

Gunnar Wolf: systemd for Linux SysAdmins

This post is a review for Computing Reviews for systemd for Linux SysAdmins , a book published in Apress
systemd. Yes, in full lowercase. If there was ever a technology to cause controversy in the Linux world, this is it. Since its inception in 2010, systemd s goals were set quite high: to replace the vital part in every Linux system that takes care of the system boot process. It quickly reached maturity, allowing it to be adopted as the main init system in most major distributions just five years later. Despite describing events that happened over a decade ago, systemd adoption still raises the temperature in any Linux-related discussion. David Both s comprehensive book tackles the what, why, and how issues surrounding systemd. Carefully divided into 16 chapters, going from the basics and some of the technical and political history behind the project to the different subsystems and aspects covered by systemd, its almost 450 pages can scare people away. But the text is written in a very clear, tutorial-like fashion, and while it can be read sequentially, cover-to-cover, readers can also pick a single aspect and jump straight to the relevant chapter. A frequent criticism of the systemd project is that it aims to basically rewrite all of a Linux system, and just looking at this book s index shows there is some truth to it. The first chapter is an introduction to the systemd project and a brief overview of its history (including the controversies around it), and the following four chapters deal with understanding and controlling the system boot process. That leaves ten chapters to cover different aspects or subprojects of systemd, such as time and date issues (synchronization, time specifications, and controlling repetitive tasks), understanding and leveraging the system journal that strongly departs from the old syslog system, network configuration and firewall management, system health and performance debugging all aspects that in the traditional Unix philosophy were managed by independent programs. And I can identify several systemd subprojects not covered by this book! We long-time Unix and Linux administrators took pride in how highly performant and stable systems were supported by the simplicity of our tools; systemd critics point out this massive project has absorbed dozens of individual tools, yielding corporate control over vast swaths of vital system tooling. Truth is, as a sysadmin myself, systemd is today one of my greatest allies. I appreciate how the author evaluates every component independently, including his personal evaluation of each even acknowledging when he prefers working with the traditional programs. If I had to note one criticism: given the many console captures, having a maximum width below 70 characters means several lines are unnaturally cut short (and continued with odd indentations). There is probably no right way to solve this, but it does affect the reading experience.

19 June 2026

Wouter Verhelst: Agentic coding and Free Software

Through work, I have paid license to windsurf (recently renamed to "devin"), an application for LLM-based (aka, "Agentic") development. I hadn't been using it that much, but in an effort to more clearly understand how this whole AI development thing works, I decided to give it a closer look recently. My conclusions: In its current form, this whole LLM wave is problematic for multiple reasons. But ignoring that, and looking at the technology only, I can say that:

Problems Lest someone (incorrectly) assume that I am arguing in favour of the current state of affairs with regards to LLMs, let me state this first. The way LLMs are built today is highly parasitic. Websites are downloaded in whole, at unsustainable rates, regardless of the consent of the people who made the original content. The result is predictable: servers get overloaded, server administrators attempt to implement various mitigations. Some of these mitigations work; some do, for a while; some are entirely useless. In actual fact, the mitigations are an arms race -- if too many people implement the same mitigation, then the people who try to build yet another LLM so they can extract rent will just try to work around the mitigation, eventually they will succeed, and you'll just have to come up with another mitigation. It's a bit like spam; you introduce regex-based spam filters, they introduce spelling mistakes, you introduce bayesian filters, they add a large batch of markov chain-generated semi-nonsense words made invisible by markup, you add filters to block emails with such markup, they move the text into an image. We have working mitigations today, but eventually we'll run out of ideas. LLMs glob up everything they can while ignoring the license of the source material. The people who push those LLMs claim that pushing the source material through the machine learning algorithms makes the output of the algorithm distinct enough from the source material that the license no longer applies; I'm not so sure that this is true. I guess the New York Times v OpenAI lawsuit will teach us some of the answer to that question here, but even so the ethical questions about "is it OK to bring down another server just so we can download the internet for another for-pay LLM" are still open. And regardless of what the law states, my opinion on "you're using my copyleft code to generate code under a different license" is not something you might like if you agree with the rent seekers' opinion on the subject. That all being said and true, the technology works. You can have a "conversation" with an LLM that resembles a human one. If you pass it some data, you can use plain english to ask it questions about that data, which is a lot easier than to ask it about that in a formal way. You can request it to generate some code, and it will generate something that looks like what you need and that will be mostly correct for like 95% of the time. Now, yes, 95% of the time is not 100% of the time, and no, you can't ask it to "write me a piece of software that implements this 300-page requirements document and get back to me when you're done", because it will fail, and you won't know where it has failed, and you'll take it into production and expect everything to be fine because it won't and this one minor logic bug will cause half your servers to spin and consume credits with your infrastructure provider with nothing to show for it. But that doesn't mean you can't use an LLM to build a large piece of software. It just means you have to understand the LLMs limitations and strenghts, and use them correctly. Here's what an LLM is good at:
  • Generating plausible text
  • Interpreting text to figure out what a plausible meaning or summary of that text is
  • Giving vague indications as to what the probable context of a given body of text is.
It turns out that that's enough to use the LLM to build a reliable piece of software, provided you do it right.

Paradigm shift An LLM can generate text by the truckful. The generated text could be code. Given a good enough LLM, the generated text might even run and do something useful. You can try to blindly run the code, and if it doesn't run correctly, you can paste the error message to the LLM, and it can tell you what went wrong and how you could possibly fix it. This creates a feedback loop: you ask it for an amount of code, you run the code, you receive an error, you tell it that the code is problematic and give it the error message, it makes changes to the code, now you have something that at least no longer fails at startup. If you ask it to add tests to make sure that your code acts as per your specification, now you get an error if and when the code doesn't act as per your specification. Or, well, at least not as per the part of the specification that was correctly turned into a unit test by the LLM. LLMs have a context window, so if the error message is pasted in the same conversation as where the code was generated, it is able to reuse the earlier prompts to refine how it should interpret the error message that you received. You can't really paste the source code of an entire application into the prompt of your LLM, that would quickly overrun its context window. But LLMs also allow you to provide some form of background information -- a document, say -- on which you ask it to reason. It will interpret that document, but doing so uses less of the LLMs context window. So providing the LLM with your application's source code as background information can help it understand better how your code interacts. This is especially helpful if you only provide the LLM the background information relevant to the actual question. So now if you are able to:
  • Create background context with your application's source code
  • Have the LLM generate a first draft of your requested change, plus the tests to make sure it works
  • Compile (if applicable) the generated code (and tests) and run said tests
  • Return any error messages to the LLM with a request to correct the error
Then the combination of "getting it 95% right off the bat" and the above feedback loop means you can generate syntactically correct code, that probably does what you need, in minutes. I say "probably" for a reason. There are going to be cases where you specify a request without a number of details (because they are implied), and the LLM will get most of those details right but just not implement the one bit because it's an automaton and it doesn't think. Or you will ask it to make sure that two bits of the application look exactly the same, without specifying that they must act the same, now and in the future, and it will just generate the same block of code twice and then in a future change it will change one but not the other. But if you review the changes, and you have experience as a programmer, you will be able to spot most cases where the LLM got it wrong. And so it's possible, if not necessarily easy at first, to use an LLM to generate mostly correct code. There are certain places where "mostly correct" code is not desireable. But equally, there are also cases where, "mostly correct" is good enough. After all, most of the software you run today -- the bits of it that weren't, yet, generated by an LLM -- is only "mostly correct", too, because to err is human and we all make mistakes. If not, there wouldn't be any CVEs and your software would never do anything wrong. Now, doing the feedback loop described above is certainly something you could do manually. You could open an account on one of the LLM websites, upload the source code of your application, ask it to generate some new feature, download the newly generated feature, run it, and then copy/paste any error messages back into the LLM. But that's a lot of manual work of the type that computers are pretty good at. So that's what the "windsurf" tool helps you with: you run it inside your IDE -- either a VSCode-based tool that you download from their website which comes with their product preinstalled, or a separate JetBrains plugin that you can install. You can then open your entire relevant codebase in a workspace in your IDE. You then ask the LLM, through the IDE, to generate a new feature in your codebase, and to also generate the test while it's at it. It will use a mixture of LLM interpretation and non-LLM functionality to scoop out the relevant bits of your codebase to send to the LLM as background information, will send it your prompt, will download the generated code and patch or create files, will compile (if required) and run the newly generated code and tests, and will refine the generated code if the tests produce any errors. All mostly automatic; by default, running anything requires explicit confirmation. You can turn that off completely (probably not a good idea), or you can give it a whitelist of things that you don't want to confirm (perhaps OK), and the tool also passes standing instructions to the LLM to never generate any command that deletes a file (which, like with any LLM, can be overridden, but it requires you to be very stubborn and to use more credits than you'd probably like). All this put together means you can build something without writing any piece of code, provided you do it right.

A technically positive evolution Don't go and say, "here's a 300-page document, read it and write whatever the document says". It will get it wrong, it will write a massive test suite that it will only run at the end, it will choke itself up trying to interpret the massive amount of failures it encounters, it will fill up its context window and it will start to forget some of the requirements. That won't work. But what you can do -- what I did, in fact -- is this. First, create an empty workspace. Don't put any code in it. Then, tell the LLM to generate a backend framework using technology X and a frontend framework using technology Y that initially only says "hello, world". Also add tests to it, and run the tests. It will do that. You'll not get much, but it will work. Then, ask it to add some UI elements. A login page, perhaps. A navigation bar. Small things. Most of it doesn't have to be functional -- but tests must be there for the bits that are, and have it run the tests and evaluate the results. Rinse, repeat, until you have a working application. Importantly, in between the steps, you should also run the application yourself and see if the change was implemented correctly. Sometimes it won't be. Sometimes there will be a subtle bug -- I at one point had a the application hang after a few minutes. Sometimes you tell it that there's a subtle bug, and it will discover it more quickly than you could, and it will fix it, and in implementing the fix it will uncover another bug, and then you have to fix that one -- the fix it came up with for the hang was to move something to an async process on the server, which caused the application to start spinning while trying to create hundreds of async jobs (this is when I realized that the hang was a deadlock due to some part of the codebase doing something that indirectly triggered itself). Sometimes it will try to fix the bug you tell it about, and you'll see that it's going off on a tangent that has nothing to do with what you're seeing. It's important to keep an eye on what it's doing, so you can guide it back on track when that happens -- when I told it about the hang, it started investigating the part of the code which sends out emails, thinking that it could hang while waiting for sendmail to finish, but the hang was happening when the application was idle, not when it was sending out emails, and only when I told it about it happening when it was idle did it find the deadlock. So it's not a fully automatic process, and it needs to be guided by someone who knows what they're doing. But if that is the case, you can come up with something that works. I spent evenings and breaks for about a week, and I managed to create a working application which, had I written it by hand, would have taken me a few months of full-time work to come up with. And I now have a side project, fully complete and working, that I had been thinking about doing for more than a decade, but never got around to actually doing, because of all the work that would be involved and I just didn't see myself having the time for. It's not perfect code. But it's mostly good enough, and it will perform the job it needs to. And it looks far slicker than most of the side projects I've done in the past, because in the past I would prioritize between implementing new features or making something look slick, and I would decide that the new feature was more important because it's only for me and there's only me and nobody cares if it looks good or not and I don't have three weeks to come up with something that looks better. But here, I found myself sometimes spending 10 minutes writing a prompt with instructions on making things look better. Because what's 10 minutes when you just spent an hour writing down and refining specifications for functionality and tests? There are a number of other things in which an LLM can help a programmer. For instance. I received a bug report recently in a project I'm paid to maintain that I couldn't make heads or tails of. I opened the source code in my windsurf IDE, pasted the bug report in the prompt, and then requested the tool to analyze the source code and the associated logs and tell me how the described behavior could be happening. It turned out that I had overlooked something, but with the help of the tool, I found the bug in minutes. I was trying to understand a particular part of a large codebase that I didn't really grasp very well. I loaded the codebase in the tool, and asked it to explain to me how a particular action is performed by the code. I requested specific functions and line numbers. I now have a far better understanding of how the code works, and will be able to write that patch that I've been wanting to write for years -- without using the LLM. I have been struggling for, literally, years with understanding why another tool that I maintain was misbehaving in a particular way but only in Firefox. I opened the codebase in Firefox, explained the buggy behavior in plain English, and asked it to explain how this could be happening. It picked up some obscure corner case behavior of ffmpeg and mp4 containers that I was not aware of and that perfectly explained why things were misbehaving in the way that they were. At the same time, there are limitations. Giving an LLM a codebase that was originally generated by an LLM (either the same one or another one) seems to work well. Giving it a codebase that was written by a human and expecting it to correctly update it seems to be more error-prone. I did one or two of those as a trial, and it is more problematic than anything. An LLM is also not intelligent, notwithstanding the popular term of "Artificial Intelligence". On multiple occasions, I've asked it to write a test case for some code that was not set up to do so; and rather than suggesting a refactor is required, it would instead copy the code that needed to be tested and then test the copy, rather than the original. The tool has made multiple similar errors. I have sometimes people describe agentic coding as "similar to interacting with junior programmers", but that is not the case. A junior programmer will either fill in the gaps in your specifications, or ask for clarification when something seems off. The LLM will not do that; it will do what you ask, exactly that and nothing more. If you missed a corner case in your specification, then all bets are off. I remember learning about programming language generations in college. A first-generation language is "machine code", a second-generation language is "assembler", a third-generation language is any high-level language such as C, Perl, or Pascal. I've forgotten what set a 3rd-generation language apart from a 4th-generation language. But I remember the definition they gave me for a 5th-generation language: "you tell the computer what to do, and it will do it". At the time, I thought it was ridiculous. Nobody could ever write something like that. But it's here. And it's a threat to free software.

A threat to free software? Yes. There is the obvious part where most of the well-known LLMs are non-free software. I mean, there are some "open source" LLM models. The windsurf tool that I used doesn't allow you to use them (directly), but they're there. There are also open source applications that implement what the windsurf editor does. So it's definitely possible to work like this without resorting to non-free software and non-free services, even though the non-free LLMs might be a bit ahead of the curve of the free ones. But that's not what I mean. And there is also the obvious thing which I mentioned earlier in this post, which is that the people who try to build LLMs are doing it in unethical, disgusting ways, causing downtimes and disregarding licenses for whatever they can get their grubby hands on. Ideally we wouldn't be in that situation, and ideally this wouldn't be a problem, but we are where we are. And there's the obvious thing where the OSI sold itself out and declared that a machine learning program can be open source even when the very things it was built from -- the training data -- is not available. That's a major issue that the free software community needs to fight against, but there's not really anything that that is a threat to free software. You just build your own, free software, LLM, and you're done. The actual threat is in funding and developer support. Most large businesses do not care about free-as-in-freedom software. They like the free-as-in-beer part, and they appreciate that the free-as-in-freedom bits can make the software more customizable. They are (mostly) happy to do sponsorships of the free-as-in-freedom projects that they use if that means their free-as-in-beer usage of the software gets improved. But why would you care about all that when you can just generate the code you need, rather than interacting with an open source community that may or may not care about your business's interests?

Where to go from here Although I think the moral and environmental issues with LLMs are real and problematic, given the experiments I did I am not convinced that the concept of interacting with a computer system in natural language and to use it to generate code is necessarily deficient. There are pitfalls, but they can be managed. It is possible to use such a system to create throwaway, proof-of-concept type "good enough" code bases. It can be used to interpret code bases and to understand bug reports. I believe that the major issue with LLMs has to do with that saying about hammers and nails:
If all you have is a hammer, then everything looks like a nail.
LLMs are an outgrowth of machine learning, pushed by large corporations. These large corporations have a lot of money. If all you have is money, then every problem can be fixed by throwing more money at it. The initial language models were promising but not (yet) good enough, and it seemed that one way in which they could be improved was to increase the scale of the statistics: throw more hardware (and thus money) at it, and rather than improving the efficiency of the models, just scale up. Scaling up is something that megacorporations are very good at. It's only a money problem, after all. Does that mean that "scaling up" is the only way to improve the models, though? I'm not convinced. Some hardware, such as most modern Apple and Samsung devices, ship with accelerator hardware for machine learning algorithms. There are some models that are small enough to be able to run on these devices. I don't see why it should not be possible to create a small(er) language model that can do some useful part of the above-described use cases; if not locally, then at least on a server that one can run on-prem rather than requiring that you pay rent to one of the LLM companies. The Software Freedom Conservancy has published an aspirational statement on machine learning-assisted programming that, I think, gets a lot right. It's not quite a definition, but it's something to keep in mind. Perhaps that's the way forward? More questions than answers at this point, anyway.

16 June 2026

Vincent Bernat: Building a Soviet Nail Factory: how KPIs killed efficiency

In 2008, I landed my second job, in the network team at Orange Portails1, the division behind the websites and search engine of the French telecom operator Orange. The place ran like clockwork: a comprehensive technical setup, a dedicated team for every part of the business, and room to focus on what I do best. A few years later, none of that mattered: thanks to an obsession with the numbers, we could no longer deliver new services on time.

Disclaimer This is a story I like to tell to warn people about Goodhart s law.2 As these events happened almost 15 years ago, my recollection is a bit fuzzy. I left in 2012.

The first years During my first years, the department operated like a startup. Its cradle was the French company Echo. They built a search engine. France T l com bought it and renamed it Voila. It was the most visited search engine in France in the early 2000s. France T l com consolidated the portal activities into the Wanadoo Portails division, later renamed Orange Portails. The technical environment was excellent. We had many internal tools:3 a ticket system, an RRD-based graphing tool, an IPAM, a reporting tool, and an SNMP-based alerting tool.4 We deployed our Linux servers with CFEngine. We installed systems and applications from internal Debian repositories. We documented everything in a private MediaWiki instance. Supervision was performed with an ancestor of Xymon. The network architecture was clean and scalable with little legacy. We onboarded new people in a day. It was a nurturing environment for me. I developed several tools: lldpd, an 802.1AB implementation, Snimpy, a pythonic binding for Net-SNMP, Wiremaps, a layer-2 discovery tool with a time machine to know which device is connected where, Kit r , a tool to simulate network conditions, QCSS-3, a controller for load-balancers, and ipoo, a service available through a Jabber chatbot and a Greasemonkey script to expose IP-related information. I added SNMP support for Keepalived and Quagga. I also started this blog, with articles like Anycast DNS, TLS-related articles like TLS computational DoS mitigation, SNMP-related articles like Integration of Net-SNMP into an event loop, Linux-related articles like Tuning Linux IPv4 route cache, and an article about VXLAN long before it was cool.

The collapse When we needed new servers, the on-site team would take a set from the inventory, install our base Linux distribution on them, put them in the datacenter, and cable them to the top-of-the-rack switches. We opened a ticket describing the servers we needed, and one week later, our servers were available. Orange wanted to know if this team was performing well, so they asked for KPIs. They decided to use the number of tickets completed in a year. They asked to double this number. So instead of one ticket for a new service, we would open six tickets one per server. By the end of the year, the KPIs had more than doubled. Everybody saw it as a success for performance management. So, they asked to do the same for the next year. Now, we needed to open a ticket per server and per step. Again, the KPIs doubled. Behind the scenes, the tickets went to different people and were no longer handled in order. So, for the next year, it was decided to have meta-tickets and meetings to follow the progress of these tickets. Of course, all these extra steps pushed the KPI even higher. This performance management method spread to the other teams.5 Everything became slower. Instead of a couple of weeks, a new service now took six months. We built a Soviet nail factory. But the KPIs were good, and we stopped caring. Let me give you another example. We had to estimate the impact of each night operation. We weren t half bad: we declared most operations without any expected impact. Most of the time, there was no impact. One time out of five, there was a 5-second impact. We were told to try harder to meet our expected impact. What did we do? We started declaring a 5-second expected impact. One day, we got a 30-second impact and were told we failed to match the expected impact. In the end, we declared most operations with a 10-minute expected impact, and we stopped caring: instead of carefully shifting traffic around, we allowed ourselves a 5-minute impact. And our KPIs were never better.
Graph showing the impact of night operations. Year after year, the impact tolerance has been increased. In the final year, the expected impact is 10 minutes, and all operations remain under this threshold. However, the impacts are much more significant than they were in the first year.
An artist's rendering of the evolution of impacts over the years.

KPIs are not bad, but they are easy to break. Use them carefully: let the people doing the work help choose the metrics, and tie those metrics to the quality of the service for example, with service level objectives. Otherwise, even dedicated people stop caring, game the system, and eventually quit.6

  1. Internally, this division was named Hebex. It was located in Bagnolet (next to Paris) and Sophia-Antipolis (near Nice).
  2. Goodhart s law often gets the credit, but Campbell s law describes my experience even better: the more you lean on a number to make decisions, the faster people corrupt it.
  3. At the time, SaaS was not really a thing. I remember we considered, with a couple of colleagues, selling Wiremaps as a SaaS, with homomorphic encryption for the database. But who would outsource their observability stack?
  4. Snalert was a metacircular alerting tool in Perl. It was able to poll a very large number of SNMP targets in a short timespan. All our monitoring was SNMP-based, including system monitoring.
  5. My team also managed the rules of many Linux-based firewalls. To increase our KPIs, we used the same method: rather than accepting one ticket with a flow matrix, we requested one ticket per flow.
  6. Orange is not unique. Google s promotion process is another well-known example of a broken KPI. Michael Lynch explains it in Why I Quit Google to Work for Myself.

15 June 2026

Freexian Collaborators: Debian Contributions: Go default compatibility, Trimming build-essential, Python upstream engagement and more! (by Anupa Ann Joseph)

Debian Contributions: 2026-05 Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

Go default compatibility, by Helmut Grohne At the MiniDebConf Hamburg, Andrew Lee had prepared a talk on how Debian accidentally chooses Go compatibility. Helmut joined Tobias Quathammer and Andrew Lee in looking into the problem. Go has a compatibility system where modules declare a desired Go version to be compatible with. This influences various features such as whether RSA keys smaller than 1024 bits are accepted. Unfortunately, Debian s way of building Go packages is unique in setting GO111MODULE=off, which practically implies a very old compatibility version that enables a number of insecure settings. Most Linux distributions use the default GO111MODULE=on and therefore consult a go.mod file that often declares a sensible version. While doing so is the way for Debian longer term, getting there involves major changes so we also sought a more short term workaround. We developed a patch to the Go compiler that would enable it to pick up a compatibility version from the environment. Tobias uploaded it to unstable. The next step is communicating the declared compatibility version from go.mod to the compiler via the new variable. Then, rebuilding the archive resolves the immediate symptoms. This does not save us from having to perform the larger transition to GO111MODULE=on, but this shortcut can be backported to trixie.

Trimming build-essential, by Helmut Grohne One of the harder problems of the architecture cross bootstrap is correctly expressing the Build-Depends of glib during the toolchain bootstrap. It implicitly depends on build-essential, which happens to depend on libc6-dev. This poses a cycle. It applies even for cross building, because it is interpreted for the host architecture and that there is no way of satisfying this dependency during the toolchain bootstrap. Given discussions at MiniDebConf Hamburg with Jochen Sprickerhof and others, a seemingly stupid idea evolved: Let s delete build-essential. What looks insane on the surface might deserve a second look. Given how we moved away from C, C++ and autotools, what is in build-essential no longer is required by much of the archive. With the rise of debputy, debian/rules no longer has to be a makefile. While the task would be huge, those packages relevant to architecture bootstrap could explicitly support building without the implied dependency making their dependencies explicit. In a number of cases, this amounts to issuing a dependency on g++-for-host. This dependency requires the use of architecture-prefixed tools. Therefore, Helmut wrote a debhelper change that makes it always pass build tools to various build systems. This also enables more packages to honour environment variables such as CC and CXX.

Python upstream engagement, by Stefano Rivera Stefano attended PyCon US (at personal expense) to improve upstream relations and ensure Debian s voice is heard where it needs to be. On Friday there was a packaging summit (notes) with good discussion on the future of the wheel format, and some discussion of the new abi3t shared library format for free-threaded python. In preparation for the event, Stefano did a complete review of the current patch stack. Stefano s primary goal was to get some of Debian s patches merged during the sprints, and results were mixed. Some trivial patches (e.g. GH-150098, made progress and merged, but the most consequential patch Debian is carrying is still blocked. Stefano will continue to try to drive progress on this.

Miscellaneous contributions
  • Carles worked on po-debconf-manager: Reviewed Catalan translations for 6 packages, submitted 10 packages to maintainers, and removed 3 packages from po-debconf-manager.
  • Carles worked on check-relations: Continued improving the backend, including importing source package build dependencies to better support analysis of Debian blends. Added support for ignoring packages using regular expressions and source package names in response to user feedback. Used the tool to report 5 new bugs and followed up on previously reported issues.
  • Helmut sent a cross build patch on behalf of a customer.
  • Helmut uploaded debvm and guess_concurrency both featuring improved reproducibility and documentation.
  • Helmut continued maintaining rebootstrap and made it correctly handle binNMUs of gcc-defaults. Additionally, he poked at existing gcc patches giving answers, rebasing or closing them.
  • Helmut supported the video team in Hamburg mixing audio.
  • Helmut continued to report undeclared file conflicts of various kinds and corresponded with maintainers about them.
  • Antonio attended a debate during the Brazil Internet Forum about the impacts of the child protection regulation (ECA Digital) on free software operating systems.
  • Antonio worked on Debian CI to improve the system transparency for users. This included listing any pending jobs explicitly in the job lists for each package/architecture/suite page, as well as adding a queue status page that users can check for an estimate of test latency.
  • Antonio worked on several Debian CI maintenance tasks, including but not limited to some monitoring improvements, replacing usage of fonts-font-awesome with fonts-fork-awesome, and adding the ability in debci to configure a global notice (which is being used in Debian CI to point to the system status pages).
  • Antonio started doing some tests related to the change of default Debian CI backend from lxc to incus-lxc. This helped identify an omission in the creation of incus-lxc images. It was missing dpkg-dev, which caused a few packages that assumed its presence to fail. In the end, the incus-lxc backend will be fixed to include dpkg-dev by default in the image, but that uncovered an undeclared dependency in gem2deb (Ruby packaging helper) and in ruby-byebug, both already fixed in unstable.
  • Stefano did some minimal work on debian-reimbursements to get it working with current versions of django-allauth.
  • May included the discovery of several high-severity Linux kernel root exploits. Stefano updated kernels and rebooted debian.social infrastructure several times.
  • Stefano supported the Hamburg miniDebConf s wafer website during the event, and set up an instance for the 2027 edition too.
  • Stefano supported the bursary team issuing bursaries for DebConf 26.
  • Stefano uploaded routine updates of python-pip, pystemmer, snowball-data, snowball (making up a mini, uncoordinated snowball transition), python-authlib, python-discovery, python-installer, python-mitogen, python-pipx, python-cachecontrol, platformdirs, and python-virtualenv.
  • Stefano fixed a small number of bugs in dh-python, culminating in the 7.20260524 upload.
  • Thorsten finally managed to upload a new upstream version of hplip. He also uploaded a new upstream version of epson-inkjet-printer-escpr. Last but not least with the help of other contributors he could fix bugs in lprng.
  • Lucas and Santiago contributed significantly to the DebConf 26 Content team; helping to organize the team, review and rate talk proposals.
  • Lucas also supported a packaging sprint held in India by rebuilding and publishing the latest results of the Ruby 3.4 transition effort.
  • Santiago continued contributing to the efforts to organize DebConf 26, especially supporting the local team with different tasks.
  • In collaboration with Emmanuel Arias, Santiago is mentoring Aryan Karamtoth, a GSoC participant that is working to introduce linux live-patching support in Debian. The GSoC project started in May, with community bonding and coding. Santiago reviewed a merge request to prepare the clang-extract package for debian. clang-extract is one of the building blocks that will help to extract specific functions from large C code, so only relevant code can be patched, without recompiling the whole original basecode.
  • Anupa assisted Jean-Pierre Giraud with the point release announcements for Debian 13.5 and Debian 12.14.
  • Colin backported various security fixes from OpenSSH 10.3 to all supported releases (including LTS and ELTS).
  • Colin backported IP quality-of-service fixes to OpenSSH in trixie. The situation there had been unsatisfactory for some time, and upstream reworked their QoS support in OpenSSH 10.1 in a way that typically produces much better results.
  • Colin imported new upstream versions of 26 Python packages, and fixed around 25 RC bugs for the Python team.

29 May 2026

Ravi Dwivedi: Budapest Travel

In September 2025, I attended the annual LibreOffice conference in Budapest, Hungary. This gave me an opportunity to explore the city, which I will cover in this post. Let s start with the currency. Although Hungary is a part of the European Union (EU), it doesn t use the euro as its currency. Instead, it uses Hungarian forints (denoted by Ft ). During my time in Hungary, 1 Indian rupee was equal to 4 Hungarian forints. After reaching the Budapest airport, I bought a 15-day public transport pass. The public transport counter is after you pass customs and immigration. The pass allows unlimited use of public transport in the city. I had to show my passport and pay 5950 Ft to get the pass. The pass had my passport number mentioned on it. The public transport passes can also be bought at any of the tram stations as well.
This is the counter from where I bought my public transport pass. This is the counter from where I bought my public transport pass.
Budapest pass. My unlimited public transport pass for Budapest. I have redacted my passport number from it.
An automatic ticket machine An automatic ticket machine at a tram station in Budapest.
Budapest is a union of two cities Buda and Pest lying on opposite sides of the Danube River. My hotel Corvin Hotel was on the Pest side. Budapest had good public transport. The buses, metros, and trams complemented each other. For example, the airport didn t have metro or tram connectivity, but it was served by the bus. Most of the metro was on the Pest side, with only a couple of stations falling in Buda. However, both sides had an extensive network of trams. Furthermore, the information about the public transport was easily accessible. For instance, the map of tram stops inside the trams also included the bus routes one could get after alighting at those stops. From the airport, I took a bus followed by taking a metro on the M3 line to reach within walking distance of my hotel.
An M3 line metro in Budapest. An M3 line metro in Budapest.
During the conference I would take the tram to the conference venue. The trams were modern and fast. They also had a smiley face at the front, which gave them a friendly look. It seemed like the trams were happily doing their job. The city also had a good pedestrian infrastructure along with separate cycling tracks.
A tram in Budapest. A tram in Budapest having a smiley face at the front.
Budapest s tap water is officially safe to drink, which was mentioned on a sticker posted on the wall of the bathroom of my hotel room. So, I did not need to buy any water bottles while I was there. On the 6th of September, I went on a sightseeing tour of Budapest with my Dione. Our friend Attila, who was a local (from Hungary), joined us. We went to the central market from our hotel by metro. If you read my post on Vienna, I mentioned that the metro stations don t have AFC gates but ticket validators instead. Budapest s metro also has the same system. If you buy individual tickets, you need to validate them using the validators on the station before boarding the metro. If you are using a public transport pass like I was, then you do not need to validate, and you can board the metro directly.
A ticket validator at a metro station in Budapest. A ticket validator at a metro station in Budapest.
In 10-15 minutes, we reached the central market. Attila showed us around. I bought a fridge magnet and paprika powder as souvenirs. Paprika powder is a signature spice of Hungary. It is mainly available in two forms one is sweet and the other being spicy. I wanted the spicy one, but I didn t get that in that market. Therefore, I had to contend with buying the sweet version. The sweet version isn t sweet though, it is just not spicy. After bringing that paprika powder home, it is mainly used for food coloring. I like it though and use it frequently in my omelets and other dishes.
Central Market. Central market.
A building with a tram in front of it. The building right behind the tram is the central market building.
At some point, Atilla had to join the The Document Foundation (TDF) sightseeing group, so we parted ways at the central market. Dione and I continued our sightseeing and decided to start with visiting the Hungarian parliament, which is a tourist attraction. It was because we were on the Pest side and the parliament was also on the same side, while other tourist attractions were on the Buda side. So, Dione and I hopped on a tram and went to the parliament. We got off at a tram station just outside the parliament. The parliament is the icon of Budapest. The building has a gothic architecture and colored brown and white. One can buy tickets and take an inside tour. However, we didn t have a lot of time, so we stayed outside the building.
Hungarian Parliament building. Hungarian Parliament building.
After spending some time outside the parliament building, we took a tram to the Chain Bridge. As I mentioned earlier, Budapest has two parts Buda and Pest separated by the Danube River. To go from one of the sides to the other requires crossing a bridge. Although Budapest has many bridges linking the two sides, the main one is the Chain Bridge. We walked on the chain bridge to get to the other side. The bridge gave a good view of the Danube River. It also had a statue of a lion. The Buda Castle (another major landmark of Budapest) was visible from the bridge.
Chain Bridge. A shot of Chain Bridge.
A lion statue The lion statue on the Chain Bridge.
After reaching the other side of the bridge (the Buda side), we sat on a bench for some time and then planned on where to go next. We decided to go to Fisherman s Bastion, which is another tourist attraction. We used the OSMAnd~ app to figure out which bus to take and hopped on one. Soon we reached Fisherman s Bastion, where we found a flight of stairs that led upwards. Upon climbing the stairs, we got a panoramic view of the city. It also gave us a good view of the Hungarian parliament across the river. Going further upstairs, we found a statue of Stephen I of Hungary. He was the first king of Hungary, getting the crown in the year 1900.
A view of Hungarian parliament from Fisherman's bastion A view of Hungarian parliament from Fisherman s bastion.
I found Fisherman s Bastion to be the best tourist attraction in the city. As mentioned earlier, it offers a panoramic view of the city, which I liked. I liked the arhitecture and open space there. If you find yourself in Budapest, I would highly recommend that you visit Fisherman s Bastion.
Fisherman's Bastion. Fisherman s Bastion.
A green colored statue of King Stephen Statue of Stephen I of Hungary at Fisherman s Bastion.
Next, we went downstairs and returned to where the bus dropped us. From here on, we walked in random streets to see the residential and non-touristy side of Budapest. It was not so random as we walked towards Batthy ny t r metro station. Upon reaching the metro station, we found a caf where we stopped for a while for some coffee. After injecting some caffeine into our blood, we proceeded to find a place to have lunch.
A metro station Batthy ny t r metro station.
For lunch, we decided to go to R k czi t r metro station after reading on the internet about the food options there. Upon exiting the metro station, we found a market inside a building that had a lot of shops, but most of them were closed. After roaming around inside a bit, we found an Italian place open and decided to eat there. The name of this place was Matteos. We ordered an eggplant parmigiana, a lasagna artichoke, and a classic tiramisu. It wasn t very tasty but filled us up for the day.
The Italian place we had our lunch at. A picture of Matteos, where we had our lunch.
Budapest has four metro lines, and we had been to three of them, so we decided to try the remaining line, which was the M1 line. It is the oldest line in the city and has a different vibe than the modern lines. This line was opened in 1896, one of the oldest subway systems in the world. The coaches were much smaller than the other metro lines, and the seating arrangement was something you would expect from a bus than a typical metro train. We rode all the way to the last stop, Mexik i t. Upon going outside, we found out there wasn t much to do here. At this point, I checked the map and realized that Heroes Square is just a couple of metro stations away. Heroes Square is a tourist attraction in Budapest. It is located in Zugl a and is a historically significant place in Budapest. It has a monument which features the Seven chieftains of the Magyars.
M1 line station and tracks. M1 line station and tracks. It is the oldest metro transit of Budapest and one of the oldest in the world. It started operations in 1896.
Here, our unlimited public transport pass was handy because if it was paid per trip, we would think of the stop as a wasted one because we would have to buy a ticket again, but in this case we could just hop on again without any regrets.
A metro train entering a station. An M1 line metro train entering the station.
So we took the M1 line again and deboarded at H s k tere station, followed by walking to the square. After roaming around for a while, we saw a trolleybus and decided to ride on that.
Heroes' Square Heroes Square.
A trolleybus This is the trolleybus we took in Budapest.
A trolleybus is an electric bus that is powered by overhead electric cables. It is like a tram but runs on roads instead of tracks. We got down at D zsa Gy rgy t metro station. Then we took a metro to our hotel. Before going to the hotel, we went to a place to eat something. We had coffee and l ngos. L ngos is a deep-fried Hungarian dish, which looks exactly like the Indian flatbread bhatura. I found it tasty, but since it was deep-fried, that was almost a given.
A deep friend dish called L ngos. L ngos a dish which looks like the Indian flatbread bhatura.
The next day we went to Vienna the capital of Austria which I have already posted about. Check it out here. I had a good time in Budapest, and it is a beautiful city with good public transport and some amazing sites to visit. That s it for now, and see you next time! Credits: Thanks Dione and Badri for proofreading.

24 May 2026

Vincent Bernat: Sharding a routing table for lock-free reads in Go

To associate routing information like AS paths or BGP communities to flows, Akvorado can import routes through the BGP Monitoring Protocol (BMP). As the Internet routing table contains more than 1 million routes, Akvorado needs to scale to tens of millions of routes.1 This has been a long-standing challenge,2 but I expect this issue is now fixed by using RIB sharding, a method that splits the routing database into several parts to enable concurrent updates.

Previous implementation Akvorado connects 2 elements to build its RIB:
  1. a prefix tree, and
  2. a list of routes attached to each prefix.
Akvorado BMP RIB implementation before sharding with the memory layout of each structure and a single lock.
Akvorado BMP RIB implementation without sharding. One single read/write lock.
In the diagram above, the RIB stores five IPv4 prefixes and two IPv6 prefixes. One of them, 2001:db8:1::/48, contains three routes:
  • from peer 3, next hop 2001:db8::3:1, AS 65402, AS path 65402, community 65402:31,
  • from peer 4, next hop 2001:db8::4:1, same ASN, AS path, and community,
  • from peer 5, next hop 2001:db8::5:1, AS 65402, AS path 65401 65402, community 65402:31.
The rib structure is defined in Go as follows:
type rib struct  
    tree          *bart.Table[prefixIndex]
    routes        map[routeKey]route
    nlris         *intern.Pool[nlri]
    nextHops      *intern.Pool[nextHop]
    rtas          *intern.Pool[routeAttributes]
    nextPrefixID  prefixIndex
    freePrefixIDs []prefixIndex
 
The prefix tree uses the bart package, an adaptation of Donald Knuth s ART algorithm. The benchmarks demonstrate it outperforms other packages for lookups, insertions, and memory usage.3 Plus, the author is quite helpful.

Storing routes in a map The list of routes for each prefix is not stored directly in the prefix tree: it would put too much pressure on the garbage collector by allocating per-prefix arrays. Instead, the RIB assigns a unique 32-bit prefix identifier for each prefix, either by picking the last available prefix identifier from the freePrefixIDs array if any, or using the nextPrefixID value before incrementing it. Then, the routes are stored in the routes map, leveraging the optimized Swiss table in Go. To retrieve routes attached to a prefix, we look them up one by one in the routes map with a 64-bit key combining the 32-bit prefix index with a 32-bit route index matching the position of the route in the list. Akvorado scans routes from the first to the last to find the best one.4 It knows there is no more route if the route key returns no result.
type prefixIndex uint32
type routeIndex uint32
type routeKey uint64

Interning routes A route contains a BGP peer identifier, a partial NLRI5, the next hop, and the attributes.
type route struct  
    peer       uint32
    nlri       intern.Reference[nlri]
    nextHop    intern.Reference[nextHop]
    attributes intern.Reference[routeAttributes]
    prefixLen  uint8
 
type nlri struct  
    family bgp.Family
    path   uint32
    rd     RD
 
type nextHop netip.Addr
type routeAttributes struct  
    asn              uint32
    asPath           []uint32
    communities      []uint32
    largeCommunities []bgp.LargeCommunity
 
To save memory and allocations, NLRI, next hops, and route attributes are interned : a 32-bit integer replaces the real value. The mechanism predates the unique package introduced in Go 1.23. We keep it because it has different trade-offs:
  • It uses explicit reference counting instead of relying on weak pointers.
  • It works with non-comparable values implementing Hash() and Equal() methods.6
  • It uses explicit pool instances. This will be useful for sharding.
  • It has better performance. See for example this benchmark.
  • It consumes half the memory thanks to unsigned 32-bit references instead of pointers.
  • But it is not safe for concurrent use.

Why does it not scale?

Note At AS 12322, we don t use BMP yet.7 But Gerhard Bogner had the patience, availability, and technical skills to help me debug this issue.

The global read/write lock is a bottleneck in this implementation. But how? There are several users of the RIB, each with its own set of constraints:
  • The Kafka workers look up the RIB to enrich flows with routing information. They are bound by the number of Kafka partitions.8 Akvorado also adjusts their number to ensure efficient batching to ClickHouse. On our setup, the number of workers oscillates between 8 and 16. As we want to observe the latest data, we cannot afford for the Kafka workers to lag too much.
  • The monitored routers send route updates through the BMP protocol. When connecting, they can send millions of routes.9 After the initial synchronization, updates are sent continuously and may spike from time to time. The router detects a stuck BMP station when its TCP window is full and resets the session in this case. While Akvorado implements a large incoming buffer, it still needs to update the received routes with the write lock held fast enough to avoid being detected as stuck.
  • When a remote BGP peer goes down, Akvorado flushes the associated routes by walking the RIB with the write lock held. When a monitored router goes down, Akvorado waits a bit but eventually flushes all the associated routes.
In short: on a busy setup, lock contention is high for both readers and writers, and neither can lag too much behind.

RIB sharding

First step: basic sharding To remove the global lock, the RIB is split into several shards, each one handling a subset of the prefixes:
Akvorado BMP RIB implementation after sharding with the memory layout of each structure and one lock per shard.
Akvorado BMP RIB implementation with sharding.
The prefix tree stays global and is protected by a single lock. Each shard gets its read/write lock, its route map, and its intern pools to store NLRIs, next hops, and route attributes, which would not have been possible with Go s unique package. The prefix indexes are also sharded: the 8 most significant bits are the shard index and the 24 remaining bits are the local prefix index. Gerhard confirmed that after this blind change, the BMP receiver chugged steadily. Later, I wrote a concurrent benchmark over half a million synthetic but plausible routes10 partitioned over 0 to 8 writers, churning routes as fast as possible, while 1 to 16 readers continuously look up a set of 10,000 routes. I don t know if this benchmark is realistic, but it confirms the improvements for both read and write latencies:
Two heatmaps. One for read latency ratio, the other for write latency ratio. Both of them comparing the speedup with colored tiles between the code before sharding and after sharding. Most tiles are green.
Read and write latency performance improvement after sharding.
It also shows that a high number of writers degrades read latency.

Second step: lock-free reads The single read/write lock protecting the prefix tree is the next target. The bart package provides alternative mutation methods returning an updated tree using copy-on-write. Readers don t need the global lock any more, leaving it only to synchronize writers. The prefix tree is boxed in an atomic pointer.
Akvorado BMP RIB implementation for sharding with lock-free reads. It shows the memory layout of each structure.
Akvorado BMP RIB implementation with sharding and lock-free reads.
Without a lock, readers can now fetch a stale prefix index when walking their copy of the tree if a concurrent writer removes the last route attached to this prefix index and recycles it for another prefix. To avoid this issue, we combine the prefix index with a generation number and store them in the tree:
type generation uint32
type prefixRef struct  
    idx prefixIndex
    gen generation
 
type rib struct  
    mu     sync.Mutex
    tree   atomic.Pointer[bart.Table[prefixRef]]
    shards []*ribShard
 
Each shard stores the generation number for each local prefix index. The generation number increases by one if the associated prefix index is freed. When looking up the routes attached to a prefix index, the reader checks if the generation number matches. Otherwise, it assumes the index was recycled and the list of routes is empty.11 You can see this case in the diagram above for prefix index 5, stored with a generation index of 3, while the current value in the []generations array is 4. The generation number could overflow, but it is not a problem as lookups are quick. Running the concurrent benchmark against this new implementation shows the improvements for the read latency as soon as the cost of the copy-on-write prefix tree is amortized.
Six heatmaps. Three for read latency ratio, three others for write latency ratio. They compare the numbers without sharding, with sharding, and with lock-free reads, pair by pair. For read latency, most tiles are green, showing an improvement of the second step. For write latency, the speedup is negative for a low number of readers.
Read and write latency performance improvement after lock-free reads. The middle column shows the cumulative improvements of both steps.

Among the multiple attempts to optimize the BMP component, RIB sharding is one of the more satisfying. Akvorado 2.2 implements the first step. PR #2433, drafted while writing this blog post, implements the second step and was released with Akvorado 2.4.

  1. Each router exporting flows doesn t need to send its routes. When Akvorado does not find a route from a specific device, it falls back to a route sent by another device. It is up to the operator to decide if this is a good enough approximation.
  2. I made many attempts to scale the BMP component. See for example PR #254, PR #255, PR #278, PR #2244, and PR #2245. Despite these efforts, this component remained problematic for some users. See discussion #2287 as the latest example.
  3. It keeps improving: bart 0.28.0 features a new implementation that trades a bit of memory for greater lookup performance. I did not test it yet, as I have been preparing this blog post for a couple of months already.
  4. Akvorado prefers the route matching the exact next hop. Otherwise, it falls back to any other route. This is an approximation. An alternative would be to have one prefix tree for each BGP peer but it would require configuring all routers to export their routes. pmacct s BMP daemon implements this approach.
  5. If we consider the BGP RIB as a database, the Network Layer Reachability Information (NLRI) is the primary key. Its content depends on the BGP family. With IPv4 or IPv6 unicast, this is the prefix. For VPNv4 and VPNv6 families, it includes the route distinguisher. If you enable the ADD-PATH extension, the NLRI also contains a path identifier. In our implementation, we don t store the prefix as we get it from the looked-up IP address using the prefix length stored separately.
  6. The Hash() methods rely on the hash/maphash package and on the unsafe package to avoid memory copies. See for example the Hash() function for the nlri structure.
  7. Despite being an author or co-author of the first BMP-related RFCs since 2016 (RFC 7854, RFC 8671, RFC 9069), Cisco did not implement it in a usable way in IOS XR until version 24.2.1. We still need to upgrade a few routers to enable this feature.
  8. KIP-932 introduces, in Kafka 4.2, the concept of share groups to enable cooperative consumption on the same partition. This is not supported in Akvorado yet.
  9. You can configure BMP to send routes for each BGP peer before or after applying the incoming policies. In this case, you can get more than one million routes for each transit peer. You can also tell BMP to send the local RIB, which only contains the best path for each prefix.
  10. The prefixes are random, but the prefix size distribution and the AS path length distribution follow the data provided by Geoff Huston.
  11. Alternatively, we could retry the lookup, but it would be pointless: the RIB is an eventually consistent database, and an empty list was a correct answer at some point in the recent past.

23 May 2026

Petter Reinholdtsen: Command line Norse God of Wind Hr svelg move the clouds

A while back, I came across the AI Fabric system created by Daniel Miessler. I liked its approach of providing command-line tools for filtering text using artificial idiocy services, allowing stepwise operations to be applied to a piece of text. The output of one operation can then serve as the input for another in other words, Unix pipeline processing powered by large language models. I do no longer remember exactly how I discovered it, but suspect it was via Matthew Berman's video "How To Install Fabric - Open-Source AI Framework That Can Automate Your Life". While the idea and concept behind AI Fabric appealed to me, its implementation has continued to rub me the wrong way. It started off as a Python project that I could only get running by downloading random programs from the internet using Poetry. I tried to assess how much work it would take to package all its missing dependencies for Debian. However, before I got very far, the project shifted away from Python and over to Go. This new implementation also relied on a build system that seemed to encourage users to run arbitrary code downloaded from the internet to get software working, and further moved to a language I do not master as well as Python. The change bothered me enough that I set my effort to set up a working command line LLM tool in Debian aside for several months. By chance, I came across a simple Python recipe in January demonstrating how to communicate with a llama.cpp API server. I had already been working on packaging llama.cpp for Debian together with the rest of Debian's AI team, and was fortunate enough to own a working instance with a 24 GiB VRAM GPU from AMD, allowing me to run useful models. Until that point, I had only used the basic web client provided by the Debian package, lacking the spare time to explore what else could be done. Then, I found this simple 50 line Python script demonstrating how to interact with llama.cpp's OpenAI-compatible API. I decided to revive the AI Fabric concept, and implement the Unix pipeline filter tool with as few dependencies as possible. It is now operational and working very well, relying solely on standard Python features. The tool include a copy of the LLM recipes from the AI Fabric project (called "patterns"), enabling easy access to request summaries, translations, code review and other useful tasks. Several hundred patterns are included, though I have only tested about ten so far. The LLM API server can be specified in ~/.config/hraesvelgr/config.ini like this:
[server]
url=https://some.llm.example.com:8080/v1/
model=Qwen/Qwen3.6-27B-FP8
With this configuration in place (you can also specify these values directly on the command line), you can specify a pattern and a file to process like this:
% bin/hraesvelgr --pattern explain_code bin/hraesvelgr
EXPLANATION:
This Python script is a client tool for interacting with an AI
service (likely a local LLM server) to process text using prompts
defined in the "AI Fabric" repository. It reads system and user
prompts from markdown files, sends them along with input text to a
chat completion API endpoint, and prints the generated response.
Key components:
1. It uses argparse for command-line argument parsing
2. The  send_chat_completion_request  function formats messages
   (system, user, query) into JSON and sends them via HTTP POST to
   an AI service endpoint
3.  read_file  function reads markdown files, replacing placeholders
   like  lang_code  with actual values from arguments
4. In main():
 - Parses command-line arguments for input file, API base URL,
   pattern type, language code, and debug flag
 - Ensures the base URL ends with a slash
 - Reads system prompt from data/patterns/ pattern /system.md
 - Optionally reads user prompt from data/patterns/ pattern /user.md
 - Reads input text either from stdin (when "-" is passed) or a file
 - Handles encoding fallback to ISO-8859-1 if UTF-8 fails
 - Sends the formatted request to the AI service and prints the response
The script assumes it's running in a directory containing a git
clone of https://github.com/danielmiessler/fabric/, which contains
the necessary prompt files.
This tool is designed to interface with local LLM servers that
support OpenAI-compatible chat completion APIs.
%
The list of available patterns can be viewed by running bin/hraesvelgr --list-patterns. I have found the summarize, translate, improve_writing, review_code, and explain_terms_and_conditions patterns particularly useful. For example using the latter combined with a text based web browser capable of dumping a page as plain text, can be done like this (originally formatted in markdown, I converted to HTML using pandoc for easier readability):
% w3m  -dump https://runbox.com/about/terms-service/   \
  hraesvelgr --pattern explain_terms_and_conditions
Executive Summary
This is a transparent, privacy-focused contract from
a Norwegian provider that generally respects user data rights and
operates under strict EU/EEA standards. However, it carries
strict liability limitations and an aggressive
data-deletion policy upon cancellation. The vibe is  Professional &
Privacy-First,  but you must manage your own backups and understand that
the company heavily shields itself from financial responsibility during
technical failures.
Key Takeaways
  • Your Data Stays Yours: Section 10.2 explicitly states Runbox will never use your transmitted or stored data for commercial purposes. This is a major privacy win.
[... trimmed output, as it is not the focus of this blog post ...] If you sign:
  1. Set up automated backups immediately. Use IMAP sync to a local drive or a secondary email provider before storing any critical documents or emails. Do not rely on Runbox as your only archive.
  2. Mark your calendar for the 30-day trial end date. Miss the payment window, and access closes instantly with no recovery period.
  3. Monitor price changes at renewal. Since they can adjust fees anytime, check their pricing page a few days before your subscription renews to avoid unexpected charges.
NO FORCED ARBITRATION CLAUSE FOUND.
REFUND POLICY IS STRICTLY CONDITIONAL (see Sections 4.2 4.5).
As you might have already noticed, I name my project after the Norse God of Wind. I found a nice description of the origin of the name on Wikipedia:
In Vaf r nism l (The Lay of Vaf r nir), Odin questions the wise j tunn Vaf r nir about the origin of the wind, and the j tunn answers:
He is called Hr svelg,
who sits at heaven s end,
a giant, in the shape of an eagle;
from his wings
they say the wind comes over all people.
(translated by John Lindow in Norse Mythology: A Guide to Gods, Heroes, Rituals, and Beliefs 2002)
The latest version of the code can be found at https://codeberg.org/pere/hraesvelgr/. Perhaps you will find it as useful as I did? As usual, if you use Bitcoin and wish to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

21 May 2026

Amin Bandali: ffs 0.2.2 released

ffs provides a minor mode for simple plain text presentations in Emacs, where the slides are separated using the page-delimiter, by default the form feed character (^L). I wrote ffs in early 2022 for my LibrePlanet 2022 presentation the Net beyond the Web, and earlier this year decided to polish it towards being a proper package and submit it to GNU ELPA. The manual still needs some more work, but the overall package is in pretty good shape so I submitted for inclusion in GNU ELPA. ffs and I owe a debt of gratitude to Protesilaos for rounds of code review and feedback for improving and polishing the package in preparation for submission to GNU ELPA. You can watch videos of these sessions posted earlier on my website: Further, inspiration for parts of ffs's implementation was gratefully drawn from Protesilaos's Logos package for Emacs. Dedicated to the loving memory of Farangis Yousefinia. Below are the release notes.

Version 0.2.2 on 2026-05-21
First release of ffs on GNU ELPA. The attempted build of ffs 0.2.1 within GNU ELPA build sandbox failed with an Error: void-function (org-texinfo-kbd-macro) due to use of #+macro: kbd (eval (org-texinfo-kbd-macro $1)) in ffs.org for better formatting of key sequences in the exported Texinfo copy. This seems to have happened for the specific case of generating a plain text README using ox-ascii where ELPA didn't load ox-texinfo. To try and mitigate this, a README.md has been added for use as the package README instead of ffs.org. If not sufficient, a Texinfo copy of the ffs manual will be shipped instead of the Org one in the next release. ffs 0.2.2 also includes small fixes and improvements throughout ffs.el from Stefan Monnier, and additional feedback to be addressed in future releases.

Version 0.2.1 on 2026-05-20
The attempted build of ffs 0.2.0 within GNU ELPA build sandbox failed with a "Cannot include file" error on the "#+include: fdl.org" in the manual. So, as a workaround, we switch to using the official Texinfo copy of the GNU FDL license rather than an Org copy.

Version 0.2.0 on 2026-05-19
First release of ffs intended for GNU ELPA. After a few years of inactivity, in early 2026 I decided to dust off ffs.el, polish and document it, and offer for inclusion in GNU ELPA as a proper package.

Default value of ffs-default-face-height changed to nil
To minimize unexpected and/or unnecessary changes out-of-the-box, the default value of ffs-default-face-height has been changed to nil.

ffs-edit-buffer-name demoted from user option to variable
This is not an important user-facing setting, so to help avoid overwhelming users with many options, this has been demoted from a user option to a variable.

Several new user options for customizing ffs's behaviour
As part of the effort to bring ffs more in line with the conventions of other existing Emacs packages, the mechanisms for toggling various parts of Emacs's interface to minimize visual clutter were changed from being minor modes to being customizable user options. These are the replacement new user options, with a default value of nil:
  • ffs-hide-cursor
  • ffs-hide-mode-line
  • ffs-hide-header-line
Their value is buffer-local, and may be set globally using setq-default. See the sample configuration in the manual for an example of how to customize them. The new ffs-page-delimiter user option defines the page delimiter inserted by ffs-edit-done when inserting a new slide. Emacs's page-delimiter regexp should be able to match ffs-page-delimiter's value, so if you use a custom page-delimiter be sure to customize ffs-page-delimiter accordingly. The new ffs-echo-progress user option controls whether to display in echo area the progress through the slides. When non-nil, changing slides will also display the progress through the slides in the echo area. The format of the displayed progress can be customized using the new ffs-echo-progress-format user option. The new ffs-edit-display-buffer-alist user option may be used to control the Window configuration for the ffs-edit buffer. By default, it will display the ffs-edit buffer in the same window. The new ffs-edit-done-hook user option may be used to define hooks to be run at the end of ffs-edit-done after returning to the main ffs presentation buffer. Lastly, a new ffs-find-speaker-notes-function variable was added to allow customizing the find function used for opening the speaker's notes file, defaulting to find-file-other-frame.

Version 0.1.0 on 2022-05-19
Initial publication of ffs.el as part of my personal configurations for GNU Emacs. My first attempt at this concept was a now-archived ffsanim.el, a major mode implementation that used Emacs's animate library to animate slide texts onto the screen. Shortly after realizing the shortcomings of that approach, I abandoned it in favour a minor mode implementation and published version 0.1.0 of what is now ffs in my personal configs repository. I used this implementation for presenting my LibrePlanet 2022 talk, The Net beyond the Web. I picked "ffs" as the package name, the acronym for form feed slides.

Dirk Eddelbuettel: nanotime 0.3.15 on CRAN: Coping

Another very minor update, now at 0.3.15, for our nanotime package is now on CRAN, and has been built for r2u and Debian. nanotime relies on the RcppCCTZ package (as well as the RcppDate package for additional C++ operations) and offers efficient high(er) resolution time parsing and formatting up to nanosecond resolution, using the bit64 package for the actual integer64 arithmetic. Initially implemented using the S3 system, it has benefitted greatly from a rigorous refactoring by Leonardo who not only rejigged nanotime internals in S4 but also added new S4 types for periods, intervals and durations. This release adjusts the package for the maybe overly hasty switch R 4.6.0 has undertaken with respect to using C++20 as a default C++ compilation standard. I am of course largely in favour of such a switch to more modern C++. But I am also cognizant of the fact that not all compilers and machines are ready. And just as I have already seen one other package fail to compile on a particular CRAN system (!!) under C++20, this package all of a sudden, and only on that same system, started to throw two (harmless) compiler warnings. We could call these erroneous as newer versions of the same compiler do not throw them but it does not matter. The decision to default to C++20 has been made, and now we live with it. But maybe some hardware platforms should be moved behind the barn. Either way, this release both adds an explicit cast to two lines that may not really need it (but this will not hurt) and also dials the compilation standard down to C++17 on one particular platform. So once again there are no user-facing changes, or behavioural changes or enhancements, in this release. The NEWS snippet below has the fuller details.

Changes in version 0.3.15 (2026-05-21)
  • Add extra const_cast as one CRAN machine with more ancient setup whines otherwise and is obviously less C++20 ready than it thinks
  • tools/configure also checks where this is being built and as needed' downgrades the compilation to C++17

Thanks to my CRANberries, there is a diffstat report for this release. More details and examples are at the nanotime page; code, issue tickets etc at the GitHub repository and all documentation is provided at the nanotime documentation site.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub. You can also sponsor my Tour de Shore 2026 ride in support of the Maywood Fine Arts Center.

20 May 2026

Michael Prokop: The mysterious XF86AudioPlay issue

I was getting <XF86AudioPlay> is undefined in the status bar of Emacs displayed every 2-3 seconds. Nowhere else I noticed any misbehavior or problems, and also couldn t find any related log entries. It didn t stop, though didn t want to reboot my system to see whether that would fix the problem, but it was driving me nuts. Now, as a starting point I adjusted my sway configuration, to react to the XF86AudioPlay key press event:
bindsym XF86AudioPlay exec playerctl play-pause
After reloading sway, my music player started to play for 2-3 seconds, stopped playing, started again, etc. It wasn t a Emacs bug, but something indeed seemed to send the XF86AudioPlay key event every 2-3 seconds. It wasn t my USB keyboard or any stuck key on it, as verified also by unplugging it. So which device was causing this? libinput from libinput-tools to the rescue:
% sudo libinput debug-events
[...]
-event12  KEYBOARD_KEY                 +0.000s  KEY_PLAYPAUSE (164) pressed
 event12  KEYBOARD_KEY                 +0.000s  KEY_PLAYPAUSE (164) released
 event12  KEYBOARD_KEY                 +2.887s  KEY_PLAYPAUSE (164) pressed
 event12  KEYBOARD_KEY                 +2.887s  KEY_PLAYPAUSE (164) released
 event12  KEYBOARD_KEY                 +5.773s  KEY_PLAYPAUSE (164) pressed
 event12  KEYBOARD_KEY                 +5.774s  KEY_PLAYPAUSE (164) released
[...]
The event12 device was sending this event, what s behind this?
% sudo udevadm info /dev/input/event12
P: /devices/pci0000:00/0000:00:1f.3/skl_hda_dsp_generic/sound/card0/input17/event12
M: event12
R: 12
J: c13:76
U: input
D: c 13:76
N: input/event12
L: 0
S: input/by-path/pci-0000:00:1f.3-platform-skl_hda_dsp_generic-event
E: DEVPATH=/devices/pci0000:00/0000:00:1f.3/skl_hda_dsp_generic/sound/card0/input17/event12
E: DEVNAME=/dev/input/event12
E: MAJOR=13
E: MINOR=76
E: SUBSYSTEM=input
E: USEC_INITIALIZED=12468722
E: ID_INPUT=1
E: ID_INPUT_KEY=1
E: ID_INPUT_SWITCH=1
E: ID_PATH=pci-0000:00:1f.3-platform-skl_hda_dsp_generic
E: ID_PATH_TAG=pci-0000_00_1f_3-platform-skl_hda_dsp_generic
E: XKBMODEL=pc105
E: XKBLAYOUT=us
E: XKBOPTIONS=lv3:ralt_switch,compose:rctrl
E: BACKSPACE=guess
E: LIBINPUT_DEVICE_GROUP=0/0/0:ALSA
E: DEVLINKS=/dev/input/by-path/pci-0000:00:1f.3-platform-skl_hda_dsp_generic-event
E: TAGS=:power-switch:
E: CURRENT_TAGS=:power-switch:
% sudo udevadm info -a /dev/input/event12   grep -iE 'kernels drivers name'
    KERNELS=="input17"
    DRIVERS==""
    ATTRS name =="sof-hda-dsp Headphone"
    KERNELS=="card0"
    DRIVERS==""
    KERNELS=="skl_hda_dsp_generic"
    DRIVERS=="skl_hda_dsp_generic"
    KERNELS=="0000:00:1f.3"
    DRIVERS=="sof-audio-pci-intel-tgl"
    KERNELS=="pci0000:00"
    DRIVERS==""
Behind this event12 is sof-hda-dsp Headphone, and evtest confirms that:
% sudo evtest
No device specified, trying to scan all of /dev/input/event*
Available devices:
/dev/input/event0:      AT Translated Set 2 keyboard
/dev/input/event1:      Sleep Button
/dev/input/event10:     ThinkPad Extra Buttons
/dev/input/event11:     sof-hda-dsp Mic
/dev/input/event12:     sof-hda-dsp Headphone
/dev/input/event13:     sof-hda-dsp HDMI/DP,pcm=3
/dev/input/event14:     sof-hda-dsp HDMI/DP,pcm=4
/dev/input/event15:     sof-hda-dsp HDMI/DP,pcm=5
/dev/input/event16:     Yubico YubiKey OTP+FIDO+CCID
/dev/input/event17:     Apple Inc. Magic Keyboard with Numeric Keypad
/dev/input/event18:     Apple Inc. Magic Keyboard with Numeric Keypad
[...]
Select the device event number [0-24]: ^C
We can even get further information:
% sudo evtest /dev/input/event12
Input driver version is 1.0.1
Input device ID: bus 0x0 vendor 0x0 product 0x0 version 0x0
Input device name: "sof-hda-dsp Headphone"
Supported events:
  Event type 0 (EV_SYN)
  Event type 1 (EV_KEY)
    Event code 114 (KEY_VOLUMEDOWN)
    Event code 115 (KEY_VOLUMEUP)
    Event code 164 (KEY_PLAYPAUSE)
    Event code 582 (KEY_VOICECOMMAND)
  Event type 5 (EV_SW)
    Event code 2 (SW_HEADPHONE_INSERT) state 0
Properties:
Testing ... (interrupt to exit)
Event: time 1779295060.175766, type 5 (EV_SW), code 2 (SW_HEADPHONE_INSERT), value 1
Event: time 1779295060.175766, -------------- SYN_REPORT ------------
Event: time 1779295061.951168, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 1
Event: time 1779295061.951168, -------------- SYN_REPORT ------------
Event: time 1779295061.951194, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 0
Event: time 1779295061.951194, -------------- SYN_REPORT ------------
Event: time 1779295064.548671, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 1
Event: time 1779295064.548671, -------------- SYN_REPORT ------------
Event: time 1779295064.548689, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 0
Event: time 1779295064.548689, -------------- SYN_REPORT ------------
Event: time 1779295067.437172, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 1
Event: time 1779295067.437172, -------------- SYN_REPORT ------------
Event: time 1779295067.437187, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 0
Event: time 1779295067.437187, -------------- SYN_REPORT ------------
Event: time 1779295070.323775, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 1
Event: time 1779295070.323775, -------------- SYN_REPORT ------------
Event: time 1779295070.323790, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 0
Event: time 1779295070.323790, -------------- SYN_REPORT ------------
Event: time 1779295073.200350, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 1
Event: time 1779295073.200350, -------------- SYN_REPORT ------------
Event: time 1779295073.200373, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 0
Event: time 1779295073.200373, -------------- SYN_REPORT ------------
Event: time 1779295076.076228, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 1
Event: time 1779295076.076228, -------------- SYN_REPORT ------------
Event: time 1779295076.076250, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 0
Event: time 1779295076.076250, -------------- SYN_REPORT ------------
Event: time 1779295078.961740, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 1
Event: time 1779295078.961740, -------------- SYN_REPORT ------------
Event: time 1779295078.961754, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 0
Event: time 1779295078.961754, -------------- SYN_REPORT ------------
Event: time 1779295081.850156, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 1
Event: time 1779295081.850156, -------------- SYN_REPORT ------------
Event: time 1779295081.850175, type 1 (EV_KEY), code 164 (KEY_PLAYPAUSE), value 0
Event: time 1779295081.850175, -------------- SYN_REPORT ------------
Event: time 1779295083.306612, type 5 (EV_SW), code 2 (SW_HEADPHONE_INSERT), value 0
Event: time 1779295083.306612, -------------- SYN_REPORT ------------
So when I plug in my headphone (see the SW_HEADPHONE_INSERT event), the unexpected behavior starts, unplugging stops the problem.
Good! But what was totally unexpected for me: my headphone, being a Beyerdynamic DT-990 Pro, does not have any keys. 8-) As it turned out, the headphone jack seemed to have been not entirely clean. The analog side of the jack triggers a behavior within the audio codec, where it seems to interpret the fluctuating impedance as a play button of the headset, being pressed, again and again. I cleaned the jack of my headphone and my XF86AudioPlay problem is gone, case closed.

18 May 2026

Sergio Durigan Junior: Fixing a 20+ year old bug in Debian curl

I have been helping co-maintain the Debian curl package for a few years now, and even though Samuel and Charles do most of the work, I'm happy to jump in and help when needed. This is one of those cases. Nowadays the package is maintained by 3 people (with help from others occasionally), but it hasn't always been like this. Samuel adopted the package back in 2021, and since then it has received a lot of love and care to make sure it lives up to Debian's standards. Again, kudos to both him and Charles who have been doing great work on this front. But a little more than 20 years ago, the situation in Debian (and curl!) was "a bit" different.

Once upon a time... According to d/changelog, the Debian curl maintainer in 2005 introduced changes to the packaging that allowed it to generate a version of libcurl for each TLS backend available: OpenSSL and GnuTLS. This meant that curl would have two binary library packages:
  • libcurl3-openssl and its respective -dev variant, for libcurl linked against OpenSSL; and
  • libcurl3-gnutls and its respective -dev variant, for libcurl linked against GnuTLS.
But then, around 2006/2007 or so, upstream curl decided to bump the SONAME version of libcurl from 3 to 4. At the time, they apparently did not version their library symbols like they do now, which was... less than ideal. I don't judge them: curl and a lot of other important projects have come a long way when we consider best practices to write shared libraries. Meanwhile, on Debian land, the release team was having trouble with other transitions going on at the time. For those who are not versed in Debian's vocabulary, a transition happens when a shared library gets its SONAME version bumped: when this happens, we have to make sure that all reverse dependencies of that library still build with the new version, and fix things that fail. The more reverse dependencies the library has, the harder this work gets. When upstream curl bumping the SONAME version of libcurl, the Debian curl maintainer at the time correctly renamed the binary packages from libcurl3- openssl,gnutls (and their -dev variants) to libcurl4- openssl,gnutls (and their -dev variants), which obviously triggered a transition. And a big one, because libcurl is used by several projects. Long story short, the Debian release team found themselves between a rock and a hard place. According to the late Steve Langasek at the time:
We talked a while back about the curl transition, and about how upstream's change from libcurl.so.3 to libcurl.so.4 is gratuitously painful for us in light of the large number of reverse dependencies. The libcurl transition has at this point gotten tangled with soname transitions in jasper, exiv2, kexiv2, and God only knows what else. So I'd like to revisit this question, because tracking this transition is costing the release team a lot of time that would be better spent elsewhere, and removing the need for a libcurl transition promises to reduce the complexity of the other components by an order of magnitude. On looking at the curl package, I've come to understand that the symbol versioning in place in this library is the result of a Debian-local patch. That's great news, because it suggests a solution to this quandary that doesn't require an unreasonable amount of developer time.
Yeah, it wasn't pretty. Here's what was proposed:
I am proposing the following:
  • Keep the library soname the same as it currently is upstream. Because upstream uses unversioned symbols, our package will be binary-compatible with applications built against the upstream libcurl regardless of what we do with symbol versioning, so leaving the soname alone minimizes the amount of patching to be done against upstream code here.
  • Revert the Debian symbol versioning to the libcurl3 version, and make libcurl.so.3 a symlink to libcurl.so.4. We have already established that libcurl.so.4 is still API-compatible with libcurl.so.3, in spite of the soname change upstream; reverting the symbol versioning will make it fully ABI-compatible with libcurl.so.3, and adding the symlink lets previously-built binaries find it.
  • Revert the Debian package names to the curl 7.15.5 versions. Because compatibility has been restored with libcurl3 and libcurl3-gnutls, restoring the package names provides the best upgrade path from etch to lenny; and because the symbol versions have been reverted, the libraries are not binary-compatible with the Debian packages currently named libcurl4/libcurl4-gnutls/libcurl4-openssl (in spite of being binary-compatible with upstream), so it would be wrong to keep the current names regardless.
  • Drop the SSL-less variant of the library, which was not present in curl 7.15.5; AFAICS, there is no use case where a user of curl needs to not have SSL support, so this split seems to be unnecessary overhead. Please correct me if I'm mistaken.
  • Leave the -dev package names alone otherwise, to simplify binNMUing of the reverse-dependencies (some packages have already added versioned build-deps on libcurl4.*-dev -- I have no idea why -- so reverting the names would mean more work to chase down those packages). Drop libcurl4-dev as a binary package, though, in favor of being Provided by libcurl4-gnutls-dev. Many of the packages currently build-depending on libcurl4-dev -- including some that wrongly used libcurl3-dev before -- are GPL, and these are apparently all packages where having SSL support missing in libcurl4 wasn't hurting them, so libcurl4-gnutls-dev seems to be the reasonable "default" here.
  • Schedule binNMUs for all reverse-dependencies.
Again, no judgement here: this was what needed to be done at the time, and I believe it was a good solution given the circumstances. In the end, the binary library packages got renamed again: from libcurl4- openssl,gnutls back to libcurl3- openssl,gnutls (but not their -dev variants!), but they continued shipping libcurl libraries whose SONAME version was 4. This solved the immediate problem of untangling the transitions mentioned by Steve, but introduced a technical debt that would stick with the package literally for decades. The situation at the end of 2007 was:
  • libcurl3-openssl with libcurl4-openssl-dev; and
  • libcurl3-gnutls with libcurl4-gnutls-dev.

More discrepancy is added Eventually the libcurl3-openssl package got renamed to libcurl3, but aside from that the situation with mismatched library names vs. SONAME versions stayed relatively unchanged until around 2018, when the Debian curl maintainer at the time (a different person) renamed libcurl3 to libcurl4 to fix a bug. This was the right thing to do for libcurl3, and at the time upstream curl was already properly versioning their symbols, but for some reason libcurl3-gnutls got left behind. So now we had:
  • libcurl4 with libcurl4-dev; and
  • libcurl3-gnutls with libcurl4-gnutls-dev.
In other words, we now have a discrepancy between the OpenSSL and GnuTLS variants' names. Yeah, confusing. And this is the situation right now, on May 2026, while I write this post. To make matters worse, the Debian curl package has been carrying a patch to facilitate the split of OpenSSL and GnuTLS flavours for decades now, and, for some reason I didn't bother to investigate, the patch pins the SONAME version of libcurl3-gnutls to CURL_GNUTLS_3, effectively overriding upstream's decision to version the symbols as CURL_GNUTLS_4.

A call to make things right Back in 2022, Simon McVittie filed a Debian bug to try and call our attention to the fact that we were shipping this messy set of curl packages. I had just started to get involved in the package maintenance and Samuel asked me to take a look at the bug. I noticed it was going to take more time than I had available, so I decided to put it in my TODO list (TM). Simon was generous enough to lay out a possible plan to tackle the problem, but I had a feeling that this was going to be harder than it looked. I kept postponing working on the bug, but also kept thinking about it now and then because it's an interesting thing to solve. Then, a month or so ago the Debian Brasil community got together for MiniDebConf Campinas 2026 and we decided to do a bug squashing party there. I started working on a few FTBFS bugs with GCC 16, but then got remembered about the curl bug and thought that that was the perfect time and place to start working on it, for a few reasons:
  • Samuel and Charles were also attending the conference, so I could talk to them about my plans and show them a PoC.
  • I was going to give a presentation about symbols (in pt_BR), so I could use this bug as an example of symbol versioning.
  • I wanted to have fun.

The initial plan The plan I had in mind was a variant of Simon's proposed plan:
  • I would have to adjust our GnuTLS-specific patch so that it did not override the SONAME version for libcurl-gnutls. Then,
  • For each symbol from libcurl3-gnutls I would have to:
    • Explicitly version it as curl_symbol_name@@CURL_GNUTLS_4.
    • Create an alias for the symbol (let's call it __curl_compat_symbol_name).
    • Explicitly version this alias as __curl_compat_symbol_name@CURL_GNUTLS_3.
  • Have a separate version of curl's linker script to make it possible to create a hierarchy between CURL_GNUTLS_3 and CURL_GNUTLS_4 symbols.
Note that this whole dance is needed because it is a hard requirement that programs linked against libcurl3-gnutls keep working when we ship libcurl4-gnutls, without needing to recompile them. Due to the fact that we will not really bump the SONAME of libcurl-gnutls (but instead fix the symbol versions shipped by it), we cannot expect programs to break given that they are actually using the exact same ABI as before. Unfortunately (as it is common with low level tools) the documentation for ld's versioning syntax is quite incomplete and hard to find. One of the best sources I found was this blog post. For this reason, let me quickly explain the different notations for symbol versioning used above.

curl_symbol_name@@CURL_GNUTLS_4 When we use curl_symbol_name@@CURL_GNUTLS_4 (note the @@) we are telling the linker that this should be considered the default version of curl_symbol_name. In other words, when a binary that links against libcurl-gnutls calls curl_symbol_name, the linker should use curl_symbol_name@@CURL_GNUTLS_4 to resolve the symbol. There are a few ways to specify a symbol version in C/C++:
__attribute__((__symver__("curl_symbol_name@@CURL_GNUTLS_4")))
void curl_symbol_name()
 
  /* ... */
 

/* or... */
void curl_symbol_name()
 
  /* ... */
 
__asm__(".symver curl_symbol_name, curl_symbol_name@@CURL_GNUTLS_4");

Function alias Creating an alias for a function is basically saying that a function can be called by another name. You can do that in C/C++ like:
void curl_symbol_name()
 
  /* ... */
 

void __curl_compat_symbol_name()
  __attribute__((alias("curl_symbol_name")));

__curl_compat_symbol_name@CURL_GNUTLS_3 Finally, when we use __curl_compat_symbol_name@CURL_GNUTL_3 (note the single @) we are telling the linker that this symbol exists, but it should not be used as the default symbol. In fact, this notation will basically hide the symbol and make it only available for those programs that have already been linked against it. It's a way of saying "don't offer this symbol when linking, but it's here in case a program needs it to run" (it's a bit more complicated than that, but you get the point). The reason I had to create an alias to the function before versioning the symbol with @CURL_GNUTLS_3 is because, once I've versioned the main symbol as @@CURL_GNUTLS_4, I can't create another version of it. It's also important to mention that to be able to create a version for the alias I also had to change its visibility to default. In the end, the alias ended up being defined as:
extern void __curl_compat_symbol_name()
  __attribute__((alias("curl_symbol_name"), visibility("default")));

First attempt and lessons learned For my PoC I decided to tackle a small subset of the problem. The symbols file for libcurl3-gnutls contains around 100 symbols that need to be fixed, so I chose two of them and started trying to write a patch to see if I could make things work. And after some time struggling with GCC's syntax and inspecting nm -D's output I finally got something that looked like it was going to work. The two symbols I had chosen to work with got correctly versioned (both as @@CURL_GNUTLS_4 and @CURL_GNUTLS_3), and a quick-and-dirty C program that used those symbols correctly compiled and ran with the expected symbols. I showed the results to Samuel and Charles, we got excited about what we saw, and then the conference ended.

Second attempt and some adjustments After getting back home I resumed the work on my branch and wrote an Emacs function that semi-automatically adjusted all 100+ symbols listed in the symbols file so that they all looked like:
__attribute__((__symver__("curl_symbol_name@@CURL_GNUTLS_4")))
void curl_symbol_name()
 
  /* ... */
 

extern void __curl_compat_symbol_name()
  __attribute__((alias("curl_symbol_name"), visibility("default"),
                 symver("__curl_compat_symbol_name@CURL_GNUTLS_3")));
The patch was big but mostly repetitive, and I was happy to have come up with a solution that looked clean. Until I tried to build the package, that is. I started seeing some strange errors that happened when ld was trying to link the final libcurl4-gnutls object (yes, at that point I had already renamed the binary package). This is one of the errors I was getting from ld (I got variants of this error as I was trying to fix the approach):
/usr/bin/x86_64-linux-gnu-ld.bfd: .libs/libcurl_gnutls_la-easy.o: in function  dupeasy_meta_freeentry':
./debian/build-gnutls/lib/./debian/build-gnutls/lib/easy.c:1024: multiple definition of  curl_easy_cleanup'; .libs/libcurl_gnutls_la-easy.o:./debian/build-gnutls/lib/./debian/build-gnutls/lib/easy.c:908: first defined here
/usr/bin/x86_64-linux-gnu-ld.bfd: .libs/libcurl-gnutls.so.4.8.0: version node not found for symbol curl_easy_duphandle@CURL_GNUTLS3
/usr/bin/x86_64-linux-gnu-ld.bfd: failed to set dynamic section sizes: bad value
This was strange. I did some tests with very simple versions of a shared library using the versioning mechanism I had implemented and it all worked. I could not reproduce the problem, and that's not a great feeling to have. Then, after reading a lot of documentation and blog posts throughout the internet I found something interesting. Apparently ld has a limitation when it comes to dealing with symbols versioned with @@. If there is a single symbol versioned like that in a source file (the actual term is TU, which means Translation Unit, but let's simplify), then ld is happy and generates the expected version without issues. But when we're dealing with multiple definitions of @@ symbols in a source file (which is exactly what happens in curl), then ld can get confused and start giving errors during the link stage. To solve that limitation, we have to resort to yet another symbol versioning notation: @@@. Yes, three at signs. For example:
void curl_symbol_name()
 
  /* ... */
 
__asm__(".symver curl_symbol_name, curl_symbol_name@@@CURL_GNUTLS_4");
Note that we have to use __asm__ because GCC's __attribute__ doesn't support the triple-at notation. What this does is tell the linker to create a versioned symbol for curl_symbol_name, set it as the default symbol when linking, but also remove the unversioned curl_symbol_name symbol. This makes ld happy and allows it to successfully link libcurl-gnutls. As usual, you won't find any mention of the @@@ notation inside ld's documentation. With libcurl-gnutls compiling again, I had to adjust libcurl's linker script to create a hierarchy between CURL_GNUTLS_3 and CURL_GNUTLS_4 symbols. Here's the final version of the file:
CURL_GNUTLS_3
 
  global:
    curl_easy_cleanup;
    /* lots of other symbols here */
  local: *;
 ;

CURL_GNUTLS_4
 
  global: curl_*;
  local: *;
  CURL_GNUTLS_3;

Debian package adjustments After getting the hard part out of the way, the rest was easy. It was time to finally rename libcurl3-gnutls to libcurl4-gnutls. Initially I was thinking that I'd need to ask the release team for a transition to happen, but as it turns out that won't be necessary. Because we are effectively shipping the same exact library/ABI and the only difference is the inclusion of the extra CURL_GNUTLS_4 versioned symbols, and given that we will be shipping CURL_GNUTLS_3 versioned symbols to guarantee backwards compatibility, packages won't need to get rebuild just to pick up the new dependency. Instead, we can safely turn libcurl3-gnutls into a transitional package that depends on libcurl4-gnutls.

Merge request and next steps This is the merge request where I am working on the fix. As of this writing it is in a draft state, but I expect to merge in the next couple of days. Once the fixed curl package is uploaded, we should keep an eye on the archive to make sure no unexpected bugs happen. I would like to carry this patch downstream at least until forky is released. It doesn't make sense to propose it upstream because this problem is Debian-specific and should be fixed there. We will need to make sure that all reverse dependencies of libcurl3-gnutls are recompiled before we can get rid of the transitional package, too. This was a fun bug to investigate and fix, and I am happy that we will finally have sensible names (and symbol versions!) for both of our libcurl variants. Stay tuned for the next challenge!

Next.