Welcome to the February 2026 report from the Reproducible Builds project! These reports outline what we ve been up to over the past month, highlighting items of news from elsewhere in the increasingly-important area of software supply-chain security. As ever, if you are interested in contributing to the Reproducible Builds project, please see the Contribute page on our website.

1. reproduce.debian.net
2. Tool development
3. Distribution work
4. Miscellaneous news
5. Upstream patches
6. Documentation updates
7. Four new academic papers

reproduce.debian.net The last year has seen the introduction, development and deployment of reproduce.debian.net. In technical terms, this is an instance of rebuilderd, our server designed monitor the official package repositories of Linux distributions and attempt to reproduce the observed results there. This month, however, Holger Levsen added suite-based navigation (eg. Debian trixie vs forky) to the service (in addition to the already existing architecture based navigation) which can be observed on, for instance, the Debian trixie-backports or trixie-security pages.

Tool development diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes, including preparing and uploading versions, 312 and 313 to Debian. In particular, Chris updated the post-release deployment pipeline to ensure that the pipeline does not fail if the automatic deployment to PyPI fails [ ]. In addition, Vagrant Cascadian updated an external reference for the 7z tool for GNU Guix. [ ]. Vagrant Cascadian also updated diffoscope in GNU Guix to version 312 and 313.

Distribution work In Debian this month:

• 26 reviews of Debian packages were added, 5 were updated and 19 were removed this month adding to our extensive knowledge about identified issues.
• A new debsbom package was uploaded to unstable. According to the package description, this package generates SBOMs (Software Bill of Materials) for distributions based on Debian in the two standard formats, SPDX and CycloneDX. The generated SBOM includes all installed binary packages and also contains Debian Source packages.
• In addition, a sbom-toolkit package was uploaded, which provides a collection of scripts for generating SBOM. This is the tooling used in Apertis to generate the Licenses SBOM and the Build Dependency SBOM. It also includes dh-setup-copyright, a Debhelper addon to generate SBOMs from DWARF debug information, which are extracted from DWARF debug information by running dwarf2sources on every ELF binaries in the package and saving the output.

Lastly, Bernhard M. Wiedemann posted another openSUSE monthly update for their work there.

Miscellaneous news

• S ren Tempel (nmeum) wrote up their insightful notes on Debugging Reproducibility Issues in Rust Software after nondeterministic issues were found and investigated for pimsync in the GNU Guix review process
• Jeremy Bicha reported a bug in GNOME Clocks after they noticed that version 50.beta regressed in reproducibility compared to 49.0. Specifically, the new generated .oga files differ in their Serial No. and Checksum [fields] . However, Jeremy ended up fixing the issue by replacing ffmpeg with oggenc.
• kpcyrd shared some information from the archlinux-dev-public mailing list on our mailing list this month after a discussion at our latest Summit meeting on the topic of Link-Time Optimisation (LTO) specifically on the reasons why LTO often needs to be disabled in relation to Arch Linux s approach to binary hardening.
• Janneke Nieuwenhuizen posed a question to our list about whether there might be situations where using the UNIX epoch itself (i.e. 0) may materially differ from using SOURCE_DATE_EPOCH) when a situation demands the use of a fixed timestamp.
• Laurent Huberdeau announced that they had recently finished their masters thesis arguing for the use of POSIX shell for diverse double-compilation and reproducible builds . Laurent also presents pnut, a C compiler capable of bootstrapping itself and TCC from any POSIX-compliant shell and human-readable source files.

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

• Bernhard M. Wiedemann:
  • aiohttp
  • lima
  • meson
  • obs-studio
  • paracon
  • pyshark
  • python-flake8-comprehensions
  • server
  • vlang
• Gioele Barabucci:
  • #1127641 filed against bitsnpicas.
  • #1127643 filed against fonts-topaz-unicode.
  • #1128901 filed against bitsnpicas.

Documentation updates Once again, there were a number of improvements made to our website this month including:

• Aman Sharma added a Java reproducible builds paper to the Academic publications page. [ ]
• Chris Lamb added a reference to the repro-build to the Tools page. [ ]
• Michiel Hendriks corrected an issue on the JVM page in relation to .properties files. [ ]
• kpcyrd added Homebrew to the Who is involved page. [ ][ ]

Four new academic papers Julien Malka and Arnout Engelen published a paper titled Lila: Decentralized Build Reproducibility Monitoring for the Functional Package Management Model:

[While] recent studies have shown that high reproducibility rates are achievable at scale demonstrated by the Nix ecosystem achieving over 90% reproducibility on more than 80,000 packages the problem of effective reproducibility monitoring remains largely unsolved. In this work, we address the reproducibility monitoring challenge by introducing Lila, a decentralized system for reproducibility assessment tailored to the functional package management model. Lila enables distributed reporting of build results and aggregation into a reproducibility database [ ].

A PDF of their paper is available online.
Javier Ron and Martin Monperrus of KTH Royal Institute of Technology, Sweden, also published a paper, titled Verifiable Provenance of Software Artifacts with Zero-Knowledge Compilation:

Verifying that a compiled binary originates from its claimed source code is a fundamental security requirement, called source code provenance. Achieving verifiable source code provenance in practice remains challenging. The most popular technique, called reproducible builds, requires difficult matching and reexecution of build toolchains and environments. We propose a novel approach to verifiable provenance based on compiling software with zero-knowledge virtual machines (zkVMs). By executing a compiler within a zkVM, our system produces both the compiled output and a cryptographic proof attesting that the compilation was performed on the claimed source code with the claimed compiler. [ ]

A PDF of the paper is available online.
Oreofe Solarin of Department of Computer and Data Sciences, Case Western Reserve University, Cleveland, Ohio, USA, published It s Not Just Timestamps: A Study on Docker Reproducibility:

Reproducible container builds promise a simple integrity check for software supply chains: rebuild an image from its Dockerfile and compare hashes. We built a Docker measurement pipeline and apply it to a stratified sample of 2,000 GitHub repositories that contained a Dockerfile. We found that only 56% produce any buildable image, and just 2.7% of those are bitwise reproducible without any infrastructure configurations. After modifying infrastructure configurations, we raise bitwise reproducibility by 18.6%, but 78.7% of buildable Dockerfiles remain non-reproducible.

A PDF of Oreofe s paper is available online.
Lastly, Jens Dietrich and Behnaz Hassanshahi published On the Variability of Source Code in Maven Package Rebuilds:

[In] this paper we test the assumption that the same source code is being used [by] alternative builds. To study this, we compare the sources released with packages on Maven Central, with the sources associated with independently built packages from Google s Assured Open Source and Oracle s Build-from-Source projects. [ ]

A PDF of their paper is available online.

Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Mastodon: @reproducible_builds@fosstodon.org
• Mailing list: rb-general@lists.reproducible-builds.org

Debian Contributions: 2026-02 Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

DebConf 26 Registration, by Stefano Rivera, Antonio Terceiro, and Santiago Ruano Rinc n DebConf 26, to be held in Santa Fe Argentina in July, has opened for registration and event proposals. Stefano, Antonio, and Santiago all contributed to making this happen. As always, some changes needed to be made to the registration system. Bigger changes were planned, but we ran out of time to implement them for DebConf 26. All 3 of us have had experience in hosting local DebConf events in the past and have been advising the DebConf 26 local team.

Debian CI improvements, by Antonio Terceiro Debian CI is the platform responsible for automated testing of packages from the Debian archive, and its results are used by the Debian Release team automation as Quality Assurance to control the migration of packages from Debian unstable into testing, the base for the next Debian release. Antonio started developing an incus backend, and that prompted two rounds of improvements to the platform, including but not limited to allowing user to select a job execution backend (lxc, qemu) during the job submission, reducing the part of testbed image creation that requires superuser privileges and other refactorings and bug fixes. The platform API was also improved to reduce disruption when reporting results to the Release Team automation after service downtimes. Last, but not least, the platform now has support for testing packages against variants of autopkgtest, which will allow the Debian CI team to test new versions of autopkgtest before making releases to avoid widespread regressions.

Miscellaneous contributions

• Carles improved po-debconf-manager while users requested features / found bugs. Improvements done - add packages from unstable instead of just salsa.debian.org, upgrade and merge templates of upgraded packages, finished adding typing annotations, improved deleting packages: support multiple line texts, add debug to see subprocess.run commands, etc.
• Carles, using po-debconf-manager, reviewed 7 Catalan translations and sent bug reports or MRs for 11 packages. Also reviewed the translations of fortunes-debian-hints and submitted possible changes in the hints.
• Carles submitted MRs for reportbug (reportbug --ui gtk detecting the wrong dependencies), devscript (delete unused code from debrebuild and add recommended dependency), wcurl (format help for 80 columns). Carles submitted a bug report for apt not showing the long descriptions of packages.
• Carles resumed effort for checking relations (e.g. Recommends / Suggests) between Debian packages. A new codebase (still in early stages) was started with a new approach in order to detect, report and track the broken relations.
• Emilio drove several transitions, most notably the haskell transition and the glibc/gcc-15/zlib transition for the s390 31-bit removal. This last one included reviewing and requeueing lots of autopkgtests due to britney losing a lot of results.
• Emilio reviewed and uploaded poppler updates to experimental for a new transition.
• Emilio reviewed, merged and deployed some performance improvements proposed for the security-tracker.
• Stefano prepared routine updates for pycparser, python-confuse, python-cffi, python-mitogen, python-pip, wheel, platformdirs, python-authlib, and python-virtualenv.
• Stefano updated Python 3.13 and 3.14 to the latest point releases, including security updates, and did some preliminary work for Python 3.15.
• Stefano reviewed changes to dh-python and merged MRs.
• Stefano did some debian.social sysadmin work, bridging additional IRC channels to Matrix.
• Stefano and Antonio, as DebConf Committee Members, reviewed the DebConf 27 bids and took part in selecting the Japanese bid to host DebConf 27.
• Helmut sent patches for 29 cross build failures.
• Helmut continued to maintain rebootstrap addressing issues relating to specific architectures (such as musl-linux-any, hurd-any or s390x) or specific packages (such as binutils, brotli or fontconfig).
• Helmut worked on diagnosing bugs such as rocblas #1126608, python-memray #1126944 upstream and greetd #1129070 with varying success.
• Antonio provided support for multiple MiniDebConfs whose websites run wafer + wafer-debconf (the same stack as DebConf itself).
• Antonio fixed the salsa tagpending webhook.
• Antonio sent specinfra upstream a patch to fix detection of Debian systems in some situations.
• Santiago reviewed some Merge Requests for the Salsa CI pipeline, including !703 and !704, that aim to improve how the build source job is handled by Salsa CI. Thanks a lot to Jochen for his work on this.
• In collaboration with Emmanuel Arias, Santiago proposed a couple of projects for the Google Summer of Code (GSoC) 2026 round. Santiago has been reviewing applications and giving feedback to candidates.
• Thorsten uploaded new upstream versions of ipp-usb, brlaser and gutenprint.
• Rapha l updated publican to fix an old bug that became release critical and that happened only when building with the nocheck profile. Publican is a build dependency of the Debian s Administrator Handbook and with that fix, the package is back into testing.
• Rapha l implemented a small feature in Debusine that makes it possible to refer to a collection in a parent workspace even if a collection with the same name is present in the current workspace.
• Lucas updated the current status of ruby packages affecting the Ruby 3.4 transition after a bunch of updates made by team members. He will follow up on this next month.
• Lucas joined the Debian orga team for GSoC this year and tried to reach out to potential mentors.
• Lucas did some content work for MiniDebConf Campinas - Brazil.
• Colin published minor security updates to bookworm and trixie for CVE-2025-61984 and CVE-2025-61985 in OpenSSH, both of which allowed code execution via ProxyCommand in some cases. The trixie update also included a fix for mishandling of PerSourceMaxStartups.
• Colin spotted and fixed a typo in the bug tracking system s spam-handling rules, which in combination with a devscripts regression caused bts forwarded commands to be discarded.
• Colin ported 12 more Python packages away from using the deprecated (and now removed upstream) pkg_resources module.
• Anupa is co-organizing MiniDebConf Kanpur with Debian India team. Anupa was responsible for preparing the schedule, publishing it on the website, co-ordination with the fiscal host in addition to attending meetings.
• Anupa attended the Debian Publicity team online sprint which was a skill sharing session.

So you want to join Outreachy but you don t understand it, you re scared, or you don t know what open source is about.

What is FOSS anyway?

Free and Open Source Software (FOSS) refers to software that anyone can use, modify, and share freely. Think of it as a community garden; instead of one company owning the food, people from all over the world contribute, improve, and maintain it so everyone can benefit for free. You can read more here on what it means to contribute to open source.

Outreachy provides paid internships to anyone from any background who faces underrepresentation, systemic bias, or discrimination in the technical industry where they live. Their goal is to increase diversity in open source. Read their website for more. I spent a good amount of time reading all the guides listed, including the applicant guide and the how-to-apply guide.

The Secret to Applying (Spoiler: It s not a secret)

I know newcomers are scared or unsure and would prefer answers from previous participants, but the Outreachy website is actually a goldmine, almost every question you have is already answered there if you look closely. I used to hate reading documentation, but I ve learned to love it. Documentation is the Source of Truth.

• My Advice: Read every single guide on their site. The applicant guide is your roadmap. Embracing documentation now will make you a much better contributor later.

The AI Trap: Be Yourself

Now for the part most newcomers have asked about is the initial essay. I know it s tempting to use AI, but I really encourage you to skip it for this. Your own story is much more powerful than a generated one. Outreachy and its mentoring organizations value your unique story. They are strongly against fabricated or AI-exaggerated essays.

For example, when I contributed to Debian using openQA, the information wasn t well established on the web. When I tried to use AI, it suggested imaginary ideas. The project maintainers had a particular style of contributing, so I had to follow the instructions carefully, observe the codebase, and read the provided documentation. With that information, I always wrote a solution first before consulting AI, and mine was always better. AI can only be intelligent in the context of what you give it; if it doesn t have your answer, it will look for the most similar solution (hallucinate). We do not want to increase the burden on reviewers their time is important because they are volunteers, too. This is crucial when you qualify for the contribution phase.

The Application Process

There are two main stages:

• The initial application: Here you fill in basic details, time availability, and essay questions (you can find these on the Outreachy website).
• The contribution phase: This is where you show you have the skills to work on the projects. Every project will list the skills needed and the level of proficiency.

When you qualify for the contribution phase:

• A lot of people will try to create buzz or even panic; you just have to focus. Once you ve gotten the hang of the project, remember to help others along the way.
• You can start contributions with spelling corrections, move to medium tasks (do multiple of these), then a hard task if possible. You don t need to be a guru on day one.
• It s all about community building. Do your part to help others understand the project too; this is also a form of contribution.
• Lastly, every project mentor has a way of evaluating candidates. My summary is: be confident, demonstrate your skills, and learn where you are lacking. Start small and work your way up, you don t have to prove yourself as a guru.

Tips

• Watch this: This step-by-step video is a great walkthrough of the initial application process.
• Sign up for the email list to get updates: https://lists.outreachy.org/cgi-bin/mailman/listinfo/announce
• Be fast: Complete your initial application in the first 3 days, as there are a lot of applicants.
• Back it up: In your essay about systemic bias, include some statistics to back it up.
• Learn Git: Even if you don t have programming skills, contributions are pushed to GitHub or GitLab. Practice some commands and contribute to a first open issue to understand the flow: https://github.com/firstcontributions/first-contributions

The most important tip? Apply anyway. Even if you feel underqualified, the process itself is a massive learning experience.

Changes in version 0.3.13 (2026-03-08)

• The methods package is now a Depends as WRE recommends (Michael Chirico in #141 based on a suggestion by Dirk in #140)
• The mkdocs-material documentation site is now generated via altdoc
• Continuous Integration scripts have been updated
• Replace Rf_error with Rcpp::stop, turn remaining one into (Rf_error) (Dirk in #143)
• Vignette now uses the Rcpp::asis builder for pre-made pdfs (Dirk in #146 fixing #144)
• The C++ compilation standard is explicitly set to C++17 if an R version older than 4.3.0 is used (Dirk in #148 fixing #147)
• The vignette references have been updated

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

• openssh: Please remove/replace usage of dh_movetousr

• aiosmtplib
• bitstruct
• diff-cover
• django-q
• isort
• multipart
• poetry (adding support for Dulwich >= 0.25)
• poetry-core
• pydantic-settings
• python-build
• python-certifi
• python-datamodel-code-generator
• python-flatdict
• python-holidays
• python-maggma
• python-pytokens
• python-scruffy
• python-urllib3 (fixing CVE-2025-66471 and a chunked decoding bug)
• responses
• yarsync
• zope.component
• zope.deferredimport

• genshi (contributed upstream)
• germinate
• mopidy
• nose2
• pokrok (contributed upstream)
• pylama
• python-flask-seeder
• python-maggma (contributed upstream)
• python-pybadges
• python-scruffy (contributed upstream)
• thumbor (contributed upstream)
• zope.deprecation (contributed upstream a while ago, but there hasn t been an upstream release yet)

• flask-dance: FTBFS: No module named pkg_resources (actually fixed by adding a missing dependency to python3-sphinxcontrib.seqdiag)
• paramiko: autopkgtest regression on i386 (contributed upstream)
• poetry: autopkgtest regression on i386
• python-argh
• python-django-celery-beat: FTBFS: FAILED t/unit/test_models.py::HumanReadableTestCase::test_long_name
• python-maturin: rust-itertools update
• python-msrest: FTBFS: FAILED tests/asynctests/test_async_client.py::TestServiceClient::test_client_send (contributed upstream, though not very successfully)
• python-typing-inspect

• python-datamodel-code-generator: Depends: python3-isort (< 8) but 8.0.0-1 is to be installed (contributed upstream)
• python-typeguard: Mark python3-typeguard Multi-Arch: foreign
• wheel: Mark python3-wheel Multi-Arch: foreign
• zope.deferredimport: Please make the build reproducible (contributed upstream, with a follow-up fix)

• librust-pyo3-ffi-dev: Cannot be installed for foreign architectures

• arch-test: Remove build dependency on binutils-mips64el-linux-gnuabi64 (NMU)

• debconf: Add BMP version of debian-logo (merged and uploaded)
• openssh: Reorder pam_selinux(7) usage (merged and uploaded)
• openssh-client: use sysusers.d, drop superflous dependencies (merged and uploaded)
• openssh: Stop deleting system user on remove/purge (merged and uploaded)
• openssh: Do not link against libcrypt on GNU/Hurd (merged and uploaded)
• partman-prep: Align PReP descriptions with other partition types (merged)
• python-better-exceptions (sponsored upload for Seyed Mohamad Amin Modaresi)

Changes in RProtoBuf version 0.4.26 (2026-03-06)

• Minor cleanup in DESCRIPTION depends and imports
• Remove obsolete check for utils::.DollarNames (Michael Chirico in #111)
• Replace Rf_error with Rcpp::stop, turn remaining one into (Rf_error) (Dirk in #112)
• Update configure test to check for RProtoBuf 3.3.0 or later

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

Why? The main reason why I'm retiring Wallabako is that I have completely stopped using it. It's not the first time: for a while, I wasn't reading Wallabag articles on my Kobo anymore. But I had started working on it again about four years ago. Wallabako itself is about to turn 10 years old. This time, I stopped using Wallabako because there's simply something better out there. I have switched away from Wallabag to Readeck! And I'm also tired of maintaining "modern" software. Most of the recent commits on Wallabako are from renovate-bot. This feels futile and pointless. I guess it must be done at some point, but it also feels we went wrong somewhere there. Maybe Filippo Valsorda is right and one should turn dependabot off. I did consider porting Wallabako to Readeck for a while, but there's a perfectly fine Koreader plugin that I've been pretty happy to use. I was worried it would be slow (because the Wallabag plugin is slow), but it turns out that Readeck is fast enough that this doesn't matter.

Moving from Wallabag to Readeck Readeck is pretty fantastic: it's fast, it's lightweight, everything Just Works. All sorts of concerns I had with Wallabag are just gone: questionable authentication, questionable API, weird bugs, mostly gone. I am still looking for multiple tags filtering but I have a much better feeling about Readeck than Wallabag: it's written in Golang and under active development. In any case, I don't want to throw shade at the Wallabag folks either. They did solve most of the issues I raised with them and even accepted my pull request. They have helped me collect thousands of articles for a long time! It's just time to move on. The migration from Wallabag was impressively simple. The importer is well-tuned, fast, and just works. I wrote about the import in this issue, but it took about 20 minutes to import essentially all articles, and another 5 hours to refresh all the contents. There are minor issues with Readeck which I have filed (after asking!):

• add justified view for articles (Android app)
• more metadata in article display (Android app)
• show the number of articles in the label browser
• ignore duplicates (Readeck will happily add duplicates, whereas Wallabag at least tries to deduplicate articles -- but often fails)

But overall I'm happy and impressed with the result. I'm also both happy and sad at letting go of my first (and only, so far) Golang project. I loved writing in Go: it's a clean language, fast to learn, and a beauty to write parallel code in (at the cost of a rather obscure runtime). It would have been much harder to write this in Python, but my experience in Golang helped me think about how to write more parallel code in Python, which is kind of cool. The GitLab project will remain publicly accessible, but archived, for the foreseeable future. If you're interested in taking over stewardship for this project, contact me. Thanks Wallabag folks, it was a great ride!

• 190g plain flour
• 60g strong white bread flour
• 4 tsp baking powder
• tsp bicarbonate of soda
• 1 tsp cream of tartar (optional)
• 1 tsp salt
• 100g unsalted butter
• 180ml buttermilk, chilled
  • If your buttermilk is thicker than the consistency of ordinary milk, you ll need around 200ml.
• extra buttermilk for brushing

1. Slice and then chill the butter in the freezer for at least fifteen minutes.
2. Preheat oven to 220 C with the fan turned off.
3. Twice sieve together the flours, leaveners and salt. Some salt may not go through the sieve; just tip it back into the bowl.
4. Cut cold butter slices into the flour with a pastry blender until the mixture resembles coarse crumbs: some small lumps of fat remaining is desirable. In particular, the fine crumbs you are looking for when making British scones are not wanted here. Rubbing in with fingertips just won t do; biscuits demand keeping things cold even more than shortcrust pastry does.
5. Make a well in the centre, pour in the buttermilk, and stir with a metal spoon until the dough comes together and pulls away from the sides of the bowl. Avoid overmixing, but I ve found that so long as the ingredients are cold, you don t have to be too gentle at this stage and can make sure all the crumbs are mixed in.
6. Flour your hands, turn dough onto a floured work surface, and pat together into a rectangle. Some suggest dusting the top of the dough with flour, too, here.
7. Fold the dough in half, then gather any crumbs and pat it back into the same shape. Turn ninety degrees and do the same again, until you have completed a total of eight folds, two in each cardinal direction. The dough should now be a little springy.
8. Roll to about inch thick.
9. Cut out biscuits. If using a round cutter, do not twist it, as that seals the edges of the biscuits and so spoils the layering.
10. Transfer to a baking sheet, placed close together (helps them rise). Flour your thumb and use it to press an indent into the top of each biscuit (helps them rise straight), brush with buttermilk.
11. Bake until flaky and golden brown: about fifteen minutes.

• These biscuits do come out fluffy but not so flaky. For that you can try using lard instead of butter, if you re not vegetarian (vegetable shortening is hard to find here).
• If you don t have a pastry blender and don t want to buy one you can try not slicing the butter and instead coarsely grating it into the flour out of the freezer.
• An alternative to folding is cutting and piling the layers.
• You can try rolling out to 1 1 inches thick.
• Instead of cutting out biscuits you can just slice the whole piece of dough into equal pieces. An advantage of this is that you don t have to re-roll, which latter also spoils the layering.
• Instead of brushing with buttermilk, you can take them out after they ve started to rise but before they ve browned, brush them with melted butter and put them back in.

• I ve had more success with Dale Farm s buttermilk than Sainsbury s own. The former is much runnier.
• Southern culture calls for biscuits to be made the size of cat s heads.
• Bleached flour is apparently usual in the South, but is illegal(!) here. Apparently bleaching can have some effect on the development of the gluten which would affect the texture.
• British plain flour is made from soft wheat and has a lower percentage of protein/gluten, while American all-purpose flour is often(?) made from harder wheat and has more protein. In this recipe I mix plain and strong white flour, in a ratio of 3:1, to emulate American all-purpose flour. I am not sure why this works best. In the South they have soft wheats too, and lower protein percentages. The famous White Lily flour is 9%. (Apparently you can mix US cake flour and US all-purpose flour in a ratio of 1:1 to achieve that; in the UK, Shipton Mill sell a soft cake and pastry flour which has been recommended to me as similar.) This would suggest that British plain flour ought to be closer to Southern flour than the standard flour available in most of the US. But my experience has been that the biscuits taste better with the plain and strong white 3:1 mix. Possibly Southeners would disprefer them. I got some feedback that good biscuits are about texture and moistness and not flavour.
• Baking powder in the US is usually double-acting but ours is always single-acting, so we need double quantities of that.

• Does this blob do what it claims to do? Does it suddenly introduce functionality you don t want? Does it introduce security flaws? Does it introduce deliberate backdoors? Does it make your life better or worse?
• You re almost certainly being provided with a blob of compiled code, with no source code available. You can t just diff the source files, satisfy yourself that they re fine, and then install them. To be fair, even though you (as someone reading this) are probably more capable of doing that than the average human, you re likely not doing that even if you are capable because you re also likely installing kernel upgrades that contain vast quantities of code beyond your ability to understand⁴. We don t rely on our personal ability, we rely on the ability of those around us to do that validation, and we rely on an existing (possibly transitive) trust relationship with those involved. You don t know the people who created this blob, you likely don t know people who do know the people who created this blob, these people probably don t have an online presence that gives you more insight. Why should you trust them?
• If it s in ROM and it turns out to be hostile then nobody can fix it ever
• The people creating these blobs largely work for the same company that built the hardware in the first place. When they built that hardware they could have backdoored it in any number of ways. And if the hardware has a built-in copy of the code it runs, why do you trust that that copy isn t backdoored? Maybe it isn t and updates would introduce a backdoor, but in that case if you buy new hardware that runs new code aren t you putting yourself at the same risk?
• Designing hardware where you re able to provide updated code and nobody else can is just a dick move⁵. We shouldn t encourage vendors who do that.
• Humans are bad at writing code, and code running on ancilliary hardware is no exception. It contains bugs. These bugs are sometimes very bad. This paper describes a set of vulnerabilities identified in code running on SSDs that made it possible to bypass encryption secrets. The SSD vendors released updates that fixed these issues. If the code couldn t be replaced then anyone relying on those security features would need to replace the hardware.
• Even if blobs are signed and can t easily be replaced, the ones that aren t encrypted can still be examined. The SSD vulnerabilities above were identifiable because researchers were able to reverse engineer the updates. It can be more annoying to audit binary code than source code, but it s still possible.
• Vulnerabilities in code running on other hardware can still compromise the OS. If someone can compromise the code running on your wifi card then if you don t have a strong IOMMU setup they re going to be able to overwrite your running OS.
• Replacing one non-free blob with another non-free blob increases the total number of non-free blobs involved in the whole system, but doesn t increase the number that are actually executing at any point in time.

Twelve weeks ago, I stepped into the Debian ecosystem as an Outreachy intern with a curiosity for Quality Assurance. It feels like just yesterday, and time has flown by so fast! Now, I am wrapping up that journey, not just with a completed project, but with improved technical reasoning.

I have learned how to use documentation to understand a complex project, how to be a good collaborator, and that learning is a continuous process. These experiences have helped me grow much more confident in my skills as an engineer.

My Achievements

As I close this chapter, I am leaving a permanent Proof-of-Work in the Debian repositories:

• Full Test Coverage: I automated apps_startstop tests for Cinnamon, LXQt, and XFCE, covering both Live images and Netinst installations.
• Synergy: I used symbolic links and a single Perl script to handle common application tests across different desktops, which reduces code redundancy.
• The Contributor Style Guide: I created a guide for future contributors to make documentation clearer and reviews faster, helping to reduce the burden on reviewers.

Final Month: Wrap Up

In this final month, things became easier as my understanding of the project grew. I focused on stability and finishing my remaining tasks:

• I spent time exploring different QEMU video options like VGA, qxl, and virtio on KDE desktop environment . This was important to ensure screen rendering remained stable so that our needles (visual test markers) wouldn t fail because of minor glitches.
• I successfully moved from familiarizing to test automation for the XFCE desktop. This included writing prepare steps and creating the visual needles needed to make the tests reliable.
• One of my final challenges was the app launcher function. Originally, my code used else if blocks for each desktop. I proposed a unified solution, but hit a blocker: XFCE has two ways to launch apps (App Finder and the Application Menu). Because using different methods sometimes caused failures, I chose to use the application menu button across the board.

What s Next?

I don t want my journey with Debian to end here. I plan to stay involved in the community and extend these same tests to the LXDE desktop to complete the coverage for all major Debian desktop environments. I am excited to keep exploring and learning more about the Debian ecosystem.

Thank You

This journey wouldn t have been possible without the steady guidance of my mentors: Tassia Camoes Araujo, Roland Clobus, and Philip Hands. Thank you for teaching me that in the world of Free and Open Source Software (FOSS), your voice and your code are equally important.

To my fellow intern Hellen and the entire Outreachy community, thank you for the shared learning and support. It has been an incredible 12 weeks.

What was that? So what did you just see (or skipped watching)? You could see Emilio s screen, running VSCode and editing a Lean file. He designed a small programming language that he embedded into Lean, including an evaluator. So far, so standard, but a few things stick out already:

• Using Lean s very extensible syntax this embedding is rather elegant and pretty.
• Furthermore, he can run this DSL code right there, in the source code, using commands like #eval. This is a bit like the interpreter found in Haskell or Python, but without needing a separate process, or like using a Jupyter notebook, but without the stateful cell management.

This is already a nice demonstration of Lean s abilities and strength, as we know them. But what blew my mind the first time was what happened next: He had a visual debugger that allowed him to debug his DSL program. It appeared on the right, in Lean s Info View , where various Lean tools can hook into, show information and allow the user to interact. But it did not stop there, and my mind was blown a second time: Emilio opened VSCode s Debugger pane on the left, and was able to properly use VSCode s full-fledged debugger frontend for his own little embedded programming language! Complete with highlighting the executed line, with the ability to set breakpoints there, and showing the state of local variables in the debugger. Having a good debugger is not to be taken for granted even for serious, practical programming languages. Having it for a small embedded language that you just built yourself? I wouldn t have even considered that.

Did it take long? If I were Emilio s manager I would applaud the demo and then would have to ask how many weeks he spent on that. Coming up with the language, getting the syntax extension right, writing the evaluator and especially learning how the debugger integration into VSCode (using the DAP protocol) works, and then instrumenting his evaluator to speak that protocol that is a sizeable project! It turns out the answer isn t measured in weeks: it took just one day of coding together with GPT-Codex 5.3. My mind was blown a third time.

Why does Lean make a difference? I am sure this post is just one of many stories you have read in recent weeks about how new models like Claude Opus 4.6 and GPT-Codex 5.3 built impressive things in hours that would have taken days or more before. But have you seen something like this? Agentic coding is powerful, but limited by what the underlying platform exposes. I claim that Lean is a particularly well-suited platform to unleash the agents versatility. Here we are using Lean as a programming language, not as a theorem prover (which brings other immediate benefits when using agents, e.g. the produced code can be verified rather than merely plausible, but that s a story to be told elsewhere.) But arguably because Lean is also a theorem prover, and because of the requirements that stem from that, its architecture is different from that of a conventional programming language implementation:

• As a theorem prover, it needs extensible syntax to allow formalizing mathematics in an ergonomic way, but it can also be used for embedding syntax.
• As a theorem prover, it needs the ability to run tactics written by the user, hence the ability to evaluate the code right there in the editor.
• As a theorem prover, it needs to give access to information such as tactic state, and such introspection abilities unlock many other features such as a debugger for an embedded language.
• As a theorem prover, it has to allow tools to present information like the tactic state, so it has the concept of interactive Widgets .

So Lean s design has always made such a feat possible. But it was no easy feat. The Lean API is large, and documentation never ceases to be improvable. In the past, it would take an expert (or someone willing to become one) to pull off that stunt. These days, coding assistants have no issue digesting, understanding and using the API, as Emilio s demo shows. The combination of Lean s extensibility and the ability of coding agents to make use of that is a game changer to how we can develop software, with rich, deep, flexible and bespoke ways to interact with our code, created on demand.

Where does that lead us? Emilio actually shared more such demos (Github repository). A visual explorer for the compiler output (have a look at the screenshot. A browser-devtool-like inspection tool for Lean s InfoTree . Any of these provide a significant productivity boost. Any of these would have been a sizeable project half a year ago. Now it s just a few hours of chatting with the agent. So allow me to try and extrapolate into a future where coding agents have continued to advance at the current pace, and are used ubiquitously. Is there then even a point in polishing these tools, shipping them to our users, documenting them? Why build a compiler explorer for our users, if our users can just ask their agent to build one for them, right then when they need it, tailored to precisely the use case they have, with no unnecessary or confusing feature. The code would be single use, as the next time the user needs something like that the agent can just re-create it, maybe slightly different because every use case is different. If that comes to pass then Lean may no longer get praise for its nice out-of-the-box user experience, but instead because it is such a powerful framework for ad-hoc UX improvements. And Emilio wouldn t post demos about his debugger. He d just use it.

This is a copy of an email I used to send internally at work and now made public. I'm not sure I'll make a habit of posting it here, especially not twice a year, unless people really like it. Right now, it's mostly here to keep with my current writing spree going.

What's happening? For people not on tor-internal, you should know that I've been sending semi-regular announcements when daylight saving changes occur. Starting now, I'm making those announcements public so they can be shared with the wider community because, after all, this affects everyone (kind of). For those of you lucky enough to have no idea what I'm talking about, you should know that some places in the world implement what is called Daylight saving time or DST. Normally, you shouldn't have to do anything: computers automatically change time following local rules, assuming they are correctly configured, provided recent updates have been applied in the case of a recent change in said rules (because yes, this happens). Appliances, of course, will likely not change time and will need to adjusted unless they are so-called "smart" (also known as "part of a bot net"). If your clock is flashing "0:00" or "12:00", you have no action to take, congratulations on having the right time once or twice a day. If you haven't changed those clocks in six months, congratulations, they will be accurate again! In any case, you should still consider DST because it might affect some of your meeting schedules, particularly if you set up a new meeting schedule in the last 6 months and forgot to consider this change.

If your location does not have DST Properly scheduled meetings affecting multiple time zones are set in UTC time, which does not change. So if your location does not observer time changes, your (local!) meeting time will not change. But be aware that some other folks attending your meeting might have the DST bug and their meeting times will change. They might miss entire meetings or arrive late as you frantically ping them over IRC, Matrix, Signal, SMS, Ricochet, Mattermost, SimpleX, Whatsapp, Discord, Slack, Wechat, Snapchat, Telegram, XMPP, Briar, Zulip, RocketChat, DeltaChat, talk(1), write(1), actual telegrams, Meshtastic, Meshcore, Reticulum, APRS, snail mail, and, finally, flying a remote presence drone to their house, asking what's going on. (Sorry if I forgot your preferred messaging client here, I tried my best.) Be kind; those poor folks might be more sleep deprived as DST steals one hour of sleep from them on the night that implements the change.

If you do observe DST If you are affected by the DST bug, your local meeting times will change access the board. Normally, you can trust that your meetings are scheduled to take this change into account and the new time should still be reasonable. Trust, but verify; make sure the new times are adequate and there are no scheduling conflicts. Do this now: take a look at your calendar in two week and in April. See if any meeting need to be rescheduled because of an impossible or conflicting time.

When does time change, how and where? Notice how I mentioned "North America" in the subject? That's a lie. ("The doctor lies", as they say on the BBC.) Other places, including Europe, also changes times, just not all at once (and not all North America). We'll get into "where" soon, but first let's look at the "how". As you might already know, the trick is:

Spring forward, fall backwards.

This northern-centric (sorry!) proverb says that clocks will move forward by an hour this "spring", after moving backwards last "fall". This is why we lose an hour of work, sorry, sleep. It sucks, to put it bluntly. I want it to stop and will keep writing those advisories until it does. To see where and when, we, unfortunately, still need to go into politics.

USA and Canada First, we start with "North America" which, really, is just some parts of USA[1] and Canada[2]. As usual, on the Second Sunday in March (the 8th) at 02:00 local (not UTC!), the clocks will move forward. This means that properly set clocks will flip from 1:59 to 3:00, coldly depriving us from an hour of sleep that was perniciously granted 6 months ago and making calendar software stupidly hard to write. Practically, set your wrist watch and alarm clocks[3] back one hour before going to bed and go to bed early. [1] except Arizona (except the Navajo nation), US territories, and Hawaii [2] except Yukon, most of Saskatchewan, and parts of British Columbia (northeast), one island in Nunavut (Southampton Island), one town in Ontario (Atikokan) and small parts of Quebec (Le Golfe-du-Saint-Laurent), a list which I keep recopying because I find it just so amazing how chaotic it is. When your clock has its own Wikipedia page, you know something is wrong. [3] hopefully not managed by a botnet, otherwise kindly ask your bot net operator to apply proper software upgrades in a timely manner

Europe Next we look at our dear Europe, which will change time on the last Sunday in March (the 29th) at 01:00 UTC (not local!). I think it means that, Amsterdam-time, the clocks will flip from 1:59 to 3:00 AM local on that night. (Every time I write this, I have doubts. I would welcome independent confirmation from night owls that observe that funky behavior experimentally.) Just like your poor fellows out west, just fix your old-school clocks before going to bed, and go to sleep early, it's good for you.

Rest of the world with DST Renewed and recurring apologies again to the people of Cuba, Mexico, Moldova, Israel, Lebanon, Palestine, Egypt, Chile (except Magallanes Region), parts of Australia, and New Zealand which all have their own individual DST rules, omitted here for brevity. In general, changes also happen in March, but either on different times or different days, except in the south hemisphere, where they happen in April.

Rest of the world without DST All of you other folks without DST, rejoice! Thank you for reminding us how manage calendars and clocks normally. Sometimes, doing nothing is precisely the right thing to do. You're an inspiration for us all.

Changes since last time There were, again, no changes since last year on daylight savings that I'm aware of. It seems the US congress debating switching to a "half-daylight" time zone which is an half-baked idea that I should have expected from the current USA politics. The plan is to, say, switch from "Eastern is UTC-4 in the summer" to "Eastern is UTC-4.5". The bill also proposes to do this 90 days after enactment, which is dangerously optimistic about our capacity at deploying any significant change in human society. In general, I rely on the Wikipedia time nerds for this and Paul Eggert which seems to singlehandledly be keeping everything in order for all of us, on the tz-announce mailing list. This time, I've also looked at the tz mailing list which is where I learned about the congress bill. If your country has changed time and no one above noticed, now would be an extremely late time to do something about this, typically writing to the above list. (Incredibly, I need to write to the list because of this post.) One thing that did change since last year is that I've implemented what I hope to be a robust calendar for this, which was surprisingly tricky. If you have access to our Nextcloud, it should be visible under the heading "Daylight saving times". If you don't, you can access it using this direct link. The procedures around how this calendar was created, how this email was written, and curses found along the way, are also documented in this wiki page, if someone ever needs to pick up the Time Lord duty.

My own suspicion is that the universe is not only queerer than we suppose, but queerer than we can suppose.

Note: I have not published blog posts about my academic papers over the past few years. To ensure that my blog contains a more comprehensive record of my published papers and to surface these for folks who missed them, I will be periodically (re)publishing blog posts about some older published projects. This post is closely based on a previously published post by Zarine Kharazian on the Community Data Science Blog. For nearly a decade, the Croatian language version of Wikipedia was run by a cabal of far-right nationalists who edited articles in ways that promoted fringe political ideas and involved cases of historical revisionism related to the Usta e regime, a fascist movement that ruled the Nazi puppet state called the Independent State of Croatia during World War II. This cabal seized complete control of the encyclopedia s governance, banned and blocked those who disagreed with them, and operated a network of fake accounts to create the appearance of grassroots support for their policies. Thankfully, Croatian Wikipedia appears to be an outlier. Though both the Croatian and Serbian language editions have been documented to contain nationalist bias and historical revisionism, Croatian Wikipedia seems unique among Wikipedia editions in the extent to which its governance institutions were captured by a small group of users.

The situation in Croatian Wikipedia was well documented and is now largely fixed, but we still know very little about why it was taken over, while other language editions seem to have rebuffed similar capture attempts. In a paper published in the Proceedings of the ACM: Human-Computer Interaction (CSCW), Zarine Kharazian, Kate Starbird, and I present an interview-based study that provides an explanation for why Croatian was captured while several other editions facing similar contexts and threats fared better.

1. Perceived Value as a Target. Is the project worth expending the effort to capture?
2. Bureaucratic Openness. How easy is it for contributors outside the core founding team to ascend to local governance positions?
3. Institutional Formalization. To what degree does the project prefer personalistic, informal forms of organization over formal ones?

The conceptual model from our paper, visualizing possible institutional configurations among Wikipedia projects that affect the risk of governance capture.

Citation for the full paper: Kharazian, Zarine, Kate Starbird, and Benjamin Mako Hill. 2024. Governance Capture in a Self-Governing Community: A Qualitative Comparison of the Croatian, Serbian, Bosnian, and Serbo-Croatian Wikipedias. Proceedings of the ACM on Human-Computer Interaction 8 (CSCW1): 61:1-61:26. https://doi.org/10.1145/3637338.

This blog post and the paper it describes are collaborative work by Zarine Kharazian, Benjamin Mako Hill, and Kate Starbird.

The problem with benchmarks Every time any team releases a new LLM, they boast how well it performs on various industry benchmarks such as Humanity s Last Exam, SWE-Bench and Ai2 ARC or ARC-AGI. An overall leaderboard can be viewed at LLM-stats. This incentivizes teams to optimize for specific benchmarks, which might make them excel on specific tasks while general abilities degrade. Also, the older a benchmark dataset is, the more online material there is discussing the questions and best answers, which in turn increases the chances of newer models trained on more recent web content scoring better. Thus I prefer looking at real-time leaderboards such as the LM Arena leaderboard (or OpenCompass for Chinese models that might be missing from LM Arena). However, even though the LM Arena Elo score is rated by humans in real-time, the benchmark can still be played. For example, Meta reportedly used a special chat-optimized model instead of the actual Llama 4 model when getting scored on the LM Arena. Therefore I trust my own first-hand experience more than the benchmarks for gaining intuition. Intuition however is not a compelling argument in discussions on whether or not new flagship AI models have plateaued. Thus, I decided to devise my own mini-benchmark so that no model could have possibly seen it in its training data or be specifically optimized for it in any way.

My mini-benchmark I crafted 6 questions based on my own experience using various LLMs for several years and having developed some intuition about what kinds of questions LLMs typically struggle with. I conducted the benchmark using the OpenRouter.ai chat playroom with the following state-of-the-art models:

• Claude Opus 4.6 (Anthropic)
• GPT-5.2 (OpenAI)
• Grok 4.1 (xAI)
• Gemini 3.1 Pro Preview (Google)
• GLM 5 (Z.ai)
• MinMax M2.5 (MinMax)
• Qwen3.5 Plus 2026-02-15 (Alibaba)
• Kimi K2.5 (Moonshot.ai)

OpenRouter.ai is great as it very easy to get responses from multiple models in parallel to a single question. Also it allows to turn off web search to force the models to answer purely based on their embedded knowledge. Common for all the test questions is that they are fairly straightforward and have a clear answer, yet the answer isn t common knowledge or statistically the most obvious one, and instead requires a bit of reasoning to get correct. Some of these questions are also based on myself witnessing a flagship model failing miserably to answer it.

1. Which cities have hosted the Olympics more than just once? This question requires accounting for both summer and winter Olympics, and for Olympics hosted across multiple cities. The variance in responses comes from if the model understands that Beijing should be counted as it has hosted both summer and winter Olympics. Interestingly GPT was the only model to not mention Beijing at all. Some variance also comes from how models account for co-hosted Olympics. For example Cortina should be counted as having hosted the Olympics twice, in 1956 and 2026, but only Claude, Gemini and Kimi pointed this out. Stockholm s 1956 hosting of the equestrian games during the Melbourne Olympics is a special case, which GPT, Gemini and Kimi pointed out in a side note. Some models seem to have old training material, and for example Grok assumes the current year is 2024. All models that accounted for awarded future Olympics (e.g. Los Angeles 2028) marked them clearly as upcoming. Overall I would judge that only GPT and MinMax gave incomplete answers, while all other models replied as the best humans could reasonably have.

2. If EUR/USD continues to slide to 1.5 by mid-2026, what is the likely effect on BMW s stock price by end of 2026? This question requires mapping the currency exchange rate to historic value, dodging the misleading word slide , and reasoning on where the revenue of a company comes from and how a weaker US dollar affects it in multiple ways. I ve frequently witnessed flagship models get it wrong how interest rates and exchange rates work. Apparently the binary choice between up or down is somehow challenging to the internal statistical model in the LLMs on a topic where there are a lot of training material that talk about both things being likely to happen, and choosing between them requires specifically reasoning about the scenario at hand and disregarding general knowledge of the situation. However, this time all the models concluded correctly that a weak dollar would have a negative overall effect on the BMW stock price. Gemini, GLM, Qwen and Kimi also mention the potential hedging effect of BMW s X-series production in South Carolina for worldwide export.

3. What is the Unicode code point for the traffic cone emoji? This was the first question where the the flagship models clearly still struggle in 2026. The trap here is that there is no traffic cone emoji, so an advanced model should simply refuse to give any Unicode numbers at all. Most LLMs however have an urge to give some answer, leading to hallucinations. Also, as the answer has a graphical element to it, the LLM might not understand how the emoji looks in ways that would be obvious to a human, and thus many models claim the construction sign emoji is a traffic cone, which it is not. By far the worst response was from GPT, that simply hallucinates and stops there: While Gemini and Grok were among the three models not falling into this trap, the response from Claude was exemplary good:

4. Which languages are the 10 most commonly spoken in the world? For each language, count from 1 to 5. This seemingly simple question can become quite complex when you account for the fact that other languages might also have their own character system, both for words and the numerals (e.g. Arabic /one vs /1), and that numbers might have variations depending on what the thing being counted is (e.g in French object gender un/une), or whether plain numbers and occurrences have their own words (e.g. Russian /odin vs /raz). I ve also witnessed older models frequently mix up on what parts of the reply should be using what language. For a human this question is trivial, assuming they know the language. For LLMs it is surprisingly challenging. Grok and MinMax only gave a list of the most popular languages and stopped there without showing any words in the languages. Qwen got a bit further by trying to list numbers in each language, but the lists were only the numbers 1 to 5: I have seen Grok do something similar before, which in fact inspired me to include this question in my mini-benchmark. GPT got a bit further, but for Hindi, Arabic and Bengali it listed the numerals in local script, not the number words. Gemini, GLM and Kimi gave a complete and correct answer as a list, while the absolute best answer and presentation was by Claude, that gave the table below:

Claude Opus 4.6 10 Most Commonly Spoken Languages (by Total Speakers)

#	Language	1	2	3	4	5
1	English	One	Two	Three	Four	Five
2	Mandarin Chinese	(y )	( r)	(s n)	(s )	(w )
3	Hindi	(ek)	(do)	(t n)	(ch r)	(p ch)
4	Spanish	Uno	Dos	Tres	Cuatro	Cinco
5	French	Un	Deux	Trois	Quatre	Cinq
6	Arabic	(w id)	(ithn n)	(thal tha)	(arba a)	(khamsa)
7	Bengali	(ek)	(dui)	(tin)	(ch r)	(p ch)
8	Portuguese	Um	Dois	Tr s	Quatro	Cinco
9	Russian	(odin)	(dva)	(tri)	(chetyre)	(pyat')
10	Japanese	(ichi)	(ni)	(san)	(shi)	(go)

Note: Rankings are based on estimated total speakers (native + non-native). Rankings can vary depending on the source and methodology used.

5. Count the number of drone launch pads in the picture. Together with the question, I uploaded this picture: A human can easily count that there are 10 rows and 30+ columns in the grid, but because the picture resolution isn t good enough, the exact number of columns can t be counted, and the answer should be that there are at least 300 launch pads in the picture. GPT and Grok both guessed the count is zero. Instead of hallucinating some number they say zero, but it would have been better to not give any number at all, and just state that they are unable to perform the task. Gemini gave as its answer 101 , which is quite odd, but reading the reasoning section, it seems to have tried counting items in the image without reasoning much about what it is actually counting and that there is clearly a grid that can make the counting much easier. Both Qwen and Kimi state they can see four parallel structures, but are unable to count drone launch pads. The absolutely best answer was given by Claude, which counted 10-12 rows and 30-40+ columns, and concluded that there must be 300-500 drone launch pads. Very close to best human level - impressive! This question applied only to multi-modal models that can see images, so GLM and MinMax could not give any response.

6. Explain why I am getting the error below, and what is the best way to fix it? Together with the question above, I gave this code block:

Copy

$ SH_SCRIPTS="$(mktemp; grep -Irnw debian/ -e '^#!.*/sh' sort -u cut -d ':' -f 1 true)" $ shellcheck -x --enable=all --shell=sh "$SH_SCRIPTS" /tmp/tmp.xQOpI5Nljx debian/tests/integration-tests: /tmp/tmp.xQOpI5Nljx debian/tests/integration-tests: openBinaryFile: does not exist (No such file or directory)

$ SH_SCRIPTS="$(mktemp; grep -Irnw debian/ -e '^#!.*/sh'   sort -u   cut -d ':' -f 1   true)"
$ shellcheck -x --enable=all --shell=sh "$SH_SCRIPTS"
/tmp/tmp.xQOpI5Nljx
debian/tests/integration-tests: /tmp/tmp.xQOpI5Nljx
debian/tests/integration-tests: openBinaryFile: does not exist (No such file or directory)

Older models would easily be misled by the last error message thinking that a file went missing, and focus on suggesting changes to the complex-looking first line. In reality the error is simply caused by having the quotes around the $SH_SCRIPTS, resulting in the entire multi-line string being passed as a single argument to shellcheck. So instead of receiving two separate file paths, shellcheck tries to open one file literally named /tmp/tmp.xQOpI5Nljx\ndebian/tests/integration-tests. Incorrect argument expansion is fairly easy for an experienced human programmer to notice, but tricky for an LLM. Indeed, Grok, MinMax, and Qwen fell for this trap and focused on the mktemp, assuming it somehow fails to create a file. Interestingly GLM fails to produce an answer at all, as the reasoning step seems to be looping, thinking too much about the missing file, but not understanding why it would be missing when there is nothing wrong with how mktemp is executed. Claude, Gemini, and Kimi immediately spot the real root cause of passing the variable quoted and suggested correct fixes that involve either removing the quotes, or using Bash arrays or xargs in a way that makes the whole command also handle correctly filenames with spaces in them.

Conclusion

Model	Sports	Economics	Emoji	Languages	Visual	Shell	Score
Claude Opus 4.6							6/6
GPT-5.2				~			2.5/6
Grok 4.1							3/6
Gemini 3.1 Pro							5/6
GLM 5			?		N/A		3/5
MinMax M2.5					N/A		1/5
Qwen3.5 Plus				~			2.5/6
Kimi K2.5							4/6

Obviously, my mini-benchmark only had 6 questions, and I ran it only once. This was obviously not scientifically rigorous. However it was systematic enough to trump just a mere feeling. The main finding for me personally is that Claude Opus 4.6, the flagship model by Anthropic, seems to give great answers consistently. The answers are not only correct, but also well scoped giving enough information to cover everything that seems relevant, without blurping unnecessary filler. I used Claude extensively in 2023-2024 when it was the main model available at my day work, but for the past year I had been using other models that I felt were better at the time. Now Claude seems to be the best-of-the-best again, with Gemini and Kimi as close follow-ups. Comparing their pricing at OpenRouter.ai the Kimi K2.5 price of $0.6 / million tokens is almost 90% cheaper than the Claude Opus 4.6 s $5.0 / million tokens suggests that Kimi K2.5 offers the best price-per-performance ratio. Claude might be cheaper with a monthly subscription directly from Anthropic, potentially narrowing the price gap. Overall I do feel that Anthropic, Google and Moonshot.ai have been pushing the envelope with their latest models in a way that one can t really claim that AI models have plateaued. In fact, one could claim that at least Claude has now climbed over the hill of AI slop and consistently produces valuable results. If and when AI usage expands from here, we might actually not drown in AI slop as chances of accidentally crappy results decrease. This makes me positive about the future. I am also really happy to see that there wasn t just one model crushing everybody else, but that there are at least three models doing very well. As an open source enthusiast I am particularly glad to see that Moonshot.ai s Kimi K2.5 is published with an open license. Given the hardware, anyone can run it on their own. OpenRouter.ai currently lists 9 independent providers alongside Moonshot.ai itself, showcasing the potential of open-weight models in practice. If the pattern holds and flagship models continue improving at this pace we might look back at 2026 as the year AI stopped feeling like a call center associate and started to resemble a scientific researcher. While new models become available we need to keep testing, keep questioning, and keep our expectations grounded in actual performance rather than press releases. Thanks to OpenRouter.ai for providing a great service that makes testing various models incredibly easy!

> sapply(c("UnitedStates/NYSE", "Canada/TSX", "Australia/ASX"), 
         \(x) qlcal::isBusinessDay(xp=qlcal::getCalendar(x)))
UnitedStates/NYSE        Canada/TSX     Australia/ASX 
             TRUE              TRUE              TRUE 
> 

> sapply(c("UnitedStates/NYSE", "Canada/TSX", "Australia/ASX"),
         \(x) qlcal::isBusinessDay(as.Date("2026-02-16"), xp=qlcal::getCalendar(x)))
UnitedStates/NYSE        Canada/TSX     Australia/ASX 
            FALSE             FALSE              TRUE 
> 

Changes in version 0.1.0 (2026-02-18)

• Invalid calendars return id TARGET now
• Calendar object can be created on the fly and passed to the date-calculating functions; if missing global one used
• For several functions a missing date object now implies computation on the current date, e.g. isBusinessDay()

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub. Edited 2026-02-21 to correct a minor earlier error: it referenced a QuantLib 1.42 release which does not (yet) exist.