Way back (more than 10 years ago) when I was doing DVD-based backups, I knew that normal DVDs/Blu-Rays are no long-term archival solutions, and that if I was real about doing optical media backups, I need to switch to M-Disc. I actually bought a (small stack) of M-Disc Blu-Rays, but never used them. I then switched to other backups solutions, and forgot about the whole topic. Until, this week, while sorting stuff, I happened upon a set of DVD backups from a range of years, and was very curious whether they are still readable after many years. And, to my surprise, there were no surprises! Went backward in time, and:

• 2014, TDK DVD+R, fully readable
• 2012, JVC DVD+R and TDK DVD+R, fully readable
• 2010, Verbatim DVD+R, fully readable
• 2009/2008/2007, Verbatim DVD+R, 4 DVDs, fully readable

I also found stack of dual-layer DVD+R from 2012-2014, some for sure Verbatim, and some unmarked (they were intended to be printed on), but likely Verbatim as well. All worked just fine. Just that, even at ~8GiB per disk, backing up raw photo files took way too many disks, even in 2014 . At this point I was happy that all 12+ DVDs I found, ranging from 10 to 14 years, are all good. Then I found a batch of 3 CDs! Here the results were mixed:

• 2003: two TDK CD-R80 , Mettalic , 700MB: fully readable, after 21 years!
• unknown year, likely around 1999-2003, but no later, Creation CD-R, 700MB: read errors to the extent I can t even read the disk signature (isoinfo -d).

I think the takeaway is that for all explicitly selected media - TDK, JVC and Verbatim - they hold for 10-20 years. Valid reads from summer 2003 is mind boggling for me, for (IIRC) organic media - not sure about the TDK metallic substrate. And when you just pick whatever ( Creation ), well, the results are mixed. Note that in all this, it was about CDs and DVDs. I have no idea how Blu-Rays behave, since I don t think I ever wrote a Blu-Ray. In any case, surprising to me, and makes me rethink a bit my backup options. Sizes from 25 to 100GB Blu-Rays are reasonable for most critical data. And they re WORM, as opposed to most LTO media, which is re-writable (and to some small extent, prone to accidental wiping). Now, I should check those M-Disks to see if they can still be written to, after 10 years

RcppDate wraps the featureful date library written by Howard Hinnant for use with R. This header-only modern C++ library has been in pretty wide-spread use for a while now, and adds to C++11/C++14/C++17 what will be (with minor modifications) the date library in C++20. This release, the first in 3 1/2 years, syncs the code with the recent date 3.0.2 release from a few days ago. It also updates a few packaging details such as URLs, badges or continuous integration.

Changes in version 0.0.4 (2024-10-14)

• Updated to upstream version 3.0.2 (and adjusting one pragma)
• Several small updates to overall packaging and testing

Courtesy of my CRANberries, there is also a diffstat report for the most recent release. More information is available at the repository or the package page. If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Happy 28th Birthday KDE!

Sorry my blog updates have been MIA. Let me tell you a story As some of you know, 3 months ago I was in a no fault car accident. Thankfully, the only injury was I ended up with a broken arm. ER sends me home in a sling and tells me it was a clean break and it will mend itself in no time. After a week of excruciating pain I went to my follow up doctor appointment, and with my x-rays in hand, the doc tells me it was far from a clean break and needs surgery. So after a week of my shattered bone scraping my nerves and causing pain I have never felt before, I finally go in for surgery! They put in a metal plate with screws to hold the bone in place so it can properly heal. The nerve pain was gone, so I thought I was on the mend. Some time goes by and the swelling still has not subsided, the doctors are not as concerned about this as I am, so I carry on until it becomes really inflamed and developed fever blisters. After no success in reaching the doctors office my husband borrows the neighbors car and rushes me to the ER. Good thing too, I had an infection. So after a 5 day stay in the hospital, they sent us home loaded with antibiotics and trained my husband in wound packing. We did everything right, kept the place immaculate, followed orders with the wound care, took my antibiotics, yet when they ran out there was still no sign of relief, or healing. Went to doctors and they gave me another month supply of antibiotics. Two days after my final dose my arm becomes inflamed again and with extra spectacular levels of pain to go with it. I call the doctor office They said to come in on my appointment day ( 4 days away ). I asked, You aren t concerned with this inflammation? , to which they replied, No. . Ok, maybe I am over reacting and it s all in my head, I can power through 4 more days. The following morning my husband observed fever blisters and the wound site was clearly not right, so once again off we go to the ER. Well thankfully we did. I was in Sepsis and could have died After deliberating with the doctor on the course of action for treatment, the doctor accepted our plea to remove the plate, rather than tighten screws and have me drive 100 miles to hospital everyday for iv antibiotics (Umm I don t have a car!?) So after another 4 day stay I am released into the world, alive and well. I am happy to report, the swelling is almost gone, the pain is minimal, and I am finally healing nicely. I am still in a sling and I have to be super careful and my arm was not fully knitted. So with that I am bummed to say, no traveling for me, no Ubuntu Summit I still need help with that car, if it weren t for our neighbor, this story would have ended much differently. https://gofund.me/00942f47 Despite my tragic few months for my right arm, my left arm has been quite busy. Thankfully I am a lefty! On to my work progress report. Kubuntu:

Kubuntu 24.10 Oracular Oriole Released

With Plasma 6! A big thank you to the Debian KDE/QT team and Rik Mills, could not have done it without you!

KDE Snaps: All release service snaps are done! Save a few problematic ones still WIP.. I have released 24.08.2 which you can find here: https://snapcraft.io/publisher/kde I completed the qt6 and KDE frameworks 6 content packs for core24 Snapcraft: I have a PR in for kde-neon-6 extension core24 support. That s all for now. Thanks for stopping by!

When setting up your YubiKey you have the option to require the user to touch the device to authorize an operation (be it signing, decrypting, or authenticating). While web browsers often provide clear prompts for this, other applications like SSH or GPG will not. Instead the operation will just hang without any visual indication that user input is required. The YubiKey itself will blink, but depending on where it is plugged in that is not very visible.
yubikey-touch-detector (fresh in unstable) solves this issue by providing a way for your desktop environment to signal the user that the device is waiting for a touch. It provides an event feed on a socket that other components can consume. It comes with libnotify support and there are some custom integrations for other environments.For GNOME and KDE libnotify support should be sufficient, however you still need to turn it on:

$ mkdir -p ~/.config/yubikey-touch-detector
$ sed -e 's/^YUBIKEY_TOUCH_DETECTOR_LIBNOTIFY=.*/YUBIKEY_TOUCH_DETECTOR_LIBNOTIFY=true/' \
  < /usr/share/doc/yubikey-touch-detector/examples/service.conf.example \
  > ~/.config/yubikey-touch-detector/service.conf
$ systemctl --user restart yubikey-touch-detector

I would still have preferred a more visible, more modal prompt. I guess that would be an exercise for another time, listening to the socket and presenting a window. But for now, desktop notifications will do for me.PS: I have not managed to get SSH's no-touch-required to work with YubiKey 4, while it works just fine with a YubiKey 5.

A long time ago . A long time ago a computer was a woman (I think almost exclusively a women, not a man) who was employed to do a lot of repetitive mathematics typically for accounting and stock / order processing. Then along came Lyons, who deployed an artificial computer to perform the same task, only with fewer errors in less time. Modern day computing was born we had entered the age of the Digital Computer. These computers were large, consumed huge amounts of power but were precise, and gave repeatable, verifiable results. Over time the huge mainframe digital computers have shrunk in size, increased in performance, and consume far less power so much so that they often didn t need the specialist CFC based, refrigerated liquid cooling systems of their bigger mainframe counterparts, only requiring forced air flow, and occasionally just convection cooling. They shrank so far and became cheep enough that the Personal Computer became to be, replacing the mainframe with its time shared resources with a machine per user. Desktop or even portable laptop computers were everywhere. We networked them together, so now we can share information around the office, a few computers were given specialist tasks of being available all the time so we could share documents, or host databases these servers were basically PCs designed to operate 24 7, usually more powerful than their desktop counterparts (or at least with faster storage and networking). Next we joined these networks together and the internet was born. The dream of a paperless office might actually become realised we can now send email (and documents) from one organisation (or individual) to another via email. We can make our specialist computers applications available outside just the office and web servers / web apps come of age. Fast forward a few years and all of a sudden we need huge data-halls filled with Rack scale machines augmented with exotic GPUs and NPUs again with refrigerated liquid cooling, all to do the same task that we were doing previously without the magical buzzword that has been named AI; because we all need another dot com bubble or block chain band waggon to jump aboard. Our AI enabled searches take slightly longer, consume magnitudes more power, and best of all the results we are given may or may not be correct . Progress, less precise answers, taking longer, consuming more power, without any verification and often giving a different result if you repeat your question AND we still need a personal computing device to access this wondrous thing. Remind me again why we are here? (time lines and huge swaves of history simply ignored to make an attempted comic point this is intended to make a point and not be scholarly work)

Wikimedia Commons, the Wikimedia project for freely licensed media files, also contains a bunch of photos by me and photos of me at various events. While I don't think Commons is going away anytime soon, I would still like to have a local copy of those images available on my own storage hardware. Obviously this requires some way to query for photos you want to download. I'm using Commons categories for this, since that's easy to implement and works for both use cases. The Commons community tends to come up with very specific categories that you can use, and if not, you can usually categorize the files yourself. There is almost an existing tool for this: Sam Wilson's mwcli project has support for exporting images one has uploaded to Commons. However I couldn't use that to upload photos of me others have uploaded, plus it's written in PHP and I don't exactly want to deal with the problem of figuring out how to package it in a way I could neatly install it on my NAS. So I wrote my own tool for it, called comload. It's written in Python because Python is easy to deploy (I can just throw it in a .deb and upload it to my internal repository), and because I did not find a Go library to handle Action API pagination for me. The basic usage is like this: This will download any files in Category:Taavi V n nen and its sub-categories to the current directory. Former image versions, as well as the image description and SDC data, if any, is also included. And it's smart enough to not download any files that are already there on future runs, so you can just throw it in a systemd timer to get any future files. I'd still like it to handle moved files without creating a duplicate copy, but otherwise I'm really happy with the current state. comload is available from PyPI and from my Git server directly, and is licensed under the GPLv3.

I've been exploring typesetting and formatting code within text documents such as papers, or my thesis. Up until now, I've been using the listings package without thinking much about it. By default, some sample Haskell code processed by listings looks like this (click any of the images to see larger, non-blurry versions): It's formatted with a monospaced font, with some keywords highlighted, but not syntactic symbols. There are several other options for typesetting and formatting code in LaTeX documents. For Haskell in particular, there is the preprocessor lhs2tex, The default output of which looks like this: A proportional font, but it's taken pains to preserve vertical alignment, which is syntactically significant for Haskell. It looks a little cluttered to me, and I'm not a fan of nearly everything being italic. Again, symbols aren't differentiated, but it has substituted them for more typographically pleasing alternatives: -> has become , and \ is now . Another option is perhaps the newest, the LaTeX package minted, which leverages the Python Pygments program. Here's the same code again. It defaults to monospace (the choice of font seems a lot clearer to me than the default for listings), no symbolic substitution, and liberal use of colour: An informal survey of the samples so far showed that the minted output was the most popular. All of these packages can be configured to varying degrees. Here are some examples of what I've achieved with a bit of tweaking All of this has got me wondering whether there are straightforward empirical answers to some of these questions of style. Firstly, I'm pretty convinced that symbolic substitution is valuable. When writing Haskell, we write ->, \, /= etc. not because it's most legible, but because it's most practical to type those symbols on the most widely available keyboards and popular keyboard layouts.¹ Of the three options listed here, symbolic substitution is possible with listings and lhs2tex, but I haven't figured out if minted can do it (which is really the question: can pygments do it?) I'm unsure about proportional versus monospaced fonts. We typically use monospaced fonts for editing computer code, but that's at least partly for historical reasons. Vertical alignment is often very important in source code, and it can be easily achieved with monospaced text; it's also sometimes important to have individual characters (., etc.) not be de-emphasised by being smaller than any other character. lhs2tex, at least, addresses vertical alignment whilst using proportional fonts. I guess the importance of identifying individual significant characters is just as true in a code sample within a larger document as it is within plain source code. From a (brief) scan of research on this topic, it seems that proportional fonts result in marginally quicker reading times for regular prose. It's not clear whether those results carry over into reading computer code in particular, and the margin is slim in any case. The drawbacks of monospaced text mostly apply when the volume of text is large, which is not the case for the short code snippets I am working with. I still have a few open questions:

• Is colour useful for formatting code in a PDF document?
  • does this open up a can of accessibility worms?
• What should be emphasised (or de-emphasised)
• Why is the minted output most popular: Could the choice of font be key? Aspects of the font other than proportionality (serifs? Size of serifs? etc)

It's been a while since I've posted about arm64 hardware. The last machine I spent my own money on was a SolidRun Macchiatobin, about 7 years ago. It's a small (mini-ITX) board with a 4-core arm64 SoC (4 * Cortex-A72) on it, along with things like a DIMM socket for memory, lots of networking, 3 SATA disk interfaces. The Macchiatobin was a nice machine compared to many earlier systems, but it took quite a bit of effort to get it working to my liking. I replaced the on-board U-Boot firmware binary with an EDK2 build, and that helped. After a few iterations we got a new build including graphical output on a PCIe graphics card. Now it worked much more like a "normal" x86 computer. I still have that machine running at home, and it's been a reasonably reliable little build machine for arm development and testing. It's starting to show its age, though - the onboard USB ports no longer work, and so it's no longer useful for doing things like installation testing. :-/ So... I was involved in a conversation in the #debian-arm IRC channel a few weeks ago, and diederik suggested the Radxa Rock 5 ITX. It's another mini-ITX board, this time using a Rockchip RK3588 CPU. Things have moved on - the CPU is now an 8-core big.LITTLE config: 4*Cortex A76 and 4*Cortex A55. The board has NVMe on-board, 4*SATA, built-in Mali graphics from the CPU, soldered-on memory. Just about everything you need on an SBC for a small low-power desktop, a NAS or whatever. And for about half the price I paid for the Macchiatobin. I hit "buy" on one of the listed websites. :-) A few days ago, the new board landed. I picked the version with 24GB of RAM and bought the matching heatsink and fan. I set it up in an existing case borrowed from another old machine and tried the Radxa "Debian" build. All looked OK, but I clearly wasn't going to stay with that. Onwards to running a native Debian setup! I installed an EDK2 build from https://github.com/edk2-porting/edk2-rk3588 onto the onboard SPI flash, then rebooted with a Debian 12.7 (Bookworm) arm64 installer image on a USB stick. How much trouble could this be? I was shocked! It Just Worked (TM) I'm running a standard Debian arm64 system. The graphical installer ran just fine. I installed onto the NVMe, adding an Xfce desktop for some simple tests. Everything Just Worked. After many years of fighting with a range of different arm machines (from simple SBCs to desktops and servers), this was without doubt the most straightforward setup I've ever done. Wow! It's possible to go and spend a lot of money on an Ampere machine, and I've seen them work well too. But for a hobbyist user (or even a smaller business), the Rock 5 ITX is a lovely option. Total cost to me for the board with shipping fees, import duty, etc. was just over 240. That's great value, and I can wholeheartedly recommend this board! The two things that are missing compared to the Macchiatobin? This is soldered-on memory (but hey, 24G is plenty for me!) It also doesn't have a PCIe slot, but it has sufficient onboard network, video and storage interfaces that I think it will cover most people's needs. Where's the catch? It seems these are very popular right now, so it can be difficult to find these machines in stock online. FTAOD, I should also point out: I bought this machine entirely with my own money, for my own use for development and testing. I've had no contact with the Radxa or Rockchip folks at all here, I'm just so happy with this machine that I've felt the need to shout about it! :-) Here's some pictures...

The diffoscope maintainers are pleased to announce the release of diffoscope version 280. This version includes the following changes: You find out more by visiting the project homepage.

Like each month, have a look at the work funded by Freexian s Debian LTS offering.

Debian LTS contributors In September, 18 contributors have been paid to work on Debian LTS, their reports are available:

• Abhijith PA did 7.0h (out of 0.0h assigned and 14.0h from previous period), thus carrying over 7.0h to the next month.
• Adrian Bunk did 51.75h (out of 9.25h assigned and 55.5h from previous period), thus carrying over 13.0h to the next month.
• Arturo Borrero Gonzalez did 10.0h (out of 0.0h assigned and 10.0h from previous period).
• Bastien Roucari s did 20.0h (out of 20.0h assigned).
• Ben Hutchings did 20.0h (out of 12.0h assigned and 12.0h from previous period), thus carrying over 4.0h to the next month.
• Chris Lamb did 18.0h (out of 18.0h assigned).
• Daniel Leidert did 23.0h (out of 26.0h assigned), thus carrying over 3.0h to the next month.
• Emilio Pozuelo Monfort did 23.5h (out of 22.25h assigned and 37.75h from previous period), thus carrying over 36.5h to the next month.
• Guilhem Moulin did 22.25h (out of 20.0h assigned and 2.5h from previous period), thus carrying over 0.25h to the next month.
• Lucas Kanashiro did 10.0h (out of 5.0h assigned and 15.0h from previous period), thus carrying over 10.0h to the next month.
• Markus Koschany did 40.0h (out of 40.0h assigned).
• Ola Lundqvist did 6.5h (out of 14.5h assigned and 9.5h from previous period), thus carrying over 17.5h to the next month.
• Roberto C. S nchez did 24.75h (out of 21.0h assigned and 3.75h from previous period).
• Santiago Ruano Rinc n did 19.0h (out of 19.0h assigned).
• Sean Whitton did 0.75h (out of 4.0h assigned and 2.0h from previous period), thus carrying over 5.25h to the next month.
• Sylvain Beucler did 16.0h (out of 42.0h assigned and 18.0h from previous period), thus carrying over 44.0h to the next month.
• Thorsten Alteholz did 11.0h (out of 11.0h assigned).
• Tobias Frost did 17.0h (out of 7.5h assigned and 9.5h from previous period).

Evolution of the situation In September, we have released 52 DLAs. September marked the first full month of Debian 11 bullseye under the responsibility of the LTS Team and the team immediately got to work, publishing more than 4 dozen updates. Some notable updates include ruby2.7 (denial-of-service, information leak, and remote code execution), git (various arbitrary code execution vulnerabilities), firefox-esr (multiple issues), gnutls28 (information disclosure), thunderbird (multiple issues), cacti (cross site scripting and SQL injection), redis (unauthorized access, denial of service, and remote code execution), mariadb-10.5 (arbitrary code execution), cups (arbitrary code execution). Several LTS contributors have also contributed package updates which either resulted in a DSA (a Debian Security Announcement, which applies to Debian 12 bookworm) or in an upload that will be published at the next stable point release of Debian 12 bookworm. This list of packages includes cups, cups-filters, booth, nghttp2, puredata, python3.11, sqlite3, and wireshark. This sort of work, contributing fixes to newer Debian releases (and sometimes even to unstable), helps to ensure that upgrades from a release in the LTS phase of its lifecycle to a newer release do not expose users to vulnerabilities which have been closed in the older release. Looking beyond Debian, LTS contributor Bastien Roucari s has worked with the upstream developers of apache2 to address regressions introduced upstream by some recent vulnerability fixes and he has also reached out to the community regarding a newly discovered security issue in the dompurify package. LTS contributor Santiago Ruano Rinc n has undertaken the work of triaging and reproducing nearly 4 dozen CVEs potentially affecting the freeimage package. The upstream development of freeimage appears to be dormant and some of the issues have languished for more than 5 years. It is unclear how much can be done without the aid of upstream, but we will do our best to provide as much help to the community as we can feasibly manage. Finally, it is sometimes necessary to limit or discontinue support for certain packages. The transition of a release from being under the responsibility of the Debian Security Team to that of the LTS Team is an occasion where we assess any pending decisions in this area and formalize them. Please see the announcement for a complete list of packages which have been designated as unsupported.

Thanks to our sponsors Sponsors that joined recently are in bold.

• Platinum sponsors:
  • TOSHIBA (for 108 months)
  • Civil Infrastructure Platform (CIP) (for 76 months)
  • VyOS Inc (for 40 months)
• Gold sponsors:
  • Roche Diagnostics International AG (for 118 months)
  • Akamai - Linode (for 112 months)
  • Babiel GmbH (for 102 months)
  • Plat Home (for 101 months)
  • CINECA (for 76 months)
  • University of Oxford (for 58 months)
  • Deveryware (for 45 months)
  • EDF SA (for 30 months)
  • Dataport A R (for 5 months)
  • CERN (for 3 months)
• Silver sponsors:
  • Domeneshop AS (for 123 months)
  • Nantes M tropole (for 117 months)
  • Univention GmbH (for 109 months)
  • Universit Jean Monnet de St Etienne (for 109 months)
  • Ribbon Communications, Inc. (for 103 months)
  • Exonet B.V. (for 92 months)
  • Leibniz Rechenzentrum (for 87 months)
  • Minist re de l Europe et des Affaires trang res (for 70 months)
  • Cloudways by DigitalOcean (for 60 months)
  • Dinahosting SL (for 58 months)
  • Bauer Xcel Media Deutschland KG (for 52 months)
  • Platform.sh SAS (for 52 months)
  • Moxa Inc. (for 46 months)
  • sipgate GmbH (for 44 months)
  • OVH US LLC (for 42 months)
  • Tilburg University (for 42 months)
  • GSI Helmholtzzentrum f r Schwerionenforschung GmbH (for 33 months)
  • Soliton Systems K.K. (for 30 months)
  • THINline s.r.o. (for 6 months)
  • Copenhagen Airports A/S
• Bronze sponsors:
  • Evolix (for 123 months)
  • Seznam.cz, a.s. (for 123 months)
  • Intevation GmbH (for 120 months)
  • Linuxhotel GmbH (for 120 months)
  • Daevel SARL (for 119 months)
  • Bitfolk LTD (for 118 months)
  • Megaspace Internet Services GmbH (for 118 months)
  • Greenbone AG (for 117 months)
  • NUMLOG (for 117 months)
  • WinGo AG (for 116 months)
  • Entr ouvert (for 107 months)
  • Adfinis AG (for 105 months)
  • Tesorion (for 100 months)
  • GNI MEDIA (for 99 months)
  • Laboratoire LEGI - UMR 5519 / CNRS (for 99 months)
  • Bearstech (for 91 months)
  • LiHAS (for 91 months)
  • Catalyst IT Ltd (for 86 months)
  • Supagro (for 81 months)
  • Demarcq SAS (for 80 months)
  • Universit Grenoble Alpes (for 66 months)
  • TouchWeb SAS (for 58 months)
  • SPiN AG (for 55 months)
  • CoreFiling (for 51 months)
  • Institut des sciences cognitives Marc Jeannerod (for 46 months)
  • Observatoire des Sciences de l Univers de Grenoble (for 42 months)
  • Tem Innovations GmbH (for 37 months)
  • WordFinder.pro (for 36 months)
  • CNRS DT INSU R sif (for 35 months)
  • Alter Way (for 28 months)
  • Institut Camille Jordan (for 18 months)
  • SOBIS Software GmbH (for 3 months)

As DebConf22 was coming to an end, in Kosovo, talking with Eeveelweezel they invited me to prepare a talk to give for the Chicago Python User Group. I replied that I m not really that much of a Python guy But would think about a topic. Two years passed. I meet Eeveelweezel again for DebConf24 in Busan, South Korea. And the topic came up again. I had thought of some ideas, but none really pleased me. Again, I do write some Python when needed, and I teach using Python, as it s the language I find my students can best cope with. But delivering a talk to ChiPy? On the other hand, I have long used a very simplistic and limited filesystem I ve designed as an implementation project at class: FIUnamFS (for Facultad de Ingenier a, Universidad Nacional Aut noma de M xico : the Engineering Faculty for Mexico s National University, where I teach. Sorry, the link is in Spanish but you will find several implementations of it from the students ). It is a toy filesystem, with as many bad characteristics you can think of, but easy to specify and implement. It is based on contiguous file allocation, has no support for sub-directories, and is often limited to the size of a 1.44MB floppy disk. As I give this filesystem as a project to my students (and not as a mere homework), I always ask them to try and provide a good, polished, professional interface, not just the simplistic menu I often get. And I tell them the best possible interface would be if they provide support for FIUnamFS transparently, usable by the user without thinking too much about it. With high probability, that would mean: Use FUSE. But, in the six semesters I ve used this project (with 30-40 students per semester group), only one student has bitten the bullet and presented a FUSE implementation. Maybe this is because it s not easy to understand how to build a FUSE-based filesystem from a high-level language such as Python? Yes, I ve seen several implementation examples and even nice web pages (i.e. the examples shipped with thepython-fuse module Stavros passthrough filesystem, Dave Filesystem based upon, and further explaining, Stavros , and several others) explaining how to provide basic functionality. I found a particularly useful presentation by Matteo Bertozzi presented ~15 years ago at PyCon4 But none of those is IMO followable enough by itself. Also, most of them are very old (maybe the world is telling me something that I refuse to understand?). And of course, there isn t a single interface to work from. In Python only, we can find python-fuse, Pyfuse, Fusepy Where to start from? So I setup to try and help. Over the past couple of weeks, I have been slowly working on my own version, and presenting it as a progressive set of tasks, adding filesystem calls, and being careful to thoroughly document what I write (but maybe my documentation ends up obfuscating the intent? I hope not and, read on, I ve provided some remediation). I registered a GitLab project for a hand-holding guide to writing FUSE-based filesystems in Python. This is a project where I present several working FUSE filesystem implementations, some of them RAM-based, some passthrough-based, and I intend to add to this also filesystems backed on pseudo-block-devices (for implementations such as my FIUnamFS). So far, I have added five stepwise pieces, starting from the barest possible empty filesystem, and adding system calls (and functionality) until (so far) either a read-write filesystem in RAM with basicstat() support or a read-only passthrough filesystem. I think providing fun or useful examples is also a good way to get students to use what I m teaching, so I ve added some ideas I ve had: DNS Filesystem, on-the-fly markdown compiling filesystem, unzip filesystem and uncomment filesystem. They all provide something that could be seen as useful, in a way that s easy to teach, in just some tens of lines. And, in case my comments/documentation are too long to read, uncommentfs will happily strip all comments and whitespace automatically! So I will be delivering my talk tomorrow (2024.10.10, 18:30 GMT-6) at ChiPy (virtually). I am also presenting this talk virtually at Jornadas Regionales de Software Libre in Santa Fe, Argentina, next week (virtually as well). And also in November, in person, at nerdear.la, that will be held in Mexico City for the first time. Of course, I will also share this project with my students in the next couple of weeks And hope it manages to lure them into implementing FUSE in Python. At some point, I shall report! Update: After delivering my ChiPy talk, I have uploaded it to YouTube: A hand-holding guide to writing FUSE-based filesystems in Python, and after presenting at Jornadas Regionales, I present you the video in Spanish here: Aprendiendo y ense ando a escribir sistemas de archivo en espacio de usuario con FUSE y Python.

Debian Contributions: 2024-09 Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

Pydantic v2, by Colin Watson Pydantic is a useful library for validating data in Python using type hints: Freexian uses it in a number of projects, including Debusine. Its Debian packaging had been stalled at 1.10.17 in testing for some time, partly due to needing to make sure everything else could cope with the breaking changes introduced in 2.x, but mostly due to needing to sort out packaging of its new Rust dependencies. Several other people (notably Alexandre Detiste, Andreas Tille, Drew Parsons, and Timo R hling) had made some good progress on this, but nobody had quite got it over the line and it seemed a bit stuck. Colin upgraded a few Rust libraries to new upstream versions, packaged rust-jiter, and chased various failures in other packages. This eventually allowed getting current versions of both pydantic-core and pydantic into testing. It should now be much easier for us to stay up to date routinely.

Reworking of glib2.0 for cross bootstrap, by Helmut Grohne Simon McVittie (not affiliated with Freexian) earlier restructured the libglib2.0-dev such that it would absorb more functionality and in particular provide tools for working with .gir files. Those tools practically require being run for their host architecture (practically this means running under qemu-user) which is at odds with the requirements of architecture cross bootstrap. The qemu requirement was expressed in package dependencies and also made people unhappy attempting to use libglib2.0-dev for i386 on amd64 without resorting to qemu. The use of qemu in architecture bootstrap is particularly problematic as it tends to not be ready at the time bootstrapping is needed. As a result, Simon proposed and implemented the introduction of a libgio-2.0-dev package providing a subset of libglib2.0-dev that does not require qemu. Packages should continue to use libglib2.0-dev in their Build-Depends unless involved in architecture bootstrap. Helmut reviewed and tested the implementation and integrated the necessary changes into rebootstrap. He also prepared a patch for libverto to use the new package and proposed adding forward compatibility to glib2.0. Helmut continued working on adding cross-exe-wrapper to architecture-properties and implemented autopkgtests later improved by Simon. The cross-exe-wrapper package now provides a generic mechanism to a program on a different architecture by using qemu when needed only. For instance, a dependency on cross-exe-wrapper:i386 provides a i686-linux-gnu-cross-exe-wrapper program that can be used to wrap an ELF executable for the i386 architecture. When installed on amd64 or i386 it will skip installing or running qemu, but for other architectures qemu will be used automatically. This facility can be used to support cross building with targeted use of qemu in cases where running host code is unavoidable as is the case for GObject introspection. This concludes the joint work with Simon and Niels Thykier on glib2.0 and architecture-properties resolving known architecture bootstrap regressions arising from the glib2.0 refactoring earlier this year.

Analyzing binary package metadata, by Helmut Grohne As Guillem Jover (not affiliated with Freexian) continues to work on adding metadata tracking to dpkg, the question arises how this affects existing packages. The dedup.debian.net infrastructure provides an easy playground to answer such questions, so Helmut gathered file metadata from all binary packages in unstable and performed an explorative analysis. Some results include:

• binutils-mipsen uses wrong file ownership
• /usr-merge is not the only cause for aliasing problems in Debian.
  • mediawiki vs vistrails
  • bmake vs ros-mk
  • golang-github-eapache-go-resiliency-dev vs golang-gopkg-eapache-go-resiliency.v1-dev
  • node-markdown-it vs libjs-markdown-it
  • dokuwiki vs php-phpseclib
  • dokuwiki vs php-random-compat

Guillem also performed a cursory analysis and reported other problem categories such as mismatching directory permissions for directories installed by multiple packages and thus gained a better understanding of what consistency checks dpkg can enforce.

Python archive rebuilds, by Stefano Rivera Last month Stefano started to write some tooling to do large-scale rebuilds in debusine, starting with finding packages that had already started to fail to build from source (FTBFS) due to the removal of setup.py test. This month, Stefano did some more rebuilds, starting with experimental versions of dh-python. During the Python 3.12 transition, we had added a dependency on python3-setuptools to dh-python, to ease the transition. Python 3.12 removed distutils from the stdlib, but many packages were expecting it to still be available. Setuptools contains a version of distutils, and dh-python was a convenient place to depend on setuptools for most package builds. This dependency was never meant to be permanent. A rebuild without it resulted in mass-filing about 340 bugs (and around 80 more by mistake). A new feature in Python 3.12, was to have unittest s test runner exit with a non-zero return code, if no tests were run. We added this feature, to be able to detect tests that are not being discovered, by mistake. We are ignoring this failure, as we wouldn t want to suddenly cause hundreds of packages to fail to build, if they have no tests. Stefano did a rebuild to see how many packages were affected, and found that around 1000 were. The Debian Python community has not come to a conclusion on how to move forward with this. As soon as Python 3.13 release candidate 2 was available, Stefano did a rebuild of the Python packages in the archive against it. This was a more complex rebuild than the others, as it had to be done in stages. Many packages need other Python packages at build time, typically to run tests. So transitions like this involve some manual bootstrapping, followed by several rounds of builds. Not all packages could be tested, as not all their dependencies support 3.13 yet. The result was around 100 bugs in packages that need work to support Python 3.13. Many other packages will need additional work to properly support Python 3.13, but being able to build (and run tests) is an important first step.

Miscellaneous contributions

• Carles prepared the update of python-pyaarlo package to a new upstream release.
• Carles worked on updating python-ring-doorbell to a new upstream release. Unfinished, pending to package a new dependency python3-firebase-messaging RFP #1082958 and its dependency python3-http-ece RFP #1083020.
• Carles improved po-debconf-manager. Main new feature is that it can open Salsa merge requests. Aiming for a lightning talk in MiniDebConf Toulouse (November) to be functional end to end and get feedback from the wider public for this proof of concept.
• Carles helped one translator to use po-debconf-manager (added compatibility for bullseye, fixed other issues) and reviewed 17 package templates.
• Colin upgraded the OpenSSH packaging to 9.9p1.
• Colin upgraded the various YubiHSM packages to new upstream versions, enabled more tests, fixed yubihsm-shell build failures on some 32-bit architectures, made yubihsm-shell build reproducibly, and fixed yubihsm-connector to apply udev rules to existing devices when the package is installed. As usual, bookworm-backports is up to date with all these changes.
• Colin fixed quite a bit of fallout from setuptools 72.0.0 removing setup.py test, backported a large upstream patch set to make buildbot work with SQLAlchemy 2.0, and upgraded 25 other Python packages to new upstream versions.
• Enrico worked with Jakob Haufe to get him up to speed for managing sso.debian.org
• Rapha l did remove spam entries in the list of teams on tracker.debian.org (see #1080446), and he applied a few external contributions, fixing a rendering issue and replacing the DDPO link with a more useful alternative. He also gave feedback on a couple of merge requests that required more work. As part of the analysis of the underlying problem, he suggested to the ftpmasters (via #1083068) to auto-reject packages having the too-many-contacts lintian error, and he raised the severity of #1076048 to serious to actually have that 4 year old bug fixed.
• Rapha l uploaded zim and hamster-time-tracker to fix issues with Python 3.12 getting rid of setuptools. He also uploaded a new gnome-shell-extension-hamster to cope with the upcoming transition to GNOME 47.
• Helmut sent seven patches and sponsored one upload for cross build failures.
• Helmut uploaded a Nagios/Icinga plugin check-smart-attributes for monitoring the health of physical disks.
• Helmut collaborated on sbuild reviewing and improving a MR for refactoring the unshare backend.
• Helmut sent a patch fixing coinstallability of gcc-defaults.
• Helmut continued to monitor the evolution of the /usr-move. With more and more key packages such as libvirt or fuse3 fixed. We re moving into the boring long-tail of the transition.
• Helmut proposed updating the meson buildsystem in debhelper to use env2mfile.
• Helmut continued to update patches maintained in rebootstrap. Due to the work on glib2.0 above, rebootstrap moves a lot further, but still fails for any architecture.
• Santiago reviewed some Merge Request in Salsa CI, such as: !478, proposed by Otto to extend the information about how to use additional runners in the pipeline and !518, proposed by Ahmed to add support for Ubuntu images, that will help to test how some debian packages, including the complex MariaDB are built on Ubuntu. Santiago also prepared !545, which will make the reprotest job more consistent with the result seen on reproducible-builds.
• Santiago worked on different tasks related to DebConf 25. Especially he drafted the fundraising brochure (which is almost ready).
• Thorsten Alteholz uploaded package libcupsfilter to fix the autopkgtest and a dependency problem of this package. After package splix was abandoned by upstream and OpenPrinting.org adopted its maintenance, Thorsten uploaded their first release.
• Anupa published posts on the Debian Administrators group in LinkedIn and moderated the group, one of the tasks of the Debian Publicity Team.
• Anupa helped organize DebUtsav 2024. It had over 100 attendees with hand-on sessions on making initial contributions to Linux Kernel, Debian packaging, submitting documentation to Debian wiki and assisting Debian Installations.

• I continued to attend and soemtimes chair Debian kernel team meetings
• For the Debian firmware-nonfree package:
  • I opened the MRs:
    • !105: nvidia-graphics: Add versioned Breaks on initramfs-tools to avoid bug #1076539
    • !107: Update to 20240909; improve orig tarball generation
  • I reviewed the MRs:
    • !79: Improve maintainer scripts of firmware-nonfree
    • !102: Backport firmware-nonfree 20240709-2~bpo12+1 to Bookworm (merged)
    • !103: Add missing symlinks for tas2563 and tas2781 (merged)
    • !106: Update to 20240811 (merged)
  • I merged my own MRs:
    • !104: Update to linux-support-6.10.6
    • !105: nvidia-graphics: Add versioned Breaks on initramfs-tools to avoid bug #1076539
  • I uploaded:
    • version 20240709-2 to unstable
    • version 20240709-2~bpo12+1 to bookworm-backports
  • I respnded to bug reports:
    • #1080492: firmware-nonfree: [i915] With 20240709-2, the external monitors randomly blank for 2-3 seconds (regression) (closed)
• For the kernel-team repository:
  • I reviewed and merged the MR !5: git-format-patch-for-debian: Support for sourcehut
  • I deleted some obsolete text from the kernel upload checklist
• For the Debian linux package:
  • I announced the end of i386 kernel packages
  • I opened or updated the MRs:
    • !741: Fix most reproducibility issues
    • !1182: [x86] ACPI: Enable ACPI_EC_DEBUGFS as module (Closes: #980555)
    • !1188: [arm64] udeb: fix duplicated modules
    • !1199: [i386] Stop building kernel packages
    • !1200: Backport CI support to bullseye
    • !1209: Build fixes for rtla
    • !1213: Build fixes for objtool and rtla
    • !1215: d/rules.real: Try harder to set the locale to C.UTF-8
    • !1218: Clean up udeb configuration
    • !1219: Remove d/b/genorig.py in favour of uscan
    • !1220: [arm64,armhf] udeb: Add all watchdog drivers to kernel-image
  • I reviewed the MRs:
    • !675: [arm64] drivers/usb/host: Enable USB_XHCI_PCI_RENESAS as module (Closes: #1032671)
    • !1732: [x86] linux-cpupower: Add intel-speed-select command (rebased and merged)
    • !1038: debian/rules.real: export LANG = C.UTF-8 for sphinx (closed)
    • !1041: Add -b flag to genorig.py (closed)
    • !1065: Enable UBSAN_BOUNDS and UBSAN_SHIFT (closed)
    • !1166: [amd64] Enable CRYPTO_DEV_IAA_CRYPTO (merged)
    • !1169: [arm64] Add additional kernel with 64k page size
    • !1172: debian/config: add DAMON support for debian kernel (merged)
    • !1177: powerpc: Explicitly disable CRASH_DUMP on 32-bit (merged)
    • !1178: mm: set CONFIG_ZONE_DEVICE=y on most 64-bit architectures, not only amd64 (merged)
    • !1185: riscv64: fix module duplication detected by kernel-wedge starting with version 2.106 (merged)
    • !1186: [arm64] udeb: fix duplicated modules (merged)
    • !1192: [loong64] enable LBT, KVM and para-virt support (merged)
    • !1193: [arm64] Update rk3588 platform support (merged)
    • !1195: [arm64] enable Qualcomm X Elite support (merged)
    • !1196: [arm64] include modules for Rockchip RK3588 (closed)
    • !1198: Revert Make linux-libc-dev provide all cross packages (merged)
    • !1204: [arm64] Enable drivers for K3-AM642 SoC on SolidRun HummingBoard-T
    • !1207: [rt] Update to 6.11-rc5-rt5 (merged)
    • !1208: Support some Wi-Fi 7 devices (merged)
    • !1211: Set CONFIG_I2C=y on alpha and sparc64 (closed)
    • !1214: [loong64] Enable USB EHCI and OHCI host support (merged)
  • I merged my own MRs:
    • !741: Fix most reproducibility issues
    • !1182: [x86] ACPI: Enable ACPI_EC_DEBUGFS as module (Closes: #980555)
    • !1188: [arm64] udeb: fix duplicated modules
    • !1199: [i386] Stop building kernel packages
    • !1200: Backport CI support to bullseye
    • !1209: Build fixes for rtla
    • !1215: d/rules.real: Try harder to set the locale to C.UTF-8
    • !1218: Clean up udeb configuration
  • I uploaded:
    • linux-6.1 versions 6.1.106-3~deb11u1, 6.1.106-3~deb11u2, and 6.1.106-3~deb11u3 to bullseye-security
    • linux version 6.11-1~exp1 to experimental
  • I updated the bullseye-security branch to upstream version 5.10.226
  • I responded to bug reports:
    • #980555: Missing ec_sys module (fixed)
    • #1041484: kernel: hpet_acpi_add: no address or irqs in _CRS (forwarded)
    • #1071468: linux-image-amd64: mess left when kernel installation fails (grub treats the uninstalled kernel as existing) (closed)
    • #1075855: Kernel panic caused by aacraid module prevents normal boot
    • #1076555: linux-image-6.9.9-amd64: boot crash RIP: 0010:kmem_cache_alloc (closed)
    • #1078997: gretap tunnel with checksum enabled: some packets have zero checksum (closed)
    • #1080975: upgrade 6.1.106 to 6.10.6 failed (reassigned)
    • #1081195: devscripts: test-patches KeyError: pae (confirmed)
    • #1082001: linux-image-6.1.0-25-amd64: TOMOYO LSM rejects execveat(AT_EMPTY_PATH) inside chroot (referred upstream)
    • #1081546: new GPU HANG: ecode 12:1:85dffdfb, in Renderer regression in i915 driver since 6.10 kernel upgrade (more info requested)
    • #1081563: Please consider adding the new Xe Graphics driver for Intel GPUs
    • #1081310: Wired ethernet connection disabled.
  • I sent patches upstream:
    • wifi: iwlegacy: Fix field-spanning write warning in il_enqueue_hcmd() which is meant to fix bug #1062421
    • tools/rtla: Fix installation from out-of-tree build
  • I sent a new response to Proposal: Switch to linear git history
• For the Debian nfs-utils package:
  • I reviewed the MRs:
    • !8: Fixes for bugs #849942 and #849608 (closed)
    • !32: d/control: nfs-common: Move python3 to Recommends (merged)
  • I closed my own old bug report #711021: mount.nfs timeout for GETPORT is much too short
• For diffoscope:
  • I commented on issue #100: Excessive memory use when comparing Linux Debian .dbg packages to report that the issue still exists, and to provide a reproducer
  • I commented on issue #342: Gets killed trying to diff very large (~5GB) images which seems like a duplicate of #100
  • I opened the MR !145: Draft: Reduce memory usage by revert to using popen() for diff input and output which fixes #100 but needs work to avoid regressions for other cases
• For the Debian nss-wrapper package, I opened MR !4: Replace wrap-and-sort autopkgtest with a CI job
• For the Debian pam-wrapper package, I opened MR !4: Replace wrap-and-sort autopkgtest with a CI job
• For initramfs-tools:
  • I opened the MR !136: Fix copy_file again; expand documentation and add tests
  • I reviewed the MRs:
    • !66: Add boot script 00_mount_efivarfs mounting efivarfs (closed)
    • !84: Allow providing UDEV_WAIT and ROUNDTTT times in environment variables (closed)
    • !125: hook-functions: Add more modules based on dracut 90kernel-modules (merged)
    • !127: Filter net kernel modules by symbol regexp from dracut
    • !134: add losetup-rootfs-img script: boot from rootfs image files
    • !135: hook-functions: fixes for Qualcomm platforms
  • I responded to bug report #1082647: copy_exec: [regression] ignores trailing slash, installs file as directory name (pending)
• For debian-kernel-handbook:
  • I reviwed the MRs:
    • !7: Add disabling SECURITY_LOCKDOWN_LSM to chapter-common-tasks
    • !8: Disable dwarf toolchain default to really disable debug info (merged)
• I responded (belatedly) to Debian tech-ctte bug #1065416: linux-libc-dev claims to provide linux-libc-dev-ARCH-cross, but it doesn t do that completely

FTP master This month I accepted 441 and rejected 29 packages. The overall number of packages that got accepted was 448. I couldn t believe my eyes, but this month I really accepted the same number of packages as last month. Debian LTS This was my hundred-twenty-third month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. During my allocated time I uploaded or worked on:

• [unstable] libcupsfilters security update to fix one CVE related to validation of IPP attributes obtained from remote printers
• [unstable] cups-filters security update to fix two CVEs related to validation of IPP attributes obtained from remote printers
• [unstable] cups security update to fix one CVE related to validation of IPP attributes obtained from remote printers
• [DSA 5778-1] prepared package for cups-filters security update to fix two CVEs related to validation of IPP attributes obtained from remote printers
• [DSA 5779-1] prepared package for cups security update to fix one CVE related to validation of IPP attributes obtained from remote printers
• [DLA 3905-1] cups-filters security update to fix two CVEs related to validation of IPP attributes obtained from remote printers
• [DLA 3904-1] cups security update to fix one CVE related to validation of IPP attributes obtained from remote printers
• [DLA 3905-1] cups-filters security update to fix two CVEs related to validation of IPP attributes obtained from remote printers

Despite the announcement the package libppd in Debian is not affected by the CVEs related to CUPS. By pure chance there is an unrelated package with the same name in Debian. I also answered some question about the CUPS related uploads. Due to the CUPS issues, I postponed my work on other packages to October. Last but not least I did a week of FD this month and attended the monthly LTS/ELTS meeting. Debian ELTS This month was the seventy-fourth ELTS month. During my allocated time I uploaded or worked on:

• [ELA-1186-1]cups-filters security update for two CVEs in Stretch and Buster to fix the IPP attribute related CVEs.
• [ELA-1187-1]cups-filters security update for one CVE in Jessie to fix the IPP attribute related CVEs (the version in Jessie was not affected by the other CVE).

I also started to work on updates for cups in Buster, Stretch and Jessie, but their uploads will happen only in October. I also did a week of FD and attended the monthly LTS/ELTS meeting. Debian Printing This month I uploaded

• libcupsfilters to also fix a dependency and autopkgtest issue besides the security fix mentioned above.
• splix for a new upstream version. This package is managed now by OpenPrinting.

Last but not least I tried to prepare an update for hplip. Unfortunately this is a nerve-stretching task and I need some more time. This work is generously funded by Freexian! Debian Matomo This month I even found some time to upload packages that are dependencies of Matomo

• php-spyc
• php-sparkline

This work is generously funded by Freexian! Debian Astro This month I uploaded a new upstream or bugfix version of:

• openvlbi
• indi-playerone
• libsbig
• indi-pentax
• indi-sbig
• indi-fishcamp
• indi-inovaplx
• libfishcamp
• libsbig
• libplayeronecamera
• libplayerone
• libahp-gt
• libahp-xc

Most of the uploads were related to package migration to testing. As some of them are in non-free or contrib, one has to build all binary versions. From my point of view handling packages in non-free or contrib could be very much improved, but well, they are not part of Debian Anyway, starting in December there is an Outreachy project that takes care of automatic updates of these packages. So hopefully it will be much easier to keep those package up to date. I will keep you informed. Debian IoT This month I uploaded new upstream or bugfix versions of:

• pywws

Debian Mobcom This month I did source uploads of all the packages that were prepared last month by Nathan and started the transition. It went rather smooth except for a few packages where the new version did not propagate to the tracker and they got stuck in old failing autopkgtest. Anyway, in the end all packages migrated to testing. I also uploaded new upstream releases or fixed bugs in:

• osmo-libasn1c

misc This month I uploaded new upstream or bugfix versions of:

• dict-element
• libcds
• libctl
• tntnet
• gnucobol3
• gnucobol4
• meep-mpi-default
• nuspell
• c2go

Most of those uploads were needed to help packages to migrate to testing.

Welcome to the September 2024 report from the Reproducible Builds project! Our reports attempt to outline what we ve been up to over the past month, highlighting news items from elsewhere in tech where they are related. As ever, if you are interested in contributing to the project, please visit our Contribute page on our website. Table of contents:

1. New binsider tool to analyse ELF binaries
2. Unreproducibility of GHC Haskell compiler 95% fixed
3. Mailing list summary
4. Towards a 100% bit-for-bit reproducible OS
5. Two new reproducibility-related academic papers
6. Distribution work
7. diffoscope
8. Other software development
9. Android toolchain core count issue reported
10. New Gradle plugin for reproducibility
11. Website updates
12. Upstream patches
13. Reproducibility testing framework

---

New binsider tool to analyse ELF binaries Reproducible Builds developer Orhun Parmaks z has announced a fantastic new tool to analyse the contents of ELF binaries. According to the project s README page:

Binsider can perform static and dynamic analysis, inspect strings, examine linked libraries, and perform hexdumps, all within a user-friendly terminal user interface!

More information about Binsider s features and how it works can be found within Binsider s documentation pages.

Unreproducibility of GHC Haskell compiler 95% fixed A seven-year-old bug about the nondeterminism of object code generated by the Glasgow Haskell Compiler (GHC) received a recent update, consisting of Rodrigo Mesquita noting that the issue is:

95% fixed by [merge request] !12680 when -fobject-determinism is enabled. [ ]

The linked merge request has since been merged, and Rodrigo goes on to say that:

After that patch is merged, there are some rarer bugs in both interface file determinism (eg. #25170) and in object determinism (eg. #25269) that need to be taken care of, but the great majority of the work needed to get there should have been merged already. When merged, I think we should close this one in favour of the more specific determinism issues like the two linked above.

Mailing list summary On our mailing list this month:

• Fay Stegerman let everyone know that she started a thread on the Fediverse about the problems caused by unreproducible zlib/deflate compression in .zip and .apk files and later followed up with the results of her subsequent investigation.
• Long-time developer kpcyrd wrote that there has been a recent public discussion on the Arch Linux GitLab [instance] about the challenges and possible opportunities for making the Linux kernel package reproducible , all relating to the CONFIG_MODULE_SIG flag. [ ]
• Bernhard M. Wiedemann followed-up to an in-person conversation at our recent Hamburg 2024 summit on the potential presence for Reproducible Builds in recognised standards. [ ]
• Fay Stegerman also wrote about her worry about the possible repercussions for RB tooling of Debian migrating from zlib to zlib-ng as reproducibility requires identical compressed data streams. [ ]
• Martin Monperrus wrote the list announcing the latest release of maven-lockfile that is designed aid building Maven projects with integrity . [ ]
• Lastly, Bernhard M. Wiedemann wrote about potential role of reproducible builds in combatting silent data corruption, as detailed in a recent Tweet and scholarly paper on faulty CPU cores. [ ]

Towards a 100% bit-for-bit reproducible OS Bernhard M. Wiedemann began writing on journey towards a 100% bit-for-bit reproducible operating system on the openSUSE wiki:

This is a report of Part 1 of my journey: building 100% bit-reproducible packages for every package that makes up [openSUSE s] minimalVM image. This target was chosen as the smallest useful result/artifact. The larger package-sets get, the more disk-space and build-power is required to build/verify all of them.

This work was sponsored by NLnet s NGI Zero fund.

Two new reproducibility-related academic papers Marvin Strangfeld published his bachelor thesis, Reproducibility of Computational Environments for Software Development from RWTH Aachen University. The author offers a more precise theoretical definition of computational environments compared to previous definitions, which can be applied to describe real-world computational environments. Additionally, Marvin provide a definition of reproducibility in computational environments, enabling discussions about the extent to which an environment can be made reproducible. The thesis is available to browse or download in PDF format. In addition, Shenyu Zheng, Bram Adams and Ahmed E. Hassan of Queen s University, ON, Canada have published an article on hermeticity in Bazel-based build systems:

A hermetic build system manages its own build dependencies, isolated from the host file system, thereby securing the build process. Although, in recent years, new artifact-based build technologies like Bazel offer build hermeticity as a core functionality, no empirical study has evaluated how effectively these new build technologies achieve build hermeticity. This paper studies 2,439 non-hermetic build dependency packages of 70 Bazel-using open-source projects by analyzing 150 million Linux system file calls collected in their build processes. We found that none of the studied projects has a completely hermetic build process, largely due to the use of non-hermetic top-level toolchains. [ ]

Distribution work In Debian this month, 14 reviews of Debian packages were added, 12 were updated and 20 were removed, all adding to our knowledge about identified issues. A number of issue types were updated as well. [ ][ ] In addition, Holger opened 4 bugs against the debrebuild component of the devscripts suite of tools. In particular:

• #1081047: Fails to download .dsc file.
• #1081048: Does not work with a proxy.
• #1081050: Fails to create a debrebuild.tar.
• #1081839: Fails with E: mmdebstrap failed to run error.

Last month, an issue was filed to update the Salsa CI pipeline (used by 1,000s of Debian packages) to no longer test for reproducibility with reprotest s build_path variation. Holger Levsen provided a rationale for this change in the issue, which has already been made to the tests being performed by tests.reproducible-builds.org. This month, this issue was closed by Santiago R. R., nicely explaining that build path variation is no longer the default, and, if desired, how developers may enable it again. In openSUSE news, Bernhard M. Wiedemann published another report for that distribution.

diffoscope diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made the following changes, including preparing and uploading version 278 to Debian:

• New features:
  • Add a helpful contextual message to the output if comparing Debian .orig tarballs within .dsc files without the ability to fuzzy-match away the leading directory. [ ]
• Bug fixes:
  • Drop removal of calculated os.path.basename from GNU readelf output. [ ]
  • Correctly invert X% similar value and do not emit 100% similar . [ ]
• Misc:
  • Temporarily remove procyon-decompiler from Build-Depends as it was removed from testing (via #1057532). (#1082636)
  • Update copyright years. [ ]

For trydiffoscope, the command-line client for the web-based version of diffoscope, Chris Lamb also:

• Added an explicit python3-setuptools dependency. (#1080825)
• Bumped the Standards-Version to 4.7.0. [ ]

Other software development disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into system calls to reliably flush out reproducibility issues. This month, version 0.5.11-4 was uploaded to Debian unstable by Holger Levsen making the following changes:

• Replace build-dependency on the obsolete pkg-config package with one on pkgconf, following a Lintian check. [ ]
• Bump Standards-Version field to 4.7.0, with no related changes needed. [ ]

In addition, reprotest is our tool for building the same source code twice in different environments and then checking the binaries produced by each build for any differences. This month, version 0.7.28 was uploaded to Debian unstable by Holger Levsen including a change by Jelle van der Waa to move away from the pipes Python module to shlex, as the former will be removed in Python version 3.13 [ ].

Android toolchain core count issue reported Fay Stegerman reported an issue with the Android toolchain where a part of the build system generates a different classes.dex file (and thus a different .apk) depending on the number of cores available during the build, thereby breaking Reproducible Builds:

We ve rebuilt [tag v3.6.1] multiple times (each time in a fresh container): with 2, 4, 6, 8, and 16 cores available, respectively:

• With 2 and 4 cores we always get an unsigned APK with SHA-256 14763d682c9286ef .
• With 6, 8, and 16 cores we get an unsigned APK with SHA-256 35324ba4c492760 instead.

New Gradle plugin for reproducibility A new plugin for the Gradle build tool for Java has been released. This easily-enabled plugin results in:

reproducibility settings [being] applied to some of Gradle s built-in tasks that should really be the default. Compatible with Java 8 and Gradle 8.3 or later.

Website updates There were a rather substantial number of improvements made to our website this month, including:

• Chris Lamb:
  • Attempt to use GitLab CI to artifact the website; hopefully useful for testing branches. [ ]
  • Correct the linting rule whilst building the website. [ ]
  • Make a number of small changes to Kees post written by Vagrant. [ ][ ][ ]
  • Add the Civil Infrastructure Platform to the Projects page. [ ]
  • Miscellaneous administration of misfiled images. [ ][ ]
• Evangelos Tzaras made a huge number of changes related to the recent Hamburg 2024 summit [ ][ ][ ][ ][ ] as well as proposed an infographic about which question Reproducible Builds is trying to answer.
• Holger Levsen added his two presentations (Reproducible Builds: The First Eleven Years and Preserving *other* build artifacts) to the website. [ ]
• Jelle van der Waa completely modernised the System Images documentation, noting that a lot has changed since 2017(!); ext4, erofs and FAT filesystems can now be made reproducible . [ ]
• Developer RyanSquared replaced the continuous integration test link for Arch Linux on our Projects page with an external instance [ ][ ] as well as updated the documentation to reflect the dependencies required to build the website [ ].
• Vagrant Cascadian pushed a lengthy interview with Linux developer Kees Cook. [ ][ ][ ][ ]

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

• Bernhard M. Wiedemann:
  • agama-integration-tests (random)
  • contrast (FTBFS-nocheck)
  • cpython (FTBFS-2038)
  • crash (parallelism, race)
  • ghostscript (toolchain date)
  • glycin-loaders (FTBFS -j1)
  • gstreamer-plugins-rs (date, other)
  • kernel-doc/Sphinx (toolchain bug, parallelism/race)
  • kernel (parallelism in BTF)
  • libcamera (random key)
  • libgtop (uname -r)
  • libsamplerate (random temporary directory)
  • lua-luarepl (FTBFS)
  • meson (toolchain)
  • netty (modification time in .a)
  • nvidia-persistenced (date)
  • nvidia-xconfig (date-related issue)
  • obs-build (build-tooling corruption)
  • perl (Perl records kernel version)
  • pinentry (make efl droppable)
  • python-PyGithub (FTBFS 2024-11-25)
  • python-Sphinx (parallelism/race)
  • python-chroma-hnswlib (CPU)
  • python-libcst
  • python-pygraphviz (random timing)
  • python312 (.pyc embeds modification time)
  • python312 (drop .pyc from documentation time)
  • scap-security-guide (date)
  • seahorse (parallelism)
  • subversion (minor Java .jar modification times)
  • xen/acpica (date-related issue in toolchain)
  • xmvn (random)
• Fridrich Strba:
  • ant (jar mtime)
  • xmvn (various fixes)
• Chris Lamb:
  • #1082702 filed against magic-wormhole-transit-relay.
  • #1082706 filed against python-sphobjinv.
  • #1082707 filed against lomiri-content-hub.
  • #1082796 filed against python-mt-940.
  • #1082806 filed against tree-puzzle.
  • #1083053 filed against muon-meson.
• James Addison:
  • guake (fix a parallelism/timing-race build bug)
  • sphobjinv (duplicates fix from Debian bug #1082706)

Reproducibility testing framework The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In September, a number of changes were made by Holger Levsen, including:

• Debian-related changes:
  • Upgrade the osuosl4 node to Debian trixie in anticipation of running debrebuild and rebuilderd there. [ ][ ][ ]
  • Temporarily mark the osuosl4 node as offline due to ongoing xfs_repair filesystem maintenance. [ ][ ]
  • Do not warn about (very old) broken nodes. [ ]
  • Add the risc64 architecture to the multiarch version skew tests for Debian trixie and sid. [ ][ ][ ]
  • Mark the virt 32,64 b nodes as down. [ ]
• Misc changes:
  • Add support for powercycling OpenStack instances. [ ]
  • Update the fail2ban to ban hosts for 4 weeks in total [ ][ ] and take care to never ban our own Jenkins instance. [ ]

In addition, Vagrant Cascadian recorded a disk failure for the virt32b and virt64b nodes [ ], performed some maintenance of the cbxi4a node [ ][ ] and marked most armhf architecture systems as being back online.

---

Finally, If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Mastodon: @reproducible_builds@fosstodon.org
• Mailing list: rb-general@lists.reproducible-builds.org
• Twitter: @ReproBuilds

The diffoscope maintainers are pleased to announce the release of diffoscope version 279. This version includes the following changes: You find out more by visiting the project homepage.

Dear Debian community, this are my bits from DPL for September. New lintian maintainer I'm pleased to welcome Louis-Philippe V ronneau as a new Lintian maintainer. He humorously acknowledged his new role, stating, "Apparently I'm a Lintian maintainer now". I remain confident that we can, and should, continue modernizing our policy checker, and I see this as one important step toward that goal. SPDX name / license tools There was a discussion about deprecating the unique names for DEP-5 and migrating to fully compliant SPDX names. Simon McVittie wrote: "Perhaps our Debian-specific names are better, but the relevant question is whether they are sufficiently better to outweigh the benefit of sharing effort and specifications with the rest of the world (and I don't think they are)." Also Charles Plessy sees the value of deprecating the Debian ones and align on SPDX. The thread on debian-devel list contains several practical hints for writing debian/copyright files. proposal: Hybrid network stack for Trixie There was a very long discussion on debian-devel list about the network stack on Trixie that started in July and was continued in end of August / beginning of September. The discussion was also covered on LWN. It continued in a "proposal: Hybrid network stack for Trixie" by Lukas M rdian. Contacting teams I continued reaching out to teams in September. One common pattern I've noticed is that most teams lack a clear strategy for attracting new contributors. Here's an example snippet from one of my outreach emails, which is representative of the typical approach: Q: Do you have some strategy to gather new contributors for your team? A: No. Q: Can I do anything for you? A: Everything that can help to have more than 3 guys :-D Well, only the first answer, "No," is typical. To help the JavaScript team, I'd like to invite anyone with JavaScript experience to join the team's mailing list and offer to learn and contribute. While I've only built a JavaScript package once, I know this team has developed excellent tools that are widely adopted by others. It's an active and efficient team, making it a great starting point for those looking to get involved in Debian. You might also want to check out the "Little tutorial for JS-Team beginners". Given the lack of a strategy to actively recruit new contributors--a common theme in the responses I've received--I recommend reviewing my talk from DebConf23 about teams. The Debian Med team would have struggled significantly in my absence (I've paused almost all work with the team since becoming DPL) if I hadn't consistently focused on bringing in new members. I'm genuinely proud of how the team has managed to keep up with the workload (thank you, Debian Med team!). Of course, onboarding newcomers takes time, and there's no guarantee of long-term success, but if you don't make the effort, you'll never find out. OS underpaid The Register, in its article titled "Open Source Maintainers Underpaid, Swamped by Security, Going Gray", summarizes the 2024 State of the Open Source Maintainer Report. I find this to be an interesting read, both in general and in connection with the challenges mentioned in the previous paragraph about finding new team members. Kind regards Andreas.

Although I've never written about them, I've been interested in music synthesisers for ages. My colleagues know this. Whilst I've been off sick, they had a whip-round and bought me a voucher for Andertons, a UK-based music store, to cheer me up. I'm absolutely floored by this generosity. And so, I'm now on a quest to buy a synthesizer! Although, not my first one. I bought my first synth, an Alesis Micron, from a colleague at $oldjob, 16 years ago. For various reasons, I've struggled to engage with it, and it's mostly been gathering dust on my desk in all that time. (I might write more about the Micron in a later blog post). "Bad Gear" sums it up better than I could: So, I'm not truly buying my "first" synth, but for all intents and purposes I'm on a similar journey to if I was, and I thought it might be fun to write about it. Goals I want something which has as many of its parameters presented physically, as knobs or sliders etc., as possible. One reason I've failed to engage with the Micron (so far) is it's at the other end of this spectrum, with hundreds of tunable parameters but a small handful of knobs. To change parameters you have to go diving into menus presented on a really old-fashioned, small LCD display. If you know what you are looking for, you can probably find it; but if you just want to experiment and play around, it's off-putting. Secondly, I want something I can use away from a computer, as much as possible. Computers are my day-job, largely dominate my existing hobbies, and are unavoidable even in some of the others (like 3d printing). Most of the computers I interact with run Linux. And for all its strengths, audio management is not one of them. If I'm going to carve out some of my extremely limited leisure time to explore this stuff, I don't to spend any of it (at least now) fighting Pulseaudio/ALSA/Pipewire/JACK/OSS/whatever, or any of the other foibles that might crop up¹. Thirdly, I'd like something which, in its soul, is an instrument. You can get some amazing little synth boxes with a huge number of features in them. Something with a limited number of features but which really feels well put together would suit me better. So next time, I'll write about the 2-3 top candidates on my list. Can you guess what they might be?

We are excited to announce and welcome Freexian into Debian Partners. Freexian specializes in Free Software with a particular focus on Debian GNU/Linux. Freexian can assist with consulting, training, technical support, packaging, or software development on projects involving use or development of Free software. All of Freexian's employees and partners are well-known contributors in the Free Software community, a choice that is integral to Freexian's business model. About the Debian Partners Program The Debian Partners Program was created to recognize companies and organizations that help and provide continuous support to the project with services, finances, equipment, vendor support, and a slew of other technical and non-technical services. Partners provide critical assistance, help, and support which has advanced and continues to further our work in providing the 'Universal Operating System' to the world. Thank you Freexian!

After (quite) a summer break, here comes the 4th article of the 5-episode blog post series on Polis, written by Guido Berh rster, member of staff at my company Fre(i)e Software GmbH. Have fun with the read on Guido's work on Polis,
Mike Table of Contents of the Blog Post Series

1. Introduction
2. Initial evaluation and adaptation
3. Issues extending Polis and adjusting our goals
4. Creating (a) new frontend(s) for Polis (this article)
5. Current status and roadmap

4. Creating (a) new frontend(s) for Polis Why a new frontend was needed... Our initial experiences of working with Polis, the effort required to implement more invasive changes and the desire of iterating changes more rapidly ultimately lead to the decision to create a new foundation for frontend development that would be independent of but compatible with the upstream project. Our primary objective was thus not to develop another frontend but rather to make frontend development more flexible and to facilitate experimentation and rapid prototyping of different frontends by providing abstraction layers and building blocks. This also implied developing a corresponding backend since the Polis backend is tightly coupled to the frontend and is neither intended to be used by third-party projects nor supporting cross-domain requests due to the expectation of being embedded as an iframe on third-party websites. The long-term plan for achieving our objectives is to provide three abstraction layers for building frontends:

• a stable cross-domain HTTP API
• a low-level JavaScript library for interacting with the HTTP API
• a high-level library of WebComponents as a framework-neutral way of rapidly building frontends

The Particiapp Project Under the umbrella of the Particiapp project we have so far developed two new components:

• the Particiapi server which provides the HTTP API
• the example frontend project which currently contains both the client library and an experimental example frontend built with it

Both the participation frontend and backend are fully compatible and require an existing Polis installation and can be run alongside the upstream frontend. More specifically, the administration frontend and common backend are required to administrate conversations and send out notifications and the statistics processing server is required for processing the voting results. Particiapi server For the backend the Python language and the Flask framework were chosen as a technological basis mainly due to developer mindshare, a large community and ecosystem and the smaller dependency chain and maintenance overhead compared to Node.js/npm. Instead of integrating specific identity providers we adopted the OpenID Connect standard as an abstraction layer for authentication which allows delegating authentication either to a self-hosted identity provider or a large number of existing external identity providers. Particiapp Example Frontend The experimental example frontend serves both as a test bed for the client library and as a tool for better understanding the needs of frontend designers. It also features a completely redesigned user interface and results visualization in line with our goals. Branded variants are currently used for evaluation and testing by the stakeholders. In order to simplify evaluation, development, testing and deployment a Docker Compose configuration is made available which contains all necessary components for running Polis with our experimental example frontend. In addition, a development environment is provided which includes a preconfigured OpenID Connect identity provider (KeyCloak), SMTP-Server with web interface (MailDev), and a database frontend (PgAdmin). The new frontend can also be tested using our public demo server.