Search Results: "reg"

17 October 2025

David Bremner: Hibernate on the pocket reform 13/n

Context

Some progress upstream Recently Sebastian Reichel at Collabora [1] has made a few related commits, apparently inspired in part by my kvetching on this blog.

Disconnecting and reconnecting PCI busses At some point I noticed error message about the nvme device on resume. I then learned how to disconnect and reconnect PCI buses in Linux. I ended up with something like the following. At least the PCI management seems to work. I can manually disconnect all the PCI busses and rescan to connect them again on a running system. It presumably helps that I am not using the nvme device in this system.
set -x
echo platform >  /sys/power/pm_test
echo reboot > /sys/power/disk
rmmod mt76x2u
sleep 2
echo 1   tee /sys/bus/pci/devices/0003:30:00.0/remove
sleep 2
echo 1   tee /sys/bus/pci/devices/0004:41:00.0/remove
sleep 2
echo 1   tee /sys/bus/pci/devices/0004:40:00.0/remove
sleep 2
echo LSPCI:
lspci -t
sleep 2
echo disk >  /sys/power/state
sleep 2
echo 1   tee /sys/bus/pci/rescan
sleep 2
modprobe mt76x2u

Minimal changes to upstream With the ongoing work at collabora I decided to try a minimal patch stack to get the pocket reform to boot. I added the following 3 commits (available from [3]).
09868a4f2eb (HEAD -> reform-patches) copy pocket-reform dts from reform-debian-packages
152e2ae8a193 pocket/panel: sleep fix v3
18f65da9681c add-multi-display-panel-driver
It does indeed boot and seems stable.
$ uname -a
Linux anthia 6.18.0-rc1+ #19 SMP Thu Oct 16 11:32:04 ADT 2025 aarch64 GNU/Linux
Running the hibernation script above I get no output from the lspci, but seemingly issues with PCI coming back from hibernate:
[  424.645109] PM: hibernation: Allocated 361823 pages for snapshot
[  424.647216] PM: hibernation: Allocated 1447292 kbytes in 3.23 seconds (448.07 MB/s)
[  424.649321] Freezing remaining freezable tasks
[  424.654767] Freezing remaining freezable tasks completed (elapsed 0.003 seconds)
[  424.661070] rk_gmac-dwmac fe1b0000.ethernet end0: Link is Down
[  424.740716] rockchip-dw-pcie a40c00000.pcie: Failed to receive PME_TO_Ack
[  424.742041] PM: hibernation: hibernation debug: Waiting for 5 second(s).
[  430.074757] pci 0004:40:00.0: [1d87:3588] type 01 class 0x060400 PCIe Root Port
F F & Zn [  watchdog: CPU4: Watchdog detected hard LOCKUP on cpu 5
[  456.039004] Modules linked in: xt_CHECKSUM xt_tcpudp nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat x_tables bridge stp llc nf_tables aes_neon_bs aes_neon_blk ccm dwmac_rk binfmt_misc mt76x2_common mt76x02_usb mt76_usb mt76x02_lib mt76 mac80211 rk805_pwrkey snd_soc_tlv320aic31xx snd_soc_simple_card reform2_lpc(OE) libarc4 rockchip_saradc industrialio_triggered_buffer kfifo_buf industrialio cfg80211 rockchip_thermal rockchip_rng hantro_vpu cdc_acm v4l2_vp9 v4l2_jpeg rockchip_rga rfkill snd_soc_rockchip_i2s_tdm videobuf2_dma_sg v4l2_h264 panthor snd_soc_audio_graph_card drm_gpuvm snd_soc_simple_card_utils drm_exec evdev joydev dm_mod nvme_fabrics efi_pstore configfs nfnetlink autofs4 ext4 crc16 mbcache jbd2 btrfs blake2b_generic xor xor_neon raid6_pq mali_dp snd_soc_meson_axg_toddr snd_soc_meson_axg_fifo snd_soc_meson_codec_glue panfrost drm_shmem_helper gpu_sched ao_cec_g12a meson_vdec(C) videobuf2_dma_contig videobuf2_memops v4l2_mem2mem videobuf2_v4l2 videodev
[  456.039060]  videobuf2_common mc dw_hdmi_i2s_audio meson_drm meson_canvas meson_dw_mipi_dsi meson_dw_hdmi mxsfb mux_mmio panel_edp imx_dcss ti_sn65dsi86 nwl_dsi mux_core pwm_imx27 hid_generic usbhid hid xhci_plat_hcd onboard_usb_dev xhci_hcd nvme nvme_core snd_soc_hdmi_codec snd_soc_core nvme_keyring nvme_auth hkdf snd_pcm_dmaengine snd_pcm snd_timer snd soundcore fan53555 rtc_pcf8523 micrel phy_package stmmac_platform stmmac pcs_xpcs rk808_regulator phylink sdhci_of_dwcmshc mdio_devres dw_mmc_rockchip of_mdio sdhci_pltfm phy_rockchip_usbdp fixed_phy dw_mmc_pltfm fwnode_mdio typec phy_rockchip_naneng_combphy phy_rockchip_samsung_hdptx pwm_rockchip sdhci dwc3 libphy dw_wdt dw_mmc ehci_platform rockchip_dfi mdio_bus cqhci ulpi ohci_platform ehci_hcd udc_core ohci_hcd rockchipdrm phy_rockchip_inno_usb2 usbcore dw_hdmi_qp analogix_dp dw_mipi_dsi cpufreq_dt dw_mipi_dsi2 i2c_rk3x usb_common drm_dp_aux_bus [last unloaded: mt76x2u]
[  456.039111] Sending NMI from CPU 4 to CPUs 5:
[  471.942262] page_pool_release_retry() stalled pool shutdown: id 9, 2 inflight 60 sec
[  532.989611] page_pool_release_retry() stalled pool shutdown: id 9, 2 inflight 121 sec
This does look like some progress, probably thanks to Sebastien. Comparing with the logs in hibernate-pocket-12, the resume process is not bailing out complaining about PHY.

Attempt to reapply PCI reset patches Following the procedure in hibernate-pocket-12, I attempted to re-apply the pci reset patches [2]. In particular I followed the hints output by b4. Unfortunately there are too many conflicts now for me to sensibly resolve.
  1. https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux.git#rockchip-devel
  2. https://lore.kernel.org/all/20250715-pci-port-reset-v6-0-6f9cce94e7bb@oss.qualcomm.com/#r
  3. https://salsa.debian.org/bremner/collabora-rockchip-3588#reform-patches

14 October 2025

Dirk Eddelbuettel: qlcal 0.0.17 on CRAN: Regular Update

The seventeenth release of the qlcal package arrivied at CRAN today, once again following a QuantLib release as 1.40 came out this morning. qlcal delivers the calendaring parts of QuantLib. It is provided (for the R package) as a set of included files, so the package is self-contained and does not depend on an external QuantLib library (which can be demanding to build). qlcal covers over sixty country / market calendars and can compute holiday lists, its complement (i.e. business day lists) and much more. Examples are in the README at the repository, the package page, and course at the CRAN package page. This releases mainly synchronizes qlcal with the QuantLib release 1.40. Only one country calendar got updated; the diffstat looks larger as the URL part of the copyright got updated throughout. We also updated the URL for the GPL-2 badge: when CRAN checks this, they always hit a timeout as the FSF server possibly keeps track of incoming requests; we now link to version from the R Licenses page to avoid this.

Changes in version 0.0.17 (2025-07-14)
  • Synchronized with QuantLib 1.40 released today
  • Calendar updates for Singapore
  • URL update in README.md

Courtesy of my CRANberries, there is a diffstat report for this release. See the project page and package documentation for more details, and more examples.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

Freexian Collaborators: Debian Contributions: Old Debian Printing software and C23, Work to decommission packages.qa.debian.org, rebootstrap uses *-for-host and more! (by Anupa Ann Joseph)

Debian Contributions: 2025-09 Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

Updating old Debian Printing software to meet C23 requirements, by Thorsten Alteholz The work of Thorsten fell under the motto gcc15 . Due to the introduction of gcc15 in Debian, the default language version was changed to C23. This means that for example, function declarations without parameters are no longer allowed. As old software, which was created with ANSI C (or C89) syntax, made use of such function declarations, it was a busy month. One could have used something like -std=c17 as compile flags, but this would have just postponed the tasks. As a result Thorsten uploaded modernized versions of ink, nm2ppa and rlpr for the Debian printing team.

Work done to decommission packages.qa.debian.org, by Rapha l Hertzog Rapha l worked to decommission the old package tracking system (packages.qa.debian.org). After figuring out that it was still receiving emails from the bug tracking system (bugs.debian.org), from multiple debian lists and from some release team tools, he reached out to the respective teams to either drop those emails or adjust them so that they are sent to the current Debian Package Tracker (tracker.debian.org).

rebootstrap uses *-for-host, by Helmut Grohne Architecture cross bootstrapping is an ongoing effort that has shaped Debian in various ways over the years. A longer effort to express toolchain dependencies now bears fruit. When cross compiling, it becomes important to express what architecture one is compiling for in Build-Depends. As these packages have become available in trixie , more and more packages add this extra information and in August, the libtool package gained a gfortran-for-host dependency. It was the first package in the essential build closure to adopt this and required putting the pieces together in rebootstrap that now has to build gcc-defaults early on. There still are hundreds of packages whose dependencies need to be updated though.

Miscellaneous contributions
  • Rapha l dropped the Build Log Scan integration in tracker.debian.org since it was showing stale data for a while as the underlying service has been discontinued.
  • Emilio updated pixman to 0.46.4.
  • Emilio coordinated several transitions, and NMUed guestfs-tools to unblock one.
  • Stefano uploaded Python 3.14rc3 to Debian unstable. It s not yet used by any packages, but it allows testing the level of support in packages to begin.
  • Stefano upgraded almost all of the debian-social infrastructure to Debian trixie .
  • Stefano published the sponsorship brochures for DebConf 26.
  • Stefano attended the Debian Technical Committee meeting.
  • Stefano uploaded routine upstream updates for a handful of Python packages (pycparser, beautifulsoup4, platformdirs, pycparser, python-authlib, python-cffi, python-mitogen, python-resolvelib, python-super-collections, twine).
  • Stefano reviewed and responded to DebConf 25 feedback.
  • Stefano investigated and fixed a request visibility bug in debian-reimbursements (for admin-altered requests).
  • Lucas reviewed a couple of merge requests from external contributors for Go and Ruby packages.
  • Lucas updated some ruby packages to its latest upstream version (thin, passenger, and puma is still WIP).
  • Lucas set up the build environment to run rebuilds of reverse dependencies of ruby using ruby3.4. As an alternative, he is looking for personal repositories provided by Debusine to perform this task more easily. This is the preparation for the transition to ruby3.4 as the default in Debian.
  • Lucas helped on the next round of the Outreachy internship program.
  • Helmut sent patches for 30 cross build failures and responded to cross building support questions on the mailing list.
  • Helmut continued to maintain rebootstrap. As gcc version 15 became the default, test jobs for version 14 had to be dropped. A fair number of patches were applied to packages and could be dropped.
  • Helmut resumed removing RC-buggy packages from unstable and sponsored a termrec upload to avoid its deletion. This work was paused to give packages some time to migrate to forky .
  • Santiago reviewed different merge requests created by different contributors. Those MRs include a new test to build reverse dependencies, created by Aquila Macedo as part of his GSoC internship; restore how lintian was used in experimental, thanks Otto Kek l inen; and the fix by Christian Bayle to support again extra repositories in deb822-style sources, whose support was broken with the move to sbuild+unshare last month.
  • While doing some new upstream release updates, thanks to Debusine s reverse dependencies autopkgtest checks, Santiago discovered that paramiko 4.0 will introduce a regression in libcloud by the drop of support for the obsolete DSA keys. Santiago finally uploaded to unstable both paramiko 4.0, and a regression fix for libcloud.
  • Santiago has taken part in different discussions and meetings for the preparation of DebConf 26. The DebConf 26 local team aims to prepare for the conference with enough time in advance.
  • Carles kept working on the missing-package-relations and reporting missing Recommends. He improved the tooling to detect and report bugs creating 269 bugs and followed up comments. 37 bugs have been resolved, others acknowledged. The missing Recommends are a mixture of packages that are gone from Debian, packages that changed name, typos and also packages that were recommended but are not packaged in Debian.
  • Carles improved the missing-package-relations to report broken Suggests only for packages that used to be in Debian but are removed from it now. No bugs have been created yet for this case but identified 1320 of them.
  • Colin spent much of the month chasing down build/test regressions in various Python packages due to other upgrades, particularly relating to pydantic, python-pytest-asyncio, and rust-pyo3.
  • Colin optimized some code in ubuntu-dev-tools (affecting e.g. pull-debian-source) that made O(n) HTTP requests when it could instead make O(1).
  • Anupa published Micronews as part of Debian Publicity team work.

13 October 2025

Russell Coker: WordPress Spam Users

Just over a year ago I configured my blog to only allow signed in users to comment to reduce spam [1]. This has stopped all spam comments, it was even more successful than expected but spammers keep registering accounts. I ve now got almost 5000 spam accounts, an average of more than 10 per day. I don t know why they keep creating them without trying to enter comments. At first I thought that they were trying to assemble a lot of accounts for a deluge of comment spam but that hasn t happened. There are some WordPress plugins for bulk deletion of users but I couldn t find one with support for delete all users who haven t submitted a comment . So I do it a page at a time, but of course I don t want to do it 100 at a time so I used the below SQL to change it to 400 at a time. I initially tried larger numbers like 2000 but got Chrome timeouts when trying to click the check-box to select all users. From experimenting it seems that the time taken to check that is worse than linear. Doing it for 2000 users is obviously much more than 5* the duration of doing it for 400. 800 users was one attempt which resulted in it being possible to select them all but then it gave an error about the URL being too long when it came to actually delete them. After a binary search I found that 450 was too many but 400 worked. So now it s 12 operations to delete all the spam accounts. Each bulk delete operation is 5 GUI operations so it s 60 operations to delete 15 months of spam users. This is annoying, but less than the other problems of spam.
UPDATE  wp_usermeta  SET  meta_value  = 400 WHERE  user_id  = 2 AND  meta_key  = 'users_per_page';
Deleting the spam users reduced the size of the backup (zstd -9 of a mysql dump) for my blog by 6.5%. Then changing from zstd -9 to -19 reduced it by another 13%. After realising this difference I configured all my mysql backups to be compressed with zstd -19, this will make a difference on the system with over 30G of zstd compressed mysql backups.

Freexian Collaborators: Monthly report about Debian Long Term Support, September 2025 (by Roberto C. S nchez)

Like each month, have a look at the work funded by Freexian s Debian LTS offering.

Debian LTS contributors In September, 20 contributors have been paid to work on Debian LTS, their reports are available:
  • Abhijith PA did 10.0h (out of 10.0h assigned and 4.0h from previous period), thus carrying over 4.0h to the next month.
  • Andreas Henriksson did 1.0h (out of 0.0h assigned and 20.0h from previous period), thus carrying over 19.0h to the next month.
  • Bastien Roucari s did 20.0h (out of 20.0h assigned).
  • Ben Hutchings did 20.0h (out of 21.0h assigned), thus carrying over 1.0h to the next month.
  • Carlos Henrique Lima Melara did 10.0h (out of 12.0h assigned), thus carrying over 2.0h to the next month.
  • Chris Lamb did 18.0h (out of 18.0h assigned).
  • Daniel Leidert did 21.0h (out of 21.0h assigned).
  • Emilio Pozuelo Monfort did 39.75h (out of 40.0h assigned), thus carrying over 0.25h to the next month.
  • Guilhem Moulin did 15.0h (out of 15.0h assigned).
  • Jochen Sprickerhof did 12.0h (out of 9.25h assigned and 11.75h from previous period), thus carrying over 9.0h to the next month.
  • Lee Garrett did 13.5h (out of 21.0h assigned), thus carrying over 7.5h to the next month.
  • Lucas Kanashiro did 8.0h (out of 20.0h assigned), thus carrying over 12.0h to the next month.
  • Markus Koschany did 15.0h (out of 3.25h assigned and 17.75h from previous period), thus carrying over 6.0h to the next month.
  • Paride Legovini did 6.0h (out of 8.0h assigned), thus carrying over 2.0h to the next month.
  • Roberto C. S nchez did 7.25h (out of 7.75h assigned and 13.25h from previous period), thus carrying over 13.75h to the next month.
  • Santiago Ruano Rinc n did 13.25h (out of 13.5h assigned and 1.5h from previous period), thus carrying over 1.75h to the next month.
  • Sylvain Beucler did 17.0h (out of 7.75h assigned and 13.25h from previous period), thus carrying over 4.0h to the next month.
  • Thorsten Alteholz did 21.0h (out of 21.0h assigned).
  • Tobias Frost did 5.0h (out of 0.0h assigned and 8.0h from previous period), thus carrying over 3.0h to the next month.
  • Utkarsh Gupta did 16.5h (out of 14.25h assigned and 6.75h from previous period), thus carrying over 4.5h to the next month.

Evolution of the situation In September, we released 38 DLAs.
  • Notable security updates:
    • modsecurity-apache prepared by Adrian Bunk, fixes a cross-site scripting vulnerability
    • cups, prepared by Thorsten Alteholz, fixes authentication bypass and denial of service vulnerabilities
    • jetty9, prepared by Adrian Bunk, fixes the MadeYouReset vulnerability (a recent, well-known denial of service vulnerability)
    • python-django, prepared by Chris Lamb, fixes a SQL injection vulnerability
    • firefox-esr and thunderbird, prepared by Emilio Pozuelo Monfort, were updated from the 128.x ESR series to the 140.x ESR series, fixing a number of vulnerabilities as well
  • Notable non-security updates:
    • wireless-regdb prepared by Ben Hutchings, updates information reflecting changes to radio regulations in many countries
There was one package update contributed by a Debian Developer outside of the LTS Team: an update of node-tar-fs, prepared by Xavier Guimard (a member of the Node packaging team). Finally, LTS Team members also contributed updates of the following packages:
  • libxslt (to stable and oldstable), prepared by Guilhem Moulin, to address a regression introduced in a previous security update
  • libphp-adodb (to stable and oldstable), prepared by Abhijith PA
  • cups (to stable and oldstable), prepared by Thorsten Alteholz
  • u-boot (to oldstable), prepared by Daniel Leidert and Jochen Sprickerhof
  • libcommongs-lang3-java (to stable and oldstable), prepared by Daniel Leidert
  • python-internetarchive (to oldstable), prepared by Daniel Leidert
One other notable contribution by a member of the LTS Team is that Sylvain Beucler proposed a fix upstream for CVE-2025-2760 in gimp2. Upstream no longer supports gimp2, but it is still present in Debian LTS, and so proposing this fix upstream is of benefit to other distros which may still be supporting the older gimp2 packages.

Thanks to our sponsors Sponsors that joined recently are in bold.

11 October 2025

John Goerzen: A Mail Delivery Mystery: Exim, systemd, setuid, and Docker, oh my!

On mail.quux, a node of NNCPNET (the NNCP-based peer-to-peer email network), I started noticing emails not being delivered. They were all in the queue, frozen, and Exim s log had entries like:
unable to set gid=5001 or uid=5001 (euid=100): local delivery to [redacted] transport=nncp
Weird. Stranger still, when I manually ran the queue with sendmail -qff -v, they all delivered fine. Huh. Well, I thought, it was a one-off weird thing. But then it happened again. Upon investigating, I observed that this issue was happening only on messages submitted by SMTP. Which, on these systems, aren t that many. While trying different things, I tried submitting a message to myself using SMTP. Nothing to do with NNCP at all. But look at this:
 jgoerzen@[redacted] R=userforward defer (-1): require_files: error for /home/jgoerzen/.forward: Permission denied
Strraaannnge . All the information I could find about this, even a FAQ entry, said that the problem is that Exim isn t setuid root. But it is:
-rwsr-xr-x 1 root root 1533496 Mar 29  2025 /usr/sbin/exim4
This problem started when I upgraded to Debian Trixie. So what changed there? There are a lot of possibilities; this is running in Docker using my docker-debian-base system, which runs a regular Debian in Docker, including systemd. I eventually tracked it down to Exim migrating from init.d to systemd in trixie, and putting a bunch of lockdowns in its service file. After a bunch of trial and error, I determined that I needed to override this set of lockdowns to make it work. These overrides did the trick:
ProtectClock=false
PrivateDevices=false
RestrictRealtime=false
ProtectKernelModules=false
ProtectKernelTunables=false
ProtectKernelLogs=false
ProtectHostname=false
I don t know for sure if the issue is related to setuid. But if it is, there s nothing that immediately jumps out at me about any of these that would indicate a problem with setuid. I also don t know if running in Docker makes any difference. Anyhow, problem fixed, but mystery not solved!

10 October 2025

Reproducible Builds: Reproducible Builds in September 2025

Welcome to the September 2025 report from the Reproducible Builds project! Welcome to the very latest report from the Reproducible Builds project. Our monthly reports outline what we ve been up to over the past month, and highlight items of news from elsewhere in the increasingly-important area of software supply-chain security. As ever, if you are interested in contributing to the Reproducible Builds project, please see the Contribute page on our website. In this report:

  1. Reproducible Builds Summit 2025
  2. Can t we have nice things?
  3. Distribution work
  4. Tool development
  5. Reproducibility testing framework
  6. Upstream patches

Reproducible Builds Summit 2025 Please join us at the upcoming Reproducible Builds Summit, set to take place from October 28th 30th 2025 in Vienna, Austria! We are thrilled to host the eighth edition of this exciting event, following the success of previous summits in various iconic locations around the world, including Venice, Marrakesh, Paris, Berlin, Hamburg and Athens. Our summits are a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort. During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field. Our aim is to create an inclusive space that fosters collaboration, innovation and problem-solving. If you re interesting in joining us this year, please make sure to read the event page which has more details about the event and location. Registration is open until 20th September 2025, and we are very much looking forward to seeing many readers of these reports there!

Can t we have nice things? Debian Developer Gunnar Wolf blogged that George V. Neville-Neil s Kode Vicious column in Communications of the ACM in which reproducible builds is mentioned without needing to introduce it (assuming familiarity across the computing industry and academia) . Titled, Can t we have nice things?, the article mentions:
Once the proper measurement points are known, we want to constrain the system such that what it does is simple enough to understand and easy to repeat. It is quite telling that the push for software that enables reproducible builds only really took off after an embarrassing widespread security issue ended up affecting the entire Internet. That there had already been 50 years of software development before anyone thought that introducing a few constraints might be a good idea is, well, let s just say it generates many emotions, none of them happy, fuzzy ones. [ ]

Distribution work In Debian this month, Johannes Starosta filed a bug against the debian-repro-status package, reporting that it does not work on Debian trixie. (An upstream bug report was also filed.) Furthermore, 17 reviews of Debian packages were added, 10 were updated and 14 were removed this month adding to our knowledge about identified issues. In March s report, we included the news that Fedora would aim for 99% package reproducibility. This change has now been deferred to Fedora 44 according to Phoronix. Lastly, Bernhard M. Wiedemann posted another openSUSE monthly update for their work there.

Tool development diffoscope version 306 was uploaded to Debian unstable by Chris Lamb. It included contributions already covered in previous months as well as some changes by Zbigniew J drzejewski-Szmek to address issues with the fdtump support [ ] and to move away from the deprecated codes.open method. [ ][ ] strip-nondeterminism version 1.15.0-1 was uploaded to Debian unstable by Chris Lamb. It included a contribution by Matwey Kornilov to add support for inline archive files for Erlang s escript [ ]. kpcyrd has released a new version of rebuilderd. As a quick recap, rebuilderd is an automatic build scheduler that tracks binary packages available in a Linux distribution and attempts to compile the official binary packages from their (purported) source code and dependencies. The code for in-toto attestations has been reworked, and the instances now feature a new endpoint that can be queried to fetch the list of public-keys an instance currently identifies itself by. [ ] Lastly, Holger Levsen bumped the Standards-Version field of disorderfs, with no changes needed. [ ][ ]

Reproducibility testing framework The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In August, however, a number of changes were made by Holger Levsen, including:
  • Setting up six new rebuilderd workers with 16 cores and 16 GB RAM each.
  • reproduce.debian.net-related:
    • Do not expose pending jobs; they are confusing without explaination. [ ]
    • Add a link to v1 API specification. [ ]
    • Drop rebuilderd-worker.conf on a node. [ ]
    • Allow manual scheduling for any architectures. [ ]
    • Update path to trixie graphs. [ ]
    • Use the same rebuilder-debian.sh script for all hosts. [ ]
    • Add all other suites to all other archs. [ ][ ][ ][ ]
    • Update SSH host keys for new hosts. [ ]
    • Move to the pull184 branch. [ ][ ][ ][ ][ ]
    • Only allow 20 GB cache for workers. [ ]
  • OpenWrt-related:
    • Grant developer aparcar full sudo control on the ionos30 node. [ ][ ]
  • Jenkins nodes:
    • Add a number of new nodes. [ ][ ][ ][ ][ ]
    • Dont expect /srv/workspace to exist on OSUOSL nodes. [ ]
    • Stop hardcoding IP addresses in munin.conf. [ ]
    • Add maintenance and health check jobs for new nodes. [ ]
    • Document slight changes in IONOS resources usage. [ ]
  • Misc:
    • Drop disabled Alpine Linux tests for good. [ ]
    • Move Debian live builds and some other Debian builds to the ionos10 node. [ ]
    • Cleanup some legacy support from releases before Debian trixie. [ ]
In addition, Jochen Sprickerhof made the following changes relating to reproduce.debian.net:
  • Do not expose pending jobs on the main site. [ ]
  • Switch the frontpage to reference Debian forky [ ], but do not attempt to build Debian forky on the armel architecture [ ].
  • Use consistent and up to date rebuilder-debian.sh script. [ ]
  • Fix supported worker architectures. [ ]
  • Add a basic excuses page. [ ]
  • Move to the pull184 branch. [ ][ ][ ][ ]
  • Fix a typo in the JavaScript. [ ]
  • Update front page for the new v1 API. [ ][ ]
Lastly, Roland Clobus did some maintenance relating to the reproducibility testing of the Debian Live images. [ ][ ][ ][ ]

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

9 October 2025

Thorsten Alteholz: My Debian Activities in September 2025

Debian LTS This was my hundred-thirty-fifth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. During my allocated time I uploaded or worked on: An upload of spim was not needed, as the corresponding CVE could be marked as ignored. I also started to work on an open-vm-tools and attended the monthly LTS/ELTS meeting. Debian ELTS This month was the eighty-sixth ELTS month. During my allocated time I uploaded or worked on: The CVEs for open-vm-tools could be marked as not-affeceted as the corresponding plugin was not yet available. I also attended the monthly LTS/ELTS meeting. Debian Printing This month I uploaded a new upstream version or a bugfix version of: This work is generously funded by Freexian! Debian Astro This month I uploaded a new upstream version or a bugfix version of: Debian IoT This month I uploaded a new upstream version or a bugfix version of: Debian Mobcom This month I uploaded a new upstream version or a bugfix version of: misc The main topic of this month has been gcc15 and cmake4, so my upload rate was extra high. This month I uploaded a new upstream version or a bugfix version of: I wonder what MBF will happen next, I guess the /var/lock-issue will be a good candidate. On my fight against outdated RFPs, I closed 30 of them in September. Meanwhile only 3397 are still open, so don t hesitate to help closing one or another. FTP master This month I accepted 294 and rejected 28 packages. The overall number of packages that got accepted was 294.

Dirk Eddelbuettel: xptr 1.2.0 on CRAN: New(ly Adopted) Package!

Excited to share that xptr is back on CRAN! The xptr package helps to create, check, modify, use, share, external pointer objects. External pointers are used quite extensively throughout R to manage external resources such as datanbase connection objects and alike, and can be very useful to pass pointers to just about any C / C++ data structure around. While described in Writing R Extensions (notably Section 5.13), they can be a little bare-bones and so this package can be useful. It had been created by Randy Lai and maintained by him during 2017 to 2020, but then fell off CRAN. In work with nanoarrow and its clean and minimal Arrow interface xptr came in handy so I adopted it. Several extensions and updates have been added: (compiled) function registration, continuous integration, tests, refreshed and extended documentation as well as a print format extension useful for PyCapsule objects when passing via reticulate. The package documentation site was switched to altdoc driving the most excellent Material for MkDocs framework (providing my first test case of altdoc replacing my older local scripts; I should post some more about that ). The first NEWS entry follow.

Changes in version 1.2.0 (2025-10-03)
  • New maintainer
  • Compiled functions are now registered, .Call() adjusted
  • README.md and DESCRIPTION edited and update
  • Simple unit tests and continunous integration have been added
  • The package documentation site has been recreated using altdoc
  • All manual pages for functions now contain \value sections

For more, see the package page, the git repo or the documentation site.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

Charles: How to Build an Incus Buster Image

It s always nice to have container images of Debian releases to test things, run applications or explore a bit without polluting your host machine. From some Brazilian friends (you know who you are ;-), I ve learned the best way to debug a problem or test a fix is spinning up an incus container, getting to it and finding the minimum reproducer. So the combination incus + Debian is something that I m very used to, but the problem is there are no images for Debian ELTS and testing security fixes to see if they actually fix the vulnerability and don t break anything else is very important. Well, the regular images don t materialize out of thin air, right? So we can learn how they are made and try to generate ELTS images in the same way - shouldn t be that difficult, right? Well, kinda ;-) The images available by default in incus come from images.linuxcontainers.org and are built by Jenkins using distrobuilder. If you follow the links, you will find the repository containing the yaml image definitions used by distrobuilder at github.com/lxc/lxc-ci. With a bit of investigation work, a fork, an incus VM with distrobuilder installed and some magic (also called trial and error) I was able to build a buster image! Whooray, but VM and stretch images are still work in progress. Anyway, I wanted to share how you can build your images and document this process so I don t forget, so here we are

Building Instructions We will use an incus trixie VM to perform the build so we don t clutter our own machine.
incus launch images:debian/trixie <instance-name> --vm
Then let s hop into the machine and install the dependencies.
incus shell <instance-name>
And
apt install git distrobuilder
Let s clone the repository with the yaml definition to build a buster container.
git clone --branch support-debian-buster https://github.com/charles2910/lxc-ci.git
# and cd into it
cd lxc-ci
Then all we need is to pass the correct arguments to distrobuilder so it can build the image. It can output the image in the current directory or in a pre-defined place, so let s create an easy place for the images.
mkdir -p /tmp/images/buster/container
# and perform the build
distrobuilder build-incus images/debian.yaml /tmp/images/buster/container/ \
            -o image.architecture=amd64 \
            -o image.release=buster \
            -o image.variant=default  \
            -o source.url="http://archive.debian.org/debian"
It requires a build definition written in yaml format to perform the build. If you are curious, check the images/ subdir. If all worked correctly, you should have two files in your pre-defined target directory. In our case, /tmp/images/buster/container/ contains:
incus.tar.xz  rootfs.squashfs
Let s copy it to our host so we can add the image to our incus server.
incus file pull <instance-name>/tmp/images/buster/container/incus.tar.xz .
incus file pull <instance-name>/tmp/images/buster/container/rootfs.squashfs .
# and import it as debian/10
incus image import incus.tar.xz rootfs.squashfs --alias debian/10
If we are lucky, we can run our Debian buster container now!
incus launch local:debian/10 <debian-buster-instance>
incus shell <debian-buster-instance>
Well, now all that is left is to install Freexian s ELTS package repository and update the image to get a lot of CVE fixes.
apt install --assume-yes wget
wget https://deb.freexian.com/extended-lts/archive-key.gpg -O /etc/apt/trusted.gpg.d/freexian-archive-extended-lts.gpg
cat <<EOF >/etc/apt/sources.list.d/extended-lts.list
deb http://deb.freexian.com/extended-lts buster-lts main contrib non-free
EOF
apt update
apt --assume-yes upgrade

8 October 2025

Colin Watson: Free software activity in September 2025

About 90% of my Debian contributions this month were sponsored by Freexian. You can also support my work directly via Liberapay or GitHub Sponsors. Some months I feel like I m pedalling furiously just to keep everything in a roughly working state. This was one of those months. Python team I upgraded these packages to new upstream versions: I had to spend a fair bit of time this month chasing down build/test regressions in various packages due to some other upgrades, particularly to pydantic, python-pytest-asyncio, and rust-pyo3: After some upstream discussion I requested removal of pydantic-compat, since it was more trouble than it was worth to keep it working with the latest pydantic version. I filed dh-python: pybuild-plugin-pyproject doesn t know about headers and added it to Python/PybuildPluginPyproject, and converted some packages to pybuild-plugin-pyproject: I updated dh-python to suppress generated dependencies that would be satisfied by python3 >= 3.11. pkg_resources is deprecated. In most cases replacing it is a relatively simple matter of porting to importlib.resources, but packages that used its old namespace package support need more complicated work to port them to implicit namespace packages. We had quite a few bugs about this on zope.* packages, but fortunately upstream did the hard part of this recently. I went round and cleaned up most of the remaining loose ends, with some help from Alexandre Detiste. Some of these aren t completely done yet as they re awaiting new upstream releases: This work also caused a couple of build regressions, which I fixed: I fixed jupyter-client so that its autopkgtests would work in Debusine. I fixed waitress to build with the nocheck profile. I fixed several other build/test failures: I fixed some other bugs: Code reviews Other bits and pieces I fixed several CMake 4 build failures: I got CI for debbugs passing (!22, !23). I fixed a build failure with GCC 15 in trn4. I filed a release-notes bug about the tzdata reorganization in the trixie cycle. I filed and fixed a git-dpm regression with bash 5.3. I upgraded libfilter-perl to a new upstream version. I optimized some code in ubuntu-dev-tools that made O(1) HTTP requests when it could instead make O(n).

3 October 2025

Dirk Eddelbuettel: #053: Adding llvm Snapshots for R Package Testing

Welcome to post 53 in the R4 series. Continuing with posts #51 from Tuesday and #52 from Wednesday and their stated intent of posting some more here is another quick one. Earlier today I helped another package developer who came to the r-package-devel list asking for help with a build error on the Fedora machine at CRAN running recent / development clang. In such situations, the best first step is often to replicate the issue. As I pointed out on the list, the LLVM team behind clang maintains an apt repo at apt.llvm.org/ making it a good resource to add to Debian-based container such as Rocker r-base or the offical r-base (the two are in fact interchangeable, and I take care of both). A small pothole, however, is that the documentation at the top of apt.llvm.org site is a bit stale and behind two aspects that changed on current Debian systems (i.e. unstable/testing as used for r-base). First, apt now prefers files ending in .sources (in a nicer format) and second, it now really requires a key (which is good practice). As it took me a few minutes to regather how to meet both requirements, I reckoned I might as well script this. Et voil the following script does that:
#!/bin/sh

## Update does not hurt but is not strictly needed
#apt update --quiet --quiet
#apt upgrade --yes

## wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key   sudo tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc
## or as we are root in container
wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key > /etc/apt/trusted.gpg.d/apt.llvm.org.asc

cat <<EOF >/etc/apt/sources.list.d/llvm-dev.sources
Types: deb
URIs: http://apt.llvm.org/unstable/
# for clang-21 
#   Suites: llvm-toolchain-21
# for current clang
Suites: llvm-toolchain
Components: main
Signed-By: /etc/apt/trusted.gpg.d/apt.llvm.org.asc
EOF

test -d ~/.R   mkdir ~/.R
cat <<EOF >~/.R/Makevars
CLANGVER=-22
# CLANGLIB=-stdlib=libc++
CXX=clang++\$(CLANGVER) \$(CLANGLIB)
CXX11=clang++\$(CLANGVER) \$(CLANGLIB)
CXX14=clang++\$(CLANGVER) \$(CLANGLIB)
CXX17=clang++\$(CLANGVER) \$(CLANGLIB)
CXX20=clang++\$(CLANGVER) \$(CLANGLIB)
CC=clang\$(CLANGVER)
SHLIB_CXXLD=clang++\$(CLANGVER) \$(CLANGLIB)
EOF

apt update
apt install --yes clang-22
Once the script is run, one can test a package (or set of packages) against clang-22 and clang++-22. This may help R package developers. The script is also generic enough for other development communities who can ignore (or comment-out / delete) the bit about ~/.R/Makevars and deploy the compiler differently. Updating the softlink as apt-preferences does is one way and done in many GitHub Actions recipes. As we only need wget here a basic Debian container should work, possibly with the addition of wget. For R users r-base hits a decent sweet spot.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

1 October 2025

Ben Hutchings: FOSS activity in September 2025

Last month I attended and spoke at Kangrejos, for which I will post a separate report later. Besides that, here s the usual categorised list of work:

Birger Schacht: Status update, September 2025

Regarding Debian packaging this was a rather quiet month. I uploaded version 1.24.0-1 of foot and version 2.8.0-1 of git-quick-stats. I took the opportunity and started migrating my packages to the new version 5 watch file format, which I think is much more readable than the previous format. I also uploaded version 0.1.1-1 of libscfg to NEW. libscfg is a C implementation of the scfg configuration file format and it is a dependency of recent version of kanshi. kanshi is a tool similar to autorandr which allows you define output profiles and kanshi switches to the correct output profile on hotplug events. Once libscfg is in unstable I can finally update kanshi to the latest version. A lot of time this month in finalizing a redesign of the output rendering of carl. carl is a small rust program I wrote that provides a calendar view similar to cal, but it comes with colors and ical file integration. That means that you can not only display a simple calendar, but also colorize/highlight dates based on various attributes or based on events on that day. In the initial versions of carl the output rendering was simply hardcoded into the app.
Screenshot of carl
This was a bit cumbersome to maintain and not configurable for users. I am using templating languages on a daily basis, so I decided I would reimplement the output generation of carl to use templates. I chose the minijinja Rust library which is based on the syntax and behavior of the Jinja2 template engine for Python . There are others out there, like tera, but minijinja seems to be more active in development currently. I worked on this implementation on and off for the last year and finally had the time to finish it up and write some additional tests for the outputs. It is easier to maintain templates than Rust code that uses write!() to format the output. I also implemented a configuration option for users to override the templates. Additional to the output refactoring I also fixed couple of bugs and finally released v0.4.0 of carl. In my dayjob I released version 0.53 of apis-core-rdf which contains the place lookup field which I implemented in August. A couple of weeks later we released version 0.54 which comes with a middleware to show pass on messages from the Django messages framework via response header to HTMX to trigger message popups. This implementation is based on the blog post Using the Django messages framework with HTMX. Version 0.55 was the last release in September. It contained preparations for refactoring the import logic as well as a couple of UX improvements.

30 September 2025

Jonathan McDowell: Local Voice Assistant Step 5: Remote Satellite

The last (software) piece of sorting out a local voice assistant is tying the openWakeWord piece to a local microphone + speaker, and thus back to Home Assistant. For that we use wyoming-satellite. I ve packaged that up - https://salsa.debian.org/noodles/wyoming-satellite - and then to run I do something like:
$ wyoming-satellite --name 'Living Room Satellite' \
    --uri 'tcp://[::]:10700' \
    --mic-command 'arecord -r 16000 -c 1 -f S16_LE -t raw -D plughw:CARD=CameraB409241,DEV=0' \
    --snd-command 'aplay -D plughw:CARD=UACDemoV10,DEV=0 -r 22050 -c 1 -f S16_LE -t raw' \
    --wake-uri tcp://[::1]:10400/ \
    --debug
That starts us listening for connections from Home Assistant on port 10700, uses the openWakeWord instance on localhost port 10400, uses aplay/arecord to talk to the local microphone and speaker, and gives us some debug output so we can see what s going on. And it turns out we need the debug. This setup is a bit too flaky for it to have ended up in regular use in our household. I ve had some problems with reliable audio setup; you ll note the Python is calling out to other tooling to grab audio, which feels a bit clunky to me and I don t think is the actual problem, but the main audio for this host is hooked up to the TV (it s a media box), so the setup for the voice assistant needs to be entirely separate. That means not plugging into Pipewire or similar, and instead giving direct access to wyoming-satellite. And sometimes having to deal with how to make the mixer happy + non-muted manually. I ve also had some issues with the USB microphone + speaker; I suspect a powered USB hub would help, and that s on my list to try out. When it does work I have sometimes found it necessary to speak more slowly, or enunciate my words more clearly. That s probably something I could improve by switching from the base.en to small.en whisper.cpp model, but I m waiting until I sort out the audio hardware issue before poking more. Finally, the wake word detection is a little bit sensitive sometimes, as I mentioned in the previous post. To be honest I think it s possible to deal with that, if I got the rest of the pieces working smoothly. This has ended up sounding like a more negative post than I meant it to. Part of the issue in a resolution is finding enough free time to poke things (especially as it involves taking over the living room and saying Hey Jarvis a lot), part of it is no doubt my desire to actually hook up the pieces myself and understand what s going on. Stay tuned and see if I ever manage to resolve it all!

Russ Allbery: Review: Deep Black

Review: Deep Black, by Miles Cameron
Series: Arcana Imperii #2
Publisher: Gollancz
Copyright: 2024
ISBN: 1-3996-1506-8
Format: Kindle
Pages: 509
Deep Black is a far-future science fiction novel and the direct sequel to Artifact Space. You do not want to start here. I regretted not reading the novels closer together and had to refresh my memory of what happened in the first book. The shorter fiction in Beyond the Fringe takes place between the two series novels and leads into some of the events in this book, although reading it is optional. Artifact Space left Marca Nbaro at the farthest point of the voyage of the Greatship Athens, an unexpected heroine and now well-integrated into the crew. On a merchant ship, however, there's always more work to be done after a heroic performance. Deep Black opens with that work: repairs from the events of the first book, the never-ending litany of tasks required to keep the ship running smoothly, and of course the trade with aliens that drew them so far out into the Deep Black. We knew early in the first book that this wouldn't be the simple, if long, trading voyage that most of the crew of the Athens was expecting, but now they have to worry about an unsettling second group of aliens on top of a potential major war between human factions. They don't yet have the cargo they came for, they have to reconstruct their trading post, and they're a very long way from home. Marca also knows, at this point in the story, that this voyage had additional goals from the start. She will slowly gain a more complete picture of those goals during this novel. Artifact Space was built around one of the most satisfying plots in military science fiction (at least to me): a protagonist who benefits immensely from the leveling effect and institutional inclusiveness of the military slowly discovering that, when working at its best, the military can be a true meritocracy. (The merchant marine of the Athens is not military, precisely, since it's modeled on the trading ships of Venice, but it's close enough for the purposes of this plot.) That's not a plot that lasts into a sequel, though, so Cameron had to find a new spine for the second half of the story. He chose first contact (of a sort) and space battle. The space battle parts are fine. I read a ton of children's World War II military fiction when I was a boy, and I always preferred the naval battles to the land battles. This part of Deep Black reminded me of those naval battles, particularly a book whose title escapes me about the Arctic convoys to the Soviet Union. I'm more interested in character than military adventure these days, but every once in a while I enjoy reading about a good space battle. This was not an exemplary specimen of the genre, but it delivered on all the required elements. The first contact part was more original, in part because Cameron chose an interesting medium ground between total incomprehensibility and universal translators. He stuck with the frustrations of communication for considerably longer than most SF authors are willing to write, and it worked for me. This is the first book I've read in a while where superficial alien fluency with the mere words of a human language masks continuing profound mutual incomprehension. The communication difficulties are neither malicious nor a setup for catastrophic misunderstanding, but an intrinsic part of learning about a truly alien species. I liked this, even though it makes for slower and more frustrating progress. It felt more believable than a lot of first contact, and it forced the characters to take risks and act on hunches and then live with the consequences. One of the other things that Cameron does well is maintain the steady rhythm of life on a working ship as a background anchor to the story. I've read a lot of science fiction that shows the day-to-day routine only until something more interesting and plot-focused starts happening and then seems to forget about it entirely. Not here. Marca goes through intense and adrenaline-filled moments requiring risk and fast reactions, and then has to handle promotion write-ups, routine watches, and studying for advancement. Cameron knows that real battles involve long periods of stressful waiting and incorporates them into the book without making them too boring, which requires a lot of writing skill. I prefer the emotional magic of finding a place where one belongs, so I was not as taken with Deep Black as I was with Artifact Space, but that's the inevitable result of plot progression and not really a problem with this book. Marca is absurdly central to the story in ways that have a whiff of "chosen one" dynamics, but if one can suspend one's disbelief about that, the rest of the book is solid. This is, fundamentally, a book about large space battles, so save it when you're in the mood for that sort of story, but it was a satisfying continuation of the series. I will definitely keep reading. Recommended if you enjoyed Artifact Space. If you didn't, Deep Black isn't going to change your mind. Followed by Whalesong, which is not yet released (and is currently in some sort of limbo for pre-orders in the US, which I hope will clear up). Rating: 7 out of 10

25 September 2025

kpcyrd: Release: rebuilderd v0.25.0

rebuilderd v0.25.0 was recently released, this version has improved in-toto support for cryptographic attestations that this blog post briefly outlines. As a quick recap, rebuilderd is an automatic build scheduler that emerged in 2019/2020 from the Reproducible Builds project doing the following:
  1. Track binary packages available in a Linux distribution
  2. Attempt to compile the official binary packages from their (alleged) source code
  3. Check if the package we compiled is bit-for-bit identical
    1. If so, mark it GOOD, issue an attestation
    2. In every other case, mark it BAD, generate a diffoscope
The binary packages in question are explicitly the packages users would also fetch and install. This project has caught the attention of Arch Linux, Debian and Fedora.

Before this version The original in-toto integration was added 4 years ago by Joy Liu during GSoC 2021, with help from Santiago Torres and Aditya Sirish (shoutout to the real ones!). Each rebuilderd-worker had its own cryptographic key and included a signed attestation along with the build result that could then be fetched from /api/v0/builds/ id /attestation. Since these workers are potentially ephemeral, and the list of worker public keys wasn t publicly known, it was difficult to make use of those signatures.

Since this version This version introduces the following:
  1. The rebuilderd daemon itself generates a long-term signing key
  2. All attestations signed by a trusted worker also get signed by the rebuilderd daemon
  3. The rebuilderd daemon gets a new endpoint that can be used to query the public-key this instance identifies with: /api/v0/public-keys
An example of this new endpoint can be found here: https://reproducible.archlinux.org/api/v0/public-keys The response looks something like this (this is the real long-term signing key used by reproducible.archlinux.org):
 
    "current": [
        "-----BEGIN PUBLIC KEY-----\r\nMCwwBwYDK2VwBQADIQBLNcEcgErQ1rZz9oIkUnzc3fPuqJEALr22rNbrBK7iqQ==\r\n-----END PUBLIC KEY-----\r\n"
    ]
 
It s a list so keys can potentially be rolled over time, and in future versions it should also list the public keys the instance has used in the past. I haven t develop any integrations for this yet (partially also to allow deployments to catch up with the new release), but I m planning to do so using the in-toto crate.

Closing words To give credit where credit is due (and because people pointed out I tend to end my blog posts too abruptly), rebuilderd is only the scheduler software, the actual build in the correct build-environment is outsourced to external tooling like archlinux-repro and debrebuild. For further reading on applied reproducible builds, see also my previous blogpost Disagreeing rebuilders and what that means. Also, there are currently efforts by the European Commission to outlaw unregulated end-to-end encrypted chat, so this may be a good time to prepare for (potential) impact and check what tools you have available to reduce unchecked trust in (open source) software authorities, to keep them operating honest and accountable. Never lose the plot~ Sincerely yours

24 September 2025

Matthew Garrett: Investigating a forged PDF

I had to rent a house for a couple of months recently, which is long enough in California that it pushes you into proper tenant protection law. As landlords tend to do, they failed to return my security deposit within the 21 days required by law, having already failed to provide the required notification that I was entitled to an inspection before moving out. Cue some tedious argumentation with the letting agency, and eventually me threatening to take them to small claims court.

This post is not about that.

Now, under Californian law, the onus is on the landlord to hold and return the security deposit - the agency has no role in this. The only reason I was talking to them is that my lease didn't mention the name or address of the landlord (another legal violation, but the outcome is just that you get to serve the landlord via the agency). So it was a bit surprising when I received an email from the owner of the agency informing me that they did not hold the deposit and so were not liable - I already knew this.

The odd bit about this, though, is that they sent me another copy of the contract, asserting that it made it clear that the landlord held the deposit. I read it, and instead found a clause reading SECURITY: The security deposit will secure the performance of Tenant s obligations. IER may, but will not be obligated to, apply all portions of said deposit on account of Tenant s obligations. Any balance remaining upon termination will be returned to Tenant. Tenant will not have the right to apply the security deposit in payment of the last month s rent. Security deposit held at IER Trust Account., where IER is International Executive Rentals, the agency in question. Why send me a contract that says you hold the money while you're telling me you don't? And then I read further down and found this:
Text reading ENTIRE AGREEMENT: The foregoing constitutes the entire agreement between the parties and may bemodified only in writing signed by all parties. This agreement and any modifications, including anyphotocopy or facsimile, may be signed in one or more counterparts, each of which will be deemed anoriginal and all of which taken together will constitute one and the same instrument. The followingexhibits, if checked, have been made a part of this Agreement before the parties  execution: Exhibit 1:Lead-Based Paint Disclosure (Required by Law for Rental Property Built Prior to 1978) Addendum 1 The security deposit will be held by (name removed) and applied, refunded, or forfeited in accordance with the terms of this lease agreement.
Ok, fair enough, there's an addendum that says the landlord has it (I've removed the landlord's name, it's present in the original).

Except. I had no recollection of that addendum. I went back to the copy of the contract I had and discovered:
The same text as the previous picture, but addendum 1 is empty
Huh! But obviously I could just have edited that to remove it (there's no obvious reason for me to, but whatever), and then it'd be my word against theirs. However, I'd been sent the document via RightSignature, an online document signing platform, and they'd added a certification page that looked like this:
A Signature Certificate, containing a bunch of data about the document including a checksum or the original
Interestingly, the certificate page was identical in both documents, including the checksums, despite the content being different. So, how do I show which one is legitimate? You'd think given this certificate page this would be trivial, but RightSignature provides no documented mechanism whatsoever for anyone to verify any of the fields in the certificate, which is annoying but let's see what we can do anyway.

First up, let's look at the PDF metadata. pdftk has a dump_data command that dumps the metadata in the document, including the creation date and the modification date. My file had both set to identical timestamps in June, both listed in UTC, corresponding to the time I'd signed the document. The file containing the addendum? The same creation time, but a modification time of this Monday, shortly before it was sent to me. This time, the modification timestamp was in Pacific Daylight Time, the timezone currently observed in California. In addition, the data included two ID fields, ID0 and ID1. In my document both were identical, in the one with the addendum ID0 matched mine but ID1 was different.

These ID tags are intended to be some form of representation (such as a hash) of the document. ID0 is set when the document is created and should not be modified afterwards - ID1 initially identical to ID0, but changes when the document is modified. This is intended to allow tooling to identify whether two documents are modified versions of the same document. The identical ID0 indicated that the document with the addendum was originally identical to mine, and the different ID1 that it had been modified.

Well, ok, that seems like a pretty strong demonstration. I had the "I have a very particular set of skills" conversation with the agency and pointed these facts out, that they were an extremely strong indication that my copy was authentic and their one wasn't, and they responded that the document was "re-sealed" every time it was downloaded from RightSignature and that would explain the modifications. This doesn't seem plausible, but it's an argument. Let's go further.

My next move was pdfalyzer, which allows you to pull a PDF apart into its component pieces. This revealed that the documents were identical, other than page 3, the one with the addendum. This page included tags entitled "touchUp_TextEdit", evidence that the page had been modified using Acrobat. But in itself, that doesn't prove anything - obviously it had been edited at some point to insert the landlord's name, it doesn't prove whether it happened before or after the signing.

But in the process of editing, Acrobat appeared to have renamed all the font references on that page into a different format. Every other page had a consistent naming scheme for the fonts, and they matched the scheme in the page 3 I had. Again, that doesn't tell us whether the renaming happened before or after the signing. Or does it?

You see, when I completed my signing, RightSignature inserted my name into the document, and did so using a font that wasn't otherwise present in the document (Courier, in this case). That font was named identically throughout the document, except on page 3, where it was named in the same manner as every other font that Acrobat had renamed. Given the font wasn't present in the document until after I'd signed it, this is proof that the page was edited after signing.

But eh this is all very convoluted. Surely there's an easier way? Thankfully yes, although I hate it. RightSignature had sent me a link to view my signed copy of the document. When I went there it presented it to me as the original PDF with my signature overlaid on top. Hitting F12 gave me the network tab, and I could see a reference to a base.pdf. Downloading that gave me the original PDF, pre-signature. Running sha256sum on it gave me an identical hash to the "Original checksum" field. Needless to say, it did not contain the addendum.

Why do this? The only explanation I can come up with (and I am obviously guessing here, I may be incorrect!) is that International Executive Rentals realised that they'd sent me a contract which could mean that they were liable for the return of my deposit, even though they'd already given it to my landlord, and after realising this added the addendum, sent it to me, and assumed that I just wouldn't notice (or that, if I did, I wouldn't be able to prove anything). In the process they went from an extremely unlikely possibility of having civil liability for a few thousand dollars (even if they were holding the deposit it's still the landlord's legal duty to return it, as far as I can tell) to doing something that looks extremely like forgery.

There's a hilarious followup. After this happened, the agency offered to do a screenshare with me showing them logging into RightSignature and showing the signed file with the addendum, and then proceeded to do so. One minor problem - the "Send for signature" button was still there, just below a field saying "Uploaded: 09/22/25". I asked them to search for my name, and it popped up two hits - one marked draft, one marked completed. The one marked completed? Didn't contain the addendum.

comment count unavailable comments

Philipp Kern: PSA: APT::Default-Release might be holding back updates from you

If you are like me that you are installing machines with testing and then go and flip them over to the current stable for a while using APT::Default-Release, you might not be receiving all relevant updates. In fact this setting is kind of discouraged in favor of more extensive pinning configuration.However, the field does support regexps, so instead of just specifying, say, "trixie", you can put this in place: APT::Default-Release "/^trixie( -security -proposed-updates -updates)$/";That should bring the security and stable updates back in.It feels like we are recently learning a lot about the drawbacks of these overlays and how they need to be configured properly...

23 September 2025

Ravi Dwivedi: Singapore Trip

In December 2024, I went on a trip through four countries - Singapore, Malaysia, Brunei, and Vietnam - with my friend Badri. This post covers our experiences in Singapore. I took an IndiGo flight from Delhi to Singapore, with a layover in Chennai. At the Chennai airport, I was joined by Badri. We had an early morning flight from Chennai that would land in Singapore in the afternoon. Within 48 hours of our scheduled arrival in Singapore, we submitted an arrival card online. At immigration, we simply needed to scan our passports at the gates, which opened automatically to let us through, and then give our address to an official nearby. The process was quick and smooth, but it unfortunately meant that we didn t get our passports stamped by Singapore. Before I left the airport, I wanted to visit the nature-themed park with a fountain I saw in pictures online. It is called Jewel Changi, and it took quite some walking to get there. After reaching the park, we saw a fountain that could be seen from all the levels. We roamed around for a couple of hours, then proceeded to the airport metro station to get to our hotel.
Jewel Changi A shot of Jewel Changi. Photo by Ravi Dwivedi. Released under the CC-BY-SA 4.0.
There were four ATMs on the way to the metro station, but none of them provided us with any cash. This was the first country (outside India, of course!) where my card didn t work at ATMs. To use the metro, one can tap the EZ-Link card or bank cards at the AFC gates to get in. You cannot buy tickets using cash. Before boarding the metro, I used my credit card to get Badri an EZ-Link card from a vending machine. It was 10 Singapore dollars ( 630) - 5 for the card, and 5 for the balance. I had planned to use my Visa credit card to pay for my own fare. I was relieved to see that my card worked, and I passed through the AFC gates. We had booked our stay at a hostel named Campbell s Inn, which was the cheapest we could find in Singapore. It was 1500 per night for dorm beds. The hostel was located in Little India. While Little India has an eponymous metro station, the one closest to our hostel was Rochor. On the way to the hostel, we found out that our booking had been canceled. We had booked from the Hostelworld website, opting to pay the deposit in advance and to pay the balance amount in person upon reaching. However, Hostelworld still tried to charge Badri s card again before our arrival. When the unauthorized charge failed, they sent an automatic message saying we tried to charge and to contact them soon to avoid cancellation, which we couldn t do as we were in the plane. Despite this, we went to the hostel to check the status of our booking. The trip from the airport to Rochor required a couple of transfers. It was 2 Singapore dollars (approx. 130) and took approximately an hour. Upon reaching the hostel, we were informed that our booking had indeed been canceled, and were not given any reason for the cancelation. Furthermore, no beds were available at the hostel for us to book on the spot. We decided to roam around and look for accommodation at other hostels in the area. Soon, we found a hostel by the name of Snooze Inn, which had two beds available. It was 36 Singapore dollars per person (around 2300) for a dormitory bed. Snooze Inn advertised supporting RuPay cards and UPI. Some other places in that area did the same. We paid using my card. We checked in and slept for a couple of hours after taking a shower. By the time we woke up, it was dark. We met Praveen s friend Sabeel to get my FLX1 phone. We also went to Mustafa Center nearby to exchange Indian rupees for Singapore dollars. Mustafa Center also had a shopping center with shops selling electronic items and souvenirs, among other things. When we were dropping off Sabeel at a bus stop, we discovered that the bus stops in Singapore had a digital board mentioning the bus routes for the stop and the number of minutes each bus was going to take. In addition to an organized bus system, Singapore had good pedestrian infrastructure. There were traffic lights and zebra crossings for pedestrians to cross the roads. Unlike in Indian cities, rules were being followed. Cars would stop for pedestrians at unmanaged zebra crossings; pedestrians would in turn wait for their crossing signal to turn green before attempting to walk across. Therefore, walking in Singapore was easy. Traffic rules were taken so seriously in Singapore I (as a pedestrian) was afraid of unintentionally breaking them, which could get me in trouble, as breaking rules is dealt with heavy fines in the country. For example, crossing roads without using a marked crossing (while being within 50 meters of it) - also known as jaywalking - is an offence in Singapore. Moreover, the streets were litter-free, and cleanliness seemed like an obsession. After exploring Mustafa Center, we went to a nearby 7-Eleven to top up Badri s EZ-Link card. He gave 20 Singapore dollars for the recharge, which credited the card by 19.40 Singapore dollars (0.6 dollars being the recharge fee). When I was planning this trip, I discovered that the World Chess Championship match was being held in Singapore. I seized the opportunity and bought a ticket in advance. The next day - the 5th of December - I went to watch the 9th game between Gukesh Dommaraju of India and Ding Liren of China. The venue was a hotel on Sentosa Island, and the ticket was 70 Singapore dollars, which was around 4000 at the time. We checked out from our hostel in the morning, as we were planning to stay with Badri s aunt that night. We had breakfast at a place in Little India. Then we took a couple of buses, followed by a walk to Sentosa Island. Paying the fare for the buses was similar to the metro - I tapped my credit card in the bus, while Badri tapped his EZ-Link card. We also had to tap it while getting off. If you are tapping your credit card to use public transport in Singapore, keep in mind that the total amount of all the trips taken on a day is deducted at the end. This makes it hard to determine the cost of individual trips. For example, I could take a bus and get off after tapping my card, but I would have no way to determine how much this journey cost. When you tap in, the maximum fare amount gets deducted. When you tap out, the balance amount gets refunded (if it s a shorter journey than the maximum fare one). So, there is incentive for passengers not to get off without tapping out. Going by your card statement, it looks like all that happens virtually, and only one statement comes in at the end. Maybe this combining only happens for international cards. We got off the bus a kilometer away from Sentosa Island and walked the rest of the way. We went on the Sentosa Boardwalk, which is itself a tourist attraction. I was using Organic Maps to navigate to the hotel Resorts World Sentosa, but Organic Maps route led us through an amusement park. I tried asking the locals (people working in shops) for directions, but it was a Chinese-speaking region, and they didn t understand English. Fortunately, we managed to find a local who helped us with the directions.
Sentosa Boardwalk A shot of Sentosa Boardwalk. Photo by Ravi Dwivedi. Released under the CC-BY-SA 4.0.
Following the directions, we somehow ended up having to walk on a road which did not have pedestrian paths. Singapore is a country with strict laws, so we did not want to walk on that road. Avoiding that road led us to the Michael Hotel. There was a person standing at the entrance, and I asked him for directions to Resorts World Sentosa. The person told me that the bus (which was standing at the entrance) would drop me there! The bus was a free service for getting to Resorts World Sentosa. Here I parted ways with Badri, who went to his aunt s place. I got to the Resorts Sentosa and showed my ticket to get in. There were two zones inside - the first was a room with a glass wall separating the audience and the players. This was the room to watch the game physically, and resembled a zoo or an aquarium. :) The room was also a silent room, which means talking or making noise was prohibited. Audiences were only allowed to have mobile phones for the first 30 minutes of the game - since I arrived late, I could not bring my phone inside that room. The other zone was outside this room. It had a big TV on which the game was being broadcast along with commentary by David Howell and Jovanka Houska - the official FIDE commentators for the event. If you don t already know, FIDE is the authoritative international chess body. I spent most of the time outside that silent room, giving me an opportunity to socialize. A lot of people were from Singapore. I saw there were many Indians there as well. Moreover, I had a good time with Vasudevan, a journalist from Tamil Nadu who was covering the match. He also asked questions to Gukesh during the post-match conference. His questions were in Tamil to lift Gukesh s spirits, as Gukesh is a Tamil speaker. Tea and coffee were free for the audience. I also bought a T-shirt from their stall as a souvenir. After the game, I took a shuttle bus from Resorts World Sentosa to a metro station, then travelled to Pasir Ris by metro, where Badri was staying with his aunt. I thought of getting something to eat, but could not find any caf s or restaurants while I was walking from the Pasir Ris metro station to my destination, and was positively starving when I got there. Badri s aunt s place was an apartment in a gated community. On the gate was a security guard who asked me the address of the apartment. Upon entering, there were many buildings. To enter the building, you need to dial the number of the apartment you want to go to and speak to them. I had seen that in the TV show Seinfeld, where Jerry s friends used to dial Jerry to get into his building. I was afraid they might not have anything to eat because I told them I was planning to get something on the way. This was fortunately not the case, and I was relieved to not have to sleep with an empty stomach. Badri s uncle gave us an idea of how safe Singapore is. He said that even if you forget your laptop in a public space, you can go back the next day to find it right there in the same spot. I also learned that owning cars was discouraged in Singapore - the government imposes a high registration fee on them, while also making public transport easy to use and affordable. I also found out that 7-Eleven was not that popular among residents in Singapore, unlike in Malaysia or Thailand. The next day was our third and final day in Singapore. We had a bus in the evening to Johor Bahru in Malaysia. We got up early, had breakfast, and checked out from Badri s aunt s home. A store by the name of Cat Socrates was our first stop for the day, as Badri wanted to buy some stationery. The plan was to take the metro, followed by the bus. So we got to Pasir Ris metro station. Next to the metro station was a mall. In the mall, Badri found an ATM where our cards worked, and we got some Singapore dollars. It was noon when we reached the stationery shop mentioned above. We had to walk a kilometer from the place where the bus dropped us. It was a hot, sunny day in Singapore, so walking was not comfortable. We had to go through residential areas in Singapore. We saw some non-touristy parts of Singapore. After we were done with the stationery shop, we went to a hawker center to get lunch. Hawker centers are unique to Singapore. They have a lot of shops that sell local food at cheap prices. It is similar to a food court. However, unlike the food courts in malls, hawker centers are open-air and can get quite hot.
Jewel Changi This is the hawker center we went to. Photo by Ravi Dwivedi. Released under the CC-BY-SA 4.0.
To have something, you just need to buy it from one of the shops and find a table. After you are done, you need to put your tray in the tray-collecting spots. I had a kaya toast with chai, since there weren t many vegetarian options. I also bought a persimmon from a nearby fruit vendor. On the other hand, Badri sampled some local non-vegetarian dishes.
A sign saying, 'No table littering, by law.' Table littering at the hawker center was prohibited by law. Photo by Ravi Dwivedi. Released under the CC-BY-SA 4.0.
Next, we took a metro to Raffles Place, as we wanted to visit Merlion, the icon of Singapore. It is a statue having the head of a lion and the body of a fish. While getting through the AFC gates, my card was declined. Therefore, I had to buy an EZ-Link card, which I had been avoiding because the card itself costs 5 Singapore dollars. From the Raffles Place metro station, we walked to Merlion. The place also gave a nice view of Marina Bay Sands. It was filled with tourists clicking pictures, and we also did the same.
Merlion from behind Merlion from behind, giving a good view of Marina Bay Sands. Photo by Ravi Dwivedi. Released under the CC-BY-SA 4.0.
After this, we went to the bus stop to catch our bus to the border city of Johor Bahru, Malaysia. The bus was more than an hour late, and we worried that we had missed the bus. I asked an Indian woman at the stop who also planned to take the same bus, and she told us that the bus was late. Finally, our bus arrived, and we set off for Johor Bahru. Before I finish, let me give you an idea of my expenditure. Singapore is an expensive country, and I realized that expenses could go up pretty quickly. Overall, my stay in Singapore for 3 days and 2 nights was approx. 5500 rupees. That too, when we stayed one night at Badri s aunt s place (so we didn t have to pay for accomodation for one of the nights) and didn t have to pay for a couple of meals. This amount doesn t include the ticket for the chess game, but includes the costs of getting there. If you are in Singapore, it is likely you will pay a visit to Sentosa Island anyway. Stay tuned for our experiences in Malaysia! Credits: Thanks to Dione, Sahil, Badri and Contrapunctus for reviewing the draft. Thanks to Bhe for spotting a duplicate sentence.

Next.

Previous.