Search Results: "edd"

2 January 2026

Birger Schacht: Status update, December 2025

December 2025 started off with a nice event, namely a small gathering of Vienna based DDs. Some of us were at DebConf25 in Brest and we thought it might be nice to have a get-together of DDs in Vienna. A couple of months after DebConf25 I picked up the idea, let someone else ping the DDs, booked a table at a local cafe and in the end we were a group of 6 DDs. It was nice to put faces to names, names to nicknames and to hear what people are up to. We are definitely planning to repeat that! December also ended with a meeting of nerds: the 39th Chaos Communication Congress in Hamburg. As usual, I did not really have that much time to watch many talks. I tend to bookmark a lot of them in the scheduling app in advance, but once I m at the congress the social aspect is much more important and I try to only attend workshop or talks that are not recorded. Watching the recordings afterward is possible anyway (and I actually try to do that!). There was also a Debian Developers meetup at day 3, combined with the usual time confusion regarding UTC and CET. We talked about having a Debian table at 40c3, so maybe the timezone won t be that much of a problem in the next time. Two talks I recommend are CSS Clicker Training: Making games in a styling language and To sign or not to sign: Practical vulnerabilities in GPG & friends. Regarding package uploads this month did not happen that much, I only uploaded the new version (0.9.3) of labwc. I created two new releases for carl. First a 0.5 release that adds Today and SpecifiedDate as properties. I forwarded an issue about dates not being parsed correctly to the icalendar issue tracker and this was fixed a couple of days later (thanks!). I then created a 0.5.1 release containing that fix. I also started planning to move the carl repository back to codeberg, because Github feels more and more like an AI Slop platform. The work on debiverse also continued. I removed the tailwind CSS framework, and it was actually not that hard to reproduce all the needed CSS classes with custom CSS. I think that CSS frameworks make sense to a point, but once you start implementing stuff that the framework does not provide, it is easier if everything comes out of one set of rules. There was also the article Vanilla CSS is all you need which goes into the same direction and which gave me some ideas how to organize the CSS directives. I also refactored the filter generation for the listing filters and the HTML filter form is now generated from the FastAPI Query Parameter Model. Screenshot of the filter form For navigation I implemented a sidebar, that is hidden on small screens but can be toggled using a burger menu. Screenshot of the navigation bar I also stumbled upon An uncomfortable but necessary discussion about the Debian bug tracker, which raises some valid points. I think debiverse could be a solution to the first point of What could be a way forward? , namely: Create a new web service that parses the existing bug data and displays it in a rich format . But if there is ever another way than email to interact with bugs.debian.org, than this approach should not rely on passing on the commands via mail. If I click a button in a web interface to raise the severity, the severity should be raised right away - not 10 minutes later when the email is received. I think the individual parts (web, database, mail interface) should be decoupled and talk to each other via APIs.

1 January 2026

Russ Allbery: 2025 Book Reading in Review

In 2025, I finished and reviewed 32 books, not counting another five books I've finished but not yet reviewed and which will therefore roll over to 2026. This was not a great reading year, although not my worst reading year since I started keeping track. I'm not entirely sure why, although part of the explanation was that I hit a bad stretch of books in spring of 2025 and got into a bit of a reading slump. Mostly, though, I shifted a lot of reading this year to short non-fiction (newsletters and doom-scrolling) and spent rather more time than I intended watching YouTube videos, and sadly each hour in the day can only be allocated one way. This year felt a bit like a holding pattern. I have some hopes of being more proactive and intentional in 2026. I'm still working on finding a good balance between all of my hobbies and the enjoyment of mindless entertainment. The best book I read this year was also the last book I reviewed (and yes, I snuck the review under the wire for that reason): Bethany Jacobs's This Brutal Moon, the conclusion of the Kindom Trilogy that started with These Burning Stars. I thought the first two books of the series were interesting but flawed, but the conclusion blew me away and improved the entire trilogy in retrospect. Like all books I rate 10 out of 10, I'm sure a large part of my reaction is idiosyncratic, but two friends of mine also loved the conclusion so it's not just me. The stand-out non-fiction book of the year was Rory Stewart's Politics on the Edge. I have a lot of disagreements with Stewart's political positions (the more I listen to him, the more disagreements I find), but he is an excellent memoirist who skewers the banality, superficiality, and contempt for competence that has become so prevailing in centrist and right-wing politics. It's hard not to read this book and despair of electoralism and the current structures of governments, but it's bracing to know that even some people I disagree with believe in the value of expertise. I also finished Suzanne Palmer's excellent Finder Chronicles series, reading The Scavenger Door and Ghostdrift. This series is some of the best science fiction I've read in a long time and I'm sad it is over (at least for now). Palmer has a new, unrelated book coming in 2026 (Ode to the Half-Broken), and I'm looking forward to reading that. This year, I experimented with re-reading books I had already reviewed for the first time since I started writing reviews. After my reading slump, I felt like revisiting something I knew I liked, and therefore re-read C.J. Cherryh's Cyteen and Regenesis. Cyteen mostly held up, but Regenesis was worse than I had remembered. I experimented with a way to add on to my previous reviews, but I didn't like the results and the whole process of re-reading and re-reviewing annoyed me. I'm counting this as a failed experiment, which means I've still not solved the problem of how to revisit series that I read long enough ago that I want to re-read them before picking up the new book. (You may have noticed that I've not read the new Jacqueline Carey Kushiel novel, for example.) You may have also noticed that I didn't start a new series re-read, or continue my semi-in-progress re-reads of Mercedes Lackey or David Eddings. I have tentative plans to kick off a new series re-read in 2026, but I'm not ready to commit to that yet. As always, I have no firm numeric goals for the next year, but I hope to avoid another reading slump and drag my reading attention back from lower-quality and mostly-depressing material in 2026. The full analysis includes some additional personal reading statistics, probably only of interest to me.

31 December 2025

kpcyrd: 2025 wrapped

Same as last year, this is a summary of what I ve been up to throughout the year. See also the recap/retrospection published by my friends (antiz, jvoisin, orhun). Thanks to everybody who has been part of my human experience, past or present. Especially those who ve been closest.

22 December 2025

Jonathan McDowell: NanoKVM: I like it

I bought a NanoKVM. I d heard some of the stories about how terrible it was beforehand, and some I didn t learn about until afterwards, but at 52, including VAT + P&P, that seemed like an excellent bargain for something I was planning to use in my home network environment. Let s cover the bad press first. apalrd did a video, entitled NanoKVM: The S stands for Security (Armen Barsegyan has a write up recommending a PiKVM instead that lists the objections raised in the video). Matej Kova i wrote an article about the hidden microphone on a Chinese NanoKVM. Various other places have picked up both of these and still seem to be running with them, 10 months later. Next, let me explain where I m coming from here. I have over 2 decades of experience with terrible out-of-band access devices. I still wince when I think of the Sun Opteron servers that shipped with an iLOM that needed a 32-bit Windows browser in order to access it (IIRC some 32 bit binary JNI blob). It was a 64 bit x86 server from a company who, at the time, still had a major non-Windows OS. Sheesh. I do not assume these devices are fit for exposure to the public internet, even if they come from reputable vendors. Add into that the fact the NanoKVM is very much based on a development board (the LicheeRV Nano), and I felt I knew what I was getting into here. And, as a TL;DR, I am perfectly happy with my purchase. Sipeed have actually dealt with a bunch of apalrd s concerns (GitHub ticket), which I consider to be an impressive level of support for this price point. Equally the microphone is explained by the fact this is a 52 device based on a development board. You re giving it USB + HDMI access to a host on your network, if you re worried about the microphone then you re concentrating on the wrong bit here. I started out by hooking the NanoKVM up to my Raspberry Pi classic, which I use as a serial console / network boot tool for working on random bits of hardware. That meant the NanoKVM had no access to the outside world (the Pi is not configured to route, or NAT, for the test network interface), and I could observe what went on. As it happens you can do an SSH port forward of port 80 with this sort of setup and it all works fine - no need for the NanoKVM to have any external access, and it copes happily with being accessed as http://localhost:8000/ (though you do need to choose MJPEG as the video mode, more forwarding or enabling HTTPS is needed for an H.264 WebRTC session). IPv6 is enabled in the kernel. My test setup doesn t have a router advertisements configured, but I could connect to the web application over the v6 link-local address that came up automatically. My device reports:
Image version:              v1.4.1
Application version:        2.2.9
That s recent, but the GitHub releases page has 2.3.0 listed as more recent. Out of the box it s listening on TCP port 80. SSH is not running, but there s a toggle to turn it on and the web interface offers a web based shell (with no extra authentication over the normal login). On first use I was asked to set a username + password. Default access, as you d expect from port 80, is HTTP, but there s a toggle to enable HTTPS. It generates a self signed certificate - for me it had the CN localhost but that might have been due to my use of port forwarding. Enabling HTTPS does not disable HTTP, but HTTP just redirects to the HTTPS URL. As others have discussed it does a bunch of DNS lookups, primarily for NTP servers but also for cdn.sipeed.com. The DNS servers are hard coded:
~ # cat /etc/resolv.conf
nameserver 192.168.0.1
nameserver 8.8.4.4
nameserver 8.8.8.8
nameserver 114.114.114.114
nameserver 119.29.29.29
nameserver 223.5.5.5
This is actually restored on boot from /boot/resolv.conf, so if you want changes to persist you can just edit that file. NTP is configured with a standard set of pool.ntp.org services in /etc/ntp.conf (this does not get restored on reboot, so can just be edited in place). I had dnsmasq on the Pi setup to hand out DNS + NTP servers, but both were ignored (though actually udhcpc does write the DNS details to /etc/resolv.conf.dhcp). My assumption is the lookup to cdn.sipeed.com is for firmware updates (as I bought the NanoKVM cube it came fully installed, so no need for a .so download to make things work); when working DNS was provided I witness attempts to connect to HTTPS. I ve not bothered digging further into this. I did go grab the latest.zip being served from the URL, which turned out to be v2.2.9, matching what I have installed, not the latest on GitHub. I note there s an iptables setup (with nftables underneath) that s not fully realised - it seems to be trying to allow inbound HTTP + WebRTC, as well as outbound SSH, but everything is default accept so none of it gets hit. Setting up a default deny outbound and tweaking a little should provide a bit more reassurance it s not going to try and connect out somewhere it shouldn t. It looks like updates focus solely on the KVM application, so I wanted to take a look at the underlying OS. This is buildroot based:
~ # cat /etc/os-release
NAME=Buildroot
VERSION=-g98d17d2c0-dirty
ID=buildroot
VERSION_ID=2023.11.2
PRETTY_NAME="Buildroot 2023.11.2"
The kernel reports itself as 5.10.4-tag-. Somewhat ancient, but actually an LTS kernel. Except we re now up to 5.10.247, so it obviously hasn t been updated in some time. TBH, this is what I expect (and fear) from embedded devices. They end up with some ancient base OS revision and a kernel with a bunch of hacks that mean it s not easily updated. I get that the margins on this stuff are tiny, but I do wish folk would spend more time upstreaming. Or at least updating to the latest LTS point release for their kernel. The SSH client/daemon is full-fat OpenSSH:
~ # sshd -V
OpenSSH_9.6p1, OpenSSL 3.1.4 24 Oct 2023
There are a number of CVEs fixed in later OpenSSL 3.1 versions, though at present nothing that looks too concerning from the server side. Yes, the image has tcpdump + aircrack installed. I m a little surprised at aircrack (the device has no WiFi and even though I know there s a variant that does, it s not a standard debug tool the way tcpdump is), but there s a copy of GNU Chess in there too, so it s obvious this is just a kitchen-sink image. FWIW it looks like the buildroot config is here. Sadly the UART that I believe the bootloader/kernel are talking to is not exposed externally - the UART pin headers are for UART1 + 2, and I d have to open up the device to get to UART0. I ve not yet done this (but doing so would also allow access to the SD card, which would make trying to compile + test my own kernel easier). In terms of actual functionality it did what I d expect. 1080p HDMI capture was fine. I d have gone for a lower resolution, but I think that would have required tweaking on the client side. It looks like the 2.3.0 release allows EDID tweaking, so I might have to investigate that. The keyboard defaults to a US layout, which caused some problems with the symbol until I reconfigured the target machine not to expect a GB layout. There s also the potential to share out images via USB. I copied a Debian trixie netinst image to /data on the NanoKVM and was able to select it in the web interface and have it appear on the target machine easily. There s also the option to fetch direct from a URL in the web interface, but I was still testing without routable network access, so didn t try that. There s plenty of room for images:
~ # df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/mmcblk0p2            7.6G    823.3M      6.4G  11% /
devtmpfs                 77.7M         0     77.7M   0% /dev
tmpfs                    79.0M         0     79.0M   0% /dev/shm
tmpfs                    79.0M     30.2M     48.8M  38% /tmp
tmpfs                    79.0M    124.0K     78.9M   0% /run
/dev/mmcblk0p1           16.0M     11.5M      4.5M  72% /boot
/dev/mmcblk0p3           22.2G    160.0K     22.2G   0% /data
The NanoKVM also appears as an RNDIS USB network device, with udhcpd running on the interface. IP forwarding is not enabled, and there s no masquerading rules setup, so this doesn t give the target host access to the management LAN by default. I guess it could be useful for copying things over to the target host, as a more flexible approach than a virtual disk image. One thing to note is this makes for a bunch of devices over the composite USB interface. There are 3 HID devices (keyboard, absolute mouse, relative mouse), the RNDIS interface, and the USB mass storage. I had a few occasions where the keyboard input got stuck after I d been playing about with big data copies over the network and using the USB mass storage emulation. There is a HID-only mode (no network/mass storage) to try and help with this, and a restart of the NanoKVM generally brought things back, but something to watch out for. Again I see that the 2.3.0 application update mentions resetting the USB hardware on a HID reset, which might well help. As I stated at the start, I m happy with this purchase. Would I leave it exposed to the internet without suitable firewalling? No, but then I wouldn t do so for any KVM. I wanted a lightweight KVM suitable for use in my home network, something unlikely to see heavy use but that would save me hooking up an actual monitor + keyboard when things were misbehaving. So far everything I ve seen says I ve got my money s worth from it.

C.J. Collier: I m learning about perlguts today.


im-learning-about-perlguts-today.png


## 0.23	2025-12-20
commit be15aa25dea40aea66a8534143fb81b29d2e6c08
Author: C.J. Collier 
Date:   Sat Dec 20 22:40:44 2025 +0000
    Fixes C-level test infrastructure and adds more test cases for upb_to_sv conversions.
    
    - **Makefile.PL:**
        - Allow  extra_src  in  c_test_config.json  to be an array.
        - Add ASan flags to CCFLAGS and LDDLFLAGS for better debugging.
        - Corrected echo newlines in  test_c  target.
    - **c_test_config.json:**
        - Added missing type test files to  deps  and  extra_src  for  convert/sv_to_upb  and  convert/upb_to_sv  test runners.
    - **t/c/convert/upb_to_sv.c:**
        - Fixed a double free of  test_pool .
        - Added missing includes for type test headers.
        - Updated test plan counts.
    - **t/c/convert/sv_to_upb.c:**
        - Added missing includes for type test headers.
        - Updated test plan counts.
        - Corrected Perl interpreter initialization.
    - **t/c/convert/types/**:
        - Added missing  test_util.h  include in new type test headers.
        - Completed the set of  upb_to_sv  test cases for all scalar types by adding optional and repeated tests for  sfixed32 ,  sfixed64 ,  sint32 , and  sint64 , and adding repeated tests to the remaining scalar type files.
    - **Documentation:**
        - Updated  01-xs-testing.md  with more debugging tips, including ASan usage and checking for double frees and typos.
        - Updated  xs_learnings.md  with details from the recent segfault.
        - Updated  llm-plan-execution-instructions.md  to emphasize debugging steps.
## 0.22	2025-12-19
commit 2c171d9a5027e0150eae629729c9104e7f6b9d2b
Author: C.J. Collier 
Date:   Fri Dec 19 23:41:02 2025 +0000
    feat(perl,testing): Initialize C test framework and build system
    
    This commit sets up the foundation for the C-level tests and the build system for the Perl Protobuf module:
    
    1.  **Makefile.PL Enhancements:**
        *   Integrates  Devel::PPPort  to generate  ppport.h  for better portability.
        *   Object files now retain their path structure (e.g.,  xs/convert/sv_to_upb.o ) instead of being flattened, improving build clarity.
        *   The  MY::postamble  is significantly revamped to dynamically generate build rules for all C tests located in  t/c/  based on the  t/c/c_test_config.json  file.
        *   C tests are linked against  libprotobuf_common.a  and use  ExtUtils::Embed  flags.
        *   Added  JSON::MaybeXS  to  PREREQ_PM .
        *   The  test  target now also depends on the  test_c  target.
    
    2.  **C Test Infrastructure ( t/c/ ):
        *   Introduced  t/c/c_test_config.json  to configure individual C test builds, specifying dependencies and extra source files.
        *   Created  t/c/convert/test_util.c  and  .h  for shared test functions like loading descriptors.
        *   Initial  t/c/convert/upb_to_sv.c  and  t/c/convert/sv_to_upb.c  test runners.
        *   Basic  t/c/integration/030_protobuf_coro.c  for Coro safety testing on core utils using  libcoro .
        *   Basic  t/c/integration/035_croak_test.c  for testing exception handling.
        *   Basic  t/c/integration/050_convert.c  for integration testing conversions.
    
    3.  **Test Proto:** Updated  t/data/test.proto  with more field types for conversion testing and regenerated  test_descriptor.bin .
    
    4.  **XS Test Harness ( t/c/upb-perl-test.h ):** Added  like_n  macro for length-aware regex matching.
    
    5.  **Documentation:** Updated architecture and plan documents to reflect the C test structure.
    6.  **ERRSV Testing:** Note that the C tests ( t/c/ ) will primarily check *if* a  croak  occurs (i.e., that the exception path is taken), but will not assert on the string content of  ERRSV . Reliably testing  $@  content requires the full Perl test environment with  Test::More , which will be done in the  .t  files when testing the Perl API.
    
    This provides a solid base for developing and testing the XS and C components of the module.
## 0.21	2025-12-18
commit a8b6b6100b2cf29c6df1358adddb291537d979bc
Author: C.J. Collier 
Date:   Thu Dec 18 04:20:47 2025 +0000
    test(C): Add integration tests for Milestone 2 components
    
    - Created t/c/integration/030_protobuf.c to test interactions
      between obj_cache, arena, and utils.
    - Added this test to t/c/c_test_config.json.
    - Verified that all C tests for Milestones 2 and 3 pass,
      including the libcoro-based stress test.
## 0.20	2025-12-18
commit 0fcad68680b1f700a83972a7c1c48bf3a6958695
Author: C.J. Collier 
Date:   Thu Dec 18 04:14:04 2025 +0000
    docs(plan): Add guideline review reminders to milestones
    
    - Added a "[ ] REFRESH: Review all documents in @perl/doc/guidelines/**"
      checklist item to the start of each component implementation
      milestone (C and Perl layers).
    - This excludes Integration Test milestones.
## 0.19	2025-12-18
commit 987126c4b09fcdf06967a98fa3adb63d7de59a34
Author: C.J. Collier 
Date:   Thu Dec 18 04:05:53 2025 +0000
    docs(plan): Add C-level and Perl-level Coro tests to milestones
    
    - Added checklist items for  libcoro -based C tests
      (e.g.,  t/c/integration/050_convert_coro.c ) to all C layer
      integration milestones (050 through 220).
    - Updated  030_Integration_Protobuf.md  to standardise checklist
      items for the existing  030_protobuf_coro.c  test.
    - Removed the single  xt/author/coro-safe.t  item from
       010_Build.md .
    - Added checklist items for Perl-level  Coro  tests
      (e.g.,  xt/coro/240_arena.t ) to each Perl layer
      integration milestone (240 through 400).
    - Created  perl/t/c/c_test_config.json  to manage C test
      configurations externally.
    - Updated  perl/doc/architecture/testing/01-xs-testing.md  to describe
      both C-level  libcoro  and Perl-level  Coro  testing strategies.
## 0.18	2025-12-18
commit 6095a5a610401a6035a81429d0ccb9884d53687b
Author: C.J. Collier 
Date:   Thu Dec 18 02:34:31 2025 +0000
    added coro testing to c layer milestones
## 0.17	2025-12-18
commit cc0aae78b1f7f675fc8a1e99aa876c0764ea1cce
Author: C.J. Collier 
Date:   Thu Dec 18 02:26:59 2025 +0000
    docs(plan): Refine test coverage checklist items for SMARTness
    
    - Updated the "Tests provide full coverage" checklist items in
      C layer plan files (020, 040, 060, 080, 100, 120, 140, 160, 180, 200)
      to explicitly mention testing all public functions in the
      corresponding header files.
    - Expanded placeholder checklists in 140, 160, 180, 200.
    - Updated the "Tests provide full coverage" and "Add coverage checks"
      checklist items in Perl layer plan files (230, 250, 270, 290, 310, 330,
      350, 370, 390) to be more specific about the scope of testing
      and the use of  Test::TestCoverage .
    - Expanded Well-Known Types milestone (350) to detail each type.
## 0.16	2025-12-18
commit e4b601f14e3817a17b0f4a38698d981dd4cb2818
Author: C.J. Collier 
Date:   Thu Dec 18 02:07:35 2025 +0000
    docs(plan): Full refactoring of C and Perl plan files
    
    - Split both ProtobufPlan-C.md and ProtobufPlan-Perl.md into
      per-milestone files under the  perl/doc/plan/  directory.
    - Introduced Integration Test milestones after each component
      milestone in both C and Perl plans.
    - Numbered milestone files sequentially (e.g., 010_Build.md,
      230_Perl_Arena.md).
    - Updated main ProtobufPlan-C.md and ProtobufPlan-Perl.md to
      act as Tables of Contents.
    - Ensured consistent naming for integration test files
      (e.g.,  t/c/integration/030_protobuf.c ,  t/integration/260_descriptor_pool.t ).
    - Added architecture review steps to the end of all milestones.
    - Moved Coro safety test to C layer Milestone 1.
    - Updated Makefile.PL to support new test structure and added Coro.
    - Moved and split t/c/convert.c into t/c/convert/*.c.
    - Moved other t/c/*.c tests into t/c/protobuf/*.c.
    - Deleted old t/c/convert.c.
## 0.15	2025-12-17
commit 649cbacf03abb5e7293e3038bb451c0406e9d0ce
Author: C.J. Collier 
Date:   Wed Dec 17 23:51:22 2025 +0000
    docs(plan): Refactor and reset ProtobufPlan.md
    
    - Split the plan into ProtobufPlan-C.md and ProtobufPlan-Perl.md.
    - Reorganized milestones to clearly separate C layer and Perl layer development.
    - Added more granular checkboxes for each component:
      - C Layer: Create test, Test coverage, Implement, Tests pass.
      - Perl Layer: Create test, Test coverage, Implement Module/XS, Tests pass, C-Layer adjustments.
    - Reset all checkboxes to  [ ]  to prepare for a full audit.
    - Updated status in architecture/api and architecture/core documents to "Not Started".
    
    feat(obj_cache): Add unregister function and enhance tests
    
    - Added  protobuf_unregister_object  to  xs/protobuf/obj_cache.c .
    - Updated  xs/protobuf/obj_cache.h  with the new function declaration.
    - Expanded tests in  t/c/protobuf_obj_cache.c  to cover unregistering,
      overwriting keys, and unregistering non-existent keys.
    - Corrected the test plan count in  t/c/protobuf_obj_cache.c  to 17.
## 0.14	2025-12-17
commit 40b6ad14ca32cf16958d490bb575962f88d868a1
Author: C.J. Collier 
Date:   Wed Dec 17 23:18:27 2025 +0000
    feat(arena): Complete C layer for Arena wrapper
    
    This commit finalizes the C-level implementation for the Protobuf::Arena wrapper.
    
    - Adds  PerlUpb_Arena_Destroy  for proper cleanup from Perl's DEMOLISH.
    - Enhances error checking in  PerlUpb_Arena_Get .
    - Expands C-level tests in  t/c/protobuf_arena.c  to cover memory allocation
      on the arena and lifecycle through  PerlUpb_Arena_Destroy .
    - Corrects embedded Perl initialization in the C test.
    
    docs(plan): Refactor ProtobufPlan.md
    
    - Restructures the development plan to clearly separate "C Layer" and
      "Perl Layer" tasks within each milestone.
    - This aligns the plan with the "C-First Implementation Strategy" and improves progress tracking.
## 0.13	2025-12-17
commit c1e566c25f62d0ae9f195a6df43b895682652c71
Author: C.J. Collier 
Date:   Wed Dec 17 22:00:40 2025 +0000
    refactor(perl): Rename C tests and enhance Makefile.PL
    
    - Renamed test files in  t/c/  to better match the  xs  module structure:
        -  01-cache.c  ->  protobuf_obj_cache.c 
        -  02-arena.c  ->  protobuf_arena.c 
        -  03-utils.c  ->  protobuf_utils.c 
        -  04-convert.c  ->  convert.c 
        -  load_test.c  ->  upb_descriptor_load.c 
    - Updated  perl/Makefile.PL  to reflect the new test names in  MY::postamble 's  $c_test_config .
    - Refactored the  $c_test_config  generation in  Makefile.PL  to reduce repetition by using a default flags hash and common dependencies array.
    - Added a  fail()  macro to  perl/t/c/upb-perl-test.h  for consistency.
    - Modified  t/c/upb_descriptor_load.c  to use the  t/c/upb-perl-test.h  macros, making its output consistent with other C tests.
    - Added a skeleton for  t/c/convert.c  to test the conversion functions.
    - Updated documentation in  ProtobufPlan.md  and  architecture/testing/01-xs-testing.md  to reflect new test names.
## 0.12	2025-12-17
commit d8cb5dd415c6c129e71cd452f78e29de398a82c9
Author: C.J. Collier 
Date:   Wed Dec 17 20:47:38 2025 +0000
    feat(perl): Refactor XS code into subdirectories
    
    This commit reorganizes the C code in the  perl/xs/  directory into subdirectories, mirroring the structure of the Python UPB extension. This enhances modularity and maintainability.
    
    - Created subdirectories for each major component:  convert ,  descriptor ,  descriptor_containers ,  descriptor_pool ,  extension_dict ,  map ,  message ,  protobuf ,  repeated , and  unknown_fields .
    - Created skeleton  .h  and  .c  files within each subdirectory to house the component-specific logic.
    - Updated top-level component headers (e.g.,  perl/xs/descriptor.h ) to include the new sub-headers.
    - Updated top-level component source files (e.g.,  perl/xs/descriptor.c ) to include their main header and added stub initialization functions (e.g.,  PerlUpb_InitDescriptor ).
    - Moved code from the original  perl/xs/protobuf.c  to new files in  perl/xs/protobuf/  (arena, obj_cache, utils).
    - Moved code from the original  perl/xs/convert.c  to new files in  perl/xs/convert/  (upb_to_sv, sv_to_upb).
    - Updated  perl/Makefile.PL  to use a glob ( xs/*/*.c ) to find the new C source files in the subdirectories.
    - Added  perl/doc/architecture/core/07-xs-file-organization.md  to document the new structure.
    - Updated  perl/doc/ProtobufPlan.md  and other architecture documents to reference the new organization.
    - Corrected self-referential includes in the newly created .c files.
    
    This restructuring provides a solid foundation for further development and makes it easier to port logic from the Python implementation.
## 0.11	2025-12-17
commit cdedcd13ded4511b0464f5d3bdd72ce6d34e73fc
Author: C.J. Collier 
Date:   Wed Dec 17 19:57:52 2025 +0000
    feat(perl): Implement C-first testing and core XS infrastructure
    
    This commit introduces a significant refactoring of the Perl XS extension, adopting a C-first development approach to ensure a robust foundation.
    
    Key changes include:
    
    -   **C-Level Testing Framework:** Established a C-level testing system in  t/c/  with a dedicated Makefile, using an embedded Perl interpreter. Initial tests cover the object cache ( 01-cache.c ), arena wrapper ( 02-arena.c ), and utility functions ( 03-utils.c ).
    -   **Core XS Infrastructure:**
        -   Implemented a global object cache ( xs/protobuf.c ) to manage Perl wrappers for UPB objects, using weak references.
        -   Created an  upb_Arena  wrapper ( xs/protobuf.c ).
        -   Consolidated common XS helper functions into  xs/protobuf.h  and  xs/protobuf.c .
    -   **Makefile.PL Enhancements:** Updated to support building and linking C tests, incorporating flags from  ExtUtils::Embed , and handling both  .c  and  .cc  source files.
    -   **XS File Reorganization:** Restructured XS files to mirror the Python UPB extension's layout (e.g.,  message.c ,  descriptor.c ). Removed older, monolithic  .xs  files.
    -   **Typemap Expansion:** Added extensive typemap entries in  perl/typemap  to handle conversions between Perl objects and various  const upb_*Def*  pointers.
    -   **Descriptor Tests:** Added a new test suite  t/02-descriptor.t  to validate descriptor loading and accessor methods.
    -   **Documentation:** Updated development plans and guidelines ( ProtobufPlan.md ,  xs_learnings.md , etc.) to reflect the C-first strategy, new testing methods, and lessons learned.
    -   **Build Cleanup:** Removed  ppport.h  from  .gitignore  as it's no longer used, due to  -DPERL_NO_PPPORT  being set in  Makefile.PL .
    
    This C-first approach allows for more isolated and reliable testing of the core logic interacting with the UPB library before higher-level Perl APIs are built upon it.
## 0.10	2025-12-17
commit 1ef20ade24603573905cb0376670945f1ab5d829
Author: C.J. Collier 
Date:   Wed Dec 17 07:08:29 2025 +0000
    feat(perl): Implement C-level tests and core XS utils
    
    This commit introduces a C-level testing framework for the XS layer and implements key components:
    
    1.  **C-Level Tests ( t/c/ )**:
        *   Added  t/c/Makefile  to build standalone C tests.
        *   Created  t/c/upb-perl-test.h  with macros for TAP-compliant C tests ( plan ,  ok ,  is ,  is_string ,  diag ).
        *   Implemented  t/c/01-cache.c  to test the object cache.
        *   Implemented  t/c/02-arena.c  to test  Protobuf::Arena  wrappers.
        *   Implemented  t/c/03-utils.c  to test string utility functions.
        *   Corrected include paths and diagnostic messages in C tests.
    
    2.  **XS Object Cache ( xs/protobuf.c )**:
        *   Switched to using stringified pointers ( %p ) as hash keys for stability.
        *   Fixed a critical double-free bug in  PerlUpb_ObjCache_Delete  by removing an extra  SvREFCNT_dec  on the lookup key.
    
    3.  **XS Arena Wrapper ( xs/protobuf.c )**:
        *   Corrected  PerlUpb_Arena_New  to use  newSVrv  and  PTR2IV  for opaque object wrapping.
        *   Corrected  PerlUpb_Arena_Get  to safely unwrap the arena pointer.
    
    4.  **Makefile.PL ( perl/Makefile.PL )**:
        *   Added  -Ixs  to  INC  to allow C tests to find  t/c/upb-perl-test.h  and  xs/protobuf.h .
        *   Added  LIBS  to link  libprotobuf_common.a  into the main  Protobuf.so .
        *   Added C test targets  01-cache ,  02-arena ,  03-utils  to the test config in  MY::postamble .
    
    5.  **Protobuf.pm ( perl/lib/Protobuf.pm )**:
        *   Added  use XSLoader;  to load the compiled XS code.
    
    6.  **New files  xs/util.h **:
        *   Added initial type conversion function.
    
    These changes establish a foundation for testing the C-level interface with UPB and fix crucial bugs in the object cache implementation.
## 0.09	2025-12-17
commit 07d61652b032b32790ca2d3848243f9d75ea98f4
Author: C.J. Collier 
Date:   Wed Dec 17 04:53:34 2025 +0000
    feat(perl): Build system and C cache test for Perl XS
    
    This commit introduces the foundational pieces for the Perl XS implementation, focusing on the build system and a C-level test for the object cache.
    
    -   **Makefile.PL:**
        -   Refactored C test compilation rules in  MY::postamble  to use a hash ( $c_test_config ) for better organization and test-specific flags.
        -   Integrated  ExtUtils::Embed  to provide necessary compiler and linker flags for embedding the Perl interpreter, specifically for the  t/c/01-cache.c  test.
        -   Correctly constructs the path to the versioned Perl library ( libperl.so.X.Y.Z ) using  $Config archlib  and  $Config libperl  to ensure portability.
        -   Removed  VERSION_FROM  and  ABSTRACT_FROM  to avoid dependency on  .pm  files for now.
    
    -   **C Cache Test (t/c/01-cache.c):**
        -   Added a C test to exercise the object cache functions implemented in  xs/protobuf.c .
        -   Includes tests for adding, getting, deleting, and weak reference behavior.
    
    -   **XS Cache Implementation (xs/protobuf.c, xs/protobuf.h):**
        -   Implemented  PerlUpb_ObjCache_Init ,  PerlUpb_ObjCache_Add ,  PerlUpb_ObjCache_Get ,  PerlUpb_ObjCache_Delete , and  PerlUpb_ObjCache_Destroy .
        -   Uses a Perl hash ( HV* ) for the cache.
        -   Keys are string representations of the C pointers, created using  snprintf  with  "%llx" .
        -   Values are weak references ( sv_rvweaken ) to the Perl objects ( SV* ).
        -    PerlUpb_ObjCache_Get  now correctly returns an incremented reference to the original SV, not a copy.
        -    PerlUpb_ObjCache_Destroy  now clears the hash before decrementing its refcount.
    
    -   **t/c/upb-perl-test.h:**
        -   Updated  is_sv  to perform direct pointer comparison ( got == expected ).
    
    -   **Minor:** Added  util.h  (currently empty), updated  typemap .
    
    These changes establish a working C-level test environment for the XS components.
## 0.08	2025-12-17
commit d131fd22ea3ed8158acb9b0b1fe6efd856dc380e
Author: C.J. Collier 
Date:   Wed Dec 17 02:57:48 2025 +0000
    feat(perl): Update docs and core XS files
    
    - Explicitly add TDD cycle to ProtobufPlan.md.
    - Clarify mirroring of Python implementation in upb-interfacing.md for both C and Perl layers.
    - Branch and adapt python/protobuf.h and python/protobuf.c to perl/xs/protobuf.h and perl/xs/protobuf.c, including the object cache implementation. Removed old cache.* files.
    - Create initial C test for the object cache in t/c/01-cache.c.
## 0.07	2025-12-17
commit 56fd6862732c423736a2f9a9fb1a2816fc59e9b0
Author: C.J. Collier 
Date:   Wed Dec 17 01:09:18 2025 +0000
    feat(perl): Align Perl UPB architecture docs with Python
    
    Updates the Perl Protobuf architecture documents to more closely align with the design and implementation strategies used in the Python UPB extension.
    
    Key changes:
    
    -   **Object Caching:** Mandates a global, per-interpreter cache using weak references for all UPB-derived objects, mirroring Python's  PyUpb_ObjCache .
    -   **Descriptor Containers:** Introduces a new document outlining the plan to use generic XS container types (Sequence, ByNameMap, ByNumberMap) with vtables to handle collections of descriptors, similar to Python's  descriptor_containers.c .
    -   **Testing:** Adds a note to the testing strategy to port relevant test cases from the Python implementation to ensure feature parity.
## 0.06	2025-12-17
commit 6009ce6ab64eccce5c48729128e5adf3ef98e9ae
Author: C.J. Collier 
Date:   Wed Dec 17 00:28:20 2025 +0000
    feat(perl): Implement object caching and fix build
    
    This commit introduces several key improvements to the Perl XS build system and core functionality:
    
    1.  **Object Caching:**
        *   Introduces  xs/protobuf.c  and  xs/protobuf.h  to implement a caching mechanism ( protobuf_c_to_perl_obj ) for wrapping UPB C pointers into Perl objects. This uses a hash and weak references to ensure object identity and prevent memory leaks.
        *   Updates the  typemap  to use  protobuf_c_to_perl_obj  for  upb_MessageDef *  output, ensuring descriptor objects are cached.
        *   Corrected  sv_weaken  to the correct  sv_rvweaken  function.
    
    2.  **Makefile.PL Enhancements:**
        *   Switched to using the Bazel-generated UPB descriptor sources from  bazel-bin/src/google/protobuf/_virtual_imports/descriptor_proto/google/protobuf/ .
        *   Updated  INC  paths to correctly locate the generated headers.
        *   Refactored  MY::dynamic_lib  to ensure the static library  libprotobuf_common.a  is correctly linked into each generated  .so  module, resolving undefined symbol errors.
        *   Overrode  MY::test  to use  prove -b -j$(nproc) t/*.t xt/*.t  for running tests.
        *   Cleaned up  LIBS  and  LDDLFLAGS  usage.
    
    3.  **Documentation:**
        *   Updated  ProtobufPlan.md  to reflect the current status and design decisions.
        *   Reorganized architecture documents into subdirectories.
        *   Added  object-caching.md  and  c-perl-interface.md .
        *   Updated  llm-guidance.md  with notes on  upb/upb.h  and  sv_rvweaken .
    
    4.  **Testing:**
        *   Fixed  xt/03-moo_immutable.t  to skip tests if no Moo modules are found.
    
    This resolves the build issues and makes the core test suite pass.
## 0.05	2025-12-16
commit 177d2f3b2608b9d9c415994e076a77d8560423b8
Author: C.J. Collier 
Date:   Tue Dec 16 19:51:36 2025 +0000
    Refactor: Rename namespace to Protobuf, build system and doc updates
    
    This commit refactors the primary namespace from  ProtoBuf  to  Protobuf 
    to align with the style guide. This involves renaming files, directories,
    and updating package names within all Perl and XS files.
    
    **Namespace Changes:**
    
    *   Renamed  perl/lib/ProtoBuf  to  perl/lib/Protobuf .
    *   Moved and updated  ProtoBuf.pm  to  Protobuf.pm .
    *   Moved and updated  ProtoBuf::Descriptor  to  Protobuf::Descriptor  (.pm & .xs).
    *   Removed other  ProtoBuf::*  stubs (Arena, DescriptorPool, Message).
    *   Updated  MODULE  and  PACKAGE  in  Descriptor.xs .
    *   Updated  NAME ,  *_FROM  in  perl/Makefile.PL .
    *   Replaced  ProtoBuf  with  Protobuf  throughout  perl/typemap .
    *   Updated namespaces in test files  t/01-load-protobuf-descriptor.t  and  t/02-descriptor.t .
    *   Updated namespaces in all documentation files under  perl/doc/ .
    *   Updated paths in  perl/.gitignore .
    
    **Build System Enhancements (Makefile.PL):**
    
    *   Included  xs/*.c  files in the common object files list.
    *   Added  -I.  to the  INC  paths.
    *   Switched from  MYEXTLIB  to  LIBS => ['-L$(CURDIR) -lprotobuf_common']  for linking.
    *   Removed custom keys passed to  WriteMakefile  for postamble.
    *    MY::postamble  now sources variables directly from the main script scope.
    *   Added  all :: $ common_lib  dependency in  MY::postamble .
    *   Added  t/c/load_test.c  compilation rule in  MY::postamble .
    *   Updated  clean  target to include  blib .
    *   Added more modules to  TEST_REQUIRES .
    *   Removed the explicit  PM  and  XS  keys from  WriteMakefile , relying on  XSMULTI => 1 .
    
    **New Files:**
    
    *    perl/lib/Protobuf.pm 
    *    perl/lib/Protobuf/Descriptor.pm 
    *    perl/lib/Protobuf/Descriptor.xs 
    *    perl/t/01-load-protobuf-descriptor.t 
    *    perl/t/02-descriptor.t 
    *    perl/t/c/load_test.c : Standalone C test for UPB.
    *    perl/xs/types.c  &  perl/xs/types.h : For Perl/C type conversions.
    *    perl/doc/architecture/upb-interfacing.md 
    *    perl/xt/03-moo_immutable.t : Test for Moo immutability.
    
    **Deletions:**
    
    *   Old test files:  t/00_load.t ,  t/01_basic.t ,  t/02_serialize.t ,  t/03_message.t ,  t/04_descriptor_pool.t ,  t/05_arena.t ,  t/05_message.t .
    *   Removed  lib/ProtoBuf.xs  as it's not needed with  XSMULTI .
    
    **Other:**
    
    *   Updated  test_descriptor.bin  (binary change).
    *   Significant content updates to markdown documentation files in  perl/doc/architecture  and  perl/doc/internal  reflecting the new architecture and learnings.
## 0.04	2025-12-14
commit 92de5d482c8deb9af228f4b5ce31715d3664d6ee
Author: C.J. Collier 
Date:   Sun Dec 14 21:28:19 2025 +0000
    feat(perl): Implement Message object creation and fix lifecycles
    
    This commit introduces the basic structure for  ProtoBuf::Message  object
    creation, linking it with  ProtoBuf::Descriptor  and  ProtoBuf::DescriptorPool ,
    and crucially resolves a SEGV by fixing object lifecycle management.
    
    Key Changes:
    
    1.  ** ProtoBuf::Descriptor :** Added  _pool  attribute to hold a strong
        reference to the parent  ProtoBuf::DescriptorPool . This is essential to
        prevent the pool and its C  upb_DefPool  from being garbage collected
        while a descriptor is still in use.
    
    2.  ** ProtoBuf::DescriptorPool :**
        *    find_message_by_name : Now passes the  $self  (the pool object) to the
             ProtoBuf::Descriptor  constructor to establish the lifecycle link.
        *   XSUB  pb_dp_find_message_by_name : Updated to accept the pool  SV*  and
            store it in the descriptor's  _pool  attribute.
        *   XSUB  _load_serialized_descriptor_set : Renamed to avoid clashing with the
            Perl method name. The Perl wrapper now correctly calls this internal XSUB.
        *    DEMOLISH : Made safer by checking for attribute existence.
    
    3.  ** ProtoBuf::Message :**
        *   Implemented using Moo with lazy builders for  _upb_arena  and
             _upb_message .
        *    _descriptor  is a required argument to  new() .
        *   XS functions added for creating the arena ( pb_msg_create_arena ) and
            the  upb_Message  ( pb_msg_create_upb_message ).
        *    pb_msg_create_upb_message  now extracts the  upb_MessageDef*  from the
            descriptor and uses  upb_MessageDef_MiniTable()  to get the minitable
            for  upb_Message_New() .
        *    DEMOLISH : Added to free the message's arena.
    
    4.  ** Makefile.PL :**
        *   Added  -g  to  CCFLAGS  for debugging symbols.
        *   Added Perl CORE include path to  MY::postamble 's  base_flags .
    
    5.  **Tests:**
        *    t/04_descriptor_pool.t : Updated to check the structure of the
            returned  ProtoBuf::Descriptor .
        *    t/05_message.t : Now uses a descriptor obtained from a real pool to
            test  ProtoBuf::Message->new() .
    
    6.  **Documentation:**
        *   Updated  ProtobufPlan.md  to reflect progress.
        *   Updated several files in  doc/architecture/  to match the current
            implementation details, especially regarding arena management and object
            lifecycles.
        *   Added  doc/internal/development_cycle.md  and  doc/internal/xs_learnings.md .
    
    With these changes, the SEGV is resolved, and message objects can be successfully
    created from descriptors.
## 0.03	2025-12-14
commit 6537ad23e93680c2385e1b571d84ed8dbe2f68e8
Author: C.J. Collier 
Date:   Sun Dec 14 20:23:41 2025 +0000
    Refactor(perl): Object-Oriented DescriptorPool with Moo
    
    This commit refactors the  ProtoBuf::DescriptorPool  to be fully object-oriented using Moo, and resolves several issues related to XS, typemaps, and test data.
    
    Key Changes:
    
    1.  **Moo Object:**  ProtoBuf::DescriptorPool.pm  now uses  Moo  to define the class. The  upb_DefPool  pointer is stored as a lazy attribute  _upb_defpool .
    2.  **XS Lifecycle:**  DescriptorPool.xs  now has  pb_dp_create_pool  called by the Moo builder and  pb_dp_free_pool  called from  DEMOLISH  to manage the  upb_DefPool  lifecycle per object.
    3.  **Typemap:** The  perl/typemap  file has been significantly updated to handle the conversion between the  ProtoBuf::DescriptorPool  Perl object and the  upb_DefPool *  C pointer. This includes:
        *   Mapping  upb_DefPool *  to  T_PTR .
        *   An  INPUT  section for  ProtoBuf::DescriptorPool  to extract the pointer from the object's hash, triggering the lazy builder if needed via  call_method .
        *   An  OUTPUT  section for  upb_DefPool *  to convert the pointer back to a Perl integer, used by the builder.
    4.  **Method Renaming:**  add_file_descriptor_set_binary  is now  load_serialized_descriptor_set .
    5.  **Test Data:**
        *   Added  perl/t/data/test.proto  with a sample message and enum.
        *   Generated  perl/t/data/test_descriptor.bin  using  protoc .
        *   Removed  t/data/  from  .gitignore  to ensure test data is versioned.
    6.  **Test Update:**  t/04_descriptor_pool.t  is updated to use the new OO interface, load the generated descriptor set, and check for message definitions.
    7.  **Build Fixes:**
        *   Corrected  #include  paths in  DescriptorPool.xs  to be relative to the  upb/  directory (e.g.,  upb/wire/decode.h ).
        *   Added  -I../upb  to  CCFLAGS  in  Makefile.PL .
        *   Reordered  INC  paths in  Makefile.PL  to prioritize local headers.
    
    **Note:** While tests now pass in some environments, a SEGV issue persists in  make test  runs, indicating a potential memory or lifecycle issue within the XS layer that needs further investigation.
## 0.02	2025-12-14
commit 6c9a6f1a5f774dae176beff02219f504ea3a6e07
Author: C.J. Collier 
Date:   Sun Dec 14 20:13:09 2025 +0000
    Fix(perl): Correct UPB build integration and generated file handling
    
    This commit resolves several issues to achieve a successful build of the Perl extension:
    
    1.  **Use Bazel Generated Files:** Switched from compiling UPB's stage0 descriptor.upb.c to using the Bazel-generated  descriptor.upb.c  and  descriptor.upb_minitable.c  located in  bazel-bin/src/google/protobuf/_virtual_imports/descriptor_proto/google/protobuf/ .
    2.  **Updated Include Paths:** Added the  bazel-bin  path to  INC  in  WriteMakefile  and to  base_flags  in  MY::postamble  to ensure the generated headers are found during both XS and static library compilation.
    3.  **Removed Stage0:** Removed references to  UPB_STAGE0_DIR  and no longer include headers or source files from  upb/reflection/stage0/ .
    4.  **-fPIC:** Explicitly added  -fPIC  to  CCFLAGS  in  WriteMakefile  and ensured  $(CCFLAGS)  is used in the custom compilation rules in  MY::postamble . This guarantees all object files in the static library are compiled with position-independent code, resolving linker errors when creating the shared objects for the XS modules.
    5.  **Refined UPB Sources:** Used  File::Find  to recursively find UPB C sources, excluding  /conformance/  and  /reflection/stage0/  to avoid conflicts and unnecessary compilations.
    6.  **Arena Constructor:** Modified  ProtoBuf::Arena::pb_arena_new  XSUB to accept the class name argument passed from Perl, making it a proper constructor.
    7.  **.gitignore:** Added patterns to  perl/.gitignore  to ignore generated C files from XS ( lib/*.c ,  lib/ProtoBuf/*.c ), the copied  src_google_protobuf_descriptor.pb.cc , and the  t/data  directory.
    8.  **Build Documentation:** Updated  perl/doc/architecture/upb-build-integration.md  to reflect the new build process, including the Bazel prerequisite, include paths,  -fPIC  usage, and  File::Find .
    
    Build Steps:
    1.   bazel build //src/google/protobuf:descriptor_upb_proto  (from repo root)
    2.   cd perl 
    3.   perl Makefile.PL 
    4.   make 
    5.   make test  (Currently has expected failures due to missing test data implementation).
## 0.01	2025-12-14
commit 3e237e8a26442558c94075766e0d4456daaeb71d
Author: C.J. Collier 
Date:   Sun Dec 14 19:34:28 2025 +0000
    feat(perl): Initialize Perl extension scaffold and build system
    
    This commit introduces the  perl/  directory, laying the groundwork for the Perl Protocol Buffers extension. It includes the essential build files, linters, formatter configurations, and a vendored Devel::PPPort for XS portability.
    
    Key components added:
    
    *   ** Makefile.PL **: The core  ExtUtils::MakeMaker  build script. It's configured to:
        *   Build a static library ( libprotobuf_common.a ) from UPB, UTF8_Range, and generated protobuf C/C++ sources.
        *   Utilize  XSMULTI => 1  to create separate shared objects for  ProtoBuf ,  ProtoBuf::Arena , and  ProtoBuf::DescriptorPool .
        *   Link each XS module against the common static library.
        *   Define custom compilation rules in  MY::postamble  to handle C vs. C++ flags and build the static library.
        *   Set up include paths for the project root, UPB, and other dependencies.
    
    *   **XS Stubs ( .xs  files)**:
        *    lib/ProtoBuf.xs : Placeholder for the main module's XS functions.
        *    lib/ProtoBuf/Arena.xs : XS interface for  upb_Arena  management.
        *    lib/ProtoBuf/DescriptorPool.xs : XS interface for  upb_DefPool  management.
    
    *   **Perl Module Stubs ( .pm  files)**:
        *    lib/ProtoBuf.pm : Main module, loads XS.
        *    lib/ProtoBuf/Arena.pm : Perl class for Arenas.
        *    lib/ProtoBuf/DescriptorPool.pm : Perl class for Descriptor Pools.
        *    lib/ProtoBuf/Message.pm : Base class for messages (TBD).
    
    *   **Configuration Files**:
        *    .gitignore : Ignores build artifacts, editor files, etc.
        *    .perlcriticrc : Configures Perl::Critic for static analysis.
        *    .perltidyrc : Configures perltidy for code formatting.
    
    *   ** Devel::PPPort **: Vendored version 3.72 to generate  ppport.h  for XS compatibility across different Perl versions.
    
    *   ** typemap **: Custom typemap for XS argument/result conversion.
    
    *   **Documentation ( doc/ )**: Initial architecture and plan documents.
    
    This provides a solid foundation for developing the UPB-based Perl extension.

18 December 2025

Dirk Eddelbuettel: dang 0.0.17: New Features, Plus Maintenance

dang image A new release of my mixed collection of things package dang package arrived at CRAN earlier today. The dang package regroups a few functions of mine that had no other home as for example lsos() from a StackOverflow question from 2009 (!!), the overbought/oversold price band plotter from an older blog post, the market monitor blogged about as well as the checkCRANStatus() function tweeted about by Tim Taylor. And more so take a look. This release retires two functions: the social media site nobody ever visits anymore shut down its API too, so no way to mute posts by a given handle. Similarly, the (never official) ability by Google to supply financial data is no more, so the function to access data this way is gone too. But we also have two new ones: one that helps with CRAN entries for ORCiD ids, and another little helper to re-order microbenchmark results by summary column (defaulting to the median). Other than the usual updates to continuous integrations, as well as a switch to Authors@R which will result in CRAN nagging me less about this, and another argument update. The detailed NEWS entry follows.

Changes in version 0.0.17 (2025-12-18)
  • Added new funtion reorderMicrobenchmarkResults with alias rmr
  • Use tolower on email argument to checkCRANStatus
  • Added new function cranORCIDs bootstrapped from two emails by Kurt Hornik
  • Switched to using Authors@R in DESCRIPTION and added ORCIDs where available
  • Switched to r-ci action with included bootstrap step; updated updated the checkout action (twice); added (commented-out) log accessor
  • Removed googleFinanceData as the (unofficial) API access point no longer works
  • Removed muteTweeters because the API was turned off

Via my CRANberries, there is a comparison to the previous release. For questions or comments use the the issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

17 December 2025

Dirk Eddelbuettel: RcppArmadillo 15.2.3-1 on CRAN: Upstream Update

armadillo image Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language and is widely used by (currently) 1272 other packages on CRAN, downloaded 43.2 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 661 times according to Google Scholar. This versions updates to the 15.2.3 upstream Armadillo release from yesterday. It brings minor changes over the RcppArmadillo 15.2.2 release made last month (and described in this post). As noted previously, and due to both the upstream transition to C++14 coupled with the CRAN move away from C++11, the package offers a transition by allowing packages to remain with the older, pre-15.0.0 legacy Armadillo yet offering the current version as the default. If and when CRAN will have nudged (nearly) all maintainers away from C++11 (and now also C++14 !!) we can remove the fallback. Our offer to help with the C++ modernization still stands, so please get in touch if we can be of assistance. As a reminder, the meta-issue #475 regroups all the resources for the C++11 transition. There were no R-side changes in this release. The detailed changes since the last release follow.

Changes in RcppArmadillo version 15.2.3-1 (2025-12-16)
  • Upgraded to Armadillo release 15.2.3 (Medium Roast Deluxe)
    • Faster .resize() for vectors
    • Faster repcube()

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

14 December 2025

Dirk Eddelbuettel: BH 1.90.0-1 on CRAN: New Upstream

Boost Boost is a very large and comprehensive set of (peer-reviewed) libraries for the C++ programming language, containing well over one hundred individual libraries. The BH package provides a sizeable subset of header-only libraries for (easier, no linking required) use by R. It is fairly widely used: the (partial) CRAN mirror logs (aggregated from the cloud mirrors) show over 41.5 million package downloads. Version 1.90.0 of Boost was released a few days ago following the regular Boost release schedule of April, August and December releases. As before, we packaged it almost immediately and started testing following our annual update cycle which strives to balance being close enough to upstream and not stressing CRAN and the user base too much. The reverse depends check revealed only one really minor issue among the over three hundred direct reverse dependencies. And that issue was addressed yesterday within hours by a truly responsove maintainer (and it helped that a related issue had been addressed months earlier with version 1.89.). So big thanks to Jean-Romain Roussel for the prompt fix, and to Andrew Johnson for the earlier test with 1.89.0. As last year with 1.87.0, no new Boost libraries were added to BH so the (considerable) size is more or less unchanged. It lead to CRAN doing a manual inspection but as there were no other issues it sailed through as is now in the CRAN repository. The short NEWS entry follows.

Changes in version 1.90.0-1 (2025-12-13)
  • Upgrade to Boost 1.90.0, patched as usual to comment-out diagnostic suppression messages per the request of CRAN
  • Minor upgrades to continuous integration

Via my CRANberries, there is a diffstat report relative to the previous release. Comments and suggestions about BH are welcome via the issue tracker at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

11 December 2025

Dirk Eddelbuettel: #056: Running r-ci with R-devel

Welcome to post 56 in the R4 series. The recent post #54 reviewed a number of earlier posts on r-ci, our small (but very versatile) runner for continunous integration (CI) with R. The post also introduced the notion of using a container in the matrix of jobs defined and running in parallel. The initial motivation was the (still ongoing, and still puzzling) variation in run-times of GitHub Actions. So when running CI and relying on r2u for the fast, easy, reliable: pick all three! provision of CRAN packages as Ubuntu binaries, a small amount of time is spent prepping a basic Ubuntu instance with the necessary setup. This can be as fast as maybe 20 to 30 seconds, but it can also stretch to almost two minutes when GitHub is busier or out of sorts for other reasons. When the CI job itself is short, that is a nuisance. We presented relying on a pre-made r2u4ci container that adds just a few commands to the standard r2u container to be complete for CI. And with that setup CI runs tend to be reliably faster. This situation is still evolving. I have not converted any of my existing CI scripts (apart from a test instance or two), but I keep monitoring the situation. However, this also offered another perspective: why not rely on a different container for a different CI aspect? When discussing the CI approach with Jeff the other day (and helping add CI to his mmap repo), it occurred to me we could also use on of the Rocker containers for R-devel. A minimal change to the underlying run.sh script later, this was accomplished. An example is provided as both a test and an illustration in the repo for package RcppInt64 in its script ci.yaml:
    strategy:
      matrix:
        include:
          -   name: container, os: ubuntu-latest, container: rocker/r2u4ci  
          -   name: r-devel,   os: ubuntu-latest, container: rocker/drd  
          -   name: macos,     os: macos-latest  
          -   name: ubuntu,    os: ubuntu-latest  

    runs-on: $  matrix.os  
    container: $  matrix.container  
This runs both a standard Ubuntu setup (fourth entry) and the alternate just described relying on the container (first entry) along with the (usually commented-out) optional macOS setup (third entry). And line two brings the drd container from Rocker. The CI runner script now checks for a possible Rdevel binary as provided inside drd (along with alias RD) and uses it when present. And that is all that there is: no other change on the user side; tests now run under R-devel. You can see some of the initial runs at the rcppint64 repo actions log. Another example is now also at Jeff s mmap repo. It should be noted that this relies on R-devel running packages made with R-release. Every few years this breaks when R needs to break its binary API. If and when that happens this option will be costlier as the R-devel instance will then have to (re-)install its R package dependencies. This can be accomodated easily as a step in the yaml file. And under normal circumstances it is not needed. Having easy access to recent builds of R-devel (the container refreshes weekly on a schedule) with the convenience of r2u gives another option for package testing. I may continue to test locally with R-devel as my primary option, and most likely keep my CI small and lean (usually just one R-relase run on Ubuntu) but having another option at GitHub Actions is also a good thing.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

7 December 2025

Vincent Bernat: Compressing embedded files in Go

Go s embed feature lets you bundle static assets into an executable, but it stores them uncompressed. This wastes space: a web interface with documentation can bloat your binary by dozens of megabytes. A proposition to optionally enable compression was declined because it is difficult to handle all use cases. One solution? Put all the assets into a ZIP archive!

Code The Go standard library includes a module to read and write ZIP archives. It contains a function that turns a ZIP archive into an io/fs.FS structure that can replace embed.FS in most contexts.1
package embed
import (
  "archive/zip"
  "bytes"
  _ "embed"
  "fmt"
  "io/fs"
  "sync"
)
//go:embed data/embed.zip
var embeddedZip []byte
var dataOnce = sync.OnceValue(func() *zip.Reader  
  r, err := zip.NewReader(bytes.NewReader(embeddedZip), int64(len(embeddedZip)))
  if err != nil  
    panic(fmt.Sprintf("cannot read embedded archive: %s", err))
   
  return r
 )
func Data() fs.FS  
  return dataOnce()
 
We can build the embed.zip archive with a rule in a Makefile. We specify the files to embed as dependencies to ensure changes are detected.
common/embed/data/embed.zip: console/data/frontend console/data/docs
common/embed/data/embed.zip: orchestrator/clickhouse/data/protocols.csv 
common/embed/data/embed.zip: orchestrator/clickhouse/data/icmp.csv
common/embed/data/embed.zip: orchestrator/clickhouse/data/asns.csv
common/embed/data/embed.zip:
    mkdir -p common/embed/data && zip --quiet --recurse-paths --filesync $@ $^
The automatic variable $@ is the rule target, while $^ expands to all the dependencies, modified or not.

Space gain Akvorado, a flow collector written in Go, embeds several static assets:
  • CSV files to translate port numbers, protocols or AS numbers, and
  • HTML, CSS, JS, and image files for the web interface, and
  • the documentation.
Breakdown of space used by each package before and after introducing embed.zip. It is displayed as a treemap and we can see many embedded files replaced by a bigger one.
Breakdown of the space used by each component before (left) and after (right) the introduction of embed.zip.
Embedding these assets into a ZIP archive reduced the size of the Akvorado executable by more than 4 MiB:
$ unzip -p common/embed/data/embed.zip   wc -c   numfmt --to=iec
7.3M
$ ll common/embed/data/embed.zip
-rw-r--r-- 1 bernat users 2.9M Dec  7 17:17 common/embed/data/embed.zip

Performance loss Reading from a compressed archive is not as fast as reading a flat file. A simple benchmark shows it is more than 4 slower. It also allocates some memory.2
goos: linux
goarch: amd64
pkg: akvorado/common/embed
cpu: AMD Ryzen 5 5600X 6-Core Processor
BenchmarkData/compressed-12     2262   526553 ns/op   610 B/op   10 allocs/op
BenchmarkData/uncompressed-12   9482   123175 ns/op     0 B/op    0 allocs/op
Each access to an asset requires a decompression step, as seen in this flame graph:
🖼 Flame graph when reading data from embed.zip compared to reading data directly
CPU flame graph comparing the time spent on CPU when reading data from embed.zip (left) versus reading data directly (right). Because the Go testing framework executes the benchmark for uncompressed data 4 times more often, it uses the same horizontal space as the benchmark for compressed data. The graph is interactive.
While a ZIP archive has an index to quickly find the requested file, seeking inside a compressed file is currently not possible.3 Therefore, the files from a compressed archive do not implement the io.ReaderAt or io.Seeker interfaces, unlike directly embedded files. This prevents some features, like serving partial files or detecting MIME types when serving files over HTTP.
For Akvorado, this is an acceptable compromise to save a few mebibytes from an executable of almost 100 MiB. Next week, I will continue this futile adventure by explaining how I prevented Go from disabling dead code elimination!

  1. You can safely read multiple files concurrently. However, it does not implement ReadDir() and ReadFile() methods.
  2. You could keep frequently accessed assets in memory. This reduces CPU usage and trades cached memory for resident memory.
  3. SOZip is a profile that enables fast random access in a compressed file. However, Go s archive/zip module does not support it.

3 December 2025

Reproducible Builds: Reproducible Builds in November 2025

Welcome to the report for November 2025 from the Reproducible Builds project! These monthly reports outline what we ve been up to over the past month, highlighting items of news from elsewhere in the increasingly-important area of software supply-chain security. As always, if you are interested in contributing to the Reproducible Builds project, please see the Contribute page on our website. In this report:

  1. 10 years of Reproducible Build at SeaGL
  2. Distribution work
  3. Tool development
  4. Website updates
  5. Miscellaneous news
  6. Software Supply Chain Security of Web3
  7. Upstream patches

10 years of Reproducible Builds at SeaGL 2025 On Friday 8th November, Chris Lamb gave a talk called 10 years of Reproducible Builds at SeaGL in Seattle, WA. Founded in 2013, SeaGL is a free, grassroots technical summit dedicated to spreading awareness and knowledge about free source software, hardware and culture. Chris talk:
[ ] introduces the concept of reproducible builds, its technical underpinnings and its potentially transformative impact on software security and transparency. It is aimed at developers, security professionals and policy-makers who are concerned with enhancing trust and accountability in our software. It also provides a history of the Reproducible Builds project, which is approximately ten years old. How are we getting on? What have we got left to do? Aren t all the builds reproducible now?

Distribution work In Debian this month, Jochen Sprickerhof created a merge request to replace the use of reprotest in Debian s Salsa Continuous Integration (CI) pipeline with debrebuild. Jochen cites the advantages as being threefold: firstly, that only one extra build needed ; it uses the same sbuild and ccache tooling as the normal build ; and works for any Debian release . The merge request was merged by Emmanuel Arias and is now active. kpcyrd posted to our mailing list announcing the initial release of repro-threshold, which implements an APT transport that defines a threshold of at least X of my N trusted rebuilders need to confirm they reproduced the binary before installing Debian packages. Configuration can be done through a config file, or through a curses-like user interface. Holger then merged two commits by Jochen Sprickerhof in order to address a fakeroot-related reproducibility issue in the debian-installer, and J rg Jaspert deployed a patch by Ivo De Decker for a bug originally filed by Holger in February 2025 related to some Debian packages not being archived on snapshot.debian.org. Elsewhere, Roland Clobus performed some analysis on the live Debian trixie images, which he determined were not reproducible. However, in a follow-up post, Roland happily reports that the issues have been handled. In addition, 145 reviews of Debian packages were added, 12 were updated and 15 were removed this month adding to our knowledge about identified issues. Lastly, Jochen Sprickerhof filed a bug announcing their intention to binary NMU a very large number of the R programming language after a reproducibility-related toolchain bug was fixed.
Bernhard M. Wiedemann posted another openSUSE monthly update for their work there.
Julien Malka and Arnout Engelen launched the new hash collection server for NixOS. Aside from improved reporting to help focus reproducible builds efforts within NixOS, it collects build hashes as individually-signed attestations from independent builders, laying the groundwork for further tooling.

Tool development diffoscope version 307 was uploaded to Debian unstable (as well as version 309). These changes included further attempts to automatically attempt to deploy to PyPI by liaising with the PyPI developers/maintainers (with this experimental feature). [ ][ ][ ] In addition, reprotest versions 0.7.31 and 0.7.32 were uploaded to Debian unstable by Holger Levsen, who also made the following changes:
  • Do not vary the architecture personality if the kernel is not varied. (Thanks to Ra l Cumplido). [ ]
  • Drop the debian/watch file, as Lintian now flags this as error for native Debian packages. [ ][ ]
  • Bump Standards-Version to 4.7.2, with no changes needed. [ ]
  • Drop the Rules-Requires-Root header as it is no longer required.. [ ]
In addition, however, Vagrant Cascadian fixed a build failure by removing some extra whitespace from an older changelog entry. [ ]

Website updates Once again, there were a number of improvements made to our website this month including:

Miscellaneous news

Software Supply Chain Security of Web3 Via our mailing list, Martin Monperrus let us know about their recently-published page on the Software Supply Chain Security of Web3. The abstract of their paper is as follows:
Web3 applications, built on blockchain technology, manage billions of dollars in digital assets through decentralized applications (dApps) and smart contracts. These systems rely on complex, software supply chains that introduce significant security vulnerabilities. This paper examines the software supply chain security challenges unique to the Web3 ecosystem, where traditional Web2 software supply chain problems intersect with the immutable and high-stakes nature of blockchain technology. We analyze the threat landscape and propose mitigation strategies to strengthen the security posture of Web3 systems.
Their paper lists reproducible builds as one of the mitigating strategies. A PDF of the full text is available to download.

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

2 December 2025

Dirk Eddelbuettel: duckdb-mlpack 0.0.5: Added kmeans, version helpers, documentation

A new release of the still-recent duckdb extension for mlpack, the C++ header-only library for machine learning, was merged into the duckdb community extensions repo today, and has been updated at its duckdb mlpack extension page. This release 0.0.5 adds one new method: kmeans clustering. We also added two version accessors for both mlpack and armadillo. We found during the work on random forests (added in 0.0.4) that the multithreaded random number generation was not quite right in the respective upstream codes. This has by now been corrected in armadillo 15.2.2 as well as the trunk version of mlpack so if you build with those, and set a seed, then your forests and classification will be stable across reruns. We added a second state variable mlpack_silent that can be used to suppress even the minimal prediction quality summary some methods show, and expanded the documentation. For more details, see the repo for code, issues and more, and the extension page for more about this duckdb community extension.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

26 November 2025

Dirk Eddelbuettel: tidyCpp 0.0.8 on CRAN: Maintenance

Another maintenance release of the tidyCpp package arrived on CRAN this morning, the first in about two years. The packages offers a clean C++ layer (as well as one small C++ helper class) on top of the C API for R which aims to make use of this robust (if awkward) C API a little easier and more consistent. See the (now updated, see below) vignette for motivating examples. This release contains mostly internal upkeep of the usual type: refreshing continuous integration, updating links, switching to Authors@R. But as we wrap the C API of R here too, changes made in R-devel this week affected the two reverse-dependencies (i.e. downstream ) packages (of mine) using this. So we commented-out the definitions for the five now-hidden accessors so that these downstream packages can build again under R-devel. The NEWS entry follows.

Changes in tidyCpp version 0.0.8 (2025-11-25)
  • Updated continuous integration setup several times
  • Updated README.md documentation with link to R API site
  • Updated example snippets to use of Protect
  • Updated documentation in defines.h header
  • Updated internals.h header reflecting in R API changes

As it happens, hours after the release at CRAN a helpful issue ticket was opened detailing more than a handful of typos in the vignette. This has been corrected, and I am not exporting the vignette via GitHub Pages so the motivating examples vignette contains the corrections. Thanks to my CRANberries, there is also a diffstat report for this release. For questions, suggestions, or issues please use the issue tracker at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

24 November 2025

Dirk Eddelbuettel: RcppQuantuccia 0.1.3 on CRAN: Micro Maintenance

A minor release of RcppQuantuccia arrived on CRAN moments ago. RcppQuantuccia started from the Quantuccia header-only subset / variant of QuantLib which it brings it to R. This project validated the idea of making the calendaring functionality of QuantLib available in a more compact and standalone project which we now do with qlcal which can be seen as a successor package to this earlier package. qlcal tracks QuantLib (releases) closely and provides approximately quarterly updates. Switching to using qlcal is generally recommended. This release, the first in almost exactly two years, only updates internals (as detailed below). Notably it switches to Authors@R to avoid a nag from CRAN on two platforms. The complete list changes for this release follows.

Changes in version 0.1.3 (2025-11-24)
  • A badge URL and link have been updated in README.md
  • The continuous integration sript switched first to r-ci-setup and then to the r-ci action with embedded setup
  • The DESCRIPTION file now uses Authors@R

Courtesy of CRANberries, there is also a diffstat report relative to the previous release. More information is on the RcppQuantuccia page. Issues and bugreports should go to the GitHub issue tracker. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

22 November 2025

Dirk Eddelbuettel: RcppArmadillo 15.2.2-1 on CRAN: Upstream Update, OpenMP Updates

armadillo image Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language and is widely used by (currently) 1286 other packages on CRAN, downloaded 42.6 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 659 times according to Google Scholar. This versions updates to the 15.2.2 upstream Armadillo release made two days ago. It brings a few changes over the RcppArmadillo 15.2.0 release made only to GitHub (and described in this post), and of course even more changes relative to the last CRAN release described in this earlier post. As described previously, and due to both the upstream transition to C++14 coupled with the CRAN move away from C++11, the package offers a transition by allowing packages to remain with the older, pre-15.0.0 legacy Armadillo yet offering the current version as the default. During the transition we did not make any releases to CRAN allowing both the upload cadence to settle back to the desired about six in six months that the CRAN Policy asks for, and for packages to adjust to any potential changes. Most affected packages have done so (as can be seen in the GitHub issues #489 and #491) which is good to see. We appreciate all the work done by the respective package maintainers. A number of packages are still under a (now formally expired) deadline at CRAN and may get removed. Our offer to help where we can still stands, so please get in touch if we can be of assistance. As a reminder, the meta-issue #475 regroups all the resources for the transition. With respect to changes in the package, we once more overhauled the OpenMP detection and setup, following the approach take by package data.table but sticking with an autoconf-based configure. The detailed changes since the last CRAN release follow.

Changes in RcppArmadillo version 15.2.2-1 (2025-11-21)
  • Upgraded to Armadillo release 15.2.2 (Medium Roast Deluxe)
    • Improved reproducibility of random number generation when using OpenMP
  • Skip a unit test file under macOS as complex algebra seems to fail under newer macOS LAPACK setting
  • Further OpenMP detection rework for macOS (Dirk in #497, #499)
  • Define ARMA_CRIPPLED_LAPACK on Windows only if 'LEGACY' Armadillo selected

Changes in RcppArmadillo version 15.2.1-0 (2025-10-28) (GitHub Only)
  • Upgraded to Armadillo release 15.2.1 (Medium Roast Deluxe)
    • Faster handling of submatrices with one row
  • Improve OpenMP detection (Dirk in #495 fixing #493)

Changes in RcppArmadillo version 15.2.0-0 (2025-10-20) (GitHub Only)
  • Upgraded to Armadillo release 15.2.0 (Medium Roast Deluxe)
    • Added rande() for generating matrices with elements from exponential distributions
    • shift() has been deprecated in favour of circshift(), for consistency with Matlab/Octave
    • Reworked detection of aliasing, leading to more efficient compiled code
  • OpenMP detection in configure has been simplified

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

19 November 2025

Dirk Eddelbuettel: digest 0.6.39 on CRAN: Micro Update

Release 0.6.39 of the digest package arrived at CRAN today and has also been uploaded to Debian. digest creates hash digests of arbitrary R objects. It can use a number different hashing algorithms (md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, spookyhash, blake3,crc32c, xxh3_64 and xxh3_128), and enables easy comparison of (potentially large and nested) R language objects as it relies on the native serialization in R. It is a mature and widely-used package (with 86.8 million downloads just on the partial cloud mirrors of CRAN which keep logs) as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation to quickly identify the various objects. As noted last week in the 0.6.38 release note, hours after it was admitted to CRAN, I heard from the ever-so-tireless Brian Ripley about an SAN issue on arm64 only (and apparently non-reproducible elsewhere). He kindly provided a fix; it needed a cast. Checking this on amd64 against our Rocker-based ASAN and UBSAN containers (where is remains impossible to replicate, this issue is apparently known for some arm64 issues) another micro-issue (of a missing final argument NULL missing in one .Call()) was detected. Both issues were fixed the same day, and constitute the only change here. I merely waited a week to avoid a mechanical nag triggered when release happen within a week. My CRANberries provides a summary of changes to the previous version. For questions or comments use the issue tracker off the GitHub repo. For documentation (including the changelog) see the documentation site. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel: #055: More Frequent r2u Updates

Welcome to post 55 in the R4 series. r2u brings CRAN packages for R to Ubuntu. We mentioned it in the R4 series within the last year in posts #54 about faster CI, #48 about the r2u keynote at U Mons, #47 reviewing r2u it at its third birthday, #46 about adding arm64 support, and #44 about the r2u for mlops talk. Today brings news of an important (internal) update. Following both the arm64 builds as well as the last bi-annual BioConductor package update (and the extension of BioConductor coverage to arm64), more and more of our build setup became automated at GitHub. This has now been unified. We dispatch builds for amd64 packages for jammy (22.04) and noble (24.04) (as well as for the arm64 binaries for noble ) from the central build repository and enjoy the highly parallel build of the up to fourty available GitHub Runners. In the process we also switched fully to source builds. In the past, we had relied on p3m.dev (formerly known as ppm and rspm) using its binaries. These so-called naked binaries are what R produces when called as R CMD INSTALL --build. They are portable with the same build architecture and release, but do not carry packaging information. Now, when a Debian or Ubuntu .deb binary is built, the same step of R CMD INSTALL --build happens. So our earlier insight was to skip the compilation step, use the p3m binary, and then wrap the remainder of a complete package around it. Which includes the all-important dependency information for both the R package relations (from hard Depends / Imports / LinkingTo or soft Suggests declarations) as well as the shared library dependency resolution we can do when building for a Linux distribution. That served us well, and we remain really grateful for the p3m.dev build service. But it also meant were dependending on the clock and cadence of p3m.dev. Which was not really a problem when it ran reliably daily, and early too, included weekends, and showed a timestamp of last updates. By now it is a bit more erratic, frequently late, skips weekends more regularly and long stopped showing when it was last updated. Late afternoon releases reflecting the CRAN updates ending one and half-days earlier is still good, it s just not all that current. Plus there was always the very opaque occurrencem where maybe one in 50 packages or so would not even be provided as a binary so we had to build it anyway the fallback always existing, and was used for both BioConductor (no binaries) and arm64 (no binaries at first, this now changed). So now we just build packages the standard way, albeit as GitHub Actions. In doing so we can ignore p3m.dev, and rather follow the CRAN clock and cadence (as for example CRANberries does), and can update several times a day. For example early this morning (Central time) we ran update for the then-new 28 source packages resulting in 28 jammy and 36 noble binary packages; right now in mid-afternoon we are running another build for 37 source packages resuling in 37 jammy and 47 noble packages. (Packages without a src/ directory and hence no compilation can be used across amd64 and arm64; those that do have src/ are rebuilt for arm64 hence the different sets of jammy and noble packages as only the latter has arm64 now.) This gets us packages from this morning into r2u which p3m.dev should have by tomorrow afternoon or so. And with that r2u remains Fast. Easy. Reliable. Pick all three! and also a little more predictable and current in its delivery. What s not to like?

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can now sponsor me at GitHub.

12 November 2025

Dirk Eddelbuettel: digest 0.6.38 on CRAN: Several Updates

Release 0.6.38 of the digest package arrived at CRAN today and has also been uploaded to Debian. digest creates hash digests of arbitrary R objects. It can use a number different hashing algorithms (md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, spookyhash, blake3,crc32c, xxh3_64 and xxh3_128), and enables easy comparison of (potentially large and nested) R language objects as it relies on the native serialization in R. It is a mature and widely-used package (with 86.8 million downloads just on the partial cloud mirrors of CRAN which keep logs) as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation to quickly identify the various objects. This release, the first in about fifteen months, updates a number of items. Carl Pearson suggested, and lead, a cleanup of the C API in order to make more of the functionality accessibly at the source level of other packages. This is ongoing / not yet complete but lead to several nice internal cleanups, mostly done by Carl. Several typos were corrected, mostly in Rd files, by Bill Denney, who also improved the test coverage statistics. Thierry Onkelinx and I improved the sha1 functionality. Sergey Fedorov improved an endianness check that matter for his work on PowerPC. I updated the blake3 hasher, expanded the set of ORCID IDs for listed contributors, updated the continuous integration setup, reinstated code coverage reports, refreshed / converted the documentation site setup, and made general updates and edits to the documentation. The release was prepared a week ago, and held up a few days until an affected package was updated: it requested raw returns where none were previously delivered (for xxhash64) but now are so it needed to not reqest them. It was then seen that another package made some assumptions about our DESCRIPTION file; this has been addressed at its end via a pull request we submitted (that remains unmerged). This delayed processing at CRAN for a few days and as it happens, hours after the packages was updated at CRAN today I heard from the ever-so-tireless Brian Ripley about an SAN issue on arm64 only. He kindly provided a fix; it needed a cast. Checking this on amd64 against our Rocker-based ASAN and UBSAN containers (where is remains impossible to replicate, this issue is apparently known for some arm64 issues) another micro-issue (of a missing final argument NULL missing in one .Call()) was detected. I plan to fix both of these in a follow-up release next week. My CRANberries provides a summary of changes to the previous version. For questions or comments use the issue tracker off the GitHub repo. For documentation (including the changelog) see the documentation site. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

11 November 2025

Dirk Eddelbuettel: duckdb-mlpack 0.0.4: Added random forest and logistic regression

A new release of the budding duckdb extension for mlpack, the C++ header-only library for machine learning, was merged into the duckdb community extensions repo today, and has been updated at its duckdb mlpack extension page. This release 0.0.4 adds two new methods (random forests, and regularized logistic regression), reworked the interface a little to now consistently provide fit (or train) and predict methods, adds a new internal state variable mlpack_verbose which can trigger (or suppress) verbose mode directly from SQL, expanded the documentation and added more unit tests. For more details, see the repo for code, issues and more, and the extension page for more about this duckdb community extension.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

4 November 2025

Dirk Eddelbuettel: RcppCNPy 0.2.14 on CRAN: Minor Maintenance

Another (again somewhat minor) maintenance release of the RcppCNPy package arrived on CRAN just now. RcppCNPy provides R with read and write access to NumPy files thanks to the cnpy library by Carl Rogers along with Rcpp for the glue to R. The changes are all minor chores. As R now checks usage of packages in demos, we added the rbenchmark to Suggests: in DESCRIPTION. We refreshed the main continuous integration script for a minor update, and also replaced one URL in a badge to avoid a timeout during checks at CRAN. So nothing user-facing this time! Full details are below.

Changes in version 0.2.14 (2024-11-03)
  • The rbenchmark package is now a Suggests: as it appears in demo
  • The continuous integration setup now uses r-ci with its embedded setup step
  • The URL used for the GPL-2 is now the R Project copy

CRANberries also provides a diffstat report for the latest release. As always, feedback is welcome and the best place to start a discussion may be the GitHub issue tickets page. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Next.

Previous.