A new maintenance release 0.0.7 of RcppXts is now on CRAN, and has been built for r2u. The RcppXts package
demonstrates how to access the export C API of xts which we
contributed a looong time ago. There are by now a more example packages
around this C level access to another package, but this one was an early
example.
This release is strictly maintenance, updating continuous
integration, the README.md file and other packaging conventions adopted
since the last release four years ago.
The NEWS entries follow.
Changes in version 0.0.7
(2026-09-09)
Corrected a docstring for the module
Updated continuous integration setup several times
Simplified setup by removing no-longer-needed Makevars
Debian LTS/ELTS
This was my hundred-forty-sixth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.
Unfortunately the number of distributed working hours had been rather low this month, so the list contains much less entries than normal. During my allocated time I uploaded or worked on:
[DLA 4713-1] sslh security update to fix one CVE in Bookworm related to a so-called link following
vulnerability.
[DLA 4714-1] libmodbus security update to fix one CVEs in Bookworm related to a stack-based Buffer Overflow vulnerability.
[DLA 4715-1] kissfft security update to fix two CVEs in Bookworm related to integer overflows.
Last but not least I spent days with FD work at the beginning of the month. The last remaining hours I continued to work on cups and hplip. Unfortunately this work did not result in an upload yet.
Debian Printing
This month I did not upload any package but just worked on some bugs.
This work is generously funded by Freexian!
Debian Lomiri
This month I worked on new Lomiri Apps. Due to a broken disk, the progress was not as expected. But stay tuned!
Not really related to Lomiri, but to Debian EDU, I fixed an incus related bug in sitesummary.
This work is generously funded by Fre(i)e Software GmbH!
Debian Astro
This month I uploaded a new upstream version or a bugfix version of:
supernovas to unstable, sponsored upload of a new upstream version.
Debian IoT
Unfortunately I had no time to work in this category this month.
Debian Mobcom
This month I uploaded a new upstream version or a bugfix version of:
Armadillo is a powerful
and expressive C++ template library for linear algebra and scientific
computing. It aims towards a good balance between speed and ease of use,
has a syntax deliberately close to Matlab, and is useful for algorithm
development directly in C++, or quick conversion of research code into
production environments. RcppArmadillo
integrates this library with the R environment and language and is
widely used by (currently) 1331 other packages on CRAN, downloaded 48.5 million
times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint
/ vignette) by Conrad and myself has been cited 727 times according
to Google Scholar.
This versions updates to the 15.6.0 upstream Armadillo release made
yesterday. It extends solver options for poorly conditioned systems, and
brings some updates and extension to the cube data type.
For this release, we once again ran the usual complete
reverse-dependency check which came back spotless, and did CRAN so no
email exchange needed despite nearly 1300 reverse dependencies (but it
ended up taking more than a single business day). Still, automation can
be helpful when used with a well-maintained software stack. The package
has also already been updated for Debian, built for r2u and r-universe, and
will build shortly at CRAN for
the different binary releases.
All changes since the last CRAN release follow.
Changes in
RcppArmadillo version 15.6.0-1 (2026-09-07)
Upgraded to Armadillo release 15.6.0 (Medium Roast Cortado)
Expanded solve() with
solve_opts::scale_thresh option to widen detection of
poorly conditioned systems
Expanded trans() and .t() to handle
cubes
Added permute() to rearrange dimensions of cubes
(generalised transpose)
Added cubemul() for batched matrix multiplication of
cube slices
This post is part of a series called "Pigeon". If this is the first post
you've found, it'd be worth reading the
intro post first and then looking over
all posts in the series.
The final step to all of this was to tie together all my PHY RF
code, Link layer parsers, and my
background with operating systems
to make this all feel like a normal thing my computer should be doing.
At the end of the day, I want my host system to know how to talk with a
pigeon daemon, so I don t have to reimplement basically everything else.
My ability to use normal tools like curl or ping6 is pretty important
here, so I need to reach for my old friend, the
TUN interface. The
TAP/TUN interface allows the kernel to route ethernet frames (TAP) or ip
packets (TUN) to a userspace program responsible for handling delivery and
reception avoiding the need for a kernelspace driver for something that
can be handled in userland.
wondering if doing this violates FCC Part 97 rules? I wrote up
notes on my test setup
for exactly this question, but the short answer is "no"!
I m nostranger to
playing with TAP/TUN, so this was pretty easy to snap together although this
time I avoided the whole ethernet proxying thing (to side-step lossy
translations, maintaining two sets of mac address tables, and handle proxying
NDP/ARP messages) it was kinda a bad idea last time so I just used TUN
and straight IP for now. While implementing this, I decided I d make a key
assertion about all pigeon networks namely, all pigeon IPv6 networks are a
/64 in size, no more, no less. The reason why I m doing this here is that,
since pigeond does still does need a MAC address for the pigeon layer 2
protocol, we can write our daemon to always use
SLAAC
to set the TUN IP address without any new information.
Which leads us to a bit of an aside, but I have a point, I swear. A few years
ago, my recreational RF adventures have lead me down a path where I decided to
engage with ARIN to solve (once and
for all) the massive headache I was running into with IPv6 numbering (really:
always renumbering) my multi-site radio processing networks. It s a lot
of work to keep running correctly, but it s solved a huge amount of
problems for me.
The only internal thing we really need to outline for this post is that, at
the highest level, my network (paultag.net) is split into an IP plan that
looks roughly like:
64 "sites" per region. A "site" is assigned to a physical or logical location.
/64
1024 subnets per site. A subnet used by directly attached devices.
For this exercise, I used IP space from paultag.net s experimental region
( region 8 ), named side.band (2602:810:6008::/48) to connect my RF lab
( site 1 - 2602:810:6008:400::/54), and my two pigeon-specific subnets,
subnet 0 (2602:810:6008:400::/64) and subnet 1 (2602:810:6008:401::/64)
to my wider network. The first subnet ( subnet 0 ) is a simple ethernet network
to enable my RF-only nodes to communicate with the side.band gateway. The
second subnet ( subnet 1 ) is an RF-only pigeon network local to my lab.
back to radios
With all that set up, I assigned my first two nodes their MAC addresses, and
set up the local RF only network segment. The nodes I brought online were the
following:
Callsign
IP
K3XEC/MN
2602:810:6008:401:8e1f:64ff:fe35:4001
K3XEC/TH
2602:810:6008:401:8e1f:64ff:fe35:4002
testing the pigeon network
And with that, I could begin to test that the host operating systems and RF
links could properly exchange data locally from SDR to SDR. We can use
ping6 to see if a plain-ole ICMPv6 ping round trips between hosts correctly:
$ ping6 2602:810:6008:401:8e1f:64ff:fe35:4001
PING 2602:810:6008:401:8e1f:64ff:fe35:4001 (2602:810:6008:401:8e1f:64ff:fe35:4001) 56 data bytes
64 bytes from 2602:810:6008:401:8e1f:64ff:fe35:4001: icmp_seq=1 ttl=64 time=426 ms
64 bytes from 2602:810:6008:401:8e1f:64ff:fe35:4001: icmp_seq=2 ttl=64 time=397 ms
64 bytes from 2602:810:6008:401:8e1f:64ff:fe35:4001: icmp_seq=3 ttl=64 time=419 ms
64 bytes from 2602:810:6008:401:8e1f:64ff:fe35:4001: icmp_seq=4 ttl=64 time=418 ms
64 bytes from 2602:810:6008:401:8e1f:64ff:fe35:4001: icmp_seq=5 ttl=64 time=436 ms
64 bytes from 2602:810:6008:401:8e1f:64ff:fe35:4001: icmp_seq=6 ttl=64 time=391 ms
64 bytes from 2602:810:6008:401:8e1f:64ff:fe35:4001: icmp_seq=7 ttl=64 time=397 ms
And it does! Latency is horrid (and there s a bunch of tx artifacts that
cause issues for us) but both of those things are problems for later. Let s
see how it handles a TCP connection by firing off a quick cURL across the
Pigeon network:
$ curl http://[2602:810:6008:401:8e1f:64ff:fe35:4001]:8000/testing.txt
The rock dove (Columba livia), also known as the common pigeon or rock pigeon
(but see also Petrophassa), is a member of the bird family Columbidae (doves
and pigeons).
As expected, our remote end here running the server reports the correct peer
IP address, which is another indication (beyond the log messages and blinking
LEDs) that we re routing over our TUN interface.
Serving HTTP on 2602:810:6008:401:8e1f:64ff:fe35:4001 port 8000 (http://[2602:810:6008:401:8e1f:64ff:fe35:4001]:8000/) ...
2602:810:6008:401:8e1f:64ff:fe35:4002 - - [28/May/2026 12:53:21] "GET /testing.txt HTTP/1.1" 200 -
That worked? First shot! Nice! It s pretty slow and seems like we have a
lot of packet loss, but it does, however, beg the question can it nethack?
nethack!
Yes! It can nethack! No clickbait here. The way I went about this one is a bit
anit-cimatic I set up a nethack server (using inetd in this case) on one
of the hosts pigeon0 network interface, and hit that port over RF from
the other:
However, when playing it, it becomes very obvious (as you can likely see) that
there s a fair amount of packet loss (understandable) and probably some packet
collisions taking place.
iperf
Let s try and put a number to exactly how bad the bandwidth and packet loss
is by running iperf between the two pigeon hosts over rf:
$ iperf -c 2602:810:6008:401:8e1f:64ff:fe35:4002
------------------------------------------------------------
Client connecting to 2602:810:6008:401:8e1f:64ff:fe35:4002, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 1] local 2602:810:6008:401:: port 58248 connected with 2602:810:6008:401:8e1f:64ff:fe35:4002 port 5001
[ ID] Interval Transfer Bandwidth
[ 1] 0.0000-20.2348 sec 76.8 KBytes 31.1 Kbits/sec
Shockingly, not nearly as bad as I thought it was going to be. Given I ve
spent exactly zero time making this operate to a level that I would call
acceptable, this is a very fucking solid start. I expect I could get that
number up if I spent a few weeks on it it s just not been a priority
at any point yet (and the first time I ve instrumented it, even!).
This ll be good enough to get started. Let s see what else we can pull
off here.
IP multicast to some rtl-sdrs
Back when I designed what I wanted Mode A to
look like, I intentionally picked a signal bandwidth that could be received
by an rtl-sdr so let s put that to use. It may go without saying,
but just to say it the rtl-sdr can not transmit, so this will be capable
of receiving pigeon frames but not sending any in reply.
However, this means I can use a bunch of low-cost computers (raspberry
pi-class), and low-cost SDRs (rtl-sdr) and still receive IP traffic from
transmitting pigeon network stations. This could be a lot of fun for things
like fountain coding a data
stream, or adapting multicast streaming protocols to work over RF links.
Anywho, I swapped my far end to an rtl-sdr (one config file change!), and
figured I d start with some (basic) multicast traffic, transmitting the time
once a second:
If I had more time to burn, I was planning on bridging
APRS traffic
to UDP multicast within a pigeon network subnet. However, since I m already 4
years late on this blog post, I figured this would be enough for now (and you
can imagine that fun project in this space if you so wish!)
I fired up pigeond again (except this time connected to an rtl-sdr),
and was pleasantly surprised to be greeted by some decoded traffic right
off the bat:
Looks great! tcpdump is showing multicast packets show up (as we assumed they
would), on the pigeon0 interface, on the machine connected to an rtl-sdr. Of
course, any replies will get sent to the
bit bucket, but it can definitely
decode things just fine! Very fucking cool.
Well right, ok! Let s go back to two rx/tx radios, and see what we can do with
our newfound network stack over ham radio frequencies let s try to do some
fun (and traditional!) ham radio things with it!
Winlink
Winlink is a ham radio mail relay system for ham radio
operators to send, receive or relay mail over the internet, or RF (usually HF
or VHF/2M). Winlink relays are accessible via whatever transport you can
find most commonly telnet (using the internet), ax.25 (usually 2m VHF)
or VARA HF (unsurprisingly, on HF). I use pat as my
Winlink client it s written in Go, doesn t require windows, and is just
generally nice to work with.
Let s try the easy thing first let s connect by proxying the Winlink server
into the pigeon network using socat (lightly edited to remove date/times)
$ pat connect pigeon
Connecting to WL2K (telnet)...
Connected to [2602:810:6008:401:8e1f:64ff:fe35:4002]:8772 (tcp)
[WL2K-5.0-B2FWIHJM$]
;PQ: 54509561
CMS>
>FC EM OLU6BP5HKMG2 240 205 0
>F> 95
FS Y
Remote accepted OLU6BP5HKMG2
Transmitting [Hello, World] [offset 0]
Hello, World: 100%
FF
>FQ
Disconnected.
$
Lo and behold, shortly after, I got this delightful message to my
email address, relayed in from WINLINK:
The only shame is I won t be able to check in to a winlink
wednesday using this scheme unless I further
proxy this message over AX.25 instead of relaying to Winlink s servers over
telnet (which, to be fair, is definitely also possible I just got lazy when
I glued this one together see note above about being 4 years late on this
post).
But, you know, connecting to a host that is using socat to proxy a
connection to an internet resource is interesting but you know what, fuck it
hang on, dear reader let s bang a hard left turn and just ship this thing
hard and directly connect it to the internet. Let s take our dinky,
home-built PHY and Layer 2 and see if we can wire it directly into the internet
something that, every time I go to think about it, reminds me of
Tim FitzHigham and his crapper.
Crossing the english channel in a bathtub
Ok, ok. I decided to bury the lede a bit here I didn t mention that the
side.band network is currently
BGP announced. Although we
haven t used it this does mean that we re most of the way to sending packets
to the wider internet, and we should be able to just fix a few routing
tables, and see packets begin to flow.
After tweaking the local routing tables (and restarting pigeond for good
measure), I decided to test my newfound connectivity by pinging something over
our new network transport.
ping github
Why don t we start with the world s premier software engineering platform,
operated by one of the largest companies in the world, GitHub! After all,
they have an all knowing (and, apparently, arguably sentiant?) AI on hand to
instantly and automatically fix any stray
reliability issues in the
background, so we should definitely see replies right off the bat:
$ ping6 github.com
ping6: github.com: Address family for hostname not supported
Wait, oh no that can t be right?
After all, it s 2026, and both
Google
and CloudFlare
(in North America) are reporting
over half of all traffic they see is IPv6 and GitHub still doesn t support
IPv6? Definitely not, this is for sure a bug with my code or network.
lets ping something that supports ipv6 instead
That being said, just for completeness sake, since that error is also given
when there s no IPv6 DNS record, let s go ahead and double check with
Hurricane Electric too, you know, just to be sure.
$ ping6 he.net
PING he.net (2001:470:0:503::2)56 data bytes
64 bytes from he.net (2001:470:0:503::2): icmp_seq=1 ttl=53 time=514 ms
64 bytes from he.net (2001:470:0:503::2): icmp_seq=2 ttl=53 time=230 ms
64 bytes from he.net (2001:470:0:503::2): icmp_seq=3 ttl=53 time=248 ms
64 bytes from he.net (2001:470:0:503::2): icmp_seq=4 ttl=53 time=246 ms
64 bytes from he.net (2001:470:0:503::2): icmp_seq=5 ttl=53 time=265 ms
64 bytes from he.net (2001:470:0:503::2): icmp_seq=6 ttl=53 time=240 ms
Well, shit. Right, OK, i ll be damned. 18 years in and GitHub still can t crack
that nut.
cURL works!
Right, anyway, yes, back on track good news! Our uplink is up and routing,
and wait, holy shit! Check it out! pigeon is
exchanging packets with the internet and no one is any the wiser!
Literlaly amazing. Let s try a cURL across the internet now (although no
TLS allowed, so, http only for now):
IRC works, too
Sweeeeet. That all works! Forget HTTP, let s do some other 90 s era
stuff, it s high-time to log into IRC with a quick /connect -notls, and see
what s going on in the #debian-hams channel pleased that I got online
fairly quickly, and was able to even talk to myself!
THE GOPHERSPACE
Naturally, let s keep this train of nostalga running, and give
the 2026 gopherspace a
shot.
I know the kind folks over at tilde.town (hello,
townies!) have a robust gopherspace, so let s give it a dial! Let s try and see
if we can load vilmibm s
slug over gopher:
Yes! I forgot to make this one a video, so no gif. I did wind up having a bit
if trouble with a few gopher clients and IPv6 support I may send some
patches if I can find the time.
So, what s next?
Alright, that s it. I have a few more fun ideas but they re going to have to
wait for another day. Carrying IP is fun and all but kinda not the point behind
pigeon, after all. Rather than trying to make this into a thing , I m planning
on exploring the loose ends first different types of modulation schemes
(like QAM-NUC), implementing LDPC error correction and some layer 2 logic
into the pigeond (like switching traffic, and gain control). I also plan
on spending some time with my (currently, very basic) simulator to better dial
in tradeoffs throughout the stack.
Since, structurally, pigeon is something I feel like I can work with, I m
hoping i ll be able to find the time for some (much smaller!) followup posts
without it taking 4 years this time. If I do, they ll show up under the
pigeon tag and I ll be sure to update this post with
a link below (and the intro post).
I m hoping that this series (which was supposed to be one post) was helpful
to someone out there if it was, feel free to reach out and let me know!
This post is part of a series called "Pigeon". If this is the first post
you've found, it'd be worth reading the
intro post first and then looking over
all posts in the series.
If you're looking for it, the intro to the Link (layer 2) that this belongs at a
high level and listing of the parts that make it up (including this one) is on
the link post.
Built-in to the pigeon link protocol is a message type called cal (short for,
you guessed it, calibration). A pigeon frame with a type of cal
(which is 0x02) carries a JSON encoded payload in the body, which can
either be a beacon, requesting signal reports in response,
or a report, describing the received beacons.
This serves a few interesting purposes firstly, network operators can better
understand the coverage footprint, propagation under different conditions, and
how gain impacts reception when tuning for the lowest practical power levels.
Secondly, this can be used (and I plan to eventually implement!) to construct
a mapping of minimum power level and peer mac address to dynamically control
the transmission power based on the destination station.
That being said, for now, all I ve used this for is getting a rough sense for
what gain value(s) make sense between two nodes (manually). In the future,
beyond all the fancy neighbor gain stuff, I plan to wire this into the daemon
to happen automatically, debouncing for beacon and report messages, such
that transmitting stations only beacon, and receiving stations only report a
max of once over some time period for a given peer.
Version
Given all the above, I do intend to make some massive changes to this protocol
(I promise to blog all about it) when I get around to hacking on switching
Layer 2 frames within a network segment. Just to avoid having to dig myself out
of a hole later, I m going to explicitly send (and check) the version field
to avoid having a big flag day switchover or needing to use a new linktype.
Version ID
Description
V1
this version of the cal protocol
Location
Both flavors of cal messages (beacon and report) may contain a location,
which is the location that the beacon was transmitted, or for or the location
where the beacon was heard for a response. This can be used to derive a
coverage map and to (operationally) better understand what stations should be
within range, and generally what gain level(s) are effective.
All fields assume WGS84 latitude and longitude values, and elevation is
distance, in meters, above the WGS84 ellipsoid NOT height above
sea level, or altitude above the ground.
Field
Description
lat
WGS84 Latitude
lon
WGS84 Longitude
elevation
height, in meters above the WGS84 ellipsoid
Sequence
Each beacon contains a Sequence identifier, which is used to communicate
which message number is being heard, and how many total were transmitted
by the originating station. The current approach with Beacon messages is to
transmit some number of Beacon messages at different gain levels, each
with a unique Sequence identifier.
Field
Description
number
beacon sequence number
total
total number of beacons transmitted
Gains
Recorded gain setting(s). For a Beacon this indicates the gain settings
(which, in spite of its name, includes things like amplifiers, or attenuators).
Changing this over different Beacon frames enables a better understanding
of what an appropriate gain level is for the transmitting station over time.
Field
Description
name
gain stage name
db
gain value, in dBm
Cal
All messages contained in a cal frame are of this type. The type field
communicates if this is a Beacon or Report message type.
Type
Description
beacon
sent intermittently by idle stations
report
reception report in response to a beacon
Beacon
A beacon message may be sent periodically by pigeon nodes capable of
transmitting to announce their prescience to peers and, implicitly, to receive
signal reports from nearby listeners who are capable and configured to transmit
reports.
The beacon JSON message is made up of the following fields:
Report
A report message may be sent in response to a beacon message by
pigeon nodes capable of receiving and transmitting to assist with setting the
lowest usable gain value, and to better understand the area of coverage and
propagation.
The report JSON message is made up of the following fields:
This post is part of a series called "Pigeon". If this is the first post
you've found, it'd be worth reading the
intro post first and then looking over
all posts in the series.
Now that we have a working Layer 1, we have a way to send a block of bits
from one place to anyone who cares to listen to us. This is very welcome
news, but we are now facing a new, different and just as fun question what
shape should that data take?
Need a bit more of a crash course on what "Layer 1" and "Layer 2" mean? No
problem, I wrote up short summary here to
help.
Given our incredibly limited functionality of our nodes, we could definitely
skip all this work and just stuff an IP packet into the link; but I decided
to not since I am (eventually) interested in adding some sort of
spanning tree-like
protocol to implement network
switching so not all nodes
need to communicate directly with all other nodes but that day is not today.
Given i m going to stub most of that out, let s take a look at what some
similar Layer 2 protocols use things like Ethernet or WiFi frames. Both
contain structured information regarding the transmitter, desired recipient,
type of data, and the higher-level data itself (such as IP packets). As a
result of attempting to learn from others, the Pigeon Layer 2 (called, simply,
link ) is also split into a fixed-length header, followed by the contents
described by the header.
The header is a fixed-length (23 byte) structure, which contains the source
MAC address (src mac), destination MAC address (dst mac), the ITU
coordinated ham radio callsign of the control operator of this message
(callsign), the type of payload to follow (type; defined below), and
the length of the data to follow the header (length as a 16 bit big-endian
unsigned integer).
The type field indicates how the payload is to be interpreted currently
I ve only defined 3 possible payload types so far:
Keen observers will perhaps infer that there used to be an Ipv4 type at
0x03 which is true however, i ve since removed it since i ve never once
used it and the codepath was more trouble than it was worth. As is my wont,
I ve optend to just lean into Ipv6-only IP transport it s easy enough to
shim ipv4 in, if someone REALLY wanted to using something like 64:ff9b:1::/48
and a bit of code in the transmitter/receiver (or even using something like
jool and unbound s
dns64-prefix
at the router). I don t think I ll bring it back, but just in case I have
to for some reason in the future, it s there.
Additionally, friends of the pod may also recognize that this structure is
basically the exact same structure as what I had in PACKRAT,
which, is also true. I started this project off maintaining interoperability in
the Layer 2 for pigeon and packrat, but at some point just gave up on it during
one of the many cleanups. I m hopeful I can maintain compatibility going
forward, and that won t have to muck with this header too much more. We ll see
what happens once I start to push the bounds of what is possible with pigeon.
Hopefully it feels like carrying IP data inside this frame to be a pretty
self-explanatory exercise the 0th byte of the payload is the 0th byte of
an IPv6 header (followed by all the usual stuff, like UDP or TCP header(s) and
any carried data, just like you d find anywhere else.
Pigeon s calabration protocol
This post is part of a series called "Pigeon". If this is the first post
you've found, it'd be worth reading the
intro post first and then looking over
all posts in the series.
If you're looking for it, the intro to the PHY (layer 1) that this belongs at a
high level and listing of the parts that make it up (including this one) is on
the phy post.
While developing Pigeon, I ve called the group of all the configuration of the
Layer 1 PHY parameters the Mode . I ve experimented with a few different
modes , but one in particular has been the most resilient to the innumerable
mistakes and bugs i ve wrought into existence and that is the first mode
I wrote down, Mode A . This is even (mostly) backwards compatible to my
original Go implementation of Pigeon Mode A (back in 2022) over the air,
and has largely withstood the problems I ve thrown at it.
I ve removed the bulk of the support I wrote out for other modes, but i m
likely to bring them back over time as I use pigeon to learn more (such as
Mode B (QAM-16), Mode C (QAM-16 NUC), and Mode AW which is the exact same
as Mode A, except 5MHz in bandwidth. More to come on those as I get further
along but for now let s braindump the parameters i ve picked out for Mode A:
The last bit to describe here is the
Subcarrier Plan. The plan is ordered
negative first (meaning the 0th bin in-memory is the most negative frequency
domain bin of the fft), and
within a Mode A OFDM symbol, there are 64
frequency domain bins (so, just to
make it explicit: 64 subcarrier usages that make up our Mode A
subcarrier plan ).
We ll follow the same structure and conventions that we went through in
the post all about OFDM Symbols which means,
we ll need to place our guard bins, data bins, and pilot bins. I ll include
a copy-paste-able version of the images to follow at the end.
Guard Bins
First up, let s place our guard bins.
As we ve already gone over, we re looking to clear some space right up against
the high and low end of the frequency range, so let s go ahead and do that:
I gave up the center bin (0 Hz) and 8 of the 64 bits on each side (1/4 of the
signal!) to give myself a bit of elbow room. This is perhaps definitely a bit
overkill, but it s been an extremely robust choice. If you multiply that
through, this accounts for 312.5 kHz of frequency domain padding at the high
and low end of the bandwidth, or 625.0 kHz of bandwidth which is not to be used.
Pilot Bins
Next up was the pilot bins. We ve already gone
over the purpose (and
use) of our pilots, but I ve found there to be an art
to the placement of the pilots. Interpolation between pilot bins has turned out
to be very reliable, but extrapolation, on the other hand, has been a major
pain, for reasons I don t fully understand yet.
My intent in placement was to pick out roughly even stretches of data bins
bracketed between pilots, with as few data subcarriers as practical outside
of a pilot (using extrapolation). I ve played a bit with my AGWN simulator(s),
as well as logging errors between two SDRs, and the configuration I have this
in has been fairly resillant (for whatever reason), and withstood a few rounds
of tweaking.
Data Bins
Almost as an afterthought all of the remaining bins become
data bins.
This puts the total number of data bins at 43, which, since Mode A carries data
in QPSK/QAM-4 (two bits per data subcarrier), means we can carry 86 bits of
data per OFDM symbol. That fairly modest capacity is largely due to the
modulation scheme (or fft size, but increasing that has been fraught) we re
using for Mode A but I ve made up for it by including 168 OFDM symbols
in a single burst in order to have enough data to carry IP traffic without
splitting the packet into two bursts.
With all that designed and on paper, we re ready to start to tackle the next
layer up our Layer 2, named, creatively, link .
Let s send some link layer data
Following along at home? Nice! As promised, I've put the full plan
copy-pasted from the pigeon source tree below so that no one has to feel
the need to transcribe this from the images above. Unlike most of the things
I've "left to the reader", manual transcription from images is not a
particularly useful task for anyone to do.
The following table is Mode A s OFDM Subcarrier Plan. This is in negative
first ordering (meaning the 0th member is the most negative fft bin,
and the Nth is the highest frequency fft bin).
Still following along at home? Nicer! I've put the
preamble sequence
promised previously
copy-pasted from the pigeon source tree below in case that's actually
important (I don't think it is, but...).
The following table is Mode A s frequency-domain preamble. This is, as above,
in negative first ordering. I don t actually think these values matter much
(at all)? but in case they do, here s what I have. I muck with these a lot
and haven t found many changes in quality of detection or frequency correction
yet.
This post is part of a series called "Pigeon". If this is the first post
you've found, it'd be worth reading the
intro post first and then looking over
all posts in the series.
If you're looking for it, the intro to the PHY (layer 1) that this belongs at a
high level and listing of the parts that make it up (including this one) is on
the phy post.
The time has come.
If you re following along at home, we now have all the basics we need to glue
these parts together and see what this looks like.
We re going to build the highest-level constructs for the PHY in code
something that takes some number of bytes in and writes out IQ samples fit for
transmit over the airwaves (we ll call this the Encoder), and something that
takes chunks of IQ samples in, writing out decoded bytes (which we ll call the
Decoder).
Encoder
Let s begin with the Encoder, since it s slightly less involved. I ve tried
to make this a bit more accessible by drawing a diagram out before describing
the order of operations, so that it s possible to follow along visually.
While the process here can look like a lot, it s really not that bad.
We begin by taking the incoming bytes, converting the bytes into bits, and chunk
those bits into parts which are sized to fit completely within an LDPC message. We will then
encode incoming data into LDPC messages, using our
configured LDPC Matrix (the table we appropriated from 802.3an). Next, we
apply whitning over all the bits in our
encoded (and packed) LDPC messages, using our configured whitening constant. In
the case of QPSK, pairs of bits will then be
modulated into a QAM subcarrier, where each
QAM point represents a range of bits in the message. We ll go through each of
those modulated IQ subcarriers, and
set each corresponding data subcarrier
in order, for each OFDM symbol contained in the pigeon Burst. The preamble configuration
is then used to generate (or, more likely, can be used at startup to precompute)
the Schmidl-Cox preamble, which is written to the
first IQ samples in our output IQ buffer. Finally, we will do a series of
inverse FFT operations to convert each
OFDM symbol to the time domain, including their cyclic prefix.
Let s take a look at doing that, but in code this time now:
// (lightly edited for clarity)
impl Encoder
../// Encode the provided bits into the output time-domain IQ samples.
fnencode(
&mut self,
dst: &mut [IQ],
src: &Vector,
) -> Result<Burst, Error>let src =letmut raw = Vector::new(self.fec.message_len());
// Set raw 's data bits, compute and set LDPC
// checkbits.
self.fec.add(&mut raw, src);
// Apply whitening, and return
raw.xor(&self.whitening)
;
// copy in the precomputed schmidl-cox preamble to dst
let preamble_len = self.preamble_iq.len();
dst[..preamble_len].copy_from_slice(&self.preamble_iq);
// allocate a new (frequency domain) 'Burst' container.
letmut burst = Burst::new(
&self.mode.ofdm.plan,
self.mode.ofdm.symbols
);
// modulate bits from 'src' as iq, and set each
// data subcarrier for each ofdm symbol in the
// burst.
self.burst_encoder.multiplex(&mut burst, &src);
// convert from frequency-domain data into time
// domain iq samples, writing out ofdm symbols
// and cyclic prefixes to dst .
self.burst_encoder
.transform(&mut dst[preamble_len..], &burst)?;
// normalize all IQ samples; the maximum magnitude
// in the IQ buffer may be very small, which weakens
// our transmitted signal. Scale all IQ samples such
// that the maximum IQ sample magnitude will be '1.0'.
dst.norm();
Ok(burst)
Using the Encoder should hopefully be fairly straightforward we ll
give it a bag of bytes, and get back some IQ samples that we can ask our
nearest SDR to transmit.
As for what happens on the other end?
Decoder
Next up is the mirror image of our Encoder the, imaginatively named,
Decoder. The Decoder is slightly more involved (since it has to find the
packet in the IQ stream, as well as correct for channel error(s)), so we ll do
the same thing as above start with a diagram. My hope is going over the
Encoder first helps us only really focus on the new stuff, otherwise it
should feel like running the Encoder backwards.
Here, we start with an incoming stream of IQ, where we will process
scan detections as they come in
from our Schmidl-Cox detector and burst Scanner. This will give us a snippit
of IQ, sized to exactly our Burst. We ll begin to correct our IQ samples by
first doing
frequency estimation and correction
in the time domain using our preamble and ofdm configuration. We ll then do
a series of inverse FFTs to extract each OFDM symbol
in our Burst, where we can then do
channel estimation and correction.
With the OFDM symbols (hopefully) good enough, we can now
map each data subcarrier back to bits, and
unapply whitning. The resulting
bits are then chunked back up into LDPC messages,
which are then checked, and concatanated data extracted. Finally the bits
are turned back into bytes, which are written to our output buffer.
However, before we get into the code to do this there s one last detail. We
know bursts won t overlap (if they do, it s likely not possible to recover
right now even though other PHYs can and do), so any time we see something we believe
to be a burst, we can skip ahead by the burst s (constant) length within
the IQ, and avoid trying to decode anything else in there.
The nice side-effect here is this also gives us an interesting property for the
Decoder namely, we know the maximum number of Burst detections we can
get for a given block of incoming IQ data if they were packed end-to-end and
we can pre-allocate the memory we need, avoiding allocations for every
demodulation attempt (which may or may not even be a valid Burst).
This pre-allocated block of memory to hold the burst s data is something that
I ve called a frame buffer internally. Each frame buffer contains exactly
sized buffers to hold decoded information from the burst an iq buffer
that is exactly the same number of samples required to encode the preamble
and data, exactly the number of bits needed to store pre and post FEC data,
pre-allocated byte array, etc.
Not shockingly, the code looks like this:
#[derive(Clone)]pubstructFrameBuffer/// Corrected IQ samples
pub samples: Samples,
/// post-correction OFDM burst
pub burst: Burst,
/// demodulated bits from the OFDM burst
pub bits: Vector,
/// demodulated bits from the OFDM burst,
/// after FEC, and cleaned
pub raw_bits: Vector,
/// Layer 2 contents of the Frame
pub contents: Vec<u8>,
Of course, that alone is handy but we need to use them. So let s go ahead
and do what we promised above each Decoder uses a fixed number of
pre-allocated FrameBuffers to store packets in-flight, packed into what
is, creatively, called FrameBuffers within my code.
As an aside, I likely should have called this a
Memory Pool, since that s the
commonand accepted
name for this design pattern but being stuck with unfortunate names is
the burden of those of us who stumble into sensible ideas over time. The only
nuance here is that I use the pools strictly sequentially we only save
the FrameBuffer if the LDPC checksum is correct, allowing us to only keep
track of how many successful packets we have and being able to get the valid
FrameBuffers, rather than storing a handle to each FrameBuffer as we go
a promise that most memory pools do not make, since blocks can usually be
taken and returned in any order.
Let s go ahead and do the whole Decoder dance now:
// (lightly edited for clarity)
impl Decoder
../// Process incoming IQ for Pigeon Bursts, and
/// demodulate them.
pubfndecode(
&mut self,
buf: &[IQ],
) -> Result<Vec<(Detection, &FrameBuffer)>, Error>letmut ret = Vec::new();
let mode = self.scanner.mode().clone();
// reset the "valid frame buffer count" back to 0
self.frames.reset();
// call the scanner and get scan detections for
// this block of iq ( buf )
for detection in self.scanner.scan(buf)
// for each detection, we're (only) going to process
// the iq snippit, but pass along the metadata
// such as SNR.
let ScanDetection
snippit,
snr,
range,
m,
= detection;
// grab the next free frame buffer to work within.
let frame_buffer = self.frames.next_mut();
// Copy the snippit into the frame buffer (a mutable
// location)
frame_buffer.samples.copy_from_slice(snippit);
// estimate the frequency offset based on the
// Burst's Schmidl-Cox preamble.
let preamble_fo = preamble::estimate_frequency_offset(
&mode.preamble,
mode.rate,
&frame_buffer.samples[..mode.preamble.samples()],
);
// Shift the IQ stream by the estimated frequency
// offset -- hopefully we're closer to 0Hz
frame_buffer.samples.shift(mode.rate, preamble_fo);
// estimate the frequency offset based on the
// each burst's **cyclic prefix** -- exactly like
// we did with the Burst Schmidl-Cox preamble,
// but this time on each OFDM symbol.
let ofdm_fo = ofdm::estimate_frequency_offset(
&mode.ofdm,
mode.rate,
&frame_buffer.samples[mode.preamble.samples()..],
);
// Shift the IQ stream closer yet; hopefully this
// is a very small nudge even closer still to 0Hz.
frame_buffer.samples.shift(mode.rate, ofdm_fo);
// Do a bunch of inverse fft operations for each
// OFDM symbol, filling the frequency-domain Symbol
// structs in frame_buffer.burst (Burst) struct.
//
// this will also do channel estimation and
// correction before returning.
self
.decoder
.transform(
&mut frame_buffer.burst,
&frame_buffer.samples[mode.preamble.samples()..],
)?;
// "demultiplex" each data subcarrier's frequency-domain
// IQ constellation point, setting the correct bit range.
self.decoder.demultiplex(
&mut frame_buffer.raw_bits,
&frame_buffer.burst
);
// unapply whitening by XOR-ing the buffer with
// the well-known whitening vector.
frame_buffer.raw_bits = frame_buffer.raw_bits.xor(
&self.whitening);
// verify that the LDPC message(s) are all correct,
// and if so, concatanate the the data bits (no check
// bits) to the bits vector.
if self.fec.decode(
&mut frame_buffer.bits,
&frame_buffer.raw_bits
).is_err()
// this is where invalid packets fail. we gave it a good go.
// next packet please.
continue;
// copy the raw bits out, as bytes, to the contents buffer.
frame_buffer.bits.copy_as_bytes(&mut frame_buffer.contents);
// store metadata/metrics on the demodulation.
ret.push(Detection
m,
snr,
index: range.start,
);
// save the contents of this frame buffer (don't
// reuse this buffer next go-around).
self.frames.save();
// We're going to take out the borrow on the frame at
// the end since we don't want to deal with telling the
// compiler via code gymnastics that the mut and non-mut
// borrows are OK since they're non-overlapping.
Ok(ret.into_iter().zip(self.frames.iter()).collect())
Phew. That was kinda a lot. In fact it s basically the whole thing. This
function is as close to how do you read an OFDM packet as it gets, and
perhaps the most important part of this whole series. Beyond that, though, this
is a huge conceptual unlock. This means we now have an incredibly powerful
primitive; the ability to take bytes and go to/from IQ samples over the air.
Let s talk about pigeon modes
I asked the banker who is my reference at the bank something like this:
Give that we are talking about the consequences of the tantrum of a fascist
foreign government, what happened to them (who are also people close and dear
to me), in some future can very well happen to me.
Suddenly my risk profile shot up under the roof.
What do you suggest me to do? Should I find a trusted source of gold bullions
to bury under the cellar at home?
The answer was something like this:
Sadly YES, given that the USA have a sort of financial monopoly they can
entitle themselves to arbitrarily define a person/organization as a terrorist
without any trial or judicial course, and as a consequence apply sanctions
that cannot be effectively counteracted, not even abroad.
I didn't have this in my 2026 bingo card, but here we are.
For more details, see:
the A/I press release under "12. Banca Etica: the sanction also affected an Italian bank account"
I asked the banker who is my reference at the bank something like this:
Give that we are talking about the consequences of the tantrum of a fascist
foreign government, what happened to them (who are also people close and dear
to me), in some future can very well happen to me.
Suddenly my risk profile shot up under the roof.
What do you suggest me to do? Should I find a trusted source of gold bullions
to bury under the cellar at home?
The answer was something like this:
Sadly YES, given that the USA have a sort of financial monopoly they can
entitle themselves to arbitrarily define a person/organization as a terrorist
without any trial or judicial course, and as a consequence apply sanctions
that cannot be effectively counteracted, not even abroad.
I didn't have this in my 2026 bingo card, but here we are.
For more details, see:
the A/I press release under "12. Banca Etica: the sanction also affected an Italian bank account"
My Debian contributions this month were all sponsored by Freexian.
You can also support my work directly via Liberapay or GitHub Sponsors.
Personal note
This month, my Dad unexpectedly passed away after a short illness. As a result I obviously got less work done than usual, and I still have a lot to take care of (since I m the executor of his will, as well as helping with funeral arrangements) while grieving and generally having less focus and energy. Having routine work to do is one of the ways I cope with this sort of thing, but all the same, I hope people will bear with me and maybe remind me if I seem to be dropping the ball on something you especially need.
LLM vote
[Content note: strong opinions.]
I voted in General Resolution: LLM usage in Debian. My vote was pretty much the opposite of what ended up winning, so I m quite disappointed. My personal opinion is that LLMs are cognitive hazards to their users that impose ecological costs far out of proportion to their utility at a time when the world absolutely cannot afford them. When the impossible economics of the large commercial models are finally allowed to catch up with reality, I expect there to be significant macroeconomic consequences, and that people who have become dependent on them will have problems; and who knows what the copyright situation on their output really is. I m not convinced that local models are better enough on these axes to be worth the costs.
Debian s direct contribution to all that will be negligible on a global scale, and even the most radical proposals in the GR didn t expect that we could do much about upstreams that have gone all-in on LLMs. Even so, I d hoped that my fellow developers might be more willing to lean on our position in the free software ecosystem to make at least a moderately radical statement. Instead, we ve at best presented an undistinguished fence-sitting position to the world, and further entrenched the idea that humans can reliably do a good job of reviewing the output of tools that are designed to produce output plausible to humans. I certainly don t trust my own code review skills that far.
Since I ve never voluntarily used an LLM (not counting LLMs being foisted on me by things like search results, support chatbots, or incoming pull requests, regardless of whether I asked for them), and don t intend to for the foreseeable future, I doubt this will change much for me in terms of the way I work. The winning option is a very weak one that imposes no new requirements on developers, which means that it also does nothing to stop me continuing to reject LLM-generated material from Debian bug reports and merge requests in my areas of responsibility. I know this probably won t do much to satisfy people who have decided that Debian is slop now, but it s the best I can do.
OpenSSH
I finally landed the GSS-API key exchange package split in our OpenSSH packaging. Here s the NEWS entry:
I fixed a flaky autopkgtest.
I upgraded from 10.4p1 to 10.5p1, which was a good test of keeping openssh and the new openssh-gssapi source package in sync.
PuTTY
I upgraded from 0.84 to 0.85.
Python packaging
New upstream versions:
The version treadmill continues: we ve just finished dropping Python 3.13 as a supported version, so now we ve started working on enabling Python 3.15 as a supported version. Maximiliano Curia has been very helpfully driving this. I didn t get as much done here as I d have liked (see the top of this post), but I fixed a couple of packages:
The AVIF image format gets more and more popular in the web dev community, so I needed to teach XFCE4's thunar (file manager) and Ristretto (image viewer) to thumbnail these.
Luckily that is not too hard:
Debian Trixie separates its gdk-pixbuf libraries slightly differently than previous versions. That's why it is not "automatically there". Ensure you have the libavif-gdk-pixbuf plugin and the tumbler service (which XFCE uses to process thumbnails):
Thunar has likely tried (and failed) to load your AVIF files before you installed the package, it will have saved a blank or "broken image" placeholder in a thumbnail cache directory. It will not attempt to regenerate them unless you clear this cache:
# Clear the thumbnail cache rm-rf ~/.cache/thumbnails/*
# Force-quit thunar and the tumblerd background service thunar -q pkill tumblerd
Tumbled will restart on its own when it is needed. When you open thunar again and navigate to your image directory ... your AVIF images will now generate thumbnails automatically like the other image format did already.
I am a heavy user of sidenotes:1 they keep optional content next to the
text instead of sending the reader to the bottom of the page and back. Tufte
CSS renders them without JavaScript but only accepts inline content. CSS
anchor positioning, now supported by recent browsers,2 is an
elegant alternative. Sidenotes can hold several blocks, still without
JavaScript, and fall back below the paragraph referencing them on narrow
viewports and older browsers.
In 2023, Eric Meyer demonstrated this technique in Nuclear Anchored
Sidenotes. The main improvement over other solutions is that the notes can
sit anywhere in the HTML document. You can place them after the paragraph
referencing them, as regular block elements for text browsers, screen readers,
feed readers, and reader mode to render them properly:
Rendering in Lynx, a text browser
When the viewport is too narrow or the browser does not support CSS anchor
positioning, you can style them so the reader can skip them or glance at them
without losing their position in the text:
Rendering below the paragraph on a narrow viewport
Once the viewport is large enough, they appear in the margin, at the same
vertical position as the matching reference mark, unless they would collide with
a previous sidenote, as in the example below:3
Rendering in the margin on a large viewport
The gist of CSS anchoring is to position an element relative to another
element the anchor. For the sidenotes, the anchor is the reference mark. I use
the following markup, with a data attribute to specify the anchor name:
The matching note is an <aside> element carrying the same data attribute for
the anchor name. We put it after the paragraph holding the reference mark:
<asiderole="note"id="sidenote-YYY"data-anchor="--lf-sn-YYY"><sup>1</sup><p>A first paragraph.</p><p>A second paragraph.</p></aside>
On a narrow viewport or when the browser is too old for CSS anchoring, we style
the sidenote, which stays below its paragraph, with a muted color:
aside[role="note"]margin-block:1rlh;color:#444;
On a wide viewport and when the browser is recent enough, we move the sidenote
to the right margin:
attr() extracts the anchor name for the reference mark from the data-anchor
attribute. It returns a string, unless we specify a CSS unit or a type, like
here: the browser parses the data attribute as a custom
identifier, which anchor-name validates as a dashed
identifier, a custom identifier starting with two
dashes.4
The note itself is absolutely positioned past the right edge of the main block.
It selects the matching reference mark as its anchor with position-anchor set
to the value of the data-anchor attribute. Each note is also an anchor named
--lf-sidenote. We use it to keep the next note from colliding with this one.
The anchor() CSS function lets us position the note s top
edge relative to its anchor: anchor(top) aligns the top edge of the note with
the top edge of the reference mark. It can also take another anchor as a
parameter: anchor(--lf-sidenote bottom) would align the top edge of the note
with the bottom edge of the closest preceding anchor named --lf-sidenote so
the previous note.5 Like attr(), anchor() accepts a fallback
value as its second parameter and use it when the named anchor does not exist.
The top property handles three cases, illustrated in the following diagram:
The three cases for the vertical position of a note
The first note s top edge aligns with the top edge of its reference mark: as
there is no previous note, anchor(--lf-sidenote bottom, -1rlh) + 1rlh
resolves to 0 and max() returns anchor(top).
When the reference mark of a later note sits above the bottom of the previous
note, plus some vertical space, the note goes below the previous one to avoid
a collision. max() returns anchor(--lf-sidenote bottom) + 1rlh.
Otherwise, the note s top edge aligns with the reference mark s top edge, as
max() returns anchor(top).
Have a look at the complete stylesheet, which also adapts the
reference mark to the location of the note: a arrow when the note sits below
the paragraph, a arrow when it moves to the margin. Gwern s Sidenotes In
Web Design lists more implementations and their trade-offs.
Some bloggers aim to write a post in 30 minutes. I planned to publish
three web-related articles this weekend. Instead, I spent an inordinate amount
of time elsewhere: about 15 commits on the build system, a pull request to
update CSS highlighting for nested selectors in Pygments, and a
small correction to MDN s article on the anchor() CSS function. The
SVG illustration took a bit less than an hour and the article itself a handful
of hours. The attr() function came in after I thought inline style looks
ugly, isn t there a better way? But, hey, I still think this is worth it!
My PhD advisor told me this is unwise.
The first bits of anchor positioning are supported from Chrome 125 (May 2024), Firefox 147 (January 2026), and Safari 26 (September 2025).
Before Safari 26.5, sidenotes may collide due to a bug in how dependency
chains are handled. You can detect this situation with some JavaScript.
It is, however, not needed in the solution described here as we depend on a
more recent feature.
If you noticed the runt in the first note, I share your pain and lament that
Firefox does not implement text-wrap: pretty.
Typed attr() is supported from Chrome 133 (February 2025),
Firefox 155 (September 2026), and Safari 27 (September 2026). Check Una
Kravets article for details. To support more
browsers, you can inline the anchor name and the position anchor directly in
the HTML:
The exact rule for the target anchor element is more complex: if an
ancestor of [the note] satisfies the following conditions, return the
nearest such element to [the note]. Otherwise, return the last element in
tree order that satisfies the conditions. One of these conditions is that
[the candidate] is an acceptable anchor element for [the note], which
requires that [the candidate] is laid out strictly before [the note],
where the relevant clause is that [the candidate] is either not absolutely
positioned or occurs earlier in the flat tree order than [the note].
About the project and me
Your program crashes. You open gdb and get ?? instead of a stack trace. So
you go find the right -dbgsym package, for the right version, for the right
architecture, install it, and start again.
Debuginfod removes that
entire detour: gdb asks a server for symbols by the build-ID baked into the
binary. Debusine already built packages, already produced -dbgsym files, and
already hosted the archives; it just couldn t answer the question.
This summer I made it answer. My project was to add debuginfod server
functionality to Debusine so that it not only hosts -dbgsym packages, but
also serves their debug symbols over the
debuginfod(8)
protocol. Debian developers can then debug binaries by setting a single URL
that gdb uses to fetch the matching debug symbols. This project took me
through design, backend work, an extraction pipeline on the worker, HTTP
serving, documentation, and testing from the first blueprint all the way to
a live demo on debusine.debian.net.
Initial planning and design changes
A design first, in
!3030.
The proposal submitted for GSoC 2026 was just an overview of how things will
work, but in reality there were a lot of design questions which needed to be
answered before starting with contribution. Debusine keeps development
blueprints in its docs tree, reviewed like code, it s basically a blueprint
of what feature or new changes are we going to make. I was assigned the work
item
#957,
which was basically about how the idea of implementing a debuginfod server
functionality inside Debusine was initially proposed by a fellow member
which later became a project idea under GSoC 2026. My developer blueprint
pinned down the four decisions everything else depends on: extraction
happens on the worker after the build, symbols are stored as artifacts keyed
by build-ID, they re published into suites alongside their binaries, and
they re served from the archive root rather than per-suite. Settling that up
front meant the design discussions happened in a document instead of across
three merged branches.
One of those arguments became its own fix. My wording implied symbols were
unpacked inside the isolated sbuild environment (the consequence was I was
handed a
bug to
be solved in the first week of contribution period), when they re actually
extracted afterwards on the worker, where the build output already sits, a
distinction that matters, because doing work inside the unshare environment
means extra tooling in the chroot and more ways to affect the build.
!3119
corrected it before the wrong model spread into the code.
A new artifact type
Artifacts
are a major concept in Debusine overall, so as per the developer blueprint
we introduced a new artifact which was
debian:debug-symbols.
It holds every .debug file from one -dbgsym package. Its data is a validated
list of lowercase 40-character build-IDs, and each file is stored under its
build-ID as the path, so answering what are the symbols for this ID? is a
direct lookup, with no path translation in the request handler. One artifact
per package rather than per file: a util-linux build would otherwise spray
hundreds of artifacts, collection items and relations across the database
for no benefit. For implementing debian:debug-symbols artifact, I changed
the main
models.py
file, along with that since it s a norm to write unit tests, all mentioned
under
!3088.
Publishing workflow and solving a bug
Extracting symbols is only useful if they reach the archive people actually
install from, so
!3180
taught package_publish to follow the relates-to relation: copying binaries
into a suite now brings their debug symbols along automatically, with
nothing extra for the publisher to configure. Each build-ID becomes its own
collection item, for example debugsym:hello_2.10-5_amd64_fcc9064 each
carrying the package name, version and architecture copied from the binary,
so the item is meaningful on its own without dereferencing anything.
Uniqueness is enforced at both the suite and archive level, because the
serving URLs are archive-wide and two suites must never disagree about what
a build-ID means: republishing an identical file is accepted quietly, while
two different files claiming the same ID is an error worth failing on. A
partial index on the build-ID keeps the eventual HTTP lookup fast.
That looked finished until symbols started arriving in target suites
disconnected from their binaries published, but unfindable, because copying
items between collections silently dropped their artifact relations, and
that relation is the only thing tying the two together. The fix sat one
level above my feature, in the generic CopyCollectionItems task that does
the copying, and since it was reusable infrastructure rather than anything
debuginfod-specific, Colin implemented it himself in
!3228.
My project needed it to work at all; every other Debusine feature that
copies items now gets it for free.
Endpoint and CI tests
With symbols in the archive,
!3212
added the part users actually touch: GET / scope / workspace /buildid/<build-id>/debuginfo looks the ID up across
every suite in that workspace s archive, streams the file, and sets the
X-DEBUGINFOD-FILE and X-DEBUGINFOD-SIZE headers the protocol expects. It
also handles the two things gdb actually does: a HEAD probe before
committing to a download, and ranged requests to pull individual ELF
sections instead of the whole file. Scoping it to the archive rather than
the suite is what lets one URL cover a whole workspace, so the developer
never has to know which suite their binary came from.
Every merge request above landed with unit tests, but those only tell you
that the pieces behave correctly. What Colin and I wanted was a real gdb
fetching real symbols from a real instance, so
!3261
adds an autopkgtest that builds a package, publishes it, checks the HTTP
headers, then sets DEBUGINFOD_URLS and makes gdb go and get the symbols,
wired into the CI integration tests so it runs on every change. It took me a
day to learn that skipping the signing worker doesn t simplify that test, it
just hangs until the 30-minute timeout, because update_suites needs signing
to produce a usable repository.
The last piece,
!3301
covers the new artifact, the suite and archive changes, the new archive URL,
and a how-to for using it. My first how-to draft explained how everything
worked and offered four ways to set DEBUGINFOD_URLS; the version that
shipped gives one recommended setup and gets out of the way. The same pass
trimmed the blueprint down to only what s still unimplemented, since a
design document describing merged code is just an obstacle for the next
reader.
What s left
Only one item on my original plan didn t land: an archive-level
build_debug_symbols switch, modelled on Launchpad s equivalent, letting an
archive skip building -dbgsym packages entirely by passing
DEB_BUILD_OPTIONS=noautodbgsym to sbuild. It was always the stretch goal
rather than core scope, landing the extract-publish-serve path solidly
mattered more than landing it broadly. The design is written up in the
blueprint, and I intend to implement it myself.
The other gaps were deliberately out of scope from the start, and the
blueprint says so. DWZ supplement files aren t ingested, so packages using
compressed debug info may render without the alternate strings table;
debugging still works, it s just less complete. Source-file serving runs
into the same Debian packaging limits that constrain debuginfod.debian.net
today, making it a design question rather than a coding one. Executable
serving, the metrics and metadata endpoints, and federation to upstream
debuginfod servers were excluded for similar reasons, none of them are
needed for Debusine s core use case, and each would have crowded out the
parts that are.
One open bug is left too. On the last day of the coding period, Stefano
Rivera found that publishing ledger and linux was failing, because I had
told the database that a build-ID identifies one exact debug file which
isn t true in Debian, since dh_dwz runs once per binary package, so when one
object ships in two binary packages their .debug files differ while
describing identical code. How to fix it is still an open discussion
#1582,
though it may not land before the formal end of the project.
None of that is a handoff. GSoC s timeline is ending, my involvement isn t,
I m carrying on with Debusine until both the build_debug_symbols switch and
DWZ supplement support are merged, and I expect to keep contributing beyond
that. This project got me familiar with a codebase I enjoy working in, and
the remaining pieces are mine to finish.
Thanks!
The biggest thanks go to my mentor, Colin Watson, whose reviews consistently
found the thing I hadn t thought about. He also gave me room to get things
wrong first and understand why, which taught me more than being handed the
answer would have.
Thanks as well to Rapha l Hertzog, Enrico Zini, Stefano Rivera, Carles Pina
i Estany and Helmut Grohne and everyone else around Debusine and Freexian
for reviews, comments and patience with my questions.
Special thanks to Freexian for developing Debusine in the open and for
giving me access to test on
debusine.debian.net.
Finally, thanks to the wider Debian community, whose build-ID and -dbgsym
conventions did most of the hard work before I arrived and to Google Summer
of Code for providing a platform and the time to do this properly.
Looking at the reactions to the Debian AI vote, I think some people still think
the clock can be turned back, as if that ever worked in history. Rather than cry
about spilled milk, I prefer to find a path forward in the new world. There are
many ways to use LLMs, some of them are straightforward, others not so much.
One of the not so clear areas for me is the focus on agentic workloads. For
complex tasks, sure, you want something that can work in the background, but in
general, why does every single tool go the agentic way? I much prefer the
chat/ask approach, or even the code one, but if I m at the keyboard, why
would I send a task to an agent, and see it work, instead of directly
implementing it?
And then, this past Friday, I finally understood one part of that. I was in the
airport, sitting at the gate and waiting to board a flight, and because I
arrived much earlier at the airport (fearing crowds due to Labour Day weekend),
I got one hour of work before boarding started. As the time for boarding
approached, I did one more
commit
after making sure tests pass, pushed, closed laptop, and went to walk a bit
before getting on the plane.
As I was getting up, I get a phone notification from GitHub that the CI run
failed.
I was quite surprised, as the local tests passed, so I open the notification,
and realize that tests via make test vs CI (which additionally uses
--pedantic) had slightly different settings, and of course I missed a build
warning (which in CI is an error).
I thought I d fix that on the plane, but then I saw a Copilot agent button in
the mobile app. I was curious what it did, I click it, and I see Copilot
starting a draft pull request, and saying:
Thanks for asking me to work on this. I will get started on it and keep this
PR s description up to date as I form a plan and make progress.
Fix the failing GitHub Actions job. Analyze the Actions logs, identify the
root cause of the failure, and implement a fix.
Then it goes, finds the failure, writes the fix, and tries to run the tests.
Well, it can t do it (it runs in a restricted container, so no network, so
stack install couldn t actually work). The agent sees that, acknowledges it
has no way to validate the fix, but the error message was clear enough that it
was confident the fix is mostly correct, so it sends the pull request.
I allow full CI to run on the pull request, and go buy a bottle of water. After
that, I check and see that the CI failed again, as not one but two test files
were broken, and I didn t have --keep-going, so the build stopped at the first
failure. I write a comment in the pull request, no reaction, I realize I need to
tag Copilot explicitly, I do that, and it starts another investigation.
I m waiting now in the boarding queue, with phone in hand, while Copilot is
fixing my bug. While I scan my boarding pass and walk towards the plane, the
pull request is updated, I trigger another CI, it passes, and I merge it.
And then, it hit me. Agents allow me to make progress while being not at
keyboard , whether that s physically not at keyboard , or while working on
something else. Fixing a simple test failure is not something that needs human
attention per se, whereas improving the test layout might be.
In that airport, using otherwise-unusable downtime, and without explicitly
intending to, I made progress in understanding a different way to use AI. Now I
have three ways to work with LLMs: ask (tutor mode), code (implement my
request), and agent (fix simple or complex problems, autonomously). I still
don t know about plan mode and really complex tasks, like asking it to
implement features from scratch. That will probably be the next area to tackle.
And today (Sunday), while waiting for a running race to start, I opened GitHub,
and asked Copilot to increase test coverage for a simple module. It did, and yes
it still can t run tests (I learned in the meantime that you can
configure
the environment in which the agent runs, nice), but after two back-and-forth
messages, I have a pull request
ready to review. All in the 20 minutes before a race, where I could either
browse social media or actually do some meaningful work.
Checking now my GitHub billing, it looks like all of this Copilot use only cost
$1.92. Yes, that is under two dollars! And while it did use compute resources,
the person across the aisle who watched TikTok or Instagram for half an hour
while waiting for takeoff also consumed a lot of compute, and so do the
gazillion cat videos uploaded to YouTube every day.
To me, this is another tool in the toolbox, that might one day replace me (as it
did to the 19th-century textile workers), or make me five times more productive
we ll see where we end up. In the meantime, I can move faster, and make better
use of my limited free time.
Enjoy the ride!
Another minor maintenance release of the RcppFarmHash
package is now on CRAN as
version 0.0.4.
RcppFarmHash
wraps the Google FarmHash family of hash
functions (written by Geoff Pike and contributors) that are used for
example by Google BigQuery for the FARM_FINGERPRINT
digest.
This releases updates several of package internal files for
continuous intergration and package data.
The brief NEWS entry follows:
Changes in version 0.0.4
(2026-09-06)
Minor updates to continuous integration, README.md and
DESCRIPTION
I have just tried CoMaps, a free mapping program released under the Apache license [1]. I have tried it on Android on a Pixel 6a but it also runs on Linux so I ll try it on a PinePhone or similar at some convenient time. On Android it is in the F-Droid repository among others and for Linux there s a Flatpak package.
The data it uses is from Open Street Map project [2] which has extensive and accurate coverage of every place I ve looked at (Australia and a few other first-world countries). The first thing it does after being installed is start downloading the world data set from Open Street Map and prompt to download the data for the detected region (Melbourne in my case).
The UI is decent and allows most of the features that I am used to using in Google Maps. The quality of directions seems good, I ve only tested it with one journey so far which was a 50 minute drive across the city and it gave a set of directions that Google Maps often gives.
It gives spoken directions which is an important feature but sometimes the way the directions are presented is confusing. When turning off a freeway it didn t give a spoken direction to do that, it gave a direction to turn right which was AFTER leaving the freeway, fortunately the map was clearly displayed.
In terms of use practices of this program the main difference I recommend is checking which off ramp to use from a freeway before entering the freeway. With Google Maps you can rely on it giving clear directions in that case.
I recommend this program without reservation. It can do everything that Google Maps does apart from detecting traffic jams because there s no way of detecting traffic without spying on users. It is designed to preserve user privacy and works well in that regard.
After having witnessed how easy it is for good people to lose a .org domain
over a fascist tantrum (you can follow the Autistici/Inventati story
here and
here), I've started moving all my infrastructure to
differently managed TLDs.
enricozini.org and enricozini.com will keep being functional for the time
being, as dropping a domain makes it available for squatting and impersonation.
These new domains are now online, with working web and emails:
ezini.it (probably equally subject to fascist
tantrums these days)
It will take ages to migrate countless accounts that are tied to my primary
email address, so better start early.
Waiting to see what will happen with .meow domains,
which I supported despite not identifying as a cat.
After having witnessed how easy it is for good people to lose a .org domain
over a fascist tantrum (you can follow the Autistici/Inventati story
here and
here), I've started moving all my infrastructure to
differently managed TLDs.
enricozini.org and enricozini.com will keep being functional for the time
being, as dropping a domain makes it available for squatting and impersonation.
These new domains are now online, with working web and emails:
ezini.it (probably equally subject to fascist
tantrums these days)
It will take ages to migrate countless accounts that are tied to my primary
email address, so better start early.
Waiting to see what will happen with .meow domains,
which I supported despite not identifying as a cat.