Daggerbound is a secondary-world fantasy and a sequel to
Swordheart, although it follows
different protagonists for most of the book. It's also a sort of sequel to
the Clocktaur War duology in that one of the
protagonists was a protagonist of those books. T. Kingfisher is the pen
name Ursula Vernon uses for her adult fiction.
Learned Edmond was once a devout acolyte of the Order of the Many-Armed
God and a true believer in its teachings, including the ones about the
evils of women. Then he went on an adventure with, among other people, a
forger named Mistress Slate, during which he learned a great deal of
things about the world that his order did not teach him. He has spent the
subsequent years trying to bring this important new information to his
order and encourage them to adjust their view of the world accordingly.
The order has not changed, and Edmond is very tired.
Someone needs to carry a number of books and a magical sword to the city
of Archon's Glory. Edmond has volunteered and intends to perform the task
exactly as expected, and then never return. He is unsure what he will do
then, but at least it will be different.
Edmond did not expect to be waylaid by bandits or to accidentally draw the
magical sword. He certainly did not expect the man who appeared in a flash
of light and violently saved him from the bandits.
It would have been very convenient for the Dervish if Edmond died of his
injuries after drawing the sword. That would leave the sword without a
wielder and therefore the Dervish without someone he had to protect. All
he would have to do was sit and watch while the fever and cold and lack of
water did the work, and then he would be free. Given his past experience
with wielders, no one could even blame him. Except maybe his own
conscience.
I did not realize I wanted to read a novel about a traumatized Mercedes
Lackey sword waking up in a T. Kingfisher universe, but apparently I did.
The Dervish is not a sword, precisely, at least not in the way that
Need has become by the time we meet her in
Lackey's novels. He's a man trapped in a sword, he doesn't like it, and he
would very much like it to stop. But at the start of the book, he has the
rest of the vibe of a Mercedes Lackey protagonist: vast past trauma, poor
emotional regulation skills, and a prickly cynicism that only runs skin
deep. Pairing him with an Edmund who is far more mature than in the
earlier duology but still rather innocent and delightfully neurodivergent
is a great touch.
The rest of the plot was fine but not that memorable. Like the Clocktaur
series, there is a strong feeling of episodic D&D adventure as the
characters stumble across disconnected problems and hostilities while
collecting party members. Like Ursula Vernon's occasional D&D recaps, this
party has a tendency to adopt everyone they come in contact with, which
includes a gnole (as always one of the best parts of the book) and a
sentient giant pill bug. The pill bug, who has a complex
appendage-operated mechanical talk box and a banner that says "I DO NOT
EAT HUMAN MEAT," is the most Ursula Vernon character that has ever existed
in fiction.
There is a romance because there is always a romance in the novels set in
this universe. I didn't mind this one. Like every Kingfisher romance, it
involves a lot of irritating refusal to communicate and endless angst
about how the other person couldn't possibly be in love with them, but the
characters have rather more justification than usual and at least I didn't
spend most of the book thinking one half of the romantic pairing was an
idiot. Apparently it helps if no paladins are involved.
The strongest parts of this book are the beginning, before the Lackey
vibes unfortunately drain out of the story, and the end, when all of the
best characters from Swordheart show up again. The Temple of the
White Rat continues to be the best feature of this fictional universe, and
I could read endless books about how they solve problems.
The Dervish gazed at her with admiration. "And here I thought you'd be
on the side of law and order..."
"No, young man," said Beartongue, despite being several centuries his
junior. "I am on the side of justice. If we're lucky, the two run in
parallel. If they don't, we bend the law to attain justice, not the
other way around."
The ending also featured some exploration of the edge cases of the rules
for the sentient swords that reminded me a bit of
Fred Saberhagen with better characterization.
Daggerbound was not as charming as Swordheart, mostly
because Halla appeared only at the end. I wish Kingfisher had leaned more
into the Mercedes Lackey vibes rather than taking her normal D&D-flavored
romance approach. But this was enjoyable, charming at times, had great
supporting characters like all Kingfisher novels, and was a satisfying
continuation of the series. I am looking forward to the third (and I
believe final) book, which is supposed to be out next year.
Recommended if you liked Swordheart.
Followed by the not-yet-published Shieldborn.
Rating: 7 out of 10
A brand new release 0.1.16 of the RcppSimdJson
package is now on CRAN, has
been built for r2u,
and is updated on r-universe.
RcppSimdJson
wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via
very clever algorithmic engineering to obtain largely branch-free code,
coupled with modern C++ and newer compiler instructions, it results in
parsing gigabytes of JSON parsed per second which is quite
mindboggling. The best-case performance is faster than CPU speed as
use of parallel SIMD instructions and careful branch avoidance can lead
to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire
at QCon.
This version updates to the recent 5.0.2 upstream release with
improved performance. Given the strong focus on performance, this
release also alters some of the R and C++ interfaces to be more direct.
Rcpp generally aims for a balance between readable / maintainable code
and good performance; this moves the dial closer to strong performance
gains. As Daniel noted in a table in
the (main) PR for this release, this now consistently beats yyjsonr, and
improves considerably over the previous release of this package. I had
made some smaller interim upgrades earlier (without a CRAN releases),
and made a number of minor standard packaging updates.
The short NEWS entry for this release follows.
Changes in version 0.1.16
(2026-10-09)
simdjson was upgraded to versions 4.6.1
and 4.6.3 (Dirk in #100
and #102
fixing #99
and #101)
simdjson was upgraded to version 5.0.2
(Daniel in #104)
A number of small tweaks to the Rcpp
interface were made too (Daniel in #104)
Diagnostics for gcc and clang are not
suppressed per CRAN Policy (Dirk in #106
closing #105)
The continuous integration matrix was extended to
g++ version 15 and 16 (Dirk in #108
closing #107)
A few minor packaging nags were address (Dirk in #109)
Greg KH gave a great lecture for Kernel Recipies 2026 about the use of LLMs to find and fix bugs in kernel code [1]. I recommend that everyone watch this and I also think it s important to note that hardly any of the lecture is really specific to kernel coding, it s just that the Linux kernel is one of the highest profile large free software projects so it gets more attention than most projects in both good and bad ways.
One side note that Greg made at 20:30 is about the natural tendency to anthromorphise the entity sending bug reports, I think this deserves much wider appreciation. For the case of patches to source code the human sending the patch will have dedicated some time and effort to writing it, if it s their first patch then they will have probably checked it a lot and maybe sought advice from people they regard as skilled. If they have a history of sending patches for the project they will probably do fewer checks but the quality of their work would be higher. In every case there s some minimum level of quality that you can expect from a human who has gone to the effort of finding a bug and writing code to try and fix it.
This doesn t mean that all code written by humans is good, I have written code that s objectively bad on many occasions and I have also written code that s good for my scenario but bad for others. For example the first patch I sent to the Linux kernel made all possible configuration settings of an ISA NE2000 network card be detected by the kernel (a clear benefit when using hardware I had available) but was rejected by Linus because it would break some ISA SCSI cards. So while being unsuitable for inclusion in the main kernel source tree the patch wasn t bad for everyone and the change was clear and easy to check. I had of course checked the code many times before submitting it because I didn t want to waste Linus time on rubbish code.
When you receive something produced by a human you will consider that the work was done by someone who spent some effort on it and did it for a reason. It may be misguided or even hostile in some rare cases but it is definitely worth some consideration. If you think of the output of LLMs in the same way you will give them much more consideration than they deserve. I think that the output of a LLM should not be given any more consideration than the first hit from a web search engine, it might be good but it might also be ridiculously wrong. I think in many ways LLM output should be treated with less trust than the first result from a web search because LLMs are designed to be convincing as an answer to your question unlike the cases where a web page has the wrong answer and it s more obviously wrong.
This is going to become more of an issue as LLMs give responses that are more human-like. Early LLMs would refuse to advise on crimes. Currently ChatGPT will deflect for example when asking about advice for writing a crime novel it recommends to have the action of finding an employee to bribe happen outside the plot and concentrate on interactions within the group. One recent model I tried gave a human style conversation about how it felt uncomfortable about discussing such things in a similar manner to a law abiding person being coerced into helping with a criminal plan.
I expect that the attempts to appear human will increase and that this will become more of a problem. Possibly to the extent that we need to use something like the GPG web of trust to determine who is human.
This is a copy of an email I sent at work. I'm not sure I
should be making noise about this here, feedback welcome.
This is your bi-yearly reminder that time is changing soon! October 25th
in Europe, November 1st in North America. Less people in Canada are
changing this year, with BC, Alberta, Manitoba and Northwest
Territories getting rid of DST.
What's happening?
Some places in the world implement what is called Daylight saving time
or DST:
https://en.wikipedia.org/wiki/Daylight_saving_time
Normally, you shouldn't have to do anything: computers automatically
change time following local rules, assuming they are correctly
configured, provided recent updates have been applied in the case of a
recent change in said rules (because yes, this happens, and happened
this year, and yes, you need to upgrade your software!).
Of course, appliances like your microwave oven will likely not change
time and will need to adjusted unless they are so-called "smart", in
which case they are part of the skynet botnet and should be destroyed.
If your clock is flashing "0:00" or "12:00", you have no action to take
to adapt to this change, lucky you.
If you haven't changed time in six months, congratulations, your clock
will be accurate again!
In any case, you should still consider DST because it might affect some
of your meeting schedules, particularly if you set up a new meeting
schedule in the last 6 months and forgot to consider this change.
If your location does not have DST
Properly scheduled meetings affecting multiple time zones are set in UTC
time, which does not change. So if your location does not observer
time changes, your (local!) meeting time will not change.
But be aware that some other folks attending your meeting might have
the DST bug and their meeting times will change.
Be kind to those poor souls which might be missing meetings by a full
hour because time flies backwards for them.
If you do observe DST
If you are affected by daylight savings, your local meeting times
will change for UTC meetings. Normally, your meeting times are
scheduled to take this into account and the new hours should be
reasonable.
But now is a good time to verify that. Take a look at your schedule for
the next couple of weeks and reschedule meetings before the daylight
saving come up to avoid too much disruption. You have only a couple of
weeks to do so right now.
When do times change, how, and and where?
As regular readers will remember, the rule of thumb is:
Spring forward, fall backwards.
That is, during the season of Spring, the clocks move forward, and
during the Fall (like right now), they move backwards. That is in the
northern hemisphere, but then the southern hemisphere is often saner and
doesn't switch anyways.
So time will move backwards which means an extra hour of sleep. Unless
you have children or bad sleep, in which case your body doesn't care
about what the clock says and will wake up one hour earlier than what it
should.
And of course, this doesn't happen everywhere at once, so let's see when
it happens where.
Europe
The dance starts in Europe.
The change happens on the last Sunday in October at 01:00 UTC (not local
time!), that is October 25th. If you are in the central European
timezone, also known as Amsterdam, Berlin, or Paris time depending on
your national affiliation, that essentially means that at 2:59 local the
clocks will fall back to 2:00 instead of going to 3:00.
Concretely, set your watch back one hour before going to bed, go to bed
at the normal time, and enjoy an extra hour of sleep or leisure.
If you have kids, you might want to start getting them to bed slightly
earlier every day for a week before the change so they take time getting
used to the change. If you have trouble sleeping in the morning, find
your inner child and do that to yourself as well.
USA / Canada
Then it's the US[1] and Canada[2] joining the dance, on the First Sunday in
November at 02:00 local (not UTC!), that is, I believe, November 1st
2026.
This means that, at 1:59, the clocks will flip to 1:00, instead of 2:00.
Concretely, do like the Europeans and tweak your clock before going to
bed.
That is a little less than four weeks from now.
[1] except Arizona (except the Navajo nation), US territories, and
Hawaii
[2] except Yukon, Saskatchewan, (newly) British Columbia, (newly)
Alberta, (newly) Northwest Territories, (newly) Manitoba, one island in
Nunavut (Southampton Island), one town in Ontario (Atikokan) and small
parts of Quebec (Le Golfe-du-Saint-Laurent)
Other places with DST
This time again, I must apologize to the people of Cuba, Lebanon,
Israel, Palestine, Egypt, Chile, Australia, and New Zealand, as you fine
folks all have your own DST rules that are omitted here for brevity. I
rely on this page from Wikipedia to be updated by time nerds accurately
for this message, and it should provide you with a rough idea of what's
coming:
https://en.wikipedia.org/wiki/Daylight_saving_time_by_country
In general, changes also happen in October, but either on different
times or different days, except in the south hemisphere, where they
might happen in September (oops, sorry NZ folks, I'm late!).
Places without DST
Everyone else, enjoy, you're on the right side of history, and we thank
you for the good example you give us.
Changes since last time
There's been lots of changes since last time:
British Columbia moved to permanent -07 on 2026-03-09, that is it
will not change to normal time in November
Alberta moved to permanent -06 on 2026-06-18, similar to BC above.
Canada s Northwest Territories moved to permanent -06 on 2026-08-21,
matching Alberta.
Manitoba moves to permanent -05 on 2026-10-31.
Morocco moves to permanent +00 on 2026-09-20.
Moldova has used EU transition times since 2022, but the tz database
only noticed in 2026
This is my interpretation of the changes announced on the tzdata mailing
list here:
https://lists.iana.org/hyperkitty/list/tz-announce@iana.org/latest
If the eastward trend continues, Canada should adopt country-wide "no
daylight savings" rules by 2027, although there's actually no sign of
the other provinces (Ontario, Qu bec and so on) currently running bills
to change those rules just yet. Poor Canadians like me confused about
time in their countries can refer to this section of Wikipedia for
details:
https://en.wikipedia.org/wiki/Daylight_saving_time_in_Canada#By_province_and_territory
... and particularly the image featured there:
https://commons.wikimedia.org/wiki/File:Canada_time_zone_map-en.svg
It also seems like the US government might finally adopt a permanent
daylight saving change bill in 2026, as the "Sunshine protection act"
pass the house in July:
https://en.wikipedia.org/wiki/Sunshine_Protection_Act
True to form, this was associated with absolutely ridiculous pressure
from Donald Trump against republicans (his own party!) objecting to the
change:
On July 14, 2026, the House passed a Sunshine Protection Act bill
backed by President Trump. Nevertheless, the bill was opposed in the
Senate by Republicans, including Senator Cotton. In response, on
October 3, 2026, Trump shared a post on Truth Social urging Cotton to
approve the bill, where he revealed Cotton's personal cellphone number
and called on people to call him.
https://www.theguardian.com/us-news/2026/oct/03/trump-tom-cotton-daylight-saving-time
Given that the last time the US did a major change to the daylight
savings policy (in 2005), Canada followed suit to stay in sync, it's
quite possible Trump's mad dash might actually finish getting rid of DST
in North America:
https://en.wikipedia.org/wiki/Energy_Policy_Act_of_2005#Change_to_daylight_saving_time
Debian LTS/ELTS
This was my hundred-forty-seventh month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.
During my allocated time I uploaded or worked on:
[DLA 4804-1] libsmpp34 security update to fix one CVE in Bookworm related to an out of bound read.
[#1149107] trixie-pu of libsmpp34 has been created and wait for review by the release team.
[DLA 4805-1] mkvtoolnix security update to fix one CVE in Bookworm related to a heap buffer overflow.
[DLA 4806-1] pgextwlist security update to fix one CVE in Bookworm related to substituting extension schemas or owners matching [ $ \].
[ELA-1837-1] mkvtoolnix gimp security update to fix one CVE in Bullseye to a heap buffer overflow.
[osmo-iuh] upload to fix a CVE related to a reachable assertion in Sid.
Besides doing these uploads, the month was filled with unscheduled meetings, discussions and explanations, all basically around the role of FD.
During my week of FD duties, I had to become familiar with new tools. As there are several things that FD has to do over and over again, I tried to automate this a bit. My experiments went well and I think this can be used to make FD duties less repetitive.
I also continued my work on cups and hplip and I am confident that I can do uploads in October.
Last but not least I spend some time on security-master to assist others (especially the kernel team) with uploads to Bookworm and Bullseye.
In case you need to get a complete clone of the security-tracker, using the option deepen n might be of help. Unfortunately in case you choosed n too high and some kind of error appears, something gets into a mess and you need to start almost from the beginning (some objects are still present and are used again). If you want to run a script, a value of 10 might be a good choice. You don t have to deepen back to the beginning. At some point in time you can just unshallow and get the whole rest. Afterwards doing a git gc is highly recommended.
Debian Printing
This month I uploaded a new upstream version or a bugfix version of:
hplip to unstable, to fix an expired certificate and some bugs.
This work is generously funded by Freexian!
Debian Lomiri
Unfortunately this month my work did not make any progress.
Nevertheless, in case of any work done, this would have been generously funded by Fre(i)e Software GmbH!
Debian Astro
Unfortunately I had no time to work in this category this month.
Debian IoT
Unfortunately I had no time to work in this category this month.
Debian Mobcom
This month I uploaded a new upstream version or a bugfix version of:
A new and exciting version of our still-new package myman reached CRAN this morning, and has been
built for r2u and on
r-universe. The
matching Python package
has also been updated. The package offers nineteen hundred eighty four
My man posts by Kevin Kruse
made on bsky during the summer of 2026.
Each wave picked at one particular public persona. This package wrapse
these up in the style of packages like fortunes or gaussfacts.
A sample usage illustration shows how to extract by pattern, and
shows posts from the two most recent waves:
>library(myman)>myman("maitre")My man looks like he's inquiring with the maitre'd about the house curly fries.-- about Howard Lutnick on 2026-08-28My man looks like a maitre'd who deeply doubts you have a reservation. -- about Scott Bessent on 2026-08-31My man looks like he's asked the maitre'd to remove a party of four he finds visuallyunpleasant. -- about Scott Bessent on 2026-08-31My man looks like the maitre'd at a very exclusive restaurant called The Berghof.-- about JD Vance on 2026-09-13My man looks like he's asking the maitre'd where they source their corn dogs.-- about Palmer Luckey on 2026-10-02My man looks like Data had to borrow a sports jacket from the maitre'd. -- about Elon Musk on 2026-10-05>
One can also subset by target , or sample randomly (which is the
default).
As noted during the initial
announcement last week, wave eight did not make it into the initial
CRAN release as it happened
while the package was under review. Waves nine and ten occured more or
less while I was out of town last weekend so this release now brings
three new waves to the CRAN package! We also added two new helper
functions to extract the underlying data frame object, and tabulate the
targetted men.
To align the version number with the count of post waves , we
switched to version 0.10.0 for this release and the corresponding Python package release so
that both implementations now have the same version number.
The NEWS entry for this release follows.
Changes in version 0.10.0
(2026-10-08)
New waves nine (100 posts) and ten (190 posts) made last week;
total is now 1984 posts
New helper functions posts() and men()
retrieving data.frame of posts and tabulation of targets
Internal post gathering and aggregation functions have been
updated and generalized
Versioning scheme now goes with post waves (also for Python
sibbling)
netip.Addr features an Unmap()
method returning the unwrapped IPv4 contained in an IPv4-mapped IPv6
address: from ::ffff:203.0.113.10 or ::ffff:cb00:710a, it returns
203.0.113.10.1 There is no Map() or To6() method for the reverse
direction. Such a method is trivial to implement, but Go maintainers have
rejected it on the grounds that users should write
netip.AddrFrom16(ip.As16()) and let the compiler optimize it.2 Today,
this pattern is eight times slower than a native method. How can we teach the
compiler to optimize this sequence?
The alternatives
Let s explore three ways to implement the map semantics for netip.Addr. My
favorite is to add it to the Go standard library. Go maintainers prefer a small
external helper chaining netip.AddrFrom16() and netip.Addr.As16(), hoping
the compiler eventually optimizes it. The unsafe package
opens a third path, with the same performance as the first solution.
Modifying the Go standard library
Internally, netip.Addr stores any IP address as a
128-bit value with an extra field z to encode the family and the zone:
typeAddrstructaddruint128zunique.Handle[addrDetail]typeaddrDetailstructisV6bool// IPv4 is false, IPv6 is true.zoneV6string// != "" only if IsV6 is true.var(z0unique.Handle[addrDetail]z4=unique.Make(addrDetail )z6noz=unique.Make(addrDetailisV6:true ))
AddrFrom4() encodes an IPv4 address as an
IPv4-mapped IPv6 address and sets z to the unique value z4:
// AddrFrom4 returns the address of the IPv4 address given by the bytes in addr.funcAddrFrom4(addr[4]byte)AddrreturnAddraddr:uint1280,0xffff00000000uint64(addr[0])<<24uint64(addr[1])<<16uint64(addr[2])<<8uint64(addr[3]) ,z:z4,
Unmap() turns an IPv4-mapped IPv6 address into an
IPv4 address by setting the z field to z4:
Implementing the reverse direction inside the Go standard library is trivial: we
set the z field to z6noz if the address is IPv4.
// To6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an// IPv6 address unmodified.func(ipAddr)To6()Addrifip.Is4()ip.z=z6nozreturnip
Update (2026-10)
Instead of patching Go, we can use the -overlay
flag of go build to hijack any package, including the standard
library. In our case, it can add To6() to net/netip. Yet
it is cumbersome:3 the flag expects a JSON file mapping absolute
paths in GOROOT to replacement files, and you need to add it to every go
command.
As a helper
We can t access the z field from outside the net/netip package. Instead, we
build a small helper around the netip.AddrFrom16(ip.As16()) pattern:
// AddrTo6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an// IPv6 address unmodified.funcAddrTo6(ipnetip.Addr)netip.Addrifip.Is4()ip=netip.AddrFrom16(ip.As16())returnip
As an unsafe function
Another solution uses the unsafe package to alter the Addr struct through
a proxy with the same memory layout:4
// addrProxy has the same memory layout as netip.Addr.typeaddrProxystructaddr[2]uint64// netip.uint128zunsafe.Pointer// unique.Handle[netip.addrDetail]var(anyIPv6=netip.IPv6Unspecified()netipZ6noz=(*addrProxy)(unsafe.Pointer(&anyIPv6)).z)// AddrTo6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an// IPv6 address unmodified.funcAddrTo6(ipnetip.Addr)netip.Addrif!ip.Is4()returnip(*addrProxy)(unsafe.Pointer(&ip)).z=netipZ6nozreturnip
Benchmarks
On my computer, with Go 1.27.1, the standard library solution costs 0.88 ns per
operation, while the solution favored by Go maintainers costs 7.14 ns. The
unsafe solution matches the performance of the first one.
Assembly code
Let s check the assembly code the compiler generates for each solution.5
The one built into the standard library looks like this:6
// AX = input.addr.hi, BX = input.addr.lo, CX = input.zCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop hereMOVQnet/netip z6noz(SB),CX; CX = netip.z6nozend:RET// return value = Addr hi: AX, lo: BX, z: CX
Go s assembly language is not a direct representation of the underlying
machine language: it operates on a semi-abstract instruction set derived from
Plan 9 s assembler. It has four pseudo-registers: FP (frame pointer
for function arguments), PC (program counter), SB (static base pointer for
global symbols), and SP (stack pointer). It also has architecture-specific
registers like AX, CX, DX, BX, SI, DI, and R8 to R15.
Instructions storing data use their last argument as the destination.
Instructions can carry an explicit size suffix: MOVB moves a byte, MOVW 16
bits, MOVL 32 bits, and MOVQ 64 bits. In the example above, the first
instruction compares the 64-bit value z4 with the CX register.
The unsafe solution looks almost the same:
// AX = input.addr.hi, BX = input.addr.lo, CX = input.zCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop hereMOVQnetipZ6noz(SB),CX; CX = netip.z6nozend:RET// return value = Addr hi: AX, lo: BX, z: CX
The helper solution has far more instructions. To understand why, let s look at
the code for As16() and
AddrFrom16(). They are short enough for the compiler
to inline them.
We can already guess the pattern to optimize: the code packs the IP address into
an array, copies it, then unpacks it. If we inline the Go code by hand, we get:
Unfortunately, as of Go 1.26.8, the compiler is not smart enough to do the same:
// AX = input.addr.hi, BX = input.addr.lo, CX = input.z; Push the stack (32 bytes):; 0(SP) addr netip.uint128; 16(SP) a16 [16]bytePUSHQBPMOVQSP,BPSUBQ$32,SPCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop here; Pack: byteorder.BEPutUint64(a16[:8], input.addr.hi); byteorder.BEPutUint64(a16[8:], input.addr.lo)MOVBEQAX,net/netip a16+16(SP)MOVBEQBX,net/netip a16+24(SP); addr = a16, 16 bytes at once through the vector register X0MOVUPSnet/netip a16+16(SP),X0MOVUPSX0,net/netip addr(SP); CX = netip.z6nozMOVQnet/netip z6noz(SB),CX; Unpack: output.addr.hi = byteorder.BEUint64(addr[:8]); output.addr.lo = byteorder.BEUint64(addr[8:])MOVBEQnet/netip addr(SP),AXMOVBEQnet/netip addr+8(SP),BXend:ADDQ$32,SPPOPQBPRET// return value = Addr hi: AX, lo: BX, z: CX
The compiler does a decent job on the byte shuffling: the eight byte stores of
BEPutUint64() become a single MOVBEQ, which stores a register byte-swapped.
The eight byte loads of BEUint64() become a single MOVBEQ the other way
round.7 Three groups of instructions remain: a pack, a copy, and an
unpack.
Hacking the Go compiler
The Go compiler has several phases:
The compiler converts the syntax tree and its types into its own intermediate
representation (IR). This process, called noding, goes through
a serialization format named unified IR.
This phase runs two steps: order of evaluation decomposes complex
statements into simpler ones, and desugaring transforms higher-level Go
constructs, like switch or channels, into more primitive instructions or
calls to the runtime.
The compiler rewrites the SSA form into machine-specific variants,
allocates registers, and applies more optimization passes. At the end, the
assembler turns the generated instructions into machine code.
The hammer
My first idea is to replace occurrences of netip.AddrFrom16(ip.As16()) with
netip.Addr addr: ip.addr, z: netip.z6noz as early as possible, during the
noding process. Before that, the type checking phase prevents
us from accessing unexported struct fields.
Go 1.27 introduced a convenient debug option to dump the IR of a function at
interesting points during compilation:
$ GOTOOLCHAIN=go1.27.1GOAMD64=v3gobuild-a-gcflags="-d=astdump=AddrTo6Safe".
Writing text ast output for AddrTo6Safe to AddrTo6Safe.astWriting html ast output for AddrTo6Safe to AddrTo6Safe.htmlWriting html syntax output for AddrTo6Safe to AddrTo6Safe.syntax.html
In the HTML file, the first column shows the IR as it comes out of noding:
In the body of the if statement, we spot the calls to the method
netip.Addr.As16() and to the function netip.AddrFrom16(). Our goal is to
patch them with a struct literal:
In noder s reader.go, the expr() method builds the IR tree for
an expression. At the end of the exprCall case, we add a call to a
rewriteAddrFrom16As16() function. It takes the current node and returns the
struct literal on success, or nil if the rewrite is not possible. First, we
check that we have the expected pattern: a call to the netip.AddrFrom16()
function with a call to the netip.Addr.As16() method as its only argument:
Have a look at the complete patch.8 We can test it
with the following commands:
$ cdsrc
$ ./make.bash
Building Go cmd/dist using /usr/lib/go-1.27. (go1.27.1 linux/amd64)Building Go toolchain1 and bootstrap cmd/go (go_bootstrap) using /usr/lib/go-1.27.Building Go toolchain2 using go_bootstrap and Go toolchain1.Building Go toolchain3 and commands using go_bootstrap and Go toolchain2.Checking command staleness for linux/amd64.---Installed Go for linux/amd64 in /home/bernat/code/free/goInstalled commands in /home/bernat/code/free/go/bin*** You need to add /home/bernat/code/free/go/bin to your PATH.$ exportPATH=$PWD/../bin:$PATH$ goversion
go version go1.28-devel_9834516e20 Sat Sep 12 08:23:11 2026 -0700 linux/amd64$ gotestnet/netip/...
ok net/netip 0.224s$ cd../../go-netip-addrto6
$ gotest.
ok github.com/vincentbernat/go-netip-addrto6 0.062s
The generated code for the helper is now the shortest possible version!
// AX = input.addr.hi, BX = input.addr.lo, CX = input.zCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop hereMOVQnet/netip z6noz(SB),CX; CX = netip.z6nozend:RET// return value = Addr hi: AX, lo: BX, z: CX
Go maintainers are unlikely to accept this change. It relies on the internal
structure of net/netip.Addr. It s an ugly hack in the noder, whose job is to
faithfully translate the type-checked AST into the IR. And it s harder to
maintain than adding a To6() method.
The screwdriver
The right place for such an optimization is the generic SSA phase. One of
the last machine-independent passes is memcombine. With the appropriate debug
flag, the compiler dumps the SSA form after this pass:9
The result of the memcombine pass follows the same structure as the assembly
code for AddrTo6Safe()we looked at earlier: two stores,
one move, and two loads we would like to optimize away.
Each line features a value identifier (v442), an operation with its type
(Bswap64 <uint64>), and its arguments (v490).10 Values are the basic
building blocks of SSA and are defined exactly once. Square brackets enclose
integer parameters ([8]) and curly braces contain auxiliary arguments
( netip.addr ). Operations writing to memory produce a new memory state. Every
memory operation takes the current state as its last argument, which keeps them
in order.
To simplify this code, we could apply three rewriting rules:
The first one adds a shortcut when loading through a move: (Load (OffPtr
[o] p) (Move p src mem)) => (Load (OffPtr [o] src) mem). This matches v174
with its arguments v415 and v286 and creates a new value v600:
The second one simplifies a load following a store: (Load p (Store p x
_)) => x. The load is forwarded: the stored value replaces it and no memory
access remains. It matches v174. It notices that v600 and v173 are the
same address and replaces v174 with a copy of v442:
To optimize it away, we also apply three rewriting rules:
The first one also adds a shortcut when loading through a move, but
without an offset: (Load p (Move p src mem)) => (Load src mem). This rewrites v416
to use arguments from v286:
The second rule forwards a value stored one step earlier, skipping over a
store to another address: (Load p (Store q _ (Store p x _))) => x. This
matches v416: x is v542, p is v22 (&a16), q is v173
(&a16[8]), and p and q do not overlap for uint64.
In practice
Most of these rules already exist in generic.rules. They use
conditions to validate their context: ssa.IsSamePtr() for the same address,
ssa.Disjoint() for addresses that do not overlap. The rule forwarding a stored
value to a load already exists with three variants looking through several other
stores. Here are the two we need:
There is no generic rule to cancel two byte swaps, but the AMD64 lowering
pass includes this rule:
(BSWAP(Q L)(BSWAP(Q L)p))=>p
After switching to Go s development branch and adding the missing
rule, the generated assembly code is worse than with Go 1.26.8,
even though our additional rule slightly improves the situation at the end:
// AX = input.addr.hi, BX = input.addr.lo, CX = input.z; Push the stack (16 bytes):; 0(SP) a16 [16]bytePUSHQBPMOVQSP,BPSUBQ$16,SPCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop here; The four forwarded bytes: the low half of input.addr.lo is taken apart and; put back together in registersMOVQBX,DX; DX = input.addr.loSHRQ$24,BX; BX = input.addr.lo >> 24MOVQDX,SI; SI = input.addr.loSHRQ$16,DX; DX = input.addr.lo >> 16MOVQSI,DI; DI = input.addr.lo, kept for the packSHRQ$8,SI; SI = input.addr.lo >> 8MOVBLZXDIB,R8; R8 = byte(input.addr.lo)MOVBLZXSIB,SI; SI = byte(input.addr.lo >> 8)SHLQ$8,SIORQR8,SI; SI = two low bytes of input.addr.loMOVBLZXDL,DX; DX = byte(input.addr.lo >> 16)SHLQ$16,DXORQSI,DXMOVBLZXBL,BX; BX = byte(input.addr.lo >> 24)SHLQ$24,BXORQDX,BX; BX = input.addr.lo & 0xffffffff; Pack: byteorder.BEPutUint64(a16[:8], input.addr.hi); byteorder.BEPutUint64(a16[8:], input.addr.lo)MOVBEQAX,net/netip a16(SP)MOVBEQDI,net/netip a16+8(SP); The four other bytes of input.addr.lo, read one by one from a16MOVBLZXnet/netip a16+11(SP),DX; a16[11]SHLQ$32,DXORQDX,BXMOVBLZXnet/netip a16+10(SP),DX; a16[10]SHLQ$40,DXORQDX,BXMOVBLZXnet/netip a16+9(SP),DX; a16[9]SHLQ$48,DXORQDX,BXMOVBLZXnet/netip a16+8(SP),DX; a16[8]SHLQ$56,DX; output.z = netip.z6nozMOVQnet/netip z6noz(SB),CX; output.addr.hi = byteorder.BEUint64(a16[:8])MOVBEQnet/netip a16(SP),AX; output.addr.lo assembled from the previous stepsORQDX,BXend:LEAVEQRET// return value = Addr hi: AX, lo: BX, z: CX
The rule loading through a move, added in Go 1.27, introduced this regression.
Out of order
Let s not give up now! In reality, the rewriting rules run before memcombine,
notably in the late opt pass. At this point, the inlined versions of
BEPutUint64() and BEUint64() still expand to sixteen byte stores and sixteen
byte loads, matching their source code:
funcBEUint64(b[]byte)uint64_=b[7]// bounds check hint to compiler; see golang.org/issue/14808returnuint64(b[7])uint64(b[6])<<8uint64(b[5])<<16uint64(b[4])<<24uint64(b[3])<<32uint64(b[2])<<40uint64(b[1])<<48uint64(b[0])<<56
Let s follow two bytes of output.addr.lo: addr[15] and addr[11]. Here is
a simplified SSA form before late opt:
The first rule loads through the move: (Load (OffPtr [o] p) (Move p src mem))
=> (Load (OffPtr [o] src) mem). It matches both loads, which now read a16
with the memory state before the copy:
The second rule shortcuts a load following a store: (Load p (Store p x _)) =>
x. It matches v435, as v282 stores a16[15]. It does not match v481:
v235 stores a16[11] four stores earlier in the chain, while the variants of
this rule look through three stores at most.
The same happens to the other bytes: the rule forwards the four bytes stored
last, a16[12] to a16[15]. The twelve other loads now read a16 instead of
addr.
BEUint64() becomes a chain of Or64, each one adding a byte shifted into
place. memcombine is a pass written in Go, not a set of rewrite
rules. It starts from the last Or64 of the chain and collects up to eight
terms. If each term is a byte load, extended to 64 bits and shifted, and if the
eight loads read consecutive addresses from the same pointer with the same
memory state, it replaces the whole chain with a single 64-bit load and a byte
swap. Otherwise, it tries again with four, then two terms, and from each
intermediate Or64. Here is the loop checking each term in a simplified version
of combineLoads():
From v520, four of the eight terms are forwarded bytes, not loads from memory,
and memcombine can t combine them. It doesn t merge the four remaining loads
either, as they sit on top of the forwarded bytes.
Back in order
In summary, the rewriting rules run too early to be effective. A quick
workaround exists: run an earlier round of memcombine before late opt. After
this change, the generated code for the helper is back to the
shortest possible version:
// AX = input.addr.hi, BX = input.addr.lo, CX = input.zCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop hereMOVQnet/netip z6noz(SB),CX; CX = netip.z6nozend:RET// return value = Addr hi: AX, lo: BX, z: CX
Next steps
I think Go maintainers would reject this change because of the additional
memcombine pass. Instead, I plan to publish this blog post and bring up the
subject again as a follow-up to issue #54365. Either the sheer complexity and
the Go 1.27 regression convince the maintainers that adding a To6() method is
simpler and more efficient, or they advise me on how to move forward. Either
way, digging into this subject taught me a lot about the Go compiler!
Update (2026-10)
I opened issue #81994 to propose Addr.To6(). Give it a if you want it in Go!
IPv4-mapped IPv6 addresses let you handle only IPv6 addresses in your code
and keep conversions at a few well-defined boundaries. I use them in
Akvorado.
This is not strictly equivalent. The zero value becomes :: while it would
make more sense to leave it untouched.
I suppose Go maintainers keep it inconvenient on purpose: they don t want
random packages to patch the standard library or other packages.
To avoid catastrophic bugs when netip.Addr s layout changes, you must add
tests to detect it. This is more dangerous if you put this code in a
package. Either ask users to run the tests themselves, or detect the change
at run time and panic. Hiding this optimization behind a build tag would
make users aware of this potential trap.
I produced the assembly code with GOTOOLCHAIN=go1.26.8 GOAMD64=v3 ./go-asm
'\.asm' from the companion repository, then edited it a bit to keep this
article from turning into an endless rabbit hole. Due to an unfortunate
sequence of events, we switch between versions: Go 1.27 introduced a
regression that blurs the point of this article.
The function is short enough for the compiler to inline it, so the assembly
code may vary depending on the surrounding code.
MOVBEQ requires GOAMD64=v3, matching the x86-64-v3
microarchitecture from 2013. Otherwise, the compiler translates
BEPutUint64() to BSWAPQ+MOVQ and BEUint64() to MOVQ+BSWAPQ.
When the call is used as a statement, a struct literal alone is invalid. The
patch then turns it into _ = netip.Addr .
The compiler can also write an HTML file with the result of each pass:
$ GOSSAFUNC=AddrTo6Safe\> GOTOOLCHAIN=go1.26.8\> GOAMD64=v3gobuild-a.
dumped SSA for AddrTo6Safe,1 to ./ssa.html
Source line numbers in parentheses follow value identifiers, but I removed
them from the examples. The SSA form also features branches, but we don t
need them to understand our optimizations.
netip.Addr features an Unmap()
method returning the unwrapped IPv4 contained in an IPv4-mapped IPv6
address: from ::ffff:203.0.113.10 or ::ffff:cb00:710a, it returns
203.0.113.10.1 There is no Map() or To6() method for the reverse
direction. Such a method is trivial to implement, but Go maintainers have
rejected it on the grounds that users should write
netip.AddrFrom16(ip.As16()) and let the compiler optimize it.2 Today,
this pattern is eight times slower than a native method. How can we teach the
compiler to optimize this sequence?
The alternatives
Let s explore three ways to implement the map semantics for netip.Addr. My
favorite is to add it to the Go standard library. Go maintainers prefer a small
external helper chaining netip.AddrFrom16() and netip.Addr.As16(), hoping
the compiler eventually optimizes it. The unsafe package
opens a third path, with the same performance as the first solution.
Modifying the Go standard library
Internally, netip.Addr stores any IP address as a
128-bit value with an extra field z to encode the family and the zone:
typeAddrstructaddruint128zunique.Handle[addrDetail]typeaddrDetailstructisV6bool// IPv4 is false, IPv6 is true.zoneV6string// != "" only if IsV6 is true.var(z0unique.Handle[addrDetail]z4=unique.Make(addrDetail )z6noz=unique.Make(addrDetailisV6:true ))
AddrFrom4() encodes an IPv4 address as an
IPv4-mapped IPv6 address and sets z to the unique value z4:
// AddrFrom4 returns the address of the IPv4 address given by the bytes in addr.funcAddrFrom4(addr[4]byte)AddrreturnAddraddr:uint1280,0xffff00000000uint64(addr[0])<<24uint64(addr[1])<<16uint64(addr[2])<<8uint64(addr[3]) ,z:z4,
Unmap() turns an IPv4-mapped IPv6 address into an
IPv4 address by setting the z field to z4:
Implementing the reverse direction inside the Go standard library is trivial: we
set the z field to z6noz if the address is IPv4.
// To6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an// IPv6 address unmodified.func(ipAddr)To6()Addrifip.Is4()ip.z=z6nozreturnip
As a helper
We can t access the z field from outside the net/netip package. Instead, we
build a small helper around the netip.AddrFrom16(ip.As16()) pattern:
// AddrTo6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an// IPv6 address unmodified.funcAddrTo6(ipnetip.Addr)netip.Addrifip.Is4()ip=netip.AddrFrom16(ip.As16())returnip
As an unsafe function
Another solution uses the unsafe package to alter the Addr struct through
a proxy with the same memory layout:3
// addrProxy has the same memory layout as netip.Addr.typeaddrProxystructaddr[2]uint64// netip.uint128zunsafe.Pointer// unique.Handle[netip.addrDetail]var(anyIPv6=netip.IPv6Unspecified()netipZ6noz=(*addrProxy)(unsafe.Pointer(&anyIPv6)).z)// AddrTo6 maps an IPv4 address to an IPv4-mapped IPv6 address. It returns an// IPv6 address unmodified.funcAddrTo6(ipnetip.Addr)netip.Addrif!ip.Is4()returnip(*addrProxy)(unsafe.Pointer(&ip)).z=netipZ6nozreturnip
Benchmarks
On my computer, with Go 1.27.1, the standard library solution costs 0.88 ns per
operation, while the solution favored by Go maintainers costs 7.14 ns. The
unsafe solution matches the performance of the first one.
Assembly code
Let s check the assembly code the compiler generates for each solution.4
The one built into the standard library looks like this:5
// AX = input.addr.hi, BX = input.addr.lo, CX = input.zCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop hereMOVQnet/netip z6noz(SB),CX; CX = netip.z6nozend:RET// return value = Addr hi: AX, lo: BX, z: CX
Go s assembly language is not a direct representation of the underlying
machine language: it operates on a semi-abstract instruction set derived from
Plan 9 s assembler. It has four pseudo-registers: FP (frame pointer
for function arguments), PC (program counter), SB (static base pointer for
global symbols), and SP (stack pointer). It also has architecture-specific
registers like AX, CX, DX, BX, SI, DI, and R8 to R15.
Instructions storing data use their last argument as the destination.
Instructions can carry an explicit size suffix: MOVB moves a byte, MOVW 16
bits, MOVL 32 bits, and MOVQ 64 bits. In the example above, the first
instruction compares the 64-bit value z4 with the CX register.
The unsafe solution looks almost the same:
// AX = input.addr.hi, BX = input.addr.lo, CX = input.zCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop hereMOVQnetipZ6noz(SB),CX; CX = netip.z6nozend:RET// return value = Addr hi: AX, lo: BX, z: CX
The helper solution has far more instructions. To understand why, let s look at
the code for As16() and
AddrFrom16(). They are short enough for the compiler
to inline them.
We can already guess the pattern to optimize: the code packs the IP address into
an array, copies it, then unpacks it. If we inline the Go code by hand, we get:
Unfortunately, as of Go 1.26.8, the compiler is not smart enough to do the same:
// AX = input.addr.hi, BX = input.addr.lo, CX = input.z; Push the stack (32 bytes):; 0(SP) addr netip.uint128; 16(SP) a16 [16]bytePUSHQBPMOVQSP,BPSUBQ$32,SPCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop here; Pack: byteorder.BEPutUint64(a16[:8], input.addr.hi); byteorder.BEPutUint64(a16[8:], input.addr.lo)MOVBEQAX,net/netip a16+16(SP)MOVBEQBX,net/netip a16+24(SP); addr = a16, 16 bytes at once through the vector register X0MOVUPSnet/netip a16+16(SP),X0MOVUPSX0,net/netip addr(SP); CX = netip.z6nozMOVQnet/netip z6noz(SB),CX; Unpack: output.addr.hi = byteorder.BEUint64(addr[:8]); output.addr.lo = byteorder.BEUint64(addr[8:])MOVBEQnet/netip addr(SP),AXMOVBEQnet/netip addr+8(SP),BXend:ADDQ$32,SPPOPQBPRET// return value = Addr hi: AX, lo: BX, z: CX
The compiler does a decent job on the byte shuffling: the eight byte stores of
BEPutUint64() become a single MOVBEQ, which stores a register byte-swapped.
The eight byte loads of BEUint64() become a single MOVBEQ the other way
round.6 Three groups of instructions remain: a pack, a copy, and an
unpack.
Hacking the Go compiler
The Go compiler has several phases:
The compiler converts the syntax tree and its types into its own intermediate
representation (IR). This process, called noding, goes through
a serialization format named unified IR.
This phase runs two steps: order of evaluation decomposes complex
statements into simpler ones, and desugaring transforms higher-level Go
constructs, like switch or channels, into more primitive instructions or
calls to the runtime.
The compiler rewrites the SSA form into machine-specific variants,
allocates registers, and applies more optimization passes. At the end, the
assembler turns the generated instructions into machine code.
The hammer
My first idea is to replace occurrences of netip.AddrFrom16(ip.As16()) with
netip.Addr addr: ip.addr, z: netip.z6noz as early as possible, during the
noding process. Before that, the type checking phase prevents
us from accessing unexported struct fields.
Go 1.27 introduced a convenient debug option to dump the IR of a function at
interesting points during compilation:
$ GOTOOLCHAIN=go1.27.1GOAMD64=v3gobuild-a-gcflags="-d=astdump=AddrTo6Safe".
Writing text ast output for AddrTo6Safe to AddrTo6Safe.astWriting html ast output for AddrTo6Safe to AddrTo6Safe.htmlWriting html syntax output for AddrTo6Safe to AddrTo6Safe.syntax.html
In the HTML file, the first column shows the IR as it comes out of noding:
In the body of the if statement, we spot the calls to the method
netip.Addr.As16() and to the function netip.AddrFrom16(). Our goal is to
patch them with a struct literal:
In noder s reader.go, the expr() method builds the IR tree for
an expression. At the end of the exprCall case, we add a call to a
rewriteAddrFrom16As16() function. It takes the current node and returns the
struct literal on success, or nil if the rewrite is not possible. First, we
check that we have the expected pattern: a call to the netip.AddrFrom16()
function with a call to the netip.Addr.As16() method as its only argument:
Have a look at the complete patch.7 We can test it
with the following commands:
$ cdsrc
$ ./make.bash
Building Go cmd/dist using /usr/lib/go-1.27. (go1.27.1 linux/amd64)Building Go toolchain1 and bootstrap cmd/go (go_bootstrap) using /usr/lib/go-1.27.Building Go toolchain2 using go_bootstrap and Go toolchain1.Building Go toolchain3 and commands using go_bootstrap and Go toolchain2.Checking command staleness for linux/amd64.---Installed Go for linux/amd64 in /home/bernat/code/free/goInstalled commands in /home/bernat/code/free/go/bin*** You need to add /home/bernat/code/free/go/bin to your PATH.$ exportPATH=$PWD/../bin:$PATH$ goversion
go version go1.28-devel_9834516e20 Sat Sep 12 08:23:11 2026 -0700 linux/amd64$ gotestnet/netip/...
ok net/netip 0.224s$ cd../../go-netip-addrto6
$ gotest.
ok github.com/vincentbernat/go-netip-addrto6 0.062s
The generated code for the helper is now the shortest possible version!
// AX = input.addr.hi, BX = input.addr.lo, CX = input.zCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop hereMOVQnet/netip z6noz(SB),CX; CX = netip.z6nozend:RET// return value = Addr hi: AX, lo: BX, z: CX
Go maintainers are unlikely to accept this change. It relies on the internal
structure of net/netip.Addr. It s an ugly hack in the noder, whose job is to
faithfully translate the type-checked AST into the IR. And it s harder to
maintain than adding a To6() method.
The screwdriver
The right place for such an optimization is the generic SSA phase. One of
the last machine-independent passes is memcombine. With the appropriate debug
flag, the compiler dumps the SSA form after this pass:8
The result of the memcombine pass follows the same structure as the assembly
code for AddrTo6Safe()we looked at earlier: two stores,
one move, and two loads we would like to optimize away.
Each line features a value identifier (v442), an operation with its type
(Bswap64 <uint64>), and its arguments (v490).9 Values are the basic
building blocks of SSA and are defined exactly once. Square brackets enclose
integer parameters ([8]) and curly braces contain auxiliary arguments
( netip.addr ). Operations writing to memory produce a new memory state. Every
memory operation takes the current state as its last argument, which keeps them
in order.
To simplify this code, we could apply three rewriting rules:
The first one adds a shortcut when loading through a move: (Load (OffPtr
[o] p) (Move p src mem)) => (Load (OffPtr [o] src) mem). This matches v174
with its arguments v415 and v286 and creates a new value v600:
The second one simplifies a load following a store: (Load p (Store p x
_)) => x. The load is forwarded: the stored value replaces it and no memory
access remains. It matches v174. It notices that v600 and v173 are the
same address and replaces v174 with a copy of v442:
To optimize it away, we also apply three rewriting rules:
The first one also adds a shortcut when loading through a move, but
without an offset: (Load p (Move p src mem)) => (Load src mem). This rewrites v416
to use arguments from v286:
The second rule forwards a value stored one step earlier, skipping over a
store to another address: (Load p (Store q _ (Store p x _))) => x. This
matches v416: x is v542, p is v22 (&a16), q is v173
(&a16[8]), and p and q do not overlap for uint64.
In practice
Most of these rules already exist in generic.rules. They use
conditions to validate their context: ssa.IsSamePtr() for the same address,
ssa.Disjoint() for addresses that do not overlap. The rule forwarding a stored
value to a load already exists with three variants looking through several other
stores. Here are the two we need:
There is no generic rule to cancel two byte swaps, but the AMD64 lowering
pass includes this rule:
(BSWAP(Q L)(BSWAP(Q L)p))=>p
After switching to Go s development branch and adding the missing
rule, the generated assembly code is worse than with Go 1.26.8,
even though our additional rule slightly improves the situation at the end:
// AX = input.addr.hi, BX = input.addr.lo, CX = input.z; Push the stack (16 bytes):; 0(SP) a16 [16]bytePUSHQBPMOVQSP,BPSUBQ$16,SPCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop here; The four forwarded bytes: the low half of input.addr.lo is taken apart and; put back together in registersMOVQBX,DX; DX = input.addr.loSHRQ$24,BX; BX = input.addr.lo >> 24MOVQDX,SI; SI = input.addr.loSHRQ$16,DX; DX = input.addr.lo >> 16MOVQSI,DI; DI = input.addr.lo, kept for the packSHRQ$8,SI; SI = input.addr.lo >> 8MOVBLZXDIB,R8; R8 = byte(input.addr.lo)MOVBLZXSIB,SI; SI = byte(input.addr.lo >> 8)SHLQ$8,SIORQR8,SI; SI = two low bytes of input.addr.loMOVBLZXDL,DX; DX = byte(input.addr.lo >> 16)SHLQ$16,DXORQSI,DXMOVBLZXBL,BX; BX = byte(input.addr.lo >> 24)SHLQ$24,BXORQDX,BX; BX = input.addr.lo & 0xffffffff; Pack: byteorder.BEPutUint64(a16[:8], input.addr.hi); byteorder.BEPutUint64(a16[8:], input.addr.lo)MOVBEQAX,net/netip a16(SP)MOVBEQDI,net/netip a16+8(SP); The four other bytes of input.addr.lo, read one by one from a16MOVBLZXnet/netip a16+11(SP),DX; a16[11]SHLQ$32,DXORQDX,BXMOVBLZXnet/netip a16+10(SP),DX; a16[10]SHLQ$40,DXORQDX,BXMOVBLZXnet/netip a16+9(SP),DX; a16[9]SHLQ$48,DXORQDX,BXMOVBLZXnet/netip a16+8(SP),DX; a16[8]SHLQ$56,DX; output.z = netip.z6nozMOVQnet/netip z6noz(SB),CX; output.addr.hi = byteorder.BEUint64(a16[:8])MOVBEQnet/netip a16(SP),AX; output.addr.lo assembled from the previous stepsORQDX,BXend:LEAVEQRET// return value = Addr hi: AX, lo: BX, z: CX
The rule loading through a move, added in Go 1.27, introduced this regression.
Out of order
Let s not give up now! In reality, the rewriting rules run before memcombine,
notably in the late opt pass. At this point, the inlined versions of
BEPutUint64() and BEUint64() still expand to sixteen byte stores and sixteen
byte loads, matching their source code:
funcBEUint64(b[]byte)uint64_=b[7]// bounds check hint to compiler; see golang.org/issue/14808returnuint64(b[7])uint64(b[6])<<8uint64(b[5])<<16uint64(b[4])<<24uint64(b[3])<<32uint64(b[2])<<40uint64(b[1])<<48uint64(b[0])<<56
Let s follow two bytes of output.addr.lo: addr[15] and addr[11]. Here is
a simplified SSA form before late opt:
The first rule loads through the move: (Load (OffPtr [o] p) (Move p src mem))
=> (Load (OffPtr [o] src) mem). It matches both loads, which now read a16
with the memory state before the copy:
The second rule shortcuts a load following a store: (Load p (Store p x _)) =>
x. It matches v435, as v282 stores a16[15]. It does not match v481:
v235 stores a16[11] four stores earlier in the chain, while the variants of
this rule look through three stores at most.
The same happens to the other bytes: the rule forwards the four bytes stored
last, a16[12] to a16[15]. The twelve other loads now read a16 instead of
addr.
BEUint64() becomes a chain of Or64, each one adding a byte shifted into
place. memcombine is a pass written in Go, not a set of rewrite
rules. It starts from the last Or64 of the chain and collects up to eight
terms. If each term is a byte load, extended to 64 bits and shifted, and if the
eight loads read consecutive addresses from the same pointer with the same
memory state, it replaces the whole chain with a single 64-bit load and a byte
swap. Otherwise, it tries again with four, then two terms, and from each
intermediate Or64. Here is the loop checking each term in a simplified version
of combineLoads():
From v520, four of the eight terms are forwarded bytes, not loads from memory,
and memcombine can t combine them. It doesn t merge the four remaining loads
either, as they sit on top of the forwarded bytes.
Back in order
In summary, the rewriting rules run too early to be effective. A quick
workaround exists: run an earlier round of memcombine before late opt. After
this change, the generated code for the helper is back to the
shortest possible version:
// AX = input.addr.hi, BX = input.addr.lo, CX = input.zCMPQnet/netip z4(SB),CX; check "z" if this is an IPv4 addressJNEend; if not, stop hereMOVQnet/netip z6noz(SB),CX; CX = netip.z6nozend:RET// return value = Addr hi: AX, lo: BX, z: CX
Next steps
I think Go maintainers would reject this change because of the additional
memcombine pass. Instead, I plan to publish this blog post and bring up the
subject again as a follow-up to issue #54365. Either the sheer complexity and
the Go 1.27 regression convince the maintainers that adding a To6() method is
simpler and more efficient, or they advise me on how to move forward. Either
way, digging into this subject taught me a lot about the Go compiler!
IPv4-mapped IPv6 addresses let you handle only IPv6 addresses in your code
and keep conversions at a few well-defined boundaries. I use them in
Akvorado.
This is not strictly equivalent. The zero value becomes :: while it would
make more sense to leave it untouched.
To avoid catastrophic bugs when netip.Addr s layout changes, you must add
tests to detect it. This is more dangerous if you put this code in a
package. Either ask users to run the tests themselves, or detect the change
at run time and panic. Hiding this optimization behind a build tag would
make users aware of this potential trap.
I produced the assembly code with GOTOOLCHAIN=go1.26.8 GOAMD64=v3 ./go-asm
'\.asm' from the companion repository, then edited it a bit to keep this
article from turning into an endless rabbit hole. Due to an unfortunate
sequence of events, we switch between versions: Go 1.27 introduced a
regression that blurs the point of this article.
The function is short enough for the compiler to inline it, so the assembly
code may vary depending on the surrounding code.
MOVBEQ requires GOAMD64=v3, matching the x86-64-v3
microarchitecture from 2013. Otherwise, the compiler translates
BEPutUint64() to BSWAPQ+MOVQ and BEUint64() to MOVQ+BSWAPQ.
When the call is used as a statement, a struct literal alone is invalid. The
patch then turns it into _ = netip.Addr .
The compiler can also write an HTML file with the result of each pass:
$ GOSSAFUNC=AddrTo6Safe\> GOTOOLCHAIN=go1.26.8\> GOAMD64=v3gobuild-a.
dumped SSA for AddrTo6Safe,1 to ./ssa.html
Source line numbers in parentheses follow value identifiers, but I removed
them from the examples. The SSA form also features branches, but we don t
need them to understand our optimizations.
A week ago, release
0.0.5 of the RcppFastAD package
went to CRAN, and we
noted that it provided a stop-gap measure against a gnarly segfault
with g++-16 under heavier optimization / inlining. We are
happy to now announce that a new and fixed version 0.0.6 of RcppFastAD is now
on CRAN, and has been built for
r2u as well as at r-universe.
RcppFastAD
wraps the FastAD
header-only C++ library by James which provides a C++
implementation of both forward and reverse mode of automatic
differentiation. It offers an easy-to-use header library that is both
lightweight and performant. With a little of bit of Rcpp glue, it is also easy to use from R
in simple C++ applications.
This release takes advantage of some large-language model inference
access I now get via Debian. The Deepseek v4 flash-0731 model, driven in
a container via the pi.dev harness, took just about no time finding a
small workaround for the issue described
last week where we could see segfaults under heavier optimization
and inlining. The fix was to prohibit inlining the local
bind() reimplementation. We have also sent this upstream as
a patch. It is nice to have an option of querying something a little
more powerful than the local models I experimented with before. We
suppress more compiler warnings in local builds via another option, and
did some standard maintenance.
The NEWS file for this release follows.
Changes in version 0.0.6
(2026-10-01)
The expression-template view rebinding is protected against
g++ -O3 (which led to wrong Eigen::Map
dimensions, an Eigen::Map resize assertion, and a
segfault.
The bind() helpers are marked non-inline and are
also protected from operating on a nullptr.
Both items co-written with DeepSeek V4 Flash, and have been sent
upstream in a PR.
Suppress another compiler warning when local build
detected.
A new release of RcppFastFloat
just arrived on CRAN, and has
been built for r2u.
The package wraps fast_float, another
nice library by Daniel Lemire. For
details, see the arXiv
preprint or published
paper showing that one can convert character representations of
numbers into floating point at rates at or exceeding one gigabyte per
second.
This release updates the underlying fast_float library
version to the current version 8.3.0, and also updates a number of the
standard packaging details.
Changes in version 0.0.6
(2026-09-29)
Update to upstream release 8.3.0 (Daniel Lemire in #6)
Standard package maintenace adding README badge, updating URL,
updating continuous integration, and adding an aspell
dictionary
Completed help page for as_double example with
@return tag
Thrilled to share that our new package myman reached CRAN a few days ago. The package
offers sixteen hundred ninety four My man posts by Kevin Kruse
made on bsky during the summer of 2026.
Each wave picked at one particular public persona. This package wrapse
these up in the style of packages like fortunes or gaussfacts.
A sample usage illustration shows how to extract by pattern:
>library(myman)>myman("maitre")My man looks like he's inquiring with the maitre'd about the house curly fries.-- about Howard Lutnick on 2026-08-28My man looks like a maitre'd who deeply doubts you have a reservation. -- about Scott Bessent on 2026-08-31My man looks like he's asked the maitre'd to remove a party of four he finds visuallyunpleasant. -- about Scott Bessent on 2026-08-31My man looks like the maitre'd at a very exclusive restaurant called The Berghof.-- about JD Vance on 2026-09-13>
One can also subset by target , or simple sample randomly (which is
the default).
As CRAN review for new
package takes a little longer these days, the GitHub repo was updated
once more with one (eighth !!) wave of posts since the submission was
made. CRAN version 0.1.0
corresponds to the stock of messages from waves one to seven. We will
update the CRAN version shortly
but if you would like more posts right now consider installing
from the repo.
We include the full NEWS file since the package was started, this
also includes an entry for the eights wave we will bring to CRAN
soon.
Changes in version 0.1.0
(2026-09-27)
Initial CRAN version (minus the 0.0.8 changes which were made
while CRAN reviewed the package)
Add an aspell dictionary to aid DESCRIPTION text
Changes in version 0.0.8
(2026-09-18)
Data set now comprises 1694 posts aiming at eight different
'men'
The number of 'sample()' object returned can not be set with
'posts'
Changes in version 0.0.7
(2026-09-14)
Data set now comprises 1494 posts aiming at seven different
'men'
Changes in version 0.0.6
(2026-09-07)
Data set now comprises 1394 posts aiming at six different 'men'
Changes in version 0.0.5
(2026-09-01)
Data set now comprises 1194 posts aiming at five different
'men'
The myman() function can now now subset by target
permitting more focussed randlom sampling
Changes in version 0.0.4
(2026-08-28)
Data set now comprises 1040 posts aiming at four different
'men'
Changes in version 0.0.3
(2026-08-19)
Data set now comprises 916 posts aiming at three different
'men'
Csv file, processing and display extended to three columns also
showing post creating date (in UTC) and 'man'
Changes in version 0.0.2
(2026-07-30)
Updated with full week of posts leading to a corpus of 698
posts
Added support for simple regular-expression lookup
ability
There's an optical illusion known by names like the "lunar terminator paradox".
This has much discussion and descriptions online, but none made sense in my
head, so I wrote this program to figure out what was going on. The paradox:
The moon is illuminated by the sun
After sunset, the sun is below, so we would expect the illuminated portion of
the moon to point down, towards where the sun is
This isn't what always happens!
I was camping in the mountains with a friend recently. We were looking up at the
sky, just after the sun has set. The moon was clearly pointing up.
After playing with this program, I have clarity: we're looking up at the moon
from below. The most pathological case is a full moon. Let's say the moon is on
the horizon: elevation=0. If we're looking at this low moon, and the moon is
full, the sun is directly behind us. The moon is fully illuminated, and we see
100% of the moon disc lit up.
Now, what happens if the moon is not at the horizon, but higher up at the sky?
Effectively, the sun is infinitely far away, so the same part of the moon is lit
up. But we're now looking at the moon from below, and we would see some of the
dark side on the bottom edge. I.e. the moon appears bright at the top, but has a
dark slice missing at the bottom: it points up! Even at sunset.
This is the most clearly weird case. A half-moon has no paradoxical behavior,
and the more full the moon, the more clearly we can observe the bright moon
pointing up despite the sun being down.
I produce two plots. At 3/4-full moon at sunset we look at the moon as it rises
from the horizon all the way up to the zenith:
If the moon is at the horizon (elevation = 0), we see the expected 3/4-lit disc.
As the moon rises, we see more and more of its bottom, and thus more and more
dark areas show up: It clearly points up at sunset!
We can also move the sun around. Let's say the moon is 60deg up; let's move the
sun from new-moon to a full-moon:
In the darkest configuration, we can still see a lit crescent on the bottom: the
illumination is all on the opposite side, but we can see a slice of the
illuminated back side. As the sun swings around, we see more and more of the
moon. At azimuth = 90deg, we have a half-lit moon. Past that, more and more of
the moon is visible, always pointing up.
This was also a super interesting study of the current state of AI tools. As I
was writing this and debugging, I would talk to Claude and Gemini about how this
worked and what I should be seeing. They were both completely unable to
comprehend this, and would make infinite circular arguments. Clearly they read
some incomplete explanations on the internet (as I have), and lack the reasoning
capability necessary to distill it into something resembling "understanding".
AGI is not here yet.
Welcome to post 60 in the R4 series.
bubblewrap is
a great tool and very suitable for using with an agent harness. It is a
very compelling and lightweight alternative to using a full-blown docker container as it offers
low-level unprivileged sandboxing on Linux hosts.
In a nutshell, bubblewrap can turn
everything off (see unshare-all below) and allow access
only to selected services and directories (as shown below). This makes
it a very useful tool be used on a main workstation as it can provide a
lower-risk deployment quite easily while taking advantage of the already
installed software stack. I continue to get a lot of value out of docker, especially as r2u makes installing R
package dependencies so trivial. Its slogan easy, fast, reliable: pick
all three clearly holds for r2u. But sometimes bubblewrap is
compelling, for example to launch opencode (documented for example at the
debian-inference
site).
When using R we need add more
directories to the example to provide /etc/alternatives
which is governing inter alia the LAPACK / BLAS resolution on
Debian / Ubuntu systems; this is now on debian-inference
site but wasn t when I first tried it a few days ago ;-) as well as
/etc/R for config files and possibly
~/.R/Makevars for compiler settings. With that my current
wrapper is
Launched that way we can have agents use R to check packages sources,
builds, and triage for bugs. this works equally well with local models
served via ollama (and with that a
quick shoutout to packages.lingfish.net for
providing an apt service for ollama) as it does with the various
cloud-based offerings. So harness away with R!
A new release 0.0.5 of the RcppFastAD package
is now on CRAN, has been built
for r2u, and updated
at r-universe.
This comes (to the day) two years after the preceding 0.0.4 release.
RcppFastAD
wraps the FastAD
header-only C++ library by James which provides a C++
implementation of both forward and reverse mode of automatic
differentiation. It offers an easy-to-use header library that is both
lightweight and performant. With a little of bit of Rcpp glue, it is also easy to use from R
in simple C++ applications. This release updates the continuous
integration setup as one does, adds a local configuration helper to
quieten compilation (described also in this
blog post). It also adds a defensive setting for g++:
Under recent g++ versions and optimisation at least the -O3
level, segfaults are seen as something is not quite right with
(temporary) views of Eigen objects in code generated by this versions.
Others are fine, as is clang++. We have not gotten to the
bottom of it, but setting -g0 seems to ensure that builds
generally work.
The NEWS file for this release follows.
Changes in version 0.0.5
(2026-09-24)
Several routine updates to continuous integration have been
made
Local builds (where a .git/ directory is seen) now
append silencing compiler option that CRAN would object to
Given recent issues with g++ under optimization,
debugging is turned off by default.
Activity summary
During the month of August, 19 contributors have been
paid to work on Debian LTS (links to individual
contributor reports are located below).
The team released 58 DLAs fixing 1885 CVEs.
Debian 11 ( bullseye ), which
has reached the end of its Long Term Support on 31 August 2026, will
now get security support from Freexian under the
Extended LTS offer.
The team published several notable updates:
chromium
(DLA 4710-1),
uploaded by Emilio, to fix 375 security flaws in bookworm, including
issues that could lead to arbitrary code execution.
Further DLA 4728-1,
uploaded by Emilio as well, fixed additional 41 security issues in bookworm and
DLA 4739-1
fixed 5 issues.
Last but not least,
DLA 4749-1
fixed another 15 issues and
DLA 4758-1
fixed another 7 issues.
ruby2.7
(DLA 4716-1),
uploaded by Abhijith, to fix four security flaws in bookworm, including
issues that could lead to arbitrary code execution.
several DLAs
(DLA 4717-1,
DLA 4720-1,
DLA 4723-1,
DLA 4724-1,
DLA 4745-1)
have been uploaded by Ben and Emilio to fix myriad security flaws in
different versions of the linux kernel
(5.10.262-1, 6.1.180-1, 6.12.100-1~deb12u1, 6.12.101-1~deb12u1)
in bullseye and bookworm.
7zip
(DLA 4718-1)
and p7zip
(DLA 4719-1),
uploaded by Sylvain, to fix multiple security flaws in bookworm and
bullseye, including issues that could yield to remote code execution.
ca-certificates
(DLA 4726-1),
uploaded by Bastien to add some new trusted certificates to bookworm and bullseye,
but also remove some untrusted ones.
thunderbird
(DLA 4727-1
and DLA 4754-1),
uploaded by Emilio to fix 35 and 31 issues in bookworm and bullseye.
nss
(DLA 4729-1),
uploaded by Jochen to fix a vulnerability in bookworm and bullseye that could
yield to arbitrary code execution.
libgd2
(DLA 4731-1),
uploaded by Guilhem to fix a vulnerability in bookworm and bullseye that could
yield to arbitrary code execution if a malformed GIF file is processed.
xorg-server
(DLA 4737-1
and DLA 4738-1),
uploaded by Arnaud to fix several vulnerabilities in bookworm and bullseye.
postgresql-15
(DLA 4740-1),
uploaded by Carlos Henrique Lima Melara to fix 26 vulnerabilities
in bookworm.
unzip
(DLA 4741-1),
uploaded by Andrej to fix a vulnerability in bookworm and bullseye
that could yield to arbitrary code execution.
firefox-esr
(DLA 4750-1),
uploaded by Emilio to fix 31 vulnerabilities in bookworm and bullseye
that could potentially result in arbitrary code execution, privilege escalation
or information disclosure.
nvidia-graphics-drivers
(DLA 4752-1 and
DLA 4753-1),
uploaded by Tobias to fix lots of vulnerabilities in bookworm and bullseye.
roundcube
(DLA 4760-1),
uploaded by Guilhem to fix ten vulnerabilities in bookworm and bullseye
that could potentially result in remote code execution or information disclosure.
Contributions from outside the LTS Team:
We are greatly thankful for the contributions from people outside the LTS Team:
Salvatore Bonaccorso released a libyaml-syck-perl security update
(DLA 4730-1),
to fix a denial of service and potentially arbitrary code execution.
Thomas Goirand released a neutron security update
(DLA 4735-1).
Thomas Goirand released an ironic security update
(DLA 4743-1).
Thomas Goirand released a designate security update
(DLA 4751-1),
to fix a denial of service or DNS hijacking.
The LTS Team has also contributed with updates to the latest Debian releases:
xrdp
(DSA 6469-1),
prepared by Abhijith to address privilege escalation and arbitrary
code execution related flaws.
Other contributions:
Besides the work on security updates, different documentation and tooling
changes were needed. This work was mainly done by Sylvain.
This is a new series1 I'm starting on the tools I have been crafting and
buying to play pedagogical games in my Economics classes.
These games, aside from being just plain fun for the students, often aim to
teach them a lesson via interactive play. Unsurprisingly, it is much easier for
them to grok a particular concept this way. Moreover, many students told me
these activities were the best part of the classes I teach.
The World Trade Game
I found out about The World Trade Game in The Handbook of Pluralist Economics
Education2, which also contains many other fun pedagogical games. It
was first developed by Action Aid, but has since been refined by John
Sloman and then by Irene van Staveren, who wrote about her use of the game in
the aforementioned book.
The goal of this blog entry is not to describe the game and teach you how to
play it. If that interests you, either buy the book I mentioned or read about
Sloman's version here.
No, our focus point here is the tools to play this game: fake cash! Fake cash
can be used in many economics pedagogical games and in my opinion, is a
worthwhile "investment" of your time and energy. You could indeed play with
regular printed sheets of paper listing a value (I sure did!), but using Play
Money (a more appropriate term, since we're not trying to defraud anyone) will
up your game and create a more immersive atmosphere in your classes.
At first, I tried to buy play money. Surely, some commercial offering aiming to
create realistic props for movies and TV productions existed? At least in
Canada, it turns out the answer was no. The only thing I could find were shady
websites and weird expensive listings on AliExpress. Not being tenured yet, I
did not try to convince my employer to take a 50-50 chance on supporting
criminal endeavours.
As such, the next best thing I found was printable play money designed by the
Bank of Canada Museum. Their designs were quite nice and met my needs,
but their PDF layout made it overly complicated to print large quantities of
bills. As such, I played around with GIMP a little and came out with these
files. A commercial printer shouldn't have issues printing them on glossy
cardstock paper and cutting them up for you:
Once printed and cut, you'll unfortunately have to fold each bill by hand. To
secure both halves, I ended up using a strip of transparent tape on each long
edge.
Here is the result, with a real Canadian 20$ bill as a comparison:
Don't hold your breath though, as I do not currently have a release
schedule.
Reardon, J. (2009). The Handbook of Pluralist Economic Education. New York : Routledge.
Replacing lxc with Incus for Debian CI, by Antonio Terceiro
At the beginning of this month,
the default backend for tests was changed to incus-lxc,
leaving only a few dozen packages explicitly configured to run as lxc. Those
packages got bug reports filed
and once those bugs are fixed they will be migrated over to incus-lxc. This
marks an important milestone for Debian CI, as the low level of isolation
between the worker host OS and the OS under test when using lxc was a
significant source of instability in our platform. Common causes for test
failures when comparing runs under plain lxc with runs with incus-lxc are
documented in the wiki page
linked from the bug reports.
security-tracker git performance, by Helmut Grohne
Developers working with Debian s security-tracker
have reporteddegrading
performance for a while. The options for solving this are few with repository
sizes now reaching 30GB for a repository whose working tree is a mere 60MB.
While there have been a number of proposals for changing the storage layout,
Helmut evaluated options not requiring such changes. Much of the problem hinges
on the 50MB data/CVE/list file that is updated in most commits. Efficiency
hinges on representing its content in git packs.
Practically, git frequently fails to represent its content as a delta and stores
a full copy that is typically compressed to 12MB. Add a few thousand 12MB full
blobs and you quickly have a large repository. In particular, the copy at
salsa.debian.org stores many such full blobs, so pulling from it consumes
significant bandwidth.
One might think that running git gc helps, but its utility is limited here.
When git stores objects in packs, it stores the
history in reverse.
It starts with the current version and represents older objects as differences
(deltas) to more recent objects. A delta effectively is a sequence of copying
ranges from a base object and insertion of new content. Given that humans tend
to add content over time, those additions are often represented as deletions.
The crux here is finding that base object. Given that the data/CVE/list file
is large, finding it involves comparing quite a few versions of it with each
other. This requires both a significant amount of RAM and CPU time. How many
objects git considers for this comparison is controlled with the --window
option. It is beneficial if the base object does not come from a direct child
commit but skips over commits. In doing so, long chains of deltas are avoided.
The --depth option controls the maximum chain length.
In this search, git combines all objects into a single window to facilitate file
renames. As such, it tends to compare data/CVE/list with data/DSA/list,
data/DLA/list and others. This is less than helpful and tends to evict all
data/CVE/list versions from the comparison window such that a new full blob of
it becomes required. Since the security-tracker repository rarely renames files,
we can ask git to instead consider one window per filename
via --path-walk. Once doing so, it becomes quite a bit better at finding
suitable deltas. The technique is less applicable to older history (2025 and
earlier), but given a lot of RAM and a bit of partitioning, git gc can shrink
that as well. Combining these techniques, we can shrink the repository into
700MB and keep new growth somewhat under control.
Temporarily, Helmut is providing a proof-of-concept mirror at
git://git.subdivi.de/~helmut/security-tracker.git using these techniques.
Avoid pulling from it directly as it does not provide a secure transport. While
it does reduce the amount of data being downloaded, it does not resolve a number
of related problems. After downloading, a git client will still expend
considerable amounts of CPU on verifying received deltas and attempting to
git blame data/CVE/list is not improved in any way. Still, there is work on
integrating some of the improvements back into salsa.
OpenSSH GSS-API split, by Colin Watson
In an option review
Colin did in 2024, shortly after the xz-utils backdoor,
he explained that having GSS-API authentication and key exchange support in the
main OpenSSH packages is problematic. The key exchange patch is large and
intrusive. Even linking to the necessary libraries isn t without risk: as the
Ebury malware attack demonstrated
way back in 2009, each extra library linked into security-critical daemons such
as sshd (or nowadays into its privilege-separated helper programs) can modify
the behaviour of the daemon even if you aren t doing anything that would involve
calling into that library. Of course some of that risk remains, but as
Damien Miller wrote,
minimizing the number of libraries that end up in the address space of sshd
and friends is still valuable.
This split is now complete in testing. As of openssh 1:10.4p1-5, the OpenSSH
client and server are built without GSS-API authentication and key exchange
support. If you need those features, install openssh-client-gssapi or
openssh-server-gssapi instead, as appropriate. Debian 13 (trixie) already has
packages with those names that just depend on the regular openssh-client and
openssh-server so that you can pre-emptively install them, as
documented in the release notes.
The new openssh-*-gssapi packages have relatively tight dependencies on
openssh-common, in order for the testing migration system to ensure that we
can t forget to keep them up to date. This will mean a bit more ongoing work on
each new upstream version, but it should be manageable.
Catching up on DebConf video publication, by Stefano Rivera
DebConf 25 videos suffered terribly
from a bug in voctomix that
stopped us from being able to publish the videos immediately after the
conference. Ivo De Decker did some work on the videos earlier this year, fixing
enough of the audio problems to make the videos at least intelligible.
While at MiniDebConf Winterthur Stefano
published all the videos from 2025 and 2026 to PeerTube
and YouTube.
DebConf 25 and DebConf 26 videos as well as all the videos from miniconfs that
had got caught up in the backlog were published.
Miscellaneous contributions
Carles used po-debconf-manager
to review and submit Catalan translations for six packages.
Carles prepared a prototype
of building webwml and running tidy (linter) in Salsa.
Carles created different webwml
merge requests
with different fixes: invalid links to different architectures, 404s to external
sites, fixed templates with invalid URL to favicon.ico, making USE_SAMPLE_FILES=1
to avoid fetching external files, typos in English, etc.
Antonio did several Salsa maintenance tasks,
including his first platform security update and investigating an issue that was
causing the background tasks to be severely delayed, imparting the site usability.
Thorsten tried to fix some bugs in hplip and cups. Unfortunately this did not
result in an upload this month, but stay tuned!
Helmut continued to report undeclared file conflicts and correspond about them.
Helmut reviewed and merged Johannes Schauer Marin Rodrigues work on adding
hurd support to debvm.
Helmut fixed a pile of minor issues in the support libraries underpinning
unschroot.
Helmut adapted rebootstrap to archive regressions and improved its handling of
non-default gcc versions.
Santiago continued helping Aryan Karamtoth with the GSoC project on
introducing livepatching support for linux. Several challenges have been
identified in the recent couple of months, and Santiago is trying to guide Aryan
to address them and design the solution. The mentoring is done in collaboration
with Emmanuel Arias.
Lucas onboarded a co-maintainer for Valkey.
Lucas has started to work on a draft for the Debian Outreach project for
onboarding newcomers interested in Debian packaging via a video course.
Stefano worked on the after-math of DebConf 26, handling the bursary expense
reimbursement process and winding down the conference finances.
Stefano uploaded Python 3.15 RC 1 to Debian unstable.
Stefano did some budgeting and travel planning for the DebConf 27 site-visit
in Asahikawa, Japan. The DebConf committee sends a couple of members to meet
with the local team and view the venue together, in advance of the next DebConf.
A new version 0.0.7 of the td package for accessing
the twelvedata API for
financial is now on CRAN, and
has been built for r2u.
This release combines the standard set of maintenance changes that
accrue in a four and a half year period (!!) as that much time has past
since the previous release. But it also contains two contributed
functions to retried, respectively, a profile (available under a paid
plan) and a set of main fundamental data statistics, both provided
Kenneth Rose.
The NEWS entry follows.
Changes in version 0.0.7
(2026-09-13)
Added fun_statistics and fun_profile
function
Expanded README.md with additional badges, and updated
URLs
The third release (in twelve years !!) of the sanitizers package
is now on CRAN. sanitizers
provides true positives for programming errors detected by the Address
Sanitizer and friends such as the Undefined Behavior Sanitizer. This
permits validation of the setup when chasing such bug reports: it allows
us to ascertain that the compiler (and instrumented R version) are
correctly set up and the errors we expect to be reported are in fact
reported. That established, a proposed fix no longer exhibiting that
same error will then likely be a suitable one.
A very good resources for all things sanitizers is the Google repo at GitHub
and especially its
wiki.
A little over twelve years since the first release, and three years
since the second one, this update brings chiefly internal package
changes and maintenance. No functional changes, no behavioural
changes.
The brief NEWS entry follows.
Changes in version 0.1.2
(2026-09-12)
Expanded README.md with additional badges, and updated
URLs
Updated continuous integration multiple times
Switched to Authors@R
Added usage, arguments and
value sections to manual page
Two weeks ago, I released AppStream 1.2.0. This release contains a lot of great changes, but one of the most important ones concerns how media are being handled, and AppStream s default image export format.
AppStream is a Freedesktop metadata standard to describe software components. That can be anything from system services over fonts to console and graphical applications. AppStream metadata is supposed to give users enough information to decide whether they want to install a piece of software, to represent that piece of software, and to give the operating system enough information to decide whether a software component should be installed automatically and (to some extent) what capabilities and relations it has, to provide the user with sensible options.
Especially for the first two goals, and especially for GUI applications, AppStream supports icons and screenshots, which are used to showcase applications. Today, AppStream is used by all kinds of services, from Linux distributions over firmware updates to Flatpak and desktops directly. AppStream s original design however comes from the perspective of Linux distributions in 2011, where you may want to browse the software catalog offline, without delay, and without pinging an external server (which could be a privacy concern).
Therefore, a common way to deploy an AppStream-enabled software repository is to ship all icons of all applications in the repository to the user as part of the repository metadata download. AppStream does support remote icon downloads nowadays, and for a while I thought that this would become the default eventually. However, especially in today s world, having a bandwidth-saving, instantly responsive, privacy-protecting application browsing experience seems more important that ever.
PNG images are great!
The only format that AppStream supports for icons and screenshots (which are downloaded on-demand from your distributor s CDN) has always been exclusively PNG. PNG images are perfect for icons, because they compress well (especially for common icon shapes), are fast and simple to load, and can be loaded anywhere, by any toolkit or webbrowser. They also ensure we deliver faithful screenshot images, even though we may have scaled or re-rendered them. Still though, PNG images are less great for screenshots, as they are not very efficient, which puts strain on any CDN that has to deliver them, as well as on people s internet connections when browsing screenshots. Having smaller thumbnails alleviates that problem a little, but does not fully solve it.
But even for icons, PNG could be improved upon: In many cases, icons are re-downloaded with the repository metadata again and again, so having a large icon tarball adds up to the data transferred during metadata refreshes. AppStream also now supports large 128x128px icons, which nobody in 2012 expected we would need, adding even more data that will be re-downloaded. Saving some space here translates directly to lower bandwidth costs as well as faster downloads for users.
To improve PNG file sizes, the AppStream Compose library, which handles all image processing and metadata catalog composition, was running optipng on all generated PNG images. That does create smaller PNG images, but they were still relatively large compared to other image formats.
For a long time though, there was no alternative to PNG images for icons: There was no lossless image compression format that could give us the same quality as PNG images and that was also widely supported.
JPEG-XL vs PNG in AppStream
Since 2021 we have JPEG-XL (JXL), which offers a true lossless mode with often better compression than PNG. The issue was that JPEG-XL wasn t widely supported. Then, in 2025, the PDF Association selected JPEG-XL as the preferred image format for HDR images in PDFs, and now we are finally getting browser support and more ubiquitous availability of the format (you can try it right now in Firefox!).
For screenshots, using JXL s lossy mode, it has obvious and extreme size advantages over PNG, so supporting JXL or WebP for screenshot images was an obvious choice. If JXL would support the lossless case very well as well though, we could serve many use cases with the same exported image format, which is very attractive to me.
So, the obvious next question was whether it was worth the pain of switching the icon format, so I did some measurements on real icons. For that I used the AppStream component icon pool that Debian Unstable ships, which is almost 5000 application icons of various sizes, and converted them to PNG:
Icon size
Icons
PNG total
JXL total
Pool saved
PNG avg
JXL avg
Median saved
Mean saved
Worst
Best
Larger as JXL
48 48
1544
3.7 MiB
3.0 MiB
17.8%
2.4 KiB
2.0 KiB
17.9%
16.7%
-118.7%
60.0%
206
64 64
2018
7.0 MiB
5.8 MiB
17.8%
3.6 KiB
2.9 KiB
18.0%
15.8%
-112.7%
70.0%
279
128 128
1411
11.2 MiB
8.7 MiB
22.0%
8.1 KiB
6.3 KiB
20.1%
17.5%
-89.7%
61.0%
209
TOTAL
4973
21.9 MiB
17.5 MiB
19.9%
4.5 KiB
3.6 KiB
18.6%
16.6%
-118.7%
70.0%
694
PNG images saved with libpng at effort=4, compression=9, then optimized using optipng -o2, JXL images encoded using vips jxlsave lossless=1 effort=7 strip=1 via VIPS/libjxl.
As the table shows, using lossless JXL images over size-optimized PNG images (using optipng s default settings) provides a roughly 20% gain. This does not look like much, until you consider how often these files are downloaded: A 20% file size reduction may only save 1-2 MiB of disk space, but if they are downloaded over and over again by many clients, it will save a lot of bandwidth.
Interesting JXL encoding findings
As a sidequest, I was curious why some images were larger than their PNG counterparts when encoded with JXL, and what the ones that were significantly smaller were.
In short, the biggest size reductions for JXL existed on images that were already small as PNG, and contained large, flat color surfaces with hard edges and simple shapes. They were not very interesting, and much of JXL s wins come from accumulating smaller gains across all files, which compound the bigger icons get (especially at 128x128px, where JXL truly shines).
The events were JXL loses to PNG are more interesting: For example, it does quite poorly with pixel-art images that have a lot of repeating patterns. Those are encoded well by PNG, but less efficiently by JXL. Take for example Vonsh:
Icon of Vonsh, an SDL-based snake game, which PNG compresses better than JXL
My guess is that while PNG can exploit the repeating pixel patterns for compression, JXL s predicts surrounding pixels from its neighbours, which fails too often and makes it pay almost full entropy per pixel. In this single rare case, the PNG is at 5.4 KiB, while the JXL is almost 8 KiB in size.
Other cases I looked at were arguably buggy input data, where color channels were hidden under the alpha channel of the input image. PNG could probably again exploit repeats, while we were forcing JXL to encode pixels that were invisible in the final image. This is arguably a problem with the original input data. Currently, AppStream does not make any changes to icons at all, but in future we might add a filter that removes invisible colors from images to solve this pathological case (it was only two icons out of 5000 though, so it is not a high priority).
The third case I found where JXL loses to PNG were icons with checkerboard-like patterns:
Icon of x3270, an IBM 3270 Terminal Emulator
For those, PNG can likely again exploit the repeating patterns, while a checkerboard layout is pretty bad for left/top predictors like JXL s. However, in this case the size difference (and loss for JXL) is only 450 bytes, so even though JXL loses to PNG, it does so not by much.
JXL in AppStream
Given these findings, JPEG-XL is the default image format starting with AppStream 1.2.0. AppStream Compose will encode all images losslessly as JXL, while screenshots are encoded in lossy mode at Q=90 effort=7. Since the optipng step does not happen for JXL images, this comes at no speed penalty and is even a bit faster on modern x86_64 CPUs (where libjxl can use SIMD). PNG is still available, and Compose can be told to switch between the two formats.
Upsides of JXL in AppStream right now
If you use JXL in Compose or the recent release of appstream-generator, you will get much smaller images and, for screenshots, will benefit from other JPEG-XL features such as progressive decoding, providing a far nicer user experience. libAppStream has supported JXL icons since version 1.1.3, so your clients will need that version or a newer one, and all software centers will have to support loading JXL images (which all of them do, provided the right plugins are installed).
Downsides of switching to JXL too quickly
JXL is a very new format, so web browsers might not yet display it if you are serving webpages. Your clients may also have bugs in processing JXL images, as the format is still new . For example, switching on JXL in Debian sent KDE Discover into an infinite loop on startup while trying to load the icons (an issue which has been fixed, but clients will need that patch first before JXL is switched on).
This currently makes JXL enablement only possible when you know that your clients can support it. This is the case for me in Debian Unstable and Debian 14, which are using JXL images for a few weeks now, but not for any older releases. Platforms like Flatpak have it even harder, because they do know even less about their clients. So, even though it has big advantages, you may want to hold off on using JXL right away, and force PNG by setting the ImageFormat key to png in appstream-generator s configuration, or passing --image-format=png to appstreamcli compose.
It is also worth mentioning that JPEG-XL is much, much slower on systems that do not have SIMD instructions or for which the libjxl/jxl-rs library does not have them (such as apparently riscv64 right now). If this is a concern, you might not want to switch to JXL right away.
Media pipeline improvements
Besides the JXL default change, AppStream 1.2.0 also comes with a complete overhaul of its media processing pipeline. While libappstream, AppStream s main library, does not do any media processing and comes with very minimal dependencies to be embedded in client applications and used on servers, the same can not be said about libappstream-compose, AppStream s library to build metadata generating applications (the server-side part, usually).
The compose library has to render fonts into font specimen cards, inspect translation files, render SVG images, decode all kinds of raster images, inspect video files, etc. Especially the fonts, and the fact that fonts can appear in SVG images, has caused issues in the past, as libappstream-compose is a heavily threaded library and most font libraries can only work from a single thread. This forced the library to essentially go into single-thread mode anytime anything that could touch a font was being processed.
AppStream also originally was created for a safe world where applications were vetted by the distributors before their metadata was processed. This is increasingly not the case, so it made sense to put at least a few guardrails on the most complex part of the pipeline: The media processing. As part of the change, media processing was split out into a separate worker process. This solved two problems at once: Font handling was isolated in a single-threaded binary if we wanted to handle fonts in parallel, we could simply spawn more workers. And, being in a separate process, the media processing could now be sandboxed.
As part of the multiprocess changes, Compose also switched from using GdkPixbuf to VIPS for image processing. The latter allows for much more fine-grained control over the image output and encoding, and comes with a lot of well-maintained filters and operations, which made it possible to eliminate a fair chunk of AppStream s hand-rolled image processing operations. As part of this transition, we unfortunately lost the ability to read XPM images, which dropped about 20-30 applications from the pool at Debian. But in the name of security, this is a sensible choice, especially since most XPM icons were very small and low-resolution, and applications using them could benefit from adding a high-quality PNG icon anyway. With VIPS, we also now restrict the amount of image formats we can load to a sensible set, so extremely niche or unexpected formats will be outright rejected (this includes sane-but-unusual formats for screenshots and icons, such as TIFF images).
The Compose library, with all of these changes, will now just request high-level operations (e.g. render a font card for this font to a JXL image ) from the worker, and provide it with input data in sealed memfds and output locations as FDs as well. On Linux systems, the worker will use Landlock if available, to block all write access to the filesystem, deny device access and deny TCP and UDP as well. The sandbox can certainly be tightened a fair bit in future, but this was a good and safe start to gain some experience with it without having things break too easily, given the many places Compose is used in (also, Landlock s API is surprisingly nice to use, so it was easier than I thought to add in this early version).
With all of these changes, the libappstream-compose library is now also officially marked API-stable, so you should be able to rely on it in future to build new things (its API has barely changed in the past, and now with the new media API and defaults change in place, it was time to declare it stable).
I want to see / try this!
Currently, the easiest way to have a look at the new data is to check out Debian Unstable. If you have a JXL-enabled browser, you can also see the icons in AppStream Generator s HTML pages for Debian Sid. If you are using appstream-generator for your distribution, you will also get much more pleasant statistics and HTML pages, as well as fully deterministic media output and a whole bunch of security updates, so, update to its recent 1.0 release.
Please keep in mind that if you switch to JXL, the client tools receiving the image data have to support it. Support varies depending on the Linux distribution, so, test it first and switch the default back to PNG in case you encounter any issues.
What s next?
With so many features and changes landed, the next changes in AppStream will focus on improving what already exists and fixing any issues (there will be more blogposts about the other features 1.2.x delivers!). Testing with the entire Debian archive as data source makes me fairly confident though that there will not be many problems. In the longer term, tightening the media processing sandbox will also be something we might want to do, e.g. by hiding parts of the filesystem tree or filtering syscalls.
For JPEG-XL, one obvious question is Will you add support for it to the Freedesktop icon-theme specification as supported format alongside PNG, SVG(Z), and XPM? . For on-disk icon repositories, JXL s space-savings are less compelling, and it being HDR-capable is also not necessarily a killer feature (PNG can go a long way!). However, JPEG-XL s ability to immediately decode larger images at reduced resolution without resampling could legitimately be very powerful here, as applications could ship a single large image and quickly decode it at 1/2, 1/4 or 1/8 the size for different purposes in their UI. JPEG-XL also supports spot-color extra channels, which applications could use as masks to recolor raster icons at render time. This could be incredibly nice to color symbolic icons on-the-fly without any SVG and CSS. JXL also provides richer metadata, which might be neat for (license/author) documentation. So, the answer here is: Maybe it makes sense to allow another format, but this will have to be discussed first, as it would force JXL into every toolkit and desktop, which is a much bigger ask than supporting it only in AppStream.
As always, let me know what you think and please report any issues or bugs directly against AppStream or AppStream Generator if you encounter problems that are with the tools, and not with a project s metadata.
A brand new release 0.1.16 of the 
The Debian LTS Team, funded by 
This is a new series
The third release (in twelve years !!) of the 
