Search Results: "ana"

9 April 2025

Taavi V n nen: Writing a custom rsync server to automatically update a static site

Inspired by some friends,1 I too wanted to make a tiny website telling which event I am at this exact moment. Thankfully I already had an another toy project with that information easily available, so generating the web page was a matter of just querying that project's API and feeding that data to a HTML template. Now the obvious way to host that would be to hook up the HTML-generating code to a web server, maybe add some caching for the API calls, and then route external HTTPS traffic to it. However, that'd a) require that server to be constantly available to serve traffic, and b) be boring. For context: I have an existing setup called staticweb, which is effectively a fancy name for a couple of Puppet-managed servers that run Apache httpd to serve static web pages and have a bunch of systemd timers running rsync to ensure they're serving the same content. It works really well and I use it for things ranging from my website or whyisbetabroken.com to things like my internal apt repository. Now, there are two ways to get new content into that mechanism: it can be manually pushed in from e.g. a CI job, or the system can be configured to periodically pull it from a separate server. The latter mechanism was initially created so that I could pull the Debian packages from my separate reprepro server into the staticweb setup. It turns out that the latter makes a really neat method for handling other dynamically-generated static sites as well. So, for my "where is Taavi at" site, I ended up writing the server part in Go, and included an rsync server using the gokrazy/rsync package. Initially I just implemented a static temporary directory with a timer to regularly update the HTML file in it, but then I got an even more cursed idea: what if the HTML was dynamically generated when an rsync client connected to the server? So I did just that. For deployment, I slapped the entire server part in a container and deployed it to my Kubernetes cluster. The rsync server is exposed directly as a service to my internal network with no authentication or encryption - I think that's fine since that's a read-only service in a private LAN and the resulting HTML is going to be publicly exposed anyway. (Thanks to some DNS magic, just creating a LoadBalancer Service object with a special annotation is enough to have a DNS name provisioned for the assigned IP address, which is neat.) Overall the setup works nice, at least for now. I need to add some sort of a cache to not fetch unchanged information from the API since for every update. And I guess I could write some cursed rsyncd reverse proxy with per-module rules if I end up creating more sites like this to avoid creating new LoadBalancer services for each of them.

  1. Mostly from Sammy's where.fops.at.

8 April 2025

Petter Reinholdtsen: Some notes on Linux LUKS cracking

A few months ago, I found myself in the unfortunate position that I had to try to recover the password used to encrypt a Linux hard drive. Tonight a few friends of mine asked for details on this effort. I guess it is a good idea to expose the recipe I found to a wider audience, so here are a few relevant links and key findings. I've forgotten a lot, so part of this is taken from memory. I found a good recipe in a blog post written in 2019 by diverto, titled Cracking LUKS/dm-crypt passphrases. I tried both the john the ripper approach where it generated password candidates and passed it to cryptsetup and the luks2jack.py approach (which did not work for me, if I remember correctly), but believe I had most success with the hashcat approach. I had it running for several days on my Thinkpad X230 laptop from 2012. I do not remember the exact hash rate, but when I tested it again just now on the same machine by running "hashcat -a 0 hashcat.luks longlist --force", I got a hash rate of 7 per second. Testing it on a newer machine with a 32 core AMD CPU, I got a hash rate of 289 per second. Using the ROCM OpenCL approach on the same machine I managed to get a hash rate of 2821 per second.
Session..........: hashcat                                
Status...........: Quit
Hash.Mode........: 14600 (LUKS v1 (legacy))
Hash.Target......: hashcat.luks
Time.Started.....: Tue Apr  8 23:06:08 2025 (1 min, 10 secs)
Time.Estimated...: Tue Apr  8 23:12:49 2025 (5 mins, 31 secs)
Kernel.Feature...: Pure Kernel
Guess.Base.......: File (/usr/share/dict/bokm l)
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........:     2821 H/s (8.18ms) @ Accel:128 Loops:128 Thr:32 Vec:1
Recovered........: 0/1 (0.00%) Digests (total), 0/1 (0.00%) Digests (new)
Progress.........: 0/935405 (0.00%)
Rejected.........: 0/0 (0.00%)
Restore.Point....: 0/935405 (0.00%)
Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:972928-973056
Candidate.Engine.: Device Generator
Candidates.#1....: A-aksje -> fiskebil
Hardware.Mon.#1..: Temp: 73c Fan: 77% Util: 99% Core:2625MHz Mem: 456MHz Bus:16
Note that for this last test I picked the largest word list I had on my machine (dict/bokm l) as a fairly random work list and not because it is useful for cracking my particular use case from a few months ago. As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

4 April 2025

Gunnar Wolf: Naming things revisited

How long has it been since you last saw a conversation over different blogs syndicated at the same planet? Well, it s one of the good memories of the early 2010s. And there is an opportunity to re-engage! I came across Evgeni s post naming things is hard in Planet Debian. So, what names have I given my computers? I have had many since the mid-1990s I also had several during the decade before that, but before Linux, my computers didn t hve a formal name. Naming my computers something nice Linux gave me. I have forgotten many. Some of the names I have used:

Evgeni Golov: naming things is hard

I got a new laptop (a Lenovo Thinkpad X1 Carbon Gen 12, more on that later) and as always with new pets, it needed a name. My naming scheme is roughly "short japanese words that somehow relate to the machine". The current (other) machines at home are (not all really in use): Then, I happen to rent a few servers: I also had machines in the past, that are no longer with me: And also servers from the past: So, what shall I call the new one? It will be "juuni" ( ), which means 12. Creative, huh?

1 April 2025

Colin Watson: Free software activity in March 2025

Most of my Debian contributions this month were sponsored by Freexian. You can also support my work directly via Liberapay. OpenSSH Changes in dropbear 2025.87 broke OpenSSH s regression tests. I cherry-picked the fix. I reviewed and merged patches from Luca Boccassi to send and accept the COLORTERM and NO_COLOR environment variables. Python team Following up on last month, I fixed some more uscan errors: I upgraded these packages to new upstream versions: In bookworm-backports, I updated python-django to 3:4.2.19-1. Although Debian s upgrade to python-click 8.2.0 was reverted for the time being, I fixed a number of related problems anyway since we re going to have to deal with it eventually: dh-python dropped its dependency on python3-setuptools in 6.20250306, which was long overdue, but it had quite a bit of fallout; in most cases this was simply a question of adding build-dependencies on python3-setuptools, but in a few cases there was a missing build-dependency on python3-typing-extensions which had previously been pulled in as a dependency of python3-setuptools. I fixed these bugs resulting from this: We agreed to remove python-pytest-flake8. In support of this, I removed unnecessary build-dependencies from pytest-pylint, python-proton-core, python-pyzipper, python-tatsu, python-tatsu-lts, and python-tinycss, and filed #1101178 on eccodes-python and #1101179 on rpmlint. There was a dnspython autopkgtest regression on s390x. I independently tracked that down to a pylsqpack bug and came up with a reduced test case before realizing that Pranav P had already been working on it; we then worked together on it and I uploaded their patch to Debian. I fixed various other build/test failures: I enabled more tests in python-moto and contributed a supporting fix upstream. I sponsored Maximilian Engelhardt to reintroduce zope.sqlalchemy. I fixed various odds and ends of bugs: I contributed a small documentation improvement to pybuild-autopkgtest(1). Rust team I upgraded rust-asn1 to 0.20.0. Science team I finally gave in and joined the Debian Science Team this month, since it often has a lot of overlap with the Python team, and Freexian maintains several packages under it. I fixed a uscan error in hdf5-blosc (maintained by Freexian), and upgraded it to a new upstream version. I fixed python-vispy: missing dependency on numpy abi. Other bits and pieces I fixed debconf should automatically be noninteractive if input is /dev/null. I fixed a build failure with GCC 15 in yubihsm-shell (maintained by Freexian). Prompted by a CI failure in debusine, I submitted a large batch of spelling fixes and some improved static analysis to incus (#1777, #1778) and distrobuilder. After regaining access to the repository, I fixed telegnome: missing app icon in About dialogue and made a new 0.3.7 release.

Guido G nther: Free Software Activities March 2025

Another short status update of what happened on my side last month. Some more ModemManager bits landed, Phosh 0.46 is out, haptic feedback is now better tunable plus some more. See below for details (no April 1st joke in there, I promise): phosh phoc phosh-osk-stub phosh-mobile-settings phosh-tour pfs xdg-desktop-portal-gtk xdg-desktop-portal-phosh meta-phosh feedbackd feedbackd-device-themes gmobile livi Debian git-buildpackage feedbackd-device-themes wlroots ModemManager Tuba xdg-spec gnome-calls Reviews This is not code by me but reviews on other peoples code. The list is (as usual) slightly incomplete. Thanks for the contributions! Help Development If you want to support my work see donations. Comments? Join the Fediverse thread

31 March 2025

Russell Coker: Links March 2025

Anarcat s review of Fish is interesting and shows some benefits I hadn t previously realised, I ll have to try it out [1]. Longnow has an insightful article about religion and magic mushrooms [2]. Brian Krebs wrote an informative artivle about DOGE and the many security problems that it has caused to the US government [3]. Techdirt has an insightful article about why they are forced to become a democracy blog after the attacks by Trump et al [4]. Antoine wrote an insightful blog post about the war for the Internet and how in many ways we are losing to fascists [5]. Interesting story about people working for free at Apple to develop a graphing calculator [6]. We need ways for FOSS people to associate to do such projects. Interesting YouTube video about a wiki for building a cheap road legal car [7]. Interesting video about powering spacecraft with Plutonion 238 and how they are running out [8]. Interesting information about the search for mh370 [9]. I previously hadn t been convinced that it was hijacked but I am now. The EFF has an interesting article about the Rayhunter, a tool to detect cellular spying that can run with cheap hardware [10].
  • [1] https://anarc.at/blog/2025-02-28-fish/
  • [2] https://longnow.org/ideas/is-god-a-mushroom/
  • [3] https://tinyurl.com/27wbb5ec
  • [4] https://tinyurl.com/2cvo42ro
  • [5] https://anarc.at/blog/2025-03-21-losing-war-internet/
  • [6] https://www.pacifict.com/story/
  • [7] https://www.youtube.com/watch?v=x8jdx-lf2Dw
  • [8] https://www.youtube.com/watch?v=geIhl_VE0IA
  • [9] https://www.youtube.com/watch?v=HIuXEU4H-XE
  • [10] https://tinyurl.com/28psvpx7
  • Simon Josefsson: On Binary Distribution Rebuilds

    I rebuilt (the top-50 popcon) Debian and Ubuntu packages, on amd64 and arm64, and compared the results a couple of months ago. Since then the Reproduce.Debian.net effort has been launched. Unlike my small experiment, that effort is a full-scale rebuild with more architectures. Their goal is to reproduce what is published in the Debian archive. One differences between these two approaches are the build inputs: The Reproduce Debian effort use the same build inputs which were used to build the published packages. I m using the latest version of published packages for the rebuild. What does that difference imply? I believe reproduce.debian.net will be able to reproduce more of the packages in the archive. If you build a C program using one version of GCC you will get some binary output; and if you use a later GCC version you are likely to end up with a different binary output. This is a good thing: we want GCC to evolve and produce better output over time. However it means in order to reproduce the binaries we publish and use, we need to rebuild them using whatever build dependencies were used to prepare those binaries. The conclusion is that we need to use the old GCC to rebuild the program, and this appears to be the Reproduce.Debian.Net approach. It would be a huge success if the Reproduce.Debian.net effort were to reach 100% reproducibility, and this seems to be within reach. However I argue that we need go further than that. Being able to rebuild the packages reproducible using older binary packages only begs the question: can we rebuild those older packages? I fear attempting to do so ultimately leads to a need to rebuild 20+ year old packages, with a non-negligible amount of them being illegal to distribute or are unable to build anymore due to bit-rot. We won t solve the Trusting Trust concern if our rebuild effort assumes some initial binary blob that we can no longer build from source code. I ve made an illustration of the effort I m thinking of, to reach something that is stronger than reproducible rebuilds. I am calling this concept a Idempotent Rebuild, which is an old concept that I believe is the same as John Gilmore has described many years ago.
    The illustration shows how the Debian main archive is used as input to rebuild another stage #0 archive. This stage #0 archive can be compared with diffoscope to the main archive, and all differences are things that would be nice to resolve. The packages in the stage #0 archive is used to prepare a new container image with build tools, and the stage #0 archive is used as input to rebuild another version of itself, called the stage #1 archive. The differences between stage #0 and stage #1 are also useful to analyse and resolve. This process can be repeated many times. I believe it would be a useful property if this process terminated at some point, where the stage #N archive was identical to the stage #N-1 archive. If this would happen, I label the output archive as an Idempotent Rebuild of the distribution. How big is N today? The simplest assumption is that it is infinity. Any build timestamp embedded into binary packages will change on every iteration. This will cause the process to never terminate. Fixing embedded timestamps is something that the Reproduce.Debian.Net effort will also run into, and will have to resolve. What other causes for differences could there be? It is easy to see that generally if some output is not deterministic, such as the sort order of assembler object code in binaries, then the output will be different. Trivial instances of this problem will be caught by the reproduce.debian.net effort as well. Could there be higher order chains that lead to infinite N? It is easy to imagine the existence of these, but I don t know how they would look like in practice. An ideal would be if we could get down to N=1. Is that technically possible? Compare building GCC, it performs an initial stage 0 build using the system compiler to produce a stage 1 intermediate, which is used to build itself again to stage 2. Stage 1 and 2 is compared, and on success (identical binaries), the compilation succeeds. Here N=2. But this is performed using some unknown system compiler that is normally different from the GCC version being built. When rebuilding a binary distribution, you start with the same source versions. So it seems N=1 could be possible. I m unhappy to not be able to report any further technical progress now. The next step in this effort is to publish the stage #0 build artifacts in a repository, so they can be used to build stage #1. I already showed that stage #0 was around ~30% reproducible compared to the official binaries, but I didn t save the artifacts in a reusable repository. Since the official binaries were not built using the latest versions, it is to be expected that the reproducibility number is low. But what happens at stage #1? The percentage should go up: we are now compare the rebuilds with an earlier rebuild, using the same build inputs. I m eager to see this materialize, and hope to eventually make progress on this. However to build stage #1 I believe I need to rebuild a much larger number packages in stage #0, it could be roughly similar to the build-essentials-depends package set. I believe the ultimate end goal of Idempotent Rebuilds is to be able to re-bootstrap a binary distribution like Debian from some other bootstrappable environment like Guix. In parallel to working on a achieving the 100% Idempotent Rebuild of Debian, we can setup a Guix environment that build Debian packages using Guix binaries. These builds ought to eventually converge to the same Debian binary packages, or there is something deeply problematic happening. This approach to re-bootstrap a binary distribution like Debian seems simpler than rebuilding all binaries going back to the beginning of time for that distribution. What do you think? PS. I fear that Debian main may have already went into a state where it is not able to rebuild itself at all anymore: the presence and assumption of non-free firmware and non-Debian signed binaries may have already corrupted the ability for Debian main to rebuild itself. To be able to complete the idempotent and bootstrapped rebuild of Debian, this needs to be worked out.

    30 March 2025

    Russ Allbery: Review: Cascade Failure

    Review: Cascade Failure, by L.M. Sagas
    Series: Ambit's Run #1
    Publisher: Tor
    Copyright: 2024
    ISBN: 1-250-87126-3
    Format: Kindle
    Pages: 407
    Cascade Failure is a far-future science fiction adventure with a small helping of cyberpunk vibes. It is the first of a (so far) two-book series, and was the author's first novel. The Ambit is an old and small Guild ship, not much to look at, but it holds a couple of surprises. One is its captain, Eoan, who is an AI with a deep and insatiable curiosity that has driven them and their ship farther and farther out into the Spiral. The other is its surprisingly competent crew: a battle-scarred veteran named Saint who handles the fighting, and a talented engineer named Nash who does literally everything else. The novel opens with them taking on supplies at Aron Outpost. A supposed Guild deserter named Jalsen wanders into the ship looking for work. An AI ship with a found-family crew is normally my catnip, so I wanted to love this book. Alas, I did not. There were parts I liked. Nash is great: snarky, competent, and direct. Eoan is a bit distant and slightly more simplistic of a character than I was expecting, but I appreciated the way Sagas put them firmly in charge of the ship and departed from the conventional AI character presentation. Once the plot starts in earnest (more on that in a moment), we meet Anke, the computer hacker, whose charming anxiety reaction is a complete inability to stop talking and who adds some needed depth to the character interactions. There's plenty of action, a plot that makes at least some sense, and a few moments that almost achieved the emotional payoff the author was attempting. Unfortunately, most of the story focuses on Saint and Jal, and both of them are irritatingly dense cliches. The moment Jal wanders onto the Ambit in the first chapter, the reader is informed that Jal, Saint, and Eoan have a history. The crew of the Ambit spent a year looking for Jal and aren't letting go of him now that they've found him. Jal, on the other hand, clearly blames Saint for something and is not inclined to trust him. Okay, fine, a bit generic of a setup but the writing moved right along and I was curious enough. It then takes a full 180 pages before the reader finds out what the hell is going on with Saint and Jal. Predictably, it's a stupid misunderstanding that could have been cleared up with one conversation in the second chapter. Cascade Failure does not contain a romance (and to the extent that it hints at one, it's a sapphic romance), but I swear Saint and Jal are both the male protagonist from a certain type of stereotypical heterosexual romance novel. They're both the brooding man with the past, who is too hurt to trust anyone and assumes the worst because he's unable to use his words or ask an open question and then listen to the answer. The first half of this book is them being sullen at each other at great length while both of them feel miserable. Jal keeps doing weird and suspicious things to resolve a problem that would have been far more easily resolved by the rest of the crew if he would offer any explanation at all. It's not even suspenseful; we've read about this character enough times to know that he'll turn out to have a heart of gold and everything will be a misunderstanding. I found it tedious. Maybe people who like slow burn romances with this character type will have a less negative reaction. The real plot starts at about the time Saint and Jal finally get their shit sorted out. It turns out to have almost nothing to do with either of them. The environmental control systems of worlds are suddenly failing (hence the book title), and Anke, the late-arriving computer programmer and terraforming specialist, has a rather wild theory about what's happening. This leads to a lot of action, some decent twists, and a plot that felt very cyberpunk to me, although unfortunately it culminates in an absurdly-cliched action climax. This book is an action movie that desperately wants to make you feel all the feels, and it worked about as well as that typically works in action movies for me. Jaded cynicism and an inability to communicate are not the ways to get me to have an emotional reaction to a book, and Jal (once he finally starts talking) is so ridiculously earnest that it's like reading the adventures of a Labrador puppy. There was enough going on that it kept me reading, but not enough for the story to feel satisfying. I needed a twist, some depth, way more Nash and Anke and way less of the men, something. Everyone is going to compare this book to Firefly, but Firefly had better banter, created more complex character interactions due to the larger and more varied crew, and played the cynical mercenary for laughs instead of straight, all of which suited me better. This is not a bad book, particularly once it gets past the halfway point, but it's not that memorable either, at least for me. If you're looking for a space adventure with heavy action hero and military SF vibes that wants to be about Big Feelings but gets there in mostly obvious ways, you could do worse. If you're looking for a found-family starship crew story more like Becky Chambers, I think you'll find this one a bit too shallow and obvious. Not really recommended, although there's nothing that wrong with it and I'm sure other people's experience will differ. Followed by Gravity Lost, which I'm unlikely to read. Rating: 6 out of 10

    25 March 2025

    Otto Kek l inen: Debian Salsa CI in Google Summer of Code 2025

    Featured image of post Debian Salsa CI in Google Summer of Code 2025Are you a student aspiring to participate in the Google Summer of Code 2025? Would you like to improve the continuous integration pipeline used at salsa.debian.org, the Debian GitLab instance, to help improve the quality of tens of thousands of software packages in Debian? This summer 2025, I and Emmanuel Arias will be participating as mentors in the GSoC program. We are available to mentor students who propose and develop improvements to the Salsa CI pipeline, as we are members of the Debian team that maintains it. A post by Santiago Ruano Rinc n in the GitLab blog explains what Salsa CI is and its short history since inception in 2018. At the time of the article in fall 2023 there were 9000+ source packages in Debian using Salsa CI. Now in 2025 there are over 27,000 source packages in Debian using it, and since summer 2024 some Ubuntu developers have started using it for enhanced quality assurance of packaging changes before uploading new package revisions to Ubuntu. Personally, I have been using Salsa CI since its inception, and contributing as a team member since 2019. See my blog post about GitLab CI for MariaDB in Debian for a description of an advanced and extensive use case. Helping Salsa CI is a great way to make a global impact, as it will help avoid regressions and improve the quality of Debian packages. The benefits reach far beyond just Debian, as it will also help hundreds of Debian derivatives, such as Ubuntu, Linux Mint, Tails, Purism PureOS, Pop!_OS, Zorin OS, Raspberry Pi OS, a large portion of Docker containers, and even the Windows Subsystem for Linux.

    Improving Salsa CI: more features, robustness, speed While Salsa CI with contributions from 71 people is already quite mature and capable, there are many ideas floating around about how it could be further extended. For example, Salsa CI issue #147 describes various static analyzers and linters that may be generally useful. Issue #411 proposes using libfaketime to run autopkgtest on arbitrary future dates to test for failures caused by date assumptions, such as the Y2038 issue. There are also ideas about making Salsa CI more robust and code easier to reuse by refactoring some of the yaml scripts into independent scripts in #230, which could make it easier to run Salsa CI locally as suggested in #169. There are also ideas about improving the Salsa CI s own CI to avoid regressions from pipeline changes in #318. The CI system is also better when it s faster, and some speed improvement ideas have been noted in #412. Improvements don t have to be limited to changes in the pipeline itself. A useful project would also be to update more Debian packages to use Salsa CI, and ensure they adopt it in an optimal way as noted in #416. It would also be nice to have a dashboard with statistics about all public Salsa CI pipeline runs as suggested in #413. These and more ideas can be found in the issue list by filtering for tags Newcomer, Nice-To-Have or Accepting MRs. A Google Summer of Code proposal does not have to be limited to these existing ideas. Participants are also welcome to propose completely novel ideas!

    Good time to also learn Debian packaging Anyone working with Debian team should also take the opportunity to learn Debian packaging, and contribute to the packaging or maintenance of 1-2 packages in parallel to improving the Salsa CI. All Salsa CI team members are also Debian Developers who can mentor and sponsor uploads to Debian. Maintaining a few packages is a great way to eat your own cooking and experience Salsa CI from the user perspective, and likely to make you better at Salsa CI development.

    Apply now! The contributor applications opened yesterday on March 24, so to participate act now! If you are an eligible student and want to attend, head over to summerofcode.withgoogle.com to learn more. There are over a thousand participating organizations, with Debian, GitLab and MariaDB being some examples. Within these organizations there may be multiple subteams and projects to choose from. The full list of participating Debian projects can be found in the Debian wiki. If you are interested in GSoC for Salsa CI specifically, feel free to
    1. Reach out to me and Emmanuel by email at otto@ and eamanu@ (debian.org).
    2. Sign up at salsa.debian.org for an account (note it takes a few days due to manual vetting and approval process)
    3. Read the project README, STRUCTURE and CONTRIBUTING to get a developer s overview
    4. Participate in issue discussions at https://salsa.debian.org/salsa-ci-team/pipeline/-/issues/
    Note that you don t have to wait for GSoC to officially start to contribute. In fact, it may be useful to start immediately by submitting a Merge Request to do some small contribution, just to learn the process and to get more familiar with how everything works, and the team maintaining Salsa CI. Looking forward to seeing new contributors!

    24 March 2025

    Jonathan McDowell: Who pays the cost of progress in software?

    I am told, by friends who have spent time at Google, about the reason Google Reader finally disappeared. Apparently it had become a 20% Project for those who still cared about it internally, and there was some major change happening to one of it upstream dependencies that was either going to cause a significant amount of work rearchitecting Reader to cope, or create additional ongoing maintenance burden. It was no longer viable to support it as a side project, so it had to go. This was a consequence of an internal culture at Google where service owners are able to make changes that can break downstream users, and the downstream users are the ones who have to adapt. My experience at Meta goes the other way. If you own a service or other dependency and you want to make a change that will break things for the users, it s on you to do the migration, or at the very least provide significant assistance to those who own the code. You don t just get to drop your new release and expect others to clean up; doing that tends to lead to changes being reverted. The culture flows the other way; if you break it, you fix it (nothing is someone else s problem). There are pluses and minuses to both approaches. Users having to drive the changes to things they own stops them from blocking progress. Service/code owners having to drive the changes avoids the situation where a wildly used component drops a new release that causes a lot of high priority work for folk in order to adapt. I started thinking about this in the context of Debian a while back, and a few incidents since have resulted in my feeling that we re closer to the Google model than the Meta model. Anyone can upload a new version of their package to unstable, and that might end up breaking all the users of it. It s not quite as extreme as rolling out a new service, because it s unstable that gets affected (the clue is in the name, I really wish more people would realise that), but it can still result in release critical bugs for lots other Debian contributors. A good example of this are toolchain changes. Major updates to GCC and friends regularly result in FTBFS issues in lots of packages. Now in this instance the maintainer is usually diligent about a heads up before the default changes, but it s still a whole bunch of work for other maintainers to adapt (see the list of FTBFS bugs for GCC 15 for instance - these are important, but not serious yet). Worse is when a dependency changes and hasn t managed to catch everyone who might be affected, so by the time it s discovered it s release critical, because at least one package no longer builds in unstable. Commercial organisations try to avoid this with a decent CI/CD setup that either vendors all dependencies, or tracks changes to them and tries rebuilds before allowing things to land. This is one of the instances where a monorepo can really shine; if everything you need is in there, it s easier to track the interconnections between different components. Debian doesn t have a CI/CD system that runs for every upload, allowing us to track exact causes of regressions. Instead we have Lucas, who does a tremendous job of running archive wide rebuilds to make sure we can still build everything. Unfortunately that means I am often unfairly grumpy at him; my heart sinks when I see a bug come in with his name attached, because it often means one of my packages has a new RC bug where I m going to have to figure out what changed elsewhere to cause it. However he s just (very usefully) surfacing an issue someone else created, rather than actually being the cause of the problem. I don t know if I have a point to this post. I think it s probably that I wish folk in Free Software would try and be mindful of the incompatible changes they might introducing, and the toil they create for other volunteer developers, often not directly visible to the person making the change. The approach done by the Debian toolchain maintainers strikes me as a good balance; they do a bunch of work up front to try and flag all the places that might need to make changes, far enough in advance of the breaking change actually landing. However they don t then allow a tardy developer to block progress.

    22 March 2025

    Antoine Beaupr : Minor outage at Teksavvy business

    This morning, internet was down at home. The last time I had such an issue was in February 2023, when my provider was Oricom. Now I'm with a business service at Teksavvy Internet (TSI), in which I pay 100$ per month for a 250/50 mbps business package, with a static IP address, on which I run, well, everything: email services, this website, etc.

    Mitigation

    Email The main problem when the service goes down like this for prolonged outages is email. Mail is pretty resilient to failures like this but after some delay (which varies according to the other end), mail starts to drop. I am actually not sure what the various settings are among different providers, but I would assume mail is typically kept for about 24h, so that's our mark. Last time, I setup VMs at Linode and Digital Ocean to deal better with this. I have actually kept those VMs running as DNS servers until now, so that part is already done. I had fantasized about Puppetizing the mail server configuration so that I could quickly spin up mail exchangers on those machines. But now I am realizing that my Puppet server is one of the service that's down, so this would not work, at least not unless the manifests can be applied without a Puppet server (say with puppet apply). Thankfully, my colleague groente did amazing work to refactor our Postfix configuration in Puppet at Tor, and that gave me the motivation to reproduce the setup in the lab. So I have finally Puppetized part of my mail setup at home. That used to be hand-crafted experimental stuff documented in a couple of pages in this wiki, but is now being deployed by Puppet. It's not complete yet: spam filtering (including DKIM checks and graylisting) are not implemented yet, but that's the next step, presumably to do during the next outage. The setup should be deployable with puppet apply, however, and I have refined that mechanism a little bit, with the run script. Heck, it's not even deployed yet. But the hard part / grunt work is done.

    Other The outage was "short" enough (5 hours) that I didn't take time to deploy the other mitigations I had deployed in the previous incident. But I'm starting to seriously consider deploying a web (and caching) reverse proxy so that I endure such problems more gracefully.

    Side note on proper servics Typically, I tend to think of a properly functioning service as having four things:
    1. backups
    2. documentation
    3. monitoring
    4. automation
    5. high availability
    Yes, I miscounted. This is why you have high availability.

    Backups Duh. If data is maliciously or accidentally destroyed, you need a copy somewhere. Preferably in a way that malicious joe can't get to. This is harder than you think.

    Documentation I have an entire template for this. Essentially, it boils down to using https://diataxis.fr/ and this "audit" guide. For me, the most important parts are:
    • disaster recovery (includes backups, probably)
    • playbook
    • install/upgrade procedures (see automation)
    You probably know this is hard, and this is why you're not doing it. Do it anyways, you'll think it sucks, but you'll be really grateful for whatever scraps you wrote when you're in trouble.

    Monitoring If you don't have monitoring, you'll know it fails too late, and you won't know it recovers. Consider high availability, work hard to reduce noise, and don't have machine wake people up, that's literally torture and is against the Geneva convention. Consider predictive algorithm to prevent failures, like "add storage within 2 weeks before this disk fills up". This is harder than you think.

    Automation Make it easy to redeploy the service elsewhere. Yes, I know you have backups. That is not enough: that typically restores data and while it can also include configuration, you're going to need to change things when you restore, which is what automation (or call it "configuration management" if you will) will do for you anyways. This also means you can do unit tests on your configuration, otherwise you're building legacy. This is probably as hard as you think.

    High availability Make it not fail when one part goes down. Eliminate single points of failures. This is easier than you think, except for storage and DNS (which, I guess, means it's harder than you think too).

    Assessment In the above 5 items, I check two:
    1. backups
    2. documentation
    And barely: I'm not happy about the offsite backups, and my documentation is much better at work than at home (and even there, I have a 15 year backlog to catchup on). I barely have monitoring: Prometheus is scraping parts of the infra, but I don't have any sort of alerting -- by which I don't mean "electrocute myself when something goes wrong", I mean "there's a set of thresholds and conditions that define an outage and I can look at it". Automation is wildly incomplete. My home server is a random collection of old experiments and technologies, ranging from Apache with Perl and CGI scripts to Docker containers running Golang applications. Most of it is not Puppetized (but the ratio is growing). Puppet itself introduces a huge attack vector with kind of catastrophic lateral movement if the Puppet server gets compromised. And, fundamentally, I am not sure I can provide high availability in the lab. I'm just this one guy running my home network, and I'm growing older. I'm thinking more about winding things down than building things now, and that's just really sad, because I feel we're losing (well that escalated quickly).

    Resolution In the end, I didn't need any mitigation and the problem fixed itself. I did do quite a bit of cleanup so that feels somewhat good, although I despaired quite a bit at the amount of technical debt I've accumulated in the lab.

    Timeline Times are in UTC-4.
    • 6:52: IRC bouncer goes offline
    • 9:20: called TSI support, waited on the line 15 minutes then was told I'd get a call back
    • 9:54: outage apparently detected by TSI
    • 11:00: no response, tried calling back support again
    • 11:10: confirmed bonding router outage, no official ETA but "today", source of the 9:54 timestamp above
    • 12:08: TPA monitoring notices service restored
    • 12:34: call back from TSI; service restored, problem was with the "bonder" configuration on their end, which was "fighting between Montr al and Toronto"

    Antoine Beaupr : Losing the war for the free internet

    Warning: this is a long ramble I wrote after an outage of my home internet. You'll get your regular scheduled programming shortly. I didn't realize this until relatively recently, but we're at war. Fascists and capitalists are trying to take over the world, and it's bringing utter chaos. We're more numerous than them, of course: this is only a handful of people screwing everyone else over, but they've accumulated so much wealth and media control that it's getting really, really hard to move around. Everything is surveilled: people are carrying tracking and recording devices in their pockets at all time, or they drive around in surveillance machines. Payments are all turning digital. There's cameras everywhere, including in cars. Personal data leaks are so common people kind of assume their personal address, email address, and other personal information has already been leaked. The internet itself is collapsing: most people are using the network only as a channel to reach a "small" set of "hyperscalers": mind-boggingly large datacenters that don't really operate like the old internet. Once you reach the local endpoint, you're not on the internet anymore. Netflix, Google, Facebook (Instagram, Whatsapp, Messenger), Apple, Amazon, Microsoft (Outlook, Hotmail, etc), all those things are not really the internet anymore. Those companies operate over the "internet" (as in the TCP/IP network), but they are not an "interconnected network" as much as their own, gigantic silos so much bigger than everything else that they essentially dictate how the network operates, regardless of standards. You access it over "the web" (as in "HTTP") but the fabric is not made of interconnected links that cross sites: all those sites are trying really hard to keep you captive on their platforms. Besides, you think you're writing an email to the state department, for example, but you're really writing to Microsoft Outlook. That app your university or border agency tells you to install, the backend is not hosted by those institutions, it's on Amazon. Heck, even Netflix is on Amazon. Meanwhile I've been operating my own mail server first under my bed (yes, really) and then in a cupboard or the basement for almost three decades now. And what for? So I can tell people I can? Maybe! I guess the reason I'm doing this is the same reason people are suddenly asking me about the (dead) mesh again. People are worried and scared that the world has been taken over, and they're right: we have gotten seriously screwed. It's the same reason I keep doing radio, minimally know how to grow food, ride a bike, build a shed, paddle a canoe, archive and document things, talk with people, host an assembly. Because, when push comes to shove, there's no one else who's going to do it for you, at least not the way that benefits the people. The Internet is one of humanity's greatest accomplishments. Obviously, oligarchs and fascists are trying to destroy it. I just didn't expect the tech bros to be flipping to that side so easily. I thought we were friends, but I guess we are, after all, enemies. That said, that old internet is still around. It's getting harder to host your own stuff at home, but it's not impossible. Mail is tricky because of reputation, but it's also tricky in the cloud (don't get fooled!), so it's not that much easier (or cheaper) there. So there's things you can do, if you're into tech. Share your wifi with your neighbours. Build a LAN. Throw a wire over to your neighbour too, it works better than wireless. Use Tor. Run a relay, a snowflake, a webtunnel. Host a web server. Build a site with a static site generator and throw it in the wind. Download and share torrents, and why not a tracker. Run an IRC server (or Matrix, if you want to federate and lose high availability). At least use Signal, not Whatsapp or Messenger. And yes, why not, run a mail server, join a mesh. Don't write new software, there's plenty of that around already. (Just kidding, you can write code, cypherpunk.) You can do many of those things just by setting up a FreedomBox. That is, after all, the internet: people doing their own thing for their own people. Otherwise, it's just like sitting in front of the television and watching the ads. Opium of the people, like the good old time. Let a billion droplets build the biggest multitude of clouds that will storm over this world and rip apart this fascist conspiracy. Disobey. Revolt. Build. We are more than them.

    21 March 2025

    Jamie McClelland: AI's Actual Impact

    Two years after OpenAI launched ChatGPT 3.5, humanity is not on the cusp of extinction and Elon Musk seems more responsible for job loss than any AI agent. However, ask any web administrator and you will learn that large language models are having a significant impact on the world wide web (or, for a less technical account, see Forbes articles on bots). At May First, a membership organization that has been supporting thousands of web site for over 20 years, we have never seen anything like this before.

    Now what? So, no, unfortunately, your web site is not suddenly getting really popular. And, you are blessed with a whole new set of strategic decisions. Fortunately, May First has undergone a major infrastructure transition, resulting in centralized logging of all web sites and a fleet of web proxy servers that intercept all web traffic. Centralized logging means we can analyze traffic and identify bots more easily, and a web proxy fleet allows us to more easily implement rules across all web sites. However, even with all of our latest changes and hours upon hours of work to keep out the bots, our members are facing some hard decisions about maintaining an open web. One member of May First provides Google translations of their web site to every language available. But wow, that is now a disaster because instead of having every bot under the sun scrapping all 843 (a made up number) pieces of unique content on their site, the same bots are scraping 843 * (number of available languages) pieces of content on their site. Should they stop providing this translation service in order to ensure people can access their site in the site s primary language? Should web sites turn off their search features that include drop down options of categories to prevent bots from systematically refreshing the search page with every possible combination of search terms? Do we need to alter our calendar software to avoid providing endless links into the future (ok, that is an easy one)?

    What s next? Something has to change.
    • Lock down web 2.0. Web 2.0 brought us wonderful dynamic web sites, which Drupal and WordPress and many other pieces of amazing software have supported for over a decade. This is the software that is getting bogged down by bots. Maybe we need to figure out a way to lock down the dynamic aspects of this software to logged in users and provide static content for everyone else?
    • Paywalls and accounts everywhere. There s always been an amazing non-financial reward to providing a web site with high quality movement oriented content for free. It populates the search engines, provides links to inspiring and useful content in moments of crises, and can galvanize movements. But these moments of triumph happen between long periods of hard labor that now seems to mostly feed capitalist AI scumbags. If we add a new set of expenses and labor to keep the sites running for this purpose, how sustainable is that? Will our treasure of free movement content have to move behind paywalls or logins? If we provide logins, will that keep the bots out or just create a small hurdle for them to automate the account creation process? What happens when we can t search for this kind of content via search engines?
    • Cutting deals. What if our movement content providers are forced to cut deals with the AI entrepreneurs to allow the paying scumbags to fund the content creation. Eww. Enough said.
    • Bot detection. Maybe we just need to get better at bot detection? This will surely be an arms race, but would have some good benefits. Bots have also been filling out our forms and populating our databases with spam, testing credit cards against our donation pages, conducting denial of service attacks and all kinds of other irritating acts of vandalism. If we were better at stopping bots automatically it would have a lot of benefits. But what impact would it have on our web sites and the experience of using them? What about good bots (RSS feed readers, payment processors, web hooks, uptime detectors)? Will we cut the legs off any developer trying to automate something?
    I m not really sure where this is going, but it seems that the world wide web is about to head in a new direction.

    18 March 2025

    Matthew Garrett: Failing upwards: the Twitter encrypted DM failure

    Almost two years ago, Twitter launched encrypted direct messages. I wrote about their technical implementation at the time, and to the best of my knowledge nothing has changed. The short story is that the actual encryption primitives used are entirely normal and fine - messages are encrypted using AES, and the AES keys are exchanged via NIST P-256 elliptic curve asymmetric keys. The asymmetric keys are each associated with a specific device or browser owned by a user, so when you send a message to someone you encrypt the AES key with all of their asymmetric keys and then each device or browser can decrypt the message again. As long as the keys are managed appropriately, this is infeasible to break.

    But how do you know what a user's keys are? I also wrote about this last year - key distribution is a hard problem. In the Twitter DM case, you ask Twitter's server, and if Twitter wants to intercept your messages they replace your key. The documentation for the feature basically admits this - if people with guns showed up there, they could very much compromise the protection in such a way that all future messages you sent were readable. It's also impossible to prove that they're not already doing this without every user verifying that the public keys Twitter hands out to other users correspond to the private keys they hold, something that Twitter provides no mechanism to do.

    This isn't the only weakness in the implementation. Twitter may not be able read the messages, but every encrypted DM is sent through exactly the same infrastructure as the unencrypted ones, so Twitter can see the time a message was sent, who it was sent to, and roughly how big it was. And because pictures and other attachments in Twitter DMs aren't sent in-line but are instead replaced with links, the implementation would encrypt the links but not the attachments - this is "solved" by simply blocking attachments in encrypted DMs. There's no forward secrecy - if a key is compromised it allows access to not only all new messages created with that key, but also all previous messages. If you log out of Twitter the keys are still stored by the browser, so if you can potentially be extracted and used to decrypt your communications. And there's no group chat support at all, which is more a functional restriction than a conceptual one.

    To be fair, these are hard problems to solve! Signal solves all of them, but Signal is the product of a large number of highly skilled experts in cryptography, and even so it's taken years to achieve all of this. When Elon announced the launch of encrypted DMs he indicated that new features would be developed quickly - he's since publicly mentioned the feature a grand total of once, in which he mentioned further feature development that just didn't happen. None of the limitations mentioned in the documentation have been addressed in the 22 months since the feature was launched.

    Why? Well, it turns out that the feature was developed by a total of two engineers, neither of whom is still employed at Twitter. The tech lead for the feature was Christopher Stanley, who was actually a SpaceX employee at the time. Since then he's ended up at DOGE, where he apparently set off alarms when attempting to install Starlink, and who today is apparently being appointed to the board of Fannie Mae, a government-backed mortgage company.

    Anyway. Use Signal.

    comment count unavailable comments

    Christian Kastner: 15th Anniversary of My First Debian Upload

    Time flies! 15 years ago, on 2010-03-18, my first upload to the Debian archive was accepted. Debian had replaced Windows as my primary OS in 2005, but it was only when I saw that package zd1211-firmware had been orphaned that I thought of becoming a contributor. I owned a Zyxel G-202 USB WiFi fob that needed said firmware, and as is so often is with open-source software, I was going to scratch my own itch. Bart Martens thankfully helped me adopt the package, and sponsored my upload. I then joined Javier Fern ndez-Sanguino Pe a as a cron maintainer and upstream, and also worked within the Debian Python Applications, Debian Python Modules, and Debian Science Teams, where Jakub Wilk and Yaroslav Halchenko were kind enough to mentor me and eventually support my application to become a Debian Maintainer. Life intervened, and I was mostly inactive in Debian for the next two years. Upon my return in 2014, I had Vincent Cheng to thank for sponsoring most of my newer work, and for eventually supporting my application to become a Debian Developer. It was around that time that I also attended my first DebConf, in Portland, which remains one of my fondest memories. I had never been to an open-source software conference before, and DebConf14 really knocked it out of the park in so many ways. After another break, I returned in 2019 to work mostly on Python and machine learning libraries. In 2020, I finally completed a process that I had first started in 2012 but had never managed to finish before: converting cron from source format 1.0 (one big diff) to source format 3.0 (quilt) (a series of patches). This was a process where I converted 25 years worth of organic growth into a minimal series of logically grouped changes (more here). This was my white whale. In early 2023, shortly after the launch of ChatGPT which triggered an unprecedented AI boom, I started contributing to the Debian ROCm Team, where over the following year, I bootstrapped our CI at ci.rocm.debian.net. Debian's current tooling lack a way to express dependencies on specific hardware other than CPU ISA, nor does it have the means to run autopkgtests using such hardware. To get autopkgtests to make use of AMD GPUs in QEMU VMs and in containers, I had to fork autopkgtest, debci, and a few other components, as well as create a fair share of new tooling for ourselves. This worked out pretty well, and the CI has grown to support 17 different AMD GPU architectures. I will share more on this in upcoming posts. I have mentioned a few contributors by name, but I have countless others to thank for collaborations over the years. It has been a wonderful experience, and I look forward to many years more.

    Dirk Eddelbuettel: RcppArmadillo 14.4.0-1 on CRAN: New Upstream

    armadillo image Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language and is widely used by (currently) 1234 other packages on CRAN, downloaded 38.8 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 617 times according to Google Scholar. Conrad released a new minor version 14.4.0 last month. That was preceding by several extensive rounds of reverse-dependency checks covering the 1200+ packages at CRAN. We eventually narrowed the impact down to just eight packages, and I opened issue #462 to manage the transition along with GitHub-only release 14.4.0-0 of RcppArmadillo. Several maintainers responded very promptly and updated within days this is truly appreciated. Yesterday the last package updated at CRAN coinciding nicely with our planned / intended upload to CRAN one month after the release. So this new release, at version -1, is now on CRAN. It brings the usual number of small improvements to Armadillo itself as well as updates to packaging. The changes since the last CRAN release are summarised below.

    Changes in RcppArmadillo version 14.4.0-1 (2025-03-17)
    • CRAN release having given a few packages time to catch-up to small upstream change as discussed and managed in #462
    • Updated bibliography, and small edits to sparse matrix vignette
    • Switched continuous integration action to r-ci with implicit bootstrap

    Changes in RcppArmadillo version 14.4.0-0 (2025-02-17) (GitHub Only)
    • Upgraded to Armadillo release 14.4.0 (Filtered Espresso)
      • Faster handling of pow() and square() within accu() and sum() expressions
      • Faster sort() and sort_index() for complex matrices
      • Expanded the field class with .reshape() and .resize() member functions
      • More efficient handling of compound expressions by sum(), reshape(), trans()
      • Better detection of vector expressions by pow(), imag(), conj()
    • The package generator helper function now supports additional DESCRIPTIONs
    • This release revealed a need for very minor changes for a handful reverse-dependency packages which will be organized via GitHub issue tracking

    Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

    This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.

    Sergio Talens-Oliag: Using actions to build this site

    As promised on my previous post, on this entry I ll explain how I ve set up forgejo actions on the source repository of this site to build it using a runner instead of doing it on the public server using a webhook to trigger the operation.

    Setting up the systemThe first thing I ve done is to disable the forgejo webhook call that was used to publish the site, as I don t want to run it anymore. After that I added a new workflow to the repository that does the following things:
    • build the site using my hugo-adoc image.
    • push the result to a branch that contains the generated site (we do this because the server is already configured to work with the git repository and we can use force pushes to keep only the last version of the site, removing the need of extra code to manage package uploads and removals).
    • uses curl to send a notification to an instance of the webhook server installed on the remote server that triggers a script that updates the site using the git branch.

    Setting up the webhook serviceOn the server machine we have installed and configured the webhook service to run a script that updates the site. To install the application and setup the configuration we have used the following script:
    #!/bin/sh
    set -e
    # ---------
    # VARIABLES
    # ---------
    ARCH="$(dpkg --print-architecture)"
    WEBHOOK_VERSION="2.8.2"
    DOWNLOAD_URL="https://github.com/adnanh/webhook/releases/download"
    WEBHOOK_TGZ_URL="$DOWNLOAD_URL/$WEBHOOK_VERSION/webhook-linux-$ARCH.tar.gz"
    WEBHOOK_SERVICE_NAME="webhook"
    # Files
    WEBHOOK_SERVICE_FILE="/etc/systemd/system/$WEBHOOK_SERVICE_NAME.service"
    WEBHOOK_SOCKET_FILE="/etc/systemd/system/$WEBHOOK_SERVICE_NAME.socket"
    WEBHOOK_TML_TEMPLATE="/srv/blogops/action/webhook.yml.envsubst"
    WEBHOOK_YML="/etc/webhook.yml"
    # Config file values
    WEBHOOK_USER="$(id -u)"
    WEBHOOK_GROUP="$(id -g)"
    WEBHOOK_LISTEN_STREAM="172.31.31.1:4444"
    # ----
    # MAIN
    # ----
    # Install binary from releases (on Debian only version 2.8.0 is available, but
    # I need the 2.8.2 version to support the systemd activation mode).
    curl -fsSL -o "/tmp/webhook.tgz" "$WEBHOOK_TGZ_URL"
    tar -C /tmp -xzf /tmp/webhook.tgz
    sudo install -m 755 "/tmp/webhook-linux-$ARCH/webhook" /usr/local/bin/webhook
    rm -rf "/tmp/webhook-linux-$ARCH" /tmp/webhook.tgz
    # Service file
    sudo sh -c "cat >'$WEBHOOK_SERVICE_FILE'" <<EOF
    [Unit]
    Description=Webhook server
    [Service]
    Type=exec
    ExecStart=webhook -nopanic -hooks $WEBHOOK_YML
    User=$WEBHOOK_USER
    Group=$WEBHOOK_GROUP
    EOF
    # Socket config
    sudo sh -c "cat >'$WEBHOOK_SOCKET_FILE'" <<EOF
    [Unit]
    Description=Webhook server socket
    [Socket]
    # Set FreeBind to listen on missing addresses (the VPN can be down sometimes)
    FreeBind=true
    # Set ListenStream to the IP and port you want to listen on
    ListenStream=$WEBHOOK_LISTEN_STREAM
    [Install]
    WantedBy=multi-user.target
    EOF
    # Config file
    BLOGOPS_TOKEN="$(uuid)" \
      envsubst <"$WEBHOOK_TML_TEMPLATE"   sudo sh -c "cat >$WEBHOOK_YML"
    chmod 0640 "$WEBHOOK_YML"
    chwon "$WEBHOOK_USER:$WEBHOOK_GROUP" "$WEBHOOK_YML"
    # Restart and enable service
    sudo systemctl daemon-reload
    sudo systemctl stop "$WEBHOOK_SERVICE_NAME.socket"
    sudo systemctl start "$WEBHOOK_SERVICE_NAME.socket"
    sudo systemctl enable "$WEBHOOK_SERVICE_NAME.socket"
    # ----
    # vim: ts=2:sw=2:et:ai:sts=2
    As seen on the code, we ve installed the application using a binary from the project repository instead of a package because we needed the latest version of the application to use systemd with socket activation. The configuration file template is the following one:
    - id: "update-blogops"
      execute-command: "/srv/blogops/action/bin/update-blogops.sh"
      command-working-directory: "/srv/blogops"
      trigger-rule:
        match:
          type: "value"
          value: "$BLOGOPS_TOKEN"
          parameter:
            source: "header"
            name: "X-Blogops-Token"
    The version on /etc/webhook.yml has the BLOGOPS_TOKEN adjusted to a random value that has to exported as a secret on the forgejo project (see later). Once the service is started each time the action is executed the webhook daemon will get a notification and will run the following update-blogops.sh script to publish the updated version of the site:
    #!/bin/sh
    set -e
    # ---------
    # VARIABLES
    # ---------
    # Values
    REPO_URL="ssh://git@forgejo.mixinet.net/mixinet/blogops.git"
    REPO_BRANCH="html"
    REPO_DIR="public"
    MAIL_PREFIX="[BLOGOPS-UPDATE-ACTION] "
    # Address that gets all messages, leave it empty if not wanted
    MAIL_TO_ADDR="blogops@mixinet.net"
    # Directories
    BASE_DIR="/srv/blogops"
    PUBLIC_DIR="$BASE_DIR/$REPO_DIR"
    NGINX_BASE_DIR="$BASE_DIR/nginx"
    PUBLIC_HTML_DIR="$NGINX_BASE_DIR/public_html"
    ACTION_BASE_DIR="$BASE_DIR/action"
    ACTION_LOG_DIR="$ACTION_BASE_DIR/log"
    # Files
    OUTPUT_BASENAME="$(date +%Y%m%d-%H%M%S.%N)"
    ACTION_LOGFILE_PATH="$ACTION_LOG_DIR/$OUTPUT_BASENAME.log"
    # ---------
    # Functions
    # ---------
    action_log()  
      echo "$(date -R) $*" >>"$ACTION_LOGFILE_PATH"
     
    action_check_directories()  
      for _d in "$ACTION_BASE_DIR" "$ACTION_LOG_DIR"; do
        [ -d "$_d" ]   mkdir "$_d"
      done
     
    action_clean_directories()  
      # Try to remove empty dirs
      for _d in "$ACTION_LOG_DIR" "$ACTION_BASE_DIR"; do
        if [ -d "$_d" ]; then
          rmdir "$_d" 2>/dev/null   true
        fi
      done
     
    mail_success()  
      to_addr="$MAIL_TO_ADDR"
      if [ "$to_addr" ]; then
        subject="OK - updated blogops site"
        mail -s "$ MAIL_PREFIX $ subject " "$to_addr" <"$ACTION_LOGFILE_PATH"
      fi
     
    mail_failure()  
      to_addr="$MAIL_TO_ADDR"
      if [ "$to_addr" ]; then
        subject="KO - failed to update blogops site"
        mail -s "$ MAIL_PREFIX $ subject " "$to_addr" <"$ACTION_LOGFILE_PATH"
      fi
      exit 1
     
    # ----
    # MAIN
    # ----
    ret="0"
    # Check directories
    action_check_directories
    # Go to the base directory
    cd "$BASE_DIR"
    # Remove the old build dir if present
    if [ -d "$PUBLIC_DIR" ]; then
      rm -rf "$PUBLIC_DIR"
    fi
    # Update the repository checkout
    action_log "Updating the repository checkout"
    git fetch --all >>"$ACTION_LOGFILE_PATH" 2>&1   ret="$?"
    if [ "$ret" -ne "0" ]; then
      action_log "Failed to update the repository checkout"
      mail_failure
    fi
    # Get it from the repo branch & extract it
    action_log "Downloading and extracting last site version using 'git archive'"
    git archive --remote="$REPO_URL" "$REPO_BRANCH" "$REPO_DIR" \
        tar xf - >>"$ACTION_LOGFILE_PATH" 2>&1   ret="$?"
    # Fail if public dir was missing
    if [ "$ret" -ne "0" ]   [ ! -d "$PUBLIC_DIR" ]; then
      action_log "Failed to download or extract site"
      mail_failure
    fi
    # Remove old public_html copies
    action_log 'Removing old site versions, if present'
    find $NGINX_BASE_DIR -mindepth 1 -maxdepth 1 -name 'public_html-*' -type d \
      -exec rm -rf   \; >>"$ACTION_LOGFILE_PATH" 2>&1   ret="$?"
    if [ "$ret" -ne "0" ]; then
      action_log "Removal of old site versions failed"
      mail_failure
    fi
    # Switch site directory
    TS="$(date +%Y%m%d-%H%M%S)"
    if [ -d "$PUBLIC_HTML_DIR" ]; then
      action_log "Moving '$PUBLIC_HTML_DIR' to '$PUBLIC_HTML_DIR-$TS'"
      mv "$PUBLIC_HTML_DIR" "$PUBLIC_HTML_DIR-$TS" >>"$ACTION_LOGFILE_PATH" 2>&1  
        ret="$?"
    fi
    if [ "$ret" -eq "0" ]; then
      action_log "Moving '$PUBLIC_DIR' to '$PUBLIC_HTML_DIR'"
      mv "$PUBLIC_DIR" "$PUBLIC_HTML_DIR" >>"$ACTION_LOGFILE_PATH" 2>&1  
        ret="$?"
    fi
    if [ "$ret" -ne "0" ]; then
      action_log "Site switch failed"
      mail_failure
    else
      action_log "Site updated successfully"
      mail_success
    fi
    # ----
    # vim: ts=2:sw=2:et:ai:sts=2

    The hugo-adoc workflowThe workflow is defined in the .forgejo/workflows/hugo-adoc.yml file and looks like this:
    name: hugo-adoc
    # Run this job on push events to the main branch
    on:
      push:
        branches:
          - 'main'
    jobs:
      build-and-push:
        if: $  vars.BLOGOPS_WEBHOOK_URL != '' && secrets.BLOGOPS_TOKEN != ''  
        runs-on: docker
        container:
          image: forgejo.mixinet.net/oci/hugo-adoc:latest
        # Allow the job to write to the repository (not really needed on forgejo)
        permissions:
          contents: write
        steps:
          - name: Checkout the repo
            uses: actions/checkout@v4
            with:
              submodules: 'true'
          - name: Build the site
            shell: sh
            run:  
              rm -rf public
              hugo
          - name: Push compiled site to html branch
            shell: sh
            run:  
              # Set the git user
              git config --global user.email "blogops@mixinet.net"
              git config --global user.name "BlogOps"
              # Create a new orphan branch called html (it was not pulled by the
              # checkout step)
              git switch --orphan html
              # Add the public directory to the branch
              git add public
              # Commit the changes
              git commit --quiet -m "Updated site @ $(date -R)" public
              # Push the changes to the html branch
              git push origin html --force
              # Switch back to the main branch
              git switch main
          - name: Call the blogops update webhook endpoint
            shell: sh
            run:  
              HEADER="X-Blogops-Token: $  secrets.BLOGOPS_TOKEN  "
              curl --fail -k -H "$HEADER" $  vars.BLOGOPS_WEBHOOK_URL  
    The only relevant thing is that we have to add the BLOGOPS_TOKEN variable to the project secrets (its value is the one included on the /etc/webhook.yml file created when installing the webhook service) and the BLOGOPS_WEBHOOK_URL project variable (its value is the URL of the webhook server, in my case http://172.31.31.1:4444/hooks/update-blogops); note that the job includes the -k flag on the curl command just in case I end up using TLS on the webhook server in the future, as discussed previously.

    ConclusionNow that I have forgejo actions on my server I no longer need to build the site on the public server as I did initially, a good thing when the server is a small OVH VPS that only runs a couple of containers and a web server directly on the host. I m still using a notification system to make the server run a script to update the site because that way the forgejo server does not need access to the remote machine shell, only the webhook server which, IMHO, is a more secure setup.

    17 March 2025

    Sergio Talens-Oliag: Configuring forgejo actions

    Last week I decided I wanted to try out forgejo actions to build this blog instead of using webhooks, so I looked the documentation and started playing with it until I had it working as I wanted. This post is to describe how I ve installed and configured a forgejo runner, how I ve added an oci organization to my instance to build, publish and mirror container images and added a couple of additional organizations (actions and docker for now) to mirror interesting actions. The changes made to build the site using actions will be documented on a separate post, as I ll be using this entry to test the new setup on the blog project.

    Installing the runnerThe first thing I ve done is to install a runner on my server, I decided to use the OCI image installation method, as it seemed to be the easiest and fastest one. The commands I ve used to setup the runner are the following:
    $ cd /srv
    $ git clone https://forgejo.mixinet.net/blogops/forgejo-runner.git
    $ cd forgejo-runner
    $ sh ./bin/setup-runner.sh
    The setup-runner.sh script does multiple things:
    • create a forgejo-runner user and group
    • create the necessary directories for the runner
    • create a .runner file with a predefined secret and the docker label
    The setup-runner.sh code is available here. After running the script the runner has to be registered with the forgejo server, it can be done using the following command:
    $ forgejo forgejo-cli actions register --name "$RUNNER_NAME" \
        --secret "$FORGEJO_SECRET"
    The RUNNER_NAME variable is defined on the setup-runner.sh script and the FORGEJO_SECRET must match the value used on the .runner file.

    Starting it with docker-composeTo launch the runner I m going to use a docker-compose.yml file that starts two containers, a docker in docker service to run the containers used by the workflow jobs and another one that runs the forgejo-runner itself. The initial version used a TCP port to communicate with the dockerd server from the runner, but when I tried to build images from a workflow I noticed that the containers launched by the runner were not going to be able to execute another dockerd inside the dind one and, even if they were, it was going to be expensive computationally. To avoid the issue I modified the dind service to use a unix socket on a shared volume that can be used by the runner service to communicate with the daemon and also re-shared with the job containers so the dockerd server can be used from them to build images.
    Warning: The use of the same docker server that runs the jobs from them has security implications, but this instance is for a home server where I am the only user, so I am not worried about it and this way I can save some resources (in fact, I could use the host docker server directly instead of using a dind service, but just in case I want to run other containers on the host I prefer to keep the one used for the runner isolated from it). For those concerned about sharing the same server an alternative would be to launch a second dockerd only for the jobs (i.e. actions-dind) using the same approach (the volume with its socket will have to be shared with the runner service so it can be re-shared, but the runner does not need to use it).
    The final docker-compose.yaml file is as follows:
    services:
      dind:
        image: docker:dind
        container_name: 'dind'
        privileged: 'true'
        command: ['dockerd', '-H', 'unix:///dind/docker.sock', '-G', '$RUNNER_GID']
        restart: 'unless-stopped'
        volumes:
          - ./dind:/dind
      runner:
        image: 'data.forgejo.org/forgejo/runner:6.2.2'
        links:
          - dind
        depends_on:
          dind:
            condition: service_started
        container_name: 'runner'
        environment:
          DOCKER_HOST: 'unix:///dind/docker.sock'
        user: $RUNNER_UID:$RUNNER_GID
        volumes:
          - ./config.yaml:/config.yaml
          - ./data:/data
          - ./dind:/dind
        restart: 'unless-stopped'
        command: '/bin/sh -c "sleep 5; forgejo-runner daemon -c /config.yaml"'
    There are multiple things to comment about this file:
    1. The dockerd server is started with the -H unix:///dind/docker.sock flag to use the unix socket to communicate with the daemon instead of using a TCP port (as said, it is faster and allows us to share the socket with the containers started by the runner).
    2. We are running the dockerd daemon with the RUNNER_GID group so the runner can communicate with it (the socket gets that group which is the same used by the runner).
    3. The runner container mounts three volumes: the data directory, the dind folder where docker creates the unix socket and a config.yaml file used by us to change the default runner configuration.
    The config.yaml file was originally created using the forgejo-runner:
    $ docker run --rm data.forgejo.org/forgejo/runner:6.2.2 \
        forgejo-runner generate-config > config.yaml
    The changes to it are minimal, the runner capacity has been increased to 2 (that allows it to run two jobs at the same time) and the /dind/docker.sock value has been added to the valid_volumes key to allow the containers launched by the runner to mount it when needed; the diff against the default version is as follows:
    @@ -13,7 +13,8 @@
       # Where to store the registration result.
       file: .runner
       # Execute how many tasks concurrently at the same time.
    -  capacity: 1
    +  # STO: Allow 2 concurrent tasks
    +  capacity: 2
       # Extra environment variables to run jobs.
       envs:
         A_TEST_ENV_NAME_1: a_test_env_value_1
    @@ -87,7 +88,9 @@
       # If you want to allow any volume, please use the following configuration:
       # valid_volumes:
       #   - '**'
    -  valid_volumes: []
    +  # STO: Allow to mount the /dind/docker.sock on the containers
    +  valid_volumes:
    +    - /dind/docker.sock
       # overrides the docker client host with the specified one.
       # If "-" or "", an available docker host will automatically be found.
       # If "automount", an available docker host will automatically be found and ...
    To start the runner we export the RUNNER_UID and RUNNER_GID variables and call docker-compose up to start the containers on the background:
    $ RUNNER_UID="$(id -u forgejo-runner)" RUNNER_GID="$(id -g forgejo-runner)" \
        docker compose up -d
    If the server was configured right we are now able to start using actions with this runner.

    Preparing the system to run things locallyTo avoid unnecessary network traffic we are going to create a multiple organizations in our forgejo instance to maintain our own actions and container images and mirror remote ones. The rationale behind the mirror use is that we reduce a lot the need to connect to remote servers to download the actions and images, which is good for performance and security reasons. In fact, we are going to build our own images for some things to install the tools we want without needing to do it over and over again on the workflow jobs.

    Mirrored actionsThe actions we are mirroring are on the actions and docker organizations, we have created the following ones for now (the mirrors were created using the forgejo web interface and we have disabled manually all the forgejo modules except the code one for them):
    To use our actions by default (i.e., without needing to add the server URL on the uses keyword) we have added the following section to the app.ini file of our forgejo server:
    [actions]
    ENABLED = true
    DEFAULT_ACTIONS_URL = https://forgejo.mixinet.net

    Setting up credentials to push imagesTo be able to push images to the oci organization I ve created a token with package:write permission for my own user because I m a member of the organization and I m authorized to publish packages on it (a different user could be created, but as I said this is for personal use, so there is no need to complicate things for now). To allow the use of those credentials on the actions I have added a secret (REGISTRY_PASS) and a variable (REGISTRY_USER) to the oci organization to allow the actions to use them. I ve also logged myself on my local docker client to be able to push images to the oci group by hand, as I it is needed for bootstrapping the system (as I m using local images on the worflows I need to push them to the server before running the ones that are used to build the images).

    Local and mirrored imagesOur images will be stored on the packages section of a new organization called oci, inside it we have created two projects that use forgejo actions to keep things in shape:
    • images: contains the source files used to generate our own images and the actions to build, tag and push them to the oci organization group.
    • mirrors: contains a configuration file for the regsync tool to mirror containers and an action to run it.
    On the next sections we are going to describe the actions and images we have created and mirrored from those projects.

    The oci/images projectThe images project is a monorepo that contains the source files for the images we are going to build and a couple of actions. The image sources are on sub directories of the repository, to be considered an image the folder has to contain a Dockerfile that will be used to build the image. The repository has two workflows:
    • build-image-from-tag: Workflow to build, tag and push an image to the oci organization
    • multi-semantic-release: Workflow to create tags for the images using the multi-semantic-release tool.
    As the workflows are already configured to use some of our images we pushed some of them from a checkout of the repository using the following commands:
    registry="forgejo.mixinet.net/oci"
    for img in alpine-mixinet node-mixinet multi-semantic-release; do
      docker build -t $registry/$img:1.0.0 $img
      docker tag $registry/$img:1.0.0 $registry/$img:latest
      docker push $registry/$img:1.0.0
      docker push $registry/$img:latest
    done
    On the next sub sections we will describe what the workflows do and will show their source code.

    build-image-from-tag workflowThis workflow uses a docker client to build an image from a tag on the repository with the format image-name-v[0-9].[0-9].[0-9]+. As the runner is executed on a container (instead of using lxc) it seemed unreasonable to run another dind container from that one, that is why, after some tests, I decided to share the dind service server socket with the runner container and enabled the option to mount it also on the containers launched by the runner when needed (I only do it on the build-image-from-tag action for now). The action was configured to run using a trigger or when new tags with the right format were created, but when the tag is created by multi-semantic-release the trigger does not work for some reason, so now it only runs the job on triggers and checks if it is launched for a tag with the right format on the job itself. The source code of the action is as follows:
    name: build-image-from-tag
    on:
      workflow_dispatch:
    jobs:
      build:
        # Don't build the image if the registry credentials are not set, the ref is not a tag or it doesn't contain '-v'
        if: $  vars.REGISTRY_USER != '' && secrets.REGISTRY_PASS != '' && startsWith(github.ref, 'refs/tags/') && contains(github.ref, '-v')  
        runs-on: docker
        container:
          image: forgejo.mixinet.net/oci/node-mixinet:latest
          # Mount the dind socket on the container at the default location
          options: -v /dind/docker.sock:/var/run/docker.sock
        steps:
          - name: Extract image name and tag from git and get registry name from env
            id: job_data
            run:  
              echo "::set-output name=img_name::$ GITHUB_REF_NAME%%-v* "
              echo "::set-output name=img_tag::$ GITHUB_REF_NAME##*-v "
              echo "::set-output name=registry::$(
                echo "$  github.server_url  "   sed -e 's%https://%%'
              )"
              echo "::set-output name=oci_registry_prefix::$(
                echo "$  github.server_url  /oci"   sed -e 's%https://%%'
              )"
          - name: Checkout the repo
            uses: actions/checkout@v4
          - name: Export build dir and Dockerfile
            id: build_data
            run:  
              img="$  steps.job_data.outputs.img_name  "
              build_dir="$(pwd)/$ img "
              dockerfile="$ build_dir /Dockerfile"
              if [ -f "$dockerfile" ]; then
                echo "::set-output name=build_dir::$build_dir"
                echo "::set-output name=dockerfile::$dockerfile"
              else
                echo "Couldn't find the Dockerfile for the '$img' image"
                exit 1
              fi
          - name: Login to the Container Registry
            uses: docker/login-action@v3
            with:
              registry: $  steps.job_data.outputs.registry  
              username: $  vars.REGISTRY_USER  
              password: $  secrets.REGISTRY_PASS  
          - name: Set up Docker Buildx
            uses: docker/setup-buildx-action@v3
          - name: Build and Push
            uses: docker/build-push-action@v6
            with:
              push: true
              tags:  
                $  steps.job_data.outputs.oci_registry_prefix  /$  steps.job_data.outputs.img_name  :$  steps.job_data.outputs.img_tag  
                $  steps.job_data.outputs.oci_registry_prefix  /$  steps.job_data.outputs.img_name  :latest
              context: $  steps.build_data.outputs.build_dir  
              file: $  steps.build_data.outputs.dockerfile  
              build-args:  
                OCI_REGISTRY_PREFIX=$  steps.job_data.outputs.oci_registry_prefix  /
    Some notes about this code:
    1. The if condition of the build job is not perfect, but it is good enough to avoid wrong uses as long as nobody uses manual tags with the wrong format and expects things to work (it checks if the REGISTRY_USER and REGISTRY_PASS variables are set, if the ref is a tag and if it contains the -v string).
    2. To be able to access the dind socket we mount it on the container using the options key on the container section of the job (this only works if supported by the runner configuration as explained before).
    3. We use the job_data step to get information about the image from the tag and the registry URL from the environment variables, it is executed first because all the information is available without checking out the repository.
    4. We use the job_data step to get the build dir and Dockerfile paths from the repository (right now we are assuming fixed paths and checking if the Dockerfile exists, but in the future we could use a configuration file to get them, if needed).
    5. As we are using a docker daemon that is already running there is no need to use the docker/setup-docker-action to install it.
    6. On the build and push step we pass the OCI_REGISTRY_PREFIX build argument to the Dockerfile to be able to use it on the FROM instruction (we are using it in our images).

    multi-semantic-release workflowThis workflow is used to run the multi-semantic-release tool on pushes to the main branch. It is configured to create the configuration files on the fly (it prepares things to tag the folders that contain a Dockerfile using a couple of template files available on the repository s .forgejo directory) and run the multi-semantic-release tool to create tags and push them to the repository if new versions are to be built. Initially we assumed that the tag creation pushed by multi-semantic-release would be enough to run the build-tagged-image-task action, but as it didn t work we removed the rule to run the action on tag creation and added code to trigger the action using an api call for the newly created tags (we get them from the output of the multi-semantic-release execution). The source code of the action is as follows:
    name: multi-semantic-release
    on:
      push:
        branches:
          - 'main'
    jobs:
      multi-semantic-release:
        runs-on: docker
        container:
          image: forgejo.mixinet.net/oci/multi-semantic-release:latest
        steps:
          - name: Checkout the repo
            uses: actions/checkout@v4
          - name: Generate multi-semantic-release configuration
            shell: sh
            run:  
              # Get the list of images to work with (the folders that have a Dockerfile)
              images="$(for img in */Dockerfile; do dirname "$img"; done)"
              # Generate a values.yaml file for the main packages.json file
              package_json_values_yaml=".package.json-values.yaml"
              echo "images:" >"$package_json_values_yaml"
              for img in $images; do
                echo " - $img" >>"$package_json_values_yaml"
              done
              echo "::group::Generated values.yaml for the project"
              cat "$package_json_values_yaml"
              echo "::endgroup::"
              # Generate the package.json file validating that is a good json file with jq
              tmpl -f "$package_json_values_yaml" ".forgejo/package.json.tmpl"   jq . > "package.json"
              echo "::group::Generated package.json for the project"
              cat "package.json"
              echo "::endgroup::"
              # Remove the temporary values file
              rm -f "$package_json_values_yaml"
              # Generate the package.json file for each image
              for img in $images; do
                tmpl -v "img_name=$img" -v "img_path=$img" ".forgejo/ws-package.json.tmpl"   jq . > "$img/package.json"
                echo "::group::Generated package.json for the '$img' image"
                cat "$img/package.json"
                echo "::endgroup::"
              done
          - name: Run multi-semantic-release
            shell: sh
            run:  
              multi-semantic-release   tee .multi-semantic-release.log
          - name: Trigger builds
            shell: sh
            run:  
              # Get the list of tags published on the previous steps
              tags="$(
                sed -n -e 's/^\[.*\] \[\(.*\)\] .* Published release \([0-9]\+\.[0-9]\+\.[0-9]\+\) on .*$/\1-v\2/p' \
                  .multi-semantic-release.log
              )"
              rm -f .multi-semantic-release.log
              if [ "$tags" ]; then
                # Prepare the url for building the images
                workflow="build-image-from-tag.yaml"
                dispatch_url="$  github.api_url  /repos/$  github.repository  /actions/workflows/$workflow/dispatches"
                echo "$tags"   while read -r tag; do
                  echo "Triggering build for tag '$tag'"
                  curl \
                    -H "Content-Type:application/json" \
                    -H "Authorization: token $  secrets.GITHUB_TOKEN  " \
                    -d " \"ref\":\"$tag\" " "$dispatch_url"
                done
              fi
    Notes about this code:
    1. The use of the tmpl tool to process the multi-semantic-release configuration templates comes from previous uses, but on this case we could use a different approach (i.e. envsubst could be used) but we left it because it keeps things simple and can be useful in the future if we want to do more complex things with the template files.
    2. We use tee to show and dump to a file the output of the multi-semantic-release execution.
    3. We get the list of pushed tags using sed against the output of the multi-semantic-release execution and for each one found we use curl to call the forgejo API to trigger the build job; as the call is against the same project we can use the GITHUB_TOKEN generated for the workflow to do it, without creating a user token that has to be shared as a secret.
    The .forgejo/package.json.tmpl file is the following one:
     
      "name": "multi-semantic-release",
      "version": "0.0.0-semantically-released",
      "private": true,
      "multi-release":  
        "tagFormat": "$ name -v$ version "
       ,
      "workspaces":   .images   toJson  
     
    As can be seen it only needs a list of paths to the images as argument (the file we generate contains the names and paths, but it could be simplified). And the .forgejo/ws-package.json.tmpl file is the following one:
     
      "name": "  .img_name  ",
      "license": "UNLICENSED",
      "release":  
        "plugins": [
          [
            "@semantic-release/commit-analyzer",
             
              "preset": "conventionalcommits",
              "releaseRules": [
                  "breaking": true, "release": "major"  ,
                  "revert": true, "release": "patch"  ,
                  "type": "feat", "release": "minor"  ,
                  "type": "fix", "release": "patch"  ,
                  "type": "perf", "release": "patch"  
              ]
             
          ],
          [
            "semantic-release-replace-plugin",
             
              "replacements": [
                 
                  "files": [ "  .img_path  /msr.yaml" ],
                  "from": "^version:.*$",
                  "to": "version: $ nextRelease.version ",
                  "allowEmptyPaths": true
                 
              ]
             
          ],
          [
            "@semantic-release/git",
             
              "assets": [ "msr.yaml" ],
              "message": "ci(release):   .img_name  -v$ nextRelease.version \n\n$ nextRelease.notes "
             
          ]
        ],
        "branches": [ "main" ]
       
     

    The oci/mirrors projectThe repository contains a template for the configuration file we are going to use with regsync (regsync.envsubst.yml) to mirror images from remote registries using a workflow that generates a configuration file from the template and runs the tool. The initial version of the regsync.envsubst.yml file is prepared to mirror alpine containers from version 3.21 to 3.29 (we explicitly remove version 3.20) and needs the forgejo.mixinet.net/oci/node-mixinet:latest image to run (as explained before it was pushed manually to the server):
    version: 1
    creds:
      - registry: "$REGISTRY"
        user: "$REGISTRY_USER"
        pass: "$REGISTRY_PASS"
    sync:
      - source: alpine
        target: $REGISTRY/oci/alpine
        type: repository
        tags:
          allow:
            - "latest"
            - "3\\.2\\d+"
            - "3\\.2\\d+.\\d+"
          deny:
            - "3\\.20"
            - "3\\.20.\\d+"

    mirror workflowThe mirror workflow creates a configuration file replacing the value of the REGISTRY environment variable (computed by removing the protocol from the server_url), the REGISTRY_USER organization value and the REGISTRY_PASS secret using the envsubst command and running the regsync tool to mirror the images using the configuration file. The action is configured to run daily, on push events when the regsync.envsubst.yml file is modified on the main branch and can also be triggered manually. The source code of the action is as follows:
    .forgejo/workflows/mirror.yaml
    name: mirror
    on:
      schedule:
        - cron: '@daily'
      push:
        branches:
          - main
        paths:
          - 'regsync.envsubst.yml'
      workflow_dispatch:
    jobs:
      mirror:
        if: $  vars.REGISTRY_USER != '' && secrets.REGISTRY_PASS != ''  
        runs-on: docker
        container:
          image: forgejo.mixinet.net/oci/node-mixinet:latest
        steps:
          - name: Checkout
            uses: actions/checkout@v4
          - name: Sync images
            run:  
              REGISTRY="$(echo "$  github.server_url  "   sed -e 's%https://%%')" \
              REGISTRY_USER="$  vars.REGISTRY_USER  " \
              REGISTRY_PASS="$  secrets.REGISTRY_PASS  " \
                envsubst <regsync.envsubst.yml >.regsync.yml
              regsync --config .regsync.yml once
              rm -f .regsync.yml

    ConclusionWe have installed a forgejo-runner and configured it to run actions for our own server and things are working fine. This approach allows us to have a powerful CI/CD system on a modest home server, something very useful for maintaining personal projects and playing with things without needing SaaS platforms like github or gitlab.

    James Valleroy: What s New for FreedomBox in Debian 13 trixie

    FreedomBox is a Debian blend that makes it easier to run your own server. Approximately every two years, there is a new stable release of Debian. This year s release will be called Debian 13 "trixie". This post will provide an overview of changes between FreedomBox 23.6 (the version that shipped in Debian 12 "bookworm") and 25.5 (the latest release). Note: Debian 13 "trixie" is not yet released, so things may still change, be added or removed, before the official release. General Diagnostics Name Services Networks Privacy Users and Groups Deluge Ejabberd Feather Wiki GitWeb GNOME ikiwiki Kiwix Matrix Synapse MediaWiki MiniDLNA Miniflux Nextcloud OpenVPN Postfix/Dovecot Shadowsocks Server SOGo TiddlyWiki Tor Proxy Transmission Conclusion Over the past two years, FreedomBox has been increasing the number of features and applications available to its users. We have also focused on improving the reliability of the system, detecting unexpected situations, and providing means to return to a known good state. With these improvements, FreedomBox has become a good solution for people with limited time or energy to set up and start running a personal server, at home or in the cloud. Looking forward, we would like to focus on making more powerful hardware available with FreedomBox pre-installed and ready to be used. This hardware would also support larger storage devices, making it suitable as a NAS or media server. We are also very interested in exploring new features such as atomic updates, which will further enhance the reliability of the system.

    Next.

    Previous.