Taavi V n nen: Writing a custom rsync server to automatically update a static site
Inspired by some friends,1 I too wanted to make a tiny website telling which
event I am at this exact moment. Thankfully I already had an another toy project
with that information easily available, so generating the web page was a matter
of just querying that project's API and feeding that data to a HTML template.
Now the obvious way to host that would be to hook up the HTML-generating code
to a web server, maybe add some caching for the API calls, and then route
external HTTPS traffic to it. However, that'd a) require that server to be
constantly available to serve traffic, and b) be boring.
For context: I have an existing setup called
staticweb, which is effectively
a fancy name for a couple of Puppet-managed servers that run Apache httpd to
serve static web pages and have a bunch of systemd timers running rsync to
ensure they're serving the same content. It works really well and I use it for
things ranging from my website or whyisbetabroken.com to things like my
internal apt repository.
Now, there are two ways to get new content into that mechanism: it can be
manually pushed in from e.g. a CI job, or the system can be configured to
periodically pull it from a separate server. The latter mechanism was initially
created so that I could pull the Debian packages from my separate reprepro
server into the staticweb setup. It turns out that the latter makes a really
neat method for handling other dynamically-generated static sites as well.
So, for my "where is Taavi at" site, I ended up writing the server part in
Go, and included an rsync server using the gokrazy/rsync package. Initially I
just implemented a static temporary directory with a timer to regularly update
the HTML file in it, but then I got an even more cursed idea: what if the HTML
was dynamically generated when an rsync client connected to the server? So I
did just that.
For deployment, I slapped the entire server part in a container and deployed it
to my Kubernetes cluster. The rsync server is exposed directly as a service
to my internal network with no authentication or encryption - I think that's
fine since that's a read-only service in a private LAN and the resulting HTML
is going to be publicly exposed anyway. (Thanks to some DNS magic, just
creating a LoadBalancer Service object with a special annotation is enough to
have a DNS name provisioned for the assigned IP address, which is neat.)
Overall the setup works nice, at least for now. I need to add some sort of a
cache to not fetch unchanged information from the API since for every update.
And I guess I could write some cursed rsyncd reverse proxy with per-module
rules if I end up creating more sites like this to avoid creating new
LoadBalancer services for each of them.
- Mostly from Sammy's where.fops.at.
How long has it been since you last saw a conversation over different blogs
syndicated at the same planet? Well, it s one of the good memories of the
early 2010s. And there is an opportunity to re-engage!
I came across
I got a new laptop (a Lenovo Thinkpad X1 Carbon Gen 12, more on that later) and as always with new pets, it needed a name.
My naming scheme is roughly "short japanese words that somehow relate to the machine".
The current (other) machines at home are (not all really in use):
Most of my Debian contributions this month were
Another short status update of what happened on my side last
month. Some more ModemManager bits landed, 
Are you a student aspiring to participate in the
I am told, by friends who have spent time at Google, about the reason
Almost two years ago, Twitter launched encrypted direct messages. I wrote about their technical implementation
As promised on my previous