This post marks the beginning my yearly roundups of the favourite books and movies that I read and watched in 2022 that I plan to publish over the next few days. Just as I did for 2020 and 2021, I won't reveal precisely how many books I read in the last year. I didn't get through as many books as I did in 2021, though, but that's partly due to reading a significant number of long nineteenth-century novels in particular, a fair number of those books that American writer Henry James once referred to as "large, loose, baggy monsters." However, in today's post I'll be looking at my favourite books that are typically filed under fiction, with 'classic' fiction following tomorrow. Works that just missed the cut here include John O'Brien's Leaving Las Vegas, Colson Whitehead's Sag Harbor and possibly The Name of the Rose by Umberto Eco, or Elif Batuman's The Idiot. I also feel obliged to mention (or is that show off?) that I also read the 1,079-page Infinite Jest by David Foster Wallace, but I can't say it was a favourite, let alone recommend others unless they are in the market for a good-quality under-monitor stand.

Mona (2021) Pola Oloixarac Mona is the story of a young woman who has just been nominated for the 'most important literary award in Europe'. Mona sees the nomination as a chance to escape her substance abuse on a Californian campus and so speedily decamps to the small village in the depths of Sweden where the nominees must convene for a week before the overall winner is announced. Mona didn't disappear merely to avoid pharmacological misadventures, though, but also to avoid the growing realisation that she is being treated as something of an anthropological curiosity at her university: a female writer of colour treasured for her flourish of exotic diversity that reflects well upon her department. But Mona is now stuck in the company of her literary competitors who all have now gathered from around the world in order to do what writers do: harbour private resentments, exchange empty flattery, embody the selfsame racialised stereotypes that Mona left the United States to avoid, stab rivals in the back, drink too much, and, of course, go to bed together. But as I read Mona, I slowly started to realise that something else is going on. Why does Mona keep finding traces of violence on her body, the origins of which she cannot or refuses to remember? There is something eerily defensive about her behaviour and sardonic demeanour in general as well. A genre-bending and mind-expanding novel unfolded itself, and, without getting into spoiler territory, Mona concludes with such a surprising ending that, according to Adam Thirlwell:

Perhaps we need to rethink what is meant by a gimmick. If a gimmick is anything that we want to reject as extra or excessive or ill-fitting, then it may be important to ask what inhibitions or arbitrary conventions have made it seem like excess, and to revel in the exorbitant fictional constructions it produces. [...]

Mona is a savage satire of the literary world, but it's also a very disturbing exploration of trauma and violence. The success of the book comes in equal measure from the author's commitment to both ideas, but also from the way the psychological damage component creeps up on you. And, as implied above, the last ten pages are quite literally out of this world.

My Brilliant Friend (2011)
The Story of a New Name (2012)
Those Who Leave and Those Who Stay (2013)
The Story of the Lost Child (2014) Elena Ferrante Elena Ferrante's Neopolitan Quartet follows two girls, both brilliant in their own way. Our protagonist-narrator is Elena, a studious girl from the lower rungs of the middle class of Naples who is inspired to be more by her childhood friend, Lila. Lila is, in turn, far more restricted by her poverty and class, but can transcend it at times through her fiery nature, which also brands her as somewhat unique within their inward-looking community. The four books follow the two girls from the perspective of Elena as they grow up together in post-war Italy, where they drift in-and-out of each other's lives due to the vicissitudes of change and the consequences of choice. All the time this is unfolding, however, the narrative is very always slightly charged by the background knowledge revealed on the very first page that Lila will, many years later, disappear from Elena's life. Whilst the quartet has the formal properties of a bildungsroman, its subject and conception are almost entirely different. In particular, the books are driven far more by character and incident than spectacular adventures in picturesque Italy. In fact, quite the opposite takes place: these are four books where ordinary-seeming occurrences take on an unexpected radiance against a background of poverty, ignorance, violence and other threats, often bringing to mind the films of the Italian neorealism movement. Brilliantly rendered from beginning to end, Ferrante has a seemingly studious eye for interpreting interactions and the psychology of adolescence and friendship. Some utterances indeed, perhaps even some glances are dissected at length over multiple pages, something that Vittorio De Sica's classic Bicycle Thieves (1948) could never do. Potential readers should not take any notice of the saccharine cover illustrations on most editions of the books. The quartet could even win an award for the most misleading artwork, potentially rivalling even Vladimir Nabokov's Lolita. I wouldn't be at all surprised if it is revealed that the drippy illustrations and syrupy blurbs ("a rich, intense and generous-hearted story ") turn out to be part of a larger metatextual game that Ferrante is playing with her readers. This idiosyncratic view of mine is partially supported by the fact that each of the four books has been given a misleading title, the true ambiguity of which often only becomes clear as each of the four books comes into sharper focus. Readers of the quartet often fall into debating which is the best of the four. I've heard from more than one reader that one has 'too much Italian politics' and another doesn't have enough 'classic' Lina moments. The first book then possesses the twin advantages of both establishing the environs and finishing with a breathtaking ending that is both satisfying and a cliffhanger as well but does this make it 'the best'? I prefer to liken the quartet more like the different seasons of The Wire (2002-2008) where, personal favourites and preferences aside, although each season is undoubtedly unique, it would take a certain kind of narrow-minded view of art to make the claim that, say, series one of The Wire is 'the best' or that the season that focuses on the Baltimore docks 'is boring'. Not to sound like a neo-Wagnerian, but each of them adds to final result in its own. That is to say, both The Wire and the Neopolitan Quartet achieve the rare feat of making the magisterial simultaneously intimate.

Out There: Stories (2022) Kate Folk Out There is a riveting collection of disturbing short stories by first-time author Kate Fork. The title story first appeared in the New Yorker in early 2020 imagines a near-future setting where a group of uncannily handsome artificial men called 'blots' have arrived on the San Francisco dating scene with the secret mission of sleeping with women, before stealing their personal data from their laptops and phones and then (quite literally) evaporating into thin air. Folk's satirical style is not at all didactic, so it rarely feels like she is making her points in a pedantic manner. But it's clear that the narrator of Out There is recounting her frustration with online dating. in a way that will resonate with anyone who s spent time with dating apps or indeed the contemporary hyper-centralised platform-based internet in general. Part social satire, part ghost story and part comic tales, the blurring of the lines between these factors is only one of the things that makes these stories so compelling. But whilst Folk constructs crazy scenarios and intentionally strange worlds, she also manages to also populate them with characters that feel real and genuinely sympathetic. Indeed, I challenge you not to feel some empathy for the 'blot' in the companion story Big Sur which concludes the collection, and it complicates any primary-coloured view of the dating world of consisting entirely of predatory men. And all of this is leavened with a few stories that are just plain surreal. I don't know what the deal is with Dating a Somnambulist (available online on Hobart Pulp), but I know that I like it.

Solaris (1961) Stanislaw Lem When Kelvin arrives at the planet Solaris to study the strange ocean that covers its surface, instead of finding an entirely physical scientific phenomenon, he soon discovers a previously unconscious memory embodied in the physical manifestation of a long-dead lover. The other scientists on the space station slowly reveal that they are also plagued with their own repressed corporeal memories. Many theories are put forward as to why all this is occuring, including the idea that Solaris is a massive brain that creates these incarnate memories. Yet if that is the case, the planet's purpose in doing so is entirely unknown, forcing the scientists to shift focus and wonder whether they can truly understand the universe without first understanding what lies within their own minds and in their desires. This would be an interesting outline for any good science fiction book, but one of the great strengths of Solaris is not only that it withholds from the reader why the planet is doing anything it does, but the book is so forcefully didactic in its dislike of the hubris, destructiveness and colonial thinking that can accompany scientific exploration. In one of its most vitriolic passages, Lem's own anger might be reaching out to the reader:

We are humanitarian and chivalrous; we don t want to enslave other races, we simply want to bequeath them our values and take over their heritage in exchange. We think of ourselves as the Knights of the Holy Contact. This is another lie. We are only seeking Man. We have no need of other worlds. We need mirrors. We don t know what to do with other worlds. A single world, our own, suffices us; but we can t accept it for what it is. We are searching for an ideal image of our own world: we go in quest of a planet, of a civilisation superior to our own, but developed on the basis of a prototype of our primaeval past. At the same time, there is something inside us that we don t like to face up to, from which we try to protect ourselves, but which nevertheless remains since we don t leave Earth in a state of primal innocence. We arrive here as we are in reality, and when the page is turned, and that reality is revealed to us that part of our reality that we would prefer to pass over in silence then we don t like it anymore.

An overwhelming preoccupation with this idea infuses Solaris, and it turns out to be a common theme in a lot of Lem's work of this period, such as in his 1959 'anti-police procedural' The Investigation. Perhaps it not a dislike of exploration in general or the modern scientific method in particular, but rather a savage critique of the arrogance and self-assuredness that accompanies most forms of scientific positivism, or at least pursuits that cloak themselves under the guise of being a laudatory 'scientific' pursuit:

Man has gone out to explore other worlds and other civilizations without having explored his own labyrinth of dark passages and secret chambers and without finding what lies behind doorways that he himself has sealed.

I doubt I need to cite specific instances of contemporary scientific pursuits that might meet Lem's punishing eye today, and the fact that his critique works both in 2022 and 1961 perhaps tells us more about the human condition than we'd care to know. Another striking thing about Solaris isn't just the specific Star Trek and Stargate SG-1 episodes that I retrospectively realised were purloined from the book, but that almost the entire register of Star Trek: The Next Generation in particular seems to be rehearsed here. That is to say, TNG presents itself as hard and fact-based 'sci-fi' on the surface, but, at its core, there are often human, existential and sometimes quite enormously emotionally devastating human themes being discussed such as memory, loss and grief. To take one example from many, the painful memories that the planet Solaris physically materialises in effect asks us to seriously consider what it actually is taking place when we 'love' another person: is it merely another 'mirror' of ourselves? (And, if that is the case, is that... bad?) It would be ahistorical to claim that all popular science fiction today can be found rehearsed in Solaris, but perhaps it isn't too much of a stretch:

[Solaris] renders unnecessary any more alien stories. Nothing further can be said on this topic ...] Possibly, it can be said that when one feels the urge for such a thing, one should simply reread Solaris and learn its lessons again. Kim Stanley Robinson [...]

I could go on praising this book for quite some time; perhaps by discussing the extreme framing devices used within the book at one point, the book diverges into a lengthy bibliography of fictional books-within-the-book, each encapsulating a different theory about what the mechanics and/or function of Solaris is, thereby demonstrating that 'Solaris studies' as it is called within the world of the book has been going on for years with no tangible results, which actually leads to extreme embarrassment and then a deliberate and willful blindness to the 'Solaris problem' on the part of the book's scientific community. But I'll leave it all here before this review gets too long... Highly recommended, and a likely reread in 2023.

Brokeback Mountain (1997) Annie Proulx Brokeback Mountain began as a short story by American author Annie Proulx which appeared in the New Yorker in 1997, although it is now more famous for the 2005 film adaptation directed by Taiwanese filmmaker Ang Lee. Both versions follow two young men who are hired for the summer to look after sheep at a range under the 'Brokeback' mountain in Wyoming. Unexpectedly, however, they form an intense emotional and sexual attachment, yet life intervenes and demands they part ways at the end of the summer. Over the next twenty years, though, as their individual lives play out with marriages, children and jobs, they continue reuniting for brief albeit secret liaisons on camping trips in remote settings. There's no feigned shyness or self-importance in Brokeback Mountain, just a close, compassionate and brutally honest observation of a doomed relationship and a bone-deep feeling for the hardscrabble life in the post-War West. To my mind, very few books have captured so acutely the desolation of a frustrated and repressed passion, as well as the particular flavour of undirected anger that can accompany this kind of yearning. That the original novella does all this in such a beautiful way (and without the crutch of the Wyoming landscape to look at ) is a tribute to Proulx's skills as a writer. Indeed, even without the devasting emotional undertones, Proulx's descriptions of the mountains and scree of the West is likely worth the read alone.

Luster (2020) Raven Leilani Edie is a young Black woman living in New York whose life seems to be spiralling out of control. She isn't good at making friends, her career is going nowhere, and she has no close family to speak of as well. She is, thus, your typical NYC millennial today, albeit seen through a lens of Blackness that complicates any reductive view of her privilege or minority status. A representative paragraph might communicate the simmering tone:

Before I start work, I browse through some photos of friends who are doing better than me, then an article on a black teenager who was killed on 115th for holding a weapon later identified as a showerhead, then an article on a black woman who was killed on the Grand Concourse for holding a weapon later identified as a cell phone, then I drown myself in the comments section and do some online shopping, by which I mean I put four dresses in my cart as a strictly theoretical exercise and then let the page expire.

She starts a sort-of affair with an older white man who has an affluent lifestyle in nearby New Jersey. Eric or so he claims has agreed upon an 'open relationship' with his wife, but Edie is far too inappropriate and disinhibited to respect any boundaries that Eric sets for her, and so Edie soon becomes deeply entangled in Eric's family life. It soon turns out that Eric and his wife have a twelve-year-old adopted daughter, Akila, who is also wait for it Black. Akila has been with Eric's family for two years now and they aren t exactly coping well together. They don t even know how to help her to manage her own hair, let alone deal with structural racism. Yet despite how dark the book's general demeanour is, there are faint glimmers of redemption here and there. Realistic almost to the end, Edie might finally realise what s important in her life, but it would be a stretch to say that she achieves them by the final page. Although the book is full of acerbic remarks on almost any topic (Dogs: "We made them needy and physically unfit. They used to be wolves, now they are pugs with asthma."), it is the comments on contemporary race relations that are most critically insightful. Indeed, unsentimental, incisive and funny, Luster had much of what I like in Colson Whitehead's books at times, but I can't remember a book so frantically fast-paced as this since the Booker-prize winning The Sellout by Paul Beatty or Sam Tallent's Running the Light.

This year had a lot of things happen in it, world-wide, as is always the case. Being more selfish here are the things I remember, in brief unless there are comments/questions:

• I learned more Finnish.
• Lots of things with our child.
  • I helped teach him to swim.
  • He learned to tell the time with an analog clock/watch.
  • I took him to a circus for the first (only) time ever.
  • He cut his hair for the first time in six years.
  • He spent his a birthday with my parents, in the UK - His languages skills were on top-form, understanding the various UK accents.
  • And another with family here in Finland - Where he watched me roll, naked, in the snow after sauna, and then he asked "Daddy why you did that?"

On the topic of Finnish I'm getting pretty damn good at understanding, albeit less good in speaking. Finnish is all about the suffixes, so:

• A car
  • auto
• My car
  • autoni
• In a car
  • autossa
• In my car
  • autossani
• With my car
  • autoll ni
• From a car
  • autosta
• Car-less
  • autoton

Most of this is regular, so you can be childless via the "ton" suffix - lapsiton is "lapsi" (child) "ton" (less). The hard part in communication is thus twofold:

• Knowing the word you want to use, be it car, cake, spoon, or smile.
• Getting the appropriate suffix for the use you want.

Our child turned six recently, and most of the year was spent doing things with him, for him, and to him. He's on the verge of learning to read (English and Finnish), he's interested in maths and completes little puzzles freely and happily. He likes to help with Sodoku, for example and not just the child-versions. In the past couple of weeks I let him play Super Mario & Super Mario Bros 3, on a NES Classic, and he dies constantly, with a smile on his face. But he does love to tell me what to do when he watches me play! He's learned to ice-skate, and ski, and almost learned to swim. (I'll say he can swim 3m inside a pool, without aid, but then he starts to sink.) We've got a couple of regular rituals with each other - including going to sauna every week or two, and other similar things. He's gotten more interested in helping me cook, and his mother too. (My wife and I live in separate houses..) I guess the next big milestone will be him walking to school by himself, which will start next year. As things stand I wake up early, go over to his house, and do all the morning-things, before I take him there. I expect I'll still want to go there, give him his breakfast, his medicine, and help him get dressed. After that I guess I kick him out, and he makes his own way there. Happily the walk to school is a few hundred meters, and doesn't involve crossing any roads. But of course it does bring other complications: if he's not collected, and walks home himself, then he needs a key to one/other house, and there's the potential need for a phone to say "I'm late", "I'm lost", or us to say "Where are you?". Anyway .. interesting year .. good year. Despite everything else.

A few years ago, I downsized my personal infrastructure. Until 2018, there were a dozen containers running on a single Hetzner server.¹ I migrated my emails to Fastmail and my DNS zones to Gandi. It left me with only my blog to self-host. As of today, my low-scale infrastructure is composed of 4 virtual machines running NixOS on Hetzner Cloud and Vultr, a handful of DNS zones on Gandi and Route 53, and a couple of Cloudfront distributions. It is managed by CDK for Terraform (CDKTF), while NixOS deployments are handled by NixOps. In this article, I provide a brief introduction to Terraform, CDKTF, and the Nix ecosystem. I also explain how to use Nix to access these tools within your shell, so you can quickly start using them.

CDKTF: infrastructure as code Terraform is an infrastructure-as-code tool. You can define your infrastructure by declaring resources with the HCL language. This language has some additional features like loops to declare several resources from a list, built-in functions you can call in expressions, and string templates. Terraform relies on a large set of providers to manage resources.

Managing servers Here is a short example using the Hetzner Cloud provider to spawn a virtual machine:

variable "hcloud_token"  
  sensitive = true
 
provider "hcloud"  
  token = var.hcloud_token
 
resource "hcloud_server" "web03"  
  name = "web03"
  server_type = "cpx11"
  image = "debian-11"
  datacenter = "nbg1-dc3"
 
resource "hcloud_rdns" "rdns4-web03"  
  server_id = hcloud_server.web03.id
  ip_address = hcloud_server.web03.ipv4_address
  dns_ptr = "web03.luffy.cx"
 
resource "hcloud_rdns" "rdns6-web03"  
  server_id = hcloud_server.web03.id
  ip_address = hcloud_server.web03.ipv6_address
  dns_ptr = "web03.luffy.cx"
 

HCL expressiveness is quite limited and I find a general-purpose language more convenient to describe all the resources. This is where CDK for Terraform comes in: you can manage your infrastructure using your preferred programming language, including TypeScript, Go, and Python. Here is the previous example using CDKTF and TypeScript:

import   App, TerraformStack, Fn   from "cdktf";
import   HcloudProvider   from "./.gen/providers/hcloud/provider";
import * as hcloud from "./.gen/providers/hcloud";
class MyStack extends TerraformStack  
  constructor(scope: Construct, name: string)  
    super(scope, name);
    const hcloudToken = new TerraformVariable(this, "hcloudToken",  
      type: "string",
      sensitive: true,
     );
    const hcloudProvider = new HcloudProvider(this, "hcloud",  
      token: hcloudToken.value,
     );
    const web03 = new hcloud.server.Server(this, "web03",  
      name: "web03",
      serverType: "cpx11",
      image: "debian-11",
      datacenter: "nbg1-dc3",
      provider: hcloudProvider,
     );
    new hcloud.rdns.Rdns(this, "rdns4-web03",  
      serverId: Fn.tonumber(web03.id),
      ipAddress: web03.ipv4Address,
      dnsPtr: "web03.luffy.cx",
      provider: hcloudProvider,
     );
    new hcloud.rdns.Rdns(this, "rdns6-web03",  
      serverId: Fn.tonumber(web03.id),
      ipAddress: web03.ipv6Address,
      dnsPtr: "web03.luffy.cx",
      provider: hcloudProvider,
     );
   
 
const app = new App();
new MyStack(app, "cdktf-take1");
app.synth();

Running cdktf synth generates a configuration file for Terraform, terraform plan previews the changes, and terraform apply applies them. Now that you have a general-purpose language, you can use functions.

Managing DNS records While using CDKTF for 4 web servers may seem a tad overkill, this is quite different when it comes to managing a few DNS zones. With DNSControl, which is using JavaScript as a domain-specific language, I was able to define the bernat.ch zone with this snippet of code:

D("bernat.ch", REG_NONE, DnsProvider(DNS_BIND, 0), DnsProvider(DNS_GANDI),
  DefaultTTL('2h'),
  FastMailMX('bernat.ch',  subdomains: ['vincent'] ),
  WebServers('@'),
  WebServers('vincent');

This generated 38 records. With CDKTF, I use:

new Route53Zone(this, "bernat.ch", providers.aws)
  .sign(dnsCMK)
  .registrar(providers.gandiVB)
  .www("@", servers)
  .www("vincent", servers)
  .www("media", servers)
  .fastmailMX(["vincent"]);

All the magic is in the code that I did not show you. You can check the dns.ts file in the cdktf-take1 repository to see how it works. Here is a quick explanation:

• Route53Zone() creates a new zone hosted by Route 53,
• sign() signs the zone with the provided master key,
• registrar() registers the zone to the registrar of the domain and sets up DNSSEC,
• www() creates A and AAAA records for the provided name pointing to the web servers,
• fastmailMX() creates the MX records and other support records to direct emails to Fastmail.

Here is the content of the fastmailMX() function. It generates a few records and returns the current zone for chaining:

fastmailMX(subdomains?: string[])  
  (subdomains ?? [])
    .concat(["@", "*"])
    .forEach((subdomain) =>
      this.MX(subdomain, [
        "10 in1-smtp.messagingengine.com.",
        "20 in2-smtp.messagingengine.com.",
      ])
    );
  this.TXT("@", "v=spf1 include:spf.messagingengine.com ~all");
  ["mesmtp", "fm1", "fm2", "fm3"].forEach((dk) =>
    this.CNAME( $ dk ._domainkey ,  $ dk .$ this.name .dkim.fmhosted.com. )
  );
  this.TXT("_dmarc", "v=DMARC1; p=none; sp=none");
  return this;
 

I encourage you to browse the repository if you need more information.

About Pulumi My first tentative around Terraform was to use Pulumi. You can find this attempt on GitHub. This is quite similar to what I currently do with CDKTF. The main difference is that I am using Python instead of TypeScript because I was not familiar with TypeScript at the time.² Pulumi predates CDKTF and it uses a slightly different approach. CDKTF generates a Terraform configuration (in JSON format instead of HCL), delegating planning, state management, and deployment to Terraform. It is therefore bound to the limitations of what can be expressed by Terraform, notably when you need to transform data obtained from one resource to another.³ Pulumi needs specific providers for each resource. Many Pulumi providers are thin wrappers encapsulating Terraform providers. While Pulumi provides a good user experience, I switched to CDKTF because writing providers for Pulumi is a chore. CDKTF does not require such a step. Outside the big players (AWS, Azure and Google Cloud), the existence, quality, and freshness of the Pulumi providers are inconsistent. Most providers rely on a Terraform provider and they may lag a few versions behind, miss a few resources, or have a few bugs of their own. When a provider does not exist, you can write one with the help of the pulumi-terraform-bridge library. The Pulumi project provides a boilerplate for this purpose. I had a bad experience with it when writing providers for Gandi and Vultr: the Makefile automatically installs Pulumi using a curl sh pattern and does not work with /bin/sh. There is a lack of interest for community-based contributions⁴ or even for providers for smaller players.

NixOS & NixOps Nix is a functional, purely-functional programming language. Nix is also the name of the package manager that is built on top of the Nix language. It allows users to declaratively install packages. nixpkgs is a repository of packages. You can install Nix on top of a regular Linux distribution. If you want more details, a good resource is the official website, and notably the learn section. There is a steep learning curve, but the reward is tremendous.

NixOS: declarative Linux distribution NixOS is a Linux distribution built on top of the Nix package manager. Here is a configuration snippet to add some packages:

environment.systemPackages = with pkgs;
  [
    bat
    htop
    liboping
    mg
    mtr
    ncdu
    tmux
  ];

It is possible to alter an existing derivation⁵ to use a different version, enable a specific feature, or apply a patch. Here is how I enable and configure Nginx to disable the stream module, add the Brotli compression module, and add the IP address anonymizer module. Moreover, instead of using OpenSSL 3, I keep using OpenSSL 1.1.⁶

services.nginx =  
  enable = true;
  package = (pkgs.nginxStable.override  
    withStream = false;
    modules = with pkgs.nginxModules; [
      brotli
      ipscrub
    ];
    openssl = pkgs.openssl_1_1;
   );

If you need to add some patches, it is also possible. Here are the patches I added in 2019 to circumvent the DoS vulnerabilities in Nginx until they were fixed in NixOS:⁷

services.nginx.package = pkgs.nginxStable.overrideAttrs (old:  
  patches = oldAttrs.patches ++ [
    # HTTP/2: reject zero length headers with PROTOCOL_ERROR.
    (pkgs.fetchpatch  
      url = https://github.com/nginx/nginx/commit/dbdd[ ].patch;
      sha256 = "a48190[ ]";
     )
    # HTTP/2: limited number of DATA frames.
    (pkgs.fetchpatch  
      url = https://github.com/nginx/nginx/commit/94c5[ ].patch;
      sha256 = "af591a[ ]";
     )
    #  HTTP/2: limited number of PRIORITY frames.
    (pkgs.fetchpatch  
      url = https://github.com/nginx/nginx/commit/39bb[ ].patch;
      sha256 = "1ad8fe[ ]";
     )
  ];
 );

If you are interested, have a look at my relatively small configuration: common.nix contains the configuration to be applied to any host (SSH, users, common software packages), web.nix contains the configuration for the web servers, isso.nix runs Isso into a systemd container.

NixOps: NixOS deployment tool On a single node, NixOS configuration is in the /etc/nixos/configuration.nix file. After modifying it, you have to run nixos-rebuild switch. Nix fetches all possible dependencies from the binary cache and builds the remaining packages. It creates a new entry in the boot loader menu and activates the new configuration. To manage several nodes, there exists several options, including NixOps, deploy-rs, Colmena, and morph. I do not know all of them, but from my point of view, the differences are not that important. It is also possible to build such a tool yourself as Nix provides the most important building blocks: nix build and nix copy. NixOps is one of the first tools available but I encourage you to explore the alternatives. NixOps configuration is written in Nix. Here is a simplified configuration to deploy znc01.luffy.cx, web01.luffy.cx, and web02.luffy.cx, with the help of the server and web functions:

let
  server = hardware: name: imports:  
    deployment.targetHost = "$ name .luffy.cx";
    networking.hostName = name;
    networking.domain = "luffy.cx";
    imports = [ (./hardware/. + "/$ hardware .nix") ] ++ imports;
   ;
  web = hardware: idx: imports:
    server hardware "web$ lib.fixedWidthNumber 2 idx " ([ ./web.nix ] ++ imports);
in  
  network.description = "Luffy infrastructure";
  network.enableRollback = true;
  defaults = import ./common.nix;
  znc01 = server "exoscale" [ ./znc.nix ];
  web01 = web "hetzner" 1 [ ./isso.nix ];
  web02 = web "hetzner" 2 [];
 

Tying everything together with Nix The Nix ecosystem is a unified solution to the various problems around software and configuration management. A very interesting feature is the declarative and reproducible developer environments. This is similar to Python virtual environments, except it is not language-specific.

Brief introduction to Nix flakes I am using flakes, a new Nix feature improving reproducibility by pinning all dependencies and making the build hermetic. While the feature is marked as experimental,⁸ it is widely used and you may see flake.nix and flake.lock at the root of some repositories. As a short example, here is the flake.nix content shipped with Snimpy, an interactive SNMP tool for Python relying on libsmi, a C library:

 
  inputs =  
    nixpkgs.url = "nixpkgs";
    flake-utils.url = "github:numtide/flake-utils";
   ;
  outputs =   self, ...  @inputs:
    inputs.flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = inputs.nixpkgs.legacyPackages."$ system ";
      in
       
        # nix build
        packages.default = pkgs.python3Packages.buildPythonPackage  
          name = "snimpy";
          src = self;
          preConfigure = ''echo "1.0.0-0-000000000000" > version.txt'';
          checkPhase = "pytest";
          checkInputs = with pkgs.python3Packages; [ pytest mock coverage ];
          propagatedBuildInputs = with pkgs.python3Packages; [ cffi pysnmp ipython ];
          buildInputs = [ pkgs.libsmi ];
         ;
        # nix run + nix shell
        apps.default =   
          type = "app";
          program = "$ self.packages."$ system ".default /bin/snimpy";
         ;
        # nix develop
        devShells.default = pkgs.mkShell  
          name = "snimpy-dev";
          buildInputs = [
            self.packages."$ system ".default.inputDerivation
            pkgs.python3Packages.ipython
          ];
         ;
       );
 

If you have Nix installed on your system:

• nix run github:vincentbernat/snimpy runs Snimpy,
• nix shell github:vincentbernat/snimpy provides a shell with Snimpy ready-to-use,
• nix build github:vincentbernat/snimpy builds the Python package, tests included, and
• nix develop . provides a shell to hack around Snimpy when run from a fresh checkout.⁹

For more information about Nix flakes, have a look at the tutorial from Tweag.

Nix and CDKTF At the root of the repository I use for CDKTF, there is a flake.nix file to set up a shell with Terraform and CDKTF installed and with the appropriate environment variables to automate my infrastructure. Terraform is already packaged in nixpkgs. However, I need to apply a patch on top of the Gandi provider. Not a problem with Nix!

terraform = pkgs.terraform.withPlugins (p: [
  p.aws
  p.hcloud
  p.vultr
  (p.gandi.overrideAttrs
    (old:  
      src = pkgs.fetchFromGitHub  
        owner = "vincentbernat";
        repo = "terraform-provider-gandi";
        rev = "feature/livedns-key";
        hash = "sha256-V16BIjo5/rloQ1xTQrdd0snoq1OPuDh3fQNW7kiv/kQ=";
       ;
     ))
]);

CDKTF is written in TypeScript. I have a package.json file with all the dependencies needed, including the ones to use TypeScript as the language to define infrastructure:

 
  "name": "cdktf-take1",
  "version": "1.0.0",
  "main": "main.js",
  "types": "main.ts",
  "private": true,
  "dependencies":  
    "@types/node": "^14.18.30",
    "cdktf": "^0.13.3",
    "cdktf-cli": "^0.13.3",
    "constructs": "^10.1.151",
    "eslint": "^8.27.0",
    "prettier": "^2.7.1",
    "ts-node": "^10.9.1",
    "typescript": "^3.9.10",
    "typescript-language-server": "^2.1.0"
   
 

I use Yarn to get a yarn.lock file that can be used directly to declare a derivation containing all the dependencies:

nodeEnv = pkgs.mkYarnModules  
  pname = "cdktf-take1-js-modules";
  version = "1.0.0";
  packageJSON = ./package.json;
  yarnLock = ./yarn.lock;
 ;

The next step is to generate the CDKTF providers from the Terraform providers and turn them into a derivation:

cdktfProviders = pkgs.stdenvNoCC.mkDerivation  
  name = "cdktf-providers";
  nativeBuildInputs = [
    pkgs.nodejs
    terraform
  ];
  src = nix-filter  
    root = ./.;
    include = [ ./cdktf.json ./tsconfig.json ];
   ;
  buildPhase = ''
    export HOME=$(mktemp -d)
    export CHECKPOINT_DISABLE=1
    export DISABLE_VERSION_CHECK=1
    export PATH=$ nodeEnv /node_modules/.bin:$PATH
    ln -nsf $ nodeEnv /node_modules node_modules
    # Build all providers we have in terraform
    for provider in $(cd $ terraform /libexec/terraform-providers; echo */*/*/*); do
      version=''$ provider##*/ 
      provider=''$ provider%/* 
      echo "Build $provider@$version"
      cdktf provider add --force-local $provider@$version   cat
    done
    echo "Compile TS   JS"
    tsc
  '';
  installPhase = ''
    mv .gen $out
    ln -nsf $ nodeEnv /node_modules $out/node_modules
  '';
 ;

Finally, we can define the development environment:

devShells.default = pkgs.mkShell  
  name = "cdktf-take1";
  buildInputs = [
    pkgs.nodejs
    pkgs.yarn
    terraform
  ];
  shellHook = ''
    # No telemetry
    export CHECKPOINT_DISABLE=1
    # No autoinstall of plugins
    export CDKTF_DISABLE_PLUGIN_CACHE_ENV=1
    # Do not check version
    export DISABLE_VERSION_CHECK=1
    # Access to node modules
    export PATH=$PWD/node_modules/.bin:$PATH
    ln -nsf $ nodeEnv /node_modules node_modules
    ln -nsf $ cdktfProviders  .gen
    # Credentials
    for p in \
      njf.nznmba.pbz/Nqzvavfgengbe \
      urgmare.pbz/ivaprag@oreang.pu \
      ihyge.pbz/ihyge@ivaprag.oreang.pu; do
        eval $(pass show $(echo $p   tr 'A-Za-z' 'N-ZA-Mn-za-m')   grep '^export')
    done
    eval $(pass show personal/cdktf/secrets   grep '^export')
    export TF_VAR_hcloudToken="$HCLOUD_TOKEN"
    export TF_VAR_vultrApiKey="$VULTR_API_KEY"
    unset VULTR_API_KEY HCLOUD_TOKEN
  '';
 ;

The derivations listed in buildInputs are available in the provided shell. The content of shellHook is sourced when starting the shell. It sets up some symbolic links to make the JavaScript environment built at an earlier step available, as well as the generated CDKTF providers. It also exports all the credentials.¹⁰ I am also using direnv with an .envrc to automatically load the development environment. This also enables the environment to be available from inside Emacs, notably when using lsp-mode to get TypeScript completions. Without direnv, nix develop . can activate the environment. I use the following commands to deploy the infrastructure:¹¹

$ cdktf synth
$ cd cdktf.out/stacks/cdktf-take1
$ terraform plan --out plan
$ terraform apply plan
$ terraform output -json > ~-automation/nixops-take1/cdktf.json

The last command generates a JSON file containing various data to complete the deployment with NixOps.

NixOps The JSON file exported by Terraform contains the list of servers with various attributes:

 
  "hardware": "hetzner",
  "ipv4Address": "5.161.44.145",
  "ipv6Address": "2a01:4ff:f0:b91::1",
  "name": "web05.luffy.cx",
  "tags": [
    "web",
    "continent:NA",
    "continent:SA"
  ]
 

In network.nix, this list is imported and transformed into an attribute set describing the servers. A simplified version looks like this:

let
  lib = inputs.nixpkgs.lib;
  shortName = name: builtins.elemAt (lib.splitString "." name) 0;
  domainName = name: lib.concatStringsSep "." (builtins.tail (lib.splitString "." name));
  server = hardware: name: imports:  
    networking =  
      hostName = shortName name;
      domain = domainName name;
     ;
    deployment.targetHost = name;
    imports = [ (./hardware/. + "/$ hardware .nix") ] ++ imports;
   ;
  cdktf-servers-json = (lib.importJSON ./cdktf.json).servers.value;
  cdktf-servers = map
    (s:
      let
        tags-maybe-import = map (t: ./. + "/$ t .nix") s.tags;
        tags-import = builtins.filter (t: builtins.pathExists t) tags-maybe-import;
      in
       
        name = shortName s.name;
        value = server s.hardware s.name tags-import;
       )
    cdktf-servers-json;
in
 
  // [ ]
  // builtins.listToAttrs cdktf-servers

For web05, this expands to:

web05 =  
  networking =  
    hostName = "web05";
    domainName = "luffy.cx";
   ;
  deployment.targetHost = "web05.luffy.cx";
  imports = [ ./hardware/hetzner.nix ./web.nix ];
 ;

As for CDKTF, at the root of the repository I use for NixOps, there is a flake.nix file to set up a shell with NixOps configured. Because NixOps do not support rollouts, I usually use the following commands to deploy on a single server:¹²

$ nix flake update
$ nixops deploy --include=web04
$ ./tests web04.luffy.cx

If the tests are OK, I deploy the remaining nodes gradually with the following command:

$ (set -e; for h in web 03..06 ; do nixops deploy --include=$h; done)

nixops deploy rolls out all servers in parallel and therefore could cause a short outage where all Nginx are down at the same time.

---

This post has been a work-in-progress for the past three years, with the content being updated and refined as I experimented with different solutions. There is still much to explore¹³ but I feel there is enough content to publish now.

---

1. It was an AMD Athlon 64 X2 5600+ with 2 GB of RAM and 2 400 GB disks with software RAID. I was paying something around 59 per month for it. While it was a good deal in 2008, by 2018 it was no longer cost-effective. It was running on Debian Wheezy with Linux-VServer for isolation, both of which were outdated in 2018.
2. I also did not use Python because Poetry support in Nix was a bit broken around the time I started hacking around CDKTF.
3. Pulumi can apply arbitrary functions with the apply() method on an output. It makes it easy to transform data that are not known during the planning stage. Terraform has functions to serve a similar purpose, but they are more limited.
4. The two mentioned pull requests are not merged yet. The second one is superseded by PR #61, submitted two months later, which enforces the use of /bin/bash. I also submitted PR #56, which was merged 4 months later and quickly reverted without an explanation.
5. You may consider packages and derivations to be synonyms in the Nix ecosystem.
6. OpenSSL 3 has outstanding performance regressions.
7. NixOS can be a bit slow to integrate patches since they need to rebuild parts of the binary cache before releasing the fixes. In this specific case, they were fast: the vulnerability and patches were released on August 13^th 2019 and available in NixOS on August 15th. As a comparison, Debian only released the fixed version on August 22^nd, which is unusually late.
8. Because flakes are experimental, many documentations do not use them and it is an additional aspect to learn.
9. It is possible to replace . with github:vincentbernat/snimpy, like in the other commands, but having Snimpy dependencies without Snimpy source code is less interesting.
10. I am using pass as a password manager. The password names are only obfuscated to avoid spam.
11. The cdktf command can wrap the terraform commands, but I prefer to use them directly as they are more flexible.
12. If the change is risky, I disable the server with CDKTF. This removes it from the web service DNS records.
13. I would like to replace NixOps with an alternative handling progressive rollouts and checks. I am also considering switching to Nomad or Kubernetes to deploy workloads.

Review: Shutdown, by Adam Tooze

Publisher:	Viking
Copyright:	September 2021
ISBN:	0-593-29756-3
Format:	Kindle
Pages:	305

Shutdown is a history of the world macroeconomic response to COVID-19, covering 2020 and the very beginning of 2021. But wait, you might be saying. It's only the end of 2022 right now, and this book was published in September of 2021. That's not history, that's journalism. And yes, I think that's a valid critique. Shutdown is doing something rather odd, and I'm not certain it was a good idea, but I do think it has a (somewhat narrow) audience. Descriptions first. After an awkward introduction (more on that later), Tooze launches into an essentially chronological history in four parts: the initial viral spread and early political and public health response, the economic hard stop and macroeconomic response, the summer fallout and political complications, and the more-organized aftershocks of the fall. The early chapters are closer to a history, with a clear timeline and the tracery of cause and effect. The closer the narrative comes to the time Tooze was writing it, the more that clarity drops away. The last few chapters feel like a collection of simultaneous events that may or may not be related or have long-term significance. Everyone reading this lived through those events, and if you're at all like me, consumed far more news coverage of them than was healthy. The obvious question, then, is why read a book that rehashes all of that? For me, there are two answers: Tooze pays attention to more of the world than makes it into the local headlines and tries to synthesize a larger picture, and the focus of this book is the macroeconomic facilities used in the response. I remember the fights over school closures in the US. I didn't know about the impact of unprecedented Federal Reserve action on the bond market for emerging market government debt. If you're the sort of person who reads The Economist religiously (which I am not), you may not learn anything new here. If you're not, and you have a general interest in international finance, there were probably some wrinkles you missed. Even if you stay up-to-date on the more technical news, Shutdown provides an intermediate consolidation and restructuring. It's not ready to be a history in the traditional sense, but it's a first pass at putting events in order and tracing the implications for the global financial system. Read in that sense, Shutdown felt like a continuation of Tooze's Crashed, following the same themes of a hegemonic but unstable dollar system and a Federal Reserve that has increasingly taken on the role of backstop to the entire world (and how the only tools it has available tend to increase wealth inequality). If you've not read Crashed, read it first. I think it's the stronger and more thorough book (not the least because it had more time and data to construct a coherent history), and it's the best introduction to the international macroeconomic risks that Tooze traces in Shutdown. This book was, for me, an update of the Tooze's thinking in Crashed for the COVID era. Tooze is very good at clearly describing macroeconomic shocks. I recently read Lev Menand's The Fed Unbound, which is in part about the same intervention during the COVID shock that Tooze describes here, and yet it wasn't until I read Shutdown that I grasped that the Treasury purchases by the Fed had arguably crossed the political red line of monetizing government debt, even though the Fed probably had no choice and few people noticed in the middle of the crisis. This is where Tooze's background as a historian is helpful. Most people writing on this topic are either economists, who seem to take inferences like that for granted and dive into the technical debate, or journalists, who rarely understand the nuance and often jump to facile conclusions. Tooze is a historian with an extensive economics background; he can explain the mechanics while still focusing on the limitations of politics, which is the sort of analysis that I want to read. The problem with this book as a history is that it necessarily raises more questions than it answers. In the conclusion, Tooze writes:

A severe tightening in U.S. monetary policy or even a full-fledged taper tantrum would put global resilience to a stern test. So too would a violent escalation of geopolitical tension in one of the major regions of the world economy.

Both of those events have subsequently happened, which throws any tentative conclusions Tooze can offer into question. For another example, when Tooze wrote Shutdown, China's zero COVID policy was widely celebrated inside China and had enabled a fast economic recovery while the rest of the world was still in serious difficulty. Before Omicron, it was conceivable (if extremely risky) that China could continue to avoid a major COVID surge. In December of 2022, it's obvious they only managed to delay, and while delay meant vaccination and the price in deaths may well be smaller than was paid for the route taken by most of the rest of the world, the trade-offs are now even harder to analyze. There are, of course, other examples, the most obvious of which is the rise of global inflation simultaneous with a strengthening dollar. A history written from a distance of several decades would have included that aftermath. A snap history with a distance of only a few months cannot. The other problem with this book is that it's not as polished. Viking did an amazing job turning publication around in roughly six months, dramatically faster than publishing normally works, but there are parts that could have used more editing. The introduction, in particular, reads more like a blog post than an edited book, and while I'm a happy reader of Tooze's Chartbook, the recap of COVID's impact was a bit trite and Tooze's abstract musings on polycrisis could have used tightening and clarity. The meat of the book is better, but there is a messy stream-of-consciousness feel that is inherent in a book written to this tight of a timeline. I think the audience of this book is, narrowly, people who have read Crashed and want to follow that line of reasoning into the COVID era. Crashed is one of the better books on macroeconomic history that I've read, and I did indeed want to follow that train of thought, so I am part of that audience. This is not the sort of book that I would widely recommend, however. If you want to read it, you probably know that already; if you are new to Tooze's analysis, Crashed is the place to start. Rating: 7 out of 10

I have reviewed, updated and revised my short book about the Rust programming language, Rust for the Polyglot Programmer. It now covers some language improvements from the past year (noting which versions of Rust they re available in), and has been updated for changes in the Rust library ecosystem. With (further) assistance from Mark Wooding, there is also a new table of recommendations for numerical conversion. Recap about Rust for the Polyglot Programmer There are many introductory materials about Rust. This one is rather different. Compared to much other information about Rust, Rust for the Polyglot Programmer is:

• Dense: I assume a lot of starting knowledge. Or to look at it another way: I expect my reader to be able to look up and digest non-Rust-specific words or concepts.
• Broad: I cover not just the language and tools, but also the library ecosystem, development approach, community ideology, and so on.
• Frank: much material about Rust has a tendency to gloss over or minimise the bad parts. I don t do that. That also frees me to talk about strategies for dealing with the bad parts.
• Non-neutral: I m not afraid to recommend particular libraries, for example. I m not afraid to extol Rust s virtues in the areas where it does well.
• Terse, and sometimes shallow: I often gloss over what I see as unimportant or fiddly details; instead I provide links to appropriate reference materials.

After reading Rust for the Polyglot Programmer, you won t know everything you need to know to use Rust for any project, but should know where to find it. Comments are welcome of course, via the Dreamwidth comments or Salsa issue or MR. (If you re making a contribution, please indicate your agreement with the Developer Certificate of Origin.)

edited 2022-12-20 01:48 to fix a typo

comments

tl;dr: Ok-wrapping as needed in today s Rust is a significant distraction, because there are multiple ways to do it. They are all slightly awkward in different ways, so are least-bad in different situations. You must choose a way for every fallible function, and sometimes change a function from one pattern to another. Rust really needs #[throws] as a first-class language feature. Code using #[throws] is simpler and clearer. Please try out withoutboats s fehler. I think you will like it. Contents

• A recent personal experience in coding style
• What is Ok wrapping? Intro to Rust error handling
• A minor inconvenience, or a significant distraction?
  • Idioms for Ok-wrapping - a bestiary
• What is to be done, then?
  • Limitations of fehler
  • Please can we have #[throws] in the Rust language
  • Explicitness
• Appendix - examples showning code with Ok wrapping is worse than code using #[throws]

A recent personal experience in coding style Ever since I read withoutboats s 2020 article about fehler, I have been using it in most of my personal projects. For Reasons I recently had a go at eliminating the dependency on fehler from Hippotat. So, I made a branch, deleted the dependency and imports, and started on the whack-a-mole with the compiler errors. After about a half hour of this, I was starting to feel queasy. After an hour I had decided that basically everything I was doing was making the code worse. And, bizarrely, I kept having to make individual decisons about what idiom to use in each place. I couldn t face it any more. After sleeping on the question I decided that Hippotat would be in Debian with fehler, or not at all. Happily the Debian Rust Team generously helped me out, so the answer is that fehler is now in Debian, so it s fine. For me this experience, of trying to convert Rust-with-#[throws] to Rust-without-#[throws] brought the Ok wrapping problem into sharp focus. What is Ok wrapping? Intro to Rust error handling (You can skip this section if you re already a seasoned Rust programer.) In Rust, fallibility is represented by functions that return Result<SuccessValue, Error>: this is a generic type, representing either whatever SuccessValue is (in the Ok variant of the data-bearing enum) or some Error (in the Err variant). For example, std::fs::read_to_string, which takes a filename and returns the contents of the named file, returns Result<String, std::io::Error>. This is a nice and typesafe formulation of, and generalisation of, the traditional C practice, where a function indicates in its return value whether it succeeded, and errors are indicated with an error code. Result is part of the standard library and there are convenient facilities for checking for errors, extracting successful results, and so on. In particular, Rust has the postfix ? operator, which, when applied to a Result, does one of two things: if the Result was Ok, it yields the inner successful value; if the Result was Err, it returns early from the current function, returning an Err in turn to the caller. This means you can write things like this:

    let input_data = std::fs::read_to_string(input_file)?;

and the error handling is pretty automatic. You get a compiler warning, or a type error, if you forget the ?, so you can t accidentally ignore errors. But, there is a downside. When you are returning a successful outcome from your function, you must convert it into a Result. After all, your fallible function has return type Result<SuccessValue, Error>, which is a different type to SuccessValue. So, for example, inside std::fs::read_to_string, we see this:

        let mut string = String::new();
        file.read_to_string(&mut string)?;
        Ok(string)
     

string has type String; fs::read_to_string must return Result<String, ..>, so at the end of the function we must return Ok(string). This applies to return statements, too: if you want an early successful return from a fallible function, you must write return Ok(whatever). This is particularly annoying for functions that don t actually return a nontrivial value. Normally, when you write a function that doesn t return a value you don t write the return type. The compiler interprets this as syntactic sugar for -> (), ie, that the function returns (), the empty tuple, used in Rust as a dummy value in these kind of situations. A block ( ... ) whose last statement ends in a ; has type (). So, when you fall off the end of a function, the return value is (), without you having to write it. So you simply leave out the stuff in your program about the return value, and your function doesn t have one (i.e. it returns ()). But, a function which either fails with an error, or completes successfuly without returning anything, has return type Result<(), Error>. At the end of such a function, you must explicitly provide the success value. After all, if you just fall off the end of a block, it means the block has value (), which is not of type Result<(), Error>. So the fallible function must end with Ok(()), as we see in the example for std::fs::read_to_string. A minor inconvenience, or a significant distraction? I think the need for Ok-wrapping on all success paths from fallible functions is generally regarded as just a minor inconvenience. Certainly the experienced Rust programmer gets very used to it. However, while trying to remove fehler s #[throws] from Hippotat, I noticed something that is evident in codebases using vanilla Rust (without fehler) but which goes un-remarked. There are multiple ways to write the Ok-wrapping, and the different ways are appropriate in different situations. See the following examples, all taken from a real codebase. (And it s not just me: I do all of these in different places, - when I don t have fehler available - but all these examples are from code written by others.) Idioms for Ok-wrapping - a bestiary Wrap just a returned variable binding If you have the return value in a variable, you can write Ok(reval) at the end of the function, instead of retval.

    pub fn take_until(&mut self, term: u8) -> Result<&'a [u8]>  
        // several lines of code
        Ok(result)
     

If the returned value is not already bound to variable, making a function fallible might mean choosing to bind it to a variable. Wrap a nontrivial return expression Even if it s not just a variable, you can wrap the expression which computes the returned value. This is often done if the returned value is a struct literal:

    fn take_from(r: &mut Reader<'_>) -> Result<Self>  
        // several lines of code
        Ok(AuthChallenge   challenge, methods  )
     

Introduce Ok(()) at the end For functions returning Result<()>, you can write Ok(()). This is usual, but not ubiquitous, since sometimes you can omit it. Wrap the whole body If you don t have the return value in a variable, you can wrap the whole body of the function in Ok( ). Whether this is a good idea depends on how big and complex the body is.

    fn from_str(s: &str) -> std::result::Result<Self, Self::Err>  
        Ok(match s  
            "Authority" => RelayFlags::AUTHORITY,
            // many other branches
            _ => RelayFlags::empty(),
         )
     

Omit the wrap when calling fallible sub-functions If your function wraps another function call of the same return and error type, you don t need to write the Ok at all. Instead, you can simply call the function and not apply ?. You can do this even if your function selects between a number of different sub-functions to call:

    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result  
        if flags::unsafe_logging_enabled()  
            std::fmt::Display::fmt(&self.0, f)
          else  
            self.0.display_redacted(f)
         
     

But this doesn t work if the returned error type isn t the same, but needs the autoconversion implied by the ? operator. Convert a fallible sub-function error with Ok( ... ?) If the final thing a function does is chain to another fallible function, but with a different error type, the error must be converted somehow. This can be done with ?.

     fn try_from(v: i32) -> Result<Self, Error>  
         Ok(Percentage::new(v.try_into()?))
      

Convert a fallible sub-function error with .map_err Or, rarely, people solve the same problem by converting explicitly with .map_err:

     pub fn create_unbootstrapped(self) -> Result<TorClient<R>>  
         // several lines of code
         TorClient::create_inner(
             // several parameters
         )
         .map_err(ErrorDetail::into)
      

What is to be done, then? The fehler library is in excellent taste and has the answer. With fehler:

• Whether a function is fallible, and what it s error type is, is specified in one place. It is not entangled with the main return value type, nor with the success return paths.
• So the success paths out of a function are not specially marked with error handling boilerplate. The end of function return value, and the expression after return, are automatically wrapped up in Ok. So the body of a fallible function is just like the body of an infallible one, except for places where error handling is actually involved.
• Error returns occur through ? error chaining, and with a new explicit syntax for error return.
• We usually talk about the error we are possibly returning, and avoid talking about Result unless we need to.

fehler provides:

• An attribute macro #[throws(ErrorType)] to make a function fallible in this way.
• A macro throws!(error) for explicitly failing.

This is precisely correct. It is very ergonomic. Consequences include:

• One does not need to decide where to put the Ok-wrapping, since it s automatic rather than explicitly written out.
• Specifically, what idiom to adopt in the body (for example write!(...)?; vs write!(...) in a formatter) does not depend on whether the error needs converting, how complex the body is, and whether the final expression in the function is itself fallible.
• Making an infallible function fallible involves only adding #[throws] to its definition, and ? to its call sites. One does not need to edit the body, or the return type.
• Changing the error returned by a function to a suitably compatible different error type does not involve changing the function body.
• There is no need for a local Result alias shadowing std::result::Result, which means that when one needs to speak of Result explciitly, the code is clearer.

Limitations of fehler But, fehler is a Rust procedural macro, so it cannot get everything right. Sadly there are some wrinkles.

• You can t write #[throws] on a closure.
• Sometimes you can get quite poor error messages if you have a sufficiently broken function body.
• Code inside a macro call isn t properly visible to fehler so sometimes return statements inside macro calls are untreated. This will lead to a type error, so isn t a correctness hazard, but it can be nuisance if you like other syntax extensions eg if_chain.
• #[must_use] #[throws(Error)] fn obtain() -> Thing; ought to mean that Thing must be used, not the Result<Thing, Error>.

But, Rust-with-#[throws] is so much nicer a language than Rust-with-mandatory-Ok-wrapping, that these are minor inconveniences. Please can we have #[throws] in the Rust language This ought to be part of the language, not a macro library. In the compiler, it would be possible to get the all the corner cases right. It would make the feature available to everyone, and it would quickly become idiomatic Rust throughout the community. It is evident from reading writings from the time, particularly those from withoutboats, that there were significant objections to automatic Ok-wrapping. It seems to have become quite political, and some folks burned out on the topic. Perhaps, now, a couple of years later, we can revisit this area and solve this problem in the language itself ? Explicitness An argument I have seen made against automatic Ok-wrapping, and, in general, against any kind of useful language affordance, is that it makes things less explicit. But this argument is fundamentally wrong for Ok-wrapping. Explicitness is not an unalloyed good. We humans have only limited attention. We need to focus that attention where it is actually needed. So explicitness is good in situtions where what is going on is unusual; or would otherwise be hard to read; or is tricky or error-prone. Generally: explicitness is good for things where we need to direct humans attention. But Ok-wrapping is ubiquitous in fallible Rust code. The compiler mechanisms and type systems almost completely defend against mistakes. All but the most novice programmer knows what s going on, and the very novice programmer doesn t need to. Rust s error handling arrangments are designed specifically so that we can avoid worrying about fallibility unless necessary except for the Ok-wrapping. Explicitness about Ok-wrapping directs our attention away from whatever other things the code is doing: it is a distraction. So, explicitness about Ok-wrapping is a bad thing. Appendix - examples showning code with Ok wrapping is worse than code using #[throws] Observe these diffs, from my abandoned attempt to remove the fehler dependency from Hippotat. I have a type alias AE for the usual error type (AE stands for anyhow::Error). In the non-#[throws] code, I end up with a type alias AR<T> for Result<T, AE>, which I think is more opaque but at least that avoids typing out -> Result< , AE> a thousand times. Some people like to have a local Result alias, but that means that the standard Result has to be referred to as StdResult or std::result::Result.

With fehler and #[throws]	Vanilla Rust, Result<>, mandatory Ok-wrapping
Return value clearer, error return less wordy:
impl Parseable for Secret	impl Parseable for Secret
#[throws(AE)]
fn parse(s: Option<&str>) -> Self	fn parse(s: Option<&str>) -> AR<Self>
let s = s.value()?;	let s = s.value()?;
if s.is_empty() throw!(anyhow!( secret value cannot be empty ))	if s.is_empty() return Err(anyhow!( secret value cannot be empty ))
Secret(s.into())	Ok(Secret(s.into()))
No need to wrap whole match statement in Ok( ):
#[throws(AE)]
pub fn client<T>(&self, key: & static str, skl: SKL) -> T	pub fn client<T>(&self, key: & static str, skl: SKL) -> AR<T>
where T: Parseable + Default	where T: Parseable + Default
match self.end	Ok(match self.end
LinkEnd::Client => self.ordinary(key, skl)?,	LinkEnd::Client => self.ordinary(key, skl)?,
LinkEnd::Server => default(),	LinkEnd::Server => default(),
	)
Return value and Ok(()) entirely replaced by #[throws]:
impl Display for Loc	impl Display for Loc
#[throws(fmt::Error)]
fn fmt(&self, f: &mut fmt::Formatter)	fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result
write!(f, :? : , &self.file, self.lno)?;	write!(f, :? : , &self.file, self.lno)?;
if let Some(s) = &self.section	if let Some(s) = &self.section
write!(f, )?;	write!(f, )?;
	Ok(())
Call to write! now looks the same as in more complex case shown above:
impl Debug for Secret	impl Debug for Secret
#[throws(fmt::Error)]
fn fmt(&self, f: &mut fmt::Formatter)	fn fmt(&self, f: &mut fmt::Formatter)-> fmt::Result
write!(f, "Secret(***)")?;	write!(f, "Secret(***)")
Much tiresome return Ok() noise removed:
impl FromStr for SectionName	impl FromStr for SectionName
type Err = AE;	type Err = AE;
#[throws(AE)]
fn from_str(s: &str) -> Self	fn from_str(s: &str) ->AR< Self>
match s	match s
COMMON => return SN::Common,	COMMON => return Ok(SN::Common),
LIMIT => return SN::GlobalLimit,	LIMIT => return Ok(SN::GlobalLimit),
_ =>	_ =>
;	;
if let Ok(n@ ServerName(_)) = s.parse() return SN::Server(n)	if let Ok(n@ ServerName(_)) = s.parse() return Ok(SN::Server(n))
if let Ok(n@ ClientName(_)) = s.parse() return SN::Client(n)	if let Ok(n@ ClientName(_)) = s.parse() return Ok(SN::Client(n))
if client == LIMIT return SN::ServerLimit(server)	if client == LIMIT return Ok(SN::ServerLimit(server))
let client = client.parse().context( client name in link section name )?;	let client = client.parse().context( client name in link section name )?;
SN::Link(LinkName server, client )	Ok(SN::Link(LinkName server, client ))

edited 2022-12-18 19:58 UTC to improve, and 2022-12-18 23:28 to fix, formatting

comments

Every year or so, I revisit the current best practices for Python packaging. This was my summary for 2021 here s the update for 2022. PyPA PyPA is still the place to go for information, best practices and tutorials for packaging Python projects. My only criticism from last year, namely that PyPA was heavily biased towards their own tooling (e.g. pipenv), has been addressed: the tool recommendations section lists now several tools for the same purpose with their own ones not necessarily being the first anymore. setup.py, setup.cfg, requirements.txt, Pipfile, pyproject.toml oh my! This is the reason why I m revisiting the documentation every year, to see what s the current way to go. Good progress has been made since last year: Bye setup.py and setup.cfg hello pyproject.toml pyproject.toml finally got mature enough to replace setup.py and setup.cfg in most cases. Recent versions of setuptools and pip now fully support pyproject.toml and even PyPA s packaging tutorial completely switched their example project from away setup.py towards pyproject.toml, making it an official recommendation. So, now you can replace your setup.py with pyproject.toml. If you had already some kind of declarative configuration in setup.cfg you can move that as well into pyproject.toml. Most tools, like pypy or pytest also support configuration in pyproject.toml (flake8 being a notable exception ) so there s no reason to keep setup.cfg around anymore. Actually, if you migrate to pyproject.toml it is best to do it properly and remove setup.py and setup.cfg as setuptools behaves a bit buggy when building a package that has either of them and the pyproject.toml. requirements.txt requirements.txt are still needed if you develop a deployable application (vs. a library) and want to provide pinned dependencies, i.e. with the specific versions that you ve tested your application with. Usually, the list of requirements in requirements.txt is the same as defined in pyproject.toml, but with pinned versions. Pipfile + Pipfile.lock I still do completely ignore Pipfile and Pipfile.lock as they are only supported by pipenv and not backed by any standard. Summary The major change this year was the proper support of pyproject.toml. I am slowly replacing all setup.py and setup.cfg in my projects with pyproject.toml and haven t discovered any issues yet. Even packaging those packages as Debian packages is well-supported by Debian s tooling. I m still running a quite boring stack based on pyproject.toml and requirements.txt, ignoring more advanced tools like poetry or such for dependency management. My build-system defined in pyproject.toml requires setuptools and I m using build for building and twine for uploading. Since PyPA changed the packaging tutorial towards pyproject.toml and away from setup.py, I think we will slowly see setup.py and setup.cfg go away over the years. Speaking of PyPA, I m happy that they changed their attitude towards a more unbiased recommendation of tooling.

I m currently reading the second book of the TriSolar Sci-Fi series by Cixin Liu, I ve only just started it so this post can t have spoilers for it and I will also only have minimal spoilers for the first book (nothing more than you will get from pop culture references to it). In the second book there are people called Wall Facers who have broad powers to shape the course of the Human response to an alien invasion in 400+ years time. The idea is that as the aliens have an ability to see everything that can be seen on Earth any ideas that leave the brain of one person can be snooped on, so if some people act independently without communicating their plans they can take the aliens by surprise. While that is probably going to work out well in the books history in general seems to show that people who act independently without any useful feedback from others tend to perform poorly, every king and dictator seems to demonstrate this. Efficient Work I ve been thinking about what I would do if I had significant powers to guide the response to an alien threat in some hundreds of years. The first thing to do would be to get all people working as efficiently as possible. Without the imminent threat of alien invasion we can have debates about how much time to spend working vs leisure time. Should we make 24 hours per week the new normal work week? But if the threat of annihilation is looming then the discussion should be about how to get as many people as possible working as much as possible. Currently 1/4 of the world population lack access to safe drinking water [1], there s a plan to achieve universal and equitable access to safe and affordable drinking water for all by 2030 . But 2030 isn t soon enough, another 8 years where 1/4 of children born won t reach their potential due to poor water is unacceptable. Currently 13% of the world population don t have access to electricity and 40% don t have access to clean fuels for cooking [2]. Lack of energy access reduces health and opportunities for education. Healthcare is another major obstacle to human development and therefore economic development. Even some allegedly first-world countries like the US lack universal affordable healthcare. I think we could reasonably get safe water to 99% of the world population before 2025 if we tried hard (IE applied a small fraction of the resources of a single war to it). Getting electricity to 95% of the world population and clean cooking fuels to 90% of the world population are probably achievable goals for 2025 as well. Healthcare is a slightly harder problem as we need to train more nurses and doctors. A registered nurse apparently needs 3 years of training after completing high school. We may have to improve high schools to get more students up to the standard of nursing degrees. If it takes 3 years to improve schools in year 9+ and then 3 years to get more high school graduates that would mean that it would take about 9 years to get an increase in nurses. Doing this would require increasing the capacity of universities and making university almost free (as it was for decades). So in about 2031 we could start sending a significant number of nurses from developed countries to help out developing countries. Becoming a doctor apparently requires 8 years of study plus a minimum of 3 years residency . So if doctors were entirely trained in first world countries then we wouldn t be able to send many doctors to developing countries until 2039. If the residency was performed in other countries then it could be as early as 2036. According to the WHO currently only half the world s population have adequate healthcare [3]. To get adequate healthcare to the world we need to more than double the number of doctors because currently we don t have enough in countries with decent healthcare systems such as Australia. It would probably take to at least 2060 to get enough doctors trained. The end goal of course would be to have every country able to train enough of it s citizens to provide all medical services, but countries that have serious widespread healthcare problems that reduce the number of people who can pursue higher education will have difficulty in that until some of the healthcare problems are alleviated. Education Obviously education is important to all achievements. Currently education seems very poorly run, it is possible to create a school system that teaches children effectively without the bullying that is common in Australia and without the sort of pressure that South Korea is infamous for. One of the main issues to resolve with the school system is the idea that everyone should learn at the same speed, that goal can only be achieved by making the majority of the students learn slowly. Students should be able to freely skip ahead as their skill permits and finish school at any age. Also high school isn t for everyone, the tech schools that teach trades need to be brought back. Deceiving Aliens A plot point in the TriSolar series is that the aliens can see each other s thoughts, the local communication (their equivalent to talking) is based on reading each other s thoughts without the possibility of deception. While deceptive written communication is potentially possible for them they haven t developed skills in that area. As a first step towards exploiting this humans could focus more on linguistic development that increases language complexity, such as the way the English language adopts words from other languages and gives them slightly different meanings for example the difference between driver and chauffeur and the difference between dog and hound is not obvious to many Europeans who otherwise speak English fluently. When involved in conversation it s possible to convey meaning without directly stating things, this is used extensively by people who are interested in security. My observations of this are based on conversations with people who do government work, but I imagine that criminal organisations also do similar things for similar reasons. An increased focus on poetry in schools might be helpful in developing skills for conveying ideas to people who think in human ways where the message is unclear to non-humans who have no experience of deception. I wonder whether the ability to understand human poetry would make aliens less hostile to humans, if they can think like us then they would be less likely to want to exterminate us. Poker is a game that depends on the ability to deceive others, I ve never been any good at it. I wonder if making it part of the school curriculum would help improve the overall human ability to deceive aliens. I don t think that such schools would become dens of sociopathy as depicted in Kakegurui, but it might have some negative results. Spreading education to a larger portion of the world s population requires more use of electronic education. Anything learned via text can be more easily assimilated by aliens than things that are learned directly from other people. For high school and the basics of a university degree this is fine. But for more advanced education it seems that having a large face to face component might help keep the value away from the aliens. More Ideas? What do you think I missed on this list? I wasn t trying to list every possibility, just the more important ones. Also for any goals other than increasing inequality for it s own sake we should improve health and education for the world.

• [1] https://ourworldindata.org/water-access
• [2] https://ourworldindata.org/energy-access
• [3] https://tinyurl.com/y7gyrjwu

The Reproducible Builds project relies on several projects, supporters and sponsors for financial support, but they are also valued as ambassadors who spread the word about our project and the work that we do. This is the sixth instalment in a series featuring the projects, companies and individuals who support the Reproducible Builds project. We started this series by featuring the Civil Infrastructure Platform project and followed this up with a post about the Ford Foundation as well as a recent ones about ARDC, the Google Open Source Security Team (GOSST), Jan Nieuwenhuizen on Bootstrappable Builds, GNU Mes and GNU Guix and Hans-Christoph Steiner of the F-Droid project. Today, however, we will be talking with David A. Wheeler, the Director of Open Source Supply Chain Security at the Linux Foundation.

Holger Levsen: Welcome, David, thanks for taking the time to talk with us today. First, could you briefly tell me about yourself? David: Sure! I m David A. Wheeler and I work for the Linux Foundation as the Director of Open Source Supply Chain Security. That just means that my job is to help open source software projects improve their security, including its development, build, distribution, and incorporation in larger works, all the way out to its eventual use by end-users. In my copious free time I also teach at George Mason University (GMU); in particular, I teach a graduate course on how to design and implement secure software. My background is technical. I have a Bachelor s in Electronics Engineering, a Master s in Computer Science and a PhD in Information Technology. My PhD dissertation is connected to reproducible builds. My PhD dissertation was on countering the Trusting Trust attack, an attack that subverts fundamental build system tools such as compilers. The attack was discovered by Karger & Schell in the 1970s, and later demonstrated & popularized by Ken Thompson. In my dissertation on trusting trust I showed that a process called Diverse Double-Compiling (DDC) could detect trusting trust attacks. That process is a specialized kind of reproducible build specifically designed to detect trusting trust style attacks. In addition, countering the trusting trust attack primarily becomes more important only when reproducible builds become more common. Reproducible builds enable detection of build-time subversions. Most attackers wouldn t bother with a trusting trust attack if they could just directly use a build-time subversion of the software they actually want to subvert.
Holger: Thanks for taking the time to introduce yourself to us. What do you think are the biggest challenges today in computing? There are many big challenges in computing today. For example:

• Lack of resilience & capacity in chip fabrication. Fabs are extraordinarily expensive, and at the high end continue to have technological advancement. As a result, supply is failing to meet demand, and geopolitical issues raise further concerns. We ve seen cars, gaming consoles and many other devices unable to be delivered due to chip shortages. More fabs are being built, and some politicians are raising concerns, but it s unclear that current efforts will be enough.
• Lack of enough developers able to develop the software that people & organizations need. Computers are far faster, and open source software has made software reuse incredibly easy. However, organizations still struggle to automate many tasks. The bottleneck is the lack of enough talented developers able to convert ideas into working software. Low-code and no-code approaches help in specialized areas, just like all previous automate the programmer efforts of the last 60 years, but there s no reason to believe they will help enough.
• Large scale of software. Small systems are easier to develop & maintain, but today s systems increasingly get bigger to meet users needs & are much harder to manage. Even small embedded systems are often supported by huge back-end systems.
• Ending tail of Moore s law & rise of smartphones. Historically people would just wait a few years for their software to speed up, but Moore s law is petering out, and smartphones are necessarily limited by power & size limits. As a result, software developers can t wait for the hardware to save their slow systems; they must redesign. Switching to faster languages, or using multiple processors, is much more difficult than waiting for performance problems to disappear.
• Continuous change in interfaces. Developers continuously find reasons to change component interfaces: perhaps they re too inflexible, too hard to use, and so on. But now that developers are reusing hundreds, thousands, or tens of thousands of components, managing the continuous change of the reused components is challenging. Package managers make updating easy but don t automatically handle interface changes. I think this is mostly a self-inflicted problem most components could support old interfaces (like the Linux kernel does) but because it s often not acknowledged as a problem, it s often not addressed.
• Security & privacy. Decades ago there were fewer computers and most computers weren t connected to a network. Today things are different. Criminals have found many ways to attack computer systems to make money, and nation-states have found many ways to attack computer systems for their own reasons. Attackers now have very strong motivations to perform attacks. Yet many developers aren t told how to develop software that resists attacks, nor how to protect their supply chains. Operations try to monitor and recover from attacks, but their job is difficult due to inadequately secure software that doesn t support those monitoring & recovery efforts well either. The results are terrible security.

Holger: Do you think reproducible builds are an important part in secure computing today already? David: Yes, but first let s put things in context. Today, when attackers exploit software vulnerabilities, they re primarily exploiting unintentional vulnerabilities that were created by the software developers. There are a lot of efforts to counter this:

• Train & education developers in how to develop secure software. The OpenSSF provides a free course on how to do that (full disclosure: I m the author). Take that course or something like it!
• Add tools to your CI pipeline to detect potential vulnerabilities. Yes, they have false positives and false negatives, so you have to also use your brain but that just means you need to be smart about using tools, instead of not using them.
• Get projects & organizations to update the components they use, since often the vulnerabilities are well-known publicly (e.g., Equifax in 2017). Add some tools to your development process to warn you about components with known vulnerabilities! GitHub & GitLab both provide tools to do this, and there are many other tools.
• When starting new projects, try to use memory-safe languages. On average 70% of the vulnerabilities in Chrome and in Microsoft are from memory safety problems; using a memory-safe language eliminates most of them.

We re just starting to get better at this, which is good. However, attackers always try to attack the easiest target. As our deployed software has started to be hardened against attack, attackers have dramatically increased their attacks on the software supply chain (Sonatype found in 2022 that there s been a 742% increase year-over-year). The software supply chain hasn t historically gotten much attention, making it the easy target. There are simple supply chain attacks with simple solutions:

• In almost every year the top attack has been typosquatting. In typo squatting, an attacker creates packages with almost the right name. This is an easy attack to counter developers just need to double-check the name of a package before adding it. But we aren t warning developers enough about it! For more information, see papers such as the Backstabber s Knife Collection.
• Last year the top software supply chain attack was dependency confusion convincing projects to use the wrong repo for a given package. There are simple solutions to this, such as specifying the package source and/or requiring a cryptographic hash to match.
• Some attacks involve takeovers of developer accounts. In almost all cases, these are caused by stolen passwords. Using a multi-factor authentication (MFA) token eliminates stolen password attacks, which is why several repositories are starting to require MFA tokens in some cases.

Unfortunately, attackers know there are other lines of attack. One of the most dangerous is subverted build systems, as demonstrated by the subversion of SolarWinds Orion system. In a subverted build system, developers can review the software source code all day and see no problem, because there is no problem there. Instead, the process to convert source code into the code people run, called the build system , is subverted by an attacker. One solution for countering subverted build systems is to make the build systems harder to attack. That s a good thing to do, but you can never be confident that it was good enough . How can you be sure it s not subverted, if there s no way to know? A stronger defense against subverted build systems is the idea of verified reproducible builds. A build is reproducible if given the same source code, build environment and build instructions, any party can recreate bit-by-bit identical copies of all specified artifacts. A build is verified if multiple different parties verify that they get the same result for that situation. When you have a verified reproducible build, either all the parties colluded (and you could always double-check it yourself), or the build process isn t subverted. There is one last turtle: What if the build system tools or machines are subverted themselves? This is not a common attack today, but it s important to know if we can address them when the time comes. The good news is that we can address this. For some situations reproducible builds can also counter such attacks. If there s a loop (that is, a compiler is used to generate itself), that s called the trusting trust attack, and that is more challenging. Thankfully, the trusting trust attack has been known about for decades and there are known solutions. The diverse double-compiling (DDC) process that I explained in my PhD dissertation, as well as the bootstrappable builds process, can both counter trusting trust attacks in the software space. So there is no reason to lose hope: there is a bottom turtle , as it were.
Holger: Thankfully, this has all slowly started to change and supply chain issues are now widely discussed, as evident by efforts like Securing the Software Supply Chain: Recommended Practices Guide for Developers which you shared on our mailing list. In there, Reproducible Builds are mentioned as recommended advanced practice, which is both pretty cool (we ve come a long way!), but to me it also sounds like this will take another decade until it s become standard normal procedure. Do you agree on that timeline? David: I don t think there will be any particular timeframe. Different projects and ecosystems will move at different speeds. I wouldn t be surprised if it took a decade or so for them to become relatively common there are good reasons for that. Today the most common kinds of attacks based on software vulnerabilities still involve unintentional vulnerabilities in operational systems. Attackers are starting to apply supply chain attacks, but the top such attacks today are typosquatting (creating packages with similar names) and dependency confusion) (convincing projects to download packages from the wrong repositories). Reproducible builds don t counter those kinds of attacks, they counter subverted builds. It s important to eventually have verified reproducible builds, but understandably other issues are currently getting prioritized first. That said, reproducible builds are important long term. Many people are working on countering unintentional vulnerabilities and the most common kinds of supply chain attacks. As these other threats are countered, attackers will increasingly target build systems. Attackers always go for the weakest link. We will eventually need verified reproducible builds in many situations, and it ll take a while to get build systems able to widely perform reproducible builds, so we need to start that work now. That s true for anything where you know you ll need it but it will take a long time to get ready you need to start now.
Holger: What are your suggestions to accelerate adoption? David: Reproducible builds need to be:

• Easy (ideally automatic). Tools need to be modified so that reproducible builds are the default or at least easier to do.
• Transparent to projects & potential users. Many projects have no idea that their results aren t reproducible, and many potential users of the project don t know either. That information needs to be obvious. I ve proposed that the OpenSSF Dashboard SIG try to reproduce builds, for at least some packages, to make it more obvious to everyone when a project isn t reproducible. I don t know if that will happen in that particular case, but the point is to help people learn that information as soon as possible.
• Deployed. Experiments are great, but experiments showing that a project could be reproducible are inadequate. We need the projects that people use to be reproducible.

I think there s a snowball effect. Once many projects packages are reproducible, it will be easier to convince other projects to make their packages reproducible. I also think there should be some prioritization. If a package is in wide use (e.g., part of minimum set of packages for a widely-used Linux distribution or framework), its reproducibility should be a special focus. If a package is vital for supporting some societally important critical infrastructure (e.g., running dams), it should also be considered important. You can then work on the ones that are less important over time.
Holger: How is the Best Practices Badge going? How many projects are participating and how many are missing? David: It s going very well. You can see some automatically-generated statistics, showing we have over 5,000 projects, adding more than 1/day on average. We have more than 900 projects that have earned at least the passing badge level.
Holger: How many of the projects participating in the Best Practices badge engaging with reproducible builds? David: As of this writing there are 168 projects that report meeting the reproducible builds criterion. That s a relatively small percentage of projects. However, note that this criterion (labelled build_reproducible) is only required for the gold badge. It s not required for the passing or silver level badge. Currently we ve been strategically focused on getting projects to at least earn a passing badge, and less on earning silver or gold badges. We would love for all projects to get earn a silver or gold badge, of course, but our theory is that projects that can t even earn a passing badge present the most risk to their users. That said, there are some projects we especially want to see implementing higher badge levels. Those include projects that are very widely used, so that vulnerabilities in them can impact many systems. Examples of such projects include the Linux kernel and curl. In addition, some projects are used within systems where it s important to society that they not have serious security vulnerabilities. Examples include projects used by chemical manufacturers, financial systems and weapons. We definitely encourage any of those kinds of projects to earn higher badge levels.
Holger: Many thanks for this interview, David, and for all of your work at the Linux Foundation and elsewhere!

---

For more information about the Reproducible Builds project, please see our website at reproducible-builds.org. If you are interested in ensuring the ongoing security of the software that underpins our civilisation and wish to sponsor the Reproducible Builds project, please reach out to the project by emailing contact@reproducible-builds.org.

Version 0.0.11 of RcppSpdlog is now on CRAN and in Debian. RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovich. This release adds support for a basic file logger as a alternative to the console logger. This can be helpful with code which suppresses or hides console output as for example unit test code does. We also expose the formatting helper function for direct use at the C level from other packages, and mention the handy wrapper spdl in the README. The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.11 (2022-12-13)

• Export the formatter at C level
• Mention package spdl in README.md
• Support simple file-based logger

Courtesy of my CRANberries, there is also a diffstat report. More detailed information is on the RcppSpdlog page, or the package documention site. If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Ever since the demise of Google Reader I have been looking for a suitable replacement RSS reader. In the past I used to use Liferea but that was when I used a single desktop machine; these days I want to be able to read on my phone and multiple machines. I moved to Feedly and it s been mostly ok, but I m hitting the limit of feeds available in the free tier, and $72/year is a bit more than I can justify to myself. Especially when I have machines already available to me where I could self host something. The problem, of course, is what to host. It seems the best options are all written in PHP, so I had to get over my adverse knee-jerk reaction to that. I ended up on FreshRSS but if it hadn t worked out I d have tried TinyTinyRSS. Of course I m hosting on Debian, and the machine I chose to use was already running nginx and PostgreSQL. So I needed to install PHP: I put my FreshRSS install in /srv/freshrss so I grabbed the 1.20.2 release from GitHub (actually 1.20.1 at the time, but I ve upgraded to the latest since) and untared it in there. I gave www-data access to the data directory (sudo chown -R www-data /srv/freshrss/data) (yes, yes, I could have created a new user specifically for FreshRSS, but I ve chosen not to for now). There s no actual need to configure things up on the filesystem, you can do the initial setup from the web interface. Which is where the trouble came. I ve been an Apache user since 1998 and as a result it s what I know and what I go to. nginx is new to me. And I wanted my FreshRSS instance to live in a subdirectory of an existing TLS-enabled host, rather than have it s own hostname. Now, at least FreshRSS copes with this (unlike far too many other projects), you just have to configure your webserver correctly. Which took me more experimentation than I d like, but I ve ended up with the following snippet: Other than the addition of the freshrss prefix this ends up differing slightly from the FreshRSS webserver configuration example. I ended up having to make the path info on the fastcgi_split_path_info optional, and my try_files in the bare directory location directive needed $is_args$args added or I just ended up in a redirect loop because the session parameters didn t get passed through. I m sure there s a better way to do it, but I did a bunch of searching and this is how I ended up making it work. Before firing up the web configuration I created a suitable database: I ran through the local configuration, creating myself a user and adding some feeds, then created a cronjob to fetch updates hourly and keep a log: Experiences so far? Reasonably happy. The interface seems snappy enough, and works well both on mobile and desktop. I m only running a single user instance at present, but am considering opening it up to some other folk and will see how that scales. And it clearly indicated a number of my feeds that were broken, so I ve cleaned some up that are still around and deleted the missing ones. Now I just need to figure out what else I should be subscribed to that I ve been putting off due to the Feedly limit!

(quoted from my other blog at since a new OS might be interesting for many and this is published in separate planets)
ALP - The Adaptable Linux Platform is a new operating system from SUSE to run containerized and virtualized workloads. It is in early prototype phase, but the development is done completely openly so it s easy to jump in to try it.For this trying out, I used the latest encrypted build as of the writing, 22.1 from ALP images. I imported it in virt-manager as a Generic Linux 2022 image, using UEFI instead of BIOS, added a TPM device (which I m interested in otherwise) and referring to an Ignition JSON file in the XML config in virt-manager.The Ignition part is pretty much fully thanks to Paolo Stivanin who studied the secrets of it before me. But here it goes - and this is required for password login in Cockpit to work in addition to SSH key based login to the VM from host - first, create config.ign file: where password SHA512 hash can be obtained using openssl passwd -6 and the ssh key is your public ssh key.That file is put to eg /tmp and referred in the virt-manager s XML like follows:Now we can boot up the VM and ssh in - or you could log in directly too but it s easier to copy-paste commands when using ssh.Inside the VM, we can follow the ALP documentation to install and start Cockpit:Check your host s IP address with ip -a, and open IP:9090 in your host s browser:Login with root / your password and you shall get the front page: and many other pages where you can manage your ALP deployment via browser:All in all, ALP is in early phases but I m really happy there s up-to-date documentation provided and people can start experimenting it whenever they want. The images from the linked directory should be fairly good, and test automation with openQA has been started upon as well.You can try out the other example workloads that are available just as well.

I m about to migrate to a new laptop, having done a brief pre-purchase review of options on Fosstodon and reaching a decision to buy the NovaCustom NV41. Given the rapid launch and decline of Mastodon instances, I thought I d better summarize my process and conclusion on my self-hosted blog until the fediverse self-hosting situation improves. Since 2010 my main portable computing device has been the Lenovo X201 that replaced the Dell Precision M65 that I bought in 2006. I have been incredibly happy with the X201, even to the point that in 2015 when I wanted to find a replacement, I couldn t settle on a decision and eventually realized I couldn t articulate what was wrong with the X201 and decided to just buy another X201 second-hand for my second office. There is still no deal-breaker with the X201, and I m doing most of my computing on it including writing this post. However, today I can better articulate what is lacking with the X201 that I desire, and the state of the available options on the market has improved since my last attempt in 2015. Briefly, my desired properties are:

• Portable weight under 1.5kg
• Screen size 9-14
• ISO keyboard layout, preferably Swedish layout
• Mouse trackpad, WiFi, USB and external screen connector
• Decent market availability: I should be able to purchase it from Sweden and have consumer protection, warranty, and some hope of getting service parts for the device
• Manufactured and sold by a vendor that is supportive of free software
• Preferably RJ45 connector (for data center visits)
• As little proprietary software as possible, inspired by FSF s Respect Your Freedom
• Able to run a free operating system

My workload for the machine is Emacs, Firefox, Nextcloud client, GNOME, Evolution (mail & calendar), LibreOffice Calc/Writer, compiling software and some podman/qemu for testing. I have used Debian as the main operating system for the entire life of this laptop, but have experimented with PureOS recently. My current X201 is useful enough for this, although support for 4K displays and a faster machine wouldn t hurt. Based on my experience in 2015 that led me to make no decision, I changed perspective. This is a judgement call and I will not be able to fulfil all criteria. I will have to decide on a balance and the final choice will include elements that I really dislike, but still it will hopefully be better than nothing. The conflict for me mainly center around these parts:

• Non-free BIOS. This is software that runs on the main CPU and has full control of everything. I want this to run free software as much as possible. Coreboot is the main project in this area, although I prefer the more freedom-oriented Libreboot.
• Proprietary and software-upgradeable parts of the main CPU. This includes CPU microcode that is not distributed as free software. The Intel Management Engine (AMD and other CPU vendors has similar technology) falls into this category as well, and is problematic because it is an entire non-free operating system running within the CPU, with many security and freedom problems. This aspect is explored in the Libreboot FAQ further. Even if these parts can be disabled (Intel ME) or not utilized (CPU microcode), I believe the mere presence of these components in the design of the CPU is a problem, and I would prefer a CPU without these properties.
• Non-free software in other microprocessors in the laptop. Ultimately, I tend agree with the FSF s secondary processor argument but when it is possible to chose between a secondary processor that runs free software and one that runs proprietary software, I would prefer as many secondary processors as possible to run free software. The libreboot binary blob reduction policy describes a move towards stronger requirements.
• Non-free firmware that has to be loaded during runtime into CPU or secondary processors. Using Linux-libre solves this but can cause some hardware to be unusable.
• WiFi, BlueTooth and physical network interface (NIC/RJ45). This is the most notable example of secondary processor problem with running non-free software and requiring non-free firmware. Sometimes these may even require non-free drivers, although in recent years this has usually been reduced into requiring non-free firmware.

A simple choice for me would be to buy one of the FSF RYF certified laptops. Right now that list only contains the 10+ year old Lenovo series, and I actually already have a X200 with libreboot that I bought earlier for comparison. The reason the X200 didn t work out as a replacement for me was the lack of a mouse trackpad, concerns about non-free EC firmware, Intel ME uncertainty (is it really neutralized?) and non-free CPU microcode (what are the bugs that it fixes?), but primarily that for some reason that I can t fully articulate it feels weird to use a laptop manufactured by Lenovo but modified by third parties to be useful. I believe in market forces to pressure manufacturers into Doing The Right Thing, and feel that there is no incentive for Lenovo to use libreboot in the future when this market niche is already fulfilled by re-sellers modifying Lenovo laptops. So I d be happier buying a laptop from someone who is natively supportive of they way I m computing. I m sure this aspect could be discussed a lot more, and maybe I ll come back to do that, and could even reconsider my thinking (the right-to-repair argument is compelling). I will definitely continue to monitor the list of RYF-certified laptops to see if future entries are more suitable options for me. Eventually I decided to buy the NovaCustom NV41 laptop, and it arrived quickly and I m in the process of setting it up. I hope to write a separate blog about it next.

It s been nearly 8 years since I last made choices about music playing. At the time, I picked Logitech Media Server (LMS, aka Slimserver and Squeezebox server) for whole-house audio and Ampache with the DSub Android app. It s time to revisit that approach. Here are the things I m looking for:

• Whole-house audio: a single control point for all the speakers in the house, which are all connected to some form of Linux (Raspberry Pi or x86). The speakers should be reasonably in sync with each other, and the control point should be able to adjust volume on them centrally. I should be able to play albums, playlists, etc. on them, and skip tracks or seek within a track.
• The ability to stream to an Android mobile device, ideally with downloading capabilities for offline use.
• If multiple solutions are used, playlist syncing between them.
• Ideally, bookmark support to resume playing a long track where it was left off.
• Ideally, podcast support.

The current setup Here are the current components:

• Logitech Media Server, which serves the music library for whole-house synchronized audio
• Squeezelite is the LMS client running on my Raspberry Pi and x86 systems
• Squeezer is a nice Android client for LMS to control playback, adjust volume, etc. It doesn t do any playback on the Android device, of course.
• Ampache provides the server for streaming clients, both browser-based and mobile
• DSub (F-Droid, Play Store) is a nice Android client for Ampache providing streaming and offline playback

LMS makes an excellent whole-house audio system. I can pull up the webpage (or use an Android app like Squeezer) to browse my music library, queue things up to play, and so forth. I can also create playlists, which it saves as m3u files. This whole setup is boringly reliable. It just works, year in, year out. The main problem with this is that LMS has no real streaming/offline mobile support. It is also a rather dated system, with a painful UI for playlist management, and in general doesn t feel very modern. (It s written largely in Perl also!) So, I paired with it is Ampache. As a streaming player, Ampache is fantastic; I can access it from a web browser, and it will transcode my FLAC files to the quality I ve set in my user prefs. The DSub app for Android is fantastic and remembers my last-play locations and such. The problem is that Ampache doesn t write its playlists back to m3u format, so I can t use them with LMS. I have to therefore maintain all the playlists in LMS, and it has a smallish limit on the number of tracks per playlist. Ampache also doesn t auto-update from LMS playlists, so I have to delete and recreate the playlists catalog periodically to get updates into Ampache. Not fun. The new experiment I m trying out a new system based on these components:

• Jellyfin is a media player. It supports not just music, but also video (in fact, the emphasis is more on video). Notably it supports controlling various devices. Its normal frontend is a web browser; Jellyfin s server won t output audio to a device itself.
• Mopidy is a media player with a web interface that does output audio to a local device. In normal use, it displays an interface to your music, letting you select, queue up, etc.
• Mopidy-Jellyfin (docs) is a plugin for Mopidy that enables two things: 1) Browsing the Jellyfin library within Mopidy, and 2) controlling Mopidy from within Jellyfin. Mode 1 barely works, but mode 2 works perfectly. Within Jellyfin, I can cast to Mopidy and queue up things, seek, skip tracks, etc.
• Snapcast is a generic solution to take audio from some sort of source and distribute it throughout the house, syncing each device (and with better syncing than LMS, too!). The source can be just about anything, and the docs include an example of how to set it up with Mopidy.
• Mopidy has selectable web interfaces, and the Mopidy-Muse interface has the added benefit of having integrated Snapcast control. (Mopidy-Iris does as well, though it wasn t documented there.) Within it, I can adjust volume on devices, mute devices, etc. I could also use the Snapcast web interface for this purpose.
• The default Jellyfin Android app lets me stream media to the mobile device, as well as control the Mopidy player.
• Finamp (F-Droid, Play Store) is a very nice Android Jellyfin music playing client, which notably supports downloads for offline playing, a feature the stock app lacks.
• The Snapcast Android app (F-Droid, Play Store) isn t strictly necessary, since the Snapcast web app is so simple to use. But it provides near-instant control of speakers and volumes.

This looks a lot more complicated than what I had before, but in reality it only has one additional layer. Since Snapcast is a general audio syncing tool, and Jellyfin doesn t itself output audio, Mopidy and its extensions is the glue . There s a lot to like about this setup. There is one single canonical source for music and playlists. Jellyfin can do a lot more besides music, and its mobile app gives me video access also. The setup, in general, works pretty well. There are a few minor glitches, but nothing huge. For instance, Jellyfin fails to clear the play queue on the mopidy side. But there is one problem, though: when playing a playlist, it is played out of order. Jellyfin itself has the same issue internally, so I m unsure where the bug lies. Rejected option: Jellyfin with jellycli This could be a nice option; instead of mopidy with a plugin, just run jellycli in headless mode as a more native client. It also has the playlist ordering bug, and in addition, fails to play a couple of my albums which Mopidy-Jellyfin handles fine. But, if those bugs were addressed, it has a ton of promise as a simpler glue between Jellyfin and Snapcast than Mopidy. Rejected option: Mopidy-Subidy Plugin with Ampache Mopidy has a Subsonic plugin, and Ampache implements the Subsonic API. This would theoretically let me use a Mopidy client to play things on the whole-house system, coming from the same Ampache system. Although I did get this connected with some trial and error (legacy auth on, API version 1.13.0), it was extremely slow. Loading the list of playlists took minutes, the list of albums and artists many seconds. It didn t cache any answers either, so it was unusably slow. Rejected option: Ampache localplay with mpd Ampache has a feature called localplay which allows it to control a mpd server. I tested this out with mpd and snapcast. It works, but is highly limited. Basically, it causes Ampache to send a playlist a literal list of URLs to the mpd server. Unfortunately, seeking within a track is impossible from within the Ampache interface. I will note that once a person is using mpd, snapcast makes a much easier whole-house solution than the streaming option I was trying to get working 8 years ago.

Wayland First up, I read Antoine Beaupr s Wayland to Sway migration with interest. While he said it s done and dusted or something similar, the post shows there s still quite a ways to go. I wouldn t say it s done or whatever till it s integrated so well that a person installs it and doesn t really need to fiddle with config files as an average user. For specific use-cases you may need to, but that should be outside of a normal user (layperson) experience. I have been using mate for a long long time and truth be told been very happy with it. The only thing I found about Wayland on mate is this discussion or rather this entry. The roadmap on Ubuntu Mate is also quite iffy. The Mate Wayland entry on Debian wiki also perhaps need an updation but dunno much as the latest update it shares is 2019 and it s 2022. One thing to note, at least according to Antoine, things should be better as and when it gets integrated even on legacy hardware. I would be interested to know how it would work on old desktops and laptops rather than new or is there some barrier? I, for one would have liked to see or know about why lightdm didn t work on Wayland and if there s support. From what little I know lightdm is much lighter than gdm3 and doesn t require much memory and from what little I have experienced works very well with mate. I have been using it since 2015/16 although the Debian changelog tells me that it has been present since 2011. I was hoping to see if there was a Wayland specific mailing list, something like debian-wayland but apparently there s not :(. Using mate desktop wayland (tried few other variations on the keywords) but search fails to find any meaningful answer :(. FWIW and I don t know the reason why but Archwiki never fails to amaze me. Interestingly, it just says No for mate. I probably would contact upstream in the coming days to know what their plans are and hopefully they will document what their plans are on integrating Wayland in both short-term and long-term with an update, or if there is something more recent they have documented elsewhere, get that update on the Debian wiki so people know. The other interesting thread I read was Russel Coker s Thinkpad X1 Carbon Gen5 entry. I would be in the market in a few months to find/buy a Thinkpad but probably of AMD rather than Intel because part of recent past history with Intel as well as AMD having a bit of an edge over Intel as far as graphics is concerned. I wonder why Russel was looking into Intel and not AMD. Would be interested to know why Intel and not AMD? Any specific reason ???

Hearing Aids I finally bought hearing aids about a couple of weeks back and have been practicing using them. I was able to have quite a few conversations although music is still I m not able to listen clearly but it is still a far cry from before and for the better. I am able to have conversations with people and also reply and they do not have to make that extra effort that they needed to. Make things easier for everybody. The one I bought is at the starting range although the hearing aids go all the way to 8 lakhs for a pair (INR 800,000), the more expensive ones having WiFi, Bluetooth and more channels, it all depends on how much can one afford. And AFAIK there is not a single Indian manufacturer who is known in this business.

One thing I did notice is while the hearing aids are remarkably sturdy if they fall down as they are small, yet you have to be careful of both dust and water . That does makes life a bit difficult as my house and city both gets sand quite a bit everyday. I don t think they made any India-specific changes, if they had, would probably make things better. I haven t yet looked at it, but it may be possible to hack it remotely. There may or may not be security issues involved, probably would try once I ve bit more time am bit more comfortable to try and see what I can find out. If I had bought it before, maybe I would have applied for the Debian event happening in Kerala, if nothing else, would have been to document what happened there in detail. I probably would have to get a new motherboard for my desktop probably in a year or two as quite a few motherboards also have WiFi (WiFi 6 ?) think on the southbridge. I at least would have a look in new year and know more as to what s been happening. For last at least 2-3 years there has been a rumor which has been confirmed time and again that the Tata Group has been in talks with multiple vendors to set chip fabrication and testing business but to date they haven t been able to find one. They do keep on giving press conferences about the same but that s all they do :(. Just shared the latest one above.

The Long War Terry Pratchett, Stephen Braxter Long Earth Terry Pratchett, Stephen Braxter ISBN13: 9780062067777 Last month there was also a seconds books sale where I was lucky enough to get my hands on the Long War. But before I share about the book itself, I had a discussion with another of my friends and had to re-share part of that conversation. While the gentleman was adamant that non-fiction books are great, my point as always is both are equal. As I shared perhaps on this blog itself, perhaps multiple times, that I had seen a YT video in which a professor shared multiple textbooks of physics and shared how they are wrong and have been wrong and kept them in a specific corner. He took the latest book which he honestly said doesn t have any mistakes as far as he know and yet still kept in that same corner denoting that it is highly possible that future understanding will make the knowledge or understanding we know different. An example of physics in the nano world and how that is different and basically turns our understanding than what we know. Now as far as the book is concerned, remember Michael Crichton s Timeline. Now that book was originally written in the 1960 s while this one was written by both the honorable gentleman in 2013. So almost 50+ years difference between the two books, and that even shows how they think about things. In this book, you no longer need a big machine, but have something called a stepper machine which is say similar to a cellphone, that size and that frame, thickness etc. In this one, the idea of multiverse is also there but done a tad differently. In this, we do not have other humans or copy humans but have multiple earths that may have same or different geography as how evolution happened. None of the multiverse earths have humans but have different species depending on the evolution that happened there. There are something called as trolls but they have a much different meaning and way about them about how most fantasy authors portray trolls. While they are big in this as well, they are as gentle as bears or rabbits. So the whole thing is about real estate and how humans have spread out on multiple earths and the politics therein. Interestingly, the story was trashed or given negative reviews on Goodreads. The sad part is/was that it was written and published in 2013 when perhaps the possibility of war or anything like that was very remote especially in the States, but now we are now in 2022 and just had an insurrection happen and whole lot of Americans are radicalized, whether you see the left or the right depending on your ideology. An American did share few weeks ago how some shares are looking at Proportional Representation and that should make both parties come more towards the center and be a bit more transparent. What was interesting to me is the fact that states have much more rights to do elections and electioneering the way they want rather than a set model which everyone has common which is what happens in India. This also does poke holes into the whole Donald Trump stolen democracy drama but that s a different story altogether. One of the more interesting things I came to know about is that there are 4 books in the long series and this was the second book in itself. I do not want to dwell on the characters themselves as frankly speaking I haven t read all the four books and it would be gross injustice on my part to talk about the characters themselves. Did I enjoy reading the book, for sure. What was interesting and very true of human nature is that even if we have the ability or had the ability to have whole worlds to ourselves, we are bound to mess it up. And in that aspect, I don t think he is too far off the mark. If I had a whole world, wouldn t I try to exploit it to the best or worse of my ability. One of the more interesting topics in the book is the barter system they have thought of that is called as favors. If you are in multiple worlds, then having a currency, even fiat money is of no use and they have to find ways and means to trade with one another. The book also touches a bit on slavery but only just and doesn t really explore it as much as it could have.

Identity Now this has many meanings to it. Couple of weeks ago, saw a transgender meet. For the uninitiated or rather people like me, basically it is about people who are born in one gender but do not identify with it but the other and they express it first through their clothes and expression and the end of the journey perhaps is with having the organs but this may or may not be feasible, as such surgery is expensive and also not available everywhere. After section 377 was repealed few years ago, we do have a third gender on forms as well as have something called a Transgender Act but how much the needle has moved in society is still a question. They were doing a roadshow near my house hence I was able to talk with them with my new hearing aids and while there was lot of traffic was able to understand some of their issues. For e.g. they find it difficult to get houses on rent, but then it is similar for bachelor guys or girls also. One could argue to what degree it is, and that perhaps maybe. Also, there is a myth that they are somehow promiscuous but that I believe is neither here or there. Osho said an average person thinks about the opposite sex every few seconds or a minute. I am sure even Freud would have similar ideas. So, if you look in that way everybody is promiscuous as far as thought is concerned. The other part being opportunity but that again is function of so many other things. Some people are able to attract a lot of people, others might not. And then whether they chose to act on that opportunity or not is another thing altogether. Another word that is or was used is called gender fluid, but that too is iffy as gender fluid may or may not mean transgender. Also, while watching some nature documentary few days/weeks back had come to know that trees have something like 18 odd genders. That just blows me out of the mind and does re-question this whole idea of sexuality and identity to only two which seems somewhat regressive at least to me. If we think humans are part of nature, then we need to be open up perhaps a bit more. But identity as I shared above has more than one meaning. For e.g. citizenship, that one is born in India is even messier to know, understand and define. I had come across this article about couple of months back. Now think about this. Now, there have been studies and surveys about citizenship and it says something like 60% birth registrations are done in metro cities. Now Metro cities are 10 as defined by Indian state. But there are roughly an odd 4k cities in India and probably twice the number of villages and those are conservative numbers as we still don t record things meticulously, maybe due to the Indian oral tradition or just being lazy or both, one part is also that if you document people and villages and towns, then you are also obligated to give them some things as a state and that perhaps is not what the Indian state wants. A small village in India could be anywhere from few hundreds of people to a few thousand. And all the new interventions whether it is PAN, Aadhar has just made holes rather than making things better. They are not inclusive but exclusive. And none of this takes into account Indian character and the way things are done in India. In most households, excluding the celebs (they are in a world of pain altogether when it comes to baby names but then it s big business but that s an entire different saga altogether, so not going to touch that.) I would use or say my individual case as that is and seems to be something which is regular even today. I was given a nickname when I was 3 years old and given a name when I was 5-6 when I was put in school. I also came to know in school few kids who didn t like their names and couple of them cajoled and actually changed their names while they were kids, most of us just stayed with what we got. I do remember sharing about nakushi or something similar a name given to few girls in Maharashtra by their parents and the state intervened and changed their names. But that too is another story in itself. What I find most problematic is that the state seems to be blind, and this seems to be by design rather than a mistake. Couple of years back, Assam did something called NRC (National Register of Citizens) and by the Govt s own account it was a failure of massive proportions. And they still want to bring in CAA, screwing up Assam more. And this is the same Govt. went shown how incorrect it was, blamed it all on the High Court and it s the same Govt. that shopped around for judges to put somebody called Mr. Saibaba (an invalid 90 year adivasi) against whom the Govt. hasn t even a single proof as of date. Apparently, they went to 6 judges who couldn t give what the decision the Govt. wanted. All this info. is in public domain. So the current party ruling, i.e. BJP just wants to make more divisions rather than taking people along as they don t have answers either on economy, inflation or issues that people are facing. One bright light has been Rahul Gandhi who has been doing a padhyatra (walking) from Kanyakumari to Kashmir and has had tremendous success although mainstream media has showed almost nothing what he is doing or why he is doing that. Not only he had people following him, there are and were many who took his example and using the same values of inclusiveness are walking where they can. And this is not to do with just a political party but more with a political thought of inclusiveness, that we are one irrespective of what I believe, eat, wear etc. And that gentleman has been giving press conferences while our dear P.M. even after 8 years doesn t have the guts to do a single press conference. Before closing, I do want to take another aspect, Rahul Gandhi s mother is an Italian or was from Italy before she married. But for BJP she is still Italian. Rishi Sunak, who has become the UK Prime Minister they think of him as Indian and yet he has sworn using the Queen s name. And the same goes for Canada Kumar (Akshay Kumar) and many others. How the right is able to blind and deaf to what it thinks is beyond me. All these people have taken an oath in the name of the Queen and they have to be loyal to her or rather now King Charles III. The disconnect continues.

This is the report for the Debian Python Team remote sprint that took place on December 2-3-4 2022. Many thanks to those who participated, namely:

• tienne Mollier (emollier)
• Taihsiang Ho (tai271828)
• Athos Ribeiro (athos)
• Stuart Prescott (stuart)
• Louis-Philippe V ronneau (pollo)
• Ileana Dumitrescu (ildumi)
• James Valleroy (jvalleroy)
• Emmanuel Arias (eamanu)
• Kurt Kremitzki (kkremitzki)
• Mohammed Bilal (rmb)
• Stefano Rivera (tumbleweed)
• Jeroen Ploemen (jcfp)

Here is a list of issues we worked on: pybuild autodep8 feature About a year ago, Antonio Terceiro contributed code to pybuild to make it possible to automatically run the upstream test suite as autopkgtests. This feature has now been merged and uploaded to unstable. Although you can find out more about it in the pybuild-autopkgtest manpage, an email providing more details should be sent to the debian-python mailing list relatively soon. Fixing packages that run tests via python3 setup.py test Last August, Stefano Rivera poked the team about the deprecation of the python3 setup.py test command to run tests in pybuild. Although this feature has been deprecated upstream for 6 years now, many packages in the archive still use it to run the upstream test suite during build. Around 29 of the 67 packages that are team-maintained by the Debian Python Team were fixed during the sprint. Ideally, all of them would be before the feature is removed from pybuild. if a package you maintain still runs this command, please consider fixing it! Fixing packages that use nose nose, provided by the python3-nose package, is an obsolete testing framework for Python and has been unmaintained since 2015. During the sprint, people worked on fixing some of the many bugs filled against packages still running tests via nose, but there are still around 240 packages affected by this issue in the archive. Again, if a package you maintain still runs this command, please consider fixing it! Removal of the remaining Python2 packages With the upload of dh-python 5.20221202, Stefano Rivera officially removed support for dh_python2 and dh_pypy, thus closing the "Python2 removal in sid/bullseye" bug. It seems some work still needs to be done for complete Python2 removal from Sid, but I expect this will be done in time for the Bookworm release. Working on Lintian tags for the Team During the sprint, I managed to work on some Lintian issues that we had targeted, namely:

• the addition of a "missing-cpython-extension" tag to flag packages that do not build CPython extensions for all the supported Python versions.
• fixing a false-positive in the "missing-prerequisite-for-pyproject-backend" tag.

I also worked on a few other Lintian tags, but they were unrelated to the Debian Python Team itself. I'm also happy to report many of the tags I wrote for the team in the past few months were merged by the awesome Russ Allbery and should land in unstable as soon as a new release is made. I'm particularly looking forward the new "uses-python-distutils" tag that should help us flag packages that still use the deprecated distutils library. Patching distro-tracker (tracker.debian.org) to show pending team MRs It's often hard to have a good overview of pending merge requests when working with team-maintained packages, as by default, Salsa doesn't notify anyone when a MR is opened. Although our workflow typically does not involve creating merge requests, some people still do and they end up sitting there, unnoticed. During the sprint, Kurt Kremitzki worked on solving this issue by having distro-tracker show the pending MRs on our team's tracker page. Sadly, it seems little progress was made, as the removal of python3-django-jsonfield from the archive and breaking changes in python3-selenium has broken the test suite. Migrate packages building with the flit plugin to the generic pyproject one pybuild has been supporting building with PEP-517 style pyproject.toml files via a generic plugin (pybuild-plugin-pyproject) for a while now. As this plugin supersedes the old flit plugin, we've been thinking of deprecating it in time for the Bookworm release. To make this possible, most of the packages in the archive that still used this plugin were migrated to the generic one and I opened bugs on the last handful of packages that were not team-maintained. Other work Many other things were done during the sprint, such as:

• improving existing packages (for example, adding or fixing autopkgtests)
• triaging and fixing Python 3.11 bugs
• updating packages to the latest upstream release
• sponsoring uploads

Thanks Thanks again to everyone who joined the sprint, and three big cheers for all the folks who donate to Debian and made it possible for us to have a food budget for the event.

Cisco IOS XR is the operating system running for the Cisco ASR, NCS, and 8000 routers. Compared to Cisco IOS, it features a candidate configuration and a running configuration. In configuration mode, you can modify the first one and issue the commit command to apply it to the running configuration.¹ This is a common concept for many NOS. Before committing the candidate configuration to the running configuration, you may want to check the changes that have accumulated until now. That s where the show commit changes diff command² comes up. Its goal is to show the difference between the running configuration (show running-configuration) and the candidate configuration (show configuration merge). How hard can it be? Let s put an interface down on IOS XR 7.6.2 (released in August 2022): The + sign before interface HundredGigE0/1/0/1 makes it look like you did create a new interface. Maybe there was a typo? No, the diff is just broken. If you look at the candidate configuration, everything is like you expect: Here is a more problematic example on IOS XR 7.2.2 (released in January 2021). We want to unconfigure three interfaces: The two first commands are correctly represented by the first two chunks of the diff: we remove GigabitEthernet0/0/0/5 and create TenGigE0/0/0/5. The two next commands are also correctly represented by the last chunk of the diff. TenGigE0/0/0/28 was already shut down, so it is expected that only description and bundle id are removed. However, the diff command forgets about the modifications for TenGigE0/0/0/29. The diff should include a chunk similar to the last one. How can the diff be correct for TenGigE0/0/0/28 but incorrect for TenGigE0/0/0/29 while they have the same configuration? How can you trust the diff command if it forgets part of the configuration? Do you remember the last time you ran an Ansible playbook and discovered the whole router ospf block disappeared without a warning? If you use automation tools, you should check how the diff is assembled. Automation tools should build it from the result of show running-config and show configuration merge. This is what NAPALM does. This is not what cisco.iosxr collection for Ansible does. The problem is not limited to the interface directives. You can get similar issues for other parts of the configuration. For example, here is what we get when removing inactive BGP neighbors on IOS XR 7.2.2: The only correct chunk is for neighbor 217.29.66.112. All the others are missing some of the removed lines. 217.29.67.15 is even missing all of them. How bad is the code providing such a diff? I could go all day with examples such as these. Cisco TAC is happy to open a case in DDTS, their bug tracker, to fix specific occurrences of this bug.³ However, I fail to understand why the XR team is not just providing the diff between show run and show configuration merge. The output would always be correct!

One of the huge benefits of WebAuthn is that it makes traditional phishing attacks impossible. An attacker sends you a link to a site that looks legitimate but isn't, and you type in your credentials. With SMS or TOTP-based 2FA, you type in your second factor as well, and the attacker now has both your credentials and a legitimate (if time-limited) second factor token to log in with. WebAuthn prevents this by verifying that the site it's sending the secret to is the one that issued it in the first place - visit an attacker-controlled site and said attacker may get your username and password, but they won't be able to obtain a valid WebAuthn response.

But what if there was a mechanism for an attacker to direct a user to a legitimate login page, resulting in a happy WebAuthn flow, and obtain valid credentials for that user anyway? This seems like the lead-in to someone saying "The Aristocrats", but unfortunately it's (a) real, (b) RFC-defined, and (c) implemented in a whole bunch of places that handle sensitive credentials. The villain of this piece is RFC 8628, and while it exists for good reasons it can be used in a whole bunch of ways that have unfortunate security consequences.

What is the RFC 8628-defined Device Authorization Grant, and why does it exist? Imagine a device that you don't want to type a password into - either it has no input devices at all (eg, some IoT thing) or it's awkward to type a complicated password (eg, a TV with an on-screen keyboard). You want that device to be able to access resources on behalf of a user, so you want to ensure that that user authenticates the device. RFC 8628 describes an approach where the device requests the credentials, and then presents a code to the user (either on screen or over Bluetooth or something), and starts polling an endpoint for a result. The user visits a URL and types in that code (or is given a URL that has the code pre-populated) and is then guided through a standard auth process. The key distinction is that if the user authenticates correctly, the issued credentials are passed back to the device rather than the user - on successful auth, the endpoint the device is polling will return an oauth token.

But what happens if it's not a device that requests the credentials, but an attacker? What if said attacker obfuscates the URL in some way and tricks a user into clicking it? The user will be presented with their legitimate ID provider login screen, and if they're using a WebAuthn token for second factor it'll work correctly (because it's genuinely talking to the real ID provider!). The user will then typically be prompted to approve the request, but in every example I've seen the language used here is very generic and doesn't describe what's going on or ask the user. AWS simply says "An application or device requested authorization using your AWS sign-in" and has a big "Allow" button, giving the user no indication at all that hitting "Allow" may give a third party their credentials.

This isn't novel! Christoph Tafani-Dereeper has an excellent writeup on this topic from last year, which builds on Nestori Syynimaa's earlier work. But whenever I've talked about this, people seem surprised at the consequences. WebAuthn is supposed to protect against phishing attacks, but this approach subverts that protection by presenting the user with a legitimate login page and then handing their credentials to someone else.

RFC 8628 actually recognises this vector and presents a set of mitigations. Unfortunately nobody actually seems to implement these, and most of the mitigations are based around the idea that this flow will only be used for physical devices. Sadly, AWS uses this for initial authentication for the aws-cli tool, so there's no device in that scenario. Another mitigation is that there's a relatively short window where the code is valid, and so sending a link via email is likely to result in it expiring before the user clicks it. An attacker could avoid this by directing the user to a domain under their control that triggers the flow and then redirects the user to the login page, ensuring that the code is only generated after the user has clicked the link.

Can this be avoided? The best way to do so is to ensure that you don't support this token issuance flow anywhere, or if you do then ensure that any tokens issued that way are extremely narrowly scoped. Unfortunately if you're an AWS user, that's probably not viable - this flow is required for the cli tool to perform SSO login, and users are going to end up with broadly scoped tokens as a result. The logs are also not terribly useful.

The infuriating thing is that this isn't necessary for CLI tooling. The reason this approach is taken is that you need a way to get the token to a local process even if the user is doing authentication in a browser. This can be avoided by having the process listen on localhost, and then have the login flow redirect to localhost (including the token) on successful completion. In this scenario the attacker can't get access to the token without having access to the user's machine, and if they have that they probably have access to the token anyway.

There's no real moral here other than "Security is hard". Sorry.

comments

Review: The Fed Unbound, by Lev Menand

Publisher:	Columbia Global Reports
Copyright:	2022
ISBN:	1-7359137-1-5
Format:	Kindle
Pages:	156

The Fed Unbound is a short non-fiction exploration of US Federal Reserve actions to reducing systemic risk caused by shadow banking. Its particular focus is the role of the Fed from the 2008 financial crisis to the present, including the COVID shock, but it includes a history of what Menand calls the "American Monetary Settlement," the political compromise that gave rise to the Federal Reserve. In Menand's view, a central cause of instability in the US financial system (and, given the influence of the dollar system, the world financial system as well) is shadow banking: institutions that act as banks without being insured like banks or subject to bank regulations. A bank, in this definition, is an organization that takes deposits. I'm simplifying somewhat, but what distinguishes a deposit from a security or an investment is that deposits can be withdrawn at any time, or at least on very short notice. When you want to spend the money in your checking account, you don't have to wait for a three-month maturity period or pay an early withdrawal penalty. You simply withdraw the money, with immediate effect. This property is what makes deposits "money," rather than something that you can (but possibly cannot) sell for money, such as stocks or bonds. Most people are familiar with the basic story of how banks work. Essentially no bank simply takes people's money and puts it in a vault until the person wants it again. If that were the case, you would need to pay the bank to store your money. Instead, a bank takes in deposits and then lends some portion of that money out to others. Those loans, for things like cars or houses or credit card spending, come due over time, with interest. The interest rate the bank charges on the loans is much higher than the rate it has to pay on its deposits, and it pockets the difference. The problem with this model, of course, is that the bank doesn't have your money, so if all the depositors go to the bank at the same time and ask for their money, the bank won't be able to repay them and will collapse. (See, for example, the movie It's a Wonderful Life, or Mary Poppins, or any number of other movies or books.) Retail banks are therefore subject to stringent regulations designed to promote public trust and to ensure that traditional banking is a boring (if still lucrative) business. Banks are also normally insured, which in the US means that if they do experience a run, federal regulators will step in, shut down the bank in an orderly fashion, and ensure every depositor gets their money back (at least up to the insurance limit). Alas, if you thought people would settle for boring work that makes a comfortable profit, you don't know the financial industry. Highly-regulated insured deposits are less lucrative than acting like a bank without all of those restrictions and rules and deposit insurance payments. As Menand relates in his brief history of US banking, financial institutions constantly invent new forms of deposits with similar properties but without all the pesky rules: eurodollars (which have nothing to do with the European currency), commercial paper, repo, and many others. These forms of deposits are primarily used by large institutions like corporations. The details vary, but they tend to be prone to the same fundamental instability as bank deposits: if there's a run on the market, there may not be enough liquidity for everyone to withdraw their money at once. Unlike bank deposits, though, there is no insurance, no federal regulator to step in and make depositors whole, and much less regulation to ensure that runs are unlikely. Instead, there's the Federal Reserve, which has increasingly become the bulwark against liquidity crises among shadow banks. This happened in 2008 during the financial crisis (which Menand argues can be seen as a shadow bank run sparked by losses on mortgage securities), and again at a larger scale in 2020 during the initial COVID crisis. Menand is clear that these interventions from the Federal Reserve were necessary. The alternative would have been an uncontrolled collapse of large sections of the financial system, with unknown consequences. But the Fed was not intended to perform those types of interventions. It has no regulatory authority to reform the underlying financial systems to make them safer, remove executives who failed to maintain sufficient liquidity for a crisis, or (as is standard for all traditional US banks) prohibit combining banking and more speculative investment on the same balance sheet. What the Federal Reserve can do, and did, is function as a buyer of last resort, bailing out shadow banks by purchasing assets with newly-created money. This works, in the sense that it averts the immediate crisis, but it creates other distortions. Most importantly, constant Fed intervention doesn't create an incentive to avoid situations that require that intervention; if anything, it encourages more dangerous risk-taking. The above, plus an all-too-brief history of the politics of US banking, is the meat of this book. It's a useful summary, as far as it goes, and I learned a few new things. But I think The Fed Unbound is confused about its audience. This type of high-level summary and capsule history seems most useful for people without an economics background and who haven't been following macroeconomics closely. But Menand doesn't write for that audience. He assumes definitions of words like "deposits" and "money" that are going to be confusing or even incomprehensible to the lay reader. For example, Menand describes ordinary retail banks as creating money, even saying that a bank loans money by simply incrementing the numbers in a customer's deposit account. This is correct in the technical economic definition of money (fractional reserve banking effectively creates new money), but it's going to sound to someone not well-versed in the terminology as if retail banks can create new dollars out of the ether. That power is, of course, reserved for the Federal Reserve, and indeed is largely the point of its existence. Much of this book relies on a very specific definition of money and money supply that will only be familiar to those with economics training. Similarly, the history of the Federal Reserve is interesting but slight, and at no point does Menand explain clearly how the record-keeping between it and retail banks works, or what the Fed's "balance sheet" means in practice. I realize this book isn't trying to give a detailed description or history of the Federal Reserve system, but the most obvious audience is likely to flounder at the level of detail Menand provides. Perhaps, therefore, this book is aimed at an audience already familiar with macroeconomics? But, if so, I'm not sure it says anything new. I follow macroeconomic policy moderately closely and found most of Menand's observations obvious and not very novel. There were tidbits here and there that I hadn't understood, but my time would probably have been better invested in another book. Menand proposes some reforms, but they mostly consist of "Congress should do its job and not create situations where the Federal Reserve has to act without the right infrastructure and political oversight," and, well, yes. It's hard to disagree with that, and it's also hard to see how it will ever happen. It's far too convenient to outsource such problems to central banking, where they are hidden behind financial mechanics that are incomprehensible to the average voter. This is an important topic, but I don't think this is the book to read about it. If you want a clearer and easier-to-understand role of the Federal Reserve in shadow banking crises, read Crashed instead. If you want to learn more about how retail bank regulation works, and hear a strong case for why the same principles should be applied to shadow banks, see Sheila Bair's Bull by the Horns. I'm still looking for a great history and explainer of the Federal Reserve system as a whole, but that is not this book. Rating: 5 out of 10

For the past four years, I ve been working on Carthage, a free-software Infrastructure as Code framework. We ve finally reached a point where it makes sense to talk about Carthage and what it can do. This is the first in a series of blog posts to introduce Carthage, discuss what it can do and show how it works. Why Another IAC Framework It seems everywhere you look, there are products designed to support the IAC pattern. On the simple side, you could check a Containerfile into Git. Products like Terraform and Vagrant allow you to template cloud infrastructure and VMs. There are more commercial offerings than I can keep up with. We were disappointed by what was out there when we started Carthage. Other products have improved, but for many of our applications we re happy with what Carthage can build. The biggest challenge we ran into is that products wanted us to specify things at the wrong level. For some of our cyber training work we wanted to say things like We want 3 blue teams, each with a couple defended networks, a red team, and some neutral infrastructure for red to exploit. Yet the tools we were trying to use wanted to lay things out at the individual machine/container level. We found ourselves contemplating writing a program to generate input for some other IAC tool. Things were worse for our internal testing. Sometimes we d be shipping hardware to a customer. But sometimes we d be virtualizing that build out in a lab. Sometimes we d be doing a mixture. So we wanted to completely separate the descriptions of machines, networks, and software from any of the information about whether that was realized on hardware, VMs, containers, or a mixture. Dimensional Breakdown In discussing Carthage with Enrico Zini, he pointed me at Cognitive Dimensions of notation as a way to think about how Carthage approaches the IAC problem. I m more interested in the idea of breaking down a design along the idea of dimensions that allow examining the design space than I am particular adherence to Green s original dimensions. Low Viscosity, High Abstraction Reuse One of the guiding principles is that we want to be able to reuse different components at different scales and in different environments. These include being able to do things like:

• Define an operation like Update a Debian system and apply that in several environments including as part of building a base VM or container image, applying to an independently managed machine, or applying to a micro service container that does not run services like ssh or systemd.
• Defining a role like DNS server that can be applied to a dedicated machine only having that role, to a traditional server with multiple roles, or in a micro service environment.
• Allowing people to write groups of functionality that can be useful in descriptions of a small number of machines, but can also be reused in large environments like modeling of cyber infrastructure to defend. In the small environments, things are simplified, but in larger environments integration like directories, authentication infrastructure and the like is needed.
• Allow grouping of functionality at multiple levels. So far I have talked about grouping of software to be installed on a single machine or container. We also want to allow groups of containers (pods or otherwise), groups of machines, groups of networks, or even enclaves (think a model of an entire company or section of a company). Each kind of grouping needs to be parametric and reusable.

Hidden Dependencies To accomplish these abstraction goals, dependencies need to be non-local. For example, a software role might need to integrate with a directory if a directory is present in the environment. When writing the role, no one is going to know which directory to use, nor whether a directory is present. Taking that as an explicit input into the role is error-prone when the role is combined into large abstract units (bigger roles or collections of machines). Instead it is better to have a non-local dependency, and to find the directory if it is available. We accomplish this using dependency injection. In addition to being non-local, dependencies are sometimes hidden. It is very easy to overwhelm our cognitive capacity with even a fairly simple IAC description. An effective notation allows us to focus on the parts that matter when working with a particular part of the description. I ve found hiding dependencies, especially indirect dependencies, to be essential in building complex descriptions. Obviously, tools are required for examining these dependencies as part of debugging. First Class Modeling Clearly one of the goals of IAC descriptions is to actually build and manage infrastructure. It turns out that there are all sorts of things you want to do with the description well before you instantiate the infrastructure. You might want to query the description to build network diagrams, understand interdependencies, or even build inventory/bill of materials. We often find ourselves building Ansible inventory, switch configurations, DNS zones, and all sorts of configuration artifacts. These artifacts may be installed into infrastructure that is instantiated by the description, but they may be consumed in other ways. Allowing the artifacts to be consumed externally means that you can avoid pre-commitment and focus on whatever part of the description you originally want to work on. You may use an existing network at first. Later the IAC description may replace that, or perhaps it never will. As a result, Carthage separates modeling from instantiation. The model can generally be built and queried without needing to interact with clouds, VMs, or containers. We ve actually found it useful to build Carthage layouts that cannot ever be fully instantiated, for example because they never specify details like whether a model should be instantiated on a container or VM, or what kind of technology will realize a modeled network. This allows developing roles before the machines that use them or focusing on how machines will interact and how the network will be laid out before the details of installing on specific hardware. The modeling separation is by far the difference I value most between Carthage and other systems. A Tool for Experts. In Neal Stephenson s essay In the Beginning Was the Command Line , Stephenson points out that the kind of tools experts need are not the same tools that beginners need. The illustration of why a beginner might not be satisfied with a Hole Hog drill caught my attention. Carthage is a tool for experts. Despite what cloud providers will tell you, IAC is not easy. Doubly so when you start making reusable components. Trying to hide that or focus on making things easy to get started can make it harder for experts to efficiently solve the problems they are facing. When we have faced trade offs between making Carthage easy to pick up and making it powerful for expert users, we have chosen to support the experts. That said, Carthage today is harder to pick up than it needs to be. It s a relatively new project with few external users as of this time. Our documentation and examples need improvement, just like every project at this level of maturity. Similarly, as the set of things people try to do expand, we will doubtless run into bugs that our current test cases don t cover. So Carthage absolutely will get easier to learn and use than it is today. Also, we ve already had success building beginner-focused applications on top of Carthage. For our cyber training, we built web applications on top of Carthage that made rebuilding and exploring infrastructure easy. We ve had success using relatively understood tools like Ansible as integration and customization points for Carthage layouts. But in all these cases, when the core layout had significant reusable components and significant complexity in the networking, only an IAC expert was going to be able to maintain and develop that layout. What Carthage can do. Carthage has a number of capabilities today. One of Carthage s strengths is its extensible design. Abstract interfaces make it easy to add new virtualization platforms, cloud services, and support for various ways of managing real hardware. This approach has been validated by incrementally adding support for virtualization architectures and cloud services. As development has progressed, adding new integrations continues to get faster because we are able to reuse existing infrastructure. Today, Carthage can model:

• Machines
• Networks
• Dynamically compose groupings of the above
• Generate model level artifacts
  • Ansible inventory
  • Various DNS integrations
  • Various switch configurations

Carthage has excellent facilities for dealing with images on which VMs and Containers can be based, although it does have a bit of a Debian/Ubuntu bias in how it thinks about images:

• Building base images from a tool like debootstrap
• Customizing these images
• Converting into VM images for kvm, VMware, and AWS
• Building from scratch OCI images for Podman, Docker and k8s
• Adding layers to existing OCI images

When instantiating infrastructure, Carthage can work with:

• systemd nspawn containers
• Podman (Docker would be easy)
• Libvirt
• VMware
• With the AWS plugin, EC2 VMs and networking

We have also looked at Oracle Cloud and I believe Openstack, although that code is not merged. Future posts will talk about core Carthage concepts and how to use Carthage to build infrastructure.

comments