Search Results: "pm"

1 January 2022

Chris Lamb: Favourite books of 2021: Classics

In my three most recent posts, I went over the memoirs and biographies, the non-fiction and fiction I enjoyed in 2021. But in the last of my 2021 book-related posts, however, I'll be going over my favourite classics. Of course, the difference between regular fiction and a 'classic' is an ambiguous, arbitrary and often-meaningless distinction: after all, what does it matter if Hemingway's The Old Man and the Sea (from 1951) is a classic or not? The term also smuggles in some of the ethnocentric gatekeeping encapsulated in the term 'Western canon' too. Nevertheless, the label of 'classic' has some utility for me in that it splits up the vast amount of non-fiction I read in two... Books that just missed the cut here include: Oscar Wilde's The Picture of Dorian Gray (moody and hilarious, but I cannot bring myself to include it due to the egregious antisemitism); Tolstoy's The Kreutzer Sonata (so angry! so funny!); and finally Notes from Underground by Fyodor Dostoevsky. Of significant note, though, would be the ghostly The Turn of the Screw by Henry James.

Heart of Darkness (1899) Joseph Conrad Heart of Darkness tells the story of Charles Marlow, a sailor who accepts an assignment from a Belgian trading company as a ferry-boat captain in the African interior, and the novella is widely regarded as a critique of European colonial rule in Africa. Loosely remade by Francis Ford Coppola as Apocalypse Now (1979), I started this book with the distinct possibility that this superb film adaptation would, for a rare treat, be 'better than the book'. However, Conrad demolished this idea of mine within two chapters, yet also elevated the film to a new level as well. This was chiefly due to how observant Conrad was of the universals that make up human nature. Some of his insight pertains to the barbarism of the colonialists, of course, but Conrad applies his shrewd acuity to the at the smaller level as well. Some of these quotes are justly famous: Ah! but it was something to have at least a choice of nightmares, for example, as well as the reference to a fastidiously turned-out colonial administrator who, with unimaginable horrors occurring mere yards from his tent, we learn he was devoted to his books, which were in applepie order . (It seems to me to be deliberately unclear whether his devotion arises from gross inhumanity, utter denial or some combination of the two.) Oh, and there's a favourite moment of mine when a character remarks that It was very fine for a time, but after a bit I did get tired of resting. Tired of resting! Yes, it's difficult to now say something original about a many-layered classic such as this, especially one that has analysed from so many angles already; from a literary perspective at first, of course, but much later from a critical postcolonial perspective, such as in Chinua Achebe's noted 1975 lecture, An Image of Africa. Indeed, the history of criticism in the twentieth century of Heart of Darkness must surely parallel the social and political developments in the Western world. (On a highly related note, the much-cited non-fiction book King Leopold's Ghost is on my reading list for 2022.) I will therefore limit myself to saying that the boat physically falling apart as it journeys deeper into the Congo may be intended to represent that our idea of 'Western civilisation' ceases to function, both morally as well as physically, in this remote environment. And, whilst I'm probably not the first to notice the potential ambiguity, when Marlow lies to Kurtz's 'Intended [wife]' in the closing section in order to save her from being exposed to the truth about Kurtz (surely a metaphor about the ignorance of the West whilst also possibly incorporating some comment on gender?), the Intended replies: I knew it. For me, though, it is not beyond doubt that what the Intended 'knows' is that she knew that Marlow would lie to her: in other words, that the alleged ignorance of everyday folk in the colonial homeland is studied and deliberate. Compact and fairly easy-to-read, it is clear that Heart of Darkness rewards even the most rudimentary analysis.

Rebecca (1938) Daphne du Maurier Daphne du Maurier creates in Rebecca a credible and suffocating atmosphere in the shape of Manderley, a grand English mansion owned by aristocratic widower Maxim de Winter. Our unnamed narrator (a young woman seemingly na ve in the ways of the world) meets Max in Monte Carlo, and she soon becomes the second Mrs. de Winter. The tale takes a turn to the 'gothic', though, when it becomes apparent that the unemotional Max, as well as potentially Manderley itself, appears to be haunted by the memory of his late first wife, the titular Rebecca. Still, Rebecca is less of a story about supernatural ghosts than one about the things that can haunt our minds. For Max, this might be something around guilt; for our narrator, the class-centered fear that she will never fit in. Besides, Rebecca doesn't need an actual ghost when you have Manderley's overbearing housekeeper, Mrs Danvers, surely one of the creepiest characters in all of fiction. Either way, the conflict of a kind between the fears of the protagonists means that they never really connect with each other. The most obvious criticism of Rebecca is that the main character is unreasonably weak and cannot quite think or function on her own. (Isn't it curious that the trait of the male 'everyman' is a kind of physical clumsiness yet the female equivalent is shorthanded by being slightly slow?) But the na vete of Rebecca's narrator makes her easier to relate to in a way, and it also makes the reader far more capable of empathising with her embarrassment. This is demonstrated best whilst she, in one of the best evocations of this particular anxiety I have yet come across, is gingerly creeping around Manderlay and trying to avoid running into the butler. A surprise of sorts comes in the latter stages of the book, and this particular twist brings us into contact with a female character who is anything but 'credulous'. This revelation might even change your idea of who the main character of this book really is too. (Speaking of amateur literary criticism, I have many fan theories about Rebecca, including that Maxim de Winter's estate manager, Frank Crawley, is actually having an affair with Max, and also that Maxim may have a lot more involvement in Mrs Danvers final act that he lets on.) An easily accessible novel (with a great-but-not-perfect 1940 adaptation by Alfred Hitchcock, Rebecca is a real indulgence.

A Clockwork Orange (1962) Anthony Burgess One of Stanley Kubrick's most prominent tricks was to use different visual languages in order to prevent the audience from immediately grasping the underlying story. In his 1975 Barry Lyndon, for instance, the intentionally sluggish pacing and elusive characters require significant digestion to fathom and appreciate, and the luminous and quasi-Renaissance splendour of the cinematography does its part to constantly distract the viewer from the film's greater meaning. This is very much the case in Kubrick's A Clockwork Orange as well whilst it ostensibly appears to be about a Saturnalia of violence, the 'greater meaning' of A Clockwork Orange pertains to the Christian conception of free will; admittedly, a much drier idea to bother making a film around. This is all made much clearer when reading Anthony Burgess' 1962 original novel. Alex became a 'true Christian' through the experimental rehabilitation process, and even offers to literally turn the other cheek at one point. But as Alex had no choice to do so (and can no longer choose to commit violence), he is incapable of making a free moral choice. Thus, is he really a Man? Yet whilst the book's central concern is our conception of free will in modern societies, it also appears to be a repudiation of two conservative principles. Firstly, A Clockwork Orange demolishes the idea that 'high art' leads to morally virtuous citizens. After all, if you can do a bit of the old ultra-violence whilst listening to the glorious 9th by old Ludvig van, then so much for the oft-repeated claims that culture makes you better as a person. (This, at least, I already knew from personal experience.) The other repudiation in A Clockwork Orange is in regard to the pervasive idea that the countryside is a refuge from crime and sin. By contrast, we see the gang commit their most horrific violence in rural areas, and, later, Alex is taken to the countryside by his former droogs for a savage beating. Although this doesn't seem to quite fit the novel, this was actually an important point for Burgess to include: otherwise his book could easily be read as a commentary on the corrupting influence of urban spaces, rather than of modernity itself. The language of this book cannot escape comment here. Alex narrates most of the book in a language called Nadsat, a fractured slang constructed by Burgess based on Russian and Cockney rhyming slang. (The language is strange for only a few pages, I promise. And note that 'Alex' is a very common Russian name.) Using Nadsat has the effect of making the book feel distinctly alien, but it also prevents it from prematurely aging too. Indeed, it comes as bit of a shock to realise that A Clockwork Orange was published 1962, the same year as The Beatles' released their first single, Love Me Do. I could probably say a whole lot more about this thoroughly engrossing book and its movie adaptation (eg. the meta-textual line in Kubrick's version: It's funny how the colours of the real world only seem really real when you watch them on a screen... appears verbatim in the textual original), but I'll leave it there. The book of A Clockwork Orange is not only worth the investment in the language, but is, again, somehow better than the film.

The Great Gatsby (1925) F. Scott Fitzgerald I'm actually being a little deceitful by including this book here: I cannot really say that The Great Gatsby was a 'favourite' read of the year, but its literary merit is so undeniable (and my respect for Fitzgerald's achievement is deep enough) that the experience was one of those pleasures you feel at seeing anything done well. Here you have a book so rich in symbolic meaning that you could easily confuse the experience with drinking Coke syrup undiluted. And a text that has made the difficulty and complexity of reading character a prominent theme of the novel, as well as a technical concern of the book itself. Yet at all times you have in your mind that The Great Gatsby is first and foremost a book about a man writing a book, and, therefore, about the construction of stories and myths. What is the myth being constructed in Gatsby? The usual answer today is that the book is really about the moral virtues of America. Or, rather, the lack thereof. Indeed, as James Boice wrote in 2016:
Could Wilson have killed Gatsby any other way? Could he have ran him over, or poisoned him, or attacked him with a knife? Not at all this an American story, the quintessential one, so Gatsby could have only died the quintessential American death.
The quintessential American death is, of course, being killed with a gun. Whatever your own analysis, The Great Gatsby is not only magnificently written, but it is captivating to the point where references intrude many months later. For instance, when reading something about Disney's 'princess culture', I was reminded of when Daisy says of her daughter: I hope she'll be a fool that's the best thing of a girl can be in this world, a beautiful little fool . Or the billboard with the eyes of 'Doctor T. J. Eckleburg'. Or the fact that the books in Gatsby's library have never been read (so what is 'Owl Eyes' doing there during the party?!). And the only plain room in Gatsby's great house is his bedroom... Okay, fine, I must have been deluding myself: I love this novel.

Caleb Adepitan: Everybody Struggles

I often get intimidated by the brilliance of others by just looking at them from a distance seeing how radiantly they gleam in their knowledge. I often feel like I'm not doing enough or I'm just meant to be a mediocre and there's nothing I can do to be more than that. I often feel like everyone except me is a genius, and are impeccable unlike me. But maybe I'm just at a point these people had previously been at, and have now gone past. Maybe it's my time to steer the wheel they once steered, or maybe they aren't even what they seem to me to be and it's all just a misrepresentation cooked up by an imperfect mind. Hmm...it has to be that. A perfect portrait painted by an imperfect mind which when found out renders every stroke in the portrait imperfect. It has to be that or all. burst.gif No one starts having it all figured out. A king is born in a day, but a king is not nutured in a day. We all have something we struggle with, it's just different all across. And because their current struggle isn't my current struggle and mine not theirs doesn't make them immuned to the system. Most times all that gleaming in thier knowledge is a collective thing and you only notice when you take a closer look; that there are more heads to the knowledge and brilliance than you think. We mostly see what people know and seldom see what they don't know. If it was the other way around, we'll see how ridiculous the things these highly esteemed people don't know are. What matters most is that they keep making it through their struggles. No man is an island! The sooner we realize this the better. We can always take those things we struggle with to others and work collectively on them, learn from a lot other experiences and gleam collectively. It's like a tug of war. When what is on the other end is more than we can pull off, we invite many more hands to pull it off.

Utkarsh Gupta: FOSS Activites in December 2021

Here s my (twenty-seventh) monthly but brief update about the activities I ve done in the F/L/OSS world.

Debian
This was my 36th month of actively contributing to Debian. I became a DM in late March 2019 and a DD on Christmas 19! \o/ Just churning through the backlog again this month. Ugh. Anyway, I did the following stuff in Debian:

Uploads and bug fixes:
  • ruby2.7 (2.7.5-1) - New upstream version fixing 3 new CVEs.

Other $things:
  • Mentoring for newcomers.
  • Moderation of -project mailing list.

Ubuntu
This was my 11th month of actively contributing to Ubuntu. Now that I ve joined Canonical to work on Ubuntu full-time, there s a bunch of things I do! \o/ I mostly worked on different things, I guess. I was too lazy to maintain a list of things I worked on so there s no concrete list atm. Maybe I ll get back to this section later or will start to list stuff from next year onward, as I was doing before. :D

Debian (E)LTS
Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success. And Debian Extended LTS (ELTS) is its sister project, extending support to the Jessie release (+2 years after LTS support). This was my twenty-seventh month as a Debian LTS and eighteenth month as a Debian ELTS paid contributor.
I was assigned 40.00 hours for LTS and 60.00 hours for ELTS and worked on the following things:
(since I had a 3-week vacation, I wanted to wrap things up that were pending and so I worked for 20h more for LTS, which I ll compensate the next month!)

LTS CVE Fixes and Announcements:

ELTS CVE Fixes and Announcements:
  • Issued ELA 525-2, fixing CVE-2021-43527, for nss.
    For Debian 8 jessie, these problems have been fixed in version 2:3.26-1+debu8u15.
  • Issued ELA 530-1, for systemd.
    For Debian 8 jessie, these problems have been fixed in version 215-17+deb8u14.
  • Issued ELA 531-1, fixing CVE-2021-41817 and CVE-2021-41819, for ruby2.1.
    For Debian 8 jessie, these problems have been fixed in version 2.1.5-2+deb8u13.
  • Issued ELA 533-1, fixing CVE-2018-12020, for python-gnupg.
    For Debian 8 jessie, these problems have been fixed in version 0.3.6-1+deb8u2.
  • Issued ELA 536-1, fixing CVE-2021-43818, for lxml.
    For Debian 8 jessie, these problems have been fixed in version Prior to version 4.6.5, the HTML Cleaner in lxml.html lets certain.
  • Started working on src:samba for CVE-2020-25717 to CVE-2020-25722 and CVE-2021-23192 for jessie and stretch, both.
    The version difference b/w the suites are a bit too much for the patch(es) to be easily backported. I ve talked to Anton to work something out. \o/
  • Found the problem w/ libjdom1-java. Will have to roll the regression upload.
    I ve prepared the patch but needs some testing to be finally rolled out. Same for stretch.

Other (E)LTS Work:
  • Front-desk duty from 29-11 to 05-12 and 20-12 to 26-12 for both LTS and ELTS.
  • Triaged ffmpeg, git, gpac, inetutils, mc, modsecurity-crs, node-object-path, php-pear, systemd-cron, node-tar, ruby2.3, gst-plugins-bad0.10, npm, nltk, request-tracker4, ros-ros-comm, mediawiki, ruby2.1, ckeditor, ntfs-3g, tiff, wordpress, and jsoup, udisks2, libgit2, python3.5, python3.4, and openssh.
  • Mark CVE-2021-38171/ffmpeg as postponed for stretch.
  • Mark CVE-2021-40330/git as no-dsa for stretch and jessie.
  • Mark CVE-2020-19481/gpac as ignored for stretch.
  • Mark CVE-2021-40491/inetutils as no-dsa for stretch.
  • Mark CVE-2021-36370/mc as no-dsa for stretch and jessie.
  • Mark CVE-2021-35368/modsecurity-crs as no-dsa for stretch.
  • Mark CVE-2021-23434/node-object-path as end-of-life for stretch.
  • Mark CVE-2021-32610/php-pear as no-dsa for stretch.
  • Mark CVE-2017-9525/systemd-cron as no-dsa for stretch.
  • Mark CVE-2021-37701/node-tar as end-of-life for stretch.
  • Mark CVE-2021-37712/node-tar as end-of-life in stretch.
  • Mark CVE-2021-39201/wordpress as not-affected for jessie.
  • Mark CVE-2020-19143/tiff as not-affected for stretch and jessie.
  • Mark CVE-2021-38562/request-tracker4 as no-dsa for stretch.
  • Mark CVE-2021-37146/ros-ros-comm as no-dsa for stretch.
  • Mark CVE-2021-28965/ruby2.1 as ignored for jessie.
  • Mark CVE-2021-37714/jsoup as ignored for jessie.
  • Mark CVE-2021-41617/openssh as no-dsa for jessie.
  • Auto EOL ed ardour, nltk, request-tracker4, python-scrapy, webkit2gtk, and linux for jessie.
  • Attended monthly Debian LTS meeting.
  • Answered questions (& discussions) on IRC (#debian-lts and #debian-elts).
  • General and other discussions on LTS private and public mailing list.

Debian LTS Survey I ve spent 5 hours on the LTS survey on the following bits:
  • Went through the old content on the previous survey.
  • Reviewed the new content - still more work to do.
  • Discussed the survey bits in the team meeting.
  • Partly reviewing the questions of the survey.
  • Walking through the instance to find the doability of the tasks discussed in the meeting.
  • Segregating and staging questions. More work to do here.

Until next time.
:wq for today.

31 December 2021

Russ Allbery: Review: The Space Between Worlds

Review: The Space Between Worlds, by Micaiah Johnson
Publisher: Del Rey
Copyright: 2020
ISBN: 0-593-13506-7
Format: Kindle
Pages: 327
Cara is valuable because, in most places, she's dead. In the world of Earth Zero, as the employees of the Eldridge Institute call it, a scientific genius named Adam Bosch developed the ability to travel between parallel worlds. This ability is not limitless, however. One restriction is that the parallel world has to be very close; large divergences of history render them unreachable. The other restriction is that anyone who attempts to travel to a world in which the local version of themselves is still alive is rejected: physically mangled in ways that result in a very short remaining lifespan. Earth Zero has not found a way to send information between worlds without sending people there physically to collect it. Those people are traversers, and their value lies in how many of their parallel selves have died. Each death in one of the 380 worlds Earth Zero can reach means another world that person can traverse to. They are the transportation system for a network of information-gathering nodes, whose collected contents are mined for stock tips, political cautions, and other information of value. Cara is dead on 372 worlds, and thus provides valuable savings on employee salaries. These related worlds are not so much post-apocalyptic as a continuation of current wealth disparity trends, although it's also clear that the climate has gotten worse. The Eldridge Institute, which controls traversing, is based in Wiley City, a walled, climate-controlled arcology of skyscrapers with a dome that filters out the dangerous sun. Its citizens are rich, with the best social support that money can buy. They are not interested in immigrants, unless they are extremely valuable. Cara is not from Wiley City. She is from Ashtown, the encampment in the desert outside of Wiley City's walls. That's part of the explanation for her death rate; in Ashtown, there are only a few ways to survive, particularly if one is not from the stiflingly religious Rurals, and most of them are dependent on being in the good graces of the local warlord and his Mad-Max-style enforcers. Being a traverser gets Cara out of Ashtown and into Wiley City, but not as a citizen, although that's dangled vaguely as a possible future prize. She's simply an employee, on a work permit, who enjoys the comforts of Wiley City for exactly as long as she's useful. Meanwhile, she juggles the demands of her job, her attraction to her watcher Dell, and her family in Ashtown. She is profoundly, aggressively cynical. Cara is also not precisely who people think she is. The Space Between Worlds pulls off a beautifully elegant combination of two science fiction subgenres: parallel universes and time travel. Both have been part of science fiction for decades, but normally parallel universes are substantially different from each other. Major historical events go differently, Nazis win World War II, Spock has a goatee, etc. Minor deviations are more often the subject of time travel stories, as travelers attempt to tweak the past and influence the future. Johnson instead provides the minor variations and small divergences of time travel stories in a parallel world framework, with no actual time travel involved or possible. The resulting story shows the same ripple effect of small differences, but the future remains unwritten and unconstrained, which avoids the stiflingly closed feeling of most time travel plots. Against that backdrop is set a story of corporate and personal intrigue, but one with a far deeper understanding of class and place than almost all of science fiction. Cara is not from Ashtown in the normal sense of science fiction novels written by comfortably middle-class white authors about protagonists from the wrong side of the tracks, who show their merit and then never look back. Cara is from Ashtown in a way that means she misses the taste of its dirt and understands its people and feels seen there. Wiley City knows very well that she's from Ashtown, and doesn't let her forget it. This type of ambiguous relationship with place and wealth, and deep connection to where one comes from, is so rare in science fiction, and it's beautifully written here. Cara wants to be in Wiley City over the alternative; the potential loss of her job is a real threat. But at the same time she is not at home there, because she is not visible there. Everything is slightly off, she has no one she can really talk to, and her reactions don't quite fit. No one understands her the way that her family in Ashtown does. And yet, by living in Wiley City, she is becoming less at home in Ashtown as well. She is becoming an outsider. It takes about 70 pages for the story in The Space Between Worlds to really get started. Those first 70 pages is very important background information that the rest of the story builds on, but they weren't that engrossing. Once the story kicks into gear, though, it's a tense, complicated story that I had a hard time predicting and an even harder time putting down. It's not perfect (more on that in a moment), but Johnson weaves together Cara's sense of place, her family connections, her sense of self, and her internal moral compass to create a memorable protagonist in a page-turning plot with a satisfying payoff. She uses our ability to look in on several versions of each character to give them additional satisfying heft and depth. Esther, Cara's highly religious sister, is the most delightful character in this book, and that's saying a lot coming from someone who usually doesn't like highly religious characters. I do have some world-building quibbles, and came up with more when I mulled over the book after finishing it, so you may need to strengthen your suspension of disbelief. The passive information gathering via traversing made a lot of sense; the bulk import of raw materials via the industrial hatch makes less sense given the constraints of the world. (Who is loading those materials into the other side? Or are they somehow traversing them directly out of the ground? Wouldn't someone notice?) The plot also partly hinges on a bit of lost technology that is extremely difficult to square with the rest of the setting, and felt like a transparent justification for introducing Mad Max elements into the setting. The quibble I noticed the most may be unavoidable given the setting: alternate worlds with slightly different versions of the same characters creates a potential explosion in cast size, which Johnson deals with by focusing on the cross-world variations of a small number of characters. I like all of those characters, but it does give the story a bit of an incestuous feel. The politics of every world revolve around the same ten people, and no one else seems to matter (or usually even has a name). That said, a small cast is a better problem to have than a confusing cast. Johnson does a great job helping the reader keep all the characters and relationships straight across their alternate world variations. I didn't realize until after I finished the book how difficult that probably was, which is the sign of a job well done. I do also have to complain about how completely dense Cara is when it comes to Dell, but I won't say any more than that to avoid spoilers. There are some things I figured out way before Cara did, though, and that made her behavior rather frustrating. This is an extremely impressive first novel that does some lovely things with genre and even more impressive things with social class and mobility. It's a little rough in places, you have to bear with the first 70 pages, and the ending, while a fitting conclusion to the emotional arc, seemed wildly unbelievable to me given the events of the plot. But it's very much worth reading despite those flaws. Johnson respects her characters and their culture and their world, and it shows. This was one of the best science fiction novels I read in 2021. (Content warning for physical and emotional partner abuse.) Rating: 8 out of 10

Matthew Garrett: Update on Linux hibernation support when lockdown is enabled

Some time back I wrote up a description of my proposed (and implemented) solution for making hibernation work under Linux even within the bounds of the integrity model. It's been a while, so here's an update.

The first is that localities just aren't an option. It turns out that they're optional in the spec, and TPMs are entirely permitted to say they don't support them. The only time they're likely to work is on platforms that support DRTM implementations like TXT. Most consumer hardware doesn't fall into that category, so we don't get to use that solution. Unfortunate, but, well.

The second is that I'd ignored an attack vector. If the kernel is configured to restrict access to PCR 23, then yes, an attacker is never able to modify PCR 23 to be in the same state it would be if hibernation were occurring and the key certification data will fail to validate. Unfortunately, an attacker could simply boot into an older kernel that didn't implement the PCR 23 restriction, and could fake things up there (yes, this is getting a bit convoluted, but the entire point here is to make this impossible rather than just awkward). Once PCR 23 was in the correct state, they would then be able to write out a new swap image, boot into a new kernel that supported the secure hibernation solution, and have that resume successfully in the (incorrect) belief that the image was written out in a secure environment.

This felt like an awkward problem to fix. We need to be able to distinguish between the kernel having modified the PCRs and userland having modified the PCRs, and we need to be able to do this without modifying any kernels that have already been released[1]. The normal approach to determining whether an event occurred in a specific phase of the boot process is to "cap" the PCR - extend it with a known value that indicates a transition between stages of the boot process. Any events that occur before the cap event must have occurred in the previous stage of boot, and since the final PCR value depends on the order of measurements and not just the contents of those measurements, if a PCR is capped before userland runs, userland can't fake the same PCR value afterwards. If Linux capped a PCR before userland started running, we'd be able to place a measurement there before the cap occurred and then prove that that extension occurred before userland had the opportunity to interfere. We could simply place a statement that the kernel supported the PCR 23 restrictions there, and we'd be fine.

Unfortunately Linux doesn't currently do this, and adding support for doing so doesn't fix the problem - if an attacker boots a kernel that doesn't cap a PCR, they can just cap it themselves from userland. So, we're faced with the same problem: booting an older kernel allows the system to be placed in an identical state to the current kernel, and a fake hibernation image can be written out. Solving this required a PCR that was being modified after kernel code was running, but before userland was started, even with existing kernels.

Thankfully, there is one! PCR 5 is defined as containing measurements related to boot management configuration and data. One of the measurements it contains is the result of the UEFI ExitBootServices() call. ExitBootServices() is called at the transition from the UEFI boot environment to the running OS, and the kernel contains code that executes before it. So, if we measure an assertion regarding whether or not we support restricted access to PCR 23 into PCR 5 before we call ExitBootServices(), this will prevent userspace from spoofing us (because userspace will only be able to extend PCR 5 after the firmware extended PCR 5 in response to ExitBootServices() being called). Obviously this depends on the firmware actually performing the PCR 5 extension when ExitBootServices() is called, but if firmware's out of spec then I don't think there's any real expectation of it being secure enough for any of this to buy you anything anyway.

My current tree is here, but there's a couple of things I want to do before submitting it, including ensuring that the key material is wiped from RAM after use (otherwise it could potentially be scraped out and used to generate another image afterwards) and, uh, actually making sure this works (I no longer have the machine I was previously using for testing, and switching my other dev machine over to TPM 2 firmware is proving troublesome, so I need to pull another machine out of the stack and reimage it).

[1] The linear nature of time makes feature development much more frustrating

comment count unavailable comments

26 December 2021

Vincent Bernat: Custom screen saver with XSecureLock

i3lock is a popular X11 screen lock utility. As far as customization goes, it only allows one to set a background from a PNG file. This limitation is part of the design of i3lock: its primary goal is to keep the screen locked, something difficult enough with X11. Each additional feature would increase the attack surface and move away from this goal.1 Many are frustrated with these limitations and extend i3lock through simple wrapper scripts or by forking it.2 The first solution is usually safe, but the second goes against the spirit of i3lock. XSecureLock is a less-known alternative to i3lock. One of the most attractive features of this locker is to delegate the screen saver feature to another process. This process can be anything as long it can attach to an existing window provided by XSecureLock, which won t pass any input to it. It will also put a black window below it to ensure the screen stays locked in case of a crash. XSecureLock is shipped with a few screen savers, notably one using mpv to display photos or videos, like the Apple TV aerial videos. I have written my own saver using Python and GTK.3 It shows a background image, a clock, and the current weather.4
Custom screen saver for XSecureLock, displaying a clock and the current weather
Custom screen saver for XSecureLock
I add two patches over XSecureLock: XSecureLock also delegates the authentication window to another process, but I was less comfortable providing a custom one as it is a bit more security-sensitive. While basic, the shipped authentication application is fine by me. I think people should avoid modifying i3lock code and use XSecureLock instead. I hope this post will help a bit.

Update (2022-01) XScreenSaver can also run arbitrary programs as a screen saver.


  1. See for example this comment or this one explaining the rationale.
  2. This Reddit post enumerates many of these alternatives.
  3. Using GTK makes it a bit difficult to use some low-level features, like embedding an application into an existing window. However, the high-level features are easier, notably drawing an image and a text with a shadow.
  4. Weather is retrieved by another script running on a timer and written to a file. The screen saver watches this file for updates.

24 December 2021

Vincent Bernat: Automatic login with startx and systemd

If your workstation is using full-disk encryption, you may want to jump directly to your desktop environment after entering the passphrase to decrypt the disk. Many display managers like GDM and LightDM have an autologin feature. However, only GDM can run Xorg with standard user privileges. Here is an alternative using startx and a systemd service:
[Unit]
Description=X11 session for bernat
After=graphical.target systemd-user-sessions.service
[Service]
User=bernat
WorkingDirectory=~
PAMName=login
Environment=XDG_SESSION_TYPE=x11
TTYPath=/dev/tty8
StandardInput=tty
UnsetEnvironment=TERM
UtmpIdentifier=tty8
UtmpMode=user
StandardOutput=journal
ExecStartPre=/usr/bin/chvt 8
ExecStart=/usr/bin/startx -- vt8 -keeptty -verbose 3 -logfile /dev/null
Restart=no
[Install]
WantedBy=graphical.target
Let me explain each block: Drop this unit in /etc/systemd/system/x11-autologin.service and enable it with systemctl enable x11-autologin.service. Xorg is now running rootless and logging into the journal. After using it for a few months, I didn t notice any regression compared to LightDM with autologin.

  1. For more information on how logind provides access to devices, see this blog post. The method names do not match the current implementation, but the concepts are still correct. Xorg takes control of the session when the TTY is active.
  2. Xorg could change the type of the session itself after taking control of it, but it does not.
  3. There is some code in Xorg to do that, but it is executed too late and fails with: xf86OpenConsole: VT_ACTIVATE failed: Operation not permitted.

23 December 2021

Russ Allbery: Review: The Empress of Salt and Fortune

Review: The Empress of Salt and Fortune, by Nghi Vo
Series: Singing Hills Cycle #1
Publisher: Tordotcom
Copyright: March 2020
ISBN: 1-250-75029-6
Format: Kindle
Pages: 119
Cleric Chih is a record-keeper and historian from the Singing Hills abbey. They have come to Lake Scarlet with their neixin, the hoopoe Almost Brilliant, because the magical imperial lock placed on the site by the Empress In-yo has just been lifted. They hope to be one of the first to catalog what may be found there, but are surprised to encounter an old woman named Rabbit. Along with a catalog of objects will come a catalog of stories. Empress In-yo came from the north, a political bride for Emperor Sung. She was exiled to Lake Scarlet, to a compound her attendants called Thriving Fortune as a bitter joke. This is the site that Cleric Chih is cataloging. The old woman named Rabbit was one of the empress's attendants. Cleric Chih slowly draws out her stories, often sparked by an object found in the compound and described in the chapter epigraphs. The Empress of Salt and Fortune is a short fantasy novella set in a slightly modified version of China. In-yo comes from a recognizably Mongolian north, but it's colder and has mammoths. Anh, the analogue of China, uses magic to ensure that winter never comes. The court politics, though, seem largely unchanged from our world. The empress from the north had one important duty: delivery of an heir. Once that was done, the empire had no more need for her. That was, as foreshadowed at the start of the novella, not the end of In-yo or Rabbit's story. This is a lovely, layered, and subtle story that was a bit too subtle for my mood when I was reading it. It is the type of story that understates the emotions of the characters and rewards close reading and paying attention. I was not paying enough attention and missed a few significant character developments, which in this concise and careful of a story is not advisable. If you read this, learn from my experience, take it slow, and don't expect major plot events to be signaled in neon. The center of this story is careful and ruthless use of power in a world that is attempting to deny you any, and the sacrifices that one makes to reach that power anyway. Both the empress and Rabbit play bad hands with great effectiveness, taking advantage of the ways in which they're underestimated. It's a sharp and difficult and quietly angry story with a good emotional payoff that doesn't deliver a typical ending. I like stories in which characters make difficult decisions with their eyes wide open, and then refuse to second-guess them or feel bad about the expected consequences. If you like stories about decisive women using and bypassing systems that were stacked against them, you will probably enjoy this novella. The Empress of Salt and Fortune won the 2021 Hugo for best novella, but there is not a lot of fantasy here. It's mostly in the background and setting of a political and character story, and I'm not sure any of it was essential to the plot. What it does have is memorable characters and concise and effective storytelling. We may be living through the golden age of the novella, at least in the science fiction and fantasy world. Chih is a quiet and careful questioner who knows the value of patience and creating space. They don't get much opportunity for characterization in this novella, but what hints we get of their order's approach are intriguing. It appears that this series of novellas will be following Chih rather than the other branches of the story. That will likely keep me reading. Followed by When the Tiger Came Down the Mountain. Rating: 7 out of 10

19 December 2021

Russ Allbery: Review: Raybearer

Review: Raybearer, by Jordan Ifueko
Series: Raybearer #1
Publisher: Amulet Books
Copyright: 2020
ISBN: 1-68335-719-1
Format: Kindle
Pages: 308
Tarisai was raised alone in Bhekina House by an array of servants and tutors who were not allowed to touch her. Glimpses of the world were fleeting and sometimes ended by nailed-shut windows. Her life revolved around her rarely-seen mother, The Lady, who treats her with deep affection but rarely offers a word of praise, instead only pushing her to study harder. The servants whispered behind her back (but still in her hearing) that she was not human. At the age of seven, in a child's attempt to locate her absent mother by sneaking out of the house, she finds her father and is told a piece of the truth: she is the daughter of the Lady and a captive ehru, a djinn. At the age of eleven she's sent with two guardians to Oluwan City, the capital of Aritsar, to enter a competition she knows nothing about, for reasons no one has ever explained. Raybearer is a young adult fantasy novel, the first of a duology. Like a lot of young adult novels, it is a coming of age story that follows Tarisai from the end of her highly manipulated childhood through her introduction to a world she was carefully never taught about. Like a lot of young adult fantasy novels, Tarisai has some unusual abilities. What those are, and why she has them, is perhaps less obvious than it may appear at first. Unlike a lot of young adult fantasy novels, Raybearer is not set in a facsimile of Western Europe, the structure of gods and religion is not obviously derived from Christianity or Greek or Norse mythology, and neither Tarisai nor most of the characters of this story are white. Some of the characters are; Ifueko draws from a grab bag of cultures that does include European as well as African, Middle Eastern, and Asian. But the food, the physical descriptions, the landscape, and the hair and hair styles feel primarily African not in the sense of specific identifiable regions, but in the same way that most fantasy feels European even if the map isn't recognizable. That gives this story a freshness that I found delightful. The mythology of this world shares some similarities to standard fantasy tropes, including a bargain with the underworld that plays a similar role to fae bargains in some European fantasy, but it also goes in different directions and finds atypical balances, which gave the story room to catch me by surprise. The magical center of this book (and series), which Tarisai is carefully not told about until the story starts, is a system for anointing and protecting the emperor: selection of people who swear loyalty to him and each other and become his innermost circle, and thereby grant him magical protection. The emperor himself is the Raybearer, possessing an artifact that makes him invulnerable to one form of death for each member of his council he anoints. At eleven council members, he becomes invulnerable to anything but old age, or an attack from one of the council themselves. As the reader learns early in the book, that last part is important. Tarisai is an assassin; her mother's goal is for her to be selected as a member of the council for the prince, who will become the next emperor. But there is rather more to this system of magic than it may first appear, in a way that adds good depth to the mythology. And there is quite a bit more to Tarisai herself than anyone expects. Tarisai as a protagonist follows a more typical young adult pattern, but it's a formula that works for me. Her upbringing isolated from any other children has left her craving connection, but it also made her self-reliant, stubborn, and good at keeping her own counsel. One of the things that I loved about this book is that she's not thrown into a nest of vipers and cynical politics. Some of that is happening in the background, but the first step of her mother's plan is for her to earn the trust of the prince in a competition with other potential council members, all of whom are, well, kids. They fight (some), but they also make friends, helped along by the goal and requirement that they join a cooperative council or be sent home. That gives the plot a more collaborative and social feel than one would otherwise expect from the setup. Ifueko does a great job juggling a challenging cast size by focusing on a few kids with whom Tarisai strikes up a friendship but giving the others distinct-enough personalities that their presence is still felt in the story. There are two character dynamics that stand out: Tarisai's relationship with Prince Ekundayo, and her friendship with Sanjeet. The first carries much of the weight of the plot, of course; Tarisai is supposed to gain his trust and then kill him, and the reader will be unsurprised that this takes twists and turns no one expected. But Ifueko, refreshingly, does not reach for the stock plot development of a romance to complicate matters, even though many of the characters expect that. To the contrary, this is a rare story that at least hints at an acknowledgment that some people are not interested in romance at all, and there are other forms that mutual respect can take. Tarisai's relationship with Sanjeet is a different type of depth: two kids with very different histories finding a common understanding in the ways that they were both abused, and create space for each other. It's a great friendship that includes some deeply touching moments. It took me a bit to get into this book, but once Tarisai starts finding her feet and navigating her new relationships, I was engrossed. The story takes a sharp and nasty turn that was hard to read, but Ifueko chooses to turn it into a story of resiliency rather than survival, which makes it much easier to read than it could have been. She also pulls off the kind of plot that complicates and deepens the motives of the obvious villains in a way that gives the story much greater heft, but without disregarding the damage that they have done. I think the plot did fall apart a bit at the end of the book, with too much quick travel and world-building revelations at the cost of development of the relationships that were otherwise at the center of the book, but I'm hoping the sequel will pull those threads back together. And it's so refreshing to read a fantasy novel of this type with a different setting. It's not perfect: Ifueko falls back on Planet of the Hats regional characterization in a few places, and Songland is so obviously Korea that it felt jarring and out of place. Christianity also snuck its nose into the world-building tent near the end in ways that bugged me a bit, although it was subtle enough that I think most readers won't notice. But compared to most fantasy settings, it feels original and fresh. More of this! Ifueko starts this book with a wonderfully memorable dedication:
For the kid scanning fairy tales for a hero with a face like theirs. And for the girls whose stories we compressed into pities and wonders, triumphs and cautions, without asking, even once, for their names.
I think she was successful on both parts of that promise, and it makes for some great reading. Recommended. Followed by Redemptor. Rating: 8 out of 10

16 December 2021

Raphaël Hertzog: Freexian s report about Debian Long Term Support, November 2021

A Debian LTS logo
Every month we review the work funded by Freexian s Debian LTS offering. Please find the report for November below. Debian project funding We continue to looking forward to hearing about Debian project proposals from various Debian stakeholders. This month has seen work on a survey that will go out to Debian Developers to gather feedback on what they think should be the priorities for funding in the project. Learn more about the rationale behind this initiative in this article. Debian LTS contributors In November 13 contributors were paid to work on Debian LTS, their reports are available below. If you re interested in participating in the LTS or ELTS teams, we welcome participation from the Debian community. Simply get in touch with Jeremiah if you are interested in participating. Evolution of the situation In November we released 31 DLAs. The security tracker currently lists 23 packages with a known CVE and the dla-needed.txt file has 16 packages needing an update. Thanks to our sponsors Sponsors that joined recently are in bold.

14 December 2021

Timo Jyrinki: Working and warming up cats

How to disable internal keyboard/touchpad when a cat arrives
I m using an external keyboard (1) and mouse (2), but the laptop lid is usually still open for better cooling. That means the internal keyboard (3) and touchpad (4) made of comfortable materials are open to be used by a cat searching for warmth (7), in the obvious every time case that a normal non-heated nest (6) is not enough.
The problem is, everything goes chaotic at that point in the default configuration. The solution is to have quick shortcuts in my Dash to Dock (8) to both disable (10) and enable (9) keyboard and touchpad at a very rapid pace.It is to be noted that I m not disabling the touch screen (5) by default, because most of the time the cat is not leaning on it there is also the added benefit that if one forgets about the internal keyboard and touchpad disabling and detaches the laptop from the USB-C monitor (11), there s the possibility of using the touch screen and on-screen keyboard to type in the password and tap on the keyboard/touchpad enabling shortcut button again. If also touch screen was disabled, the only way would be to go back to an external keyboard or reboot.So here are the scripts. First, the disabling script (pardon my copy-paste use of certain string manipulation tools):
dconf write /org/gnome/desktop/peripherals/touchpad/send-events "'disabled'"
sudo killall evtest
sudo evtest --grab $(sudo libinput list-devices grep -A 1 "AT Translated Set 2 keyboard" tail -n 1 sed 's/.*\/dev/\/dev/') &
sudo evtest --grab $(sudo libinput list-devices grep -A 1 "Dell WMI" tail -n 1 sed 's/.*\/dev/\/dev/') &
sudo evtest --grab $(sudo libinput list-devices grep -A 1 "Power" grep Kernel tail -n 1 sed 's/.*\/dev/\/dev/') &
sudo evtest --grab $(sudo libinput list-devices grep -A 1 "Power" grep Kernel head -n 1 sed 's/.*\/dev/\/dev/') &
sudo evtest --grab $(sudo libinput list-devices grep -A 1 "Sleep" grep Kernel tail -n 1 sed 's/.*\/dev/\/dev/') &
sudo evtest --grab $(sudo libinput list-devices grep -A 1 "HID" grep Kernel head -n 1 sed 's/.*\/dev/\/dev/') &
sudo evtest --grab $(sudo libinput list-devices grep -A 1 "HID" tail -n 1 sed 's/.*\/dev/\/dev/') &
#sudo evtest --grab $(sudo libinput list-devices grep -A 1 "ELAN" tail -n 1 sed 's/.*\/dev/\/dev/') # Touch screen
And the associated ~/.local/share/applications/disable-internal-input.desktop:
[Desktop Entry]
Version=1.0
Name=Disable internal input
GenericName=Disable internal input
Exec=/bin/bash -c /home/timo/Asiakirjat/helpers/disable-internal-input.sh
Icon=yast-keyboard
Type=Application
Terminal=false
Categories=Utility;Development;
Here s the enabling script:
dconf write /org/gnome/desktop/peripherals/touchpad/send-events "'enabled'"
sudo killall evtest
and the desktop file:
[Desktop Entry]
Version=1.0
Name=Enable internal input
GenericName=Enable internal input
Exec=/bin/bash -c /home/timo/Asiakirjat/helpers/enable-internal-input.sh
Icon=/home/timo/.local/share/icons/hicolor/scalable/apps/yast-keyboard-enable.png
Type=Application
Terminal=false
Categories=Utility;Development;
With these, if I sense a cat or am just proactive enough, I press Super+9. If I m about to detach my laptop from the monitor, I press Super+8. If I forget the latter (usually this is the case) and haven t yet locked the screen, I just tap the enabling icon on the touch screen.

11 December 2021

Neil Williams: Diversity and gender

As a follow on to a previous blog entry of mine, Free and Open, I feel it worthwhile to do my bit to dismantle the pseudo-science and over simplification in the idea that gender is binary at a biological level.
TL;DR: Science simply does not support binary sexes or binary genders. Truth is a bit more complicated.
There is certainty and there are binary answers in mathematics. Things get less definitive in physics, certainly as soon as quantum is broached. Processes become more of an equilibrium between states in chemistry, never wholly one or the other. Yes, there is the oddity of absolute zero but no experiment has yet achieved that fully. It is accurate to describe physics as a development of applied mathematics and to view chemistry as applied physics. Biology, at the biochemical level, is applied chemistry. The sciences build on each other, "on the shoulders of giants", but at each level, some certainty is lost, some amount of uncertainty is expanded and measurements become probabilities, proportions and percentages. Biology is dependent on biochemistry - chemistry is how a biological change results in a different organism. Physics is how that chemical change occurs - temperature, pressure and physical states are inherent to all chemical changes. Outside laboratory constraints, few chemical reactions, especially in organic chemistry, produce one and only one result from two or more known reagents. In biology, everyone is familiar with genetic mutations but a genetic mutation only happens because a biochemical reaction (hydrogen bonding of nucleobases) does not always produce the expected result. Every cell division, every viral infection, there is a finite probability that a change will occur. It might be a small number but it is never zero and can never be dismissed. This is obvious in the current Covid pandemic - genetic mutations result in new variants. Some variants are inviable, some variants produce no net change in the way that the viral particles infect adjacent cells. Sometimes, a mutation happens that changes everything. These mutations are not mistakes - these are simply changes with undetermined outcomes. Genetic changes are the foundation of biodiversity and variety is what allows lifeforms of all kinds to survive changes in environmental factors and/or changes in prevalent diseases. It is precisely the same in humans, particularly in one of the principle spheres of human life that involves replicating genetic material - the creation of gametes for sexual reproduction. Every single time any DNA is copied, there is a finite chance that a different base will be put in place compared to the original. Copying genetic material is therefore non-binary. Given precisely the same initial conditions, the result is not always predictable and the range of how the results vary from one to another increases with every iteration. Let me stress that - at the molecular level, no genetic operation in any biological lifeform has a truly binary result. Repeat that operation sufficiently often and an unexpected result WILL inevitably occur. It is a mathematical certainty that genetic changes will arise by attempting precisely the same genetic operation enough times. Genetic changes are fundamental to how lifeforms survive changing conditions. Life would likely have died out a long time ago on this planet if every genetic operation was perfect. Diversity is life. Similarity leads to extinction. Viral load is interesting at this point. Someone can be infected with a virus, including coronavirus, by encountering a small number of viral particles. Some viruses, it may be a few hundred, some viruses may need a few thousand particles to infect a vulnerable host. But here's the thing, for that host to be at risk of infecting another host, the virus needs the host to produce billions upon billions of copies of the virus by taking over the genetic machinery within a huge number of cells in the host. This, as is accepted with Covid, is before the virus has been copied enough times to produce symptoms in the host. Before those symptoms become serious, billions more copies will be made. The numbers become unimaginable - and that is within a single host, let alone the 265 million (and counting) hosts in the current Covid19 pandemic. It's also no wonder that viral infections cause tiredness, the infection is diverting huge resources to propagating itself - before even considering the activity of the immune system. It is idiocy of the highest order to expect all those copies to be identical. The rise of variants is inevitable - indeed essential - in all spheres of biology. A single viral particle is absolutely no threat of any kind - it must first get inside and then copy the genetic information in a host cell. This is where the complexity lies in the definition of life itself. A virus can be considered a lifeform but it is only able to reproduce using another, more complex, lifeform. In truth, a viral particle does not and cannot mutate. The infected host mutates the virus. The longer it takes that host to clear the infection, the more mutations that host will create and then potentially spread to others. Now apply this to the creation of gametes in humans. With seven billion humans, the amount of copying of genetic material is not as large as the pandemic but it is still easy for everyone to understand that children do not merely combine the DNA of both parents. Changes happen. Human sexual reproduction is not as simple as 1 + 1 = 2. Sometimes, the copying of the genetic material produces an unexpected result. Sexual reproduction itself is non-binary. Sexual reproduction is not easy or simple for lifeforms to adopt - the diversity which results from the non-binary operations are exactly why so many lifeforms invest so much energy in reproducing in this way. Whilst many genetic changes in humans will be benign or beneficial, I d like to take an example of a genetic disorder that results from the non-binary nature of sex. Humans can be born with the XY phenotype - i.e. at a genetic level, the individual has the same combination of chromosomes as another XY individual but there are changes within the genes in those chromosomes. We accept this, some children of blonde parents do not have blonde hair, etc. There are also genetic changes where an XY phenotype is not binary. Some people, who at a genetic level would be almost identical to another person who is genetically male, have a genetic mutation which makes it impossible for the cells of that individual to respond to androgens (testosterone). (See Androgen insensitivity syndrome). Genetically, that individual has an X and a Y chromosome, just like many other individuals. However, due to a change in how the genes on those chromosomes were copied, that individual is biologically incapable of constructing the secondary sexual characteristics of a male. At a genetic level, the individual has the XY phenotype of a male. At the physical level, the individual has all the sexual characteristics of a female and none of the sexual characteristics of a male. The gender of that individual is not binary. Treatment is centred on supporting the individual and minimising some risks from the inactive genes on the Y chromosome. Human sexual reproduction is non-binary. The results of any sexual reproduction in humans will not always produce the binary option of male or female. It is a lie to claim that human gender is binary. The science is in plain view and cannot be ignored. Identifying as non-binary is not a "cop out" - it can be a biological, genetic, scientific fact. Human sexuality and gender are malleable. Where genetic changes result in symptoms, these can be ameliorated by treatment with human sex hormones, like oestrogen and testosterone. There are valid medical uses for anabolic steroids and hormone replacement therapies to help individuals who, at a genetic level, have non-binary gender. These treatments can help align the physical outer signs with the personality and identity of the individual, whether with or without surgery. It is unacceptable to abandon such people to suffer life long discrimination and harassment by imposing a binary definition that has no basis in science. When a human being has an XY phenotype, that human being is not necessarily male. That individual will be on a spectrum from female (left unaffected by sex hormones in the womb, the foetus will be female, even with an X and a Y chromosome), to various degrees of male. So, at a genetic, biological level, it is a scientific fact that human beings do not have binary gender. There is no evidence that this is new to the modern era, there is no scientific basis for thinking that copying of genetic material was somehow perfectly reliable in earlier history, or that such mutations are specific to homo sapiens. Changes in genetic material provide the diversity to fight infections and adapt to changing environmental factors. Species have and will continue to go extinct if this diversity is absent. With that out of the way, it is no longer a stretch to encompass other aspects of human non-binary genders beyond the known genetic syndromes based on changes in the XY phenotype. Science has not uncovered all of the ways that genes affect personality, behaviour, or identity. How other, less studied, genetic changes affect the much more subtle human facets, especially anything to do with consciousness, identity, personality, sexuality and behaviour, is guesswork. All of these facets can and likely are being affected by genetic factors as well as environmental factors in an endless range of permutations. Personality traits are a beautiful and largely unknowable blend of genes and environment. Genetic information has a finite probability of changes at each and every iteration. Environmental factors are more akin to chaos theory. The idea that the results will fit into binary constructs is laughable. Human society puts huge emphasis on societal norms. Individuals who do not fit into those norms suffer discrimination. The norms themselves have evolved over time as a response to various influences on human civilisation but most are not based on science. It is up to all humans in that society to call out discrimination, to call for changes in the accepted norms and support those who are marginalised. It is a precarious balance, one that humans rarely get right, but it must be based on an acceptance that variation is the natural state. Artificial constraints, like binary genders, must be dismantled because human beings and human sexual reproduction are not binary. To those who think, "well it is for 99%", think again about Covid. 99% (or closer to 98%) of infected humans recover without notable after effects. That has still crippled the nations of the globe and humbled all those who tried to deny it. Five million human beings are dead because "most infected people recover". Just because something only affects a proportion of human beings does not invalidate the suffering of those humans and the discrimination that those humans will face. Societal norms are not necessarily correct. Religious and other influences typically obscure and ignore scientific fact and undermine human kindness. The scientific truth of life on this planet is that gender is not binary. The more complex the lifeform, the more factors will affect where on the spectrum any one individual will appear. Just because we do not yet fully understand how genes affect human personality and sexuality, does not invalidate the science that variation is the natural order. My previous blog about diversity is not just about male vs female, one nationality vs another, one ethnicity compared to another. Diversity is diverse. Diversity requires accepting that every facet of humanity is subject to variation. That leads to tension at times, it is inevitable. Tension against societal norms, tension against discrimination, tension around those individuals who would abuse the tolerance of others for their own gratification or from their own ignorance. None of us are perfect, none of us have any of this fully sorted and all of us will make mistakes. Personally, I try to respect those around me. I will use whatever pronouns and other conventions that the person requests, from their perspective and not mine. To do otherwise is to deny the natural order and to deny the science. Celebrate all diversity, it is the very stuff of life. The discussions around (typically female) bathroom facilities often miss the point. The concern is not about individuals who describe themselves as non-binary. The concern is about individuals who are fully certain of their own sexuality and who act as sexual predators for their own gratification. These people are acting out a lie for their own ends. The problem people are the predators, so stop blaming the victims who are just as at risk as anyone else who identifies as female. Maybe the best people to spot such predators are those who are non-binary, who have had to pretend to fit into societal norms. Just as travel can be a good antidote to racism, openness and discussion can be a tool to undermine the lies of sexual predators and reassure those who are justifiably fearful. There can never be a biological binary test of gender, there can never be any scientific justification for binary division of facilities. Humanity itself is not binary, even life itself has blurry borders around comas, suspended animation and locked-in syndrome. Legal definitions of human death vary around the world. The only common thread I have ever found is: Be kind to each other. If you find anything above objectionable, then I can only suggest that you reconsider the science and learn to be kind to your fellow humans. None of us are getting out of this alive. I Think You ll Find It s a Bit More Complicated Than That - Ben Goldacre ISBN 978-0-00-750514-2 https://www.amazon.co.uk/dp/B00HATQA8K/ https://en.wikipedia.org/wiki/Androgen_insensitivity_syndrome https://www.bbc.co.uk/news/world-51235105 https://en.wikipedia.org/wiki/Nucleobase My degree is in pharmaceutical sciences and I practised community and hospital pharmacy for 20 years before moving into programming. I have direct experience of supporting people who were prescribed hormones to transition their physical characteristics to match their personal identity. I had a Christian upbringing but my work showed me that those religious norms were incompatible with being kind to others, so I rejected religion and I now consider myself a secular humanist.

5 December 2021

Reproducible Builds: Reproducible Builds in November 2021

Welcome to the November 2021 report from the Reproducible Builds project. As a quick recap, whilst anyone may inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries. The motivation behind the reproducible builds effort is therefore to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. If you are interested in contributing to our project, please visit our Contribute page on our website.
On November 6th, Vagrant Cascadian presented at this year s edition of the SeaGL conference, giving a talk titled Debugging Reproducible Builds One Day at a Time:
I ll explore how I go about identifying issues to work on, learn more about the specific issues, recreate the problem locally, isolate the potential causes, dissect the problem into identifiable parts, and adapt the packaging and/or source code to fix the issues.
A video recording of the talk is available on archive.org.
Fedora Magazine published a post written by Zbigniew J drzejewski-Szmek about how to Use Diffoscope in packager workflows, specifically around ensuring that new versions of a package do not introduce breaking changes:
In the role of a packager, updating packages is a recurring task. For some projects, a packager is involved in upstream maintenance, or well written release notes make it easy to figure out what changed between the releases. This isn t always the case, for instance with some small project maintained by one or two people somewhere on GitHub, and it can be useful to verify what exactly changed. Diffoscope can help determine the changes between package releases. [ ]

kpcyrd announced the release of rebuilderd version 0.16.3 on our mailing list this month, adding support for builds to generate multiple artifacts at once.
Lastly, we held another IRC meeting on November 30th. As mentioned in previous reports, due to the global events throughout 2020 etc. there will be no in-person summit event this year.

diffoscope diffoscope is our in-depth and content-aware diff utility. Not only can it locate and diagnose reproducibility issues, it can provide human-readable diffs from many kinds of binary formats. This month, Chris Lamb made the following changes, including preparing and uploading versions 190, 191, 192, 193 and 194 to Debian:
  • New features:
    • Continue loading a .changes file even if the referenced files do not exist, but include a comment in the returned diff. [ ]
    • Log the reason if we cannot load a Debian .changes file. [ ]
  • Bug fixes:
    • Detect XML files as XML files if file(1) claims if they are XML files or if they are named .xml. (#999438)
    • Don t duplicate file lists at each directory level. (#989192)
    • Don t raise a traceback when comparing nested directories with non-directories. [ ]
    • Re-enable test_android_manifest. [ ]
    • Don t reject Debian .changes files if they contain non-printable characters. [ ]
  • Codebase improvements:
    • Avoid aliasing variables if we aren t going to use them. [ ]
    • Use isinstance over type. [ ]
    • Drop a number of unused imports. [ ]
    • Update a bunch of %-style string interpolations into f-strings or str.format. [ ]
    • When pretty-printing JSON, mark the difference as being reformatted, additionally avoiding including the full path. [ ]
    • Import itertools top-level module directly. [ ]
Chris Lamb also made an update to the command-line client to trydiffoscope, a web-based version of the diffoscope in-depth and content-aware diff utility, specifically only waiting for 2 minutes for try.diffoscope.org to respond in tests. (#998360) In addition Brandon Maier corrected an issue where parts of large diffs were missing from the output [ ], Zbigniew J drzejewski-Szmek fixed some logic in the assert_diff_startswith method [ ] and Mattia Rizzolo updated the packaging metadata to denote that we support both Python 3.9 and 3.10 [ ] as well as a number of warning-related changes[ ][ ]. Vagrant Cascadian also updated the diffoscope package in GNU Guix [ ][ ].

Distribution work In Debian, Roland Clobus updated the wiki page documenting Debian reproducible Live images to mention some new bug reports and also posted an in-depth status update to our mailing list. In addition, 90 reviews of Debian packages were added, 18 were updated and 23 were removed this month adding to our knowledge about identified issues. Chris Lamb identified a new toolchain issue, absolute_path_in_cmake_file_generated_by_meson.
Work has begun on classifying reproducibility issues in packages within the Arch Linux distribution. Similar to the analogous effort within Debian (outlined above), package information is listed in a human-readable packages.yml YAML file and a sibling README.md file shows how to classify packages too. Finally, Bernhard M. Wiedemann posted his monthly reproducible builds status report for openSUSE and Vagrant Cascadian updated a link on our website to link to the GNU Guix reproducibility testing overview [ ].

Software development The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including: Elsewhere, in software development, Jonas Witschel updated strip-nondeterminism, our tool to remove specific non-deterministic results from a completed build so that it did not fail on JAR archives containing invalid members with a .jar extension [ ]. This change was later uploaded to Debian by Chris Lamb. reprotest is the Reproducible Build s project end-user tool to build the same source code twice in widely different environments and checking whether the binaries produced by the builds have any differences. This month, Mattia Rizzolo overhauled the Debian packaging [ ][ ][ ] and fixed a bug surrounding suffixes in the Debian package version [ ], whilst Stefano Rivera fixed an issue where the package tests were broken after the removal of diffoscope from the package s strict dependencies [ ].

Testing framework The Reproducible Builds project runs a testing framework at tests.reproducible-builds.org, to check packages and other artifacts for reproducibility. This month, the following changes were made:
  • Holger Levsen:
    • Document the progress in setting up snapshot.reproducible-builds.org. [ ]
    • Add the packages required for debian-snapshot. [ ]
    • Make the dstat package available on all Debian based systems. [ ]
    • Mark virt32b-armhf and virt64b-armhf as down. [ ]
  • Jochen Sprickerhof:
    • Add SSH authentication key and enable access to the osuosl168-amd64 node. [ ][ ]
  • Mattia Rizzolo:
    • Revert reproducible Debian: mark virt(32 64)b-armhf as down - restored. [ ]
  • Roland Clobus (Debian live image generation):
    • Rename sid internally to unstable until an issue in the snapshot system is resolved. [ ]
    • Extend testing to include Debian bookworm too.. [ ]
    • Automatically create the Jenkins view to display jobs related to building the Live images. [ ]
  • Vagrant Cascadian:
    • Add a Debian package set group for the packages and tools maintained by the Reproducible Builds maintainers themselves. [ ]


If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

2 December 2021

Paul Tagliamonte: Processing IQ data formats (Part 1/5)

This post is part of a series called "PACKRAT". If this is the first post you've found, it'd be worth reading the intro post first and then looking over all posts in the series.
When working with SDRs, information about the signals your radio is receiving are communicated by streams of IQ data. IQ is short for In-phase and Quadrature , which means 90 degrees out of phase. Values in the IQ stream are complex numbers, so converting them to a native complex type in your language helps greatly when processing the IQ data for meaning. I won t get too deep into what IQ is or why complex numbers (mostly since I don t think I fully understand it well enough to explain it yet), but here s some basics in case this is your first interaction with IQ data before going off and reading more.
Before we get started at any point, if you feel lost in this post, it's OK to take a break to do a bit of learning elsewhere in the internet. I'm still new to this, so I'm sure my overview in one paragraph here won't help clarify things too much. This took me months to sort out on my own. It's not you, really! I particularly enjoyed reading visual-dsp.switchb.org when it came to learning about how IQ represents signals, and Software-Defined Radio for Engineers for a more general reference.
Each value in the stream is taken at a precisely spaced sampling interval (called the sampling rate of the radio). Jitter in that sampling interval, or a drift in the requested and actual sampling rate (usually represented in PPM, or parts per million how many samples out of one million are missing) can cause errors in frequency. In the case of a PPM error, one radio may think it s 100.1MHz and the other may think it s 100.2MHz, and jitter will result in added noise in the resulting stream. A single IQ sample is both the real and imaginary values, together. The complex number (both parts) is the sample. The number of samples per second is the number of real and imaginary value pairs per second. Each sample is reading the electrical energy coming off the antenna at that exact time instant. We re looking to see how that goes up and down over time to determine what frequencies we re observing around us. If the IQ stream is only real-valued measures (e.g., float values rather than complex values reading voltage from a wire), you can still send and receive signals, but those signals will be mirrored across your 0Hz boundary. That means if you re tuned to 100MHz, and you have a nearby transmitter at 99.9MHz, you d see it at 100.1MHz. If you want to get an intuitive understanding of this concept before getting into the heavy math, a good place to start is looking at how Quadrature encoders work. Using complex numbers means we can see up in frequency as well as down in frequency, and understand that those are different signals. The reason why we need negative frequencies is that our 0Hz is the center of our SDR s tuned frequency, not actually at 0Hz in nature. Generally speaking, it s doing loads in hardware (and firmware!) to mix the raw RF signals with a local oscillator to a frequency that can be sampled at the requested rate (fundamentally the same concept as a superheterodyne receiver), so a frequency of -10MHz means that signal is 10 MHz below the center of our SDR s tuned frequency. The sampling rate dictates the amount of frequency representable in the data stream. You ll sometimes see this called the Nyquist frequency. The Nyquist Frequency is one half of the sampling rate. Intuitively, if you think about the amount of bandwidth observable as being 1:1 with the sampling rate of the stream, and the middle of your bandwidth is 0 Hz, you would only have enough space to go up in frequency for half of your bandwidth or half of your sampling rate. Same for going down in frequency.

Float 32 / Complex 64 IQ samples that are being processed by software are commonly processed as an interleaved pair of 32 bit floating point numbers, or a 64 bit complex number. The first float32 is the real value, and the second is the imaginary value.
I#0
Q#0
I#1
Q#1
I#2
Q#2
The complex number 1+1i is represented as 1.0 1.0 and the complex number -1-1i is represented as -1.0 -1.0. Unless otherwise specified, all the IQ samples and pseudocode to follow assumes interleaved float32 IQ data streams. Example interleaved float32 file (10Hz Wave at 1024 Samples per Second)

RTL-SDR IQ samples from the RTL-SDR are encoded as a stream of interleaved unsigned 8 bit integers (uint8 or u8). The first sample is the real (in-phase or I) value, and the second is the imaginary (quadrature or Q) value. Together each pair of values makes up a complex number at a specific time instant.
I#0
Q#0
I#1
Q#1
I#2
Q#2
The complex number 1+1i is represented as 0xFF 0xFF and the complex number -1-1i is represented as 0x00 0x00. The complex number 0+0i is not easily representable since half of 0xFF is 127.5.
Complex Number Representation
1+1i []uint8 0xFF, 0xFF
-1+1i []uint8 0x00, 0xFF
-1-1i []uint8 0x00, 0x00
0+0i []uint8 0x80, 0x80 or []uint8 0x7F, 0x7F
And finally, here s some pseudocode to convert an rtl-sdr style IQ sample to a floating point complex number:
...
in = []uint8 0x7F, 0x7F 
real = (float(iq[0])-127.5)/127.5
imag = (float(iq[1])-127.5)/127.5
out = complex(real, imag)
....
Example interleaved uint8 file (10Hz Wave at 1024 Samples per Second)

HackRF IQ samples from the HackRF are encoded as a stream of interleaved signed 8 bit integers (int8 or i8). The first sample is the real (in-phase or I) value, and the second is the imaginary (quadrature or Q) value. Together each pair of values makes up a complex number at a specific time instant.
I#0
Q#0
I#1
Q#1
I#2
Q#2
Formats that use signed integers do have one quirk due to two s complement, which is that the smallest negative number representable s absolute value is one more than the largest positive number. int8 values can range between -128 to 127, which means there s bit of ambiguity in how +1, 0 and -1 are represented. Either you can create perfectly symmetric ranges of values between +1 and -1, but 0 is not representable, have more possible values in the negative range, or allow values above (or just below) the maximum in the range to be allowed. Within my implementation, my approach has been to scale based on the max integer value of the type, so the lowest possible signed value is actually slightly smaller than -1. Generally, if your code is seeing values that low the difference in step between -1 and slightly less than -1 isn t very significant, even with only 8 bits. Just a curiosity to be aware of.
Complex Number Representation
1+1i []int8 127, 127
-1+1i []int8 -128, 127
-1-1i []int8 -128, -128
0+0i []int8 0, 0
And finally, here s some pseudocode to convert a hackrf style IQ sample to a floating point complex number:
...
in = []int8 -5, 112 
real = (float(in[0]))/127
imag = (float(in[1]))/127
out = complex(real, imag)
....
Example interleaved int8 file (10Hz Wave at 1024 Samples per Second)

PlutoSDR IQ samples from the PlutoSDR are encoded as a stream of interleaved signed 16 bit integers (int16 or i16). The first sample is the real (in-phase or I) value, and the second is the imaginary (quadrature or Q) value. Together each pair of values makes up a complex number at a specific time instant. Almost no SDRs capture at a 16 bit depth natively, often you ll see 12 bit integers (as is the case with the PlutoSDR) being sent around as 16 bit integers. This leads to the next possible question, which is are values LSB or MSB aligned? The PlutoSDR sends data LSB aligned (which is to say, the largest real or imaginary value in the stream will not exceed 4095), but expects data being transmitted to be MSB aligned (which is to say the lowest set bit possible is the 5th bit in the number, or values can only be set in increments of 16). As a result, the quirk observed with the HackRF (that the range of values between 0 and -1 is different than the range of values between 0 and +1) does not impact us so long as we do not use the whole 16 bit range.
Complex Number Representation
1+1i []int16 32767, 32767
-1+1i []int16 -32768, 32767
-1-1i []int16 -32768, -32768
0+0i []int16 0, 0
And finally, here s some pseudocode to convert a PlutoSDR style IQ sample to a floating point complex number, including moving the sample from LSB to MSB aligned:
...
in = []int16 -15072, 496 
// shift left 4 bits (16 bits - 12 bits = 4 bits)
 // to move from LSB aligned to MSB aligned.
 in[0] = in[0] << 4
in[1] = in[1] << 4
real = (float(in[0]))/32767
imag = (float(in[1]))/32767
out = complex(real, imag)
....
Example interleaved i16 file (10Hz Wave at 1024 Samples per Second)

Next Steps Now that we can read (and write!) IQ data, we can get started first on the transmitter, which we can (in turn) use to test receiving our own BPSK signal, coming next in Part 2!

30 November 2021

Russell Coker: Links November 2021

The Guardian has an amusing article by Sophie Elmhirst about Libertarians buying a cruise ship to make a seasteading project off the coast of Panama [1]. It turns out that you need permits etc to do this and maintaining a ship is expensive. Also you wouldn t want to mine cryptocurrency in a ship cabin as most cabins are small and don t have enough airconditioning to remain pleasant if you dump 1kW or more into the air. NPR has an interesting article about the reaction of the NRA to the Columbine shootings [2]. Seems that some NRA person isn t a total asshole and is sharing their private information, maybe they are dying and are worried about going to hell. David Brin wrote an insightful blog post about the singleton hypothesis where he covers some of the evidence of autocratic societies failing [3]. I think he makes a convincing point about a single centralised government for human society not being viable. But something like the EU on a world wide scale could work well. Ken Shirriff wrote an interesting blog post about reverse engineering the Yamaha DX7 synthesiser [4]. The New York Times has an interesting article about a Baboon troop that became less aggressive after the alpha males all died at once from tuberculosis [5]. They established a new more peaceful culture that has outlived the beta males who avoided tuberculosis. The Guardian has an interesting article about how sequencing the genomes of the entire population can save healthcare costs while improving the health of the population [6]. This is somthing wealthy countries should offer for free to the world population. At a bit under $1000 per test that s only about $7 trillion to test everyone, and of course the price should drop significantly if there were billions of tests being done. The Strategy Bridge has an interesting article about SciFi books that have useful portrayals of military strategy [7]. The co-author is Major General Mick Ryan of the Australian Army which is noteworthy as Major General is the second highest rank in use by the Australian Army at this time. Vice has an interesting article about the co-evolution of penises and vaginas and how a lot of that evolution is based on avoiding impregnation from rape [8]. Cory Doctorow wrote an insightful Medium article about the way that governments could force interoperability through purchasing power [9]. Cory Doctorow wrote an insightful article for Locus Magazine about imagining life after capitalism and how capitalism might be replaced [10]. We need a Star Trek future! Arstechnica has an informative article about new developmenet in the rowhammer category of security attacks on DRAM [11]. It seems that DDR4 with ECC is the best current mitigation technique and that DDR3 with ECC is harder to attack than non-ECC RAM. So the thing to do is use ECC on all workstations and avoid doing security critical things on laptops because they can t juse ECC RAM.

28 November 2021

Joachim Breitner: Zero-downtime upgrades of Internet Computer canisters

TL;DR: Zero-downtime upgrades are possible if you stick to the basic actor model.

Background DFINITY s Internet Computer provides a kind of serverless compute platform, where the services are WebAssemmbly programs called canisters . These services run without stopping (or at least that s what it feels like from the service s perspective; this is called orthogonal persistence ), and process one message after another. Messages not only come from the outside ( ingress calls), but are also exchanged between canisters. On top of these uni-directional messages, the system provides the concept of inter-canister calls , which associates a respondse message with the outgoing message, and guarantees that a response will come. This RPC-like interface allows canister developers to program in the popular async/await model, where these inter-canister calls look almost like normal function calls, and the subsequent code is suspended until the response comes back.

The problem This is all very well, until you try to upgrade your canister, i.e. install new code to fix a bug or add a feature. Because if you used the await pattern, there may still be suspended computations waiting for the response. If you swap out the program now, the code of that suspended computation will no longer be present, and the response cannot be handled! Worse, because of an infelicity with the current system s API, when the response comes back, it may actually corrupt your service s state. That is why upgrading a canister requires stopping it first, which means waiting for all outstanding calls to come back. During this time, your canister is not available for new calls (so there is downtime), and worse, the length of the downtime is at the whims of the canisters you called they could withhold the response ad infinitum, rendering your canister unupgradeable. Clearly, this is not acceptable for any serious application. In this post, I ll explore some of the ways to mitigate this problem, and how to create canisters that are safely instantanously (no downtime) upgradeable.

It s a spectrum Some canisters are trivially upgradeable, for others all hope is lost; it depends on what the canister does and how. As an overview, here is the spectrum:
  1. A canister that never performs inter-canister calls can always be upgraded without stopping.
  2. A canister that only does one-way calls, and does them in a particular way (see below), can always be upgraded without stopping.
  3. A canister that performs calls, and where it is acceptable to simply drop outstanding repsonses, can always be upgraded without stopping, once the System API has been improved and your Canister Development Kit (CDK; Motoko or Rust) has adapted.
  4. A canister that performs calls, but uses explicit continuations to handle, responses instead of the await-convenience, based on an eventually fixed System API, can be upgradeded without stopping, and will even handle responses afterwards.
  5. A canister that uses await to do inter-canister call cannot be upgraded without stopping.
In this post I will explain 2, which is possible now, in more detail. Variant 3 and 4 only become reality if and when the System API has improved.

One-way calls A one-way call is a call where you don t care about the response; neither the replied data, nor possible failure conditions. Since you don t care about the response, you can pass an invalid continuation to the system (technical detail: a Wasm table index of -1). Because it is invalid for any (realistic) Wasm module, it will stay invalid even after an upgrade, and the problem of silent corruption mentioned above is avoided. And otherwise it s fine for this to be invalid: it means the canister traps once the response comes back, which is harmeless (and possibly even cheaper than a do-nothing computation). This requires your CDK to support this kind of call. Mostly incidential, Motoko (and Candid) actually have the concept of one-way call in their type system, namely shared functions with return type () instead of async ... (Motoko is actually older than the system, and not every prediction about what the system will provide has proven successful). So, pending this PR to be released, Motoko will implement one-way calls in this way. On Rust, you have to use the System API directly or wait for cdk-rs to provide this ability (patches welcome, happy to advise). You might wonder: How are calls useful if I don t get to look at the response? Of course, this is a set-back calls with responses are useful, and await is convenient. And if you have to integrate with an existing service that only provides normal calls, you are out of luck. But if you get to design the canister and all called canisters together, it may be possible to use only one-way messages. You d be programming in the plain actor model now, with all its advantages (simple concurrency, easy to upgrade, general robustness). Consider for example a token ledger canister, not unlike the ICP ledger canister. For the most part, it doesn t have to do any outgoing calls (and thus be trivially upgradeble). But say we need to add notify functionality, where the ledger canister tells other canisters about a transaction. This is a good example for a one-way call: Maybe the ledger canister doesn t care if that notification was received? The ICP leder does care (once it comes back successful, this particular notification cannot be sent again), but maybe your ledger can do it differently: let the other canister confirm the receip via another one-way call, instead of via the reply; or simply charge for each notification and do not worry about repeated notifications. Maybe you want to add archiving functionality, where the ledger canister streams its data to an archive canister. There, again, instead of using successful responses to confirm receipt, the archive canister can ping the ledger canister with the latest received index directly. Yes, it changes the programming model a bit, and all involved parties have to play together, but the gain (zero-downtime upgrades) is quite valuable, and removes a fair number of other sources of issues.

And in the future? The above is possible with today s Internet Computer. If the System API gets improves the way I hope it will be, you have a possible middle ground: You still don t get to use await and instead have to write your response handler as separate functions, but this way you can call any canister again, and you get the system s assistance in mapping responses to calls. With this in place, any canister can be rewritten to a form that supports zero-downtime upgrades, without affecting its interface or what the canister can do.

19 November 2021

Neil Williams: git worktrees

A few scenarios have been problematic with git and I've now discovered git worktrees which help with each. You could go to the trouble of making a new directory and re-cloning the same tree. However, a local commit in one tree is then not accessible to the other tree. You could commit everything every time, but with a dirty tree, that involves sorting out the .gitignore rules as well. That could well be pointless with an experimental change. Git worktrees allow multiple filesystems from a single git tree. Commits on any branch are visible from other branches, even when the commit was on a different worktree. This makes things like cherry-picking easy, without needing to push pointless changes or branches. Branches on a worktree can be rebased as normal, with the benefit that commit hashes from other local changes are available for reference and cherry-picks. I'm sure git worktrees are not new. However, I've only started using them recently and others have asked about how the worktree operates. Creating a new tree can be done with a new or existing branch. To make it easier, set the new directory at the same time, usually in ../ New branch (branched from the current branch):
git worktree add -b branch_name ../branch_name
Existing branch - note, slightly different syntax here, specify the commit-ish last (branch name, tag or hash):
git worktree add ../branch_name branch_name
git worktree list
/home/neil/Documents/testing/testrepo        0612677 [master]
/home/neil/Documents/testing/testtree        d38f5a3 [testtree]
Use git worktree remove <name> to drop the entire directory for that tree and the git tracking. I'm using this for work on the Debian Security Tracker. I have two local branches and having two worktrees allows me to have three terminals open, using the same files and the same git repository. One to run make serve and update the local SQLite database. One to access master to run git pull One to make local changes without risking collisions on master.
git add data/CVE/list
git commit
# pre commit hook runs here
git log -n 1
# copy the hash
# switch to master terminal
git pull
git cherry-pick <HASH>
git push
# switch to server terminal
git rebase master
# no git pull or fetch, it's all local
make
# switch back to changes terminal
git rebase master
Sadly, one area where this isn't as easy is with importing a new DSC into Salsa with git-buildpackage as that uses several branches at the same time. It would be possible but you'll need to have a separate upstream and possibly pristine-tar branches and supply the relevant options. Possibly something git-buildpackage to adopt - it is common to need to make changes to the packaging with a new upstream release & a lot of those changes are currently done outside git. For the rest of the support, see git worktree (1)

17 November 2021

Rapha&#235;l Hertzog: Freexian s report about Debian Long Term Support, October 2021

A Debian LTS logo
Every month we review the work funded by Freexian s Debian LTS offering. Please find the report for October below. Debian project funding We re looking forward to receiving more projects from various Debian teams! Learn more about the rationale behind this initiative in this article. Debian LTS contributors In October 12 contributors were paid to work on Debian LTS, their reports are available below. Evolution of the situation In October we released 34 DLAs.

Also, we would like to remark once again that we are constantly looking for new contributors. Please contact Jeremiah if you are interested! The security tracker currently lists 37 packages with a known CVE and the dla-needed.txt file has 22 packages needing an update. Thanks to our sponsors Sponsors that joined recently are in bold.

16 November 2021

Paul Tagliamonte: Measuring the Power Output of my SDRs

Over the last few years, I ve often wondered what the true power output of my SDRs are. It s a question with a shocking amount of complexity in the response, due to a number of factors (mostly Frequency). The ranges given in spec sheets are often extremely vague, and if I m being honest with myself, not incredibly helpful for being able to determine what specific filters and amplifiers I ll need to get a clean signal transmitted.
Hey, heads up! - This post contains extremely unvalidated and back of the napkin quality work to understand how my equipment works. Hopefully this work can be of help to others, but please double check any information you need for your own work!
I was specifically interested in what gain output (in dBm) looks like across the frequency range in particular, how variable the output dBm is when I change frequencies. The second question I had was understanding how linear the output gain is when adjusting the requested gain from the radio. Does a 2 dB increase on a HackRF API mean 2 dB of gain in dBm, no matter what the absolute value of the gain stage is? I ve finally bit the bullet and undertaken work to characterize the hardware I do have, with some outdated laboratory equipment I found on eBay. Of course, if it s worth doing, it s worth overdoing, so I spent a bit of time automating a handful of components in order to collect the data that I need from my SDRs. I bought an HP 437B, which is the cutting edge of 30 years ago, but still accurate to within 0.01dBm. I paired this Power Meter with an Agilent 8481A Power Sensor (-30 dBm to 20 dBm from 10MHz to 18GHz). For some of my radios, I was worried about exceeding the 20 dBm mark, so I used a 20db attenuator while I waited for a higher power power sensor. Finally, I was able to find a GPIB to USB interface, and get that interface working with the GPIB Kernel driver on my system. With all that out of the way, I was able to write Go bindings to my HP 437B to allow for totally headless and automated control in sync with my SDR s RF output. This allowed me to script the transmission of a sine wave at a controlled amplitude across a defined gain range and frequency range and read the Power Sensor s measured dBm output to characterize the Gain across frequency and configured Gain.

HackRF Looking at configured Gain against output power, the requested gain appears to have a fairly linear relation to the output signal power. The measured dBm ranged between the sensor noise floor to approx +13dBm. The average standard deviation of all tested gain values over the frequency range swept was +/-2dBm, with a minimum standard deviation of +/-0.8dBm, and a maximum of +/-3dBm. When looking at output power over the frequency range swept, the HackRF contains a distinctive (and frankly jarring) ripple across the Frequency range, with a clearly visible jump in gain somewhere around 2.1GHz. I have no idea what is causing this massive jump in output gain, nor what is causing these distinctive ripples. I d love to know more if anyone s familiar with HackRF s RF internals!

PlutoSDR The power output is very linear when operating above -20dB TX channel gain, but can get quite erratic the lower the output power is configured. The PlutoSDR s output power is directly related to the configured power level, and is generally predictable once a minimum power level is reached. The measured dBm ranged from the noise floor to 3.39 dBm, with an average standard deviation of +/-1.98 dBm, a minimum standard deviation of +/-0.91 dBm and a maximum standard deviation of +/-3.37 dBm. Generally, the power output is quite stable, and looks to have very even and wideband gain control. There s a few artifacts, which I have not confidently isolated to the SDR TX gain, noise (transmit artifacts such as intermodulation) or to my test setup. They appear fairly narrowband, so I m not overly worried about them yet. If anyone has any ideas what this could be, I d very much appreciate understanding why they exist!

Ettus B210 The power output on the Ettus B210 is higher (in dBm) than any of my other radios, but it has a very odd quirk where the power becomes nonlinear somewhere around -55dB TX channel gain. After that point, adding gain has no effect on the measured signal output in dBm up to 0 dB gain. The measured dBm ranged from the noise floor to 18.31 dBm, with an average standard deviation of +/-2.60 dBm, a minimum of +/-1.39 dBm and a maximum of +/-5.82 dBm. When the Gain is somewhere around the noise floor, the measured gain is incredibly erratic, which throws the maximum standard deviation significantly. I haven t isolated that to my test setup or the radio itself. I m inclined to believe it s my test setup. The radio has a fairly even and wideband gain, and so long as you re operating between -70dB to -55dB, fairly linear as well.

Summary Of all my radios, the Ettus B210 has the highest output (in dBm) over the widest frequency range, but the HackRF is a close second, especially after the gain bump kicks in around 2.1GHz. The Pluto SDR feels the most predictable and consistent, but also a very low output, comparatively - right around 0 dBm.
Name Max dBm stdev dBm stdev min dBm stdev max dBm
HackRF +12.6 +/-2.0 +/-0.8 +/-3.0
PlutoSDR +3.3 +/-2.0 +/-0.9 +/-3.7
B210 +18.3 +/-2.6 +/-1.4 +/-6.0

15 November 2021

Vincent Bernat: Git as a source of truth for network automation

The first step when automating a network is to build the source of truth. A source of truth is a repository of data that provides the intended state: the list of devices, the IP addresses, the network protocols settings, the time servers, etc. A popular choice is NetBox. Its documentation highlights its usage as a source of truth:
NetBox intends to represent the desired state of a network versus its operational state. As such, automated import of live network state is strongly discouraged. All data created in NetBox should first be vetted by a human to ensure its integrity. NetBox can then be used to populate monitoring and provisioning systems with a high degree of confidence.
When introducing Jerikan, a common feedback we got was: you should use NetBox for this. Indeed, Jerikan s source of truth is a bunch of YAML files versioned with Git.

Why Git? If we look at how things are done with servers and services, in a datacenter or in the cloud, we are likely to find users of Terraform, a tool turning declarative configuration files into infrastructure. Declarative configuration management tools like Salt, Puppet,1 or Ansible take care of server configuration. NixOS is an alternative: it combines package management and configuration management with a functional language to build virtual machines and containers. When using a Kubernetes cluster, people use Kustomize or Helm, two other declarative configuration management tools. Tapped together, these tools implement the infrastructure as code paradigm.
Infrastructure as code is an approach to infrastructure automation based on practices from software development. It emphasizes consistent, repeatable routines for provisioning and changing systems and their configuration. You make changes to code, then use automation to test and apply those changes to your systems. Kief Morris, Infrastructure as Code, O Reilly.
A version control system is a central tool for infrastructure as code. The usual candidate is Git with a source code management system like GitLab or GitHub. You get:
Traceability and visibility
Git keeps a log of all changes: what, who, why, and when. With a bit of discipline, each change is explained and self-contained. It becomes part of the infrastructure documentation. When the support team complains about a degraded experience for some customers over the last two months or so, you quickly discover this may be related to a change to an incoming policy in New York.
Rolling back
If a change is defective, it can be reverted quickly, safely, and without much effort, even if other changes happened in the meantime. The policy change at the origin of the problem spanned over three routers. Reverting this specific change and deploying the configuration let you solve the situation until you find a better fix.
Branching, reviewing, merging
When working on a new feature or refactoring some part of the infrastructure, a team member creates a branch and works on their change without interfering with the work of other members. Once the branch is ready, a pull request is created and the change is ready to be reviewed by the other team members before merging. You discover the issue was related to diverting traffic through an IX where one ISP was connected without enough capacity. You propose and discuss a fix that includes a change of the schema and the templates used to declare policies to be able to handle this case.
Continuous integration
For each change, automated tests are triggered. They can detect problems and give more details on the effect of a change. Branches can be deployed to a test infrastructure where regression tests are executed. The results can be synthesized as a comment in the pull request to help the review. You check your proposed change does not modify the other existing policies.

Why not NetBox? NetBox does not share these features. It is a database with a REST and a GraphQL API. Traceability is limited: changes are not grouped into a transaction and they are not documented. You cannot fork the database. Usually, there is one staging database to test modifications before applying them to the production database. It does not scale well and reviews are difficult. Applying the same change to the production database can be hazardous. Rolling back a change is non-trivial.

Update (2021-11) Nautobot, a fork of NetBox, will soon address this point by using Dolt, an SQL database engine allowing you to clone, branch, and merge, like a Git repository. Dolt is compatible with MySQL clients. See Nautobots, Roll Back! for a preview of this feature.

Moreover, NetBox is not usually the single source of truth. It contains your hardware inventory, the IP addresses, and some topology information. However, this is not the place you put authorized SSH keys, syslog servers, or the BGP configuration. If you also use Ansible, this information ends in its inventory. The source of truth is therefore fragmented between several tools with different workflows. Since NetBox 2.7, you can append additional data with configuration contexts. This mitigates this point. The data is arranged hierarchically but the hierarchy cannot be customized.2 Nautobot can manage configuration contexts in a Git repository, while still allowing to use of the API to fetch them. You get some additional perks, thanks to Git, but the remaining data is still in a database with a different lifecycle. Lastly, the schema used by NetBox may not fit your needs and you cannot tweak it. For example, you may have a rule to compute the IPv6 address from the IPv4 address for dual-stack interfaces. Such a relationship cannot be easily expressed and enforced in NetBox. When changing the IPv4 address, you may forget the IPv6 address. The source of truth should only contain the IPv4 address but you also want the IPv6 address in NetBox because this is your IPAM and you need it to update your DNS entries.

Why not Git? There are some limitations when putting your source of truth in Git:
  1. If you want to expose a web interface to allow an external team to request a change, it is more difficult to do it with Git than with a database. Out-of-the-box, NetBox provides a nice web interface and a permission system. You can also write your own web interface and interact with NetBox through its API.
  2. YAML files are more difficult to query in different ways. For example, looking for a free IP address is complex if they are scattered in multiple places.
In my opinion, in most cases, you are better off putting the source of truth in Git instead of NetBox. You get a lot of perks by doing that and you can still use NetBox as a read-only view, usable by other tools. We do that with an Ansible module. In the remaining cases, Git could still fit the bill. Read-only access control can be done through submodules. Pull requests can restrict write access: a bot can check the changes only modify allowed files before auto-merging. This still requires some Git knowledge, but many teams are now comfortable using Git, thanks to its ubiquity.

  1. Wikimedia manages its infrastructure with Puppet. They publish everything on GitHub. Creative Commons uses Salt. They also publish everything on GitHub. Thanks to them for doing that! I wish I could provide more real-life examples.
  2. Being able to customize the hierarchy is key to avoiding repetition in the data. For example, if switches are paired together, some data should be attached to them as a group and not duplicated on each of them. Tags can be used to partially work around this issue but you lose the hierarchical aspect.

Next.

Previous.