Search Results: "will"

1 July 2024

Russell Coker: VoLTE in Australia

Introduction In Australia the 3G mobile frequencies are to be reused so they are in the process of shutting down the 3G service. That means that everyone has to use VoLTE (Voice Over LTE) for phone calls (including emergency calls). The shutdown time varies by telco, Kogan Mobile (one of the better services which has good value for money and generally works well) shut down their 3G service in January. Aldi Mobile (another one of the good services which is slightly more expensive but has included free calls to most first-world countries and uses the largest phone network) will shut theirs down at the end of August. For background there s a Fosdem talk about OpenSIPS with VoLTE and VoNR [1], it s more complex than you want to know. Also VoNR (Voice over New Radio) is the standard for 5G voice and it s different from VoLTE and has a fallback to VoLTE. Another good lecture for background information is the Fosdem talk on VoLTE at the handset end [2]. The PinePhonePro In October 2023 I tried using my PinePhonePro as my main phone but that only lasted a few days due to problems with calls and poor battery life [3]. Since then I went back to the Huawei Mate 10 Pro that I bought refurbished in June 2019 for $389. So that has been my main phone for 5 years now, giving a cost of $1.50 per week. I had tried using a Huawei Nova 7i running Android without Google Play as an experiment but that had failed, I do many things that need Android apps [4]. I followed the PinePhone wiki to get my PinePhonePro working with VoLTE [5]. That worked fine for me, the only difference from the instructions is that I had to use device /dev/ttyUSB3 and that the modem kept resetting itself during the process and when that happened I had to kill minicom and start again. After changing the setting and saving it the PinePhonePro seemed to work well with VoLTE on a Kogan Mobile SIM (so definitely not using 3G). One issue I have found is that Plasma Mobile (my preferred FOSS phone GUI) appears to have a library issue that results in polling every 14ms even when the screen is locked [6]. If you have a few processes doing that (which means the most lightly used Plasma system) it really hurts battery use. The maintainer has quite reasonably deferred action on this bug report given the KDE 6 transition. Later on in the Trixie development cycle I hope to get this issue resolved, I don t expect it to suddenly make battery life good. But it might make battery life acceptable. I am now idly considering carrying around my PinePhonePro in a powered off state for situations where I might need to do high security operations (root logins to servers or online banking) but for which carrying a laptop isn t convenient. It will do well for the turn on, do 30 mins of work that needs security, and then turn off scenario. Huawei Mate 10 Pro and Redmi 9A The Huawei Mate 10 Pro has been my main phone for 5 years and it has worked well, so it would be ideal if it could do VoLTE as the PinePhonePro isn t ready yet. All the web pages I ve seen about the Mate 10 Pro say that it will either allow upgrading to a VoLTE configuration if run with the right SIM or only support it with the right SIM. I did a test with a Chinese SIM which gave an option of turning on VoLTE but didn t allow any firmware updates and the VoLTE option went away when I put an Australian SIM in. Some forum comments had led me to believe that it would either permanently enable VoLTE or allow upgrading the firmware to one that enables VoLTE if I used a Chinese SIM but that is not the case. I didn t expect a high probability of success but I had to give it a go as it s a nice phone. I did some tests on a Redmi 9A (a terrible phone that has really bad latency on the UI in spite of having reasonably good hardware). The one I tested on didn t have VoLTE enabled when I got it, to test that I used the code *#*#4636#*#* in the dialler to get the menu of SIM information and it showed that VoLTE was not provisioned. I then had to update to the latest release of Android for that phone and enter *#*#86583#*#* in the dialler to enable VoLTE, the message displayed after entering that magic number must end in DISABLE . I get the impression that the code in question makes the phone not check certain aspects of whether the carrier is good for VoLTE and just do it. So apparently Kogan Mobile somehow gives the Redmi 9A the impression that VoLTE isn t supported but if the phone just goes ahead and connects it will work. I don t plan to use a Redmi 9A myself as it s too slow, but I added it to my collection to offer to anyone else I know who needs a phone with VoLTE and doesn t use the phone seriously or to someone who needs a known good phone for testing things. Samsung Galaxy Note 9 I got some Samsung Galaxy Note 9 phones to run Droidian as an experiment [7]. But Droidian dropped support for the Note 9 and I couldn t figure out how to enable VoLTE via Droidian, which was very annoying after I had spent $109 on a test phone and $215 on a phone for real use (I have no plans to try Droidian again at this time). I tried installing LineageOS on one Note 9 [8] which was much easier than expected (especially after previously installing Droidian). But VoLTE wasn t an option. According to Reddit LineageOS doesn t support VoLTE on Samsung devices and you can use a magisk module or a VoLTE enabler module but those aren t supported by LineageOS either [9]. I downloaded an original image for the Note 9 from SamsMobile.com [10]. That image booted past the orange stage (where if you have problems then your phone is probably permanently useless) but didn t boot into the OS. A friend helped me out with that and it turned out that the Heimdal flash tool on Linux didn t do something it needed to do and that Odin on Windows was required. After using Odin everything was fine and I have a Note 9 with VoLTE running the latest Samsung firmware which is security patch level 1st July 2022!!! So I have a choice between using a Note 9 for data and SMS while running a current version of Lineage OS with all security fixes or running a Samsung image with no security updates for 2 years which supports phone calls. So based on this I have to recommend Pixel as the phone of choice, it has a decent level of support from Google and long term support from LineageOS. According to the LineageOS web site you can run the current version of Lineage on the original Pixel phone from 2016! Of course getting VoLTE to work on it might be another saga, but it would probably be easier to do with LineageOS on a Pixel than on a Samsung phone. Conclusion The operation of the Note 9 for me is decent now apart from the potential security issues. The same goes for selling one of the phones. The PinePhonePro still has potential to become my daily driver at some future time if I and others can optimise power use. Also a complicating factor is that I want to have both Jabber and Matrix be actually instant IM systems not IM with a 5 minute delay, so suspend mode isn t a good option. Pixel phones will be a much higher priority when looking at phones to buy in future. The older Pixel phones go for as little as $100 on eBay and can still run the latest LineageOS. VoLTE seems needlessly complicated.

Niels Thykier: Debian packaging with style black

When I started working on the language server for debputy, one of several reasons was about automatic applying a formatting style. Such that you would not have to remember to manually reformat the file. One of the problems with supporting automatic formatting is that no one agrees on the "one true style". To make this concrete, Johannes Schauer Marin Rodrigues did the numbers of which wrap-and-sort option that are most common in https://bugs.debian.org/895570#46. Unsurprising, we end up with 14-15 different styles with various degrees of popularity. To make matters worse, wrap-and-sort does not provide a way to declare "this package uses options -sat". So that begged the question, how would debputy know which style it should use when it was going to reformat file. After a couple of false-starts, Christian Hofstaedtler mentioned that we could just have a field in debian/control for supporting a "per-package" setting in responds to my concern about adding a new "per-package" config file. At first, I was not happy with it, because how would you specify all of these options in a field (in a decent manner)? But then I realized that one I do not want all these styles and that I could start simpler. The Python code formatter black is quite successful despite not having a lot of personalized style options. In fact, black makes a statement out of not allowing a lot of different styles. Combing that, the result was X-Style: black (to be added to the Source stanza of debian/control), which every possible reference to the black tool for how styling would work. Namely, you outsource the style management to the tool (debputy) and then start using your focus on something else than discussing styles. As with black, this packaging formatting style is going to be opinionated and it will evolve over time. At the starting point, it is similar to wrap-and-sort -sat for the deb822 files (debputy does not reformat other files at the moment). But as mentioned, it will likely evolve and possible diverge from wrap-and-sort over time. The choice of the starting point was based on the numbers posted by Johannes #895570. It was not my personal favorite but it seemed to have a majority and is also close to the one suggested by salsa pipeline maintainers. The delta being -kb which I had originally but removed in 0.1.34 at request of Otto Kek l inen after reviewing the numbers from Johannes one more time. To facilitate this new change, I uploaded debputy/0.1.30 (a while back) to Debian unstable with the following changes:

Support for the X-Style: black header.

When a style is defined, the debputy lsp server command will now automatically reformat deb822 files on save (if the editor supports it) or on explicit "reformat file" request from the editor (usually indirectly from the user).

New subcommand debputy reformat command that will reformat the files, when a style is defined.

A new pre-commit hook repo to run debputy lint and debputy reformat. These hooks are available from https://salsa.debian.org/debian/debputy-pre-commit-hooks version v0.1 and can be used with the pre-commit tool (from the package of same name).

The obvious omission is a salsa-pipeline feature for this. Otto has put that on to his personal todo list and I am looking forward to that.

Beyond black Another thing I dislike about our existing style tooling is that if you run wrap-and-sort without any arguments, you have a higher probability of "trashing" the style of the current package than getting the desired result. Part of this is because wrap-and-sort's defaults are out of sync with the usage (which is basically what https://bugs.debian.org/895570 is about). But I see another problem. The wrap-and-sort tool explicitly defined options to tweak the style but provided maintainers no way to record their preference in any machine readable way. The net result is that we have tons of diverging styles and that you (as a user of wrap-and-sort) have to manually tell wrap-and-sort which style you want every time you run the tool. In my opinion that is not playing to the strengths of neither human nor machine. Rather, it is playing to the weaknesses of the human if anything at all. But the salsa-CI pipeline people also ran into this issue and decided to work around this deficiency. To use wrap-and-sort in the salsa-CI pipeline, you have to set a variable to activate the job and another variable with the actual options you want. The salsa-CI pipeline is quite machine readable and wrap-and-sort is widely used. I had debputy reformat also check for the salsa-CI variables as a fallback. This fallback also works for the editor mode (debputy lsp server), so you might not even have to run debputy reformat. :) This was a deliberate trade-off. While I do not want all us to have all these options, I also want Debian packaging to be less painful and have fewer paper cuts. Having debputy go extra lengths to meet wrap-and-sort users where they are came out as the better solution for me. A nice side-effect of this trade-off is that debputy reformat now a good tool for drive-by contributors. You can safely run debputy reformat on any package and either it will apply the styling or it will back out and inform you that no obvious style was detected. In the latter case, you would have to fallback to manually deducing the style and applying it.

Differences to wrap-and-sort The debputy reformat has some limitations or known differences to wrap-and-sort. Notably, debputy reformat (nor debputy lsp server) will not invoke wrap-and-sort. Instead, debputy has its own reformatting engine that provides similar features. One reason for not running wrap-and-sort is that I want debputy reformat to match the style that debputy lsp server will give you. That way, you get consistent style across all debputy commands. Another reason is that it is important to me that reformatting is safe and does not change semantics. This leads to two regrettable known differences to the wrap-and-sort behavior due to safety in addition to one scope limitation in debputy:

debputy will ignore requests to sort the stanzas when the "keep first" option is disabled (-b --no-keep-first). This combination is unsafe reformatting. I feel it was a mistake for wrap-and-sort to ever allow this but at least it is no longer the default (-b is now -bk by default). This will be less of a problem in debhelper-compat 15, since the concept of "main package" will disappear and all multi-binary source packages will be required to use debian/package.install rather than debian/install.

debputy will not reorder the contents of debhelper packaging files such as debian/install. This is also an (theoretical) unsafe thing to do. While the average package will not experience issues with this, there are rare corner cases where the re-ordering can affect the end result. I happen to know this, because I ran into issues when trying to optimize dh_install in a way that assumed the order did not matter. Stuff broke and there is now special-case code in dh_install to back out of that optimization when that happens.

debputy has a limited list of wrap-and-sort options it understands. Some options may cause debputy to back out and disable reformatting entirely with a remark that it cannot apply that style. If you run into a case of this, feel free to file a feature request to support it. I will not promise to support everything, but if it is safe and trivially doable with the engine already, then I probably will.

As stated, where debputy cannot implement the wrap-and-sort styles fully, then it will currently implement a subset that is safe if that can be identified or back out entirely of the formatting when it cannot. In all cases, debputy will not break the formatting if it is correct. It may just fail at correcting one aspect of the wrap-and-sort style if you happen to get it wrong. It is also important to remember that the prerequisite for debputy applying any wrap-and-sort style is that you have set the salsa-CI pipeline variables to trigger wrap-and-sort with the salsa-CI pipeline. So there is still a CI check before the merge that will run the wrap-and-sort in its full glory that provides the final safety net for you.

Just give me a style In conclusion, if you, like me, are more interested in getting a consistent style rather than discussing what that style should be, now you can get that with X-Style: black. You can also have your custom wrap-and-sort style be picked up automatically for drive-by contributors.

$ apt satisfy 'dh-debputy (>= 0.1.30), python3-lsprotocol'
# Add  X-Style: black  to  debian/control  for "just give me a style"
#
# OR, if there is a specific  wrap-and-sort  style for you then set
# SALSA_CI_DISABLE_WRAP_AND_SORT=no plus set relevant options in
# SALSA_CI_WRAP_AND_SORT_ARGS in debian/salsa-ci.yml (or .gitlab-ci.yml)
$ debputy reformat

It is sadly not yet in the salsa-ci pipeline. Otto is looking into that and hopefully we will have it soon. :) And if you find yourself often doing archive-wide contributions and is tired of having to reverse engineer package formatting styles, consider using debputy reformat or debputy lsp server. If you use debputy in this way, please consider providing feedback on what would help you.

Russ Allbery: Review: Snuff

Review: Snuff, by Terry Pratchett

Series:	Discworld #39
Publisher:	Harper
Copyright:	October 2011
Printing:	January 2013
ISBN:	0-06-221886-7
Format:	Mass market
Pages:	470

Snuff is the 39th Discworld novel and the 8th (and last) Watch novel. This is not a good place to start reading. Sam Vines has been talked, cajoled, and coerced into taking a vacation. Since he is now the Duke of Ankh, he has a country estate that he's never visited. Lady Sybil is insistent on remedying this, as is Vetinari. Both of them may have ulterior motives. They may also be colluding. It does not take long for Vimes to realize that something is amiss in the countryside. It's not that the servants are uncomfortable with him talking to them, the senior servants are annoyed that he talks to the wrong servants, and the maids turn to face the wall at the sight of him. Those are just the strange customs of the aristocracy, for which he has little understanding and even less patience. There's something else going on. The nobility is wary, the town blacksmith is angry about something more than disliking the nobles, and the bartender doesn't want to get involved. Vimes smells something suspicious. When he's framed for a murder, the suspicions seem justified. It takes some time before the reader learns what the local nobility are squirming about, so I won't spoil it. What I will say is that Snuff is Pratchett hammering away at one of his favorite targets: prejudice, cruelty, and treating people like things. Vimes, with his uncompromising morality, is one of the first to realize the depth of the problem. It takes most of the rest longer to come around, even Sybil. It's both painful, and painfully accurate, to contemplate how often recognition of other people's worth only comes once they do something that you recognize as valuable. This is one of the better-plotted Discworld novels. Vimes starts out with nothing but suspicions and stubbornness, and manages to turn Snuff into a mystery novel through dogged persistence. The story is one continuous plot arc with the normal Pratchett color (Young Sam's obsession with types of poo, for example) but without extended digressions. It also has considerably better villains than most Pratchett novels: layers of foot soldiers and plotters, each of which have to be dealt with in a suitable way. Even the concluding action sequences worked for me, which is not always a given in Discworld. The problem, unfortunately, is that the writing is getting a bit wobbly. Pratchett died of early-onset Alzheimer's in 2015, four years after this book was first published, and this is the first novel where I can see some early effects. It mostly shows up in the dialogue: it's just a bit flabby and a bit repetitive, and the characters, particularly towards the end of the book, start repeating the name of the person they're talking to every other line. Once I saw it, I couldn't unsee it, and it was annoying enough to rob a bit of enjoyment from the end of the book. That aside, though, this was a solid Discworld novel. Vimes testing his moral certainty against the world and forcing it into a more ethical shape is always rewarding, and here he takes more risks, with better justification, than in most of the Watch novels. We also find out that Vimes has a legacy from the events of Thud!, which has interesting implications that I wish Pratchett had more time to explore. I think the best part of this book is how it models the process of social change through archetypes: the campaigner who knew the right choice early on, the person who formed their opinion the first time they saw injustice, the person who gets there through a more explicit moral code, the ones who have to be pushed by someone who was a bit faster, the ones who have to be convinced but then work to convince others, and of course the person who is willing to take on the unfair and far-too-heavy burden of being exceptional enough that they can be used as a tool to force other people to acknowledge them as a person. And, since this is Discworld, Vetinari is lurking in the scenery pulling strings, balancing threats, navigating politics, and giving Vimes just enough leeway to try to change the world without abusing his power. I love that the amount of leeway Vimes gets depends on how egregious the offense is, and Vetinari calibrates this quite carefully without ever saying so openly. Recommended, and as much as I don't want to see this series end, this is not a bad entry for the Watch novels to end on. Followed in publication order by Raising Steam. Rating: 8 out of 10

30 June 2024

Joachim Breitner: Do surprises get larger?

The setup Imagine you are living on a riverbank. Every now and then, the river swells and you have high water. The first few times this may come as a surprise, but soon you learn that such floods are a recurring occurrence at that river, and you make suitable preparation. Let s say you feel well-prepared against any flood that is no higher than the highest one observed so far. The more floods you have seen, the higher that mark is, and the better prepared you are. But of course, eventually a higher flood will occur that surprises you. Of course such new record floods are happening rarer and rarer as you have seen more of them. I was wondering though: By how much do the new records exceed the previous high mark? Does this excess decrease or increase over time? A priori both could be. When the high mark is already rather high, maybe new record floods will just barley pass that mark? Or maybe, simply because new records are so rare events, when they do occur, they can be surprisingly bad? This post is a leisurely mathematical investigating of this question, which of course isn t restricted to high waters; it could be anything that produces a measurement repeatedly and (mostly) independently weather events, sport results, dice rolls. The answer of course depends on the distribution of results: How likely is each possible results.

Dice are simple With dice rolls the answer is rather simple. Let our measurement be how often you can roll a die until it shows a 6. This simple game we can repeat many times, and keep track of our record. Let s say the record happens to be 7 rolls. If in the next run we roll the die 7 times, and it still does not show a 6, then we know that we have broken the record, and every further roll increases by how much we beat the old record. But note that how often we will now roll the die is completely independent of what happened before! So for this game the answer is: The excess with which the record is broken is always the same. Mathematically speaking this is because the distribution of rolls until the die shows a 6 is memoryless. Such distributions are rather special, its essentially just the example we gave (a geometric distribution), or its continuous analogue (the exponential distributions, for example the time until a radioactive particle decays).

Mathematical formulation With this out of the way, let us look at some other distributions, and for that, introduce some mathematical notations. Let X be a random variable with probability density function (x) and cumulative distribution function (x), and a be the previous record. We are interested in the behavior of Y(a) = X a X > x i.e. by how much X exceeds a under the condition that it did exceed a. How does Y change as a increases? In particular, how does the expected value of the excess e(a) = E(Y(a)) change?

Uniform distribution If X is uniformly distributed between, say, 0 and 1, then a new record will appear uniformly distributed between a and 1, and as that range gets smaller, the excess must get smaller as well. More precisely, e(a) = E(X a X > a) = E(X X > a) a = (1 a)/2 This not very interesting linear line is plotted in blue in this diagram:
The expected record surpass for the uniform distribution
The orange line with the logarithmic scale on the right tries to convey how unlikely it is to surpass the record value a: it shows how many attempts we expect before the record is broken. This can be calculated by n(a) = 1/(1 (a)).

Normal distribution For the normal distribution (with median 0 and standard derivation 1, to keep things simple), we can look up the expected value of the one-sided truncated normal distribution and obtain e(a) = E(X X > a) a = (a)/(1 (a)) a Now is this growing or shrinking? We can plot this an have a quick look:
The expected record surpass for the normal distribution
Indeed it is, too, a decreasing function! (As a sanity check we can see that e(0) = (2/ ), which is the expected value of the half-normal distribution, as it should.)

Could it be any different? This settles my question: It seems that each new surprisingly high water will tend to be less surprising than the previously assuming high waters were uniformly or normally distributed, which is unlikely to be helpful. This does raise the question, though, if there are probability distributions for which e(a) is be increasing? I can try to construct one, and because it s a bit easier, I ll consider a discrete distribution on the positive natural numbers, and consider at g(0) = E(X) and g(1) = E(X 1 X > 1). What does it take for g(1) > g(0)? Using E(X) = p + (1 p)E(X X > 1) for p = P(X = 1) we find that in order to have g(1) > g(0), we need E(X) > 1/p. This is plausible because we get equality when E(X) = 1/p, as it precisely the case for the geometric distribution. And it is also plausible that it helps if p is large (so that the next first record is likely just 1) and if, nevertheless, E(X) is large (so that if we do get an outcome other than 1, it s much larger). Starting with the geometric distribution, where P(X > n X n) = p_n = p (the probability of again not rolling a six) is constant, it seems that these p_n is increasing, we get the desired behavior. So let p₁ < p₂ < p_n < be an increasing sequence of probabilities, and define X so that P(X = n) = p₁ p_{n 1} (1 p_n) (imagine the die wears off and the more often you roll it, the less likely it shows a 6). Then for this variation of the game, every new record tends to exceed the previous more than previous records. As the p increase, we get a flatter long end in the probability distribution.

Gamma distribution To get a nice plot, I ll take the intuition from this and turn to continuous distributions. The Wikipedia page for the exponential distribution says it is a special case of the gamma distribution, which has an additional shape parameter , and it seems that it could influence the shape of the distribution to be and make the probability distribution have a longer end. Let s play around with = 2 and = 0.5, 1 and 1.5:
The expected record surpass for the gamma distribution

For = 1 (dotted) this should just be the exponential distribution, and we see that e(a) is flat, as predicted earlier.

For larger (dashed) the graph does not look much different from the one for the normal distribution not a surprise, as for , the gamma distribution turns into the normal distribution.

For smaller (solid) we get the desired effect: e(a) is increasing. This means that new records tend to break records more impressively.

The orange line shows that this comes at a cost: for a given old record a, new records are harder to come by with smaller .

Conclusion As usual, it all depends on the distribution. Otherwise, not much, it s late.

29 June 2024

Russ Allbery: Review: Bitter Angels

Review: Bitter Angels, by C.L. Anderson

Publisher:	Ballantine Books
Copyright:	2009
ISBN:	0-553-59217-3
Format:	Mass market
Pages:	438

Bitter Angels is a stand-alone science fiction novel. It won the Philip K. Dick award for best SF original paperback in 2010. C.L. Anderson is a pen name for Sarah Zettel. Terese was a Guardian, one of the agents of the Pax Solaris who find ways to keep the peace in troubled systems and high-stress situations with the help of an implanted Companion, an assistant AI. Forty years ago, on one of those missions, she was captured and her Companion was forcibly removed. She was rescued by her friend and mentor and retired afterwards, starting a new life and a new family, trying to leave the memories behind. Now, the woman who rescued her is dead. She was murdered on duty in the Erasmus system, a corporate hellhole that appears to be on the verge of exploding into a political hot spot. Bianca's last instructions asked for Terese to replace her. Terese's family is furious at her for even considering returning to the Guardians, but she can't say no. Duty, and Bianca's dying request, call too strongly. Amerand is Security on Dazzle, one of the Erasmus stations. He is one of the refugees from Oblivion, the station that the First Bloods who rule the system let die. He keeps his head above water and tries to protect his father and find his mother without doing anything that the ever-present Clerks might find concerning. Keeping an eye on newly-arriving Solaris saints is a typical assignment, since the First Bloods don't trust the meddling do-gooders. But something is not quite right, and a cryptic warning from his Clerk makes him even more suspicious. This is the second book by Sarah Zettel that I've read, and both of them have been tense, claustrophobic thrillers set in a world with harsh social inequality and little space for the characters to maneuver. In this case, the structure of her future universe reminded me a bit of Iain M. Banks's Culture, but with less advanced technology and only humans. The Pax Solaris has eliminated war within its borders and greatly extended lifespans. That peace is maintained by Guardians, who play a role similar to Special Circumstances but a bit more idealist and less lethal. They show up where there are problems and meddle, manipulating and pushing to try to defuse the problems before they reach the Pax Solaris. Like a Culture novel, nearly all of the action takes place outside the Pax Solaris in the Erasmus system. Erasmus is a corporate colony that has turned into a cross between a hereditary dictatorship and the Corporate Rim from Martha Wells's Murderbot series. Debt slavery is ubiquitous, economic inequality is inconceivably vast, and the Clerks are everywhere. Erasmus natives like Amerand have very little leeway and even fewer options. Survival is a matter of not drawing the attention of the wrong people. Terese and her fellow Guardians are appalled, but also keenly aware that destabilizing the local politics may make the situation even worse and get a lot of people killed. Bitter Angels is structured like a mystery: who killed Bianca, and what was her plan when she was killed? Unlike a lot of books of this type, the villains are not idiots and their plan is both satisfyingly complex and still depressingly relevant. I don't think I'm giving anything away by saying that I have read recent news articles about people with very similar plans, albeit involving less science-fiction technology. Anderson starts with a tense situation and increases the pressure relentlessly, leaving the heroes one step behind the villains for almost the entire novel. It is not happy or optimistic reading at times, the book is quite dark but it certainly was engrossing. The one world-building quibble that I had is that the Erasmus system is portrayed partly as a hydraulic empire, and while this is arguably feasible given that spaceship travel is strictly controlled, it seemed like a weird choice given the prevalence of water on the nearby moons. Water smuggling plays a significant role in the plot, and I wasn't entirely convinced of the politics and logistics behind it. If this sort of thing bugs you, there are some pieces that may require suspension of disbelief. Bitter Angels is the sort of tense thriller where catastrophe is barely avoided and the cost of victory is too high, so you will want to be in the mood for that before you dive in. But if that's what you're looking for, I thought Anderson delivered a complex and satisfying story. Content warning: major character suicide. Rating: 7 out of 10

28 June 2024

Matthew Palmer: Checking for Compromised Private Keys has Never Been Easier

As regular readers would know, since I never stop banging on about it, I run Pwnedkeys, a service which finds and collates private keys which have been disclosed or are otherwise compromised. Until now, the only way to check if a key is compromised has been to use the Pwnedkeys API, which is not necessarily trivial for everyone. Starting today, that s changing. The next phase of Pwnedkeys is to start offering more user-friendly tools for checking whether keys being used are compromised. These will typically be web-based or command-line tools intended to answer the question is the key in this (certificate, CSR, authorized_keys file, TLS connection, email, etc) known to Pwnedkeys to have been compromised? .

Opening the Toolbox Available right now are the first web-based key checking tools in this arsenal. These tools allow you to:

Check the key in a PEM-format X509 data structure (such as a CSR or certificate);

Check the keys in an `authorized_keys` file you upload; and

Check the SSH keys used by a user at any one of a number of widely-used code-hosting sites.

Further planned tools include live checking of the certificates presented in TLS connections (for HTTPS, etc), SSH host keys, command-line utilities for checking local `authorized_keys` files, and many other goodies.

If You Are Intrigued By My Ideas and wish to subscribe to my newsletter, now you can! I m not going to be blogging every little update to Pwnedkeys, because that would probably get a bit tedious for readers who aren t as intrigued by compromised keys as I am. Instead, I ll be posting every little update in the Pwnedkeys newsletter. So, if you want to keep up-to-date with the latest and greatest news and information, subscribe to the newsletter.

Supporting Pwnedkeys All this work I m doing on my own time, and I m paying for the infrastructure from my own pocket. If you ve got a few dollars to spare, I d really appreciate it if you bought me a refreshing beverage. It helps keep the lights on here at Pwnedkeys Global HQ.

27 June 2024

Jonathan McDowell: Sorting out backup internet #4: IPv6

The final piece of my 5G (well, 4G) based based backup internet connection I needed to sort out was IPv6. While Three do support IPv6 in their network they only seem to enable it for certain devices, and the MC7010 is not one of those devices, even though it also supports IPv6. I use v6 a lot - over 50% of my external traffic, last time I looked. One suggested option was that I could drop the IPv6 Router Advertisements when the main external link went down, but I have a number of internal services that are only presented on v6 addresses so I needed to ensure clients in the house continued to have access to those. As it happens I ve used the Hurricane Electric IPv6 Tunnel Broker in the past, so my pass was re-instating that. The 5G link has a real external IPv4 address, and it s possible to update the endpoint using a simple HTTP GET. I added the following to my /etc/dhcp/dhclient-exit-hooks.d/modem-interface-route where we are dealing with an interface IP change:

# Update IPv6 tunnel with Hurricane Electric
curl --interface $interface 'https://username:password@ipv4.tunnelbroker.net/nic/update?hostname=1234'

I needed some additional configuration to bring things up, so /etc/network/interfaces got the following, configuring the 6in4 tunnel as well as the low preference default route, and source routing via the 5g table, similar to IPv4:

pre-up ip tunnel add he-ipv6 mode sit remote 216.66.80.26
pre-up ip link set he-ipv6 up
pre-up ip addr add 2001:db8:1234::2/64 dev he-ipv6
pre-up ip -6 rule add from 2001:db8:1234::/64 lookup 5g
pre-up ip -6 route add default dev he-ipv6 table 5g
pre-up ip -6 route add default dev he-ipv6 metric 1000
post-down ip tunnel del he-ipv6

We need to deal with IPv4 changes in for the tunnel endpoint, so modem-interface-route also got:

ip tunnel change he-ipv6 local $new_ip_address

/etc/nftables.conf had to be taught to accept the 6in4 packets from the tunnel in the input chain:

# Allow HE tunnel
iifname "sfp.31" ip protocol 41 ip saddr 216.66.80.26 accept

Finally, I had to engage in something I never thought I d deal with; IPv6 NAT. HE provide a /48, and my FTTP ISP provides me with a /56, so this meant I could do a nice stateless 1:1 mapping:

table ip6 nat  
  chain postrouting  
    type nat hook postrouting priority 0
    oifname "he-ipv6" snat ip6 prefix to ip6 saddr map   2001:db8:f00d::/56 : 2001:db8:666::/56  
   
 

This works. Mostly. The problem is that HE, not unreasonably, expect your IPv4 address to be pingable. And it turns out Three have some ranges that this works on, and some that it doesn t. Which means it s a bit hit and miss whether you can setup the tunnel. I spent a while trying to find an alternative free IPv6 tunnel provider with a UK endpoint. There s less call for them these days, so I didn t manage to find any that actually worked (or didn t have a similar pingable requirement). I did consider whether I wanted to end up with routes via a VM, as I described in the failover post, but looking at costings for VMs with providers who could actually give me an IPv6 range I decided the cost didn t make it worthwhile; the VM cost ended up being more than the backup SIM is costing monthly. Finally, it turns out happy eyeballs mostly means that when the 5G ends up on an IP that we can t setup the IPv6 tunnel on, things still mostly work. Browser usage fails over quickly and it s mostly my own SSH use that needs me to force IPv4. Purists will groan, but this turns out to be an acceptable trade-off for me, at present. Perhaps if I was seeing frequent failures the diverse routes approach to a VM would start to make sense, but for now I m pretty happy with the configuration in terms of having a mostly automatic backup link take over when the main link goes down.

Russ Allbery: Review: Lyorn

Review: Lyorn, by Steven Brust

Series:	Vlad Taltos #17
Publisher:	Tor
Copyright:	2024
ISBN:	1-4668-8971-3
Format:	Kindle
Pages:	274

Lyorn is the 17th Vlad Taltos book and a direct sequel to 2014's Hawk. (Yes, actual main story progress!) You do not want to start reading here; you would be hopelessly confused. When this series is complete, I want to re-read the entire thing from the beginning and pick up more of the bits I missed the first time. Vlad is not, in fact, free to see his friends and get entangled in imperial politics again as I thought after Hawk. Despite the successes of that story, there is one remaining small problem: incredibly powerful magic users still want to kill him. His immediate solution is to shelter in a theater, since Draegaran theaters are well-known for their excellent magical shielding. This works well enough at first, but the theater is rehearsing a play about Draegaran politics that is highly offensive to the Lyorn and the theater may be shut down because of it. Vlad's enemies are also willing to lean on his friends to find him and kill him. This series continues to be thoroughly enjoyable. Lyorn is "just" more of Vlad being Vlad, meddling in everyone's business and coming up with elaborate plans with too many moving parts that he somehow manages to pull off, but I'll happily read lots of books like that. Vlad is both anxious and grumpy, both of which give the plot some needed tension without being overwhelming. There are no truly major world-building revelations here (or, if there are, I missed them), but there's a lot of processing of what the reader learned in Tsalmoth. It's increasingly looking like the payoff from those revelations is going to be the series finale. This is the first Vlad book that contains solid confirmation of where the series as a whole is headed. Brust mentioned some time ago that the last book is titled The Last Contract, and Lyorn comes close to stating explicitly what that contract will be. I am sure that it will be more complicated than it appears now and there are misdirections yet to come, but I am excited to see where Brust takes this idea. Vlad has been insistently apolitical for much of the series, meddling in politics only when he has to or to help his friends and otherwise treating it as a system that he has to navigate and survive. That was the root of the conflict in Teckla, all the way back at the start of the series. This may be starting to change, and when Brust ties it together with the Jenoine, the Great Weapon Godslayer, and the rest of the world-building he's cued up, the results are going to be explosive. Two books left: Chreotha and The Last Contract. I can hardly wait. In every Vlad book, Brust plays some sort of structural game. This time, befitting the setting, it's a musical. The action is interspersed with quotes from a fictional history about the play the theater is putting on, a work called Song of the Presses about political censorship during the reign of a Lyorn emperor in the 14th cycle, thousands of years before the time of this book. This was, at times, nearly as interesting as the main plot. The chapters are also numbered like the acts and scenes of a play, although this I didn't notice as much since books often have that structure anyway. Since this is a musical, there are also songs. Specifically, each chapter is introduced by a parody of songs from various musicals in our world, rewritten so that they fit within Brust's fictional musical. Brust is also a musician and a filker, so these songs are actually good, or at least they amused me a great deal. I'm not much of a musical fan and I still could hear the tune playing when I read most of them. Lyorn is not so good that I would rave about it. It's one of those functional connective books of a series that advances the plot, tells a good story, and has some fun along the way. The guns on the mantelpiece of this world have not gone off yet, and Vlad is still maneuvering into position. But it's looking like we're going to get the conclusion, and it's going to be spectacular. If you have read this far, you will want to keep reading. Followed by Chreotha, which may be a bit of a wait because apparently Brust is going to write The Last Contract first to make sure he ties up loose ends properly. Rating: 7 out of 10

26 June 2024

Gunnar Wolf: Many terabytes for students to play with. Thanks Debian!

LIDSOL students receiving their new hard drives

My students at LIDSOL (Laboratorio de Investigaci n y Desarrollo de Software Libre, Free Software Research and Development Lab) at Facultad de Ingenier a, UNAM asked me to help them get the needed hardware to set up a mirror for various free software projects. We have some decent servers (not new servers, but mirrors don t require to be top-performance), so A couple of weeks ago, I approached the Debian Project Leader (DPL) and suggested we should buy two 16TBhard drives for this project, as it is the most reasonable cost per byte point I found. He agreed, and I bought the drives. Today we had a lab meeting, and I handed them over the hardware. Of course, they are very happy and thankful with the Debian project In the last couple of weeks, they have already set up an Archlinux mirror (https://archlinux.org/mirrors/fi-b.unam.mx), and now that they have heaps of storage space, plans are underway to set up various other mirrors (of course, a Debian mirror will be among the first).

25 June 2024

Gunnar Wolf: Find my device - Whether you like it or not

I received a mail today from Google (noreply-findmydevice@google.com) notifying me that they would unconditionally enable the Find my device functionality I have been repeatedly marking as unwanted in my Android phone. The mail goes on to explain this functionality works even when the device is disconnected, by Bluetooth signals (aha, so turn off Bluetooth will no longer turn off Bluetooth? Hmmm ) Of course, the mail hand-waves that only I can know the location of my device. Google cannot see or use it for other ends . First, should we trust this blanket statement? Second, the fact they don t do it now means they won t ever? Not even if law enforcement requires them to? The devices will be generating this information whether we want it or not, so it s just a matter of opening the required window. Targetting an individual in a crowd

Of course, it is a feature many people will appreciate and find useful. And it s not only for finding lost (or stolen) phones, but the mail also mentions tags can be made available to store in your wallet, bike, keys or whatever. But it should be opt-in. As it is, it seems it s not even to opt out of it.

Thomas Lange: FAI 6.2.3 released, FAIme adds Trixie support

A new FAI version was released and the FAIme service is using this new release. You can now also create installation images for Debian 13 (testing aka Trixie). https://fai-project.org/FAIme/ Another new feature of the FAIme service will be announced at DebConf24 in August.

22 June 2024

Russ Allbery: Review: And the Stars Will Sing

Review: And the Stars Will Sing, by Michelle Browne

Series:	The Meaning Wars #1
Publisher:	Michelle Browne
Copyright:	2012, 2021
Printing:	2021
ASIN:	B0075G7GEA
Format:	Kindle
Pages:	85

And the Stars Will Sing is a self-published science fiction novella, the first of a (currently) five book series. I believe it may be Browne's first publication, although I don't have a good data source to confirm. Crystal Weiss is a new graduate from Mars, about to leave the solar system to her first job assignment: installation of a permanent wormhole in the vicinity of Messier 14. Her expertise is the placement calculations. The heavy mathematical lifting is of course done by computers, but humans have to do the mapping and some of the guidance. And the Stars Will Sing is an epistolary novel, told in the form of her letters to her friend Sarah. I feel bad when I stumble across a book like this. I want to stick with my habit of writing a review of each book I read, but it's one thing to pan a bad book by a famous author and another thing to pick on a self-published novella that I read due to some recommendation or mention whose details I've forgotten. Worse, I think this wasn't even the recommended book; I looked up the author, saw that the first of a series was on sale, and thought "oh, hey, I like epistolary novels and I'm in the mood for some queer space opera." This book didn't seem that queer (there is a secondary lesbian relationship but the main relationship seemed rather conventional), but I'll get to the romance in a moment. I was not the reader for this book. There's a reason why most of the books I read are from traditional publishers; I'm too critical of a reader for a lot of early self-published work. It's not that I dislike self-publishing as a concept many self-published books are excellent and the large publishers have numerous problems but publishers enforce a quality bar. Inconsistently, unfairly, and by rejecting a lot of good work, but still, they do. I'm fairly sure traditional publishers would have passed on this book; the quality of the writing isn't there yet. (It's certainly a better book than I could have written! But that's why I'm writing my reviews over in my quiet corner of the Internet and not selling fiction to other people.) The early chapters aren't too bad, although they have a choppy, cliched style that more writing experience usually smoothes out. The later chapters have more dialogue, enough that I started wondering how Crystal could remember that much dialogue verbatim to put into a letter, and it's not good. All of the characters talk roughly the same (even the aliens), the dialogue felt even more cliched than the rest of the writing, and I started getting distracted by the speech tags. Crystal comes across as very young, impulsive, and a drama magnet who likes being all up in her coworkers' business. None of these are objective flaws in the book, but I could tell early on that I was going to find her annoying. She has a heavily-foreshadowed enemies-to-lovers thing with one of her male coworkers. Her constant complaining about him at the start of the story was bad enough, but the real problem is that in the very few places where he has more personality than plastic lawn furniture, he's being obnoxious to Crystal. I'm used to being puzzled by a protagonist's choice in love interests, but this one felt less like an odd personality choice and more a lack of writing skill. Even if the relationship is being set up for failure (not true by the end of this book), you've got to help me understand what the protagonist saw in him or was getting out of the relationship. The plot was so predictable that it ironically surprised me. I was sure that some sort of twist or complication was coming, but no. I will give Browne some credit for writing a slightly more realistic character reaction to violence than most SF authors, but there was nothing in the plot to hold my interest. The world-building was generic science fiction with aliens. It had a few glimmers of promise, but there was some sort of psychic hand-waving involved in siting wormholes that didn't work for me and the plot climax made no sense to me whatsoever. This is the kind of bad book that I don't want to hold against the writer. Twelve years later and with numerous other novels and novellas under her belt, her writing is probably much better. I do think this book would have benefited from an editor telling her it wasn't good enough for publication yet, but that's not how the Kindle self-publishing world works. Mostly, this is my fault: I half-followed a recommendation into an area of publishing that I know from past experience I should avoid without a solid review from an equally critical reader. Followed by The Stolen, a two-story collection. Rating: 3 out of 10

21 June 2024

Bits from Debian: Looking for the artwork for Trixie the next Debian release

Each release of Debian has a shiny new theme, which is visible on the boot screen, the login screen and, most prominently, on the desktop wallpaper. Debian plans to release Trixie, the next release, next year. As ever, we need your help in creating its theme! You have the opportunity to design a theme that will inspire thousands of people while working in their Debian systems. For the most up to date details, please refer to the wiki. We would also like to take this opportunity to thank Juliette Taka Belin for doing the Emerald theme for bookworm. The deadlines for submissions is: 2024-09-19 The artwork is usually picked based on which themes look the most:

''Debian'': admittedly not the most defined concept, since everyone has their own take on what Debian means to them.
''plausible to integrate without patching core software'': as much as we love some of the insanely hot looking themes, some would require heavy GTK+ theming and patching GDM/GNOME.
''clean / well designed'': without becoming something that gets annoying to look at a year down the road. Examples of good themes include Joy, Lines, Softwaves and futurePrototype.

If you'd like more information or details, please post to the Debian Desktop mailing list.

Bits from Debian: Looking for the artwork for Trixie the next Debian release

''Debian'': admittedly not the most defined concept, since everyone has their own take on what Debian means to them.
''plausible to integrate without patching core software'': as much as we love some of the insanely hot looking themes, some would require heavy GTK+ theming and patching GDM/GNOME.
''clean / well designed'': without becoming something that gets annoying to look at a year down the road. Examples of good themes include Joy, Lines, Softwaves and futurePrototype.

If you'd like more information or details, please post to the Debian Desktop mailing list.

20 June 2024

C.J. Collier: Signed NVIDIA drivers on Google Cloud Dataproc 2.2

Hello folks, I ve been working this year on better integrating NVIDIA hardware with the Google Cloud Dataproc product (Hadoop on Google Cloud) running the default cluster node image. We have an open bug[1] in the initialization-actions repo regarding creation failures upon enabling secure boot. This is because with secure boot, kernel driver code has its signature verified before insmod places the symbols into kernel memory. The verification process involves reading trust root certificates from EFI variables, and validating that the signatures on the kernel driver either a) were made directly by one of the certificates in the boot sector or b) were made by certificates which chain up to one of them. This means that Dataproc disk images must have a certificate installed into them. My work on the internals will likely start producing images which have certificates from Google in them. In the meantime, however, our users are left without a mechanism to have both secure boot enabled and install out-of-tree kernel modules such as the NVIDIA GPU drivers. To that end, I ve got PR #83[2] open with the GoogleCloudDataproc/custom-images github repository. This PR introduces a new argument to the custom image creation script, trusted-cert , the argument of which is the path to a DER-encoded certificate to be included in the certificate database in the EFI variables of the disk s boot sector. I ve written up the instructions on creating a custom image with a trusted certificate here: https://github.com/cjac/custom-images/blob/secure-boot-custom-image/examples/secure-boot/README.md Here is a set of commands that can be used to create a Dataproc custom image with certificate installed to the EFI s db variable. You can run these commands from the root directory of a checkout such as this:

git clone https://github.com/cjac/custom-images.git --branch secure-boot-custom-image --single-branch
pushd custom-images
PROJECT_ID=your-project-here
PROJECT_NUMBER=your-project-nnnn-here
my_bucket=your-bucket-here
custom_image_zone=your-zone-here
gcloud projects add-iam-policy-binding $ PROJECT_ID  \
        --member=serviceAccount:$ PROJECT_NUMBER -compute@developer.gserviceaccount.com \
        --role=roles/secretmanager.secretAccessor
gcloud config set project $ PROJECT_ID 
gcloud auth login
eval $(bash examples/secure-boot/create-key-pair.sh)
metadata="public_secret_name=$ public_secret_name "
metadata="$ metadata ,private_secret_name=$ private_secret_name "
metadata="$ metadata ,secret_project=$ secret_project "
metadata="$ metadata ,secret_version=$ secret_version "
#dataproc_version=2.1-debian11
dataproc_version=2.2-debian12
#customization_script=examples/secure-boot/install-nvidia-driver-debian11.sh
customization_script=examples/secure-boot/install-nvidia-driver-debian12.sh
#image_name="nvidia-open-kernel-bullseye-$(date +%F)"
image_name="nvidia-open-kernel-bookworm-$(date +%F)"
disk_size_gb="50"
python generate_custom_image.py \
    --image-name $ image_name  \
    --dataproc-version $ dataproc_version  \
    --trusted-cert "tls/db.der" \
    --customization-script $ customization_script  \
    --metadata "$ metadata " \
    --zone "$ custom_image_zone " \
    --disk-size "$ disk_size_gb " \
    --no-smoke-test \
    --gcs-bucket "$ my_bucket "
popd

I d love to hear your feedback! [1] https://github.com/GoogleCloudDataproc/initialization-actions/issues/1058
[2] https://github.com/GoogleCloudDataproc/custom-images/pull/83

Daniel Lange: Fixing esptool read_flash above 2MB on some cheap ESP32 boards

esptool, the Espressif SoC serial bootloader utility, tends to dislike cheap Flash chips attached to the various incarnations of the ESP32 chip family. And it seems to dislike them even more when running esptool on Linux than on other OSs. The common error mode is seeing it break at the 2MB barrier when trying to dump (esptool read_flash) a 4MB flash configuration.

esptool -p /dev/ttyUSB0 -b 921600 read_flash 0 0x400000 flash_dump.bin

will fail with

esptool.py v4.7.0
Serial port /dev/ttyUSB0
Connecting....
Detecting chip type... ESP32
Chip is ESP32-D0WD-V3 (revision v3.1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
[..]
Detected flash size: 4MB
[..]
2097152 (50 %)
A fatal error occurred: Failed to read flash block (result was 01090000: CRC or checksum was invalid)

typically at the 2MB barrier. I found the solution in a rather unrelated esptool Github issue: Create an esptool.cfg file in the project directory (from where you will run esptool):

[esptool]
timeout = 30
max_timeout = 240
erase_write_timeout_per_mb = 40
mem_end_rom_timeout = 0.2
serial_write_timeout = 10

The timeout = 30 is the setting that fixed reading flash memory via esptool read_flash for me. When your esptool.cfg is read, esptool will tell you so in its second line of output:

$ esptool flash_id

esptool.py v4.7.0
Loaded custom configuration from /home/dl/[..]/Embedded_dev/ESP-32_Wemos/esptool.cfg
Found 1 serial ports
Serial port /dev/ttyUSB0
Connecting......
[..]

Thank you Radim Karnis and wibbit from the Github issue linked above.

14 June 2024

Matthew Palmer: Information Security: "We Can Do It, We Just Choose Not To"

Whenever a large corporation disgorges the personal information of millions of people onto the Internet, there is a standard playbook that is followed. Security is our top priority . Passwords were hashed . No credit card numbers were disclosed . record scratch Let s talk about that last one a bit.

A Case Study This post could have been written any time in the past well, decade or so, really. But the trigger for my sitting down and writing this post is the recent breach of wallet-finding and criminal-harassment-enablement platform Tile. As reported by Engadget, a statement attributed to Life360 CEO Chris Hulls says
The potentially impacted data consists of information such as names, addresses, email addresses, phone numbers, and Tile device identification numbers.
But don t worry though; even though your home address is now public information
It does not include more sensitive information, such as credit card numbers
Aaaaaand here is where I get salty.

Why Credit Card Numbers Don t Matter Describing credit card numbers as more sensitive information is somewhere between disingenuous and a flat-out lie. It was probably included in the statement because it s part of the standard playbook. Why is it part of the playbook, though? Not being a disaster comms specialist, I can t say for sure, but my hunch is that the post-breach playbook includes this line because (a) credit cards are less commonly breached these days (more on that later), and (b) it s a way to insinuate that all your financial data is safe, no need to worry without having to say that (because that statement would absolutely be a lie). The thing that not nearly enough people realise about credit card numbers is:

The credit card holder is not usually liable for most fraud done via credit card numbers; and

In terms of actual, long-term damage to individuals, credit card fraud barely rates a mention. Identity fraud, Business Email Compromise, extortion, and all manner of other unpleasantness is far more damaging to individuals.

Why Credit Card Numbers Do Matter Losing credit card numbers in a data breach is a huge deal but not for the users of the breached platform. Instead, it s a problem for the company that got breached. See, going back some years now, there was a wave of huge credit card data breaches. If you ve been around a while, names like Target and Heartland will bring back some memories. Because these breaches cost issuing banks and card brands a lot of money, the Payment Card Industry Security Standards Council (PCI-SSC) and the rest of the ecosystem went full goblin mode. Now, if you lose credit card numbers in bulk, it will cost you big. Massive fines for breaches (typically levied by the card brands via the acquiring bank), increased transaction fees, and even the Credit Card Death Penalty (being banned from charging credit cards), are all very big sticks.

Now Comes the Finding Out In news that should not be surprising, when there are actual consequences for failing to do something, companies take the problem seriously. Which is why no credit card numbers were disclosed is such an interesting statement. Consider why no credit card numbers were disclosed. It s not that credit card numbers aren t valuable to criminals because they are. Instead, it s because the company took steps to properly secure the credit card data. Next, you ll start to consider why, if the credit card numbers were secured, why wasn t the personal information that did get disclosed similarly secured? Information that is far more damaging to the individuals to whom that information relates than credit card numbers. The only logical answer is that it wasn t deemed financially beneficial to the company to secure that data. The consequences of disclosure for that information isn t felt by the company which was breached. Instead, it s felt by the individuals who have to spend weeks of their life cleaning up from identity fraud committed against them. It s felt by the victim of intimate partner violence whose new address is found in a data dump, letting their ex find them again. Until there are real, actual consequences for the companies which hemorrhage our personal data (preferably ones that have percentage of global revenue at the end), data breaches will continue to happen. Not because they re inevitable because as credit card numbers show, data can be secured but because there s no incentive for companies to prevent our personal data from being handed over to whoever comes along.

Support my Salt My salty takes are powered by refreshing beverages. If you d like to see more of the same, buy me one.

12 June 2024

Matthew Garrett: SSH agent extensions as an arbitrary RPC mechanism

A while back, I wrote about using the SSH agent protocol to satisfy WebAuthn requests. The main problem with this approach is that it required starting the SSH agent with a special argument and also involved being a little too friendly with the implementation - things worked because I could provide an arbitrary public key and the implementation never validated that, but it would be legitimate for it to start doing so and then break everything. And it also only worked for keys stored on tokens that ssh supports - there was no way to extend this to other keystores on the client (such as the Secure Enclave on Macs, or TPM-backed keys on PCs). I wanted a better solution.

It turns out that it was far easier than I expected. The ssh agent protocol is documented here, and the interesting part is the extension support extension mechanism. Basically, you can declare an extension and then just tunnel whatever you want over it. As before, my goto was the go ssh agent package which conveniently implements both the client and server side of this. Implementing the local agent is trivial - look up SSH_AUTH_SOCK, connect to it, create a new agent client that can communicate with that by calling NewClient, and then implement the ExtendedAgent interface, create a new socket, and call ServeAgent against that. Most of the ExtendedAgent functions should simply call through to the original agent, with the exception of Extension(). Just add a case statement against extensionType, define some reasonably namespaced extension, and you're done.

Now you need to use this agent. You probably don't want to use this for arbitrary hosts (agent forwarding should only be enabled for remote systems you trust, not arbitrary machines you connect to - if you enabled agent forwarding for github and github got compromised, github would be able to use any private keys loaded into your agent, and you probably don't want that). So the right approach is to add a Host entry to the ssh config with a ForwardAgent stanza pointing at the socket you created in your new agent. This way the configured subset of remote hosts will automatically talk to this new custom agent, while forwarding for anything else will still be at the user's discretion.

For the remote end things are even easier. Look up SSH_AUTH_SOCK and call NewClient as before, and then simply call client.Extension(). Whatever you stick in the contents argument will simply end up being received at the client end. You now have a communication channel between a the remote system and the local client, and what you do with that is up to you. I'm using it to allow a remote system to obtain auth tokens from Okta and forward WebAuthn challenges that can either be satisfied via a local WebAuthn token or by passing the query off to Mac TouchID, but there's fundamentally no constraints whatsoever on what can be done here.

(If you want to do this on Windows and still have everything work with existing clients you'll need to take this into account - Windows didn't really do Unix sockets until recently so everything there is awful)

comments

Freexian Collaborators: Monthly report about Debian Long Term Support, May 2024 (by Roberto C. S nchez)

Like each month, have a look at the work funded by Freexian s Debian LTS offering.

Debian LTS contributors In May, 17 contributors have been paid to work on Debian LTS, their reports are available:

Adrian Bunk did 34.25h (out of 24.0h assigned and 22.0h from previous period), thus carrying over 11.75h to the next month.

Bastien Roucari s did 20.0h (out of 20.0h assigned).

Ben Hutchings did 16.0h (out of 24.0h assigned), thus carrying over 8.0h to the next month.

Chris Lamb did 18.0h (out of 18.0h assigned).

Daniel Leidert did 8.0h (out of 10.0h assigned), thus carrying over 2.0h to the next month.

Emilio Pozuelo Monfort did 35.5h (out of 46.0h assigned), thus carrying over 10.5h to the next month.

Guilhem Moulin did 13.0h (out of 14.75h assigned and 5.25h from previous period), thus carrying over 7.0h to the next month.

Lee Garrett did 11.0h (out of 37.25h assigned and 8.75h from previous period), thus carrying over 35.0h to the next month.

Lucas Kanashiro did 10.0h (out of 20.0h assigned), thus carrying over 10.0h to the next month.

Markus Koschany did 40.0h (out of 40.0h assigned).

Ola Lundqvist did 6.5h (out of 22.5h assigned and 1.5h from previous period), thus carrying over 17.5h to the next month.

Roberto C. S nchez did 7.75h (out of 11.0h assigned and 1.0h from previous period), thus carrying over 4.25h to the next month.

Santiago Ruano Rinc n did 8.0h (out of 16.0h assigned), thus carrying over 8.0h to the next month.

Sean Whitton did 5.5h (out of 5.5h assigned and 0.5h from previous period), thus carrying over 0.5h to the next month.

Sylvain Beucler did 10.5h (out of 0.75h assigned and 45.25h from previous period), thus carrying over 35.5h to the next month.

Thorsten Alteholz did 14.0h (out of 14.0h assigned).

Tobias Frost did 7.75h (out of 10.0h assigned and 2.0h from previous period), thus carrying over 4.25h to the next month.

Evolution of the situation In May, we have released 20 DLAs. Notable security updates in May included:

apache2: multiple vulnerabilities which may result in HTTP response splitting, denial of service, or authorization bypass (by Bastien Roucari s, in collaboration with apache2 maintainer Yadd)

bind9: two vulnerabilities, called KeyTrap and NSEC3, which may result in denial of service (by Santiago Ruano Rinc n)

python-pymysql: potential SQL injection attack (by Chris Lamb)

The aforementioned apache2 was prepared by its Debian maintainer Yadd. This update also involved work on the package test suite in buster, which contributor Bastien Roucari s then forwarded to the apache2 package in unstable. More importantly, a regression in fossil was reported, and Bastien prepared a fix for it. Bastien coordinated the upload of both packages to minimize the introduction of regressions. Contributor Daniel Leidert also prepared an upload of runc to Debian 11 in order fix a number of CVEs still affecting that package. Finally, contributor Thorsten Alteholz prepared uploads for qtbase-opensource-src, libjwt, and libmicrohttpd in Debian 11. Note that Debian 11 will pass into the LTS phase of support in August and these updates will improve the state and long-term supportability of Debian 11. Debian 10 is presently in its final month of LTS support (as announced on the debian-lts-announce mailing list, support will end on June 30th), after which no new security updates will be made available on security.debian.org. However, Freexian and its team of paid Debian contributors will continue to maintain Debian 10 going forward for the customers of the Extended LTS offer. Subscribe right away if you sill have Debian 10 which must be kept secure (and which cannot yet be upgraded).

Thanks to our sponsors Sponsors that joined recently are in bold.

Platinum sponsors:

TOSHIBA (for 105 months)

Civil Infrastructure Platform (CIP) (for 73 months)

VyOS Inc (for 37 months)

Gold sponsors:

Roche Diagnostics International AG (for 116 months)

Linode (for 110 months)

Babiel GmbH (for 99 months)

Plat Home (for 99 months)

CINECA (for 73 months)

University of Oxford (for 55 months)

Deveryware (for 42 months)

EDF SA (for 26 months)

CERN

Silver sponsors:

Domeneshop AS (for 120 months)

Nantes M tropole (for 115 months)

Univention GmbH (for 106 months)

Universit Jean Monnet de St Etienne (for 106 months)

Ribbon Communications, Inc. (for 100 months)

Exonet B.V. (for 90 months)

Leibniz Rechenzentrum (for 84 months)

Minist re de l Europe et des Affaires trang res (for 67 months)

Cloudways by DigitalOcean (for 57 months)

Dinahosting SL (for 55 months)

Bauer Xcel Media Deutschland KG (for 49 months)

Platform.sh SAS (for 49 months)

Moxa Inc. (for 43 months)

sipgate GmbH (for 40 months)

OVH US LLC (for 38 months)

Tilburg University (for 38 months)

GSI Helmholtzzentrum f r Schwerionenforschung GmbH (for 30 months)

Soliton Systems K.K. (for 27 months)

THINline s.r.o.

Bronze sponsors:

Evolix (for 121 months)

Seznam.cz, a.s. (for 121 months)

Intevation GmbH (for 118 months)

Linuxhotel GmbH (for 118 months)

Daevel SARL (for 116 months)

Bitfolk LTD (for 115 months)

Megaspace Internet Services GmbH (for 115 months)

NUMLOG (for 115 months)

Greenbone AG (for 114 months)

WinGo AG (for 114 months)

Ecole Centrale de Nantes - LHEEA (for 110 months)

Entr ouvert (for 105 months)

Adfinis AG (for 102 months)

GNI MEDIA (for 97 months)

Laboratoire LEGI - UMR 5519 / CNRS (for 97 months)

Tesorion (for 97 months)

Bearstech (for 88 months)

LiHAS (for 88 months)

Catalyst IT Ltd (for 83 months)

Supagro (for 78 months)

Demarcq SAS (for 77 months)

Universit Grenoble Alpes (for 63 months)

TouchWeb SAS (for 55 months)

SPiN AG (for 52 months)

CoreFiling (for 47 months)

Institut des sciences cognitives Marc Jeannerod (for 42 months)

Observatoire des Sciences de l Univers de Grenoble (for 39 months)

Tem Innovations GmbH (for 34 months)

WordFinder.pro (for 33 months)

CNRS DT INSU R sif (for 32 months)

Alter Way (for 25 months)

Institut Camille Jordan (for 14 months)

8 June 2024

Reproducible Builds: Reproducible Builds in May 2024

Welcome to the May 2024 report from the Reproducible Builds project! In these reports, we try to outline what we have been up to over the past month and highlight news items in software supply-chain security more broadly. As ever, if you are interested in contributing to the project, please visit our Contribute page on our website. Table of contents:

A peek into build provenance for Homebrew Joe Sweeney and William Woodruff on the Trail of Bits blog wrote an extensive post about build provenance for Homebrew, the third-party package manager for MacOS. Their post details how each bottle (i.e. each release):
[ ] built by Homebrew will come with a cryptographically verifiable statement binding the bottle s content to the specific workflow and other build-time metadata that produced it. [ ] In effect, this injects greater transparency into the Homebrew build process, and diminishes the threat posed by a compromised or malicious insider by making it impossible to trick ordinary users into installing non-CI-built bottles.
The post also briefly touches on future work, including work on source provenance:
Homebrew s formulae already hash-pin their source artifacts, but we can go a step further and additionally assert that source artifacts are produced by the repository (or other signing identity) that s latent in their URL or otherwise embedded into the formula specification.

Distribution news In Debian this month, Johannes Schauer Marin Rodrigues (aka josch) noticed that the Debian binary package `bash` version `5.2.15-2+b3` was uploaded to the archive twice. Once to bookworm and once to sid but with differing content. This is problem for reproducible builds in Debian due its assumption that the package name, version and architecture triplet is unique. However, josch highlighted that
This example with `bash` is especially problematic since `bash` is `Essential:yes`, so there will now be a large portion of `.buildinfo` files where it is not possible to figure out with which of the two differing bash packages the sources were compiled.
In response to this, Holger Levsen performed an analysis of all `.buildinfo` files and found that this needs almost 1,500 binNMUs to fix the fallout from this bug. Elsewhere in Debian, Vagrant Cascadian posted about a Non-Maintainer Upload (NMU) sprint to take place during early June, and it was announced that there is now a `#debian-snapshot` IRC channel on OFTC to discuss the creation of a new source code archiving service to, perhaps, replace snapshot.debian.org. Lastly, 11 reviews of Debian packages were added, 15 were updated and 48 were removed this month adding to our extensive knowledge about identified issues. A number of issue types have been updated by Chris Lamb as well. [ ][ ]
Elsewhere in the world of distributions, deep within a larger announcement from Colin Percival about the release of version 14.1-BETA2, it was mentioned that the FreeBSD kernels are now built reproducibly.
In Fedora, however, the change proposal mentioned in our report for April 2024 was approved, so, per the ReproduciblePackageBuilds wiki page, the add-determinism tool is now running in new builds for Fedora 41 ( rawhide ). The add-determinism tool is a Rust program which, as its name suggests, adds determinism to files that are given as input by attempting to standardize metadata contained in binary or source files to ensure consistency and clamping to `$SOURCE_DATE_EPOCH` in all instances . This is essentially the Fedora version of Debian s strip-nondeterminism. However, strip-nondeterminism is written in Perl, and Fedora did not want to pull Perl in the `buildroot` for every package. The add-determinism tool eliminates many causes of non-determinism and work is ongoing to continue the scope of packages it can operate on.

Mailing list news On our mailing list this month, regular contributor kpcyrd wrote to the list with an update on their source code indexing project, whatsrc.org. The whatsrc.org project, which was launched last month in response to the XZ Utils backdoor, now contains and indexes almost 250,000 unique source code archives. In their post, kpcyrd gives an example of its intended purpose, noting that it shown that whilst there seems to be consensus about [the] source code for zsh 5.9 in various Linux distributions, it does not align with the contents of the zsh Git repository . Holger Levsen also posted to the list with a pre-announcement of sorts for the 2024 Reproducible Builds summit. In particular:
[Whilst] the dates and location are not fixed yet, however if you don help us with finding a suitable location soon, it is very likely that we ll meet again in Hamburg in the 2nd half of September 2024 [ ].
Lastly, Frederic-Emmanuel Picca wrote to the list asking for help understanding the non-reproducible status of the Debian `silx` package and received replies from both Vagrant Cascadian and Chris Lamb.

Miscellaneous news strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. This month strip-nondeterminism version `1.14.0-1` was uploaded to Debian unstable by Chris Lamb chiefly to incorporate a change from Alex Muntada to avoid a dependency on `Sub::Override` to perform monkey-patching and break circular dependencies related to `debhelper` [ ]. Elsewhere in our tooling, Jelle van der Waa modified reprotest because the `pipes` module will be removed in Python version 3.13 [ ].
It was also noticed that a new blog post by Daniel Stenberg detailing How to verify a Curl release mentions the `SOURCE_DATE_EPOCH` environment variable. This is because:
The [curl] release tools document also contains another key component: the exact time stamp at which the release was done using integer second resolution. In order to generate a correct tarball clone, you need to also generate the new version using the old version s timestamp. Because the modification date of all files in the produced tarball will be set to this timestamp.

Furthermore, Fay Stegerman filed a bug against the Signal messenger app for Android to report that their reproducible builds cannot, in fact, be reproduced. However, Fay is quick to note that she has:
found zero evidence of any kind of compromise. Some differences are yet unexplained but everything I found seems to be benign. I am disappointed that Reproducible Builds have been broken for months but I have zero reason to doubt Signal s security in any way.

Lastly, it was observed that there was a concise and diagrammatic overview of supply chain threats on the SLSA website.

Two new academic papers Two new scholarly papers were published this month. Firstly, Mathieu Acher, Beno t Combemale, Georges Aaron Randrianaina and Jean-Marc J z quel of University of Rennes on Embracing Deep Variability For Reproducibility & Replicability. The authors describe their approach as follows:
In this short [vision] paper we delve into the application of software engineering techniques, specifically variability management, to systematically identify and explicit points of variability that may give rise to reproducibility issues (e.g., language, libraries, compiler, virtual machine, OS, environment variables, etc.). The primary objectives are: i) gaining insights into the variability layers and their possible interactions, ii) capturing and documenting configurations for the sake of reproducibility, and iii) exploring diverse configurations to replicate, and hence validate and ensure the robustness of results. By adopting these methodologies, we aim to address the complexities associated with reproducibility and replicability in modern software systems and environments, facilitating a more comprehensive and nuanced perspective on these critical aspects.
(A PDF of this article is available.)
Secondly, Ludovic Court s, Timothy Sample, Simon Tournier and Stefano Zacchiroli have collaborated to publish a paper on Source Code Archiving to the Rescue of Reproducible Deployment. Their paper was motivated because:
The ability to verify research results and to experiment with methodologies are core tenets of science. As research results are increasingly the outcome of computational processes, software plays a central role. GNU Guix is a software deployment tool that supports reproducible software deployment, making it a foundation for computational research workflows. To achieve reproducibility, we must first ensure the source code of software packages Guix deploys remains available.
(A PDF of this article is also available.)

diffoscope diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes such as uploading versions `266`, `267`, `268` and `269` to Debian, making the following changes:

New features:

Use `xz --list` to supplement output when comparing .xz archives; essential when metadata differs. (#1069329)

Include `xz --verbose --verbose` (ie. double) output. (#1069329)

Strip the first line from the `xz --list` output. [ ]

Only include `xz --list --verbose` output if the `xz` has no other differences. [ ]

Actually append the `xz --list` after the container differences, as it simplifies a lot. [ ]

Testing improvements:

Allow Debian testing to fail right now. [ ]

Drop `apktool` from `Build-Depends`; we can still test APK functionality via autopkgtests. (#1071410)

Add a versioned dependency for at least version 5.4.5 for the `xz` tests as they fail under (at least) version 5.2.8. (#374)

Fix tests for `7zip` 24.05. [ ][ ]

Fix all tests after additon of `xz --list`. [ ][ ]

Misc:

Update copyright years. [ ]

In addition, James Addison fixed an issue where the HTML output showed only the first difference in a file, while the text output shows all differences [ ][ ][ ], Sergei Trofimovich amended the `7zip` version test for older 7z versions that include the string `[64]` [ ][ ] and Vagrant Cascadian relaxed the versioned dependency to allow version 5.4.1 for the `xz` tests [ ] and proposed updates to guix for versions 267, 268 and pushed version 269 to Guix. Furthermore, Eli Schwartz updated the diffoscope.org website in order to explain how to install diffoscope on Gentoo [ ].

Website updates There were a number of improvements made to our website this month, including Chris Lamb making the print CSS stylesheet nicer [ ]. Fay Stegerman made a number of updates to the page about the `SOURCE_DATE_EPOCH` environment variable [ ][ ][ ] and Holger Levsen added some of their presentations to the Resources page. Furthermore, IOhannes zm lnig stipulated support for `SOURCE_DATE_EPOCH` in clang version 16.0.0+ [ ], Jan Zerebecki expanded the Formal definition page and fixed a number of typos on the Buy-in page [ ] and Simon Josefsson fixed the link to Trisquel GNU/Linux on the Projects page [ ].

Upstream patches This month, we wrote a number of patches to fix specific reproducibility issues, including:

Bernhard M. Wiedemann:

`nauty` (CPU-detection issue)

`emacs` (ASLR)

Chris Lamb:

#1070754 filed against `gensio`.

#1071064 filed against `tkgate`.

#1072094 filed against `ruby-pgplot`.

Reproducibility testing framework The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In May, a number of changes were made by Holger Levsen:

Debian-related changes:

Enable the rebuilder-snapshot API on `osuosl4`. [ ]

Schedule the `i386` architecture a bit more often. [ ]

Adapt `cleanup_nodes.sh` to the new way of running our build services. [ ]

Add 8 more workers for the `i386` architecture. [ ]

Update configuration now that the `infom07` and `infom08` nodes have been reinstalled as real `i386` systems. [ ]

Make diffoscope timeouts more visible on the `#debian-reproducible-changes` IRC channel. [ ]

Mark the `cbxi4a-armhf` node as down. [ ][ ]

Only install the `hdmi2usb-mode-switch` package only on Debian bookworm and earlier [ ] and only install the `haskell-platform` package on Debian bullseye [ ].

Misc:

Install the `ntpdate` utility as we need it later. [ ]

Document the progress on the `i386` architecture nodes at Infomaniak. [ ]

Drop an outdated and unnoticed notice. [ ]

Add `live_setup_schroot` to the list of so-called zombie jobs. [ ]

In addition, Mattia Rizzolo reinstalled the `infom07` and `infom08` nodes [ ] and Vagrant Cascadian marked the `cbxi4a` node as online [ ].

If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

IRC: `#reproducible-builds` on `irc.oftc.net`.

Twitter: @ReproBuilds

Mastodon: @reproducible_builds@fosstodon.org

Mailing list: `rb-general@lists.reproducible-builds.org`

Next.

Previous.