When you re accessing services over the WEB let s pick GMail as an example couple of things have to happen upfront: The server you re connecting to (GMail in our example) has to get to know who you are. Only after getting to know who you are it s able to decide what resources you are allowed to access (e.g. your own email inbox, your Calendar, Drive etc.). Step 1 above is called authentication.

FTP master This month I accepted 13 and rejected 2 packages. The overall number of packages that got accepted was 13. As the Release Team prefers not to have any new package upload to unstable, the numbers are this low. I am afraid there is some discussion needed after the release of Bullseye Debian LTS This was my eighty-fifth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 30h. During that time I did LTS and normal security uploads of:

• [DLA 2720-1] aspell security update for two CVEs
• [DLA 2722-1] libsndfile security update for one CVE
• [DLA 2723-1] linuxptp security update for one CVE
• prepared aspell update in Buster, which resulted in DSA 4948-1

I also made further progress on gpac and started to test the package now. Last but not least I did some days of frontdesk duties. I am not sure whether it is just me, but I got the impression that nowadays lots of CVEs can be marked as not-affected in the corresponding Stretch-version. Most of the remaining CVEs only have a small security impact (if at all) and can be marked as no-dsa. So the number of packages that really need an update decreases steadily. Does that mean that all issues in older versions are fixed now? Or are people more focused on new features in software as it is easier to find issues in more or less unexplored code? Debian ELTS This month was the thirty-seventh ELTS month. During my allocated time I uploaded:

• ELA-461-1 for jasper
• ELA-462-1 for aspell
• ELA-464-1 for libsndfile

Last but not least I did some days of frontdesk duties. In ELTS the decreasing number of uploads, as mentioned above, seems to be even more clearly. Other stuff I played a bit with RISC-V and looked after some packages that did not build on that architecture. Generally this looks like fun but building packages with qemu dampens the mood a bit. So if anybody knows some hardware that runs Debian, that is available now and that does not cost more than my car, I would be happy to get some pointer. This month I uploaded new upstream versions of:

• libosmocore
• libsmpp34
• meep

to experimental. I improved packaging and fixed bugs in:

• libcds

On my neverending golang challenge I again uploaded some packages either for NEW or as source upload.

In July I was assigned 16 hours of work by Freexian's Debian LTS initiative and carried over 2.75 hours from earlier months. I worked 12.75 hours and will carry over the remainder. I applied some urgent (and some not-so-urgent) security fixes to the linux (Linux 4.9), uploaded it, and issued DLA-2713-2. I also updated the linux-4.19 package based on the stable security update, and issued DLA-2714-1.

Review: The Last Battle, by C.S. Lewis

Illustrator:	Pauline Baynes
Series:	Chronicles of Narnia #7
Publisher:	Collier Books
Copyright:	1956
Printing:	1978
ISBN:	0-02-044210-6
Format:	Mass market
Pages:	184

The Last Battle is the seventh and final book of the Chronicles of Narnia in every reading order. It ties together (and spoils) every previous Narnia book, so you do indeed want to read it last (or skip it entirely, but I'll get into that). In the far west of Narnia, beyond the Lantern Waste and near the great waterfall that marks Narnia's western boundary, live a talking ape named Shift and a talking donkey named Puzzle. Shift is a narcissistic asshole who has been gaslighting and manipulating Puzzle for years, convincing the poor donkey that he's stupid and useless for anything other than being Shift's servant. At the start of the book, a lion skin washes over the waterfall and into the Cauldron Pool. Shift, seeing a great opportunity, convinces Puzzle to retrieve it. The king of Narnia at this time is Tirian. I would tell you more about Tirian except, despite being the protagonist, that's about all the characterization he gets. He's the king, he's broad-shouldered and strong, he behaves in a correct kingly fashion by preferring hunting lodges and simple camps to the capital at Cair Paravel, and his close companion is a unicorn named Jewel. Other than that, he's another character like Rilian from The Silver Chair who feels like he was taken from a medieval Arthurian story. (Thankfully, unlike Rilian, he doesn't talk like he's in a medieval Arthurian story.) Tirian finds out about Shift's scheme when a dryad appears at Tirian's camp, calling for justice for the trees of Lantern Waste who are being felled. Tirian rushes to investigate and stop this monstrous act, only to find the beasts of Narnia cutting down trees and hauling them away for Calormene overseers. When challenged on why they would do such a thing, they reply that it's at Aslan's orders. The Last Battle is largely the reason why I decided to do this re-read and review series. It is, let me be clear, a bad book. The plot is absurd, insulting to the characters, and in places actively offensive. It is also, unlike the rest of the Narnia series, dark and depressing for nearly all of the book. The theology suffers from problems faced by modern literature that tries to use the Book of Revelation and related Christian mythology as a basis. And it is, most famously, the site of one of the most notorious authorial betrayals of a character in fiction. And yet, The Last Battle, probably more than any other single book, taught me to be a better human being. It contains two very specific pieces of theology that I would now critique in multiple ways but which were exactly the pieces of theology that I needed to hear when I first understood them. This book steered me away from a closed, judgmental, and condemnatory mindset at exactly the age when I needed something to do that. For that, I will always have a warm spot in my heart for it. I'm going to start with the bad parts, though, because that's how the book starts. MAJOR SPOILERS BELOW. First, and most seriously, this is a second-order idiot plot. Shift shows up with a donkey wearing a lion skin (badly), only lets anyone see him via firelight, claims he's Aslan, and starts ordering the talking animals of Narnia to completely betray their laws and moral principles and reverse every long-standing political position of the country... and everyone just nods and goes along with this. This is the most blatant example of a long-standing problem in this series: Lewis does not respect his animal characters. They are the best feature of his world, and he treats them as barely more intelligent than their non-speaking equivalents and in need of humans to tell them what to do. Furthermore, despite the assertion of the narrator, Shift is not even close to clever. His deception has all the subtlety of a five-year-old who doesn't want to go to bed, and he offers the Narnians absolutely nothing in exchange for betraying their principles. I can forgive Puzzle for going along with the scheme since Puzzle has been so emotionally abused that he doesn't know what else to do, but no one else has any excuse, especially Shift's neighbors. Given his behavior in the book, everyone within a ten mile radius would be so sick of his whining, bullying, and lying within a month that they'd never believe anything he said again. Rishda and Ginger, a Calormene captain and a sociopathic cat who later take over Shift's scheme, do qualify as clever, but there's no realistic way Shift's plot would have gotten far enough for them to get involved. The things that Shift gets the Narnians to do are awful. This is by far the most depressing book in the series, even more than the worst parts of The Silver Chair. I'm sure I'm not the only one who struggled to read through the first part of this book, and raced through it on re-reads because everything is so hard to watch. The destruction is wanton and purposeless, and the frequent warnings from both characters and narration that these are the last days of Narnia add to the despair. Lewis takes all the beautiful things that he built over six books and smashes them before your eyes. It's a lot to take, given that previous books would have treated the felling of a single tree as an unspeakable catastrophe. I think some of these problems are due to the difficulty of using Christian eschatology in a modern novel. An antichrist is obligatory, but the animals of Narnia have no reason to follow an antichrist given their direct experience with Aslan, particularly not the aloof one that Shift tries to give them. Lewis forces the plot by making everyone act stupidly and out of character. Similarly, Christian eschatology says everything must become as awful as possible right before the return of Christ, hence the difficult-to-read sections of Narnia's destruction, but there's no in-book reason for the Narnians' complicity in that destruction. One can argue about whether this is good theology, but it's certainly bad storytelling. I can see the outlines of the moral points Lewis is trying to make about greed and rapacity, abuse of the natural world, dubious alliances, cynicism, and ill-chosen prophets, but because there is no explicable reason for Tirian's quiet kingdom to suddenly turn to murderous resource exploitation, none of those moral points land with any force. The best moral apocalypse shows the reader how, were they living through it, they would be complicit in the devastation as well. Lewis does none of that work, so the reader is just left angry and confused. The book also has several smaller poor authorial choices, such as the blackface incident. Tirian, Jill, and Eustace need to infiltrate Shift's camp, and use blackface to disguise themselves as Calormenes. That alone uncomfortably reveals how much skin tone determines nationality in this world, but Lewis makes it far worse by having Tirian comment that he "feel[s] a true man again" after removing the blackface and switching to Narnian clothes. All of this drags on and on, unlike Lewis's normally tighter pacing, to the point that I remembered this book being twice the length of any other Narnia book. It's not; it's about the same length as the rest, but it's such a grind that it feels interminable. The sum total of the bright points of the first two-thirds of the book are the arrival of Jill and Eustace, Jill's one moment of true heroism, and the loyalty of a single Dwarf. The rest is all horror and betrayal and doomed battles and abject stupidity. I do, though, have to describe Jill's moment of glory, since I complained about her and Eustace throughout The Silver Chair. Eustace is still useless, but Jill learned forestcraft during her previous adventures (not that we saw much sign of this previously) and slips through the forest like a ghost to steal Puzzle and his lion costume out from the under the nose of the villains. Even better, she finds Puzzle and the lion costume hilarious, which is the one moment in the book where one of the characters seems to understand how absurd and ridiculous this all is. I loved Jill so much in that moment that it makes up for all of the pointless bickering of The Silver Chair. She doesn't get to do much else in this book, but I wish the Jill who shows up in The Last Battle had gotten her own book. The end of this book, and the only reason why it's worth reading, happens once the heroes are forced into the stable that Shift and his co-conspirators have been using as the stage for their fake Aslan. Its door (for no well-explained reason) has become a door to Aslan's Country and leads to a reunion with all the protagonists of the series. It also becomes the frame of Aslan's final destruction of Narnia and judging of its inhabitants, which I suspect would be confusing if you didn't already know something about Christian eschatology. But before that, this happens, which is sufficiently and deservedly notorious that I think it needs to be quoted in full.

"Sir," said Tirian, when he had greeted all these. "If I have read the chronicle aright, there should be another. Has not your Majesty two sisters? Where is Queen Susan?" "My sister Susan," answered Peter shortly and gravely, "is no longer a friend of Narnia." "Yes," said Eustace, "and whenever you've tried to get her to come and talk about Narnia or do anything about Narnia, she says 'What wonderful memories you have! Fancy your still thinking about all those funny games we used to play when we were children.'" "Oh Susan!" said Jill. "She's interested in nothing nowadays except nylons and lipstick and invitations. She always was a jolly sight too keen on being grown-up." "Grown-up indeed," said the Lady Polly. "I wish she would grow up. She wasted all her school time wanting to be the age she is now, and she'll waste all the rest of her life trying to stay that age. Her whole idea is to race on to the silliest time of one's life as quick as she can and then stop there as long as she can."

There are so many obvious and dire problems with this passage, and so many others have written about it at length, that I will only add a few points. First, I find it interesting that neither Lucy nor Edmund says a thing. (I would like to think that Edmund knows better.) The real criticism comes from three characters who never interacted with Susan in the series: the two characters introduced after she was no longer allowed to return to Narnia, and a character from the story that predated hers. (And Eustace certainly has some gall to criticize someone else for treating Narnia as a childish game.) It also doesn't say anything good about Lewis that he puts his rather sexist attack on Susan into the mouths of two other female characters. Polly's criticism is a somewhat generic attack on puberty that could arguably apply to either sex (although "silliness" is usually reserved for women), but Jill makes the attack explicitly gendered. It's the attack of a girl who wants to be one of the boys on a girl who embraces things that are coded feminine, and there's a whole lot of politics around the construction of gender happening here that Lewis is blindly reinforcing and not grappling with at all. Plus, this is only barely supported by single sentences in The Voyage of the Dawn Treader and The Horse and His Boy and directly contradicts the earlier books. We're expected to believe that Susan the archer, the best swimmer, the most sensible and thoughtful of the four kids has abruptly changed her whole personality. Lewis could have made me believe Susan had soured on Narnia after the attempted kidnapping (and, although left unstated, presumably eventual attempted rape) in The Horse and His Boy, if one ignores the fact that incident supposedly happens before Prince Caspian where there is no sign of such a reaction. But not for those reasons, and not in that way. Thankfully, after this, the book gets better, starting with the Dwarfs, which is one of the two passages that had a profound influence on me. Except for one Dwarf who allied with Tirian, the Dwarfs reacted to the exposure of Shift's lies by disbelieving both Tirian and Shift, calling a pox on both their houses, and deciding to make their own side. During the last fight in front of the stable, they started killing whichever side looked like they were winning. (Although this is horrific in the story, I think this is accurate social commentary on a certain type of cynicism, even if I suspect Lewis may have been aiming it at atheists.) Eventually, they're thrown through the stable door by the Calormenes. However, rather than seeing the land of beauty and plenty that everyone else sees, they are firmly convinced they're in a dark, musty stable surrounded by refuse and dirty straw. This is, quite explicitly, not something imposed on them. Lucy rebukes Eustace for wishing Tash had killed them, and tries to make friends with them. Aslan tries to show them how wrong their perceptions are, to no avail. Their unwillingness to admit they were wrong is so strong that they make themselves believe that everything is worse than it actually is.

"You see," said Aslan. "They will not let us help them. They have chosen cunning instead of belief. Their prison is only in their own minds, yet they are in that prison; and so afraid of being taken in that they cannot be taken out."

I grew up with the US evangelical version of Hell as a place of eternal torment, which in turn was used to justify religious atrocities in the name of saving people from Hell. But there is no Hell of that type in this book. There is a shadow into which many evil characters simply disappear, and there's this passage. Reading this was the first time I understood the alternative idea of Hell as the absence of God instead of active divine punishment. Lewis doesn't use the word "Hell," but it's obvious from context that the Dwarfs are in Hell. But it's not something Aslan does to them and no one wants them there; they could leave any time they wanted, but they're too unwilling to be wrong. You may have to be raised in conservative Christianity to understand how profoundly this rethinking of Hell (which Lewis tackles at greater length in The Great Divorce) undermines the system of guilt and fear that's used as motivation and control. It took me several re-readings and a lot of thinking about this passage, but this is where I stopped believing in a vengeful God who will eternally torture nonbelievers, and thus stopped believing in all of the other theology that goes with it. The second passage that changed me is Emeth's story. Emeth is a devout Calormene, a follower of Tash, who volunteered to enter the stable when Shift and his co-conspirators were claiming Aslan/Tash was inside. Some time after going through, he encounters Aslan, and this is part of his telling of that story (and yes, Lewis still has Calormenes telling stories as if they were British translators of the Arabian Nights):

[...] Lord, is it then true, as the Ape said, that thou and Tash are one? The Lion growled so that the earth shook (but his wrath was not against me) and said, It is false. Not because he and I are one, but because we are opposites, I take to me the services which thou hast done to him. For I and he are of such different kinds that no service which is vile can be done to me, and none which is not vile can be done to him. Therefore if any man swear by Tash and keep his oath for the oath's sake, it is by me that he has truly sworn, though he know it not, and it is I who reward him. And if any man do a cruelty in my name, then, though he says the name Aslan, it is Tash whom he serves and by Tash his deed is accepted. Dost thou understand, Child? I said, Lord, thou knowest how much I understand. But I said also (for the truth constrained me), Yet I have been seeking Tash all my days. Beloved, said the Glorious One, unless thy desire had been for me, thou wouldst not have sought so long and so truly. For all find what they truly seek.

So, first, don't ever say this to anyone. It's horribly condescending and, since it's normally said by white Christians to other people, usually explicitly colonialist. Telling someone that their god is evil but since they seem to be a good person they're truly worshiping your god is only barely better than saying yours is the only true religion. But it is better, and as someone who, at the time, was wholly steeped in the belief that only Christians were saved and every follower of another religion was following Satan and was damned to Hell, this passage blew my mind. This was the first place I encountered the idea that someone who followed a different religion could be saved, or that God could transcend religion, and it came with exactly the context and justification that I needed given how close-minded I was at the time. Today, I would say that the Christian side of this analysis needs far more humility, and fobbing off all the evil done in the name of the Christian God by saying "oh, those people were really following Satan" is a total moral copout. But, nonetheless, Lewis opened a door for me that I was able to step through and move beyond to a less judgmental, dismissive, and hostile view of others. There's not much else in the book after this. It's mostly Lewis's charmingly Platonic view of the afterlife, in which the characters go inward and upward to truer and more complete versions of both Narnia and England and are reunited (very briefly) with every character of the series. Lewis knows not to try too hard to describe the indescribable, but it remains one of my favorite visions of an afterlife because it makes so explicit that this world is neither static or the last, but only the beginning of a new adventure. This final section of The Last Battle is deeply flawed, rather arrogant, a little bizarre, and involves more lectures on theology than precise description, but I still love it. By itself, it's not a bad ending for the series, although I don't think it has half the beauty or wonder of the end of The Voyage of the Dawn Treader. It's a shame about the rest of the book, and it's a worse shame that Lewis chose to sacrifice Susan on the altar of his prejudices. Those problems made it very hard to read this book again and make it impossible to recommend. Thankfully, you can read the series without it, and perhaps most readers would be better off imagining their own ending (or lack of ending) to Narnia than the one Lewis chose to give it. But the one redeeming quality The Last Battle will always have for me is that, despite all of its flaws, it was exactly the book that I needed to read when I read it. Rating: 4 out of 10

After a very long porting journey, Launchpad is finally running on Python 3 across all of our systems. I wanted to take a bit of time to reflect on why my emotional responses to this port differ so much from those of some others who ve done large ports, such as the Mercurial maintainers. It s hard to deny that we ve had to burn a lot of time on this, which I m sure has had an opportunity cost, and from one point of view it s essentially running to stand still: there is no single compelling feature that we get solely by porting to Python 3, although it s clearly a prerequisite for tidying up old compatibility code and being able to use modern language facilities in the future. And yet, on the whole, I found this a rewarding project and enjoyed doing it. Some of this may be because by inclination I m a maintenance programmer and actually enjoy this sort of thing. My default view tends to be that software version upgrades may be a pain but it s much better to get that pain over with as soon as you can rather than trying to hold back the tide; you can certainly get involved and try to shape where things end up, but rightly or wrongly I can t think of many cases when a righteously indignant user base managed to arrange for the old version to be maintained in perpetuity so that they never had to deal with the new thing (OK, maybe Perl 5 counts here). I think a more compelling difference between Launchpad and Mercurial, though, may be that very few other people really had a vested interest in what Python version Launchpad happened to be running, because it s all server-side code (aside from some client libraries such as launchpadlib, which were ported years ago). As such, we weren t trying to do this with the internet having Strong Opinions at us. We were doing this because it was obviously the only long-term-maintainable path forward, and in more recent times because some of our library dependencies were starting to drop support for Python 2 and so it was obviously going to become a practical problem for us sooner or later; but if we d just stayed on Python 2 forever then fundamentally hardly anyone else would really have cared directly, only maybe about some indirect consequences of that. I don t follow Mercurial development so I may be entirely off-base, but if other people were yelling at me about how late my project was to finish its port, that in itself would make me feel more negatively about the project even if I thought it was a good idea. Having most of the pressure come from ourselves rather than from outside meant that wasn t an issue for us. I m somewhat inclined to think of the process as an extreme version of paying down technical debt. Moving from Python 2.7 to 3.5, as we just did, means skipping over multiple language versions in one go, and if similar changes had been made more gradually it would probably have felt a lot more like the typical dependency update treadmill. I appreciate why not everyone might want to think of it this way: maybe this is just my own rationalization. Reflections on porting to Python 3 I m not going to defend the Python 3 migration process; it was pretty rough in a lot of ways. Nor am I going to spend much effort relitigating it here, as it s already been done to death elsewhere, and as I understand it the core Python developers have got the message loud and clear by now. At a bare minimum, a lot of valuable time was lost early in Python 3 s lifetime hanging on to flag-day-type porting strategies that were impractical for large projects, when it should have been providing for bilingual strategies (code that runs in both Python 2 and 3 for a transitional period) which is where most libraries and most large migrations ended up in practice. For instance, the early advice to library maintainers to maintain two parallel versions or perhaps translate dynamically with 2to3 was entirely impractical in most non-trivial cases and wasn t what most people ended up doing, and yet the idea that 2to3 is all you need still floats around Stack Overflow and the like as a result. (These days, I would probably point people towards something more like Eevee s porting FAQ as somewhere to start.) There are various fairly straightforward things that people often suggest could have been done to smooth the path, and I largely agree: not removing the u'' string prefix only to put it back in 3.3, fewer gratuitous compatibility breaks in the name of tidiness, and so on. But if I had a time machine, the number one thing I would ask to have been done differently would be introducing type annotations in Python 2 before Python 3 branched off. It s true that it s technically possible to do type annotations in Python 2, but the fact that it s a different syntax that would have to be fixed later is offputting, and in practice it wasn t widely used in Python 2 code. To make a significant difference to the ease of porting, annotations would need to have been introduced early enough that lots of Python 2 library code used them so that porting code didn t have to be quite so much of an exercise of manually figuring out the exact nature of string types from context. Launchpad is a complex piece of software that interacts with multiple domains: for example, it deals with a database, HTTP, web page rendering, Debian-format archive publishing, and multiple revision control systems, and there s often overlap between domains. Each of these tends to imply different kinds of string handling. Web page rendering is normally done mainly in Unicode, converting to bytes as late as possible; revision control systems normally want to spend most of their time working with bytes, although the exact details vary; HTTP is of course bytes on the wire, but Python s WSGI interface has some string type subtleties. In practice I found myself thinking about at least four string-like types (that is, things that in a language with a stricter type system I might well want to define as distinct types and restrict conversion between them): bytes, text, ordinary native strings (str in either language, encoded to UTF-8 in Python 2), and native strings with WSGI s encoding rules. Some of these are emergent properties of writing in the intersection of Python 2 and 3, which is effectively a specialized language of its own without coherent official documentation whose users must intuit its behaviour by comparing multiple sources of information, or by referring to unofficial porting guides: not a very satisfactory situation. Fortunately much of the complexity collapses once it becomes possible to write solely in Python 3. Some of the difficulties we ran into are not ones that are typically thought of as Python 2-to-3 porting issues, because they were changed later in Python 3 s development process. For instance, the email module was substantially improved in around the 3.2/3.3 timeframe to handle Python 3 s bytes/text model more correctly, and since Launchpad sends quite a few different kinds of email messages and has some quite picky tests for exactly what it emits, this entailed a lot of work in our email sending code and in our test suite to account for that. (It took me a while to work out whether we should be treating raw email messages as bytes or as text; bytes turned out to work best.) 3.4 made some tweaks to the implementation of quoted-printable encoding that broke a number of our tests in ways that took some effort to fix, because the tests needed to work on both 2.7 and 3.5. The list goes on. I got quite proficient at digging through Python s git history to figure out when and why some particular bit of behaviour had changed. One of the thorniest problems was parsing HTTP form data. We mainly rely on zope.publisher for this, which in turn relied on cgi.FieldStorage; but cgi.FieldStorage is badly broken in some situations on Python 3. Even if that bug were fixed in a more recent version of Python, we can t easily use anything newer than 3.5 for the first stage of our port due to the version of the base OS we re currently running, so it wouldn t help much. In the end I fixed some minor issues in the multipart module (and was kindly given co-maintenance of it) and converted zope.publisher to use it. Although this took a while to sort out, it seems to have gone very well. A couple of other interesting late-arriving issues were around pickle. For most things we normally prefer safer formats such as JSON, but there are a few cases where we use pickle, particularly for our session databases. One of my colleagues pointed out that I needed to remember to tell pickle to stick to protocol 2, so that we d be able to switch back and forward between Python 2 and 3 for a while; quite right, and we later ran into a similar problem with marshal too. A more surprising problem was that datetime.datetime objects pickled on Python 2 require special care when unpickling on Python 3; rather than the approach that ended up being implemented and documented for Python 3.6, though, I preferred a custom unpickler, both so that things would work on Python 3.5 and so that I wouldn t have to risk affecting the decoding of other pickled strings in the session database. General lessons Writing this over a year after Python 2 s end-of-life date, and certainly nowhere near the leading edge of Python 3 porting work, it s perhaps more useful to look at this in terms of the lessons it has for other large technical debt projects. I mentioned in my previous article that I used the approach of an enormous and frequently-rebased git branch as a working area for the port, committing often and sometimes combining and extracting commits for review once they seemed to be ready. A port of this scale would have been entirely intractable without a tool of similar power to git rebase, so I m very glad that we finished migrating to git in 2019. I relied on this right up to the end of the port, and it also allowed for quick assessments of how much more there was to land. git worktree was also helpful, in that I could easily maintain working trees built for each of Python 2 and 3 for comparison. As is usual for most multi-developer projects, all changes to Launchpad need to go through code review, although we sometimes make exceptions for very simple and obvious changes that can be self-reviewed. Since I knew from the outset that this was going to generate a lot of changes for review, I therefore structured my work from the outset to try to make it as easy as possible for my colleagues to review it. This generally involved keeping most changes to a somewhat manageable size of 800 lines or less (although this wasn t always possible), and arranging commits mainly according to the kind of change they made rather than their location. For example, when I needed to fix issues with / in Python 3 being true division rather than floor division, I did so in one commit across the various places where it mattered and took care not to mix it with other unrelated changes. This is good practice for nearly any kind of development, but it was especially important here since it allowed reviewers to consider a clear explanation of what I was doing in the commit message and then skim-read the rest of it much more quickly. It was vital to keep the codebase in a working state at all times, and deploy to production reasonably often: this way if something went wrong the amount of code we had to debug to figure out what had happened was always tractable. (Although I can t seem to find it now to link to it, I saw an account a while back of a company that had taken a flag-day approach instead with a large codebase. It seemed to work for them, but I m certain we couldn t have made it work for Launchpad.) I can t speak too highly of Launchpad s test suite, much of which originated before my time. Without a great deal of extensive coverage of all sorts of interesting edge cases at both the unit and functional level, and a corresponding culture of maintaining that test suite well when making new changes, it would have been impossible to be anything like as confident of the port as we were. As part of the porting work, we split out a couple of substantial chunks of the Launchpad codebase that could easily be decoupled from the core: its Mailman integration and its code import worker. Both of these had substantial dependencies with complex requirements for porting to Python 3, and arranging to be able to do these separately on their own schedule was absolutely worth it. Like disentangling balls of wool, any opportunity you can take to make things less tightly-coupled is probably going to make it easier to disentangle the rest. (I can see a tractable way forward to porting the code import worker, so we may well get that done soon. Our Mailman integration will need to be rewritten, though, since it currently depends on the Python-2-only Mailman 2, and Mailman 3 has a different architecture.) Python lessons Our database layer was already in pretty good shape for a port, since at least the modern bits of its table modelling interface were already strict about using Unicode for text columns. If you have any kind of pervasive low-level framework like this, then making it be pedantic at you in advance of a Python 3 port will probably incur much less swearing in the long run, as you won t be trying to deal with quite so many bytes/text issues at the same time as everything else. Early in our port, we established a standard set of __future__ imports and started incrementally converting files over to them, mainly because we weren t yet sure what else to do and it seemed likely to be helpful. absolute_import was definitely reasonable (and not often a problem in our code), and print_function was annoying but necessary. In hindsight I m not sure about unicode_literals, though. For files that only deal with bytes and text it was reasonable enough, but as I mentioned above there were also a number of cases where we needed literals of the language s native str type, i.e. bytes in Python 2 and text in Python 3: this was particularly noticeable in WSGI contexts, but also cropped up in some other surprising places. We generally either omitted unicode_literals or used six.ensure_str in such cases, but it was definitely a bit awkward and maybe I should have listened more to people telling me it might be a bad idea. A lot of Launchpad s early tests used doctest, mainly in the style where you have text files that interleave narrative commentary with examples. The development team later reached consensus that this was best avoided in most cases, but by then there were far too many doctests to conveniently rewrite in some other form. Porting doctests to Python 3 is really annoying. You run into all the little changes in how objects are represented as text (particularly u'...' versus '...', but plenty of other cases as well); you have next to no tools to do anything useful like skipping individual bits of a doctest that don t apply; using __future__ imports requires the rather obscure approach of adding the relevant names to the doctest s globals in the relevant DocFileSuite or DocTestSuite; dealing with many exception tracebacks requires something like zope.testing.renormalizing; and whatever code refactoring tools you re using probably don t work properly. Basically, don t have done that. It did all turn out to be tractable for us in the end, and I managed to avoid using much in the way of fragile doctest extensions aside from the aforementioned zope.testing.renormalizing, but it was not an enjoyable experience. Regressions I know of nine regressions that reached Launchpad s production systems as a result of this porting work; of course there were various other regressions caught by CI or in manual testing. (Considering the size of this project, I count it as a resounding success that there were only nine production issues, and that for the most part we were able to fix them quickly.) Equality testing of removed database objects One of the things we had to do while porting to Python 3 was to implement the __eq__, __ne__, and __hash__ special methods for all our database objects. This was quite conceptually fiddly, because doing this requires knowing each object s primary key, and that may not yet be available if we ve created an object in Python but not yet flushed the actual INSERT statement to the database (most of our primary keys are auto-incrementing sequences). We thus had to take care to flush pending SQL statements in such cases in order to ensure that we know the primary keys. However, it s possible to have a problem at the other end of the object lifecycle: that is, a Python object might still be reachable in memory even though the underlying row has been DELETEd from the database. In most cases we don t keep removed objects around for obvious reasons, but it can happen in caching code, and buildd-manager crashed as a result (in fact while it was still running on Python 2). We had to take extra care to avoid this problem. Debian imports crashed on non-UTF-8 filenames Python 2 has some unfortunate behaviour around passing bytes or Unicode strings (depending on the platform) to shutil.rmtree, and the combination of some porting work and a particular source package in Debian that contained a non-UTF-8 file name caused us to run into this. The fix was to ensure that the argument passed to shutil.rmtree is a str regardless of Python version. We d actually run into something similar before: it s a subtle porting gotcha, since it s quite easy to end up passing Unicode strings to shutil.rmtree if you re in the process of porting your code to Python 3, and you might easily not notice if the file names in your tests are all encoded using UTF-8. lazr.restful ETags We eventually got far enough along that we could switch one of our four appserver machines (we have quite a number of other machines too, but the appservers handle web and API requests) to Python 3 and see what happened. By this point our extensive test suite had shaken out the vast majority of the things that could go wrong, but there was always going to be room for some interesting edge cases. One of the Ubuntu kernel team reported that they were seeing an increase in 412 Precondition Failed errors in some of their scripts that use our webservice API. These can happen when you re trying to modify an existing resource: the underlying protocol involves sending an If-Match header with the ETag that the client thinks the resource has, and if this doesn t match the ETag that the server calculates for the resource then the client has to refresh its copy of the resource and try again. We initially thought that this might be legitimate since it can happen in normal operation if you collide with another client making changes to the same resource, but it soon became clear that something stranger was going on: we were getting inconsistent ETags for the same object even when it was unchanged. Since we d recently switched a quarter of our appservers to Python 3, that was a natural suspect. Our lazr.restful package provides the framework for our webservice API, and roughly speaking it generates ETags by serializing objects into some kind of canonical form and hashing the result. Unfortunately the serialization was dependent on the Python version in a few ways, and in particular it serialized lists of strings such as lists of bug tags differently: Python 2 used [u'foo', u'bar', u'baz'] where Python 3 used ['foo', 'bar', 'baz']. In lazr.restful 1.0.3 we switched to using JSON for this, removing the Python version dependency and ensuring consistent behaviour between appservers. Memory leaks This problem took the longest to solve. We noticed fairly quickly from our graphs that the appserver machine we d switched to Python 3 had a serious memory leak. Our appservers had always been a bit leaky, but now it wasn t so much a small hole that we can bail occasionally as the boat is sinking rapidly : (Yes, this got in the way of working out what was going on with ETags for a while.) I spent ages messing around with various attempts to fix this. Since only a quarter of our appservers were affected, and we could get by on 75% capacity for a while, it wasn t urgent but it was definitely annoying. After spending some quality time with objgraph, for some time I thought traceback reference cycles might be at fault, and I sent a number of fixes to various upstream projects for those (e.g. zope.pagetemplate). Those didn t help the leaks much though, and after a while it became clear to me that this couldn t be the sole problem: Python has a cyclic garbage collector that will eventually collect reference cycles as long as there are no strong references to any objects in them, although it might not happen very quickly. Something else must be going on. Debugging reference leaks in any non-trivial and long-running Python program is extremely arduous, especially with ORMs that naturally tend to end up with lots of cycles and caches. After a while I formed a hypothesis that zope.server might be keeping a strong reference to something, although I never managed to nail it down more firmly than that. This was an attractive theory as we were already in the process of migrating to Gunicorn for other reasons anyway, and Gunicorn also has a convenient max_requests setting that s good at mitigating memory leaks. Getting this all in place took some time, but once we did we found that everything was much more stable: This isn t completely satisfying as we never quite got to the bottom of the leak itself, and it s entirely possible that we ve only papered over it using max_requests: I expect we ll gradually back off on how frequently we restart workers over time to try to track this down. However, pragmatically, it s no longer an operational concern. Mirror prober HTTPS proxy handling After we switched our script servers to Python 3, we had several reports of mirror probing failures. (Launchpad keeps lists of Ubuntu archive and image mirrors, and probes them every so often to check that they re reasonably complete and up to date.) This only affected HTTPS mirrors when probed via a proxy server, support for which is a relatively recent feature in Launchpad and involved some code that we never managed to unit-test properly: of course this is exactly the code that went wrong. Sadly I wasn t able to sort out that gap, but at least the fix was simple. Non-MIME-encoded email headers As I mentioned above, there were substantial changes in the email package between Python 2 and 3, and indeed between minor versions of Python 3. Our test coverage here is pretty good, but it s an area where it s very easy to have gaps. We noticed that a script that processes incoming email was crashing on messages with headers that were non-ASCII but not MIME-encoded (and indeed then crashing again when it tried to send a notification of the crash!). The only examples of these I looked at were spam, but we still didn t want to crash on them. The fix involved being somewhat more careful about both the handling of headers returned by Python s email parser and the building of outgoing email notifications. This seems to be working well so far, although I wouldn t be surprised to find the odd other incorrect detail in this sort of area. Failure to handle non-ISO-8859-1 URL-encoded form input Remember how I said that parsing HTTP form data was thorny? After we finished upgrading all our appservers to Python 3, people started reporting that they couldn t post Unicode comments to bugs, which turned out to be only if the attempt was made using JavaScript, and was because I hadn t quite managed to get URL-encoded form data working properly with zope.publisher and multipart. The current standard describes the URL-encoded format for form data as in many ways an aberrant monstrosity , so this was no great surprise. Part of the problem was some very strange choices in zope.publisher dating back to 2004 or earlier, which I attempted to clean up and simplify. The rest was that Python 2 s urlparse.parse_qs unconditionally decodes percent-encoded sequences as ISO-8859-1 if they re passed in as part of a Unicode string, so multipart needs to work around this on Python 2. I m still not completely confident that this is correct in all situations, but at least now that we re on Python 3 everywhere the matrix of cases we need to care about is smaller. Inconsistent marshalling of Loggerhead s disk cache We use Loggerhead for providing web browsing of Bazaar branches. When we upgraded one of its two servers to Python 3, we immediately noticed that the one still on Python 2 was failing to read back its revision information cache, which it stores in a database on disk. (We noticed this because it caused a deployment to fail: when we tried to roll out new code to the instance still on Python 2, Nagios checks had already caused an incompatible cache to be written for one branch from the Python 3 instance.) This turned out to be a similar problem to the pickle issue mentioned above, except this one was with marshal, which I didn t think to look for because it s a relatively obscure module mostly used for internal purposes by Python itself; I m not sure that Loggerhead should really be using it in the first place. The fix was relatively straightforward, complicated mainly by now needing to cope with throwing away unreadable cache data. Ironically, if we d just gone ahead and taken the nominally riskier path of upgrading both servers at the same time, we might never have had a problem here. Intermittent bzr failures Finally, after we upgraded one of our two Bazaar codehosting servers to Python 3, we had a report of intermittent bzr branch hangs. After some digging I found this in our logs: I d seen this before in our git hosting service: it was a bug in Twisted s Python 3 port, fixed after 20.3.0 but unfortunately after the last release that supported Python 2, so we had to backport that patch. Using the same backport dealt with this. Onwards!

Review: Piranesi, by Susanna Clarke

Publisher:	Bloomsbury Publishing
Copyright:	2020
ISBN:	1-63557-564-8
Format:	Kindle
Pages:	245

Piranesi is a story told in first-person journal entries by someone who lives in a three-floored world of endless halls full of statues. The writing style is one of the most distinctive things about this book (and something you'll have to get along with to enjoy it), so it's worth quoting a longer passage from the introductory description of the world:

I am determined to explore as much of the World as I can in my lifetime. To this end I have travelled as far as the Nine-Hundred-and-Sixtieth Hall to the West, the Eight-Hundred-and-Ninetieth Hall to to the North and the Seven-Hundred-and-Sixty-Eighth Hall to the South. I have climbed up to the Upper Halls where Clouds move in slow procession and Statues appear suddenly out of the Mists. I have explored the Drowned Halls where the Dark Waters are carpeted with white water lilies. I have seen the Derelict Halls of the East where Ceilings, Floors sometimes even Walls! have collapsed and the dimness is split by shafts of grey Light. In all these places I have stood in Doorways and looked ahead. I have never seen any indication that the World was coming to an End, but only the regular progression of Halls and Passageways into the Far Distance. No Hall, no Vestibule, no Staircase, no Passage is without its Statues. In most Halls they cover all the available space, though here and there you will find an Empty Plinth, Niche or Apse, or even a blank space on a Wall otherwise encrusted with Statues. These Absences are as mysterious in their way as the Statues themselves.

So far as the protagonist knows, the world contains only one other living person, the Other, and thirteen dead ones who exist only as bones. The Other is a scientist searching for Great and Secret Knowledge, and calls the protagonist Piranesi, which is odd because that is not the protagonist's name. Be warned that I'm skating around spoilers for the rest of this review. I don't think I'm giving away anything that would ruin the book, but the nature of the story takes some sharp turns. If knowing anything about that would spoil the book for you and you want to read this without that knowledge, you may want to stop reading here. I also want to disclose early in this review that I wanted this to be a different book than it is, and that had a significant impact on how much I enjoyed it. Someone who came to it with different expectations may have a different and more enjoyable experience. I was engrossed by the strange world, the atmosphere, and the mystery of the halls full of statues. The protagonist is also interested in the same things, and the early part of the book is full of discussion of exploration, scientific investigation, and attempts to understand the nature of the world. That led me to hope for the sort of fantasy novel in which the setting is a character and where understanding the setting is a significant part of the plot. Piranesi is not that book. The story that Clarke wants to tell is centered on psychology rather than setting. The setting does not become a character, nor do we learn much about it by the end of the book. While we do learn how the protagonist came to be in this world, my first thought when that revelation starts halfway through the book was "this is going to be disappointing." And, indeed, it was. I say all of this because I think Piranesi looks, from both its synopsis and from the first few chapters, like it's going to be a world building and exploration fantasy. I think it runs a high risk of disappointing readers in the way that it disappointed me, and that can lead to disliking a book one may have enjoyed if one had read it in a different mood and with a different set of expectations. Piranesi is, instead, about how the protagonist constructs the world, about the effect of trauma on that construction, and about the complexities hidden behind the idea of recovery. And there is a lot to like here: The ending is complex and subtle and does not arrive at easy answers (although I also found it very sad), and although Clarke, by the end of the book, is using the setting primarily as metaphor, the descriptions remain vivid and immersive. I still want the book that I thought I was reading, but I want that book in large part because the fragments of that book that are in this one are so compelling and engrossing. What did not work for me was every character in the book except for the protagonist and one supporting character. The relationship between the protagonist and the Other early in the book is a lovely bit of unsettling complexity. It's obvious that the Other has a far different outlook on the world than the protagonist, but the protagonist seems unaware of it. It's also obvious that the Other is a bit of a jerk, but I was hoping for a twist that showed additional complexity in his character. Sadly, when we get the twist, it's not in the direction of more complexity. Instead, it leads to a highly irritating plot that is unnecessarily prolonged through the protagonist being gullible and child-like in the face of blatantly obvious gaslighting. This is a pattern for the rest of the book: Once villains appear on stage, they're one-note narcissists with essentially no depth. There is one character in Piranesi that I liked as well or better than the protagonist, but they only show up late in the story and get very little character development. Clarke sketches the outline of a character I wanted to learn much more about, but never gives us the details on the page. That leads to what I thought was too much telling rather than showing in the protagonist's relationships at the end of the book, which is part of why I thought the ending was so sad. What the protagonist loses is obvious to me (and lines up with the loss I felt when the book didn't turn out to be what I was hoping it would be); what the protagonist gains is less obvious, is working more on the metaphorical level of the story than the literal level, and is more narrated than shown. In other words, this is psychological fantasy with literary sensibilities told in a frame that looks like exploration fantasy. Parts of it, particularly the descriptions and the sense of place, are quite skillful, but the plot, once revealed, is superficial, obvious, and disappointing. I think it's possible this shift in the reader's sense of what type of book they're reading is intentional on Clarke's part, since it works with the metaphorical topic of the book. But it's not the existence of a shift itself that is my primary objection. I like psychological fantasy as well as exploration fantasy. It's that I thought the book after the shift was shallower, less interesting, and more predictable than the book before the shift. The one thing that is excellent throughout Piranesi, though, is the mood. It takes a bit to get used to the protagonist's writing style (and I continue to dislike the Affectation of capitalizing Nouns when writing in English), but it's open-hearted, curious, thoughtful, observant, and capable in a way I found delightful. Some of the events in this book are quite dark, but it never felt horrifying or oppressive because the protagonist remains so determinedly optimistic and upbeat, even when yanked around by the world's most obvious and blatant gaslighting. That persistent hopefulness and lightness is a good feature in a book published in 2020 and is what carried me through the parts of the story I didn't care for. I wish this had been a different book than it was, or failing that, a book with more complex and interesting supporting characters and plot to fit its complex and interesting psychological arc. I also wish that Clarke had done something more interesting with gender in this novel; it felt like she was setting that up for much of the book, and then it never happened. Ah well. As is, I can't recommend Piranesi, but I can say the protagonist, atmosphere, and sense of place are very well done and I think it will work for some other readers better than it did for me. Rating: 6 out of 10

Via my work on the media-types package, I wanted to know which packages were using the media type application/x-xcf, which apparently is not correct (#991158). The https://codesearch.debian.net site gives the answer. (Thanks!) Moreover, one can create a user key, for command-line remote access; here is an example below (the file dcs-apikeyHeader-plessy.txt contains x-dcs-apikey: followed by my access key).

curl -X GET "https://codesearch.debian.net/api/v1/searchperpackage?query=application/x-xcf&match_mode=literal" -H @dcs-apikeyHeader-plessy.txt > result.json

The result is serialised in JSON. Here is how I transformed it to make a list of email addresses that I could easily paste in mutt.

cat result.json  
  jq --raw-output '.[]."package"'  
  dd-list --stdin  
  sed -e '/^ /d' -e '/^$/'d -e 's/$/,/' -e 's/^/  /'

I just got a new Kobo ebook reader, a Kobo Clara HD. It's pretty similar to the Glo HD I had but which has unfortunately died after 5 years, even after trying to replace the battery.

Quick hardware review This is a neat little device. It's very similar to the Glo HD, which is a bit disappointing: you'd think they would have improved on the design in the 5+ years since the Glo HD has come out.. It does have an "amber" night light which is nice, but the bezel is still not level with the display, and the device is still kind of on the thick side. A USB-C (instead of micro-USB) port would have been nice too. But otherwise, it's pretty slick, and just works. And because the hardware design didn't change, I can still hack at it like a madman, which is really why I bought this thing in the first place. Hopefully it will last longer than 5 years. Ebook readers should really last for decades, not years, but I guess that's too much to expect from our consumerist, suicidal, extinctionist society.

Configuration hacks Here are the hacks I done on the device. I had done many more hacks on the Kobo Glo HD, but I decided to take a more streamlined, minimalist and, hopefully, easier for new users than the pile of hacks I was doing before (which I expand on at the end of the article).

SD card replacement I replaced the SD card. The original card shipped with the Clara HD was 8GB which meant all my books actually fitted on the original, but just barely. The new card is 16GB. Unfortunately, I did this procedure almost at the end of this guide (right before writing the syncthing scripts, below). Next time, that should be the first thing done so the original SD card acts as a pristine copy of the upstream firmware. So even though this seems like an invasive and difficult procedure, I actually do recommend you do it first. The process is basically to:

1. crack open the Kobo case (don't worry, it sounds awful but I've done it often)
2. take the SD card out
3. copy it over to a new, larger card (say on your computer)
4. put the larger card in

This guide has all the details.

Registration bypass hack This guide (from the same author!) has this awesome trick to bypass the annoying registration step. Basically:

1. pretend you do not have wifi
2. mount the device
3. sqlite3 /media/.../KOBOeReader/.kobo/KoboReader.sqlite
4. INSERT INTO user(UserID,UserKey) VALUES('1','');
5. unmount the device

More details in the above guide, again.

Install koreader My e-reader of choise is Koreader. It's just that great. I still don't find the general user interface (ie. the "file browswer") as intuitive as the builtin one, but the book reading just feels better. And anyways it's the easier way to get a shell on the device. Follow those instructions, particularly the NickelMenu instructions (see also the NickelMenu home page). Yes, you need to install some other thing to start koreader, which doesn't start on its own. NickelMenu is the simplest and better integrated I have found. You might also want to install some dictionnaries and configure SSH:

1. mount USB
2. drop your SSH public key in .../KOBOeReader/.adds/koreader/settings/SSH/authorized_keys
3. unmount USB
4. enable SSH in koreader (Gear -> Network -> SSH -> start SSH)

Note that ed25519 keys do not work: try an RSA key. This might be because koreader ships with dropbear (or an older version), but I haven't verified this.

Install syncthing I use Syncthing to copy all my books into the device now. I was previously using Koreader's OPDS support with Calibre's web interface, but that was clunky and annoying, and I'd constantly have to copy books around. Now the entire collection is synchronized. As a bonus, I can actually synchronise (and backup!) the koreader metadata, since it's stored next to the files. So in theory, this means I could use koreader from multiple devices and have my reading progress sync'd, but I haven't tested that feature just yet. I chose Syncthing because it's simple, lightweight, supported on Linux and Android, and statically compiles by default which means it's easy to deploy on the Kobo. Here is how I installed and started Syncthing at first:

1. Download the latest version for ARM
2. extract the archive
3. copy the syncthing binary into .../KOBOeReader/.adds/
4. login over SSH (see above on how to enable) with -p 2222 -l root
5. create the following directory: ~/.config/syncthing/
6. create the following configuration file, named config.xml:

   <configuration version="18">
       <gui enabled="true" tls="false" debugging="false">
           <address>0.0.0.0:8384</address>
       </gui>
   </configuration>
   

7. copy a valid ca-certificates.crt file (say from your Linux desktop) into /etc/ssl/certs/ on the Kobo (otherwise syncthing cannot bootstrap discovery servers)
8. launch syncthing over SSH: /mnt/onboard/.adds/syncthing

You should now be able to connect to the syncthing GUI through your web browser. Immediately change the GUI admin user and password on the Settings: GUI tab. Then, figure out how to start it. Here are your options:

1. on boot (inittab or whatever). downside: power usage.
2. on wifi (udev hacks). downside: unreliable (see wallabako).
3. on demand (e.g. nickel menu, koreader terminal shortcuts). downside: kind of clunky in koreader, did not work in nickel menu.
4. manually, through shell. downside: requires a shell, but then again we already have one through koreader?

What I have done is to write trivial shell scripts (in .../KOBOeReader/scripts) to start syncthing. The first is syncthing-start.sh:

#!/bin/sh
/mnt/onboard/.adds/syncthing serve &

Then syncthing-stop.sh:

#!/bin/sh
/usr/bin/pkill syncthing

This makes those scripts usable from the koreader file browser. Then the folder can be added to the folder shortcuts and a long-hold on the script will allow you to execute it. Still have to figure out why the Nickel Menu script is not working, but it could simply reuse the above to simplify debugging. This is the script I ended up with, in .../KOBOeReader/.adds/nm/syncthing:

menu_item :main    :Syncthing (toggle)    :cmd_spawn         :exec /mnt/onboard/scripts/syncthing-stop.sh
  chain_success:skip:4
    chain_success                      :cmd_spawn          :exec /mnt/onboard/scripts/syncthing-start.sh
    chain_success                      :dbg_toast          :Started Syncthing server
    chain_failure                      :dbg_toast          :Error starting Syncthing server
    chain_always:skip:-1
  chain_success                        :dbg_toast          :Stopped Syncthing server
menu_item :main    :Syncthing (start)    :cmd_output         :exec /mnt/onboard/scripts/syncthing-start.sh
menu_item :main    :Syncthing (stop)    :cmd_output         :exec /mnt/onboard/scripts/syncthing-stop.sh

It's unclear why this doesn't work: I only get "Error starting Syncthing server" for the toggle, and no output for the (start) action. In either case, syncthing doesn't actually start.

Avoided tasks This list wouldn't be complete without listing more explicitly the stuff I have done before on the Kobo Glo HD and which I have deliberately decided not to do here because my time is precious:

• plato install: beautiful project, but koreader is good enough
• wallabako setup: too much work to maintain, Wallabag articles are too distracting and available on my phone anyways
• using calibre to transfer books: not working half the time, different file layout than the source, one less Calibre dependency
• using calibre to generate e-books based on RSS feeds (yes, I did that, and yes, it was pretty bad and almost useless)
• SSH support: builtin to koreader

Now maybe I'll have time to actually read a book...

I'm creating a program that uses the web browser for its user interface, and I'm reasonably sure I'm not the first person doing this. Normally such a problem would listen to a port on localhost, and tell the browser to connect to it. Bonus points for listening to a randomly allocated free port, so that one does not need to involve some amount of luck to get the program started. However, using a local port still means that any user on the local machine can connect to it, which is generally a security issue. A possible solution would be to use AF_UNIX Unix Domain Sockets, which are supported by various web servers, but as far as I understand not currently by browsers. I checked Firefox and Chrome, and they currently seem to fail to even acknowledge the use case. I'm reasonably sure I'm not the first person doing this, and yes, it's intended as an understatement. So, dear Lazyweb, is there a way to securely use a browser as a UI for a user's program, without exposing access to the backend to other users in the system? Access token in the URL Emanuele Di Giacomo suggests to add an access token to the URL that gets passed to the browser. This would work to protect access on localhost: even if the application cannot use HTTPS, other users cannot see packets that go through the local interface, so both the access token and the session cookie that one could send afterwards would be protected. Network namespaces I thought about isolating server and browser in a private network namespace with something like unshare(1), but it seems to require root. Johannes Schauer Marin Rodrigues wrote to correct that:

It's possible to unshare the network namespace by first unsharing the user namespace and thus becoming root which is possible without being root since #898446 got fixed. For example you can run this as the normal user: lxc-usernsexec -- lxc-unshare -s NETWORK -- ip addr If you don't want to depend on lxc, you can write a wrapper in Perl or Python. I have a Perl implementation of that in mmdebstrap.

Firewalling Martin Schuster wrote to suggest another option:

I had the same issue. My approach was "weird", but worked: Block /outgoing/ connections to the port, unless the uid is correct. That might be counter-intuitive, but of course all connections /to/ localhost will be done /from/ localhost also. Something like: iptables -A OUTPUT -p tcp -d localhost --dport 8123 -m owner --uid-owner joe -j ACCEPT iptables -A OUTPUT -p tcp -d localhost --dport 8123 -j REJECT

BBI Kenya and live Supreme Court streaming on YT The last few weeks have been unrelenting as all sorts of news have been coming in, mostly about the downturn in the Economy, Islamophobia in India on the rise, Covid, and electioneering. However, in the last few days, Kenya surpassed India in live-streaming proceeds in a Court of Appeals about BBI or Building Bridges Initiative. A background filler article on the topic can be found in BBC. The live-streaming was done via YT and if wants to they can start from

https://www.youtube.com/watch?v=JIQzpmVKvro One can also subscribe to K24TV which took the initiative of sharing the proceedings with people worldwide. If K24TV continues to share SC proceedings of Kenya, that would add to the soft power of Kenya. I will not go into the details of the case as Gautam Bhatia who has been following the goings-on in Kenya is a far better authority on the subject. In fact, just recently he shared about another Kenyan judgment from a trial which can be seen here. He has shared the proceedings and some hot takes on the Twitter thread started by him. Probably after a couple of weeks or more when he has processed what all has happened there, he may also share some nuances although many of his thoughts would probably go to his book on Comparative Constitutional Law which he hopes to publish maybe in 2021/2022 or whenever he can. Such televised proceedings are sure to alleviate the standing of Kenya internationally. There has been a proposal to do similar broadcasts by India but with surveillance built-in, so they know who is watching. The problems with the architecture and the surveillance built-in have been shared by Srinivas Kodali or DigitalDutta quite a few times, but that probably is a story for another day.

Uttar Pradesh Population Control Bill

Hindus comprise 83% of Indian couples with more than two child children

The U.P. Population Bill came and it came with lot of prejudices. One of the prejudices is the idea that Muslims create or procreate to have the most children. Even with data is presented as shared above from NFHS National Family Health Survey which is supposed to carry our surveys every few years did the last one around 4 years back. The analysis from it has been instrumental not only in preparing graphs as above but also sharing about what sort of death toll must have been in rural India. And as somebody who have had the opportunity in the past, can vouch that you need to be extremely lucky if something happens to you when you are in a rural area. Even in places like Bodh Gaya (have been there) where millions of tourists come as it is one of the places not to be missed on the Buddhism tourist circuit, the medical facilities are pretty underwhelming. I am not citing it simply because there are too many such newspaper reports from even before the pandemic, and both the State and the Central Govt. response has been dismal. Just a few months back, they were recalled. There were reports of votes being bought at INR 1000/- (around $14) and a bottle or two of liquor. There used to be a time when election monitoring whether national or state used to be a thing, and you had LTO s (Long-time Observers) and STO s (Short-Term Observers) to make sure that the election has been neutral. This has been on the decline in this regime, but that probably is for another time altogether. Although, have to point out the article which I had shared a few months ago on the private healthcare model is flawed especially for rural areas. Instead of going for cheap, telemedicine centers that run some version of a Linux distro. And can provide a variety of services, I know Kerala and Tamil Nadu from South India have experimented in past but such engagements need to be scaled up. This probably will come to know when the next time I visit those places (sadly due to the virus, not anytime soonish.:( ) . Going back to the original topic, though, I had shared Hans Rosling s famous Ted talk on population growth which shows that even countries which we would not normally associate with family planning for e.g. the middle-east and Africa have also been falling quite rapidly. Of course, when people have deeply held prejudices, then it is difficult. Even when sharing China as to how they had to let go of their old policy in 2016 as they had the thing for leftover men . I also shared the powerful movie So Long my Son. I even shared how in Haryana women were and are trafficked and have been an issue for centuries but as neither suits the RW propaganda, they simply refuse to engage. They are more repulsed by people who publish this news rather than those who are actually practicing it, as that is culture . There is also teenage pregnancy, female infanticide, sex-selective abortion, etc., etc. It is just all too horrible to contemplate. Personal anecdote I know a couple, or they used to be a couple, where the gentleman wanted to have a male child. It was only after they got an autistic child, they got their DNA tested and came to know that the gentleman had a genetic problem. He again forced and had another child, and that too turned out to be autistic. Finally, he left the wife and the children, divorced them and lived with another woman. Almost a decade of the wife s life was ruined. The wife before marriage was a gifted programmer employed at IBM. This was an arranged marriage. After this, if you are thinking of marrying, apart from doing astrology charts, also look up DNA compatibility charts. Far better than ruining yours or the women s life. Both the children whom I loved are now in heaven, god bless them If one wants to, one can read a bit more about the Uttar Pradesh Population bill here. The sad part is that the systems which need fixing, nobody wants to fix. The reason being simple. If you get good health service by public sector, who will go to the private sector. In Europe, AFAIK they have the best medical bang for the money. Even the U.S. looks at Europe and hopes it had the systems that Europe has but that again is probably for another day.

South Africa and India long-lost brothers. As had shared before, after the 2016 South African Debconf convention, I had been following South Africa. I was happy when FeesMustFall worked and in 2017 the then ANC president Zuma declared it in late 2017. I am sure that people who have been regular visitors to this blog know how my position is on student loans. They also must be knowing that even in U.S. till the 1970s it had free education all the way to be a lawyer and getting a lawyer license. It is only when people like Thurgood Marshall, Martin Luther King Jr., and others from the civil rights movement came out as a major force that the capitalists started imposing fees. They wanted people who could be sold to corporate slavery, and they won. Just last week, Biden took some steps and canceled student loans and is working on steps towards broad debt forgiveness. Interestingly, NASA has an affirmative diversity program for people from diverse backgrounds, where a couple of UC (Upper Caste) women got the job. While they got the job, the RW (Right-Wing) was overjoyed as they got jobs on merit . Later, it was found that both the women were the third or fourth generation of immigrants in U.S.

NASA Federal Equal Opportunity Policy Directive NPD 3713 2H

Going back to the original question and topic, while there has been a concerning spate of violence, some calling it the worst sort of violence not witnessed since 1994. The problem, as ascertained in that article, is the same as here in India or elsewhere. Those, again, who have been on my blog know that merit 90% of the time is a function of privilege and there is a vast amount of academic literature which supports that. If, for a moment, you look at the data that is shared in the graph above which shows that 83% of Hindus and 13% of Muslims have more than 2 children, what does it show, it shows that 83+13 = 96% of the population is living in insecurity. The 5% are the ones who have actually consolidated more power during this regime rule in India. Similarly, from what I understood living in Cape Town for about a month, it is the Dutch Afrikaans as they like to call themselves and the immigrants who come from abroad who have enjoyed the fruits of tourism and money and power while the rest of the country is dying due to poverty. It is the same there, it is the same here. Corruption is also rampant in both countries, and the judiciary is virtually absent from both communities in India and SA. Interestingly, South Africa and India have been at loggerheads, but I suspect that is more due to the money and lobbying power by the Dutch. Usually, those who have money power, do get laws and even press on their side, and it is usually the ruling party in power. I cannot help but share about the Gupta brothers and their corruption as I came to know about it in 2016. And as have shared that I m related to Gupta s on my mother s side, not those specific ones but Gupta as a clan. The history of the Gupta dynasty does go back to the 3rd-4th century. Equally interesting have been Sonali Ranade s series of articles which she wrote in National Herald, the latest on exports which is actually the key to taking India out of poverty rather than anything else. While in other countries Exporters are given all sort of subsidies, here it is being worked as how to give them less. This was in Economic times hardly a week back

Export incentive schemes being reduced

I can t imagine the incredible stupidity done by the Finance Minister. And then in an attempt to prove that, they will attempt to present a rosy picture with numbers that have nothing to do with reality. Interestingly enough, India at one time was a major exporter of apples, especially from Kashmir. Now instead of exporting, we are importing them from Afghanistan as well as Belgium and now even from the UK. Those who might not want to use the Twitter link could use this article. Of course, what India got out of this trade deal is not known. One can see that the UK got the better deal from this. Instead of investing in our own capacity expansion, we are investing in increasing the capacity of others. This is at the time when due to fuel price hike (Central taxes 66%) demand is completely flat. And this is when our own CEA (Chief Economic Adviser) tells us that growth will be at the most 6-7% and that too in 2023-2024 while currently, the inflation rate is around 12%. Is it then any wonder that almost 70% are living on Govt. ration and people in the streets of Kolkata, Assam, and other places have to sell kidneys to make sure they have some money for their kids for tomorrow. Now I have nothing against the UK but trade negotiation is an art. Sadly, this has been going on for the last few years. The politicians in India fool the public by always telling of future trade deals. Sadly, as any businessman knows, once you have compromised, you always have to compromise. And the more you compromise, the more you weaken the hand for any future trade deals.

IIT pupil tries to sell kidney to repay loan, but no takers for Dalit organ.

The above was from yesterday s Times of India. Just goes to show how much people are suffering. There have been reports in vernacular papers of quite a few people from across regions and communities are doing this so they can live without pain a bit. Almost all the time, the politicians are saved as only few understand international trade, the diplomacy and the surrounding geopolitics around it. And this sadly, is as much to do with basic education as much as it is to any other factor

Suli Deals About a month back on the holy day of Ramzan or Ramadan as it is known in the west, which is beloved by Muslims, a couple of Muslim women were targeted and virtually auctioned. Soon, there was a flood and a GitHub repository was created where hundreds of Muslim women, especially those who have a voice and fearlessly talk about their understanding about issues and things, were being virtually auctioned. One week after the FIR was put up, to date none of the people mentioned in the FIR have been arrested. In fact, just yesterday, there was an open letter which was published by livelaw. I have saved a copy on WordPress just in case something does go wrong. Other than the disgust we feel, can t say much as no action being taken by GOI and police.

IT Rules 2021 and Big Media After almost a year of sleeping when most activists were screaming hoarsely about how the new IT rules are dangerous for one and all, big media finally woke up a few weeks back and listed a writ petition in Madras High Court of the same. Although to be frank, the real writ petition was filed In February 2021, classical singer, performer T.M. Krishna in Madras High Court. Again, a copy of the writ petition, I have hosted on WordPress. On 23rd June 2021, a group of 13 media outlets and a journalist have challenged the IT Rules, 2021. The Contention came from Digital News Publishers Association which is made up of the following news companies: ABP Network Private Limited, Amar Ujala Limited, DB Corp Limited, Express Network Pvt Ltd, HT Digital Streams Limited, IE Online Media Services Pvt Ltd, Jagran Prakashan Limited, Lokmat Media Private Limited, NDTV Convergence Limited, TV Today Network Limited, The Malayala Manorama Co (P) Ltd, Times Internet Limited, and Ushodaya Enterprises Private Limited. All the above are heavyweights in the markets where they operate. The reason being simple, when these media organizations came into being, the idea was to have self-regulation, which by and large has worked. Now, the present Govt. wants each news item to be okayed by them before publication. This is nothing but blatant misuse of power and an attempt at censorship. In fact, the Tamil Nadu BJP president himself made a promise of the same. And of course, what is true and what is a lie, only GOI knows and will decide for the rest of the country. If somebody remembers Joseph Goebbels at this stage, it is merely a coincidence. Anyways, 3 days ago Supreme Court on 14th July the Honorable Supreme Court asked the Madras High Court to transfer all the petitions to SC. This, the Madras High Court denied as cited/shared by Meera Emmanuel, a reporter who works with barandbench. The Court says nothing doing, let this happen and then the SC can entertain the motion of doing it that level. At the same time, they would have the benefit of Madras High Court opinion as well. It gave the center two weeks to file a reply. So, either of end-week of July or latest by August first week, we might be able to read the Center s reply on the same. The SC could do a forceful intervention, but it would lead to similar outrage as has been witnessed in the past when a judge commented that if the SC has to do it all, then why do we need the High Courts, district courts etc. let all the solutions come from SC itself. This was, admittedly, frustration on the part of the judge, but due in part to the needless intervention of SC time and time again. But the concerns had been felt around all the different courts in the country.

Sedition Law A couple of days ago, the Supreme Court under the guidance of Honorable CJI NV Ramanna, entertained the PIL filed by Maj Gen S G Vombatkere (Retd.) which asked simply that the sedition law which was used in the colonial times by the British to quell dissent by Mahatma Gandhi and Bal Gangadhar Tilak during the Indian freedom struggle. A good background filler article can be found on MSN which tells about some recent cases but more importantly how historically the sedition law was used to quell dissent during India s Independence. Another article on MSN actually elaborates on the PIL filed by Maj Gen S. G. Vombatkere. Another article on MSN tells how sedition law has been challenged and changed in 10 odd countries. I find it equally sad and equally hilarious that the Indian media whose job is to share news and opinion on this topic is being instead of being shared more by MSN. Although, I would be bereft of my duty if I did not share the editorial on the same topic by the Hindu and Deccan Chronicle. Also, an interesting question to ask is, are there only 10 countries in the world that have sedition laws? AFAIK, there are roughly 200 odd countries as recognized by WTO. If 190 odd countries do not have sedition laws, it also tells a lot about them and a lot about the remaining 10. Also, it came to light that police are still filing laws under sec66A which was declared null and void a few years ago. It was replaced with section 124A if memory serves right and it has more checks and balances.

Danish Siddiqui, Pulitzer award-winning and death in Afghanistan Before I start with Danish Siddiqui, let me share an anecdote that I think I have shared on the blog years ago about how photojournalists are. Again, those who know me and those who follow me know how much I am mad both about trains and planes (civil aviation). A few months back, I had shared a blog post about some of the biggest railway systems in the world which shows that privatization of Railways doesn t necessarily lead to up-gradation of services but definitely leads to an increase in tariff/fares. Just had a conversation couple of days ago on Twitter and realized that need to also put a blog post about civil aviation in India and the problems it faces, but I digress. This was about a gentleman who wanted to take a photo of a particular train coming out of a valley at a certain tunnel at two different heights, one from below and one from above the train. This was several years ago, and while I did share that award-winning photograph then, it probably would take me quite a bit of time and effort to again look it up on my blog and share. The logistics though were far more interesting and intricate than I had first even thought of. We came around a couple of days before the train was supposed to pass that tunnel and the valley. More than half a dozen or maybe more shots were taken throughout the day by the cameras. The idea was to see how much light was being captured by the cameras and how much exposure was to be given so that the picture isn t whitened out or is too black. Weather is the strangest of foes for a photojournalist or even photographers, and the more you are in nature, the more unpredictable it is and can be. We were also at a certain height, so care had to be taken in case light rainfall happens or dew falls, both not good for digital cameras. And dew is something which will happen regardless of what you want. So while the two days our gentleman cameraman fiddled with the settings to figure out correct exposure settings, we had one other gentleman who was supposed to take the train from an earlier station and apprise us if the train was late or not. The most ideal time would be at 0600 hrs. When the train would enter the tunnel and come out and the mixture of early morning sun rays, dew, the flowers in the valley, and the train would give a beautiful effect. We could stretch it to maybe 0700 hrs. Anything after that would just be useless, as it wouldn t have the same effect. And of all this depended on nature. If the skies were to remain too dark, nothing we could do about it, if the dewdrops didn t fall it would all be over. On the day of the shoot, we were told by our compatriot that the train was late by half an hour. We sank a little on hearing that news. Although Photoshop and others can do touch-ups, most professionals like to take as authentic a snap as possible. Everything had been set up to perfection. The wide-angle lenses on both the cameras with protections were set up. The tension you could cut with a knife. While we had a light breakfast, I took a bit more and went in the woods to shit and basically not be there. This was too tensed up for me. Returned an hour to find everybody in a good mood. Apparently, the shoot went well. One of the two captured it for good enough. Now, this is and was in a benign environment where the only foe was the environment. A bad shot would have meant another week in the valley, something which I was not looking forward to. Those who have lived with photographers and photojournalists know how self-involved they can be in their craft, while how grumpy they can be if they had a bad shoot. For those, who don t know, it is challenging to be friends with such people for a long time. I wish they would scream more at nature and let out the frustrations they have after a bad shoot. But again, this is in a very safe environment. Now let s cut to Danish Siddiqui and the kind of photojournalism he followed. He followed a much more riskier sort of photojournalism than the one described above. Krittivas Mukherjee in his Twitter thread shared how reporters in most advanced countries are trained in multiple areas, from risk assessment to how to behave in case you are kidnapped, are in riots, hostage situations, etc. They are also trained in all sorts of medical training from treating gunshot wounds, CPR, and other survival methods. They are supposed to carry medical equipment along with their photography equipment. Sadly, these concepts are unknown in India. And even then they get killed. Sadly, he attributes his death to the thrill of taking an exclusive photograph. And the gentleman s bio reads that he is a diplomat. Talk about tone-deafness On another completely different level was Karen Hao who was full of empathy as she shared the humility, grace, warmth and kinship she describes in her interaction with the photojournalist. His body of work can be seen via his ted talk in 2020 where he shared a brief collage of his works. Latest, though in a turnaround, the Taliban have claimed no involvement in the death of photojournalist Danish Siddiqui. This could be in part to show the Taliban in a more favorable light as they do and would want to be showcased as progressive, even though they are forcing that all women within a certain age become concubines or marry the fighters and killing the minority Hazaras or doing vile deeds with them. Meanwhile, statements made by Hillary Clinton almost a decade, 12 years ago have come back into circulation which stated how the U.S. itself created the Taliban to thwart the Soviet Union and once that job was finished, forgot all about it. And then in 2001, it landed back in Afghanistan while the real terrorists were Saudi. To date, not all documents of 9/11 are in the public domain. One can find more information of the same here. This is gonna take probably another few years before Saudi Arabia s whole role in the September 11 attacks will be known. Last but not the least, came to know about the Pegasus spyware and how many prominent people in some nations were targeted, including in mine India. Will not talk more as it s already a big blog post and Pegasus revelations need an article on its own.

It seems I m the only person on the Internet who didn t know sending email to Google with bit.ly links will tank your deliverability. To my credit, I ve been answering deliverability support questions for 16 years and this has never come up. Until last week. For some reason, at May First we suddenly had about three percent of our email to Google deferred with the ominous sounding:

Our system has detected that this message is 421-4.7.0 suspicious due to the nature of the content and/or the links within.

The quantity of email that accounts for just three percent of mail to Google is high, and caused all kinds of monitoring alarms to go off, putting us into a bit of panic. Eventually we realized all but one of the email messages had bit.ly links. I m still not sure whether this issue was caused by a weird and coincidental spike in users sending bit.ly links to Google. Or whether some subtle change in the Google algorithm is responsible. Or some change in our IP address reputation placed greater emphasis on bit.ly links. In the end it doesn t really matter - the real point is that until we disrupt this growing monopoly we will all be at the mercy of Google and their algorithms for email deliverability (and much, much more).

Like each month, have a look at the work funded by Freexian s Debian LTS offering. Debian project funding In June, we put aside 5775 EUR to fund Debian projects for which we re looking forward to receive more projects from various
Debian teams! Learn more about the rationale behind this initiative in this article. Debian LTS contributors In June, 12 contributors have been paid to work on Debian LTS, their reports are available:

• Abhijith PA did 18.0h (out of 14h assigned and 19h from May), thus carrying over 15h to July.
• Anton Gladky did 12h (out of 12h assigned).
• Ben Hutchings did 13.25h (out of 14h assigned and 2h from May), thus carrying over 2.75h to July.
• Chris Lamb did 18h (out of 18h assigned).
• Emilio Pozuelo Monfort did 29h (out of 40h assigned), thus carrying over 11h to July.
• Holger Levsen s work was coordinating/managing the LTS team, he did 3.5h (out of 12h assigned) and gave back 8.5h to the pool.
• Markus Koschany did 29.75h (out of 30h assigned plus 29.75h from May), thus carrying over 30h for July.
• Ola Lundqvist did 10h (out of 12h assigned and 4.5h from May), thus carrying over 6.5h to July.
• Roberto C. S nchez did 12h (out of 32h assigned), thus carrying over 20h to July.
• Sylvain Beucler did 30h (out of 30h assigned).
• Thorsten Alteholz did 30h (out of 30h assigned).
• Utkarsh Gupta did not report back about their work so we assume they did nothing (out of 40h assigned), thus is carrying over 40h for July.

Evolution of the situation In June we released 30 DLAs. As already written last month we are looking for a Debian LTS project manager and team coordinator.
Finally, we would like to remark once again that we are constantly looking for new contributors. Please contact Holger if you are interested! The security tracker currently lists 41 packages with a known CVE and the dla-needed.txt file has 23 packages needing an update. Thanks to our sponsors Sponsors that joined recently are in bold.

• Platinum sponsors:
  • TOSHIBA (for 70 months)
  • GitHub (for 60 months)
  • Civil Infrastructure Platform (CIP) (for 38 months)
• Gold sponsors:
  • Roche Diagnostics International AG (for 81 months)
  • Linode (for 75 months)
  • Babiel GmbH (for 64 months)
  • Plat Home (for 63 months)
  • University of Oxford (for 20 months)
  • Deveryware (for 7 months)
  • VyOS
• Silver sponsors:
  • The Positive Internet Company (for 86 months)
  • Domeneshop AS (for 85 months)
  • Nantes M tropole (for 79 months)
  • Univention GmbH (for 71 months)
  • Universit Jean Monnet de St Etienne (for 71 months)
  • Ribbon Communications, Inc. (for 65 months)
  • Exonet B.V. (for 54 months)
  • Leibniz Rechenzentrum (for 48 months)
  • CINECA (for 38 months)
  • Minist re de l Europe et des Affaires trang res (for 32 months)
  • Cloudways Ltd (for 21 months)
  • Dinahosting SL (for 19 months)
  • Platform.sh (for 14 months)
  • Bauer Xcel Media Deutschland KG (for 13 months)
  • Moxa Intelligence Co., Ltd. (for 7 months)
  • sipgate GmbH (for 5 months)
  • OVH US LLC (for 3 months)
  • Tilburg University (for 3 months)
• Bronze sponsors:
  • Seznam.cz, a.s. (for 86 months)
  • Evolix (for 85 months)
  • Linuxhotel GmbH (for 83 months)
  • Intevation GmbH (for 82 months)
  • Daevel SARL (for 81 months)
  • Bitfolk LTD (for 80 months)
  • Megaspace Internet Services GmbH (for 80 months)
  • Greenbone Networks GmbH (for 79 months)
  • NUMLOG (for 79 months)
  • WinGo AG (for 78 months)
  • Ecole Centrale de Nantes LHEEA (for 75 months)
  • Entr ouvert (for 70 months)
  • Adfinis AG (for 67 months)
  • Laboratoire LEGI UMR 5519 / CNRS (for 62 months)
  • Tesorion (for 62 months)
  • GNI MEDIA (for 61 months)
  • Bearstech (for 53 months)
  • LiHAS (for 53 months)
  • People Doc (for 49 months)
  • Catalyst IT Ltd (for 47 months)
  • Demarcq SAS (for 41 months)
  • Universit Grenoble Alpes (for 28 months)
  • TouchWeb SAS (for 20 months)
  • SPiN AG (for 16 months)
  • CoreFiling (for 12 months)
  • Institut des sciences cognitives Marc Jeannerod (for 7 months)
  • Observatoire des Sciences de l Univers de Grenoble (for 3 months)

Installing a new laptop New is a 2nd hand Thinkpad T470p laptop that I intend to duel boot with windows.
I have been a Debian user for over 20 years, I use windows at work for the proprietary EDA Altium , but I have never had a windows installation on my laptop. This machine will to be different it is the first laptop that I have owned that has sufficient GPU to realistically run Altium.. I will try it in a VM later (if that works it will be my preferred choice), but for now I want to try a duel boot system. So where to start? Step one Debian wiki https://wiki.debian.org/DimentionedDualBoot/Windows My laptop was purchased from a dealer / refurbisher. This means that they had confirmed that the hardware was functional, wiped it down and then installed a clean copy of Windows on the whole system. What it doesn t mean is that the system was set for UEFI boot and that the EFI partition is set correctly . I turned on UEFI and made sure that Legacy BIOS mode was disabled. Next I re-installed Windows, making sure to leave enough disk space for may later Debian install. (if you already have UEFI / secure boot enabled then you could skip the reinstall and instead re-size your disk) Eeew! Windows now wants to show me adverts, it doesn t give me the option to never show me ads, but at least I could insist that it doesn t display tailored ads based on the obvious snooping of my web browsing habits just another reason to use Debian. Now to install Debian I want an encrypted file system, and because I want to dual boot I can t just follow the guided installation in the Debian installer. So I shall detail what I did here. Indeed I took several attempts at this and eventually asked for help as I had still messed up (I thought I was doing it correctly but had missed out a step) First the boiler plate DI

• Download your prefered Debian installation media (I am using Bullseye AMD64 netinst beta), and drop this directly onto a USB memory stick (dd)
• Put the USB stick in the laptop and select this as the boot device (on my thinkpad the boot device menu is F12)
• I chose the graphical installation option, but only because it was less key strokes to select
• Select your prefered Locale
  • UI language (English)
  • Enter your location (United Kingdom)
  • and keyboard layout (British English)
• Next DI comes up with a whole host of missing firmware for the detected WiFi I can safely ignore this as I have a network cable plugged in (select No). If I want to enable WiFi I could choose to add media with the firmware at this stage or add it later.
  • I have a network cable plugged in and DI finds and configures my network setup (IPv6 and v4 with DHCP)
• I enter a hostname (I chose to name my machines after lizards this will be called skink)
• I am asked for a domain name (I have koipond.org.uk configured)
• You are then asked for some account details
  • I do not enter a root password as I want the root account login disabled
  • But I do provide my details for a user account

Now for the interesting bit Partitioning the disk(s) Select MANUAL disk partitioning I have the following partitions: /dev/nvmen0p1
1.0MB FREE SPACE
#1 536.9 MB B K ESP
400.0 GB FREE SPACE
#3 16.8 MB Microsoft reserved partition
#4 111.6 GB ntfs Basic data partition
335.4 kB FREE SPACE

• Create an partition for /boot
  • Select the 400GB free space
  • Create a new partition
  • Enter enough space of /boot (>100MB I select 500 MB)
  • place this at the beginning of the disk
  • Name it (boot)
  • Use as ext2 we don t want journaling here
  • Mount point /boot

• Set up encrypted volumes
  • We need to write the new partition table to disk before we can continue
  • Create encrypted volumes
    • select the large remaining area of free space
    • name it (skink)
    • write disk configuration
    • finish
    • let the system overwrite the partition with random
    • enter a passphrase for the disk

• Set up LVM (inside the encrypted volume)
  • Select Configure Logical Volume Manager
  • Write changes to disk (we do this a lot)
  • Create volume group
    • Give it a name (VG-Skink)
    • Select the encrypted partition
  • Create logical volume (swap)
    • Select the volume group to use (VG-Skink)
    • Enter a name (LV-Swap)
    • Enter size of swap (32G)
  • Create logical volume (system)
    • Select the volume group to use (VG-Skink)
    • Enter a name (LV-System)
    • Enter size of swap (remaining space)
  • Finish

Set use

• Select your LVM VG for swap
  • Use as: Swap area
  • Done Setting up partition
• Select your LVM VG for system
  • Use as: Ext4 journaling file system
  • Mount point: / the root filing system
  • Mount options: I select discard (trim function as this makes a considerable improvement to the disk performance and life)

I now have the following partitions: LVM VG VG-Skink
#1 32 GB f swap swap
LVM VG VG-System
#1 367.5 GB f ext4 /
Encrypted volume
#1 399.5 GB K lvm
/dev/nvmen0p1
1.0MB FREE SPACE
#1 536.9 MB B K ESP
#2 500.2 MB F ext2 /boot
#5 399.2 GB K crypto skink
#3 16.8 MB Microsoft reserved partition
#4 111.6 GB ntfs Basic data partition
335.4 kB FREE SPACE

• Finish partitioning and write changes to disk
  • Write the changes to disk

Boiler plate debian install continues The system will install a base system

• Configure package manager Select nearest mirror (I run a local mirror so select enter information manually)
• Yes I do want to take part in popcon (Debian uses this as a guide to how many instances of each package are installed I select this for anything other than test installs)
• Software Selection
  • I will have a desktop environment and I currently use KDE
  • I would like an ssh server to be installed
  • I want the standard system utilities

Sit back and wait a for the system to install Well that didn t take very long Damn this new laptop is quick. I suspect that is nvme solid state storage, no longer limited to SATA bus speeds (and even that wasn t slow)

Back in the days of Etch, I converted this blog from Drupal to ikiwiki. I remember being very excited about this brand new concept of static web sites derived from content stored in a version control system. And now over a decade later I ve moved to hugo. I feel some loyalty to ikiwiki and Joey Hess for opening my eyes to the static web site concept. But ultimately I grew tired of splitting my time and energy between learning ikiwiki and hugo, which has been my tool of choice for new projects. When I started getting strange emails that I suspect had something to do with spammers filling out ikiwiki s commenting registration system, I choose to invest my time in switching to hugo over debugging and really understanding how ikiwiki handles user registration. I carefully reviewed anarcat s blog on converting from ikiwiki to hugo and learned about a lot of ikiwiki features I am not using. Wow, it s times like these that I m glad I keep it really simple. Based on the various ikiwiki2hugo python scripts I studied, I eventually wrote a far simpler one tailored to my needs. Also, in what could only be called a desperate act of procrastination combined with a touch of self-hatred (it s been a rough week) I rejected all the commenting options available to me and choose to implement my own in PHP. What?!?! Why would anyone do such a thing? I refer you to my previous sentence about desperate procrastination. And also I know it s fashionable to hate PHP, but honestly as the first programming language I learned, there is something comforting and familiar about it. And, on a more objective level, I can deploy it easily to just about any hosting provider in the world. I don t have to maintain a unicorn service or a nodejs service and make special configuration entries in my web configuration. All I have to do is upload the php files and I m done. Well, I m sure I ll regret this decision. Special thanks to Alexander Bilz for the anatole hugo theme. I choose it via a nearly random click to avoid the rabbit hole of choosing a theme. And, by luck, it has turned out quite well. I only had to override the commento partial theme page to hijack it for my own commenting system s use.

Hello there. I still can t believe that the first half of GSoC period is almost over. So it s been about 5 weeks working on the project and that means I have a lot to share about it. So without further ado, let s get started. I will be listing up my work done in the respective tasks.

Task: Migrating Logins to Salsa The objective of this task was that the users could log in to their account on debci using their Debian Salsa account (collaborative development server for Debian based on the GitLab software) and this is implemented with the help of OmniAuth, the ruby authentication framework. At the beginning of this, I had to discuss quite a few issues with my mentors that I was bumping into, and by the end of it with multiple revisions and discussions, the following was implemented:

• The previous users' table schema of debci comprises the username field which contained mostly the emails of the users with some exceptions and to accommodate the Salsa logins, a new uid field is added to the table to store the Salsa uid of the logged-in user with the username field storing Salsa usernames now and as the Salsa users have the liberty to change their usernames, the updation of username as well as in debci database is also taken care of.
• For Salsa login, the ruby-omniauth-gitlab strategy has been used and for login in development mode, the developer strategy which comes with ruby-omniauth has been set up.
• Added a Login Page giving the option to log in using Salsa and an additional option to login in Developer Mode which is accessible only in Development Setup so that other contributors don t have to set up dummy Salsa applications for working.
• Added specs for the new login process. This was an interesting part, as I got the chance to understand RSpec and facilities provided by OmniAuth to mock the authentication for Integration Testing.
• One blocker that I dealt with was that the Debian release from where packages were pulled out for debci have the OmniAuth version 1.8, which was not working well with the developer strategy implementation for the application so to resolve that I did a minor change to the callback API for developer strategy until the time that release have the newer version of OmniAuth.
• Another thing we discussed in one of the meetings that in the existing database structure, the tests do not have a real reference to the users' table and rather the username is stored directly as a string for the requestor field, so this thing was fixed as part of this task.

The migration of the existing users' data for the new logins was handled by my mentor Antonio Terceiro and with this, our first task is concluded. All these changes are now part of Debian Continuous Integration platform and you can find the blogpost for same by Antonio here. This task also allowed me to write my first ever tutorial Tutorial: Integrating OmniAuth with Sinatra Application to help people looking to integrate their ruby application with OmniAuth. Moving further to the next task in progress.

Task: Adding support for testing security uploads and Debian LTS This is the next task I am working on enabling private tests in debci for adding support for testing security uploads and Debian LTS. Since it s a bigger task, it is broken down into about 6-7 steps and till now, the following has been done:

• The schema of jobs' (tests) table is updated to have a boolean field to store whether the job is private or not.
• The is_private parameter is added to both API and Self-Service section so the private test can be submitted through the API as well as through GUI form on the web platform.
• Another thing which comes up through discussion in meetings that a parameter is required to add extra-apt-sources for getting packages of security repository and this is the part in progress.

So that concludes my work till now. It has been an amazing journey with lots of learning and also the guidance from the wonderful mentors of my project and I am looking forward to more exciting parts ahead. That s all for now. See you next time!

This is not only the Year of the Ox, but also the year of Debian 11, code-named bullseye. The release lies ahead, full freeze starts this week. A good opportunity to take a look at what is new in bullseye. In this post new programs and new software versions related to XMPP, also known as Jabber are presented. XMPP exists since 1999, and has a diverse and active developers community. It is a universal communication protocol, used for instant messaging, IoT, WebRTC, and social applications. You probably will encounter some oxen in this post.

• biboumi, XMPP gateway to connect to IRC servers: 8.3 9.0
  The biggest change for users is SASL support: A new field in the Configure ad-hoc command lets you set a password that will be used to authenticate to the nick service, instead of using the cumbersome NickServ method.
  Many more changes are listed in the changelog.
• Dino, modern XMPP client: 0.0.git20181129 0.2.0
  Dino in Debian 10 was practically a technology preview. In Debian 11 it is already a fully usable client, supporting OMEMO encryption, file upload, image preview, message correction and many more features in a clean and beautiful user interface.
• ejabberd, the extensible realtime platform: 18.12.1 21.01.
  Probably the most important improvement for end-users is XEP-0215 support to facilitate modern WebRTC-style audio/video calls. ejabberd also integrates more nicely with systemd (e.g., the watchdog feature if supported, now). Apart from that, a new configuration validator was introduced, which brings a more flexible (but mostly backwards-compatible) syntax. Also, error reporting in case of misconfiguration should be way more helpful, now. As a new authentication backend, JSON Web Tokens (JWT) can be used. In addition to the XMPP and SIP support, ejabberd now includes a full-blown MQTT server. A large number of smaller features has been added, performance was improved in many ways, and several bugs were fixed. See the long list of changes.
• Gajim, a GTK+-based Jabber client: 1.1.2 1.3.1
  The new Debian release brings many improvements. Gajim s network code has been completely rewritten, which leads to faster connections, better recovery from network loss, and less network related hiccups. Customizing Gajim is now easier than ever. Thanks to the new settings backend and a completely reworked Preferences window, you can adapt Gajim to your needs in just a few seconds.
  Good for newcomers: account creation is now a lot easier with Gajim s new assistant. The new Profile window gives you many options to tell people more about yourself. You can now easily crop your own profile picture before updating it.
  Group chats actions have been reorganized. It s now easier to send invitations or change your nickname for example. Gajim also received support for chat markers, which enables you to see how far your contact followed the conversation. But this is by far not everything the new release brings. There are many new and helpful features, such as pasting images from your clipboard directly into the chat or playing voice messages directly from the chat window.
  Read more about the new Gajim release in Debian 11 here.
  Furthermore, three more Gajim plugins are now in Debian: gajim-lengthnotifier, gajim-openpgp for OX (XEP-0373: OpenPGP for XMPP) and gajim-syntaxhighlight.
• NEW Kaidan Simple and user-friendly Jabber/XMPP client 0.7.0
  Kaidan is a simple, user-friendly and modern XMPP chat client. The user interface makes use of Kirigami and QtQuick, while the back-end of Kaidan is entirely written in C++ using Qt and the Qt-based XMPP library QXmpp. Kaidan runs on mobile and desktop systems including Linux, Windows, macOS, Android, Plasma Mobile and Ubuntu Touch.
• mcabber, small Jabber (XMPP) console client: 1.1.0 1.1.2
  A theme for 256 color terminals is now included, the handling of carbon message copies has been improved, and various minor issues have been fixed.
• Poezio, Console-based XMPP client: 0.12.1 0.13.1
  This new release brings many improvements, such as Message Archive (XEP-0313) support, initial support for OMEMO (XEP-0384) through a plugin, HTTP File Upload support, Consitent Color Generation (XEP-0392), and plenty of internal changes and bug fixes. Not all changes in 0.13 and 0.13.1 can be listed, see the CHANGELOG for a more extensive summary.
• Profanity, the console based XMPP client: 0.6.0 0.10.0
  We can not list all changes which have been done, but here are some highlights.
  Support of OMEMO Encryption (XEP-0384). Consistent Color Generation (XEP-0392), be aware of the changes in the command to standardize the names of commands. A clipboard feature has been added. Highlight unread messages with a different color in /wins. Keyboard switch to select the next window with unread messages with alt + a. Support for Last Message Correction (XEP-0308), Allow UTF-8 symbols as OMEMO/OTR/PGP indicator char. Add option to open avatars directly (XEP-0084). Add option to define a theme at startup and some changes to improve themes. Add possibility to easily open URLs. Add experimental OX (XEP-0373, XEP-0374) support. Add OMEMO media sharing support, ...
  There is also a Profanity light package in Debian now, the best option for systems with tight limits on resources.
• Prosody, the lightweight extensible XMPP server: 0.11.2 0.11.9
  Upgrading to the latest stable release of Prosody brings a whole load of improvements in the stability, usability and performance departments. It especially improves the performance of websockets, and PEP performance for users with many contacts. It includes interoperability improvements for a range of clients.
• prosody-modules, community modules and extensions for Prosody: 0.0~hg20190203 0.0~hg20210130
  The ever-growing collection of goodies to plug into Prosody has a number of exciting additions, including a suite of modules to handle invite-based account registration, and others for moderating messages in group chats (e.g. for removal of spam/abuse), server-to-server federation over Tor and client authentication using certificates. Many existing community modules received updates as well.
• Psi, Qt-based XMPP client: 1.3 1.5
  The new version contains important bug fixes.
• salutatoi, multi-frontends, multi-purposes communication tool: 0.7.0a4 0.8.0~hg3453
  This version is now fully running on Python 3, and has full OMEMO support (one2one, groups and files). The CLI frontend (jp) has among new commands a "jp file get" one which is comparable to wget with OMEMO support. A file sharing component is included, with HTTP Upload and Jingle support. For a list of other improvements, please consult the changelog.
  Note, that the upstream project has been renamed to "Libervia".
• NEW sms4you, Personal gateway connecting SMS to XMPP or email 0.0.7
  It runs with a GSM device over ModemManager and uses a lightweight XMPP server or a single email account to handle communication in both directions.
• NEW xmppc, XMPP Command Line Client 0.1.0
  xmppc is a new command line tool for XMPP. It supports some basic features of XMPP (request your roster, bookmarks, OMEMO Devices and fingerprints). You can send messages with both legacy PGP (XEP-0027) and the new OX (XEP-0373: OpenPGP for XMPP).

That's all for now. Enjoy Debian 11 bullseye and Happy Chatting!

How are you meant to write about an unfinished emancipation? Bodies of Light is a 2014 book by Glasgow-born Sarah Moss on the stirrings of women's suffrage in an arty clique in nineteenth-century England. Set in the intellectually smoggy cities of Manchester and London, we follow the studious and intelligent Alethia 'Ally' Moberly, who is struggling to gain the acceptance of herself, her mother and the General Medical Council. 'Alethia' may be the Greek goddess of truth, but our Ally is really searching for wisdom. Her strengths are her patience and bookish learning, and she acquires Latin as soon as she learns male doctors will use it to keep women away from the operating theatre. In fact, Ally's acquisition of language becomes a recurring leitmotif: replaying a suggestive dream involving a love interest, for instance, Ally thinks of 'dark, tumbling dreams for which she has a perfectly adequate vocabulary'. There are very few moments of sensuality in the book, and pairing it with Ally's understated wit achieves a wonderful effect. The amount we learn about a character is adapted for effect as well. There are few psychological insights about Ally's sister, for example, and she thus becomes a fey, mysterious and almost Pre-Raphaelite figure below the surface of a lake to match the artistic movement being portrayed. By contrast, we get almost the complete origin story of Ally's mother, Elizabeth, who also constitutes of those rare birds in literature: an entirely plausible Christian religious zealot. Nothing Ally does is ever enough for her, but unlike most modern portrayals of this dynamic, neither of them are aware of what is going, and it is conveyed in a way that is chillingly... benevolent. This was brought home in the annual 'birthday letters' that Elizabeth writes to her daughter:

Last year's letter said that Ally was nervous, emotional and easily swayed, and that she should not allow her behaviour to be guided by feeling but remember always to assert her reason. Mamma would help her with early hours, plain food and plenty of exercise. Ally looks at the letter, plump in its cream envelope. She hopes Mamma wrote it before scolding her yesterday.

The book makes the implicit argument that it is a far more robust argument against pervasive oppression to portray a character in, say, 'a comfortable house, a kind husband and a healthy child', yet they are nonetheless still deeply miserable, for reasons they can't quite put their finger on. And when we see Elizabeth perpetuating some generational trauma with her own children, it is telling that is pattern is not short-circuited by an improvement in their material conditions. Rather, it is arrested only by a kind of political consciousness in Ally's case, the education in a school. In fact, if there is a real hero in Bodies of Light, it is the very concept of female education. There's genuine shading to the book's ideological villains, despite finding their apotheosis in the jibes about 'plump Tories'. These remarks first stuck out to me as cheap thrills by the author; easy and inexpensive potshots that are unbecoming of the pages around them. But they soon prove themselves to be moments of much-needed humour. Indeed, when passages like this are read in their proper context, the proclamations made by sundry Victorian worthies start to serve as deadpan satire:

We have much evidence that the great majority of your male colleagues regard you as an aberration against nature, a disgusting, unsexed creature and a danger to the public.

Funny as these remarks might be, however, these moments have a subtler and more profound purpose as well. Historical biography always has the risk of allowing readers to believe that the 'issue' has already been solved hence, perhaps, the enduring appeal of science fiction. But Moss providing these snippets from newspapers 150 years ago should make a clear connection to a near-identical moral panic today. On the other hand, setting your morality tale in the past has the advantage that you can show that progress is possible. And it can also demonstrate how that progress might come about as well. This book makes the argument for collective action and generally repudiates individualisation through ever-fallible martyrs. Ally always needs 'allies' not only does she rarely work alone, but she is helped in some way by almost everyone around her. This even includes her rather problematic mother, forestalling any simplistic proportioning of blame. (It might be ironic that Bodies of Light came out in 2014, the very same year that Sophia Amoruso popularised the term 'girl boss'.) Early on, Ally's schoolteacher is coded as the primary positive influence on her, but Ally's aunt later inherits this decisive role, continuing Ally's education on cultural issues and what appears to be the Victorian version of 'self-care'. Both the aunt and the schoolteacher are, of course, surrogate mother figures. After Ally arrives in the cut-throat capital, you often get the impression you are being shown discussions where each of the characters embodies a different school of thought within first-wave feminism. This can often be a fairly tedious device in fiction, the sort of thing you would find in a Sally Rooney novel, Pilgrim's Progress or some other ponderously polemical tract. Yet when Ally appears to 'win' an argument, it is only in the sense that the narrator continues to follow her, implicitly and lightly endorsing her point. Perhaps if I knew my history better, I might be able to associate names with the book's positions, but perhaps it is better (at least for the fiction-reading experience...) that I don't, as the baggage of real-world personalities can often get in the way. I'm reminded here of Regina King's One Night in Miami... (2020), where caricatures of Malcolm X, Muhammad Ali, Jim Brown and Sam Cooke awkwardly replay various arguments within an analogous emancipatory struggle. Yet none of the above will be the first thing a reader will notice. Each chapter begins with a description of an imaginary painting, providing a title and a date alongside a brief critical exegesis. The artworks serve a different purpose in each chapter: a puzzle to be unlocked, a fear to be confirmed, an unsolved enigma. The inclusion of (artificial) provenances is interesting as well, not simply because they add colour and detail to the chapter to come, but because their very inclusion feels reflective of how we see art today.

Orphelia (1852) by Sir John Everett Millais.

To continue the question this piece began, how should an author conclude a story about an as-yet-unfinished struggle for emancipation? How can they? Moss' approach dares you to believe the ending is saccharine or formulaic, but what else was she meant to turn in yet another tale of struggle and suffering? After all, Thomas Hardy has already written Tess of the d'Urbervilles. All the same, it still feels slightly unsatisfying to end merely with Ally's muted, uncelebrated success. Nevertheless, I suspect many readers will dislike the introduction of a husband in the final pages, taking it as a betrayal of the preceding chapters. Yet Moss denies us from seeing the resolution as a Disney-style happy ending. True, Ally's husband turns out to be a rather dashing lighthouse builder, but isn't it Ally herself who is lighting the way in their relationship, warning other women away from running aground on the rocks of mental illness? And Tom feels more of a reflection of Ally's newly acquired self-acceptance instead of that missing piece she needed all along. We learn at one point that Tom's 'importance to her is frightening' this is hardly something a Disney princess would say. In fact, it is easy to argue that a heroic ending for Ally might have been an even more egregious betrayal. The evil of saints is that you can never live up to them, for the concept of a 'saint' embodies an unreachable ideal that no human can begin to copy. By being taken as unimpeachable and uncorrectable as well, saints preclude novel political action, and are therefore undoubtedly agents of reaction. Appreciating historical figures as the (flawed) people that they really were is the first step if you wish to continue or adapt their political ideas. I had acquired Bodies of Light after enjoying Moss' Summerwater (2020), which had the dubious honour of being touted as the 'first lockdown novel', despite it being finished before Covid-19. There are countless ways one might contrast the two, so I will limit myself to the sole observation that the strengths of one are perhaps the weaknesses of the other. It's not that Bodies of Light ends with a whimper, of course, as it quietly succeeds in concert with Ally. But by contrast, the tighter arc of Summerwater (which is set during a single day, switches protagonist between chapters, features a closed-off community, etc.) can reach a higher high with its handful of narrative artifices. Summerwater is perhaps like Phil Collins' solo career: 'more satisfying, in a narrower way.'

As part of my GSoC project, my first task includes that user could login into their account on debci using their Debian Salsa account (collaborative development server for Debian based on the GitLab software). The task is officially completed using OmniAuth library and while implementing it, I found that the documentation of OmniAuth is quite a mix-match and more focused on using it with Rails app and this gives me a idea to write a tutorial for people looking to integrate OmniAuth with Sinatra application. So here it is. Now, depending on the provider, Omniauth requires specific strategy which are generally released individually as RubyGems. For this tutorial, I would be using omniauth-gitlab which I used for Debian Salsa in my project, omniauth-twitter and a developer strategy which could be used for project in development mode and comes with omniauth gem itself. For simplicity purposes, I have included all routes and OmniAuth configurations in a single file app.rb. Let s start.

Register your application This can be easily done just head over to provider (Twitter, Salsa) website and find the option to create a new application and fill in the form. In the callback URL field, you need to append /auth/:provider/callback to whichever URL you used in the website field. The client-id and client-secret is obtained from the console which is used in further step to set up OmniAuth.

Gems At the top of the file, we require the necessary gems of for our project

require 'sinatra'
require 'omniauth'
require 'omniauth-gitlab'
require 'omniauth-twitter'

Enable sessions In order for OmniAuth to work and to store the logged in user across requests, sessions need to be activated and if activated, you have one session hash per user session.

configure do
    set :sessions, true
end

Set up OmniAuth configurations OmniAuth::Builder Rack middleware build up your list of OmniAuth strategies for use in your application:

use OmniAuth::Builder do
    if development?
      provider :developer,
               fields: [:name],
               uid_field: :name
    end
    provider :gitlab, #client-id , #client-secret,
             scope: "read_user",
             client_options:  
               site: 'https://salsa.debian.org/api/v4/'
              
    provider :twitter, #client-id , #client-secret,
end

Here, few things to note could be the extra options used with providers, as
In developer:
fields : to specify the form fields for login in developer mode and by default it has name and email.
uid_field: to specify that which field s value could be obtained as uid and by default it is email.
In gitlab:
scope: to limit the scope of application, by default, the api scope is requested and must be allowed in GitLab s application configuration.
client_options: to specify the server being used as client based on Gitlab software.
Note: In case you want to have a different callback url other than the default /auth/:provider/callback, it can be specified using redirect_url option in case of gitlab provider and accordingly update it in your Application Configuration at provider s console.

Extra configurations To redirect to auth/failure route in case of failure even in developer mode, following could be added:

OmniAuth.config.on_failure = proc do  env 
    OmniAuth::FailureEndpoint.new(env).redirect_to_failure
end

By default, OmniAuth will log to STDOUT but you can configure this. If you don t want OmniAuth to log to STDOUT, following could be used:

OmniAuth.config.logger.level = Logger::UNKNOWN

Setting up routes

Login route Starting with GET /login route, where you can specify the options available to login:

get '/login' do
    <<~HTML
    <form method='post' action='/auth/gitlab'>
    <input type="hidden" name="authenticity_token" value='# request.env["rack.session"]["csrf"] '>
    <button type='submit'>Login with Salsa</button>
    </form>
    <form method='post' action='/auth/twitter'>
    <input type="hidden" name="authenticity_token" value='# request.env["rack.session"]["csrf"] '>
    <button type='submit'>Login with Twitter</button>
    </form>
    <form method='post' action='/auth/developer'>
    <input type="hidden" name="authenticity_token" value='# request.env["rack.session"]["csrf"] '>
    <button type='submit'>Login with Developer</button>
    </form>
  HTML
end

The auth/:provider path is created and configured automatically by OmniAuth, so you just need to send the request to that paths and auth process will start.
As per the OmniAuth version 2.0, OmniAuth now defaults to only POST as allowed request_phase methods and authenticity_token is required to validate your requests so make sure to take care of this.

Callback routes: On success from authentication, Omniauth will return the hash of information to the auth/:provider/callback in the Rack environment under the key omniauth.auth so this is what you can use in your desired way like creating a entry to your database and storing the current user in session params.

get '/auth/:provider/callback' do
    erb "
    <h1>Hello # request.env['omniauth.auth']['info']['name'] </h1>"
end
post '/auth/developer/callback' do
    erb "
    <h1>Hello # request.env['omniauth.auth']['info']['name'] </h1>"
end

Here, POST request method is used for developer strategy and GET request method for twitter and gitlab as that is how their working is defined in their respective strategies.

Failure route: If user authentication fails on the provider side, OmniAuth will catch the response and then redirect the request to the path /auth/failure, passing a corresponding error message in a parameter named message.

get '/auth/failure' do
    halt(403, erb("<h2>Authentication Failed</h2><h4>Reason: </h4><pre># params[:message] </pre>"))
end

Final result That s it, our application is all set to be tested. And this is how it will be working. Here, I have already logged into my respective accounts so page for entring credentials does not show up but if the user has not logged into his account, he would be first asked to log in. For complete code, you can check out: OmniAuth with Sinatra Tutorial So this completes the tutorial. I hope it helps others who are looking to integrate their Ruby Applications with OmniAuth. If you have any feedback, feel free to let me know. See you next time!

In June I was assigned 14 hours of work by Freexian's Debian LTS initiative and carried over 2 hours from earlier months. I worked 13.25 hours and will carry over the remainder. I finished bringing the linux (Linux 4.9) package up to date, uploaded it, and issued DLA-2689-1. I also updated the linux-4.19 package based on the version in stable point release 10.10, and issued DLA-2690-1. Finally, I backported my recent security fixes for klibc, uploaded it and issued DLA-2695-1.