Search Results: "tar"

26 October 2022

Scarlett Gately Moore: KDE Gear snaps round 3!

While trying to stay warm in our first snow of the year, I got several apps tested and released in round 3 of https://www.scarlettgatelymoore.dev/kde-gear-snaps-round-2/ ! All of these are being ( re ) tested on both arm64 and amd64. Hence, the release is going slower than it will in the future. Thank you for your patience. More rounds of bug gardening has been done, if you find bugs, or know one of your bugs is related to it being a snap, please assign to me. https://bugs.kde.org Please consider a donation so I may bring many more apps your way. https://www.patreon.com/sgmoore I am working on one time donations.

Russ Allbery: Review: The Golden Enclaves

Review: The Golden Enclaves, by Naomi Novik
Series: The Scholomance #3
Publisher: Del Rey
Copyright: 2022
ISBN: 0-593-15836-9
Format: Kindle
Pages: 408
The Golden Enclaves is the third and concluding book of the Scholomance trilogy and picks up literally the instant after the end of The Last Graduate. The three books form a coherent and complete story that under absolutely no circumstances should be read out of order. This is an impossible review to write because everything is a spoiler. You're only going to read this book if you've read and liked the first two, and in that case you do not want to know a single detail about this book before you read it. The timing of revelations was absolutely perfect; I repeatedly figured out what was going on at exactly the same time that El did, which rarely happens in a book. (And from talking to friends I am not the only one.) If you're still deciding whether to read the series, or are deciding how to prioritize the third book, here are the things you need to know:
  1. Novik nails the ending. Absolutely knocks it out of the park.
  2. Everything is explained, and the explanation was wholly satisfying.
  3. There is more Liesel, and she's even better in the third book.
  4. El's relationship with her mother still works perfectly.
  5. Holy shit.
You can now stop reading this review here and go read, assured that this is the best work of Novik's career to date and has become my favorite fantasy series of all time, something I do not say lightly. For those who want some elaboration, I'll gush some more about this book, but the above is all you need to know. There are so many things that I loved about this series, but the most impressive to me is how each book broadens the scope of the story while maintaining full continuity with the characters and plot. Novik moves from individuals to small groups to, in this book, systems and social forces without dropping a beat and without ever losing the characters. She could have written a series only about El and her friends and it still would have been amazing, but each book takes a risky leap into a broader perspective and she pulls it off every time. This is also one of the most enjoyable first-person perspectives that I have ever read. (I think only Code Name Verity competes, and that's my favorite novel of all time.) Whether you like this series at all will depend on whether you like El, because you spend the entire series inside her head. I loved every moment of it. Novik not infrequently pauses the action to give the reader a page or four of El's internal monologue, and I not only didn't mind, I thought those were the best parts of the book. El is such a deep character: stubborn, thoughtful, sarcastic, impulsive, but also ethical and self-aware in a grudging sort of way that I found utterly compelling to read. And her friends! The friendship dynamics are so great. We sadly don't see as much of Liu in this book (for very good reasons, but I would gladly read an unnecessary sequel novella that existed just to give Liu more time with her friends), but everyone else is here, and in exchange we get much more of Liesel. There should be an Oscar for best supporting character in a novel just so that Liesel can win it. Why are there not more impatient, no-nonsense project managers in fiction? There are a couple of moments between El and Liesel that are among my favorite character interactions in fiction. This is also a series in which the author understands what the characters did in the previous books and the bonds that experience would form, and lets that influence how they interact with the rest of the world. I won't be more specific to avoid spoilers, but the characters worked so hard and were on edge for so long, and I felt like Novik understood the types of relationships that would create in a far deeper and more complex way than most novels. There are several moments in The Golden Enclaves where I paused in reading to admire how perfect the character reactions were, and how striking the contrast was with people who hadn't been through what they went through. The series as a whole is chosen-one fantasy, and if you'd told me that before I read it, I would have grimaced. But this is more evidence (which I should have learned from the romance genre) that tropes, even ones that have been written many times, do not wear out, no matter what critics will try to tell you. There's always room for a great author to pick up the whole idea, turn it sideways, and say "try looking at it from this angle." This is boarding schools, chosen one, and coming of age, with the snarky first-person voice of urban fantasy, and it respects all of those story shapes, is aware of earlier work, and turns them all into something original, often funny, startlingly insightful, and thoroughly engrossing. I am aware that anything I like this much is probably accidentally aimed at my favorite ideas as a reader and my reaction may be partly idiosyncratic. I am not at all objective, and I'm sure not everyone will like it as much as I did. But wow did I ever like this book and this series. Just the best thing I've read in a very, very long time. Highly, highly recommended. (Start at the beginning!) Rating: 10 out of 10

25 October 2022

Arturo Borrero Gonz lez: Netfilter Workshop 2022 summary

Netfilter logo This is my report from the Netfilter Workshop 2022. The event was held on 2022-10-20/2022-10-21 in Seville, and the venue was the offices of Zevenet. We started on Thursday with Pablo Neira (head of the project) giving a short welcome / opening speech. The previous iteration of this event was in virtual fashion in 2020, two years ago. In the year 2021 we were unable to meet either in person or online. This year, the number of participants was just eight people, and this allowed the setup to be a bit more informal. We had kind of an un-conference style meeting, in which whoever had something prepared just went ahead and opened a topic for debate. In the opening speech, Pablo did a quick recap on the legal problems the Netfilter project had a few years ago, a topic that was settled for good some months ago, in January 2022. There were no news in this front, which was definitely a good thing. Moving into the technical topics, the workshop proper, Pablo started to comment on the recent developments to instrument a way to perform inner matching for tunnel protocols. The current implementation supports VXLAN, IPIP, GRE and GENEVE. Using nftables you can match packet headers that are encapsulated inside these protocols. He mentioned the design and the goals, that was to have a kernel space setup that allows adding more protocols by just patching userspace. In that sense, more tunnel protocols will be supported soon, such as IP6IP, UDP, and ESP. Pablo requested our opinion on whether if nftables should generate the matching dependencies. For example, if a given tunnel is UDP-based, a dependency match should be there otherwise the rule won t work as expected. The agreement was to assist the user in the setup when possible and if not, print clear error messages. By the way, this inner thing is pure stateless packet filtering. Doing inner-conntracking is an open topic that will be worked on in the future. Pablo continued with the next topic: nftables automatic ruleset optimizations. The times of linear ruleset evaluation are over, but some people have a hard time understanding / creating rulesets that leverage maps, sets, and concatenations. This is where the ruleset optimizations kick in: it can transform a given ruleset to be more optimal by using such advanced data structures. This is purely about optimizing the ruleset, not about validating the usefulness of it, which could be another interesting project. There were a couple of problems mentioned, however. The ruleset optimizer can be slow, O(n!) in worst case. And the user needs to use nested syntax. More improvements to come in the future. Next was Stefano Brivio s turn (Red Hat engineer). He had been involved lately in a couple of migrations to nftables, in particular libvirt and KubeVirt. We were pointed to https://libvirt.org/firewall.html, and Stefano walked us through the 3 or 4 different virtual networks that libvirt can create. He evaluated some options to generate efficient rulesets in nftables to instrument such networks, and commented on a couple of ideas: having a null matcher in nftables set expression. Or perhaps having kind of subsets, something similar to a view in a SQL database. The room spent quite a bit of time debating how the nft_lookup API could be extended to support such new search operations. We also discussed if having intermediate facilities such as firewalld could provide the abstraction levels that could make developers more comfortable. Using firewalld also may have the advantage that coordination between different system components writing ruleset to nftables is handled by firewalld itself and developers are freed of the responsibility of doing it right. Next was Fernando F. Mancera (Red Hat engineer). He wanted to improve error reporting when deleting table/chain/rules with nftables. In general, there are some inconsistencies on how tables can be deleted (or flushed). And there seems to be no correct way to make a single table go away with all its content in a single command. The room agreed in that the commands destroy table and delete table should be defined consistently, with the following meanings: This topic diverted into another: how to reload/replace a ruleset but keep stateful information (such as counters). Next was Phil Sutter (Netfilter coreteam member and Red Hat engineer). He was interested in discussing options to make iptables-nft backward compatible. The use case he brought was simple: What happens if a container running iptables 1.8.7 creates a ruleset with features not supported by 1.8.6. A later container running 1.8.6 may fail to operate. Phil s first approach was to attach additional metadata into rules to assist older iptables-nft in decoding and printing the ruleset. But in general, there are no obvious or easy solutions to this problem. Some people are mixing different tooling version, and there is no way all cases can be predicted/covered. iptables-nft already refuses to work in some of the most basic failure scenarios. An other way to approach the issue could be to introduce some kind of support to print raw expressions in iptables-nft, like -m nft xyz. Which feels ugly, but may work. We also explored playing with the semantics of release version numbers. And another idea: store strings in the nft rule userdata area with the equivalent matching information for older iptables-nft. In fact, what Phil may have been looking for is not backwards but forward compatibility. Phil was undecided which path to follow, but perhaps the most common-sense approach is to fall back to a major release version bump (2.x.y) and declaring compatibility breakage with older iptables 1.x.y. That was pretty much it for the first day. We had dinner together and went to sleep for the next day. The room The second day was opened by Florian Westphal (Netfilter coreteam member and Red Hat engineer). Florian has been trying to improve nftables performance in kernels with RETPOLINE mitigations enabled. He commented that several workarounds have been collected over the years to avoid the performance penalty of such mitigations. The basic strategy is to avoid function indirect calls in the kernel. Florian also described how BPF programs work around this more effectively. And actually, Florian tried translating nf_hook_slow() to BPF. Some preliminary benchmarks results were showed, with about 2% performance improvement in MB/s and PPS. The flowtable infrastructure is specially benefited from this approach. The software flowtable infrastructure already offers a 5x performance improvement with regards the classic forwarding path, and the change being researched by Florian would be an addition on top of that. We then moved into discussing the meeting Florian had with Alexei in Zurich. My personal opinion was that Netfilter offers interesting user-facing interfaces and semantics that BPF does not. Whereas BPF may be more performant in certain scenarios. The idea of both things going hand in hand may feel natural for some people. Others also shared my view, but no particular agreement was reached in this topic. Florian will probably continue exploring options on that front. The next topic was opened by Fernando. He wanted to discuss Netfilter involvement in Google Summer of Code and Outreachy. Pablo had some personal stuff going on last year that prevented him from engaging in such projects. After all, GSoC is not fundamental or a priority for Netfilter. Also, Pablo mentioned the lack of support from others in the project for mentoring activities. There was no particular decision made here. Netfilter may be present again in such initiatives in the future, perhaps under the umbrella of other organizations. Again, Fernando proposed the next topic: nftables JSON support. Fernando shared his plan of going over all features and introduce programmatic tests from them. He also mentioned that the nftables wiki was incomplete and couldn t be used as a reference for missing tests. Phil suggested running the nftables python test-suite in JSON mode, which should complain about missing features. The py test suite should cover pretty much all statements and variations on how the nftables expression are invoked. Next, Phil commented on nftables xtables support. This is, supporting legacy xtables extensions in nftables. The most prominent problem was that some translations had some corner cases that resulted in a listed ruleset that couldn t be fed back into the kernel. Also, iptables-to-nftables translations can be sloppy, and the resulting rule won t work in some cases. In general, nft list ruleset nft -f may fail in rulesets created by iptables-nft and there is no trivial way to solve it. Phil also commented on potential iptables-tests.py speed-ups. Running the test suite may take very long time depending on the hardware. Phil will try to re-architect it, so it runs faster. Some alternatives had been explored, including collecting all rules into a single iptables-restore run, instead of hundreds of individual iptables calls. Next topic was about documentation on the nftables wiki. Phil is interested in having all nftables code-flows documented, and presented some improvements in that front. We are trying to organize all developer-oriented docs on a mediawiki portal, but the extension was not active yet. Since I worked at the Wikimedia Foundation, all the room stared at me, so at the end I kind of committed to exploring and enabling the mediawiki portal extension. Note to self: is this perhaps https://www.mediawiki.org/wiki/Portals ? Next presentation was by Pablo. He had a list of assorted topics for quick review and comment. Following this, a new topic was introduced by Stefano. He wanted to talk about nft_set_pipapo, documentation, what to do next, etc. He did a nice explanation of how the pipapo algorithm works for element inserts, lookups, and deletion. The source code is pretty well documented, by the way. He showed performance measurements of different data types being stored in the structure. After some lengthly debate on how to introduce changes without breaking usage for users, he declared some action items: writing more docs, addressing problems with non-atomic set reloads and a potential rework of nft_rbtree. After that, the next topic was kubernetes & netfilter , also by Stefano. Actually, this topic was very similar to what we already discussed regarding libvirt. Developers want to reduce packet matching effort, but also often don t leverage nftables most performant features, like sets, maps or concatenations. Some Red Hat developers are already working on replacing everything with native nftables & firewalld integrations. But some rules generators are very bad. Kubernetes (kube-proxy) is a known case. Developers simply won t learn how to code better ruleset generators. There was a good question floating around: What are people missing on first encounter with nftables? The Netfilter project doesn t have a training or marketing department or something like that. We cannot force-educate developers on how to use nftables in the right way. Perhaps we need to create a set of dedicated guidelines, or best practices, in the wiki for app developers that rely on nftables. Jozsef Kadlecsik (Netfilter coreteam) supported this idea, and suggested going beyond: such documents should be written exclusively from the nftables point of view: stop approaching the docs as a comparison to the old iptables semantics. Related to that last topic, next was Laura Garc a (Zevenet engineer, and venue host). She shared the same information as she presented in the Kubernetes network SIG in August 2020. She walked us through nftlb and kube-nftlb, a proof-of-concept replacement for kube-proxy based on nftlb that can outperform it. For whatever reason, kube-nftlb wasn t adopted by the upstream kubernetes community. She also covered latest changes to nftlb and some missing features, such as integration with nftables egress. nftlb is being extended to be a full proxy service and a more robust overall solution for service abstractions. In a nutshell, nftlb uses a templated ruleset and only adds elements to sets, which is exactly the right usage of the nftables framework. Some other projects should follow its example. The performance numbers are impressive, and from the early days it was clear that it was outperforming classical LVS-DSR by 10x. I used this opportunity to bring a topic that I wanted to discuss. I ve seen some SRE coworkers talking about katran as a replacement for traditional LVS setups. This software is a XDP/BPF based solution for load balancing. I was puzzled about what this software had to offer versus, for example, nftlb or any other nftables-based solutions. I commented on the highlighs of katran, and we discussed the nftables equivalents. nftlb is a simple daemon which does everything using a JSON-enabled REST API. It is already packaged into Debian, ready to use, whereas katran feels more like a collection of steps that you need to run in a certain order to get it working. All the hashing, caching, HA without state sharing, and backend weight selection features of katran are already present in nftlb. To work on a pure L3/ToR datacenter network setting, katran uses IPIP encapsulation. They can t just mangle the MAC address as in traditional DSR because the backend server is on a different L3 domain. It turns out nftables has a nft_tunnel expression that can do this encapsulation for complete feature parity. It is only available in the kernel, but it can be made available easily on the userspace utility too. Also, we discussed some limitations of katran, for example, inability to handle IP fragmentation, IP options, and potentially others not documented anywhere. This seems to be common with XDP/BPF programs, because handling all possible network scenarios would over-complicate the BPF programs, and at that point you are probably better off by using the normal Linux network stack and nftables. In summary, we agreed that nftlb can pretty much offer the same as katran, in a more flexible way. Group photo Finally, after many interesting debates over two days, the workshop ended. We all agreed on the need for extending it to 3 days next time, since 2 days feel too intense and too short for all the topics worth discussing. That s all on my side! I really enjoyed this Netfilter workshop round.

Russ Allbery: Review: A Spaceship Repair Girl Supposedly Named Rachel

Review: A Spaceship Repair Girl Supposedly Named Rachel, by Richard Roberts
Publisher: Mystique Press
Copyright: 2022
ISBN: 1-63789-763-4
Format: Kindle
Pages: 353
Rachel had snuck out of the house to sit on the hill, to write and draw in rare peace and quiet, when a bus fell out of the sky like a meteor and plowed into the ground in front of her. This is quickly followed by a baffling encounter with a seven-foot-tall man with a blunderbuss, two misunderstandings and a storytelling lie, and a hurried invitation to get into the bus and escape before they're both infected by math. That's how Rachel discovers that she's able to make on-the-fly repairs to bicycle-powered spaceships, and how she ends up at the Lighthouse of Ceres. The title comes from Rachel's initial hesitation to give her name, which propagates through the book to everyone she meets as certainty that Rachel isn't really her name. I enjoyed this running gag way more than I expected to. I don't read enough young adult and middle-grade books to be entirely clear on the boundaries, but this felt very middle-grade. It has a headlong plot, larger-than-life characters, excitingly imaginative scenery (such as a giant space lighthouse dwarfing the asteroid that it's attached to), a focus on friendship, and no romance. This is, to be clear, not a complaint. But it's a different feel than my normal fare, and there were a few places where I was going one direction and the book went another. The conceit of this book is that Earth is unique in the solar system in being stifled by the horrific weight of math, which infects anyone who visits and makes the routine wonders of other planets impossible. Other planets have their own styles and mythos (Saturn is full of pirates, the inhabitants of Venus are space bunnies with names like Passionfruit Nectar Ecstasy), but throughout the rest of the solar system, belief, style, and story logic reign supreme. That means Rachel's wild imagination and reflexive reliance on tall tales makes her surprisingly powerful. The first wild story she tells, to the man who crashed on earth, shapes most of the story. She had written in her sketchbook that it was the property of the Witch Queen of Eloquent Verbosity and Grandiose Ornamentation, and when challenged on it, says that she stole it to cure her partner. Much to her surprise, everyone outside of Earth takes this completely seriously. Also much to her surprise, her habit of sketching spaceships and imaginative devices makes her a natural spaceship mechanic, a skill in high demand. Some of the story is set on Ceres, a refuge for misfits with hearts of gold. That's where Rachel meets Wrench, a kobold who is by far my favorite character of the book and the one relationship that I thought had profound emotional depth. Rachel's other adventures are set off by the pirate girl Violet (she's literally purple), who is the sort of plot-provoking character that I think only works in middle-grade fiction. By normal standards, Violet's total lack of respect for other people's boundaries or consent would make her more of a villain. Here, while it often annoys Rachel, it's clear that both Rachel and the book take Violet's steamroller personality in good fun, more like the gentle coercion between neighborhood friends trying to pull each other into games. I still got rather tired of Violet, though, which caused me a few problems around the middle of the book. There's a bit of found family here (some of it quite touching), a lot of adventures, a lot of delightful spaceship repair, and even some more serious plot involving the actual Witch Queen of Charon. There is a bit of a plot arc to give some structure to the adventures, but this is not the book to read if you're looking for complex plotting or depth. I thought the story fell apart a bit at the tail end, with a conflict that felt like it was supposed to be metaphorical and then never resolved for me into something concrete. I was expecting Rachel to eventually have to do more introspection and more direct wrestling with her identity, but the resolution felt a bit superficial and unsatisfying. Reading this as an adult, I found it odd but fun. I wanted more from the ending, and I was surprised that Roberts does not do more to explain to the reader why Rachel does not regret leaving Earth and her family behind. It feels like something Rachel will have to confront eventually, but this is not the book for it. Instead we get some great friendships (some of which I agreed with wholeheartedly, and some of which I found annoying) and an imaginative, chaotic universe that Rachel takes to like a fish to water. The parts of the story focused on her surprising competence (and her delight in her own competence) were my favorites. The book this most reminds me of is Norton Juster's The Phantom Tollbooth. It is, to be clear, nowhere near as good as The Phantom Tollbooth, which is a very high bar, and it's not as focused on puns. But it has the same sense of internal logic and the same tendency to put far more weight on belief and stories than our world does, and to embrace the resulting chaos. I'm not sure this will be anyone's favorite book (although I'm also not the target age), but I enjoyed reading it. It was a great change of pace after Nona the Ninth. Recommended if you're in the mood for some space fantasy that doesn't take itself seriously. Rating: 7 out of 10

24 October 2022

Russ Allbery: Review: Nona the Ninth

Review: Nona the Ninth, by Tamsyn Muir
Series: The Locked Tomb #3
Publisher: Tordotcom
Copyright: 2022
ISBN: 1-250-85412-1
Format: Kindle
Pages: 480
Nona the Ninth is the third book of the Locked Tomb series and entirely pointless to read if you have not read the series to date. It completely spoils the previous books, assuming you would be able to figure out who these people were and why you should care about them. This is only for readers who are already invested. This series was originally supposed to be a trilogy, and this book was supposed to be Alecto the Ninth. Muir says in the acknowledgments, and has said at more length elsewhere, that Nona changed all of her plans and demanded her own book. Hence this book, postponing the end of the series and lengthening it to four books. After reading it, I understand why Muir decided to write a whole book about Nona. She's an interesting character in ways that wouldn't have come out if she was a small part of the concluding book. Unfortunately, it's also obvious that this book wasn't part of the plan. It's not entirely correct to say that Nona the Ninth is devoid of series plot, but the plot advances very little, and mostly at the end. Instead, we get Nona, who is physically a teenager who acts like someone several years younger, most of the time. She lives with her family (who I won't name to avoid spoilers for Harrow the Ninth), helps at a local school (although her level of understanding is about that of the students), and is a member of a kid's gang. She also has dreams every night about a woman with a painted face, dreams her family are very interested in. This sounds weirdly normal for this series, but Nona and her family live in a war-torn city full of fighting, refugees, and Blood of Eden operatives. The previous books of the series took place in the rarefied spaces of the Houses. Here we see a bit of the rest of the universe, although it's not obvious at first what we're looking at and who these people are. Absolutely no concession is made to the reader's fading memory, so expect to need either a re-read, help from friends with better memories, or quality time with a wiki. And, well, good luck with the latter if you've not already read this book, since the Locked Tomb Wiki has now been updated with spoilers for Nona. The other challenge, besides memory for the plot, is that this book is told from a tight third-person focus on Nona, and Nona is not a very reliable narrator. She doesn't lie, exactly, but she mostly doesn't understand what's going on, often doesn't care, and tends not to focus on what the reader is the most interested in. Nona is entirely uninterested in developing the series plot. Her focus is on her child friends (who are moderately interesting but not helpful if you're trying to figure out the rest of the story) and the other rhythms of a strange life that's normal to her. For me at least, that meant the first half of this book involved a lot of "what the heck is going on and why do I care about any of this?" I liked Harrow the Ninth a lot, despite how odd and ambiguous it was, but I was ready for revelations and plot coherence and was not thrilled by additional complexity, odd allusions, and half-revealed details. I didn't mind the layers of complexity added on by Harrow, but for me Nona was a bit too much and I started getting frustrated rather than intrigued. We do, at last, get most of the history of this universe, including the specific details of how John became God Emperor and how the Houses were founded. That happens in odd interludes with a forced and somewhat artificial writing style, but it's more straightforward and comprehensible than I feared at first. The pace of the story picks up considerably towards the end of the book, finally providing the plot momentum that I was hoping for. Unfortunately, it also gets more cryptic at the end of the book in ways that I didn't enjoy. The epilogue, which is vital to understanding the climax of the novel, took me three readings before I think I understood what happened. If you preferred the clarity of Gideon the Ninth, be warned that Nona is more like Harrow and Muir seems to be making the plot more cryptic as she goes. I am hoping this trend reverses in Alecto the Ninth. This book made me grumpy. Nona is okay as a character, but the characters in this series that I really like mostly do not appear or appear in heavily damaged and depressing forms. Muir does bring back a couple of my favorite characters, but then does something to them that's a major spoiler but that I think was intended to be a wonderful moment for them and instead left me completely cold and unhappy. There are still some great moments of humor, but overall it felt more strained. That said, I still had tons of fun discussing this book and its implications with friends who were reading it at the same time. I think that is the best way to read this series. Muir is being intentionally confusing and is inserting a blizzard of references. Some of them are pop culture jokes, but some of them are deep plot clues, and I'm not up to deciphering them all by myself. Working through them with other people is much more fun. (It also gives me an opportunity to feel smug about guessing correctly what was happening at the end of Harrow the Ninth, when I'm almost never the person who makes correct guesses about that sort of thing.) I think your opinion of this one will depend on how much you like Nona as a character, how much patience you have for the postponement of plot resolution, and how much tolerance you have for even more cryptic references. I'm still invested in this series until the end, but this was not my favorite installment. I suspect it (and the rest of the series) would benefit immensely from re-reading, but life is short and my reading backlog is long. What Muir is doing is interesting and has a lot of depth, but she's asking quite a lot of the reader. Content warning: Nona has an eating disorder, which occupied rather more of my mental space while reading this book than I was comfortable with. Followed by Alecto the Ninth, which does not have a publication date scheduled as of this writing. Rating: 7 out of 10

20 October 2022

Scarlett Gately Moore: KDE Gear Snaps round 2

As a continuation of https://www.scarlettgatelymoore.dev/new-kde-gear-snaps-in-the-works/ Todays releases, tested on both amd64 and arm64, are: This week has also been a busy week gardening snap bugs in bugs.kde.org. They are all over the place  I am trying to sort out getting them there own section. I have assigned all snap bugs I have found to myself and requested that this is default. If you have bugs, please report them at bugs.kde.org , for now under neon / Snaps. More coming next week!

Dirk Eddelbuettel: RcppQuantuccia 0.1.1 on CRAN: Maintenance

A minor release of RcppQuantuccia arrived on CRAN today. RcppQuantuccia started from the Quantuccia header-only subset / variant of QuantLib which it brings it to R. This project validated the idea of making the calendaring functionality of QuantLib available in a more compact and standalone project which we now do with qlcal which can be seen as a successor to this. This release merely updates a source file to proper encoding as clang++-15 would otherwise warn. The complete list changes for this release follows.

Changes in version 0.1.1 (2022-10-19)
  • Minor code reorganization splitting off calendars.cpp
  • Convert the Argentinian calendar sources files as utf-8 to appease clang++-15
  • Advertise the qlcal package as an alternative

Courtesy of CRANberries, there is also a diffstat report relative to the previous release. More information is on the RcppQuantuccia page. Issues and bugreports should go to the GitHub issue tracker. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

19 October 2022

Shirish Agarwal: Pune Rains, Uncosted Budgets, Hearing Loss Covid, Fracking

Pune Rains Lemme start with a slightly funny picture that tells as much about Pune, my city as anything else does.
Pune- Leave your attitude behind, we have our own
This and similar tags, puns and whatnot you will find if you are entering Pune from the road highway. You can also find similar similar symbols and Puns all over the city and they are partly sarcasm and ironic and partly the truth. Puneities work from the attitude that they know everything rather than nothing, including yours truly  . What is the basis of that or why is there such kind of confidence I have no clue or idea, it is what it is. Approximately 24 hrs. ago, apparently we had a cloudburst. What I came to know later is that we got 100 mm of rain. Sharing from local news site. Much more interesting was a thread made on Reddit where many people half-seriously asked where they can buy a boat. One of the reasons being even if it s October, in fact, we passed middle of October and it s still raining. Even today in the evening, it rained for quite a while. As I had shared in a few blog posts before, June where rains should have started, it didn t, it actually started late July or even August, so something has shifted. The current leadership does not believe in Anthropogenic Climate Change or human activity induced climate change even though that is a reality. I could share many links and even using the term above should give links to various studies. Most of the people who are opposed to it are either misinformed or influenced from the fossil fuel industry. Again, could share many links, but will share just one atm. I have talked to quite a few people about it but nobody has ever been able to give a convincing answer as to why GM had to crush the cars. Let s even take the argument that it was the worst manufactured car in history and there have been quite a few, have the others been crushed? If not, then the reason shared or given by most people sounds hollow. And if you look into it, they had an opportunity that they let go, and now most of them are scrambling and yet most of the legacy auto manufacturers will be out of existence if they don t get back into the game in the next 2-3 years. There have been a bunch of announcements but we are yet to see. The Chinese though have moved far ahead, although one has to remark that they have been doing that for the last decade, so they have a 10-year head start, hardly surprising then. But I need to get back to the subject, another gentleman on Reddit remarked that if you start to use boat, and others start to use boat, then the Govt. will tax it. In fact, somebody had shared the below the other day
Different types of taxes collected by GOI
Many of the taxes that I have shared above are by the Modi Govt. who came on the platform, manifesto that once they come to power they will reduce taxes for the common man, they have reduced taxes but only for the Corporates. For the common man, the taxes have only gone up, both direct tax and indirect tax. Any reference to the Tory party who have also done similar things and have also shared that it is labor who had done large expenditures even though they have been 8 years in power, I am sure for most is purely coincidental. Incidentally, that is the same tack that was taken even by the Republican party. They all like to give tax benefits to the 1% while for the rest is austerity claiming some reason, even if it has been proven to be false.
Corporate Tax Rate, Revenue Loss to Govt.
The figures mentioned above are findings of parliamentary panel so nobody can accuse of anybody having a bias. Also, I probably had shared this but still feel the need to re-share it as people still believe that 2G scam happened even though there are plenty of arguments I can share to prove how it was all a fabricated lie.
Vinod Rai Mafinama in Uttarakhand High Court.
Part 2 of the same Mafinama.
How pathetic Mr. Rai s understanding of economics is can be gauged from the fact that he was made Chairman of IDFC and subsequently had to be thrown out. That whole lie was engineered to throw UPA out and it worked. There are and have been many such coincidences happening over the last 8 years, parallel stories happening in India and UK. This was just yesterday, about a year back Air India was given back to the Tatas, There was controversy about the supposed auctions as only Indigo was the only other party allowed to be at auction but not allowed to buy but more as a spectator as they already have 60% of the Indian civil aviation market. And there was lot of cheering from the Govt. side that finally Air India has been bought home to its true owners, the Tatas. The Tatas too started cheering and sharing how they will take down all the workers, worker unions and everything will be happy glory within a year. In fact, just couple of days back they shared new plans. Btw for the takeover of Air India, they had bought loans from the Banks and they are in the category of too big to fail. As I have shared couple of times before, RBI has not shared any inspection reports of nationalized or private banks after 2013/14. While by law, RBI is supposed to do inspection reports every 3 months and share them in the Public domain. And if you ask any of their supporter, for everything they will say UPA did x or y, which only goes to show morally bankrupt the present Govt. is. Coming back to the topic, before I forget, the idea of sharing their plans is so that they can again borrow money from the banks. But that is not the only story. Just one day back, Smita Prakash, one of the biggest cheerleaders of the present Govt. (she is the boss at Asia News International (ANI)) posted how Air India had treated her sister and other 21 passengers. Basically, they had bought business tickets but the whole cabin was dirty, they complained and they were forced to sit in economy class, not just her cousin sister but the other 21 odd passengers too. Of course Ms Smita became calm as her sister was given free air tickets on Vistara and other goodies. Of course, after that she didn t post anything about the other 21 odd passengers after that. And yes, I understand she is supposed to be a reporter but as can be seen from the twitter thread, there is or was no follow up. Incidentally, she is one of many who has calling others about Revdi culture (freebies to the masses.) but guess that only applies to other people not her or her sister. Again, if there are any coincidences of similar nature in the UK or when Trump was P.M. of the U.S. they are just coincidental .

The Uncosted budget India and the UK have many parallels, it s mind boggling. Before we get into the nitty-gritties, saw something that would be of some interest to the people here.
For those who might not be able to see above, apparently there is place in UK called Tufton Street where there are quite a few organizations that are shadowy and whose finances are not known as to how they are financed. Ms. Truss and quite a few of the people in the cabinet are from the same shadowy organizations. Mr. Kwasi Kwarteng, the just-departed chancellor is and was part of the same group. Now even for me it was a new term to learn and understand what is an uncosted budget is. To make it much more easier I share the example using a common person who goes to the bank for a loan

Now M/S X wants a loan of say Rs. 1000/- for whatever reason. He/she/they go to the bank and asks give me a loan of say INR 1000/- The banks asks them to produce a statement of accounts to show what their financial position is. They produce a somewhat half-filled statement of accounts In which all liabilities are shown, but incomes are not. The bank says you already have so much liabilities, how are you gonna pay those, accounts have to be matched otherwise you are not solvent. M/s X adamantly refuses to do any changes citing that they don t need to. At this point, M/s X credit rating goes down and nobody in the market will give them a loan. At the same time, the assets they had held, their value also depreciates because it became known that they can t act responsibly. So whose to say whether or not M/s X has those number of assets and priced them accurately. But the drama doesn t end there, M/s X says this is the responsibility of actually Mr. Z ( cue The Bank of England) as they are my accountant/lawyer etc. M/s Z says as any lawyer/accountant should. This is not under my remit. If the clients either gives incomplete information or false information or whatever then it is their responsibility not mine. And in fact, the Chancellor is supposed to be the one who is given the responsibility of making the budget. The Chancellor is very similar to our Finance Minister. Because the UK has constitutional monarchy, I am guessing the terms are slightly different, otherwise the functionality seems to be the same. For two weeks, there was lot of chaos, lot of pension funds lost quite a bit in the market and in the end Mr. Kwasi Kwarteng was ousted out of the job. Incredibly, the same media and newspapers who had praised Mr. Kwarteng just few weeks back as the best Tory budget, they couldn t wait to bury him. And while I have attempted to simplify what happened, the best explanation of what has happened can be found in an article from the guardian. Speculation is rife in the UK as to who s ruling atm as the new Chancellor has reversed almost all the policies that Ms. Truss had bought and she is now more or less a figurehead. Mr. Hunt, the new chancellor doesn t have anybody behind him. Apparently, the gentleman wanted to throw his hat the ring in the Tory leadership contest that was held about a month back and he couldn t get 20 MP s to support him. Another thing that is different between UK and India is that in UK by law the PM has to answer questions put up to him or her by the opposition leaders. That is the way accountability is measured there. This is known as PMQ s or Prime Minister Questions and Answers. One can just go to YouTube or any streaming service and give Liz Truss and PMQ s and if they are interested of a certain date, give a date and they can see how she answered the questions thrown at her. Unfortunately, all she could do in both times were non-answers. In fact, the Tories seem to be using some of Labor s policies after they had bad-mouthed the same policies. Politics of right-wing both in the UK and the US seems so out of touch with the people whom they are supposed to protect and administer. An article about cyclists which is sort of half-truth, half irony shows how screwed up the policies are of the RW (right-wing). Now they are questions about the pensions triple-lock. Sadly, it is the working class who would suffer the most, most of the rich have already moved their money abroad several years ago. The Financial Times, did share a video about how things have been unfolding

Seems Ms. Truss forgot to add Financial Times in the list of anti-growth coalition she is so fond of. Also, the Tory party seems to want to create more tax havens in the UK and calling them investment zones. Of course, most of the top tax havens are situated around the UK itself. I wouldn t go more into as that would probably require its own article, although most of that information is all in public domain. Fracking I don t really want to take much time as the blog post has become long. There have been many articles written why Fracking is bad and that is why even the Tories had put in their Manifesto that they won t allow Fracking but apparently, today they are trying to reopen Fracking. And again how bad it is and can be found out by the article in Guardian.

Petter Reinholdtsen: Managing and using ONVIF IP cameras with Linux

Recently I have been looking at how to control and collect data from a handful IP cameras using Linux. I both wanted to change their settings and to make their imagery available via a free software service under my control. Here is a summary of the tools I found. First I had to identify the cameras and their protocols. As far as I could tell, they were using some SOAP looking protocol and their internal web server seem to only work with Microsoft Internet Explorer with some proprietary binary plugin, which in these days of course is a security disaster and also made it impossible for me to use the camera web interface. Luckily I discovered that the SOAP looking protocol is actually following the ONVIF specification, which seem to be supported by a lot of IP cameras these days. Once the protocol was identified, I was able to find what appear to be the most popular way to configure ONVIF cameras, the free software Windows tool named ONVIF Device Manager. Lacking any other options at the time, I tried unsuccessfully to get it running using Wine, but was missing a dotnet 40 library and I found no way around it to run it on Linux. The next tool I found to configure the cameras were a non-free Linux Qt client ONVIF Device Tool. I did not like its terms of use, so did not spend much time on it. To collect the video and make it available in a web interface, I found the Zoneminder tool in Debian. A recent version was able to automatically detect and configure ONVIF devices, so I could use it to set up motion detection in and collection of the camera output. I had initial problems getting the ONVIF autodetection to work, as both Firefox and Chromium refused the inter-tab communication being used by the Zoneminder web pages, but managed to get konqueror to work. Apparently the "Enhanced Tracking Protection" in Firefox cause the problem. I ended up upgrading to the Bookworm edition of Zoneminder in the process to try to fix the issue, and believe the problem might be solved now. In the process I came across the nice Linux GUI tool ONVIF Viewer allowing me to preview the camera output and validate the login passwords required. Sadly its author has grown tired of maintaining the software, so it might not see any future updates. Which is sad, as the viewer is sightly unstable and the picture tend to lock up. Note, this lockup might be due to limitations in the cameras and not the viewer implementation. I suspect the camera is only able to provide pictures to one client at the time, and the Zoneminder feed might interfere with the GUI viewer. I have asked for the tool to be included in Debian. Finally, I found what appear to be very nice Linux free software replacement for the Windows tool, named libonvif. It provide a C library to talk to ONVIF devices as well as a command line and GUI tool using the library. Using the GUI tool I was able to change the admin passwords and update other settings of the cameras. I have asked for the package to be included in Debian. As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b. Update 2022-10-20: Since my initial publication of this text, I got several suggestions for more free software Linux tools. There is a ONVIF python library (already requested into Debian) and a python 3 fork using a different SOAP dependency. There is also support for ONVIF in Home Assistant, and there is an alternative to Zoneminder called Shinobi. The latter two are not included in Debian either. I have not tested any of these so far.

17 October 2022

Scarlett Gately Moore: New KDE Gear snaps in the works

KDE Extras 22.08.2 was released! https://kde.org/announcements/gear/22.08.2/ So I am working on new snaps! This release also includes a new content snap I made with frameworks 5.98 and Qt 5.15.6. With all the new goodness, I am (Re) testing all snaps to make sure they are working as expected. You can find a link to all of my snap releases from the KDE Snap Store Releases on the menu above. Some notable releases that have new fixes and improvements are: New 22.08.2 releases re-tested on arm64 and amd64: More coming soon! Please consider donating! I am seeking employment, but until then, I need assistance with gas to power my generator to power my laptop. Solar doesn t work great in these coming winter months  Thank you for your consideration! https://www.patreon.com/sgmoore

Dima Kogan: gnuplot output in an FLTK widget

Overview
I make a lot of plots, and the fragmentation of tools in this space really bugs me. People writing Python code mostly use matplotlib, R people use ggplot2. MS people use the internal Excel thing. I've seen people use gtkdatabox for GTK widgets, rrdtool for logging, qcustomplot for qt. And so on. This is really unhelpful, and it would benefit everybody if there was a single solid plotting backend with lots of bindings to different languages and tools. For my own usage, I've been fighting this quixotic battle, using gnuplot as the plotting backend for all my use cases. gnuplot is
  • very mature
  • stable
  • fast
  • powerful
  • supported on every (with reason) platform
  • supports lots and lots of output backends
There are some things it can't do, but those can be added, and I haven't felt it to be limiting in over 20 years of using it. I rarely use it directly, and usually interact with it through one of I wrote all of these, although the Perl library was taken over by others long ago. Recently I needed a plotting widget for an FLTK program written in Python. It would be great if there was a C++ class deriving from Fl_Widget that would be wrapped by pyfltk, but there isn't. But it turns out that I already had all the tools to quickly hack together something that mostly works. This is a not-ready-for-primetime hack, but it works so well, I'd like to write it up. Hopefully this will be done "properly" someday.

Approach
Alright. So here I'm trying to tie together a Python program, gnuplot output and an FLTK widget. This is a Python program, I can use gnuplotlib to talk to the gnuplot backend. In a perfect world, gnuplot would ship a backend interfacing to FLTK. But it doesn't. What it does do is to ship an x11 backend that makes plots with X11 commands, and it allows these commands to be directed to an arbitrary X11 window. So we
  1. Make an FLTK widget that simply creates an X11 window, and never actually draws into it
  2. Tell gnuplot to plot into this window

Demo
This is really simple, and works shockingly well. Here's my Fl_gnuplotlib widget:
#!/usr/bin/python3
import sys
import gnuplotlib as gp
import fltk
class Fl_Gnuplotlib_Window(fltk.Fl_Window):
    def __init__(self, x,y,w,h, **plot_options):
        super().__init__(x,y,w,h)
        self.end()
        self._plot                 = None
        self._delayed_plot_options = None
        self.init_plot(**plot_options)
    def init_plot(self, **plot_options):
        if 'terminal' in plot_options:
            raise Exception("Fl_Gnuplotlib_Window needs control of the terminal, but the user asked for a specific 'terminal'")
        if self._plot is not None:
            self._plot = None
        self._delayed_plot_options = None
        xid = fltk.fl_xid(self)
        if xid == 0:
            # I don't have an xid (yet?), so I delay the init
            self._delayed_plot_options = plot_options
            return
        # will barf if we already have a terminal
        gp.add_plot_option(plot_options,
                           terminal = f'x11 window "0x xid:x "')
        self._plot = gp.gnuplotlib(**plot_options)
    def plot(self, *args, **kwargs):
        if self._plot is None:
            if self._delayed_plot_options is None:
                raise Exception("plot has not been initialized")
            self.init_plot(**self._delayed_plot_options)
            if self._plot is None:
                raise Exception("plot has not been initialized. Delayed initialization failed")
        self._plot.plot(*args, **kwargs)
Clearly it's simply making an Fl_Window, and pointing gnuplotlib at it. And a sample application that uses this widget:
#!/usr/bin/python3
import sys
import numpy as np
import numpysane as nps
from fltk import *
from Fl_gnuplotlib import *
window = Fl_Window(800, 600, "plot")
plot   = Fl_Gnuplotlib_Window(0, 0, 800,600)
iplot = 0
plotx = np.arange(1000)
ploty = nps.cat(plotx*plotx,
                np.sin(plotx/100),
                plotx)
def timer_callback(*args):
    global iplot, plotx, ploty, plot
    plot.plot(plotx,
              ploty[iplot],
              _with = 'lines')
    iplot += 1
    if iplot == len(ploty):
        iplot = 0
    Fl.repeat_timeout(1.0, timer_callback)
window.resizable(window)
window.end()
window.show()
Fl.add_timeout(1.0, timer_callback)
Fl.run()
This is nice and simple. Exactly what a program using a widget to make a plot (while being oblivious to the details) should look like. It creates a window, places the one plotting widget into it, and cycles the plot inside it at 1Hz (cycling between a parabola, a sinusoid and a line). Clearly we could place other UI elements around it, or add more plots, or whatever. The output looks like this:
Fl_gnuplotlib_demo.gif
To run you need to apt install python3-numpysane python3-gnuplotlib python3-fltk. If running an older distro on a non-Debian-based distro, you should grab those from source.

Discussion
This works. But it's a hack. Some issues:
  • This plotting widget currently can output only. It can make whatever plot we like, but it cannot accept UI input from the container program in any way
  • More than that, when focused it completely replaces the FLTK event logic for that window. So all keyboard input is swallowed, including the keys to access FLTK menus, to exit the application, etc, etc.
  • This approach requires us to use the x11 gnuplot terminal. This works, but it's no longer the terminal preferred by the gnuplot devs, and it it's maintained as vigilantly as the others.
  • And it has bugs. For instance, asking to plot into a window that doesn't yet exist, causes it to create a new window. This breaks FLTK applications that start up and create a plot immediately. Here's a mailing list thread discussing these issues.
So this is a very functional hack, but it's still hack. And it feels like making this solid will take a lot of work. Maybe. I'll push more on this as I need it. Stay tuned!

Jeremy Bicha: Ubuntu bug fix anniversary

I first installed Ubuntu when Ubuntu 6.06 LTS Dapper Drake was released. I was brand new to Linux. This was Ubuntu s first LTS release; the very first release of Ubuntu was only a year and a half before. I was impressed by how usable and useful the system was. It soon became my primary home operating system and I wanted to help make it better. On October 15, 2009, I was helping test the release candidates ISOs for the Ubuntu 9.10 release. Specifically, I tested Edubuntu. Edubuntu has since been discontinued but at the time it was an official Ubuntu flavor preloaded with lots of education apps. One of those education apps was Moodle, an e-learning platform. When testing Moodle, I found that a default installation would make Moodle impossible to use locally. I figured out how to fix this issue. This was really exciting: I finally found an Ubuntu bug I knew how to fix. I filed the bug report. This was very late in the Ubuntu 9.10 release process and Ubuntu was in the Final Freeze state. In Final Freeze, every upload to packages included in the default install need to be individually approved by a member of the Ubuntu Release Team. Also, I didn t have upload rights to Ubuntu. Jordan Mantha (LaserJock), an Edubuntu maintainer, sponsored my bug fix upload. I also forwarded my patch to Debian. While trying to figure out what wasn t working with Moodle, I stumbled across a packaging bug. Edubuntu provided a choice of MySQL or PostgreSQL for the system default database. MySQL was the default, but if PostgreSQL were chosen instead, Moodle wouldn t work. I figured out how to fix this bug too a week later. Jordan sponsored this upload and Steve Langasek from the Release Team approved it so it also was able to be fixed before 9.10 was released. Although the first bug was new to 9.10 because of a behavior change in a low-level dependency, this PostgreSQL bug existed in stable Ubuntu releases. Therefore, I prepared Stable Release Updates for Ubuntu 9.04 and Ubuntu 8.04 LTS. Afterwards Six months later, I was able to attend my first Ubuntu Developer Summit. I was living in Bahrain (in the Middle East) at the time and a trip to Belgium seemed easier to me than if I were living in the United States where I usually live. This was the Ubuntu Developer Summit where planning for Ubuntu 10.10 took place. I like to believe that I helped with the naming since I added Maverick to the wiki page where people contribute suggestions. I did not apply for financial sponsorship to attend and I stayed in a budget hotel on the other side of Brussels. The event venue was on the outskirts of Brussels so there wasn t a direct bus or metro line to get there. I rented a car. I didn t yet have a smartphone and I had a LOT of trouble navigating to and from the site every day. I learned then that it s best to stay close to the conference site since a lot of the event is actually in the unstructured time in the evenings. Fortunately, I managed to arrive in time for Mark Shuttleworth s keynote where the Unity desktop was first announced. This was released in Ubuntu 10.10 in the Ubuntu Netbook Remix and became the default for Ubuntu Desktop in Ubuntu 11.04. Ubuntu s switch to Unity provided me with a huge opportunity. In April 2011, GNOME 3.0 was released. I wanted to try it but it wasn t yet packaged in Ubuntu or Debian. It was suggested that I could help work on packaging the major new version in a PPA. The PPA was convenient because I was able to get permission to upload there easier than being able to upload directly to Ubuntu. My contributions there then enabled me to get upload rights to the Ubuntu Desktop packages later that year. At a later Ubuntu Developer Summit, it was suggested that I start an official Ubuntu flavor for GNOME. So along with Tim Lunn (darkxst), I co-founded Ubuntu GNOME. Years later, Canonical stopped actively developing Unity; instead, Ubuntu GNOME was merged into Ubuntu Desktop. Along the way, I became an Ubuntu Core Developer and a Debian Developer. And in January 2022, I joined Canonical on the Desktop Team. This all still feels amazing to me. It took me a long time to be comfortable calling myself a developer! Conclusion My first Ubuntu bugfix was 13 years ago this week. Because Ubuntu historically uses alphabetical adjective animal release names, 13 years means that we have rolled around to the letter K again! Later today, we begin release candidate ISO testing for Ubuntu 22.10 Kinetic Kudu . I encourage you to help us test the release candidates and report bugs that you find. If you figure out how to fix a bug, we still sponsor bug fixes. If you are an Ubuntu contributor, I highly encourage you to attend an Ubuntu Summit if you can. The first Ubuntu Summit in years will be in 3 weeks in Prague, but the intent is for the Ubuntu Summits to be recurring events again.

16 October 2022

Sven Hoexter: CentOS 9, stunnel, an openssl memory leak and a VirtualBox crash

tl;dr; OpenSSL 3.0.1 leaks memory in ssl3_setup_write_buffer(), seems to be fixed in 3.0.5 3.0.2. The issue manifests at least in stunnel and keepalived on CentOS 9. In addition I learned the hard way that running a not so recent VirtualBox version on Debian bullseye let to dh parameter generation crashing in libcrypto in bn_sqr8x_internal(). A recent rabbit hole I went down. The actual bug in openssl was nailed down and documented by Quentin Armitage on GitHub in keepalived My bugreport with all back and forth in the RedHat Bugzilla is #2128412. Act I - Hello stunnel, this is the OOMkiller Calling We started to use stunnel on Google Cloud compute engine instances running CentOS 9. The loadbalancer in front of those instances used a TCP health check to validate the backend availability. A day or so later the stunnel instances got killed by the OOMkiller. Restarting stunnel and looking into /proc/<pid>/smaps showed a heap segment growing quite quickly. Act II - Reproducing the Issue While I'm not the biggest fan of VirtualBox and Vagrant I've to admit it's quite nice to just fire up a VM image, and give other people a chance to recreate that setup as well. Since VirtualBox is no longer released with Debian/stable I just recompiled what was available in unstable at the time of the bullseye release, and used that. That enabled me now to just start a CentOS 9 VM, setup stunnel with a minimal config, grab netcat and a for loop and watch the memory grow. E.g. while true; do nc -z localhost 2600; sleep 1; done To my surprise, in addition to the memory leak, I also observed some crashes but did not yet care too much about those. Act III - Wrong Suspect, a Workaround and Bugreporting Of course the first idea was that something must be wrong in stunnel itself. But I could not find any recent bugreports. My assumption is that there are still a few people around using CentOS and stunnel, so someone else should probably have seen it before. Just to be sure I recompiled the latest stunnel package from Fedora. Didn't change anything. Next I recompiled it without almost all the patches Fedora/RedHat carries. Nope, no progress. Next idea: Maybe this is related to the fact that we do not initiate a TLS context after connecting? So we changed the test case from nc to openssl s_client, and the loadbalancer healthcheck from TCP to a TLS based one. Tada, a workaround, no more memory leaking. In addition I gave Fedora a try (they have Vagrant Virtualbox images in the "Cloud" Spin, e.g. here for Fedora 36) and my local Debian installation a try. No leaks experienced on both. Next I reported #2128412. Act IV - Crash in libcrypto and a VirtualBox Bug When I moved with the test case from the Google Cloud compute instance to my local VM I encountered some crashes. That morphed into a real problem when I started to run stunnel with gdb and valgrind. All crashes happened in libcrypto bn_sqr8x_internal() when generating new dh parameter (stunnel does that for you if you do not use static dh parameter). I quickly worked around that by generating static dh parameter for stunnel. After some back and forth I suspected VirtualBox as the culprit. Recompiling the current VirtualBox version (6.1.38-dfsg-3) from unstable on bullseye works without any changes. Upgrading actually fixed that issue. Epilog I highly appreciate that RedHat, with all the bashing around the future of CentOS, still works on community contributed bugreports. My kudos go to Clemens Lang. :) Now that the root cause is clear, I guess RedHat will push out a fix for the openssl 3.0.1 based release they have in RHEL/CentOS 9. Until that is available at least stunnel and keepalived are known to be affected. If you run stunnel on something public it's not that pretty, because already a low rate of TCP connections will result in a DoS condition.

Colin Watson: Reproducible man-db databases

I ve released man-db 2.11.0 (announcement, NEWS), and uploaded it to Debian unstable. The biggest chunk of work here was fixing some extremely long-standing issues with how the database is built. Despite being in the package name, man-db s database is much less important than it used to be: most uses of man(1) haven t required it in a long time, and both hardware and software improvements mean that even some searches can be done by brute force without needing prior indexing. However, the database is still needed for the whatis(1) and apropos(1) commands. The database has a simple format - no relational structure here, it s just a simple key-value database using old-fashioned DBM-like interfaces and composing a few fields to form values - but there are a number of subtleties involved. The issues tend to amount to this: what does a manual page name mean? At first glance it might seem simple, because you have file names that look something like /usr/share/man/man1/ls.1.gz and that s obviously ls(1). Some pages are symlinks to other pages (which we track separately because it makes it easier to figure out which entries to update when the contents of the file system change), and sometimes multiple pages are even hard links to the same file. The real complications come with whatis references . Pages can list a bunch of names in their NAME section, and the historical expectation is that it should be possible to use those names as arguments to man(1) even if they don t also appear in the file system (although Debian policy has deprecated relying on this for some time). Not only does that mean that man(1) sometimes needs to consult the database, but it also means that the database is inherently more complicated, since a page might list something in its NAME section that conflicts with an actual file name in the file system, and now you need a priority system to resolve ambiguities. There are some other possible causes of ambiguity as well. The people working on reproducible builds in Debian branched out to the related challenge of reproducible installations some time ago: can you take a collection of packages, bootstrap a file system image from them, and reproduce that exact same image somewhere else? This is useful for the same sorts of reasons that reproducible builds are useful: it lets you verify that an image is built from the components it s supposed to be built from, and doesn t contain any other skulduggery by accident or design. One of the people working on this noticed that man-db s database files were an obstacle to that: in particular, the exact contents of the database seemed to depend on the order in which files were scanned when building it. The reporter proposed solving this by processing files in sorted order, but I wasn t keen on that approach: firstly because it would mean we could no longer process files in an order that makes it more efficient to read them all from disk (still valuable on rotational disks), but mostly because the differences seemed to point to other bugs. Having understood this, there then followed several late nights of very fiddly work on the details of how the database is maintained. None of this was conceptually difficult: it mainly amounted to ensuring that we maintain a consistent well-order for different entries that we might want to insert for a given database key, and that we consider the same names for insertion regardless of the order in which we encounter files. As usual, the tricky bit is making sure that we have the right data structures to support this. man-db is written in C which is not very well-supplied with built-in data structures, and originally much of the code was written in a style that tried to minimize memory allocations; this came at the cost of ownership and lifetime often being rather unclear, and it was often difficult to make changes without causing leaks or double-frees. Over the years I ve been gradually introducing better encapsulation to make things easier to follow, and I had to do another round of that here. There were also some problems with caching being done at slightly the wrong layer: we need to make use of a trace of the chain of links followed to resolve a page to its ultimate source file, but we were incorrectly caching that trace and reusing it for any link to the same file, with incorrect results in many cases. Oh, and after doing all that I found that the on-disk representation of a GDBM database is insertion-order-dependent, so I ended up having to manually reorganize the database at the end by reading it all in and writing it all back out in sorted order, which feels really weird to me coming from spending most of my time with PostgreSQL these days. Fortunately the database is small so this takes negligible time. None of this is particularly glamorous work, but it paid off:
# export SOURCE_DATE_EPOCH="$(date +%s)"
# mkdir emptydir disorder
# disorderfs --multi-user=yes --shuffle-dirents=yes --reverse-dirents=no emptydir disorder
# export TMPDIR="$(pwd)/disorder"
# mmdebstrap --variant=standard --hook-dir=/usr/share/mmdebstrap/hooks/merged-usr \
      unstable out1.tar
# mmdebstrap --variant=standard --hook-dir=/usr/share/mmdebstrap/hooks/merged-usr \
      unstable out2.tar
# cmp out1.tar out2.tar
# echo $?
0

Aigars Mahinovs: Ryzen 7000 amdgpu boot hang

So you decided to build a brand new system using all the latest and coolest tech, so you buy a Ryzen 7000 series Zen 4 CPU, like the Ryzen 7700X that I picked, with a new mother board and DDR5 memory and all that jazz. But for now, you don't yet have a fitting GPU for that system (as the new ones will only come out in November), so you are booting a Debian system using the new build-in video card of the new CPUs (Zen 4 generation has a simple AMD GPU build-in into every CPU now - great stuff for debugging and mostly-headless systems) and you get ... nothing on the screen. Hmm. You boot into the rescue mode and the kernel message stop after:
Oct 16 13:31:25 home kernel: [    4.128328] amdgpu: Ignoring ACPI CRAT on non-APU system
Oct 16 13:31:25 home kernel: [    4.128329] amdgpu: Virtual CRAT table created for CPU
Oct 16 13:31:25 home kernel: [    4.128332] amdgpu: Topology: Add CPU node
That looks bad, right? Well, if you either ssh into the machine or reboot with module_blacklist=amdgpu in the kernel command line you will find in /var/log/kern.log.1 those messages and also the following messages that will clarify the situation a bit:
Oct 16 13:31:25 home kernel: [    4.129352] amdgpu 0000:10:00.0: firmware: failed to load amdgpu/psp_13_0_5_toc.bin (-2)
Oct 16 13:31:25 home kernel: [    4.129354] firmware_class: See https://wiki.debian.org/Firmware for information about missing firmware
Oct 16 13:31:25 home kernel: [    4.129358] amdgpu 0000:10:00.0: firmware: failed to load amdgpu/psp_13_0_5_toc.bin (-2)
Oct 16 13:31:25 home kernel: [    4.129359] amdgpu 0000:10:00.0: Direct firmware load for amdgpu/psp_13_0_5_toc.bin failed with error -2
Oct 16 13:31:25 home kernel: [    4.129360] amdgpu 0000:10:00.0: amdgpu: fail to request/validate toc microcode
Oct 16 13:31:25 home kernel: [    4.129361] [drm:psp_sw_init [amdgpu]] *ERROR* Failed to load psp firmware!
Oct 16 13:31:25 home kernel: [    4.129432] [drm:amdgpu_device_init.cold [amdgpu]] *ERROR* sw_init of IP block <psp> failed -2
Oct 16 13:31:25 home kernel: [    4.129525] amdgpu 0000:10:00.0: amdgpu: amdgpu_device_ip_init failed
Oct 16 13:31:25 home kernel: [    4.129526] amdgpu 0000:10:00.0: amdgpu: Fatal error during GPU init
Oct 16 13:31:25 home kernel: [    4.129527] amdgpu 0000:10:00.0: amdgpu: amdgpu: finishing device.
Oct 16 13:31:25 home kernel: [    4.129633] amdgpu: probe of 0000:10:00.0 failed with error -2
So what you need is to get a new set of Linux Kernel Firmware blobs and upack that in /lib/firmware. The tarball from 2022-10-12 worked well for me. After that you also need to re-create the initramfs with update-initramfs -k all -c to include the new firmware. Having kernel version 5.18 or newer is also required for stable Zen 4 support. It might be that a fresh Mesa version is also of importance, but as I am running sid on this machine I can only say that Mesa 22.2.1 that is in sid works fine.

Vincent Fourmond: Tutorial: analysis of multiwavelength fast kinetics data

The purpose of this post is to demonstrate a first approach to the analysis of multiwavelength kinetic data, like those obtained using stopped-flow data. To practice, we will use data that were acquired during the stopped flow practicals of the MetBio summer school from the FrenchBIC. During the practicals, the student monitored the reaction of myoglobin (in its Fe(III) state) with azide, which yields a fast and strong change in the absorbance spectrum of the protein, which was monitored using a diode array. The data is publicly available on zenodo. Aims of this tutorial The purpose of this tutorial is to teach you to use the free softwareQSoas to run a simple, multiwavelength exponential fit on the data, and to look at the results. This is not a kinetics lecture, so that it will not go in depth about the use of the exponential fit and its meaning. Getting started: loading the file First, make sure you have a working version of QSoas, you can download them (for free) there. Then download the data files from zenodo. We will work only on the data file Azide-1.25mm_001.dat, but of course, the purpose of this tutorial is to enable you to work on all of them. The data files contain the time evolution of the absorbance for all wavelengths, in a matrix format, in which each row correpond to a time point and each column to a wavelength. Start QSoas, and launch the command:
QSoas> load /comments='"'
Then, choose the Azide-1.25mm_001.dat data file. This should bring up a horizontal red line at the bottom of the data display, with X values between about 0 and 2.5. If you zoom on the red line with the mouse wheel, you'll realize it is data. The /comments='"' part is very important since it allows the extraction of the wavelength from the data. We will look at what it means another day. At this stage, you can look at the loaded data using the command:
QSoas> edit
You should have a window looking like this:
The rows each correspond to a data point displayed on the window below. The first column correspond to the X values, the second the Y values, and all the other ones to extra Y columns (they are not displayed by default). What is especially interesting is the first row, which contains a nan as the X value and what is obviously the wavelength for all the Y values. To tell that QSoas should take this line as the wavelength (which will be the perpendicular coordinate, the coordinate of the other direction of the matrix), first close the edit window and run:
QSoas> set-perp /from-row=0
Splitting and fitting Now, we have a single dataset containing a lot of Y columns. We want to fit all of them simultaneously with a (mono) exponential fit. For that, we first need to split the big matrix into a series of X,Y datasets (because fitting only works on the first Y). This is possible by running:
QSoas> expand /style=red-to-blue /flags=kinetics
Your screen should now look like this:
You're looking at the kinetics at all wavelengths at the same time (this may take some time to display on your computer, it is after all a rather large number of data points). The /style=red-to-blue is not strictly necessary, but it gives the red to blue color gradient which makes things easier to look at (and cooler !). The /flags=kinetics is there to attach a label (a flag) to the newly created datasets so we can easily manipulate all of them at the same time. Then it's time to fit, with the following command:
QSoas> mfit-exponential-decay flagged:kinetics
This should bring up a new window. After resizing it, you should have something that looks like this:
The bottom of the fit window is taken by the parameters, each with two checkboxes on the right to set them fixed (i.e. not determined by the fitting mechanism) and/or global (i.e. with a single value for all the datasets, here all the wavelengths). The top shows the current dataset along with the corresponding fit (in green), and, below, the residuals. You can change the dataset by clicking on the horizontal arrows or using Ctrl+PgUp or Ctrl+PgDown (keep holding it to scan fast). See the Z = 728.15 showing that QSoas has recognized that the currently displayed dataset corresponds to the wavelength 728.15. The equation fitted to the data is: $$y(x) = A_\infty + A_1 \times \exp -(x - x_0)/\tau_1$$ In this case, while the \(A_1\) and \(A_\infty\) parameters clearly depend on the wavelength, the time constant of evolution should be independent of wavelength (the process happens at a certain rate regardless of the wavelength we're analyzing), so that the \(\tau_1\) parameter should be common for all the datasets/wavelengths. Just click on the global checkbox at the right of the tau_1 parameter, make sure it is checked, and hit the Fit button... The fit should not take long (less than a minute), and then you end up with the results of the fits: all the parameters. The best way to look at the non global parameters like \(A_1\) and \(A_\infty\) is to use the Show Parameters item from the Parameters menu. Using it and clicking on A_inf too should give you a display like this one:
The A_inf parameter corresponds to the spectum at infinite time (of azide-bound heme), while the A_1 parameter corresponds to the difference spectrum between the initial (azide-free) and final (azide-bound) states. Now, the fit is finished, you can save the parameters if you want to reload them in a later fit by using the Parameters/Save menu item or export them in a form more suitable for plotting using Parameters/Export (although QSoas can also display and the parameters saved using Save). This concludes this first approach to fitting the data. What you can do is How to read the code above All the lines starting by QSoas> in the code areas above are meant to be typed into the QSoas command line (at the bottom of the window), and started by pressing enter at the end. You must remove the QSoas> bit. The other lines (when applicable) show you the response of QSoas, in the terminal just above the command-line. You may want to play with the QSoas tutorial to learn more about how to interact with QSoas. About QSoas QSoas is a powerful open source data analysis program that focuses on flexibility and powerful fitting capacities. It is released under the GNU General Public License. It is described in Fourmond, Anal. Chem., 2016, 88 (10), pp 5050 5052. Current version is 3.1. You can freely (and at no cost) download its source code or precompiled versions for MacOS and Windows there. Alternatively, you can clone from the GitHub repository.
Contact: find my email address there, or contact me on LinkedIn.

15 October 2022

Jonathan Dowland: podman generate

I've been working with and on container technology for seven years, but I still learn new things every day. Recently I read the excellent LWN article Docker and the OCI container ecosystem and this was news to me:
Running the docker CLI under a process supervisor only results in supervising the CLI process. This has a variety of consequences for users of these tools. For example, any attempt to limit a container's memory usage by running the CLI as a systemd service will fail; the limits will only apply to the CLI and its non-existent children. In addition, attempts to terminate a client process may not result in terminating all of the processes in the container.
Huh of course! I hadn't really thought about that. I run a small number of containers on my home system via docker (since I was using it at work) and systemd, and sometimes I have weird consistency issues. This explains them. Later:
As a result, Podman plays nicely with tools like systemd; using podman run with a process supervisor works as expected, because the processes inside the container are children of podman run. The developers of Podman encourage people to use it in this way by a command to generate systemd units for Podman containers.
Given the above, it seemed like a good idea to migrate my few local containers over to Podman. This was easy. The first part is copying the images from Docker's storage to Podman's. To do this, I used the skopeo tool:
sudo skopeo copy  docker-daemon,containers-storage :octoprint/octoprint:latest
(I want to launch these as a superuser, rather than use root-less containers, to isolate the containers from each other: rootless ones are forced to share a common namespace.) Once the images were copied over, I needed to start up a container instance via Podman. For most of them, running under Docker, I had volume-mounted the important bits for persistent data. I can do the same with Podman:
# podman run -d --rm \
    --name octoprint \
    -p 8080:80 \
    -v /archive/octoprint:/octoprint \
    --device /dev/ttyACM0:/dev/ttyACM0 \
    octoprint/octoprint
Once an instance was running, I can use podman generate to create a Systemd unit file which describes creating an equivalent container:
# cd /etc/systemd/system
# podman generate systemd octoprint \
    --new \
    --name \
    --files
For some of the containers I was running, there are a few more steps required: migrating docker volumes and configuring equivalents for private docker networks. But Podman's versions of those concepts are largely identical. Running a mixture of Podman and Docker containers side-by-side also meant renumbering some private addresses for migrated hosts whilst the original network was still up.

14 October 2022

Shirish Agarwal: Dowry, Racism, Railways

Dowry Few days back, had posted about the movie Raksha Bandhan and whatever I felt about it. Sadly, just couple of days back, somebody shared this link. Part of me was shocked and part of me was not. Couple of acquaintances of mine in the past had said the same thing for their daughters. And in such situations you are generally left speechless because you don t know what the right thing to do is. If he has shared it with you being an outsider, how many times he must have told the same to their wife and daughters? And from what little I have gathered in life, many people have justified it on similar lines. And while the protests were there, sadly the book was not removed. Now if nurses are reading such literature, how their thought process might be forming, you can tell :(. And these are the ones whom we call for when we are sick and tired :(. And I have not taken into account how the girls/women themselves might be feeling. There are similar things in another country but probably not the same, nor the same motivations though although feeling helplessness in both would be a common thing. But such statements are not alone. Another gentleman in slightly different context shared this as well
The above is a statement shared in a book recommended for CTET (Central Teacher s Eligibility Test that became mandatory to be taken as the RTE (Right To Education) Act came in.). The statement says People from cold places are white, beautiful, well-built, healthy and wise. And people from hot places are black, irritable and of violent nature. Now while I can agree with one part of the statement that people residing in colder regions are more fair than others but there are loads of other factors that determine fairness or skin color/skin pigmentation. After a bit of search came to know that this and similar articulation have been made in an idea/work called Environmental Determinism . Now if you look at that page, you would realize this was what colonialism is and was all about. The idea that the white man had god-given right to rule over others. Similarly, if you are fair, you can lord over others. Seems simplistic, but yet it has a powerful hold on many people in India. Forget the common man, this thinking is and was applicable to some of our better-known Freedom fighters. Pune s own Bal Gangadhar Tilak The Artic Home to the Vedas. It sort of talks about Aryans and how they invaded India and became settled here. I haven t read or have access to the book so have to rely on third-party sources. The reason I m sharing all this is that the right-wing has been doing this myth-making for sometime now and unless and until you put a light on it, it will continue to perpetuate  . For those who have read this blog, do know that India is and has been in casteism from ever. They even took the fair comment and applied it to all Brahmins. According to them, all Brahmins are fair and hence have god-given right to lord over others. What is called the Eton boy s network serves the same in this casteism. The only solution is those idea under limelight and investigate. To take the above, how does one prove that all fair people are wise and peaceful while all people black and brown are violent. If that is so, how does one count for Mahatma Gandhi, Martin Luther King Junior, Nelson Mandela, Michael Jackson the list is probably endless. And not to forget that when Mahatma Gandhiji did his nonviolent movements either in India or in South Africa, both black and brown people in millions took part. Similar examples of Martin Luther King Jr. I know and read of so many non-violent civl movements that took place in the U.S. For e.g. Rosa Parks and the Montgomery Bus Boycott. So just based on these examples, one can conclude that at least the part about the fair having exclusive rights to being fair and noble is not correct. Now as far as violence goes, while every race, every community has had done violence in the past or been a victim of the same. So no one is and can be blameless, although in light of the above statement, the question can argumentated as to who were the Vikings? Both popular imagination and serious history shares stories about Vikings. The Vikings were somewhat nomadic in nature even though they had permanent settlements but even then they went on raids, raped women, captured both men and women and sold them at slaves. So they are what pirates came to be, but not the kind Hollywood romanticizes about. Europe in itself has been a tale in conflict since time immemorial. It is only after the formation of EU that most of these countries stopped fighting each other From a historical point perspective, it is too new. So even the part of fair being non-violent dies in face of this evidence. I could go on but this is enough on that topic.

Railways and Industrial Action around the World. While I have shared about Railways so many times on this blog, it continues to fascinate me that how people don t understand the first things about Railways. For e.g. Railways is a natural monopoly. What that means is and you can look at all and any type of privatization around the world, you will see it is a monopoly. Unlike the road or Skies, Railways is and would always be limited by infrastructure and the ability to have new infrastructure. Unlike in road or Skies (even they have their limits) you cannot run train services on a whim. At any particular point in time, only a single train could and should occupy a stretch of Railway network. You could have more trains on one line, but then the likelihood of front or rear-end collisions becomes a real possibility. You also need all sorts of good and reliable communications, redundant infrastructure so if one thing fails then you have something in place. The reason being a single train can carry anywhere from 2000 to 5000 passengers or more. While this is true of Indian Railways, Railways around the world would probably have some sort of similar numbers.It is in this light that I share the below videos.
To be more precise, see the fuller video
Now to give context to the recording above, Mike Lynch is the general secretary at RMT. For those who came in late, both UK and the U.S. have been threatened by railway strikes. And the reason for the strikes or threat of strikes is similar. Now from the company perspective, all they care is to invest less and make the most profits that can be given to equity shareholders. At the same time, they have freezed the salaries of railway workers for the last 3 years. While the politicians who were asking the questions, apparently gave themselves raise twice this year. They are asking them to negotiate at 8% while inflation in the UK has been 12.3% and projected to go higher. And it is not only the money. Since the 1980s when UK privatized the Railways, they stopped investing in the infrastructure. And that meant that the UK Railway infrastructure over period of time started getting behind and is even behind say Indian Railways which used to provide most bang for the buck. And Indian Railways is far from ideal. Ironically, most of the operators on UK are nationalized Railways of France, Germany etc. but after the hard Brexit, they too are mulling to cut their operations short, they have too  There is also the EU Entry/Exit system that would come next year. Why am I sharing about what is happening in UK Rail, because the Indian Government wants to follow the same thing, and fooling the public into saying we would do it better. What inevitably will happen is that ticket prices go up, people no longer use the service, the number of services go down and eventually they are cancelled. This has happened both in Indian Railways as well as Airlines. In fact, GOI just recently announced a credit scheme just a few days back to help Airlines stay afloat. I was chatting with a friend who had come down to Pune from Chennai and the round-trip cost him INR 15k/- on that single trip alone. We reminisced how a few years ago, 8 years to be precise, we could buy an Air ticket for 2.5k/- just a few days before the trip and did it. I remember doing/experiencing at least a dozen odd trips via air in the years before 2014. My friend used to come to Pune, almost every weekend because he could afford it, now he can t do that. And these are people who are in the above 5-10% of the population. And this is not just in UK, but also in the United States. There is one big difference though, the U.S. is mainly a freight carrier while the UK Railway Operations are mostly passenger based. What was and is interesting that Scotland had to nationalize their services as they realized the Operators cannot or will not function when they were most needed. Most of the public even in the UK seem to want a nationalized rail service, at least their polls say so. So, it would definitely be interesting to see what happens in the UK next year. In the end, I know I promised to share about books, but the above incidents have just been too fascinating to not just share the news but also share what I think about them. Free markets function good where there is competition, for example what is and has been happening in China for EV s but not where you have natural monopolies. In all Railway privatization, you have to handover the area to one person, then they have no motivation. If you have multiple operators, then there would always be haggling as to who will run the train and at what time. In either scenario, it doesn t work and raises prices while not delivering anything better  I do take examples from UK because lot of things are India are still the legacy of the British. The whole civil department that was created in 1953 is/was a copy of the British civil department at that time and it is to this day. P.S. Just came to know that the UK Chancellor Kwasi Kwarteng was just sacked as UK Chancellor. I do commend Truss for facing the press even though she might be dumped a week later unlike our PM who hasn t faced a single press conference in the last 8 odd years.

https://www.youtube.com/watch?v=oTP6ogBqU7of The difference in Indian and UK politics seems to be that the English are now asking questions while here in India, most people are still sleeping without a care in the world. Another thing to note Minidebconf Palakkad is gonna happen 12-13th November 2022. I am probably not gonna go but would request everyone who wants to do something in free software to attend it. I am not sure whether I would be of any use like this and also when I get back, it would be an empty house. But for people young and old, who want to do anything with free/open source software it is a chance not to be missed. Registration of the same closes on 1st of November 2022. All the best, break a leg  Just read this, beautifully done.

Simon Josefsson: On language bindings & Relaunching Guile-GnuTLS

The Guile bindings for GnuTLS has been part of GnuTLS since spring 2007 when Ludovic Court s contributed it after some initial discussion. I have been looking into getting back to do GnuTLS coding, and during a recent GnuTLS meeting one topic was Guile bindings. It seemed like a fairly self-contained project to pick up on. It is interesting to re-read the old thread when this work was included: some of the concerns brought up there now have track record to be evaluated on. My opinion that the cost of introducing a new project per language binding today is smaller than the cost of maintaining language bindings as part of the core project. I believe the cost/benefit ratio has changed during the past 15 years: introducing a new project used to come with a significant cost but this is no longer the case, as tooling and processes for packaging have improved. I have had similar experience with Java, C# and Emacs Lisp bindings for GNU Libidn as well, where maintaining them centralized slow down the pace of updates. Andreas Metzler pointed to a similar conclusion reached by Russ Allbery. There are many ways to separate a project into two projects; just copying the files into a new git repository would have been the simplest and was my original plan. However Ludo mentioned git-filter-branch in an email, and the idea of keeping all git history for some of the relevant files seemed worth pursuing to me. I quickly found git-filter-repo which appears to be the recommend approach, and experimenting with it I found a way to filter out the GnuTLS repo into a small git repository that Guile-GnuTLS could be based on. The commands I used were the following, if you want to reproduce things.
$ git clone https://gitlab.com/gnutls/gnutls.git guile-gnutls
$ cd guile-gnutls/
$ git checkout f5dcbdb46df52458e3756193c2a23bf558a3ecfd
$ git-filter-repo --path guile/ --path m4/guile.m4 --path doc/gnutls-guile.texi --path doc/extract-guile-c-doc.scm --path doc/cha-copying.texi --path doc/fdl-1.3.texi
I debated with myself back and forth whether to include some files that would be named the same in the new repository but would share little to no similar lines, for example configure.ac, Makefile.am not to mention README and NEWS. Initially I thought it would be nice to preserve the history for all lines that went into the new project, but this is a subjective judgement call. What brought me over to a more minimal approach was that the contributor history and attribution would be quite strange for the new repository: Should Guile-GnuTLS attribute the work of the thousands of commits to configure.ac which had nothing to do with Guile? Should the people who wrote that be mentioned as contributor of Guile-GnuTLS? I think not. The next step was to get a reasonable GitLab CI/CD pipeline up, to make sure the project builds on some free GNU/Linux distributions like Trisquel and PureOS as well as the usual non-free distributions like Debian and Fedora to have coverage of dpkg and rpm based distributions. I included builds on Alpine and ArchLinux as well, because they tend to trigger other portability issues. I wish there were GNU Guix docker images available for easy testing on that platform as well. The GitLab CI/CD rules for a project like this are fairly simple. To get things out of the door, I tagged the result as v3.7.9 and published a GitLab release page for Guile-GnuTLS that includes OpenPGP-signed source tarballs manually uploaded built on my laptop. The URLs for these tarballs are not very pleasant to work with, and discovering new releases automatically appears unreliable, but I don t know of a better approach. To finish this project, I have proposed a GnuTLS merge request to remove all Guile-related parts from the GnuTLS core. Doing some GnuTLS-related work again felt nice, it was quite some time ago so thank you for giving me this opportunity. Thoughts or comments? Happy hacking!

Gunnar Wolf: Learning some Rust with Lars!

A couple of weeks ago, I read a blog post by former Debian Developer Lars Wirzenius offering a free basic (6hr) course on the Rust language to interested free software and open source software programmers. I know Lars offers training courses in programming, and besides knowing him for ~20 years and being proud to consider us to be friends, have worked with him in a couple of projects (i.e. he is upstream for vmdb2, which I maintain in Debian and use for generating the Raspberry Pi Debian images) He is a talented programmer, and a fun guy to be around. I was admitted to the first cohort of students of this course (please note I m not committing him to run free courses ever again! He has said he would consider doing so, specially to offer a different time better suited for people in Asia). I have wanted to learn some Rust for quite some time. About a year ago, I bought a copy of The Rust Programming Language, the canonical book for learning the language, and started reading it But lacked motivation and lost steam halfway through, and without having done even a simple real project beyond the simple book exercises. How has this been? I have enjoyed the course. I must admit I did expect it to be more hands-on from the beginning, but Rust is such a large language and it introduces so many new, surprising concepts. Session two did have two somewhat simple hands-on challenges; by saying they were somewhat simple does not mean we didn t have to sweat to get them to compile and work correctly! I know we will finish this Saturday, and I ll still be a complete newbie to Rust. I know the only real way to wrap my head around a language is to actually have a project that uses it And I have some ideas in mind. However, I don t really feel confident to approach an already existing project and start meddling with it, trying to contribute. What does Rust have that makes it so different? Bufff Variable ownership (borrow checking) and values lifetimes are the most obvious salient idea, but they are relatively simple, as you just cannot forget about them. But understanding (and adopting) idiomatic constructs such as the pervasive use of enums, understanding that errors always have to be catered for by using expect() and Result<T,E> It will take some time to be at ease developing in it, if I ever reach that stage! Oh, FWIW Interested related reading. I am halfway through an interesting article, published in March in the Communications of the ACM magazine, titled Here We Go Again: Why Is It Difficult for Developers to Learn Another Programming Language? , that presents an interesting point we don t always consider: If I m a proficient programmer in the X programming language and want to use the Y programming language, learning it Should be easier for me than for the casual bystander, or not? After all, I already have a background in programming! But it happens that mental constructs we build for a given language might hamper our learning of a very different one. This article presents three interesting research questions:
  1. Does cross-language interference occur?
  2. How do experienced programmers learn new languages?
  3. What do experienced programmers find confusing in new languages?
I m far from reaching the conclusions, but so far, it s been a most interesting read. Anyway, to wrap up Thanks Lars! I am learning (although at a pace that is not magically quick But I am aware of the steep learning curve of the language) quite a bit of a very interesting topic, and I m also enjoying the time I spend in front of my computer on Saturday.

Next.

Previous.