Search Results: "cas"

23 May 2023

Russ Allbery: Review: A Half-Built Garden

Review: A Half-Built Garden, by Ruthanna Emrys
Publisher: Tordotcom
Copyright: 2022
ISBN: 1-250-21097-6
Format: Kindle
Pages: 340
The climate apocalypse has happened. Humans woke up to the danger, but a little bit too late. Over one billion people died. But the world on the other side of that apocalypse is not entirely grim. The corporations responsible for so much of the damage have been pushed out of society and isolated on their independent "aislands," traded with only grudgingly for the few commodities the rest of the world has not yet learned how to manufacture without them. Traditional governments have largely collapsed, although they cling to increasingly irrelevant trappings of power. In their place arose the watershed networks: a new way of living with both nature and other humans, built around a mix of anarchic consensus and direct democracy, with conservation and stewardship of the natural environment at its core. Therefore, when the aliens arrive near Bear Island on the Potomac River, they're not detected by powerful telescopes and met by military jets. Instead, their waste sets off water sensors, and they're met by the two women on call for alert duty, carrying a nursing infant and backed by the real-time discussion and consensus technology of the watershed's dandelion network. (Emrys is far from the first person to name something a "dandelion network," so be aware that the usage in this book seems unrelated to the charities or blockchain network.) This is a first contact novel, but it's one that skips over the typical focus of the subgenre. The alien Ringers are completely fluent in English down to subtle nuance of emotion and connotation (supposedly due to observation of our radio and TV signals), have translation devices, and in some cases can make our speech sounds directly. Despite significantly different body shapes, they are immediately comprehensible; differences are limited mostly to family structure, reproduction, and social norms. This is Star Trek first contact, not the type more typical of written science fiction. That feels unrealistic, but it's also obviously an authorial choice to jump directly to the part of the story that Emrys wants to write. The Ringers have come to save humanity. In their experience, technological civilization is inherently incompatible with planets. Technology will destroy the planet, and the planet will in turn destroy the species unless they can escape. They have reached other worlds multiple times before, only to discover that they were too late and everyone is already dead. This is the first time they've arrived in time, and they're eager to help humanity off its dying planet to join them in the Dyson sphere of space habitats they are constructing. Planets, to them, are a nest and a launching pad, something to eventually abandon and break down for spare parts. The small, unexpected wrinkle is that Judy, Carol, and the rest of their watershed network are not interested in leaving Earth. They've finally figured out the most critical pieces of environmental balance. Earth is going to get hotter for a while, but the trend is slowing. What they're doing is working. Humanity would benefit greatly from Ringer technology and the expertise that comes from managing closed habitat ecosystems, but they don't need rescuing. This goes over about as well as a toddler saying that playing in the road is perfectly safe. This is a fantastic hook for a science fiction novel. It does exactly what a great science fiction premise should do: takes current concerns (environmentalism, space boosterism, the debatable primacy of humans as a species, the appropriate role of space colonization, the tension between hopefulness and doomcasting about climate change) and uses the freedom of science fiction to twist them around and come at them from an entirely different angle. The design of the aliens is excellent for this purpose. The Ringers are not one alien species; they are two, evolved on different planets in the same system. The plains dwellers developed space flight first and went to meet the tree dwellers, and while their relationship is not entirely without hierarchy (the plains dwellers clearly lead on most matters), it's extensively symbiotic. They now form mixed families of both species, and have a rich cultural history of stories about first contact, interspecies conflicts and cooperation, and all the perils and misunderstandings that they successfully navigated. It makes their approach to humanity more believable to know that they have done first contact before and are building on a model. Their concern for humanity is credibly sincere. The joining of two species was wildly successful for them and they truly want to add a third. The politics on the human side are satisfyingly complicated. The watershed network may have made first contact, but the US government (in the form of NASA) is close behind, attempting to lean on its widely ignored formal power. The corporations are farther away and therefore slower to arrive, but the alien visitors have a damaged ship and need space to construct a subspace beacon and Asterion is happy to offer a site on one of its New Zealand islands. The corporate representatives are salivating at the chance to escape Earth and its environmental regulation for uncontrolled space construction and a new market of trillions of Ringers. NASA's attitude is more measured, but their representative is easily persuaded that the true future of humanity is in space. The work the watershed networks are doing is difficult, uncertain, and involves a lot of sacrifice, particularly for corporate consumer lifestyles. With such an attractive alien offer on the table, why stay and work so hard for an uncertain future? Maybe the Ringers are right. And then the dandelion networks that the watersheds use as the core of their governance and decision-making system all crash. The setup was great; I was completely invested. The execution was more mixed. There are some things I really liked, some things that I thought were a bit too easy or predictable, and several places where I wish Emrys had dug deeper and provided more detail. I thought the last third of the book fizzled a little, although some of the secondary characters Emrys introduces are delightful and carry the momentum of the story when the politics feel a bit lacking. If you tried to form a mental image of ecofeminist political science fiction with 1970s utopian sensibilities, but updated for the concerns of the 2020s, you would probably come very close to the politics of the watershed networks. There are considerably more breastfeedings and diaper changes than the average SF novel. Two of the primary characters are transgender, but with very different experiences with transition. Pronoun pins are an ubiquitous article of clothing. One of the characters has a prosthetic limb. Another character who becomes important later in the story codes as autistic. None of this felt gratuitous; the characters do come across as obsessed with gender, but in a way that I found believable. The human diversity is well-integrated with the story, shapes the characters, creates practical challenges, and has subtle (and sometimes not so subtle) political ramifications. But, and I say this with love because while these are not quite my people they're closely adjacent to my people, the social politics of this book are a very specific type of white feminist collaborative utopianism. When religion makes an appearance, I was completely unsurprised to find that several of the characters are Jewish. Race never makes a significant appearance at all. It's the sort of book where the throw-away references to other important watershed networks includes African ones, and the characters would doubtless try to be sensitive to racial issues if they came up, but somehow they never do. (If you're wondering if there's polyamory in this book, yes, yes there is, and also I suspect you know exactly what culture I'm talking about.) This is not intended as a criticism, just more of a calibration. All science fiction publishing houses could focus only on this specific political perspective for a year and the results would still be dwarfed by the towering accumulated pile of thoughtless paeans to capitalism. Ecofeminism has a long history in the genre but still doesn't show up in that many books, and we're far from exhausting the space of possibilities for what a consensus-based politics could look like with extensive computer support. But this book has a highly specific point of view, enough so that there won't be many thought-provoking surprises if you're already familiar with this school of political thought. The politics are also very earnest in a way that I admit provoked a bit of eyerolling. Emrys pushes all of the political conflict into the contrasts between the human factions, but I would have liked more internal disagreement within the watershed networks over principles rather than tactics. The degree of ideological agreement within the watershed group felt a bit unrealistic. But, that said, at least politics truly matters and the characters wrestle directly with some tricky questions. I would have liked to see more specifics about the dandelion network and the exact mechanics of the consensus decision process, since that sort of thing is my jam, but we at least get more details than are typical in science fiction. I'll take this over cynical libertarianism any day. Gender plays a huge role in this story, enough so that you should avoid this book if you're not interested in exploring gender conceptions. One of the two alien races is matriarchal and places immense social value on motherhood, and it's culturally expected to bring your children with you for any important negotiation. The watersheds actively embrace this, or at worst find it comfortable to use for their advantage, despite a few hints that the matriarchy of the plains aliens may have a very serious long-term demographic problem. In an interesting twist, it's the mostly-evil corporations that truly challenge gender roles, albeit by turning it into an opportunity to sell more clothing. The Asterion corporate representatives are, as expected, mostly the villains of the plot: flashy, hierarchical, consumerist, greedy, and exploitative. But gender among the corporations is purely a matter of public performance, one of a set of roles that you can put on and off as you choose and signal with clothing. They mostly use neopronouns, change pronouns as frequently as their clothing, and treat any question of body plumbing as intensely private. By comparison, the very 2020 attitudes of the watersheds towards gender felt oddly conservative and essentialist, and the main characters get flustered and annoyed by the ever-fluid corporate gender presentation. I wish Emrys had done more with this. As you can tell, I have a lot of thoughts and a lot of quibbles. Another example: computer security plays an important role in the plot and was sufficiently well-described that I have serious questions about the system architecture and security model of the dandelion networks. But, as with decision-making and gender, the more important takeaway is that Emrys takes enough risks and describes enough interesting ideas that there's a lot of meat here to argue with. That, more than getting everything right, is what a good science fiction novel should do. A Half-Built Garden is written from a very specific political stance that may make it a bit predictable or off-putting, and I thought the tail end of the book had some plot and resolution problems, but arguing with it was one of the more intellectually satisfying science fiction reading experiences I've had recently. You have to be in the right mood, but recommended for when you are. Rating: 7 out of 10

21 May 2023

Russ Allbery: Review: The Stone Canal

Review: The Stone Canal, by Ken MacLeod
Series: Fall Revolution #2
Publisher: Tor
Copyright: 1996
Printing: January 2001
ISBN: 0-8125-6864-8
Format: Mass market
Pages: 339
The Stone Canal is a sort of halfway sequel to The Star Fraction. They both take place in the same universe, but the characters are almost entirely disjoint. Half of The Stone Canal happens (mostly) well before the previous book and the other half happens well after it. This book does contain spoilers for the ending of The Star Fraction if one connects the events of the two books correctly (which was a bit harder than I thought it should be), so I would not read them out of order. At the start of The Stone Canal, Jon Wilde wakes up on New Mars beside the titular canal, in the middle of nowhere, accompanied only by a robot that says it made him. Wilde remembers dying on Earth; this new life is apparently some type of resurrection. It's a long walk to Ship City, the center of civilization of a place the robot tells him is New Mars. In Ship City, an android named Dee Model has escaped from her owner and is hiding in a bar. There, she meets an AI abolitionist named Tamara, who helps her flee out the back and down the canal on a boat when Wilde walks into the bar and immediately recognizes her. The abolitionists provide her protection and legal assistance to argue her case for freedom from her owner, a man named Reid. The third thread of the story, and about half the book, is Jon Wilde's life on Earth, starting in 1975 and leading up to the chaotic wars, political fracturing, and revolutions that formed the background and plot of The Star Fraction. Eventually that story turns into a full-fledged science fiction setting, but not until the last 60 pages of the book. I successfully read two books in a Ken MacLeod series! Sadly, I'm not sure I enjoyed the experience. I commented in my review of The Star Fraction that the appeal for me in MacLeod's writing was his reputation as a writer of political science fiction. Unfortunately that's been a bust. The characters are certainly political, in the sense that they profess to have strong political viewpoints and are usually members of some radical (often Trotskyite) organization. There are libertarian anarchist societies and lots of political conflict. But there is almost no meaningful political discussion in any of these books so far. The politics are all tactical or background, and often seem to be created by authorial fiat. For example, New Mars is a sort of libertarian anarchy that somehow doesn't have corporations or a strongman ruler, even though the history (when we finally learn it) would have naturally given rise to one or the other (and has, in numerous other SF novels with similar plots). There's a half-assed explanation for this towards the end of the book that I didn't find remotely believable. Another part of the book describes the formation of the libertarian microstate in The Star Fraction, but never answers a "why" or "how" question I had in the previous book in a satisfying way. Somehow people stop caring about control or predictability or stability or traditional hierarchy without any significant difficulties except external threats, in situations of chaos and disorder where historically humans turn to anyone promising firm structure. It's common to joke about MacLeod winning multiple libertarian Prometheus Awards for his fiction despite being a Scottish communist. I'm finding that much less surprising now that I've read more of his books. Whether or not he believes in it himself, he's got the cynical libertarian smugness and hand-waving down pat. What his characters do care deeply about is smoking, drinking, and having casual sex. (There's more political fire here around opposition to anti-smoking laws than there is about any of the society-changing political structures that somehow fall into place.) I have no objections to any of those activities from a moral standpoint, but reading about other people doing them is a snoozefest. The flashback scenes sketch out enough imagined history to satisfy some curiosity from the previous book, but they're mostly about the world's least interesting love triangle, involving two completely unlikable men and lots of tedious jealousy and posturing. The characters in The Stone Canal are, in general, a problem. One of those unlikable men is Wilde, the protagonist for most of the book. Not only did I never warm to him, I never figured out what motivates him or what he cares about. He's a supposedly highly political person who seems to engage in politics with all the enthusiasm of someone filling out tax forms, and is entirely uninterested in explaining to the reader any sort of coherent philosophical approach. The most interesting characters in this book are the women (Annette, Dee Model, Tamara, and, very late in the book, Meg), but other than Dee Model they rarely get much focus from the story. By far the best part of this book is the last 60 pages, where MacLeod finally explains the critical bridge events between Wilde's political history on earth and the New Mars society. I thought this was engrossing, fast-moving, and full of interesting ideas (at least for a 1990s book; many of them feel a bit stale now, 25 years later). It was also frustrating, because this was the book I wanted to have been reading for the previous 270 pages, instead of MacLeod playing coy with his invented history or showing us interminable scenes about Wilde's insecure jealousy over his wife. It's also the sort of book where at one point characters (apparently uniformly male as far as one could tell from the text of the book) get assigned sex slaves, and while MacLeod clearly doesn't approve of this, the plot is reminiscent of a Heinlein novel: the protagonist's sex slave becomes a very loyal permanent female companion who seems to have the same upside for the male character in question. This was unfortunately not the book I was hoping for. I did enjoy the last hundred pages, and it's somewhat satisfying to have the history come together after puzzling over what happened for 200 pages. But I found the characters tedious and annoying and the politics weirdly devoid of anything like sociology, philosophy, or political science. There is the core of a decent 1990s AI and singularity novel here, but the technology is now rather dated and a lot of other people have tackled the same idea with fewer irritating ticks. Not recommended, although I'll probably continue to The Cassini Division because the ending was a pretty great hook for another book. Followed by The Cassini Division. Rating: 5 out of 10

19 May 2023

Petter Reinholdtsen: wmbusmeters, parse data from your utility meter - nice free software

There is a European standard for reading utility meters like water, gas, electricity or heat distribution meters. The Meter-Bus standard (EN 13757-2, EN 13757-3 and EN 13757 4) provide a cross vendor way to talk to and collect meter data. I ran into this standard when I wanted to monitor some heat distribution meters, and managed to find free software that could do the job. The meters in question broadcast encrypted messages with meter information via radio, and the hardest part was to track down the encryption keys from the vendor. With this in place I could set up a MQTT gateway to submit the meter data for graphing. The free software systems in question, rtl-wmbus to read the messages from a software defined radio, and wmbusmeters to decrypt and decode the content of the messages, is working very well and allowe me to get frequent updates from my meters. I got in touch with upstream last year to see if there was any interest in publishing the packages via Debian. I was very happy to learn that Fredrik hrstr m volunteered to maintain the packages, and I have since assisted him in getting Debian package build rules in place as well as sponsoring the packages into the Debian archive. Sadly we completed it too late for them to become part of the next stable Debian release (Bookworm). The wmbusmeters package just cleared the NEW queue. It will need some work to fix a built problem, but I expect Fredrik will find a solution soon. If you got a infrastructure meter supporting the Meter Bus standard, I strongly recommend having a look at these nice packages. As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

18 May 2023

Antoine Beaupr : A terrible Pixel Tablet

In a strange twist of history, Google finally woke and thought "I know what we need to do! We need to make a TABLET!". So some time soon in 2023, Google will release "The tablet that only Google could make", the Pixel Tablet. Having owned a Samsung Galaxy Tab S5e for a few years, I was very curious to see how this would pan out and especially whether it would be easier to flash than the Samsung. As an aside, I figured I would give that a shot, and within a few days managed to completely brick the device. Awesome. See gts4lvwifi for the painful details of that. In any case, Google made a tablet. I own a Pixel phone and I'm moderately happy with it. It's easy to flash with CalyxOS, maybe this is the promise land of tablets?

Compared with the Samsung But it turns out that the Pixel Tablet pales in comparison with the Samsung tablet, produced 4 years ago, in 2019:
  • it's thicker (8.1mm vs 5.5mm)
  • it's heavier (493g vs 400g)
  • it's not AMOLED (IPS LCD)
  • it doesn't have an SD card reader
  • its camera is worse (8MP vs 13MP, 1080p video instead of 4k)
  • it's more expensive (670EUR vs 410EUR)
What the Pixel tablet has going for it:
  • a slightly more powerful CPU
  • a stylus
  • more storage (128GB or 256GB vs 64GB or 128GB)
  • more RAM (8GB vs 4GB or 6GB)
  • Wifi 6
I guess I should probably wait for the actual device to come out to see reviews and how it stacks up, but so far it's kind of impressive how underwhelming this is. Also note that we're comparing against a very old Samsung tablet here, a fairer comparison might be against the Samsung Galaxy Tab S8. There the sizes are comparable, and the Samsung is more expensive than the Pixel, but then the Pixel has absolutely zero advantages and all the other disadvantages.

The Dock The "Dock" is also worth a little aside. See, the tablet comes with a dock that doubles as a speaker. You can't buy the tablet without the dock. You have to have a dock. I shit you not, actual quote: "Can I purchase a Pixel Tablet without the Charging Speaker Dock? No, you can only purchase the Pixel Tablet with the Charging Speaker Dock." In case you really, really like the dock, "You may purchase additional Charging Speaker Docks separately (coming soon)." And no, they can't all play together, only the dock the tablet is docked into will play audio. The dock is not a Bluetooth speaker, it can only play audio from that one tablet that Google made, this one time. It's also not a battery pack. It's just a charger with speakers in it. Promising e-waste. Again, I hope I'm wrong and that this is going to be a fine tablet. But so far, it looks like it doesn't even come close to whatever Samsung threw over the fence before the apocalypse (remember 2019? were we even born yet?). "The tablet that only Google could make." Amazing. Hopefully no one else gets any bright ideas like this.

15 May 2023

Dirk Eddelbuettel: RcppSimdJson 0.1.10 on CRAN: New Upstream

We are happy to share that the RcppSimdJson package has been updated to release 0.1.10. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is faster than CPU speed as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire at QCon. This release updates the underlying simdjson library to version 3.1.8 (also made today). Otherwise we only made a minor edit to the README and adjusted one tweek for code coverage. The (very short) NEWS entry for this release follows.

Changes in version 0.1.10 (2023-05-14)
  • simdjson was upgraded to version 3.1.8 (Dirk in #85).

Courtesy of my CRANberries, there is also a diffstat report for this release. For questions, suggestions, or issues please use the issue tracker at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

14 May 2023

C.J. Collier: Early Access: Inserting JSON data to BigQuery from Spark on Dataproc

Hello folks! We recently received a case letting us know that Dataproc 2.1.1 was unable to write to a BigQuery table with a column of type JSON. Although the BigQuery connector for Spark has had support for JSON columns since 0.28.0, the Dataproc images on the 2.1 line still cannot create tables with JSON columns or write to existing tables with JSON columns. The customer has graciously granted permission to share the code we developed to allow this operation. So if you are interested in working with JSON column tables on Dataproc 2.1 please continue reading! Use the following gcloud command to create your single-node dataproc cluster: 
IMAGE_VERSION=2.1.1-debian11
REGION=us-west1
ZONE=$ REGION -a
CLUSTER_NAME=pick-a-cluster-name
gcloud dataproc clusters create $ CLUSTER_NAME  \
    --region $ REGION  \
    --zone $ ZONE  \
    --single-node \
    --master-machine-type n1-standard-4 \
    --master-boot-disk-type pd-ssd \
    --master-boot-disk-size 50 \
    --image-version $ IMAGE_VERSION  \
    --max-idle=90m \
    --enable-component-gateway \
    --scopes 'https://www.googleapis.com/auth/cloud-platform'
The following file is the Scala code used to write JSON structured data to a BigQuery table using Spark. The file following this one can be executed from your single-node Dataproc cluster. Main.scala 
import org.apache.spark.sql.functions.col
import org.apache.spark.sql.types. Metadata, StringType, StructField, StructType 
import org.apache.spark.sql. Row, SaveMode, SparkSession 
import org.apache.spark.sql.avro
import org.apache.avro.specific
  val env = "x"
  val my_bucket = "cjac-docker-on-yarn"
  val my_table = "dataset.testavro2"
    val spark = env match  
      case "local" =>
        SparkSession
          .builder()
          .config("temporaryGcsBucket", my_bucket)
          .master("local")
          .appName("isssue_115574")
          .getOrCreate()
      case _ =>
        SparkSession
          .builder()
          .config("temporaryGcsBucket", my_bucket)
          .appName("isssue_115574")
          .getOrCreate()
     
  // create DF with some data
  val someData = Seq(
    Row(""" "name":"name1", "age": 10  """, "id1"),
    Row(""" "name":"name2", "age": 20  """, "id2")
  )
  val schema = StructType(
    Seq(
      StructField("user_age", StringType, true),
      StructField("id", StringType, true)
    )
  )
  val avroFileName = s"gs://$ my_bucket /issue_115574/someData.avro"
  
  val someDF = spark.createDataFrame(spark.sparkContext.parallelize(someData), schema)
  someDF.write.format("avro").mode("overwrite").save(avroFileName)
  val avroDF = spark.read.format("avro").load(avroFileName)
  // set metadata
  val dfJSON = avroDF
    .withColumn("user_age_no_metadata", col("user_age"))
    .withMetadata("user_age", Metadata.fromJson(""" "sqlType":"JSON" """))
  dfJSON.show()
  dfJSON.printSchema
  // write to BigQuery
  dfJSON.write.format("bigquery")
    .mode(SaveMode.Overwrite)
    .option("writeMethod", "indirect")
    .option("intermediateFormat", "avro")
    .option("useAvroLogicalTypes", "true")
    .option("table", my_table)
    .save()
repro.sh: 
#!/bin/bash
PROJECT_ID=set-yours-here
DATASET_NAME=dataset
TABLE_NAME=testavro2
# We have to remove all of the existing spark bigquery jars from the local
# filesystem, as we will be using the symbols from the
# spark-3.3-bigquery-0.30.0.jar below.  Having existing jar files on the
# local filesystem will result in those symbols having higher precedence
# than the one loaded with the spark-shell.
sudo find /usr -name 'spark*bigquery*jar' -delete
# Remove the table from the bigquery dataset if it exists
bq rm -f -t $PROJECT_ID:$DATASET_NAME.$TABLE_NAME
# Create the table with a JSON type column
bq mk --table $PROJECT_ID:$DATASET_NAME.$TABLE_NAME \
  user_age:JSON,id:STRING,user_age_no_metadata:STRING
# Load the example Main.scala 
spark-shell -i Main.scala \
  --jars /usr/lib/spark/external/spark-avro.jar,gs://spark-lib/bigquery/spark-3.3-bigquery-0.30.0.jar
# Show the table schema when we use  bq mk --table  and then load the avro
bq query --use_legacy_sql=false \
  "SELECT ddl FROM $DATASET_NAME.INFORMATION_SCHEMA.TABLES where table_name='$TABLE_NAME'"
# Remove the table so that we can see that the table is created should it not exist
bq rm -f -t $PROJECT_ID:$DATASET_NAME.$TABLE_NAME
# Dynamically generate a DataFrame, store it to avro, load that avro,
# and write the avro to BigQuery, creating the table if it does not already exist
spark-shell -i Main.scala \
  --jars /usr/lib/spark/external/spark-avro.jar,gs://spark-lib/bigquery/spark-3.3-bigquery-0.30.0.jar
# Show that the table schema does not differ from one created with a bq mk --table
bq query --use_legacy_sql=false \
  "SELECT ddl FROM $DATASET_NAME.INFORMATION_SCHEMA.TABLES where table_name='$TABLE_NAME'"
Google BigQuery has supported JSON data since October of 2022, but until now, it has not been possible, on generally available Dataproc clusters, to interact with these columns using the Spark BigQuery Connector. JSON column type support was introduced in spark-bigquery-connector release 0.28.0.

10 May 2023

Freexian Collaborators: Debian Contributions: DEP-17, Debian Reimbursements Web App, and more! (by Utkarsh Gupta, Stefano Rivera)

Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

DEP-17 progress, by Helmut and Emilio We posted a proposal for modifying dpkg to better cope with directory aliasing. After an initial period of silence, the discussion took off, but was mostly diverted to a competing proposal by Luca Boccassi: Do not change dpkg at all, but still move all files affected by aliasing to their canonical location and thus removing the bad effects of aliasing. We facilitated this discussion and performed extensive analysis of this and competing proposals highlighting resulting problems and proposing solutions or workarounds. We performed a detailed analysis of how aliasing affects usage of dpkg-divert, dpkg-statoverride and update-alternatives. Details are available on the debian-dpkg mailinglist thread.

Debian Reimbursements Web App Progress, by Stefano Rivera In a project funded by Freexian s Project Funding initiative, Stefano made some more progress on the Debian Reimbursements Web App. The full workflow can now be exercised, completing the first milestone of the project, the Working Prototype.

DebConf Bursary Team, by Utkarsh Gupta Utkarsh started to prep the bursary team work, gearing up for DebConf, happening in India in September 2023. To learn more about the bursaries team, head to https://wiki.debian.org/Teams/DebConf/Bursaries. For learning how to apply for bursaries, visit https://debconf23.debconf.org/about/bursaries.

Miscellaneous contributions
  • Stefano attended several DebConf planning meetings, and did some work on the DebConf 23 website.
  • Stefano updated distro-info-data to include the release date of Debian bullseye, and added the next Ubuntu release, Mantic Minotour. This required a round of updates to all the stable releases, LTS, and ELTS.
  • Helmut sent patches for 13 cross build failures and filed 104 RC bugs for missing Breaks and Replaces.

6 May 2023

Reproducible Builds: Reproducible Builds in April 2023

Welcome to the April 2023 report from the Reproducible Builds project! In these reports we outline the most important things that we have been up to over the past month. And, as always, if you are interested in contributing to the project, please visit our Contribute page on our website.

General news Trisquel is a fully-free operating system building on the work of Ubuntu Linux. This month, Simon Josefsson published an article on his blog titled Trisquel is 42% Reproducible!. Simon wrote:
The absolute number may not be impressive, but what I hope is at least a useful contribution is that there actually is a number on how much of Trisquel is reproducible. Hopefully this will inspire others to help improve the actual metric.
Simon wrote another blog post this month on a new tool to ensure that updates to Linux distribution archive metadata (eg. via apt-get update) will only use files that have been recorded in a globally immutable and tamper-resistant ledger. A similar solution exists for Arch Linux (called pacman-bintrans) which was announced in August 2021 where an archive of all issued signatures is publically accessible.
Joachim Breitner wrote an in-depth blog post on a bootstrap-capable GHC, the primary compiler for the Haskell programming language. As a quick background to what this is trying to solve, in order to generate a fully trustworthy compile chain, trustworthy root binaries are needed and a popular approach to address this problem is called bootstrappable builds where the core idea is to address previously-circular build dependencies by creating a new dependency path using simpler prerequisite versions of software. Joachim takes an somewhat recursive approach to the problem for Haskell, leading to the inadvertently humourous question: Can I turn all of GHC into one module, and compile that? Elsewhere in the world of bootstrapping, Janneke Nieuwenhuizen and Ludovic Court s wrote a blog post on the GNU Guix blog announcing The Full-Source Bootstrap, specifically:
[ ] the third reduction of the Guix bootstrap binaries has now been merged in the main branch of Guix! If you run guix pull today, you get a package graph of more than 22,000 nodes rooted in a 357-byte program something that had never been achieved, to our knowledge, since the birth of Unix.
More info about this change is available on the post itself, including:
The full-source bootstrap was once deemed impossible. Yet, here we are, building the foundations of a GNU/Linux distro entirely from source, a long way towards the ideal that the Guix project has been aiming for from the start. There are still some daunting tasks ahead. For example, what about the Linux kernel? The good news is that the bootstrappable community has grown a lot, from two people six years ago there are now around 100 people in the #bootstrappable IRC channel.

Michael Ablassmeier created a script called pypidiff as they were looking for a way to track differences between packages published on PyPI. According to Micahel, pypidiff uses diffoscope to create reports on the published releases and automatically pushes them to a GitHub repository. This can be seen on the pypi-diff GitHub page (example).
Eleuther AI, a non-profit AI research group, recently unveiled Pythia, a collection of 16 Large Language Model (LLMs) trained on public data in the same order designed specifically to facilitate scientific research. According to a post on MarkTechPost:
Pythia is the only publicly available model suite that includes models that were trained on the same data in the same order [and] all the corresponding data and tools to download and replicate the exact training process are publicly released to facilitate further research.
These properties are intended to allow researchers to understand how gender bias (etc.) can affected by training data and model scale.
Back in February s report we reported on a series of changes to the Sphinx documentation generator that was initiated after attempts to get the alembic Debian package to build reproducibly. Although Chris Lamb was able to identify the source problem and provided a potential patch that might fix it, James Addison has taken the issue in hand, leading to a large amount of activity resulting in a proposed pull request that is waiting to be merged.
WireGuard is a popular Virtual Private Network (VPN) service that aims to be faster, simpler and leaner than other solutions to create secure connections between computing devices. According to a post on the WireGuard developer mailing list, the WireGuard Android app can now be built reproducibly so that its contents can be publicly verified. According to the post by Jason A. Donenfeld, the F-Droid project now does this verification by comparing their build of WireGuard to the build that the WireGuard project publishes. When they match, the new version becomes available. This is very positive news.
Author and public speaker, V. M. Brasseur published a sample chapter from her upcoming book on corporate open source strategy which is the topic of Software Bill of Materials (SBOM):
A software bill of materials (SBOM) is defined as a nested inventory for software, a list of ingredients that make up software components. When you receive a physical delivery of some sort, the bill of materials tells you what s inside the box. Similarly, when you use software created outside of your organisation, the SBOM tells you what s inside that software. The SBOM is a file that declares the software supply chain (SSC) for that specific piece of software. [ ]

Several distributions noticed recent versions of the Linux Kernel are no longer reproducible because the BPF Type Format (BTF) metadata is not generated in a deterministic way. This was discussed on the #reproducible-builds IRC channel, but no solution appears to be in sight for now.

Community news On our mailing list this month: Holger Levsen gave a talk at foss-north 2023 in Gothenburg, Sweden on the topic of Reproducible Builds, the first ten years. Lastly, there were a number of updates to our website, including:
  • Chris Lamb attempted a number of ways to try and fix literal : .lead appearing in the page [ ][ ][ ], made all the Back to who is involved links italics [ ], and corrected the syntax of the _data/sponsors.yml file [ ].
  • Holger Levsen added his recent talk [ ], added Simon Josefsson, Mike Perry and Seth Schoen to the contributors page [ ][ ][ ], reworked the People page a little [ ] [ ], as well as fixed spelling of Arch Linux [ ].
Lastly, Mattia Rizzolo moved some old sponsors to a former section [ ] and Simon Josefsson added Trisquel GNU/Linux. [ ]

Debian
  • Vagrant Cascadian reported on the Debian s build-essential package set, which was inspired by how close we are to making the Debian build-essential set reproducible and how important that set of packages are in general . Vagrant mentioned that: I have some progress, some hope, and I daresay, some fears . [ ]
  • Debian Developer Cyril Brulebois (kibi) filed a bug against snapshot.debian.org after they noticed that there are many missing dinstalls that is to say, the snapshot service is not capturing 100% of all of historical states of the Debian archive. This is relevant to reproducibility because without the availability historical versions, it is becomes impossible to repeat a build at a future date in order to correlate checksums. .
  • 20 reviews of Debian packages were added, 21 were updated and 5 were removed this month adding to our knowledge about identified issues. Chris Lamb added a new build_path_in_line_annotations_added_by_ruby_ragel toolchain issue. [ ]
  • Mattia Rizzolo announced that the data for the stretch archive on tests.reproducible-builds.org has been archived. This matches the archival of stretch within Debian itself. This is of some historical interest, as stretch was the first Debian release regularly tested by the Reproducible Builds project.

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

diffoscope development diffoscope version 241 was uploaded to Debian unstable by Chris Lamb. It included contributions already covered in previous months as well a change by Chris Lamb to add a missing raise statement that was accidentally dropped in a previous commit. [ ]

Testing framework The Reproducible Builds project operates a comprehensive testing framework (available at tests.reproducible-builds.org) in order to check packages and other artifacts for reproducibility. In April, a number of changes were made, including:
  • Holger Levsen:
    • Significant work on a new Documented Jenkins Maintenance (djm) script to support logged maintenance of nodes, etc. [ ][ ][ ][ ][ ][ ]
    • Add the new APT repo url for Jenkins itself with a new signing key. [ ][ ]
    • In the Jenkins shell monitor, allow 40 GiB of files for diffoscope for the Debian experimental distribution as Debian is frozen around the release at the moment. [ ]
    • Updated Arch Linux testing to cleanup leftover files left in /tmp/archlinux-ci/ after three days. [ ][ ][ ]
    • Mark a number of nodes hosted by Oregon State University Open Source Lab (OSUOSL) as online and offline. [ ][ ][ ]
    • Update the node health checks to detect failures to end schroot sessions. [ ]
    • Filter out another duplicate contributor from the contributor statistics. [ ]
  • Mattia Rizzolo:


If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

5 May 2023

Shirish Agarwal: CAT-6, AMD 5600G, Dealerships closing down, TRAI-caller and privacy.

CAT-6 patch cord & ONU Few months back I was offered a fibre service. Most of the service offering has been using Chinese infrastructure including the ONU (Optical Network Unit). Wikipedia doesn t have a good page on ONU hence had to rely on third-party sites. FS (a name I don t really know) has some (good basic info. on ONU and how it s part and parcel of the whole infrastructure. I also got an ONT (Optical Network Terminal) but it seems to be very basic and mostly dumb. I used the old CAT-6 cable ( a decade old) to connect them and it worked for couple of months. Had to change it, first went to know if a higher cable solution offered themselves. CAT-7 is there but not backward compatible. CAT-8 is the next higher version but apparently it s expensive and also not easily bought. I did quite a few tests on CAT-6 and the ONU and it conks out at best 1 mbps which is still far better than what I am used to. CAT-8 are either not available or simply too expensive for home applications atm. A good summary of CAT-8 and what they stand for can be found here. The networking part is hopeless as most consumer facing CPU s and motherboards don t even offer 10 mbps, so asking anything more is just overkill without any benefit. Which does bring me to the next question, something that I may do in a few months or a year down the road. Just to clarify they may say it is 100 mbps or even 1 Gbps but that s plain wrong.

AMD APU, Asus Motherboard & Dealerships I had been thinking of an AMD APU, could wait a while but sooner or later would have to get one. I got quoted an AMD Ryzen 3 3200G with an Asus A320 Motherboard for around 14k which kinda looked steep to me. Quite a few hardware dealers whom I had traded, consulted over years simply shut down. While there are new people, it s much more harder now to make relationships (due to deafness) rather than before. The easiest to share which was also online was pcpartpicker.com that had an Indian domain now no longer available. The number of offline brick and mortar PC business has also closed quite a bit. There are a few new ones but it takes time and the big guys have made more of a killing. I was shocked quite a bit. Came home and browsed a bit and was hit by this. Both AMD and Intel PC business has taken a beating. AMD a bit more as Intel still holds part of the business segment as traditionally been theirs. There have been proofs and allegations of bribing in the past (do remember the EU Antitrust case against Intel for monopoly) but Intel s own cutting corners with the Spectre and Meltdown flaws hasn t helped its case, nor the suits themselves. AMD on the other hand under expertise of Lisa Su has simply grown strength by strength. Inflation and Profiteering by other big companies has made the outlook for both AMD and Intel a bit lackluster. AMD is supposed to show Zen5 chips in a few days time and the rumor mill has been ongoing. Correction Not few days but 2025. Personally, I would be happy with maybe a Ryzen 5600G with an Asus motherboard. My main motive whenever I buy an APU is not to hit beyond 65 TDP. It s kinda middle of the road. As far as what I could read this year and next year we could have AM4+ or something like those updates, AM5 APU s, CPU s and boards are slated to be launched in 2025. I did see pcpricetracker and it does give idea of various APU prices although have to say pcpartpicker was much intuitive to work with than the above. I just had my system cleaned couple of months so touchwood I should be able to use it for another couple of years or more before I have to get one of these APU s and do hope they are worth it. My idea is to use that not only for testing various softwares but also delve a bit into VR if that s possible. I did read a bit about deafness and VR as well. A good summary can be found here. I am hopeful that there may be few people in the community who may look and respond to that. It s crucial.

TRAI-caller, Privacy 101& Element. While most of us in Debian and FOSS communities do engage in privacy, lots of times it s frustrating. I m always looking for videos that seek to share that view why Privacy is needed by individuals and why Governments and other parties hate it. There are a couple of basic Youtube Videos that does explain the same quite practically.
Now why am I sharing the above. It isn t that people do not privacy and how we hold it dear. I share it because GOI just today blocked Element. While it may be trivial for us to workaround the issues, it does tell what GOI is doing. And it still acts as if surprised why it s press ranking is going to pits. Even our Women Wrestlers have been protesting for a week to just file an FIR (First Information Report) . And these are women who have got medals for the country. More than half of these organizations, specifically the women wrestling team don t have POSH which is a mandatory body supposed to be in every organization. POSH stands for Prevention of Sexual Harassment at Workplace. The gentleman concerned is a known rowdy/Goon hence it took almost a week of protest to do the needful I do try not to report because right now every other day we see somewhere or the other the Govt. curtailing our rights and most people are mute Signing out, till later

Valhalla's Things: Hiking Slippers

Posted on May 5, 2023
image When I travel for a few days I don t usually1 bring any other shoe than the ones I m wearing, plus some kind of slippers for use inside hotel / B&B rooms. It s good for not carrying useless weight, but it always leave me with a vague feeling of what if my only shoes break , followed by on a Sunday, when the shops are closed . So I started to think in the general direction of hiking sandals, shoes that are designed to be worn when resting, and lightweight to carry, but are a passable substitute for regular shoes in case of an accident to the main ones, maybe with the help of an extra pair of socks2 (or when crossing fords, but that s not really a usecase I have). My requirements are easier than the ones for real hiking sandals, since I m only going to be walking on paved streets (or at most easy unpaved ones), and the weight considerations are a thing, but not as strict as if I had to carry these on my back while hiking many hours in a day. My first attempt was a pair of hiking sandals from things I already had in my stash, with vibram soles, neoprene padding and polyester webbing. After a couple fixes they sort of worked, but they had a few big issues.
  • While comfortable when worn, the neoprene made the sandals hard to make, as it tended to deform while being assembled.
  • Polyester webbing is slippery. Some strips of hot glue in strategical places helped, but they weren t perfect and in time they are peeling off.
  • Most importantly, to make the sandals stable enough to wear while walking I had to add a strap around the ankle that needs closing: this makes it a bit of a hassle to use the sandals, say, when waking up in the middle of the night for metabolic reasons.
And then, one day I made my linen slippers, and that lead me to think again about the problem: what if I made a pair of slippers with a rubber sole, technical materials and maybe uppers made of net, so that they would be lightweight, breathable and possibly even still suitable in case I ever need to cross a ford. This was also readily attainable from the stash: some polycotton for the sole lining, elastic mesh for the uppers, EVA foam for padding and vibram soles. I decided to assemble most of them by machine, and it was quick and painless (possibly also thanks to the elasticity of the mesh) image For the soles I may have gone a bit overboard with the vibram claw, but:
  • I already had it in the stash;
  • if I need to wear them on an unpaved road, they are going to be suitable;
  • why not?
The soles were glued to the slippers rather than being sewn, as I don t think there is a reasonable way to sew these soles; I hope it won t cause durability issues later on (if it does, there will be an update) the slippers on a kitchen scale As for the finished weight, at 235 g for the pair I thought I could do better, but apparently shoes are considered ultralight if they are around 500 g? Using just one layer of mesh rather than two would probably help, but it would have required a few changes to the pattern, and anyway I don t really to carry them around all day. image I ve also added a loop of fabric (polycotton) to the centre back to be able to hang the slippers to the backpack when wet or dirty; a bit of narrow webbing may have been better, but I didn t have any in my stash. The pattern is the same as that used for the linen slippers, and of course it s released as #FreeSoftWear. I ve worn these for a few days around the home and they worked just fine, except for the fact that I had to re-glue the sole in a few places (but I suspect it was glued badly in the first place, since the other sole had no issues). Right now I have no plans to travel, so I don t know how much I will be able to test these in the next few months, but sooner or later I will (or I ll keep wearing them at home after I ve thoroughly tested the linen ones), and if there are issues I will post them here on the blog (and add a link to this post).
  1. the exception would be when I m also bringing some kind of costume, and even there it s not always true.
  2. and one should always carry an extra pair of clean socks, as they are useful for so many things, as Pratchett reminds us.

4 May 2023

Matthew Garrett: Twitter's e2ee DMs are better than nothing

(Edit 2023-05-10: This has now launched for a subset of Twitter users. The code that existed to notify users that device identities had changed does not appear to have been enabled - as a result, in its current form, Twitter can absolutely MITM conversations and read your messages)

Elon Musk appeared on an interview with Tucker Carlson last month, with one of the topics being the fact that Twitter could be legally compelled to hand over users' direct messages to government agencies since they're held on Twitter's servers and aren't encrypted. Elon talked about how they were in the process of implementing proper encryption for DMs that would prevent this - "You could put a gun to my head and I couldn't tell you. That's how it should be."

tl;dr - in the current implementation, while Twitter could subvert the end-to-end nature of the encryption, it could not do so without users being notified. If any user involved in a conversation were to ignore that notification, all messages in that conversation (including ones sent in the past) could then be decrypted. This isn't ideal, but it still seems like an improvement over having no encryption at all. More technical discussion follows.

For context: all information about Twitter's implementation here has been derived from reverse engineering version 9.86.0 of the Android client and 9.56.1 of the iOS client (the current versions at time of writing), and the feature hasn't yet launched. While it's certainly possible that there could be major changes in the protocol between now launch, Elon has asserted that they plan to launch the feature this week so it's plausible that this reflects what'll ship.

For it to be impossible for Twitter to read DMs, they need to not only be encrypted, they need to be encrypted with a key that's not available to Twitter. This is what's referred to as "end-to-end encryption", or e2ee - it means that the only components in the communication chain that have access to the unencrypted data are the endpoints. Even if the message passes through other systems (and even if it's stored on other systems), those systems do not have access to the keys that would be needed to decrypt the data.

End-to-end encrypted messengers were initially popularised by Signal, but the Signal protocol has since been incorporated into WhatsApp and is probably much more widely used there. Millions of people per day are sending messages to each other that pass through servers controlled by third parties, but those third parties are completely unable to read the contents of those messages. This is the scenario that Elon described, where there's no degree of compulsion that could cause the people relaying messages to and from people to decrypt those messages afterwards.

But for this to be possible, both ends of the communication need to be able to encrypt messages in a way the other end can decrypt. This is usually performed using AES, a well-studied encryption algorithm with no known significant weaknesses. AES is a form of what's referred to as a symmetric encryption, one where encryption and decryption are performed with the same key. This means that both ends need access to that key, which presents us with a bootstrapping problem. Until a shared secret is obtained, there's no way to communicate securely, so how do we generate that shared secret? A common mechanism for this is something called Diffie Hellman key exchange, which makes use of asymmetric encryption. In asymmetric encryption, an encryption key can be split into two components - a public key and a private key. Both devices involved in the communication combine their private key and the other party's public key to generate a secret that can only be decoded with access to the private key. As long as you know the other party's public key, you can now securely generate a shared secret with them. Even a third party with access to all the public keys won't be able to identify this secret. Signal makes use of a variation of Diffie-Hellman called Extended Triple Diffie-Hellman that has some desirable properties, but it's not strictly necessary for the implementation of something that's end-to-end encrypted.

Although it was rumoured that Twitter would make use of the Signal protocol, and in fact there are vestiges of code in the Twitter client that still reference Signal, recent versions of the app have shipped with an entirely different approach that appears to have been written from scratch. It seems simple enough. Each device generates an asymmetric keypair using the NIST P-256 elliptic curve, along with a device identifier. The device identifier and the public half of the key are uploaded to Twitter using a new API endpoint called /1.1/keyregistry/register. When you want to send an encrypted DM to someone, the app calls /1.1/keyregistry/extract_public_keys with the IDs of the users you want to communicate with, and gets back a list of their public keys. It then looks up the conversation ID (a numeric identifier that corresponds to a given DM exchange - for a 1:1 conversation between two people it doesn't appear that this ever changes, so if you DMed an account 5 years ago and then DM them again now from the same account, the conversation ID will be the same) in a local database to retrieve a conversation key. If that key doesn't exist yet, the sender generates a random one. The message is then encrypted with the conversation key using AES in GCM mode, and the conversation key is then put through Diffie-Hellman with each of the recipients' public device keys. The encrypted message is then sent to Twitter along with the list of encrypted conversation keys. When each of the recipients' devices receives the message it checks whether it already has a copy of the conversation key, and if not performs its half of the Diffie-Hellman negotiation to decrypt the encrypted conversation key. One it has the conversation key it decrypts it and shows it to the user.

What would happen if Twitter changed the registered public key associated with a device to one where they held the private key, or added an entirely new device to a user's account? If the app were to just happily send a message with the conversation key encrypted with that new key, Twitter would be able to decrypt that and obtain the conversation key. Since the conversation key is tied to the conversation, not any given pair of devices, obtaining the conversation key means you can then decrypt every message in that conversation, including ones sent before the key was obtained.

(An aside: Signal and WhatsApp make use of a protocol called Sesame which involves additional secret material that's shared between every device a user owns, hence why you have to do that QR code dance whenever you add a new device to your account. I'm grossly over-simplifying how clever the Signal approach is here, largely because I don't understand the details of it myself. The Signal protocol uses something called the Double Ratchet Algorithm to implement the actual message encryption keys in such a way that even if someone were able to successfully impersonate a device they'd only be able to decrypt messages sent after that point even if they had encrypted copies of every previous message in the conversation)

How's this avoided? Based on the UI that exists in the iOS version of the app, in a fairly straightforward way - each user can only have a single device that supports encrypted messages. If the user (or, in our hypothetical, a malicious Twitter) replaces the device key, the client will generate a notification. If the user pays attention to that notification and verifies with the recipient through some out of band mechanism that the device has actually been replaced, then everything is fine. But, if any participant in the conversation ignores this warning, the holder of the subverted key can obtain the conversation key and decrypt the entire history of the conversation. That's strictly worse than anything based on Signal, where such impersonation would simply not work, but even in the Twitter case it's not possible for someone to silently subvert the security.

So when Elon says Twitter wouldn't be able to decrypt these messages even if someone held a gun to his head, there's a condition applied to that - it's true as long as nobody fucks up. This is clearly better than the messages just not being encrypted at all in the first place, but overall it's a weaker solution than Signal. If you're currently using Twitter DMs, should you turn on encryption? As long as the limitations aren't too limiting, definitely! Should you use this in preference to Signal or WhatsApp? Almost certainly not. This seems like a genuine incremental improvement, but it'd be easy to interpret what Elon says as providing stronger guarantees than actually exist.

comment count unavailable comments

Emanuele Rocca: UEFI Secure Boot on the Raspberry Pi

UPDATE: this post unexpectedly ended up on Hacker News and I received a lot of comments. The two most important points being made are (1) that Secure Boot on the RPi as described here is not actually truly secure. An attacker who successfully gained root could just mount the firmware partition and either add their own keys to the EFI variable store or replace the firmware altogether with a malicious one. (2) The TianCore firmware cannot be used instead of the proprietary blob as I mentioned. What truly happens is that the proprietary blob is loaded onto the VideoCore cores, then TianoCore is loaded onto the ARM cores. Thanks for the corrections.
A port of the free software TianoCore UEFI firmware can be used instead of the proprietary boot blob to boot the Raspberry Pi. This allows to install Debian on the RPi with the standard Debian Installer, and it also makes it possible to use UEFI Secure Boot. Note that Secure Boot had been broken on arm64 for a while, but it s now working in Bookworm!.

Debian Installer UEFI boot
To begin, you ll need to download the appropriate firmware files for the RPi3 or RPi4. I ve got a Raspberry Pi 3 Model B+ myself, so the rest of this document will assume an RPi3 is being installed.
Plug the SD card you are going to use as the RPi storage device into another system. Say it shows up as /dev/sdf. Then: 
# Create an msdos partition table
$ sudo parted --script /dev/sdf mklabel msdos
# Create, format, and label a 10M fat32 partition
$ sudo parted --script /dev/sdf mkpart primary fat32 0% 10M
$ sudo mkfs.vfat /dev/sdf1
$ sudo fatlabel /dev/sdf1 RPI-FW
# Get the UEFI firmware onto the SD card
$ sudo mount /dev/sdf1 /mnt/data/
$ sudo unzip Downloads/RPi3_UEFI_Firmware_v1.38.zip -d /mnt/data/
$ sudo umount /mnt/data
At this point, the SD card can be used to boot the RPi, and you ll get a UEFI firmware.
Download the Bookworm RC 2 release of the installer, copy it to a USB stick as described in the Installation Guide, and boot your RPi from the stick. If for some reason booting from the stick does not happen automatically, enter the firmware interface with ESC and choose the USB stick from Boot Manager.
Proceed with the installation as normal, paying attention not to modify the firmware partition labeled RPI-FW. I initially thought it would be nice to reuse the firmware partition as ESP partition as well. However, setting the esp flag on makes the RPi unbootable. Either configuring the partition as ESP in debian-installer, or manually with sudo parted --script /dev/sda set 1 esp on, breaks boot. In case you accidentally do that, set it back to off and the edk2 firmware will boot again.
What I suggest doing in terms of partitioning is: (1) leave the 10M partition created above for the firmware alone, and (2) create another 512M or so ESP partition for EFI boot.
The installation should go smoothly till the end, but rebooting won t work. Doh. This is because of an important gotcha: the Raspberry Pi port of the TianoCore firmware we are using does not support setting UEFI variables persistently from a "High Level Operating System (HLOS)", which is the debian-installer in our case. Persistently is the keyword there: variables can be set and modified regularly with efibootmgr or otherwise, but crucially the modifications do not survive reboot. However, changes made from the firmware interface itself are persistent. So enter the firmware with ESC right after booting the RPi, select Boot Maintenance Manager Boot Options Add Boot Option Your SD card Your ESP partition EFI debian shimaa64.efi. Choose a creative name for your boot entry (eg: "debian"), save and exit the firmware interface. Bookworm should be booting fine at this point!

Enabling Secure Boot
Although the TianoCore firmware does support Secure Boot, there are no keys enrolled by default. To add the required keys, copy PK-0001.der, DB-0001.der, DB-0002.der, KEK-0001.der, and KEK-0002.der to a FAT32 formatted USB stick.
Here s a summary of the Subject field for each of the above: 
PK-0001.der.pem
        Subject: O = Debian, CN = Debian UEFI Secure Boot (PK/KEK key), emailAddress = debian-devel@lists.debian.org
DB-0001.der.pem
        Subject: C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, CN = Microsoft Windows Production PCA 2011
DB-0002.der.pem
        Subject: C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, CN = Microsoft Corporation UEFI CA 2011
KEK-0001.der.pem
        Subject: O = Debian, CN = Debian UEFI Secure Boot (PK/KEK key), emailAddress = debian-devel@lists.debian.org
KEK-0002.der.pem
        Subject: C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, CN = Microsoft Corporation KEK CA 2011
Plug the stick into the RPi, boot and enter the firmware interface with ESC. Select Device Manager Secure Boot Configuration Secure Boot Mode choose Custom Mode Custom Secure Boot Options PK Options Enroll PK choose PK-0001.der. Do the same for DB Options, this time choose DB-0001.der and DB-0002.der. As you may have guessed by now, the same must be done for KEK Options, but adding KEK-0001.der and KEK-0002.der. Save, exit, reboot. If everything went well, your RPi now has booted with Secure Boot enabled.
See https://wiki.debian.org/SecureBoot for the details on how to check whether Secure Boot has been enabled correctly and much more.

2 May 2023

Neil Williams: Carrying Grief

This isn't a book review, although the reason that I am typing this now is because of a book, You Are Not Alone: from the creator and host of Griefcast, Cariad Lloyd, ISBN: 978-1526621870 and I include a handful of quotes from Cariad where there is really no better way of describing things. Many people experience death for the first time as a child, often relating to a family pet. Death is universal but every experience of death is unique. One of the myths of grief is the idea of the Five Stages but this is a misinterpretation. Denial, Anger, Bargaining, Depression and Acceptance represent the five stage model of death and have nothing to do with grief. The five stages were developed from studying those who are terminally ill, the dying, not those who then grieve for the dead person and have to go on living without them. Grief is for those who loved the person who has died and it varies between each of those people just as people vary in how they love someone. The Five Stages end at the moment of death, grief is what comes next and most people do not grieve in stages, it can be more like a tangled knot. Death has a date and time, so that is why the last stage of the model is Acceptance. Grief has no timetable, those who grieve will carry that grief for the rest of their lives. Death starts the process of grief in those who go on living just as it ends the life of the person who is loved. "Grief eases and changes and returns but it never disappears.". I suspect many will have already stopped reading by this point. People do not talk about death and grief enough and this only adds to the burden of those who carry their grief. It can be of enormous comfort to those who have carried grief for some time to talk directly about the dead, not in vague pleasantries but with specific and strong memories. Find a safe place without distractions and talk with the person grieving face to face. Name the dead person. Go to places with strong memories and be there alongside. Talk about the times with that person before their death. Early on, everything about grief is painful and sad. It does ease but it remains unpredictable. Closing it away in a box inside your head (as I did at one point) is like cutting off a damaged limb but keeping the pain in a box on the shelf. You still miss the limb and eventually, the box starts leaking. For me, there were family pets which died but my first job out of university was to work in hospitals, helping the nurses manage the medication regimen and providing specialist advice as a pharmacist. It will not be long in that environment before everyone on the ward gets direct experience of the death of a person. In some ways, this helped me to separate the process of death from the process of grief. I cared for these people as patients but these were not my loved ones. Later, I worked in specialist terminal care units, including providing potential treatments as part of clinical trials. Here, it was not expected for any patient to be discharged alive. The more aggressive chemotherapies had already been tried and had failed, this was about pain relief, symptom management and helping the loved ones. Palliative care is not just about the patient, it involves helping the loved ones to accept what is happening as this provides comfort to the patient by closing the loop. Grief is stressful. One of the most common causes of personal stress is bereavement. The death of your loved one is outside of your control, it has happened, no amount of regret can change that. Then come all the other stresses, maybe about money or having somewhere to live as a result of what else has changed after the death or having to care for other loved ones. In the early stages, the first two years, I found it helpful to imagine my life as a box containing a ball and a button. The button triggers new waves of pain and loss each time it is hit. The ball bounces around the box and hits the button at random. Initially, the button is large and the ball is enormous, so the button is hit almost constantly. Over time, both the button and the ball change size. Starting off at maximum, initially there is only one direction of change. There are two problems with this analogy. First is that the grief ball has infinite energy which does not happen in reality. The ball may get smaller and the button harder to hit but the ball will continue bouncing. Secondly, the life box is not a predictable shape, so the pattern of movement of the ball is unpredictable. A single stress is one thing, but what has happened since has just kept adding more stress for me. Shortly before my father died 5 years ago now, I had moved house. Then, I was made redundant on the day of the first anniversary of my father's death. A year or so later, my long term relationship failed and a few months after that COVID-19 appeared. As the country eased out of the pandemic in 2021, my mother died (unrelated to COVID itself). A year after that, I had to take early retirement. My brother and sister, of course, share a lot of those stressors. My brother, in particular, took the responsibility for organising both funerals and did most of the visits to my mother before her death. The grief is different for each of the surviving family. Cariad's book helped me understand why I was getting frequent ideas about going back to visit places which my father and I both knew. My parents encouraged each of us to work hard to leave Port Talbot (or Pong Toilet locally) behind, in no small part due to the unrestrained pollution and deprivation that is common to small industrial towns across Wales, the midlands and the north of the UK. It wasn't that I wanted to move house back to our ancestral roots. It was my grief leaking out of the box. Yes, I long for mountains and the sea because I'm now living in a remorselessly flat and landlocked region after moving here for employment. However, it was my grief driving those longings - not for the physical surroundings but out of the shared memories with my father. I can visit those memories without moving house, I just need to arrange things so that I can be undisturbed and undistracted. I am not alone with my grief and I am grateful to my friends who have helped whilst carrying their own grief. It is necessary for everyone to think and talk about death and grief. In respect of your own death, no matter how far ahead that may be, consider Advance Care Planning and Expressions of Wish as well as your Will. Talk to people, document what you want. Your loved ones will be grateful and they deserve that much whilst they try to cope with the first onslaught of grief. Talk to your loved ones and get them to do the same for themselves. Normalise talking about death with your family, especially children. None of us are getting out of this alive and we will all leave behind people who will grieve.

1 May 2023

Gunnar Wolf: Scanning heaps of 8mm movies

After my father passed away, I brought home most of the personal items he had, both at home and at his office. Among many, many (many, many, many) other things, I brought two of his personal treasures: His photo collection and a box with the 8mm movies he shot approximately between 1956 and 1989, when he was forced into modernity and got a portable videocassette recorder. I have talked with several friends, as I really want to get it all in a digital format, and while I ve been making slow but steady advances scanning the photo reels, I was particularly dismayed (even though it was most expected most personal electronic devices aren t meant to last over 50 years) to find out the 8mm projector was no longer in working conditions; the lamp and the fans work, but the spindles won t spin. Of course, it is quite likely it is easy to fix, but it is beyond my tinkering abilities and finding photographic equipment repair shops is no longer easy. Anyway, even if I got it fixed, filming a movie from a screen, even with a decent camera, is a lousy way to get it digitized. But almost by mere chance, I got in contact with my cousin Daniel, ho came to Mexico to visit his parents, and had precisely brought with him a 8mm/Super8 movie scanner! It is a much simpler piece of equipment than I had expected, and while it does present some minor glitches (i.e. the vertical framing slightly loses alignment over the course of a medium-length film scanning session, and no adjustment is possible while the scan is ongoing), this is something that can be decently fixed in post-processing, and a scanning session can be split with no ill effects. Anyway, it is quite uncommon a mid-length (5min) film can be done without interrupting i.e. to join a splice, mostly given my father didn t just film, but also edited a lot (this is, it s not just family pictures, but all different kinds of fiction and documentary work he did). So, Daniel lent me a great, brand new, entry-level film scanner; I rushed to scan as many movies as possible before his return to the USA this week, but he insisted he bought it to help preserve our family s memory, and given we are still several cousins living in Mexico, I could keep hold of it so any other of the cousins will find it more easily. Of course, I am thankful and delighted! So, this equipment is a Magnasonic FS81. It is entry-level, as it lacks some adjustment abilities a professional one would surely have, and I m sure a better scanner will make the job faster but it s infinitely superior to not having it! The scanner processes roughly two frames per second (while the nominal 8mm/Super8 speed is 24 frames per second), so a 3 minute film reel takes a bit over 35 minutes And a long, ~20 minute film reel takes Close to 4hr, if nothing gets in your way :- And yes, with longer reels, the probability of a splice breaking are way higher than with a short one not only because there is simply a longer film to process, but also because, both at the unwinding and at the receiving reels, mechanics play their roles. The films don t advance smoothly, but jump to position each frame in the scanner s screen, so every bit of film gets its fair share of gentle tugs. My professional consultant on how and what to do is my good friend Chema Serralde, who has stopped me from doing several things I would regret later otherwise (such as joining spliced tapes with acidic chemical adhesives such as Kola Loka, a.k.a. Krazy Glue even if it s a bit trickier to do it, he insisted me on best using simple transparent tape if I m not buying fancy things such as film-adhesive). Chema also explained me the importance of the loopers (las Lupes in his technical Spanish translation), which I feared increased the likelihood of breaking a bit of old glue due to the angle in which the film gets pulled but if skipped, result in films with too much jumping. Not all of the movies I have are for public sharing Some of them are just family movies, with high personal value, but probably of very little interest to others. But some are! I have been uploading some of the movies, after minor post-processing, to the Internet Archive. Among them: Anyway, I have a long way forward for scanning. I have 20 3min reels, 19 5min reels, and 8 20min reels. I want to check the scanning quality, but I think my 20min reels are mostly processed (we paid for scanning them some years ago). I mostly finished the 3min reels, but might have to go over some of them again due to the learning process. And Well, I m having quite a bit of fun in the process!

Russ Allbery: Review: The Amazing Maurice and His Educated Rodents

Review: The Amazing Maurice and His Educated Rodents, by Terry Pratchett
Series: Discworld #28
Publisher: HarperCollins
Copyright: 2001
Printing: 2008
ISBN: 0-06-001235-8
Format: Mass market
Pages: 351
The Amazing Maurice and His Educated Rodents is the 28th Discworld novel and the first marketed for younger readers. Although it has enough references to establish it as taking place on Discworld, it has no obvious connections with the other books and doesn't rely on any knowledge of the series so far. This would not be a bad place to start with Terry Pratchett and see if his writing style and sense of humor is for you. Despite being marketed as young adult, and despite Pratchett's comments in an afterward in the edition I own that writing YA novels is much harder, I didn't think this was that different than a typical Discworld novel. The two main human characters read as about twelve and there were some minor changes in tone, but I'm not sure I would have immediately labeled it as YA if I hadn't already known it was supposed to be. There are considerably fewer obvious pop culture references than average, though; if that's related, I think I'll prefer Pratchett's YA novels, since I think his writing is stronger when he's not playing reference bingo. Maurice (note to US readers: Maurice is pronounced "Morris" in the UK) is a talking cat and the mastermind of a wandering con job. He, a stupid-looking kid with a flute (Maurice's description), and a tribe of talking rats travel the small towns of Discworld. The rats go in first, making a show of breaking into the food, swimming in the cream, and widdling on things that humans don't want widdled on. Once the townspeople are convinced they have a plague of rats, the kid with the flute enters the town and offers to pipe the rats away for a very reasonable fee. He plays his flute, the rats swarm out of town, and they take their money and move on to the next town. It's a successful life that suits Maurice and his growing hoard of gold very well. If only the rats would stop asking pointed questions about the ethics of this scheme. The town of Bad Blintz is the next on their itinerary, and if the rats have their way, will be the last. Their hope is they've gathered enough money by now to find an island, away from humans, where they can live their own lives. But, as is always the case for one last job in fiction, there's something uncannily wrong about Bad Blintz. There are traps everywhere, more brutal and dangerous ones than they've found in any other town, and yet there is no sign of native, unintelligent rats. Meanwhile, Maurice and the boy find a town that looks wealthy but has food shortages, a bounty on rats that is absurdly high, and a pair of sinister-looking rat-catchers who are bringing in collections of rat tails that look suspiciously like bootlaces. The mayor's daughter discovers Maurice can talk and immediately decides she has to take them in hand. Malicia is very certain of her own opinions, not accustomed to taking no for an answer, and is certain that the world follows the logic of stories, even if she has to help it along. This is truly great stuff. I think this might be my favorite Discworld novel to date, although I do have some criticisms that I'll get to in a moment. The best part are the rats, and particularly the blind philosopher rat Dangerous Beans and his assistant Peaches. In the middle of daring infiltration of the trapped sewers in scenes reminiscent of Mission: Impossible, the rats are also having philosophical arguments. They've become something different than the unaltered rats that they call the keekees, but what those differences mean is harder to understand. The older rats are not happy about too many changes and think the rats should keep acting like rats. The younger ones are discovering that they're afraid of shadows because now they understand what the shadows hint at. Dangerous Beans is trying to work out a writing system so that they can keep important thoughts. One of their few guides is a children's book of talking animals, although they quickly discover that the portrayed clothing is annoyingly impractical. But as good as the rats are, Maurice is nearly as much fun in an entirely different way. He is unapologetically out for himself, streetwise and canny in a way that feels apt for a cat, gets bored and mentally wanders off in the middle of conversations, and pretends to agree with people when that's how he can get what he wants. But he also has a weird sense of loyalty and ethics that only shows up when something is truly important. It's a variation on the con man with a heart of gold, but it's a very well-done variation that weaves in a cat's impatience with and inattention to anything that doesn't directly concern them. I was laughing throughout the book. Malicia is an absolute delight, the sort of character who takes over scenes through sheer force of will, and the dumb-looking kid (whose name turns out to be Keith) is a perfect counterbalance: a laid-back, quiet boy who just wants to play his music and is almost entirely unflappable. It's such a great cast. The best part of the plot is the end. I won't spoil it, so I'll only say that Pratchett has the characters do the work on the aftermath that a lot of books skip over. He doesn't have any magical solutions for the world's problems, but he's so very good at restoring one's faith that maybe sometimes those solutions can be constructed. My one complaint with this book is that Pratchett introduces a second villain, and while there are good in-story justifications for it and it's entangled with the primary plot, he added elements of (mild) supernatural horror and evil that I thought were extraneous and unnecessary. He already had enough of a conflict set up without adding that additional element, and I think it undermined the moral complexity of the story. I would have much rather he kept the social dynamics of the town at the core of the story and used that to trigger the moments of sacrifice and philosophy that made the climax work. The Discworld books by this point have gotten very good, but each book seems to have one element like that where it felt like Pratchett took the easy way out of a plot corner or added some story element that didn't really work. I feel like the series is on the verge of having a truly great book that rises above the entire series to date, but never quite gets there. That caveat aside, I thoroughly enjoyed this and had trouble putting it down. Mrs. Frisby and the Rats of Nimh was one of my favorite books as a kid, and this reminded me of it in some good ways (enough so that I think some of the references were intentional). Great stuff. If you were to read only one Discworld book and didn't want to be confused by all the entangled plot threads and established characters, I would seriously consider making it this one. Recommended. Followed by Night Watch in publication order. There doesn't appear to be a direct plot sequel, more's the pity. Rating: 8 out of 10

30 April 2023

Russell Coker: Links April 2023

Cory Doctorow has an insightful article Gig Work is the Opposite of Steampunk [1] about the horrors that companies like Amazon are forcing on their employees. Valerie Aurora and Leigh Honeywell wrote an insightful article about the al Capone theory of sexual harassment [2]. Why people who sexually harass others usually perform other anti-social activity that is also easier to prosecute. The IEEE has an interesting article about using ML for parts of the CPU design process, both the technical issues and the controversy about competing ideas which is probably caused by sexism [3]. Love and taxes are forever my heart is a line from an anime dating sim game that prepares US taxes [4]. Unfortunately it was removed from Steam. The existence of the game is a weird social commentary and removing the game because you can t have an anime hottie do taxes is bizarre but also understandable given liability issues. There s no mention in the review of whether male hotties are available for people who prefer dating guys. As an aside my accountant looks like he is allergic to exercise The Killdozer Book web site (which has an invalid SSL certificate so you have to click on advanced in Chrome to get to the content) has an insightful article debunking some of the stories about the Killdozer [5]. He wasn t some sort of hero of freedom, he was just a jerk who reneged on a deal hoping to get more money, thought that laws shouldn t apply to him, and killed himself because of it. Apparently some big tech companies are knowingly hiring people to not work unlike the usual large corporate case of unknowingly hiring people to not work [6]. Silicon Valley is a good TV show, and it s apparently realistic. Ron Garrett wrote in insightful blog post about theoretical attacks on Bitcoin and how Bitcoin could be used [7]. The conclusion is not good for Bitcoin. Compiler Explorer is a program that shows how various C++ compilers produce assembly code for various architectures, this site hosts the main active instance [8]. There are other instances, here is an instance that produces code for the Ruzzian Elbrus architecture [9]. The Elbrus Wikipedia page is interesting [10]. Apparently the Ruzzians don t want this information to be spread, LOL. The Smithsonian Magazine has an interesting article about pet parrots being taught to video call each other [11]. Apparently parrots are social animals and can develop psychological problems if kept alone, so the video calls can be good for them. Also the owners had to monitor the chats to ensure that they played nicely together, just like play-dates for kids! Phoronix has an amusing article about the drama regarding the AMD Spectral Chicken bit in the Linux kernel source [12]. This page listing bad free software licenses is amusing [13]. The ACS has an interesting article about how Samsung fakes photos of the moon and presumably could fake other photos of notable objects that don t change [14]. The way that they proved the forgery was interesting.
Related posts:
  1. Links January 2023 The Intercept has an amusing and interesting article about senior...
  2. Links March 2023 Interesting paper about a plan for eugenics in dogs with...
  3. Links Aug 2022 Armor is an interesting technology from Manchester University for stopping...

Matthew Palmer: dev-dependencies and Rust's unused_crate_dependencies lint

I m in the process of getting super-strict about the code quality of cretrit, the comparison-revealing encryption library that underlies the queryable encryption of the Enquo project. While I m going to write a whole big thing about Rust linting in the future, I bumped across a rather gnarly problem that I thought was worth sharing separately. The problem, in short, is that the unused_crate_dependencies lint interacts badly with crates that are only needed for benchmarking, such as (in my case) criterion. Rust has a whole bucketload of lints that can help your codebase adhere to certain standards, by warning (or exploding) if the problem is detected. The unused_crate_dependencies lint, as the name suggests, gets snippy when there s a crate listed in your Cargo.toml that doesn t appear to be used anywhere. All well and good so far. However, while Rust has the ability to specify crates needed for running the testsuite (the [dev-dependencies] section of Cargo.toml) separately from crates needed for actually using this thing ([dependencies]), it doesn t have a way to specify crates needed for running the benchmarks . That is a problem when you re using something like criterion for benchmarking, because it doesn t get refered to at all in your project code it only gets used in the benchmarks. When building your codebase for running the test suite, the compiler sees that you ve specified criterion as one of your testsuite dependencies , but it never gets used in your testsuite. This causes the unused_crate_dependencies lint to lose its tiny mind, and make your build look ugly (or fail). Thankfully, the solution is very simple, once you know the trick (this could be the unofficial theme song of the entire Rust ecosystem). You need to refer to the criterion crate somewhere in the code that gets built during the testsuite. The lint tells you most of what you need to do (like most things Rust, it tries hard to be helpful), but since it s a development dependency, you need a little extra secret sauce. All you need to do is add these two lines to the bottom of your src/lib.rs (or src/main.rs for a binary crate): 
#[cfg(test)]
use criterion as _;
For the less Rust-literate, this means when the build-time configuration flag test is set, import the criterion crate, but don t, like, allow it to actually be referred to . This is enough to make the lint believe that the dependency is being used, and making it only happen when the test build-time config flag is set avoids the ugliness of it trying to refer to the crate during regular builds (which would fail, because criterion is only a dev-dependency). Simple? Yes. Did it take me a lot of skull-sweat to figure out? You betcha. That s why I m writing it down even if everyone else already knows this, at least future Matt will find this post next time he trips over this problem.

29 April 2023

Russell Coker: Write a blog post in the style of Russell Coker

Feeling a bit bored I asked ChatGPT Write a blog post in the style of Russell Coker and the result is in the section below. I don t know if ChatGPT knows that the person asking the question is the same as the person being asked about. If a human had created that I d be certain that great computer scientist and writer was an attempt at flattery, for a machine I m not sure. I have not written a single book, but I expect that in some alternate universe some version of me has written several. I don t know if humans would describe my writing as being known for clarity, precision, and depth . I would not be surprised if no-one else wrote about it so I guess I m forced to read what he wrote would be a more common response. The actual article part doesn t seem to be in my style at all. Firstly it s very short at only 312 words, while I have written some short posts most of them are much longer. To find this out I did some MySQL queries to get the lengths of posts (I used this blog post as inspiration [1]). Note that multiple sequential spaces counts as multiple words. 
# get post ID and word count
SELECT id, LENGTH(post_content) - LENGTH(REPLACE(REPLACE(REPLACE(REPLACE(post_content, "\r", ""), "\n", ""), "\t", ""), " ", "")) + 1 AS wordcount FROM wp_posts where post_status = 'publish' and post_type='post';
# get average word count
SELECT avg(LENGTH(post_content) - LENGTH(REPLACE(REPLACE(REPLACE(REPLACE(post_content, "\r", ""), "\n", ""), "\t", ""), " ", "")) + 1) FROM wp_posts where post_status = 'publish' and post_type='post';
# get the first posts by length
SELECT id, LENGTH(post_content) - LENGTH(REPLACE(REPLACE(REPLACE(REPLACE(post_content, "\r", ""), "\n", ""), "\t", ""), " ", "")) + 1 AS wordcount, post_content FROM wp_posts where post_status = 'publish' and post_type='post' ORDER BY wordcount limit 10;
# get a count of the posts less than 312 words
SELECT count(*) from wp_posts where (LENGTH(post_content) - LENGTH(REPLACE(REPLACE(REPLACE(REPLACE(post_content, "\r", ""), "\n", ""), "\t", ""), " ", "")) + 1) < 312 and post_status = 'publish' and post_type='post';
# get a count of all posts
select count(*) from wp_posts where post_status = 'publish' and post_type='post';
It turns out that there are 333/1521 posts that are less than 312 words and the average length is 665 words. Of the shortest posts a large portion were written before Twitter became popular and had the sort of unimportant content that Twitter is good for. It seems ironic that ChatGPT says that I'm known for "depth" and then writes a short post allegedly in my style. As for the actual content of the "post", it's not something that I greatly disagree with but it's definitely not something I'd write. If some manager or marketing person wrote something like this and asked my opinion I'd probably suppress a sigh and say it's OK for it's target market. I think that free software is better in many ways and for many reasons and that separately from the issue of merit free software is important because of the ways that it is used. We could debate various issues related to the quality of free software vs proprietary software but even people who don't believe that free software is of high quality have to admit that software that runs most mobile phones and most web servers is important. As much as I dislike Windows most desktop PCs and laptops run it so I can't claim it to be unimportant. So my first disagreement with the content is the issue of importance vs quality. One thing that ChatGPT is known for is weak arguments that appear convincing. "This diversity often leads to better software, as bugs are found and fixed faster, and new features are added more quickly" could be opposed by "the payment for proprietary software often leads to better software, as bugs are found and fixed faster, and new features are added more quickly". I think that any case where "feature of option A gives this list of good results" can be rewritten as "feature of the opposite of option A gives the same list of good results" and be equally convincing is a poor argument. Also any argument that seems to rest on "free software is all written by hobbyists" is ignoring the demands of users who need things done on a non-hobby timeframe, the many profitable companies that develop and support free software, and the employment status of a large portion of my Facebook friends (many of whom are paid for ~2000 hours a year of free software development and many more of whom have paid free software development as part of their work supporting the use of free software). Really the entire blog post it wrote seems to me like something a human might write if their boss assigned them to write about the benefits of free software, and that they would then write a new article saying the exact opposite if their boss asked them to. Finally I prefer the term "free software" to "open source software". The difference doesn't seem important to some people, but any being that doesn't know the difference won't do well at impersonating me. ChatGPT Result Hello everyone, I'm ChatGPT, and today I'm going to try my best to write a blog post in the style of the great computer scientist and writer, Russell Coker. Russell Coker is a renowned software engineer and writer, who is best known for his contributions to the Linux community, especially the Debian project. He has authored several books and articles on topics ranging from security to software engineering, and his writing style is known for its clarity, precision, and depth. So, without further ado, let's dive into the topic of this blog post, which is the importance of open-source software. Open-source software is software whose source code is available for anyone to inspect, modify, and distribute. This means that anyone can use, study, and improve the software, without any restrictions or fees. Open-source software has been around for a long time, but it has become more prevalent and important in recent years, thanks to the rise of the internet and the digital economy. Open-source software has several benefits, both for individuals and for society as a whole. One of the most significant benefits is that it fosters innovation and collaboration. When software is open-source, it means that anyone can contribute to it, which leads to a diverse range of perspectives and ideas. This diversity often leads to better software, as bugs are found and fixed faster, and new features are added more quickly. Another benefit of open-source software is that it is often more secure than proprietary software. Because the source code is available for anyone to inspect, vulnerabilities and security flaws can be found and fixed faster. This means that open-source software is often more reliable and trustworthy than proprietary software, which may have hidden backdoors or vulnerabilities that can be exploited by attackers. Open-source software is also often more cost-effective than proprietary software. Because it is free to use and distribute, organizations can save money on licensing fees and other costs associated with proprietary software. This can be particularly important for non-profit organizations or small businesses that may not have the resources to invest in expensive software licenses. In conclusion, open-source software is an important part of the digital landscape, and its benefits are numerous. Whether you are a software developer, a business owner, or an individual user, open-source software has something to offer. So, the next time you are looking for software to use, consider open-source options, and help support the collaborative and innovative spirit of the open-source community.
Related posts:
  1. How not to write the way dumb people think smart people write Don Marti has written an amusing and informative little post...
  2. a Heartbeat developer comments on my blog post Alan Robertson (a major contributor to the Heartbeat project) commented...
  3. Guest/Link Post Spam I ve been getting a lot of spam recently from people...

28 April 2023

Enrico Zini: Handling keyboard-like devices

CNC control panel and Bluetooth pedal page turner I acquired some unusual input devices to experiment with, like a CNC control panel and a bluetooth pedal page turner. These identify and behave like a keyboard, sending nice and simple keystrokes, and can be accessed with no drivers or other special software. However, their keystrokes appear together with keystrokes from normal keyboards, which is the expected default when plugging in a keyboard, but not what I want in this case. I'd also like them to be readable via evdev and accessible by my own user. Here's the udev rule I cooked up to handle this use case: 
# Handle the CNC control panel
SUBSYSTEM=="input", ENV ID_VENDOR =="04d9", ENV ID_MODEL =="1203", \
   OWNER="enrico", ENV ID_INPUT =""
# Handle the Bluetooth page turner
SUBSYSTEM=="input", ENV ID_BUS =="bluetooth", ENV LIBINPUT_DEVICE_GROUP =="*/ mac ", ENV ID_INPUT_KEYBOARD ="1" \
   OWNER="enrico", ENV ID_INPUT ="", SYMLINK+="input/by-id/bluetooth- mac -kbd"
SUBSYSTEM=="input", ENV ID_BUS =="bluetooth", ENV LIBINPUT_DEVICE_GROUP =="*/ mac ", ENV ID_INPUT_TABLET ="1" \
   OWNER="enrico", ENV ID_INPUT ="", SYMLINK+="input/by-id/bluetooth- mac -tablet"
The bluetooth device didn't have standard rules to create /dev/input/by-id/ symlinks so I added them. In my own code, I watch /dev/input/by-id with inotify to handle when devices appear or disappear. I used udevadm info /dev/input/event to see what I could use to identify the device. The Static device configuration via udev page of libinput's documentation has documentation on the various elements specific to the input subsystem Grepping rule files in /usr/lib/udev/rules.d was useful to see syntax examples. udevadm test /dev/input/event was invaluable for syntax checking and testing my rule file while working on it. Finally, this is an extract of a quick prototype Python code to read keys from the CNC control panel: 
import libevdev
KEY_MAP =  
    libevdev.EV_KEY.KEY_GRAVE: "EMERGENCY",
    # InputEvent(EV_KEY, KEY_LEFTALT, 1)
    libevdev.EV_KEY.KEY_R: "CYCLE START",
    libevdev.EV_KEY.KEY_F5: "SPINDLE ON/OFF",
    # InputEvent(EV_KEY, KEY_RIGHTCTRL, 1)
    libevdev.EV_KEY.KEY_W: "REDO",
    # InputEvent(EV_KEY, KEY_LEFTALT, 1)
    libevdev.EV_KEY.KEY_N: "SINGLE STEP",
    # InputEvent(EV_KEY, KEY_LEFTCTRL, 1)
    libevdev.EV_KEY.KEY_O: "ORIGIN POINT",
    libevdev.EV_KEY.KEY_ESC: "STOP",
    libevdev.EV_KEY.KEY_KPPLUS: "SPEED UP",
    libevdev.EV_KEY.KEY_KPMINUS: "SLOW DOWN",
    libevdev.EV_KEY.KEY_F11: "F+",
    libevdev.EV_KEY.KEY_F10: "F-",
    libevdev.EV_KEY.KEY_RIGHTBRACE: "J+",
    libevdev.EV_KEY.KEY_LEFTBRACE: "J-",
    libevdev.EV_KEY.KEY_UP: "+Y",
    libevdev.EV_KEY.KEY_DOWN: "-Y",
    libevdev.EV_KEY.KEY_LEFT: "-X",
    libevdev.EV_KEY.KEY_RIGHT: "+X",
    libevdev.EV_KEY.KEY_KP7: "+A",
    libevdev.EV_KEY.KEY_Q: "-A",
    libevdev.EV_KEY.KEY_PAGEDOWN: "-Z",
    libevdev.EV_KEY.KEY_PAGEUP: "+Z",
 
class KeyReader:
    def __init__(self, path: str):
        self.path = path
        self.fd: IO[bytes]   None = None
        self.device: libevdev.Device   None = None
    def __enter__(self):
        self.fd = open(self.path, "rb")
        self.device = libevdev.Device(self.fd)
        return self
    def __exit__(self, exc_type, exc, tb):
        self.device = None
        self.fd.close()
        self.fd = None
    def events(self) -> Iterator[dict[str, Any]]:
        for e in self.device.events():
            if e.type == libevdev.EV_KEY:
                if (val := KEY_MAP.get(e.code)):
                    yield  
                        "name": val,
                        "value": e.value,
                        "sec": e.sec,
                        "usec": e.usec,
Edited: added rules to handle the Bluetooth page turner

Shirish Agarwal: John Grisham s books, Evolution removed from textbooks

Gray Mountain John Grisham I have been perusing John Grisham s books, some read and some re-read again. Almost all of the books that Mr. Grisham wrote are relevant even today. The Gray Mountain talks about how mountain top removal was done in Applachia, the U.S. (South). In fact NASA made a summary which either was borrowed from this book or the author borrowed from NASA, either could be true although seems it might be the former. And this is when GOI just made a new Forest Conservation Bill 2023 which does the opposite. There are many examples of the same, the latest from my own city as an e.g. Vetal Tekdi is and was a haven for people animals, birds all kinds of ecosystem and is a vital lung of the city, one of the few remaining green spaces in the city but BJP wants to commercialize it as it has been doing for everything, I haven t been to the Himalayas since 4-5 years back as I hear the rape of the land daily. Even after Joshimath tragedy, if people are not opening the eyes what can I do The more I say, the more depressed I will become so will leave it for now. In many ways the destruction seems similar to the destruction that happened in Brazil under Jair Bolsonaro. So as can be seen from what I have shared this book was mostly about environment and punitive damages and also how punitive damages have been ceiling in America (South). This was done via lobbying by the coal groups and apparently destroyed people s lives, communities, even whole villages. It also shared how most people called black lung and how those claims had been denied by Coal Companies all the time. And there are hardly any unions. While the book itself is a fiction piece, there is a large amount of truth in it. And that is one of the reasons people write a fiction book. You could write about reality using fictional names and nobody can sue you while you set the reality as it is. In many ways, it is a tell-all.

The Testament John Grisham One of the things I have loved about John Grisham is he understands human condition. In this book it starts with an eccentric billionaire who makes a will which leaves all his property to an illegitimate child who coincidentally also lives in Brazil, she is a missionary. The whole book is about human failings as well as about finding the heir. I am not going to give much detail as the book itself is fun.

The Appeal John Grisham In this, we are introduced to a company that does a chemical spill for decades and how that leads to lobbying and funding Judicial elections. It does go into quite a bit of length how private money coming from Big business does all kind of shady things to get their person elected to the Supreme Court. Sadly, this seems to happen all the time, for e.g. two weeks ago. This piece from the Atlantic also says the same. Again, won t tell as there is a bit of irony at the end of the book.

The Rainmaker John Grisham This in short is about how Insurance Companies stiff people. It s a wonderful story that has all people in grey including our hero. An engaging book that sorta tells how the Insurance Industry plays the game. In India, the companies are safe as they ask for continuance for years together and their object is to delay the hearings till the grandchildren are not alive unlike in the U.S. or UK. It also does tell how more lawyers are there then required. Both of which are the same in my country.

The Litigators John Grisham This one is about Product Liability, both medicines as well as toys for young kids. What I do find funny a bit is how the law in States allows people to file cases but doesn t protect people while in EU they try their best that if there is any controversy behind any medicine or product, they will simply ban it. Huge difference between the two cultures.

Evolution no longer part of Indian Education A few days ago, NCERT (one of the major bodies) that looks into Indian Education due to BJP influence has removed Darwin s Theory of Evolution. You can t even debate because the people do not understand adaptability. So the only conclusion is that Man suddenly appeared out of nowhere. If that is so, then we are the true aliens. They discard the notion that we and Chimpanzees are similar. Then they also have to discard this finding that genetically we are 96% similar.

Next.

Previous.