Search Results: "xam"

18 February 2021

Dirk Eddelbuettel: dang 0.0.13: New intradayMarketMonitor

sp500 intraday monitor A new release of the dang package got to CRAN earlier today, a few months since the last relase. The dang package regroups a few functions of mine that had no other home as for example lsos() from a StackOverflow question from 2009 (!!) is one, this overbought/oversold price band plotter from an older blog post is another. This release adds one function I tweeted about one month ago. It takes a function Josh Ulrich originally tweeted about in November with a reference to this gist. I refactored this into a proper functions and polished a few edges: the data now properly rolls off after a fixed delay (of two days), should work with other symbols (though we both focused on ^GSPC as a free (!!) real-time SP500 index (albeit only during trading hours), properly gaps between trading days and more. You can simply invoke it via
dang::intradayMarketMonitor()
and a chart just like the one here will grow (though there is no state : if you stop it, or reboot, or the plot starts from scratch). The short NEWS entry follows.

Changes in version 0.0.13 (2021-02-17)
  • New function intradayMarketMonitor based on an earlier gist-posted snippet by Josh Ulrich.
  • The CI setup was generalized as a test for 'r-ci' and is used essentially unchanged with three different providers.

Courtesy of my CRANberries, there is a comparison to the previous release. For questions or comments use the issue tracker off the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

17 February 2021

Louis-Philippe V ronneau: What are the incentive structures of Free Software?

When I started my Master's degree in January 2018, I was confident I would be done in a year and half. After all, I only had one year of classes and I figured 6 months to write a thesis would be plenty. Three years later, I'm finally done: the final version of my thesis was accepted on January 22nd 2021. My thesis, entitled What are the incentive structures of Free Software? An economic analysis of Free Software's specific development model, can be found here 1. If you care about such things, both the data and the final document can be built from source with the code in this git repository. Results and analysis My thesis is divided in four main sections:
  1. an introduction to FOSS
  2. a chapter discussing the incentive structures of Free Software (and arguing the so called Tragedy of the Commons isn't inevitable)
  3. a chapter trying to use empirical data to validate the theories presented in the previous chapter
  4. an annex on the various FOSS business models
If you're reading this blog post, chances are you'll find both section 1 and 4 a tad boring, as you might already be familiar with these concepts. Incentives So, why do people contribute to Free Software? Unsurprisingly, it's complicated. Many economists have studied this topic, but for some reason, most research happened in the early 2000s. Although papers don't all agree with each other and most importantly, about the variables' importance, the main incentives2 can be summarized by: Giving weights to these variables is not an easy thing: the FOSS ecosystem is highly heterogeneous and thus, people tend to write FOSS for different reasons. Moreover, incentives tend to shift with time as the ecosystem does. People writing Free Software in the 1990s probably did it for different reasons than people in 2021. These four variables can also be divided in two general categories: extrinsic and intrinsic incentives. Monetary gain expectancy is an extrinsic incentive (its value is delayed and mediated), whereas the three other ones are intrinsic (they have an immediate value by themselves). Empirical analysis Theory is nice, but it's even better when you can back it up with data. Sadly, most of the papers on the economic incentives of FOSS are either purely theoretical, or use sample sizes so small they could as well be. Using the data from the StackOverflow 2018 survey, I thus tried to see if I could somehow confirm my previous assumptions. With 129 questions and more than 100 000 respondents (which after statistical processing yields between 28 000 and 39 000 observations per variable of interest), the StackOverflow 2018 survey is a very large dataset compared to what economists are used to work with. Sadly, it wasn't entirely enough to come up with hard answers. There is a strong and significant correlation between writing Free Software and having a higher salary, but endogeneity problems3 made it hard to give a reliable estimate of how much money this would represent. Same goes for writing code has a hobby: it seems there is a strong and significant correlation, but the exact numbers I came up with cannot really be trusted. The results on community as an incentive to writing FOSS were the ones that surprised me the most. Although I expected the relation to be quite strong, the coefficients predicted were in fact quite small. I theorise this is partly due to only 8% of the respondents declaring they didn't feel like they belonged in the IT community. With such a high level of adherence, the margin for improvement has to be smaller. As for altruism, I wasn't able get any meaningful results. In my opinion this is mostly due to the fact there was no explicit survey question on this topic and I tried to make up for it by cobbling data together. Kinda anti-climatic, isn't it? I would've loved to come up with decisive conclusions on this topic, but if there's one thing I learned while writing this thesis, it is I don't know much after all.

  1. Note that the thesis is written in French.
  2. Of course, life is complex and so are people's motivations. One could come up with dozen more reasons why people contribute to Free Software. The "fun" of theoretical modelisation is trying to make complex things somewhat simpler.
  3. I'll spare you the details, but this means there is no way to know if this correlation is the result of a causal link between the two variables. There are ways to deal with this problem (using an instrumental variables model is a very popular one), but again, the survey didn't provide the proper instruments to do so. For example, it could very well be the correlation is due to omitted variables. If you are interested in this topic (and can read French), I talk about this issue in section 3.2.8.

16 February 2021

Michael Prokop: How to properly use 3rd party Debian repository signing keys with apt

(Blogging this, since this is a recurring anti-pattern I noticed at several customers and often comes up during deployments of 3rd party repositories.) Update on 2021-02-19: clarified, that Signed-By requires apt >= 1.1, thanks Vincent Bernat Many upstream projects provide Debian repository instructions like this:
curl -fsSL https://example.com/stable/debian.gpg   sudo apt-key add -
Do not follow this, for different reasons, including:
  1. You do not see what you get before adding the GPG key to your global apt trust store
  2. You can t easily script this via your preferred configuration management (the apt-key manpage clearly discourages programmatic usage)
  3. The signing key is considered valid for all your enabled Debian repositories (instead of only a specific one)
  4. You need GnuPG (either gnupg2 or gnupg1) on your system for usage with apt-key
There s a much better approach to this: download the GPG key, make sure it s in the appropriate format, then use it via deb [signed-by=/usr/share/keyrings/ ] in your apt s sources list configuration. Note and FTR: the Signed-By feature is available starting with apt 1.1 (so apt in Debian jessie/8 and older does not support it). TL;DR: As an example, let s demonstrate this with the Tailscale Debian repository for buster.
Downloading the GPG file will give you an ascii-armored GPG file:
% curl -fsSL -o buster.gpg https://pkgs.tailscale.com/stable/debian/buster.gpg
% gpg --keyid-format long buster.gpg 
gpg: WARNING: no command supplied.  Trying to guess what you mean ...
pub   rsa4096/458CA832957F5868 2020-02-25 [SC]
      2596A99EAAB33821893C0A79458CA832957F5868
uid                           Tailscale Inc. (Package repository signing key) <info@tailscale.com>
sub   rsa4096/B1547A3DDAAF03C6 2020-02-25 [E]
% file buster.gpg
buster.gpg: PGP public key block Public-Key (old)
If you have apt version >= 1.4 available (Debian >=stretch/9 and Ubuntu >=bionic/18.04), you can use this file directly as follows:
% sudo mv buster.gpg /usr/share/keyrings/tailscale.asc
% cat /etc/apt/sources.list.d/tailscale.list
deb [signed-by=/usr/share/keyrings/tailscale.asc] https://pkgs.tailscale.com/stable/debian buster main
% sudo apt update
[...]
And you re done! Iff your apt version really is older than 1.4, you need to convert the ascii-armored GPG file into a GPG key public ring file (AKA binary OpenPGP format), either by just dearmor-ing it (if you don t care about checking ID + fingerprint):
% gpg --dearmor < buster.gpg > tailscale.gpg
or if you prefer to go via GPG, you can also use a temporary GPG home directory (if you don t care about going through your personal GPG setup):
% mkdir --mode=700 /tmp/gpg-tmpdir
% gpg --homedir /tmp/gpg-tmpdir --import ./buster.gpg
gpg: keybox '/tmp/gpg-tmpdir/pubring.kbx' created
gpg: /tmp/gpg-tmpdir/trustdb.gpg: trustdb created
gpg: key 458CA832957F5868: public key "Tailscale Inc. (Package repository signing key) <info@tailscale.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1
% gpg --homedir /tmp/gpg-tmpdir --output tailscale.gpg  --export-options=export-minimal --export 0x458CA832957F5868
% rm -rf /tmp/gpg-tmpdir
The resulting GPG key public ring file should look like that:
% file tailscale.gpg 
tailscale.gpg: PGP/GPG key public ring (v4) created Tue Feb 25 04:51:20 2020 RSA (Encrypt or Sign) 4096 bits MPI=0xc00399b10bc12858...
% gpg tailscale.gpg 
gpg: WARNING: no command supplied.  Trying to guess what you mean ...
pub   rsa4096/458CA832957F5868 2020-02-25 [SC]
      2596A99EAAB33821893C0A79458CA832957F5868
uid                           Tailscale Inc. (Package repository signing key) <info@tailscale.com>
sub   rsa4096/B1547A3DDAAF03C6 2020-02-25 [E]
Then you can use this GPG file on your system as follows:
% sudo mv tailscale.gpg /usr/share/keyrings/tailscale.gpg
% cat /etc/apt/sources.list.d/tailscale.list
deb [signed-by=/usr/share/keyrings/tailscale.gpg] https://pkgs.tailscale.com/stable/debian buster main
% sudo apt update
[...]
Such a setup ensures:
  1. You can verify the GPG key file (ID + fingerprint)
  2. You can easily ship files via /usr/share/keyrings/ and refer to it in your deployment scripts, configuration management, (and can also easily update or get rid of them again!)
  3. The GPG key is valid only for the repositories with the corresponding [signed-by=/usr/share/keyrings/ ] entry
  4. You don t need to install GnuPG (neither gnupg2 nor gnupg1) on the system which is using the 3rd party Debian repository
Thanks: Guillem Jover for reviewing an early draft of this blog article.

15 February 2021

Axel Beckert: Starting a GNU Screen session via SSH's ~/.ssh/config

This is more or less a followup to this blog posting of mine about AutoSSH and GNU Screen from nearly ten years ago which by the way is still valid and still the way, I use SSH and GNU Screen. Recently a friend asked me how to automatically start or reconnect to a GNU Screen session directly via OpenSSH s configuration file. Here s how to do it: Add an entry to ~/.ssh/config similar to this one:
Host screen_on_server
    Hostname server.example.org
    RequestTTY yes
    RemoteCommand screen -RD
and then just call ssh screen_on_server and you ll get connected to an existing screen session if present, otherwise you ll a new new one. Should work with tmux, too, but might need commandlien different options.

11 February 2021

Dirk Eddelbuettel: td 0.0.1 on CRAN: New Finance Data Package

Thrilled to announce that a new package of mine just made it to CRAN: the td package accesses the twelvedata API for financial data. Currently only the time_series REST access point is supported, but it is already supported with all meaningful options (we skipped only JSON or CSV which makes no sense here) so for example any resolution between 1 minute and 1 month can be requested for any stock, etf or currency symbol for a wide array of exchanges. Historical access is available too via (optional) start and end dates. We return either raw JSON or a data.frame or an xts object making it trivial to call high-end plotting functions on the data the project and repo pages show several examples. As just one example, here is GME during the follies. We simply request via
> gme <- time_series("GME", "1min",
+                    start_date="2021-01-25 09:30:00",
+                    end_date="2021-02-04 16:00:00", as="xts")
(where the API key is either in a user-local config file accessed via the new-ish R function tools::R_user_dir("td") pointing at this package s directory, or via an environment variable; either is accessed at package load or attachment) from which we can then plot via quantmod
> quantmod::chart_Series(gme, name=paste0(attr(gme, "symbol"), "/", attr(gme, "exchange")))
which shows how we also helpfully store metadata returned by twelvedata as extra attributes of the object. The chart is You will need an API key to have up to 800 daily accesses for free, higher-performance plans (including websocket access) are available for paying customers too. I have only used the free API so far myself. I plan to add quote and price support this weekend, and generalize the time series access to returning lists of objects as the API does in fact support multi-security access. As always, feedback is welcomed. Please post comments and suggestions at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Julien Danjou: Debugging C code on macOS

Debugging C code on macOSI started to write C 25 years ago now, with many different tools over the year. As many open source developers, I spent most of my life working with the GNU tools out there.As I've been using an Apple computer over the last years, I had to adapt to this environment and learn the tricks of the trade. Here are some of my notes so a search engine can index them and I'll be able to find them later.

Debugger: lldbI was used to gdb for most of years doing C. I never managed to install gdb correctly on macOS as it needs certificates, authorization, you name it, to work properly.macOS provides a native debugger named lldb, which really looks like gdb to me it runs in a terminal with a prompt.I had to learn the few commands I mostly use, which are:
  • lldb -- myprogram -options to run the program with options
  • r to run the program
  • bt or bt N to get a backtrace of the latest N frames
  • f N to select frame N
  • p V to print some variable value or memory address
Those commands cover 99% of my use case with a debugger when writing C, so once I lost my old gdb habits, I was good to go.

Debugging Memory Overflows

On GNU/LinuxOne of my favorite tools when writing C has always been Electric Fence (and DUMA more recently). It's a library that overrides the standard memory manipulation function (e.g., malloc) and instantly makes the program crash when an out of memory error is produced, rather than corrupting the heap.Heap corruption issues are hard to debug without such tools as they can happen at any time and stay unnoticed for a while, crashing your program in a totally different location later.There's no need to compile your program with those libraries. By using the dynamic loader, you can preload them and overload the standard C library functions.
LD_PRELOAD=/usr/lib/libefence.so.0.0 my-program
Run my-program with Eletric Fence loaded on GNU/Linux
My gdb configuration has been sprinkle with my friends efence and duma, and I would activate them from gdb easily with this configuration in ~/.gdbinit:
define efence
        set environment EF_PROTECT_BELOW 0
        set environment EF_ALLOW_MALLOC_0 1
        set environment LD_PRELOAD /usr/lib/libefence.so.0.0
        echo Enabled Electric Fence\n
end
document efence
Enable memory allocation debugging through Electric Fence (efence(3)).
        See also nofence and underfence.
end
define underfence
        set environment EF_PROTECT_BELOW 1
        set environment EF_ALLOW_MALLOC_0 1
        set environment LD_PRELOAD /usr/lib/libefence.so.0.0
        echo Enabled Electric Fence for underflow detection\n
end
document underfence
Enable memory allocation debugging for underflows through Electric Fence
(efence(3)).
        See also nofence and efence.
end
define nofence
        unset environment LD_PRELOAD
        echo Disabled Electric Fence\n
end
document nofence
Disable memory allocation debugging through Electric Fence (efence(3)).
end
define duma
        set environment DUMA_PROTECT_BELOW 0
        set environment DYMA_ALLOW_MALLOC_0 1
        set environment LD_PRELOAD /usr/lib/libduma.so
        echo Enabled DUMA\n
end
document duma
Enable memory allocation debugging through DUMA (duma(3)).
        See also noduma and underduma.
end

On macOSI've been looking for equivalent features in macOS, and after many hours of research, I found out that this feature is shipped natively with libgmalloc. It works in the same way, and its features are documented by Apple.My ~/.lldbinit file now contains the following:
command alias gm _regexp-env DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib
This command alias allows enabling gmalloc by just typing gm at the lldb prompt and then run the program again to see if it crashes with gmalloc enabled.

Debugging CPythonIt's not a mystery that I spend a lot of time writing Python code that's the main reason I've been doing C lately.When playing with CPython, it can be useful to, e.g., dump the content of PyObject structs on the heap or get the Python backtrace.I've been using cpython-lldb for this with great success. It adds a few bells and whistles when debugging CPython or extensions inside lldb. For example, the alias py-bt is handy to get the Python traceback of your calls rather than a bunch of cryptic C frames.Now, you should be ready to debug your nasty issues and memory problems on macOS efficiently!

10 February 2021

Dirk Eddelbuettel: RcppArmadillo 0.10.2.1.0: New Upstream Release

armadillo image Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language and is widely used by (currently) 823 other packages on CRAN. This release brings us Armadillo 10.2 with a few updates as detailed below in the list of changes. Upstream release 10.2 was made a couple of days ago, but we need to balance new upstream updates with a responsible release cadence at CRAN. As we needed a maintenance release in early January, I opted to wait four weeks with this one which hence gets us 10.2.0 and 10.2.1 at once. As tweeted (with a follow-up) it had yet another very smooth passage at CRAN so we again appreciate the excellent work of the CRAN maintainers and say Thank You!. Anybody who desires more frequent updates show look at the RcppCore drat repo which provides more frequent interim updates. Here for example we also had 0.10.2.0.0 available for your testing pleasure. Also of note is that here is now a Python variant pyarma for those who might want enjoy Armadillo with Python. The full set of changes follows.

Changes in RcppArmadillo version 0.10.2.1.0 (2021-02-09)
  • Upgraded to Armadillo release 10.2.1 (Cicada Swarm)
    • faster handling of subcubes
    • added tgamma()
    • added .brief_print() for abridged printing of matrices & cubes
    • expanded forms of trimatu() and trimatl() with diagonal specification to handle sparse matrices
    • expanded eigs_sym() and eigs_gen() with optional shift-invert mode

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

8 February 2021

Russ Allbery: Review: The Future of Another Timeline

Review: The Future of Another Timeline, by Annalee Newitz
Publisher: Tor
Copyright: September 2019
ISBN: 0-7653-9212-7
Format: Kindle
Pages: 350
Tess is a time traveler from 2022, a member of the semi-secret Daughters of Harriet who are, under the cover of an academic research project, attempting to modify the timeline to improve women's rights in the United States. Beth is a teenager in suburban Irvine in Alta California, with an abusive father, a tight-knit group of friends, and a love of feminist punk rock. The story opens with both of them at a Grape Ape concert in 1992. Beth is hanging out with her friends, and Tess is looking for signs of a conspiracy to alter the timeline to further restrict the rights of women. The Future of Another Timeline has a great science fiction premise. There are time machines buried in geologically-stable bedrock that have been there since before any current species evolved. The first was discovered by humans thousands of years before the start of the story. They can be controlled with vibrations in the rock and therefore don't need any modern technology to operate. Humanity has therefore lived with time travel for much of recorded history, albeit with a set of rules strictly imposed by these mysterious machines: individuals can only travel to their own time or earlier, and cannot carry any equipment with them. The timeline at the start of the book is already not ours, and it shifts further over the course of the plot. Time travel has a potentially devastating effect on the foundations of narrative, so most SF novels that let the genie of time travel out of the bottle immediately start trying to stuff it back in again. Newitz does not, which is a refreshing change. The past is not immutable, there is no scientific or magical force that prevents history from changing, and people do not manage to keep something with a history of thousands of years either secret or well-controlled. It's not a free-for-all: There is a Chronology Academy that sets some rules for time travelers, the Machines themselves have rules that prevent time travel from being too casual, and most countries have laws about what time travelers are allowed to do. But it's also not horribly difficult to travel in time, not horribly uncommon to come across someone from the future, and most of the rules are not strictly enforced. This does mean there are some things that one has to agree to not think about. (To take the most obvious example, the lack of government and military involvement in time travel is not believable, even given its constraints. One has to accept this as a story premise.) But it removes the claustrophobic rules-lawyering that's so common in time travel stories and lets Newitz tell a more interesting political story about the difficulty of achieving lasting social change. Unfortunately, this is also one of those science fiction novels that is much less interested in its premise and machinery than I was as a reader. The Machines are fascinating objects: ancient, mysterious, and as we learn more about them over the course of the story, rich with intriguing detail. After reading this summary, you're probably curious where they came from, what they can do, and how they work. So am I, after reading the book. The Future of Another Timeline is completely uninterested in that or any related question. About halfway through the book, a time traveler from the future demonstrates interfaces in the time machines that no one knew existed, the characters express some surprise, and then no one asks any meaningful questions for the rest of the book. At another point, the characters have the opportunity to see a Machine in something closer to its original form before aspects of its interface have eroded away. They learn just enough to solve their immediate plot problem and show no further curiosity. I found this immensely frustrating, in part due to the mixed signaling. Normally if an author is going to use a science fiction idea as pure plot device, they avoid spending much time on it, implicitly warning the reader that this isn't where the story is going. Newitz instead provides the little details and new revelations that normally signal that understanding these objects will be a key to the plot, and then shrugs and walks away, leaving every question unanswered. Given how many people enjoyed Rendezvous with Rama, this apparently doesn't bother other readers as much as it bothers me. If you are like me, though, be warned. But, fine, this is a character story built around a plot device rather than a technology story. That's a wholly valid mode of science fiction, and that part of the book has heft. It reminded me of the second-wave feminist science fiction of authors like Russ and Charnas, except updated to modern politics. The villains are a projection forward of the modern on-line misogynists (incels, specifically), but Newitz makes the unusual choice of not focusing on their motives or interior lives. They simply exist as a malevolent hostile force, much the way that women experience them today on-line. They have to be defeated, the characters of the book set out to defeat them, and this is done without melodrama, hand-wringing, or psychoanalysis. It's refreshingly straightforward and unambiguous, and it keeps the focus on the people trying to make the world a better place rather than on the redemption arc of some screaming asshole. The part I was less enamored of is that these are two of the least introspective first-person protagonists that I've seen in a book. Normally, first-person perspective is used to provide a rich internal monologue about external events, but both Tess and Beth tell their stories as mostly-dry sequences of facts. Sometimes this includes a bit of what they're feeling, but neither character delves much into the why or how. This improves somewhat towards the end of the book, but I found the first two-thirds of the story oddly flat and had a hard time generating much interest in or sympathy for the characters. There are good in-story reasons for both Tess and Beth to heavily suppress their emotions, so I will not argue this is unrealistic, but character stories work better for me with more of an emotional hook. Hand-in-hand with that is the problem that the ending didn't provide the catharsis that I was hoping for. Beth goes through absolute hell over the course of the book, and while that does reach a resolution that I know intellectually is the best type of resolution that her story can hope for, it felt wholly insufficient. Tess's story reaches a somewhat more satisfying conclusion, but one that reverses an earlier moral imperative in a way that I found overly sudden. And everything about this book is highly contingent and temporary in a way that is true to its theme and political statement but that left me feeling more weary than satisfied. That type of ending is a valid authorial choice, and to some extent my complaint is only that this wasn't the book for me at the time I read it. But I have read other books with similarly conditional endings and withdrawn characters that still carried me along with the force and power of the writing (Daughters of the North comes to mind). The Future of Another Timeline is not poorly written, but neither do I think it achieves that level of skill. The writing is a bit wooden, the flow of sentences is a touch cliched and predictable, and the characters are a bit thin. It's serviceable writing had there been something else (such as a setting-as-character exploration of the origins and purpose of the Machines) to grab my attention and pull me along. But if the weight of the story has to be born by the quality of the writing, I don't think it was quite up to the task. Overall, I think The Future of Another Timeline has a great premise that it treats with frustrating indifference, a satisfyingly different take on time travel with some obvious holes, some solid political ideas reminiscent of an earlier age of feminist SF, a refreshing unwillingness to center evil on its own terms, characters that took more than half the book to develop much depth, and a suitable but frustrating ending. I can see why other people liked it more than I did, but I can't recommend it. Content warning: Rape, graphic violence, child abuse, gaslighting, graphic medical procedure, suicide, extreme misogyny, and mutilation, and this is spread throughout the book, not concentrated in one scene. I'm not very squeamish about non-horror fiction and it was still rather a lot, so please read with care. Rating: 6 out of 10

7 February 2021

Chris Lamb: Favourite books of 2020

I won't reveal precisely how many books I read in 2020, but it was definitely an improvement on 74 in 2019, 53 in 2018 and 50 in 2017. But not only did I read more in a quantitative sense, the quality seemed higher as well. There were certainly fewer disappointments: given its cultural resonance, I was nonplussed by Nick Hornby's Fever Pitch and whilst Ian Fleming's The Man with the Golden Gun was a little thin (again, given the obvious influence of the Bond franchise) the booked lacked 'thinness' in a way that made it interesting to critique. The weakest novel I read this year was probably J. M. Berger's Optimal, but even this hybrid of Ready Player One late-period Black Mirror wasn't that cringeworthy, all things considered. Alas, graphic novels continue to not quite be my thing, I'm afraid. I perhaps experienced more disappointments in the non-fiction section. Paul Bloom's Against Empathy was frustrating, particularly in that it expended unnecessary energy battling its misleading title and accepted terminology, and it could so easily have been an 20-minute video essay instead). (Elsewhere in the social sciences, David and Goliath will likely be the last Malcolm Gladwell book I voluntarily read.) After so many positive citations, I was also more than a little underwhelmed by Shoshana Zuboff's The Age of Surveillance Capitalism, and after Ryan Holiday's many engaging reboots of Stoic philosophy, his Conspiracy (on Peter Thiel and Hulk Hogan taking on Gawker) was slightly wide of the mark for me. Anyway, here follows a selection of my favourites from 2020, in no particular order:

Fiction Wolf Hall & Bring Up the Bodies & The Mirror and the Light Hilary Mantel During the early weeks of 2020, I re-read the first two parts of Hilary Mantel's Thomas Cromwell trilogy in time for the March release of The Mirror and the Light. I had actually spent the last few years eagerly following any news of the final instalment, feigning outrage whenever Mantel appeared to be spending time on other projects. Wolf Hall turned out to be an even better book than I remembered, and when The Mirror and the Light finally landed at midnight on 5th March, I began in earnest the next morning. Note that date carefully; this was early 2020, and the book swiftly became something of a heavy-handed allegory about the world at the time. That is to say and without claiming that I am Monsieur Cromuel in any meaningful sense it was an uneasy experience to be reading about a man whose confident grasp on his world, friends and life was slipping beyond his control, and at least in Cromwell's case, was heading inexorably towards its denouement. The final instalment in Mantel's trilogy is not perfect, and despite my love of her writing I would concur with the judges who decided against awarding her a third Booker Prize. For instance, there is something of the longueur that readers dislike in the second novel, although this might not be entirely Mantel's fault after all, the rise of the "ugly" Anne of Cleves and laborious trade negotiations for an uninspiring mineral (this is no Herbertian 'spice') will never match the court intrigues of Anne Boleyn, Jane Seymour and that man for all seasons, Thomas More. Still, I am already looking forward to returning to the verbal sparring between King Henry and Cromwell when I read the entire trilogy once again, tentatively planned for 2022.

The Fault in Our Stars John Green I came across John Green's The Fault in Our Stars via a fantastic video by Lindsay Ellis discussing Roland Barthes famous 1967 essay on authorial intent. However, I might have eventually come across The Fault in Our Stars regardless, not because of Green's status as an internet celebrity of sorts but because I'm a complete sucker for this kind of emotionally-manipulative bildungsroman, likely due to reading Philip Pullman's His Dark Materials a few too many times in my teens. Although its title is taken from Shakespeare's Julius Caesar, The Fault in Our Stars is actually more Romeo & Juliet. Hazel, a 16-year-old cancer patient falls in love with Gus, an equally ill teen from her cancer support group. Hazel and Gus share the same acerbic (and distinctly unteenage) wit and a love of books, centred around Hazel's obsession of An Imperial Affliction, a novel by the meta-fictional author Peter Van Houten. Through a kind of American version of Jim'll Fix It, Gus and Hazel go and visit Van Houten in Amsterdam. I'm afraid it's even cheesier than I'm describing it. Yet just as there is a time and a place for Michelin stars and Haribo Starmix, there's surely a place for this kind of well-constructed but altogether maudlin literature. One test for emotionally manipulative works like this is how well it can mask its internal contradictions while Green's story focuses on the universalities of love, fate and the shortness of life (as do almost all of his works, it seems), The Fault in Our Stars manages to hide, for example, that this is an exceedingly favourable treatment of terminal illness that is only possible for the better off. The 2014 film adaptation does somewhat worse in peddling this fantasy (and has a much weaker treatment of the relationship between the teens' parents too, an underappreciated subtlety of the book). The novel, however, is pretty slick stuff, and it is difficult to fault it for what it is. For some comparison, I later read Green's Looking for Alaska and Paper Towns which, as I mention, tug at many of the same strings, but they don't come together nearly as well as The Fault in Our Stars. James Joyce claimed that "sentimentality is unearned emotion", and in this respect, The Fault in Our Stars really does earn it.

The Plague Albert Camus P. D. James' The Children of Men, George Orwell's Nineteen Eighty-Four, Arthur Koestler's Darkness at Noon ... dystopian fiction was already a theme of my reading in 2020, so given world events it was an inevitability that I would end up with Camus's novel about a plague that swept through the Algerian city of Oran. Is The Plague an allegory about the Nazi occupation of France during World War Two? Where are all the female characters? Where are the Arab ones? Since its original publication in 1947, there's been so much written about The Plague that it's hard to say anything new today. Nevertheless, I was taken aback by how well it captured so much of the nuance of 2020. Whilst we were saying just how 'unprecedented' these times were, it was eerie how a novel written in the 1940s could accurately how many of us were feeling well over seventy years on later: the attitudes of the people; the confident declarations from the institutions; the misaligned conversations that led to accidental misunderstandings. The disconnected lovers. The only thing that perhaps did not work for me in The Plague was the 'character' of the church. Although I could appreciate most of the allusion and metaphor, it was difficult for me to relate to the significance of Father Paneloux, particularly regarding his change of view on the doctrinal implications of the virus, and spoiler alert that he finally died of a "doubtful case" of the disease, beyond the idea that Paneloux's beliefs are in themselves "doubtful". Answers on a postcard, perhaps. The Plague even seemed to predict how we, at least speaking of the UK, would react when the waves of the virus waxed and waned as well:
The disease stiffened and carried off three or four patients who were expected to recover. These were the unfortunates of the plague, those whom it killed when hope was high
It somehow captured the nostalgic yearning for high-definition videos of cities and public transport; one character even visits the completely deserted railway station in Oman simply to read the timetables on the wall.

Tinker, Tailor, Soldier, Spy John le Carr There's absolutely none of the Mad Men glamour of James Bond in John le Carr 's icy world of Cold War spies:
Small, podgy, and at best middle-aged, Smiley was by appearance one of London's meek who do not inherit the earth. His legs were short, his gait anything but agile, his dress costly, ill-fitting, and extremely wet.
Almost a direct rebuttal to Ian Fleming's 007, Tinker, Tailor has broken-down cars, bad clothes, women with their own internal and external lives (!), pathetically primitive gadgets, and (contra Mad Men) hangovers that significantly longer than ten minutes. In fact, the main aspect that the mostly excellent 2011 film adaption doesn't really capture is the smoggy and run-down nature of 1970s London this is not your proto-Cool Britannia of Austin Powers or GTA:1969, the city is truly 'gritty' in the sense there is a thin film of dirt and grime on every surface imaginable. Another angle that the film cannot capture well is just how purposefully the novel does not mention the United States. Despite the US obviously being the dominant power, the British vacillate between pretending it doesn't exist or implying its irrelevance to the matter at hand. This is no mistake on Le Carr 's part, as careful readers are rewarded by finding this denial of US hegemony in metaphor throughout --pace Ian Fleming, there is no obvious Felix Leiter to loudly throw money at the problem or a Sheriff Pepper to serve as cartoon racist for the Brits to feel superior about. By contrast, I recall that a clever allusion to "dusty teabags" is subtly mirrored a few paragraphs later with a reference to the installation of a coffee machine in the office, likely symbolic of the omnipresent and unavoidable influence of America. (The officer class convince themselves that coffee is a European import.) Indeed, Le Carr communicates a feeling of being surrounded on all sides by the peeling wallpaper of Empire. Oftentimes, the writing style matches the graceless and inelegance of the world it depicts. The sentences are dense and you find your brain performing a fair amount of mid-flight sentence reconstruction, reparsing clauses, commas and conjunctions to interpret Le Carr 's intended meaning. In fact, in his eulogy-cum-analysis of Le Carr 's writing style, William Boyd, himself a ventrioquilist of Ian Fleming, named this intentional technique 'staccato'. Like the musical term, I suspect the effect of this literary staccato is as much about the impact it makes on a sentence as the imperceptible space it generates after it. Lastly, the large cast in this sprawling novel is completely believable, all the way from the Russian spymaster Karla to minor schoolboy Roach the latter possibly a stand-in for Le Carr himself. I got through the 500-odd pages in just a few days, somehow managing to hold the almost-absurdly complicated plot in my head. This is one of those classic books of the genre that made me wonder why I had not got around to it before.

The Nickel Boys Colson Whitehead According to the judges who awarded it the Pulitzer Prize for Fiction, The Nickel Boys is "a devastating exploration of abuse at a reform school in Jim Crow-era Florida" that serves as a "powerful tale of human perseverance, dignity and redemption". But whilst there is plenty of this perseverance and dignity on display, I found little redemption in this deeply cynical novel. It could almost be read as a follow-up book to Whitehead's popular The Underground Railroad, which itself won the Pulitzer Prize in 2017. Indeed, each book focuses on a young protagonist who might be euphemistically referred to as 'downtrodden'. But The Nickel Boys is not only far darker in tone, it feels much closer and more connected to us today. Perhaps this is unsurprising, given that it is based on the story of the Dozier School in northern Florida which operated for over a century before its long history of institutional abuse and racism was exposed a 2012 investigation. Nevertheless, if you liked the social commentary in The Underground Railroad, then there is much more of that in The Nickel Boys:
Perhaps his life might have veered elsewhere if the US government had opened the country to colored advancement like they opened the army. But it was one thing to allow someone to kill for you and another to let him live next door.
Sardonic aper us of this kind are pretty relentless throughout the book, but it never tips its hand too far into on nihilism, especially when some of the visual metaphors are often first-rate: "An American flag sighed on a pole" is one I can easily recall from memory. In general though, The Nickel Boys is not only more world-weary in tenor than his previous novel, the United States it describes seems almost too beaten down to have the energy conjure up the Swiftian magical realism that prevented The Underground Railroad from being overly lachrymose. Indeed, even we Whitehead transports us a present-day New York City, we can't indulge in another kind of fantasy, the one where America has solved its problems:
The Daily News review described the [Manhattan restaurant] as nouveau Southern, "down-home plates with a twist." What was the twist that it was soul food made by white people?
It might be overly reductionist to connect Whitehead's tonal downshift with the racial justice movements of the past few years, but whatever the reason, we've ended up with a hard-hitting, crushing and frankly excellent book.

True Grit & No Country for Old Men Charles Portis & Cormac McCarthy It's one of the most tedious cliches to claim the book is better than the film, but these two books are of such high quality that even the Coen Brothers at their best cannot transcend them. I'm grouping these books together here though, not because their respective adaptations will exemplify some of the best cinema of the 21st century, but because of their superb treatment of language. Take the use of dialogue. Cormac McCarthy famously does not use any punctuation "I believe in periods, in capitals, in the occasional comma, and that's it" but the conversations in No Country for Old Men together feel familiar and commonplace, despite being relayed through this unconventional technique. In lesser hands, McCarthy's written-out Texan drawl would be the novelistic equivalent of white rap or Jar Jar Binks, but not only is the effect entirely gripping, it helps you to believe you are physically present in the many intimate and domestic conversations that hold this book together. Perhaps the cinematic familiarity helps, as you can almost hear Tommy Lee Jones' voice as Sheriff Bell from the opening page to the last. Charles Portis' True Grit excels in its dialogue too, but in this book it is not so much in how it flows (although that is delightful in its own way) but in how forthright and sardonic Maddie Ross is:
"Earlier tonight I gave some thought to stealing a kiss from you, though you are very young, and sick and unattractive to boot, but now I am of a mind to give you five or six good licks with my belt." "One would be as unpleasant as the other."
Perhaps this should be unsurprising. Maddie, a fourteen-year-old girl from Yell County, Arkansas, can barely fire her father's heavy pistol, so she can only has words to wield as her weapon. Anyway, it's not just me who treasures this book. In her encomium that presages most modern editions, Donna Tartt of The Secret History fame traces the novels origins through Huckleberry Finn, praising its elegance and economy: "The plot of True Grit is uncomplicated and as pure in its way as one of the Canterbury Tales". I've read any Chaucer, but I am inclined to agree. Tartt also recalls that True Grit vanished almost entirely from the public eye after the release of John Wayne's flimsy cinematic vehicle in 1969 this earlier film was, Tartt believes, "good enough, but doesn't do the book justice". As it happens, reading a book with its big screen adaptation as a chaser has been a minor theme of my 2020, including P. D. James' The Children of Men, Kazuo Ishiguro's Never Let Me Go, Patricia Highsmith's Strangers on a Train, James Ellroy's The Black Dahlia, John Green's The Fault in Our Stars, John le Carr 's Tinker, Tailor Soldier, Spy and even a staged production of Charles Dicken's A Christmas Carol streamed from The Old Vic. For an autodidact with no academic background in literature or cinema, I've been finding this an effective and enjoyable means of getting closer to these fine books and films it is precisely where they deviate (or perhaps where they are deficient) that offers a means by which one can see how they were constructed. I've also found that adaptations can also tell you a lot about the culture in which they were made: take the 'straightwashing' in the film version of Strangers on a Train (1951) compared to the original novel, for example. It is certainly true that adaptions rarely (as Tartt put it) "do the book justice", but she might be also right to alight on a legal metaphor, for as the saying goes, to judge a movie in comparison to the book is to do both a disservice.

The Glass Hotel Emily St. John Mandel In The Glass Hotel, Mandel somehow pulls off the impossible; writing a loose roman- -clef on Bernie Madoff, a Ponzi scheme and the ephemeral nature of finance capital that is tranquil and shimmeringly beautiful. Indeed, don't get the wrong idea about the subject matter; this is no over over-caffeinated The Big Short, as The Glass Hotel is less about a Madoff or coked-up financebros but the fragile unreality of the late 2010s, a time which was, as we indeed discovered in 2020, one event away from almost shattering completely. Mandel's prose has that translucent, phantom quality to it where the chapters slip through your fingers when you try to grasp at them, and the plot is like a ghost ship that that slips silently, like the Mary Celeste, onto the Canadian water next to which the eponymous 'Glass Hotel' resides. Indeed, not unlike The Overlook Hotel, the novel so overflows with symbolism so that even the title needs to evoke the idea of impermanence permanently living in a hotel might serve as a house, but it won't provide a home. It's risky to generalise about such things post-2016, but the whole story sits in that the infinitesimally small distance between perception and reality, a self-constructed culture that is not so much 'post truth' but between them. There's something to consider in almost every character too. Take the stand-in for Bernie Madoff: no caricature of Wall Street out of a 1920s political cartoon or Brechtian satire, Jonathan Alkaitis has none of the oleaginous sleaze of a Dominic Strauss-Kahn, the cold sociopathy of a Marcus Halberstam nor the well-exercised sinuses of, say, Jordan Belford. Alkaitis is dare I say it? eminently likeable, and the book is all the better for it. Even the C-level characters have something to say: Enrico, trivially escaping from the regulators (who are pathetically late to the fraud without Mandel ever telling us explicitly), is daydreaming about the girlfriend he abandoned in New York: "He wished he'd realised he loved her before he left". What was in his previous life that prevented him from doing so? Perhaps he was never in love at all, or is love itself just as transient as the imaginary money in all those bank accounts? Maybe he fell in love just as he crossed safely into Mexico? When, precisely, do we fall in love anyway? I went on to read Mandel's Last Night in Montreal, an early work where you can feel her reaching for that other-worldly quality that she so masterfully achieves in The Glass Hotel. Her f ted Station Eleven is on my must-read list for 2021. "What is truth?" asked Pontius Pilate. Not even Mandel cannot give us the answer, but this will certainly do for now.

Running the Light Sam Tallent Although it trades in all of the clich s and stereotypes of the stand-up comedian (the triumvirate of drink, drugs and divorce), Sam Tallent's debut novel depicts an extremely convincing fictional account of a touring road comic. The comedian Doug Stanhope (who himself released a fairly decent No Encore for the Donkey memoir in 2020) hyped Sam's book relentlessly on his podcast during lockdown... and justifiably so. I ripped through Running the Light in a few short hours, the only disappointment being that I can't seem to find videos online of Sam that come anywhere close to match up to his writing style. If you liked the rollercoaster energy of Paul Beatty's The Sellout, the cynicism of George Carlin and the car-crash invertibility of final season Breaking Bad, check this great book out.

Non-fiction Inside Story Martin Amis This was my first introduction to Martin Amis's work after hearing that his "novelised autobiography" contained a fair amount about Christopher Hitchens, an author with whom I had a one of those rather clich d parasocial relationship with in the early days of YouTube. (Hey, it could have been much worse.) Amis calls his book a "novelised autobiography", and just as much has been made of its quasi-fictional nature as the many diversions into didactic writing advice that betwixt each chapter: "Not content with being a novel, this book also wants to tell you how to write novels", complained Tim Adams in The Guardian. I suspect that reviewers who grew up with Martin since his debut book in 1973 rolled their eyes at yet another demonstration of his manifest cleverness, but as my first exposure to Amis's gift of observation, I confess that I was thought it was actually kinda clever. Try, for example, "it remains a maddening truth that both sexual success and sexual failure are steeply self-perpetuating" or "a hospital gym is a contradiction like a young Conservative", etc. Then again, perhaps I was experiencing a form of nostalgia for a pre-Gamergate YouTube, when everything in the world was a lot simpler... or at least things could be solved by articulate gentlemen who honed their art of rhetoric at the Oxford Union. I went on to read Martin's first novel, The Rachel Papers (is it 'arrogance' if you are, indeed, that confident?), as well as his 1997 Night Train. I plan to read more of him in the future.

The Collected Essays, Journalism and Letters: Volume 1 & Volume 2 & Volume 3 & Volume 4 George Orwell These deceptively bulky four volumes contain all of George Orwell's essays, reviews and correspondence, from his teenage letters sent to local newspapers to notes to his literary executor on his deathbed in 1950. Reading this was part of a larger, multi-year project of mine to cover the entirety of his output. By including this here, however, I'm not recommending that you read everything that came out of Orwell's typewriter. The letters to friends and publishers will only be interesting to biographers or hardcore fans (although I would recommend Dorian Lynskey's The Ministry of Truth: A Biography of George Orwell's 1984 first). Furthermore, many of his book reviews will be of little interest today. Still, some insights can be gleaned; if there is any inconsistency in this huge corpus is that his best work is almost 'too' good and too impactful, making his merely-average writing appear like hackwork. There are some gems that don't make the usual essay collections too, and some of Orwell's most astute social commentary came out of series of articles he wrote for the left-leaning newspaper Tribune, related in many ways to the US Jacobin. You can also see some of his most famous ideas start to take shape years if not decades before they appear in his novels in these prototype blog posts. I also read Dennis Glover's novelised account of the writing of Nineteen-Eighty Four called The Last Man in Europe, and I plan to re-read some of Orwell's earlier novels during 2021 too, including A Clergyman's Daughter and his 'antebellum' Coming Up for Air that he wrote just before the Second World War; his most under-rated novel in my estimation. As it happens, and with the exception of the US and Spain, copyright in the works published in his lifetime ends on 1st January 2021. Make of that what you will.

Capitalist Realism & Chavs: The Demonisation of the Working Class Mark Fisher & Owen Jones These two books are not natural companions to one another and there is likely much that Jones and Fisher would vehemently disagree on, but I am pairing these books together here because they represent the best of the 'political' books I read in 2020. Mark Fisher was a dedicated leftist whose first book, Capitalist Realism, marked an important contribution to political philosophy in the UK. However, since his suicide in early 2017, the currency of his writing has markedly risen, and Fisher is now frequently referenced due to his belief that the prevalence of mental health conditions in modern life is a side-effect of various material conditions, rather than a natural or unalterable fact "like weather". (Of course, our 'weather' is being increasingly determined by a combination of politics, economics and petrochemistry than pure randomness.) Still, Fisher wrote on all manner of topics, from the 2012 London Olympics and "weird and eerie" electronic music that yearns for a lost future that will never arrive, possibly prefiguring or influencing the Fallout video game series. Saying that, I suspect Fisher will resonate better with a UK audience more than one across the Atlantic, not necessarily because he was minded to write about the parochial politics and culture of Britain, but because his writing often carries some exasperation at the suppression of class in favour of identity-oriented politics, a viewpoint not entirely prevalent in the United States outside of, say, Tour F. Reed or the late Michael Brooks. (Indeed, Fisher is likely best known in the US as the author of his controversial 2013 essay, Exiting the Vampire Castle, but that does not figure greatly in this book). Regardless, Capitalist Realism is an insightful, damning and deeply unoptimistic book, best enjoyed in the warm sunshine I found it an ironic compliment that I had quoted so many paragraphs that my Kindle's copy protection routines prevented me from clipping any further. Owen Jones needs no introduction to anyone who regularly reads a British newspaper, especially since 2015 where he unofficially served as a proxy and punching bag for expressing frustrations with the then-Labour leader, Jeremy Corbyn. However, as the subtitle of Jones' 2012 book suggests, Chavs attempts to reveal the "demonisation of the working class" in post-financial crisis Britain. Indeed, the timing of the book is central to Jones' analysis, specifically that the stereotype of the "chav" is used by government and the media as a convenient figleaf to avoid meaningful engagement with economic and social problems on an austerity ridden island. (I'm not quite sure what the US equivalent to 'chav' might be. Perhaps Florida Man without the implications of mental health.) Anyway, Jones certainly has a point. From Vicky Pollard to the attacks on Jade Goody, there is an ignorance and prejudice at the heart of the 'chav' backlash, and that would be bad enough even if it was not being co-opted or criminalised for ideological ends. Elsewhere in political science, I also caught Michael Brooks' Against the Web and David Graeber's Bullshit Jobs, although they are not quite methodical enough to recommend here. However, Graeber's award-winning Debt: The First 5000 Years will be read in 2021. Matt Taibbi's Hate Inc: Why Today's Media Makes Us Despise One Another is worth a brief mention here though, but its sprawling nature felt very much like I was reading a set of Substack articles loosely edited together. And, indeed, I was.

The Golden Thread: The Story of Writing Ewan Clayton A recommendation from a dear friend, Ewan Clayton's The Golden Thread is a journey through the long history of the writing from the Dawn of Man to present day. Whether you are a linguist, a graphic designer, a visual artist, a typographer, an archaeologist or 'just' a reader, there is probably something in here for you. I was already dipping my quill into calligraphy this year so I suspect I would have liked this book in any case, but highlights would definitely include the changing role of writing due to the influence of textual forms in the workplace as well as digression on ergonomic desks employed by monks and scribes in the Middle Ages. A lot of books by otherwise-sensible authors overstretch themselves when they write about computers or other technology from the Information Age, at best resulting in bizarre non-sequiturs and dangerously Panglossian viewpoints at worst. But Clayton surprised me by writing extremely cogently and accurate on the role of text in this new and unpredictable era. After finishing it I realised why for a number of years, Clayton was a consultant for the legendary Xerox PARC where he worked in a group focusing on documents and contemporary communications whilst his colleagues were busy inventing the graphical user interface, laser printing, text editors and the computer mouse.

New Dark Age & Radical Technologies: The Design of Everyday Life James Bridle & Adam Greenfield I struggled to describe these two books to friends, so I doubt I will suddenly do a better job here. Allow me to quote from Will Self's review of James Bridle's New Dark Age in the Guardian:
We're accustomed to worrying about AI systems being built that will either "go rogue" and attack us, or succeed us in a bizarre evolution of, um, evolution what we didn't reckon on is the sheer inscrutability of these manufactured minds. And minds is not a misnomer. How else should we think about the neural network Google has built so its translator can model the interrelation of all words in all languages, in a kind of three-dimensional "semantic space"?
New Dark Age also turns its attention to the weird, algorithmically-derived products offered for sale on Amazon as well as the disturbing and abusive videos that are automatically uploaded by bots to YouTube. It should, by rights, be a mess of disparate ideas and concerns, but Bridle has a flair for introducing topics which reveals he comes to computer science from another discipline altogether; indeed, on a four-part series he made for Radio 4, he's primarily referred to as "an artist". Whilst New Dark Age has rather abstract section topics, Adam Greenfield's Radical Technologies is a rather different book altogether. Each chapter dissects one of the so-called 'radical' technologies that condition the choices available to us, asking how do they work, what challenges do they present to us and who ultimately benefits from their adoption. Greenfield takes his scalpel to smartphones, machine learning, cryptocurrencies, artificial intelligence, etc., and I don't think it would be unfair to say that starts and ends with a cynical point of view. He is no reactionary Luddite, though, and this is both informed and extremely well-explained, and it also lacks the lazy, affected and Private Eye-like cynicism of, say, Attack of the 50 Foot Blockchain. The books aren't a natural pair, for Bridle's writing contains quite a bit of air in places, ironically mimics the very 'clouds' he inveighs against. Greenfield's book, by contrast, as little air and much lower pH value. Still, it was more than refreshing to read two technology books that do not limit themselves to platitudinal booleans, be those dangerously naive (e.g. Kevin Kelly's The Inevitable) or relentlessly nihilistic (Shoshana Zuboff's The Age of Surveillance Capitalism). Sure, they are both anti-technology screeds, but they tend to make arguments about systems of power rather than specific companies and avoid being too anti-'Big Tech' through a narrower, Silicon Valley obsessed lens for that (dipping into some other 2020 reading of mine) I might suggest Wendy Liu's Abolish Silicon Valley or Scott Galloway's The Four. Still, both books are superlatively written. In fact, Adam Greenfield has some of the best non-fiction writing around, both in terms of how he can explain complicated concepts (particularly the smart contract mechanism of the Ethereum cryptocurrency) as well as in the extremely finely-crafted sentences I often felt that the writing style almost had no need to be that poetic, and I particularly enjoyed his fictional scenarios at the end of the book.

The Algebra of Happiness & Indistractable: How to Control Your Attention and Choose Your Life Scott Galloway & Nir Eyal A cocktail of insight, informality and abrasiveness makes NYU Professor Scott Galloway uncannily appealing to guys around my age. Although Galloway definitely has his own wisdom and experience, similar to Joe Rogan I suspect that a crucial part of Galloway's appeal is that you feel you are learning right alongside him. Thankfully, 'Prof G' is far less err problematic than Rogan (Galloway is more of a well-meaning, spirited centrist), although he, too, has some pretty awful takes at time. This is a shame, because removed from the whirlwind of social media he can be really quite considered, such as in this long-form interview with Stephanie Ruhle. In fact, it is this kind of sentiment that he captured in his 2019 Algebra of Happiness. When I look over my highlighted sections, it's clear that it's rather schmaltzy out of context ("Things you hate become just inconveniences in the presence of people you love..."), but his one-two punch of cynicism and saccharine ("Ask somebody who purchased a home in 2007 if their 'American Dream' came true...") is weirdly effective, especially when he uses his own family experiences as part of his story:
A better proxy for your life isn't your first home, but your last. Where you draw your last breath is more meaningful, as it's a reflection of your success and, more important, the number of people who care about your well-being. Your first house signals the meaningful your future and possibility. Your last home signals the profound the people who love you. Where you die, and who is around you at the end, is a strong signal of your success or failure in life.
Nir Eyal's Indistractable, however, is a totally different kind of 'self-help' book. The important background story is that Eyal was the author of the widely-read Hooked which turned into a secular Bible of so-called 'addictive design'. (If you've ever been cornered by a techbro wielding a Wikipedia-thin knowledge of B. F. Skinner's behaviourist psychology and how it can get you to click 'Like' more often, it ultimately came from Hooked.) However, Eyal's latest effort is actually an extended mea culpa for his previous sin and he offers both high and low-level palliative advice on how to avoid falling for the tricks he so studiously espoused before. I suppose we should be thankful to capitalism for selling both cause and cure. Speaking of markets, there appears to be a growing appetite for books in this 'anti-distraction' category, and whilst I cannot claim to have done an exhausting study of this nascent field, Indistractable argues its points well without relying on accurate-but-dry "studies show..." or, worse, Gladwellian gotchas. My main criticism, however, would be that Eyal doesn't acknowledge the limits of a self-help approach to this problem; it seems that many of the issues he outlines are an inescapable part of the alienation in modern Western society, and the only way one can really avoid distraction is to move up the income ladder or move out to a 500-acre ranch.

6 February 2021

Andrew Cater: Debian 10.8 release process - Yay, it's a lot faster

Thanks to the changes behind the scenes, images are now being produced significantly faster than they were: the embarrassingly parallel speed up has worked, though at slight cost to the predictability of when we get each architecture produced for us to test.Thanks, as ever, to Sledge, RattusRattus and Isy over in Cambridge and to schweer who painstakingly tests all the debian-edu releases.83 out of 94 steps done and it's only 13:50 GMT - less popular architectures are coming on apace. There's still a lot of testing to do: we will never be able to test s390x for example. Other architectures - including mips* - we have no machines, we're building as best efforts and we can't guarantee how well they will work for anyone.Anybody installing the less popular architectures please let us know that you've installed them and how you get on. We'd like to see a positive report as well as bugs: we don't see much feedback at all to know how useful they are: evidence of installs on any architecture is always helpful to us.

Dirk Eddelbuettel: RcppFastFloat 0.0.1: New Package, Already on CRAN

A new package, once again based on wonderful library by Daniel Lemire, is now on CRAN in its initial version 0.0.1. Daniel, in a recent arXiv paper shows that one can convert character representations of numbers into floating point at rates at or exceeding one gigabyte per second. His tests show a fourfold gain over library functions such as strtod. We put a simply package together showing use of the example parser, and containing a simple all-in comparison benchmark (where we time the function call overhead as well) and get roughly 3x. See the repo for details; we are borring the table and figure here:
> source("comparison.R")
Unit: milliseconds
      expr      min       lq     mean   median       uq      max neval cld
     scanf 218.8936 224.1223 238.5650 227.1901 229.9116 1343.433   100   c
      atof 124.8087 127.3274 129.4104 128.5858 130.9138  146.334   100  b 
    strtod 124.5705 127.2157 129.1238 129.1042 130.7504  137.143   100  b 
      stod 127.1751 129.7343 131.7339 131.4854 133.1425  147.763   100  b 
 fastfloat  40.6219  41.3042  42.5729  42.3209  43.1738   57.788   100 a  
> 
Or in chart form: Not much to say yet for the initial release:

Changes in version 0.0.1 (2021-01-31)
  • Initial version and CRAN upload

While the package was waiting to be added to CRAN, Brendan already added a potential as.double() replacement which will be in the next version. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

4 February 2021

Ulrike Uhlig: On anger, misunderstandings, and hearing with different ears

Anger Anger is a feeling that is mostly taboo in our society. People tend to think that anger and rage are the same thing and reject anger. We are taught to suppress it. But: The suppression of anger can cause a lot of trouble, giving rise to virulent progeny such as malice, passive aggression, hostility, rage, sabotage, hate, blame, guilt, controlling behavior, shame, self-blame, and self-destruction. (Quoted from: Anne Katherine, Where to draw the line ). When we talk about anger, we need to distinguish between acting out anger, and feeling anger. In general, when you hear me talking about anger, I m talking about the feeling named anger in English. Anger is a normal feeling with a super power: it gives us the energy to change a situation that we consider to be unsustainable. We can express the feeling of anger verbally by saying for example: That thing makes me really angry , I can t talk right now, I am very angry about what I just heard . Let s also note that there may be other feelings lying below anger, such as sadness, frustration, grief, fear, etc. Mixing emotions This is a page from the wonderful Making Comics by Scott McCloud. (Yes, emotions are complex! See some more of Scott McCloud s pages about emotion in comics)

Bottling up feelings Most of the time, people don t get angry suddenly, even though it might seem like that from the outside. Instead, they ve been bottling up feelings for a while, and at some point, a small trigger is enough to make the bottle overflow. The reasons for bottling up feelings can be as diverse as people:
  • Low self-esteem: not thinking one has the right and the capacity to express unpleasant feelings
  • Not knowing how to express unpleasant feelings. For example not having learnt to say: I am not comfortable, I ll leave now, I ll get back to you once I know what s going on / if this is about me / if I feel like it.
  • Not trusting one s own feelings, specifically common in people who have been victims of gaslighting
  • Repressing emotions, specifically anger, sometimes to the point of not being able to feel one s own feelings anymore
  • Not being able to express disagreement and instead applying the fawn response: trying to please the other in order to avoid further conflict. See: Fight, flight, freeze, or fawn
  • Not being able to express boundaries: stop , this is enough , I don t accept this , etc.
  • Not being in a situation or a space in which feelings can be expressed freely, for example a workplace or a hierarchical situation where this might be disadvantageous
These are just some examples I can come up with in 3 minutes, the list is non exhaustive. When someone s bottle overflows, their peers can be confused and not know how to react to what they perceive as a sudden outburst while what they are seeing might just be the other person s first time tentative of saying no or stop . Oftentimes, the responsibility for the situation is then put on the shoulders of the person who supposedly exploded : they have been behaving differently than usual or not within the expectations, right? Well, it s not that simple. Conflicts are relational. The other party might have ignored signs, requests, feelings, and needs of the angry person for a while. Maybe the relationship has been deteriorating since some time? Maybe there was a power imbalance, that has never been revisited, updated, questioned? Or maybe their bottle is being filled by something else in their life and getting too small to contain all the drops of suppressed emotions. People change, and relationships change. We cannot assume that because a person has accepted something for months or years, that they always will. I think that anger can be a sign of such change or need thereof.

A message has four sides and that creates misunderstandings Friedemann Schulz von Thun has created a theory, the four-sides model, that establishes four facets of a message:
  • Fact: What is the message about?
  • Self-disclosure: What does the speaker reveal about herself (with or without intention)?
  • Relation: What is the speaker s relationship towards the receiver of the message?
  • Appeal: What does the speaker want to obtain?
Let me adapt Schulz von Thun s example to a situation that happened to me once: I came to a friend s house and I smelled some unknown thing when I entered the apartment. I asked: what s that smell? I actually never bake, so I don t know anything about whatever it is people put into cakes. I wanted to say: I don t know what that smell is, tell me what it is? She replied: You never like anything I do, my furniture gets criticized, my cake s not right, and you criticize me all the time. We ended up in such a big misunderstanding that I left her house 5 minutes after I arrived. With Schulz von Thun s model we can understand what happened here: The speaker s question What s that smell in the kitchen? has four sides:
  1. Factual: There is a smell.
  2. Self-disclosure: I don t know what it is.
  3. Relational: You know what it is.
  4. Appeal: Tell me what it is!
The receiver can hear the question with 4 different ears:
  1. Factual part: There is a smell.
  2. Self-disclosure: I don t like the smell.
  3. Relationship: You re not good at baking cake.
  4. Appeal: Don t bake cake anymore.
At this point, the receiver will probably reply: Bake the cake yourself next time!

Behind the message Sometimes, we hear more strongly with one ear than with the other ears. For example, some people hear more on the relational side and they will always hear You re not good at . Other people hear more on the appeal side and will always try to guess into a message what the other person wants or expects of them.

Form and contents of a message Getting back to anger, I think that it s often not the contents i.e. the factual facet of a message that triggers our bottle to overflow, but the (perceived) intention of the speaker, i.e. the appeal facet. For example, a speaker might have an intention to silence us by using gaslighting, or tone policing. Or a speaker wants to explicitly hurt us because that s how they learnt to deal with their own feelings of hurt. The actual relation between speaker and receiver also seems to play a role in how anger can get triggered, when we hear with the relational ear. There are many nonverbal underlying layers to our communication:
  • The position of both speaker and receiver to each other: are they equals or is there any kind of power imbalance between them? A power imbalance can be a (perceived) dependency: For example, one person in a friendship has a kid and the other one regularly helps them so that the parent has some time to advance their work or career: this can create a feeling of not being good enough by oneself and having to rely on others. Or there is indeed a dependency in which the speaker is a team lead and the receiver a subordinate.
  • The needs of each person: a need to solve problems quickly for one person might conflict with the need to be involved in decision making of the other person. (See Taibi Kahler s drivers)
  • The inner beliefs of each person: in childhood we might have constructed the inner belief: You re okay, I m not okay , or Nobody cares about what I want , or I have to be nice (friendly, hardworking, strong, etc.) all the time otherwise nobody loves me - as some examples.
So, suppose person A offers to help person B, and what person B hears instead is their overprotective mother instead of their friend. Person A might be surprised to see B overreact or disappear for a while to get back their feelings of autonomy and integrity as an adult. Or person C expresses she d like to handle problem X like this while person D might well not hear this as a proposal to handle a problem, but as a decision made without involving her. Which might propel D back to her childhood in which she could also not take decisions autonomously. Then D might react like she would react as a child, slamming a door, shout, run away, freeze, or fawn.

Communication is complex! So, whenever we hear a message, not only do we hear the message with four ears, but also, we hear it with our position, our needs, and our inner beliefs. And sometimes, our bottle is already full, and then it overflows. At that point, I find it important that both sides reflect on the situation and stay in contact. Using empathy and compassion, we can try to better understand what s going on, where we might have hurt the other person, for example, or what was being misunderstood. Did we hear only one side of the message? Do our respective strategies conflict with each other? If we cannot hear each other anymore or always hear only one side of the message, we can try to do a mediation. Obviously, if at that point one side does not actually want to solve the problem, or thinks it s not their problem, then there s not much we can do, and mediation would not help. The examples above might sound familiar to you. I chose them because I ve seen them happen around me often, and I understand them as shared patterns. While all beings on this planets are unique, we share a common humanity, for example through such patterns and common experiences.

John Goerzen: A Simple, Delay-Tolerant, Offline-Capable Mesh Network with Syncthing (+ optional NNCP)

A little while back, I spent a week in a remote area. It had no Internet and no cell phone coverage. Sometimes, I would drive in to town where there was a signal to get messages, upload photos, and so forth. I had to take several devices with me: my phone, my wife s, maybe a laptop or a tablet too. It seemed there should have been a better way. And there is. I ll use this example to talk about a mesh network, but it could just as well apply to people wanting to communicate on a 12-hour flight that has no in-flight wifi, or spacecraft with an intermittent connection, or a person traveling. Syncthing makes a wonderful solution for things like these. Here are some interesting things about Syncthing: Syncthing works by having you define devices and folders. You can choose which devices to share folders with. A shared folder has an ID that is unique across Sycnthing. You can share a folder from device A to device B, and then device B can share it with device C, even if A and C don t know about each other or have no way to communicate. More commonly, though, all the devices would know about each other and will opportunistically communicate the best way they can. Syncthing uses something akin to a Bittorrent protocol. Say you re syncing videos from your phone, and they re going to 3 machines. It doesn t mean that Syncthing has to send it three times from the phone. Syncthing will send each block, most likely, just once; the other nodes in the swarm will register the block availability from the first other node to get it and will exchange blocks with themselves. Syncthing will typically look for devices on the local LAN. Failing that, it will use an introduction server to see if it can reach them directly using P2P. Failing that, perhaps due to restrictive firewalls or NAT, communication can be relayed through volunteer-run Syncthing servers on the Internet. All Syncthing communications are cryptographically encrypted and verified. You can also configure Syncthing arbitrarily; for instance, to run over ssh or Tor tunnels. So, let s look at how Syncthing might help with the example I laid out up front. All the devices at the remote location could communicate with each other. The Android app is quite capable of syncing photos and videos using Syncthing, for instance. Then one device could be taken to the Internet location and it would transmit data on behalf of all the others perhaps back to a computer at your home, or to a server somewhere. Perhaps a script running on the remote server would then move files out of the syncthing synced folder into permanent storage elsewhere, triggering a deletion to be sent to the phone to free up storage. When the phone gets back to the other devices, the deletion can be propagated to them to free up storage there too. Or maybe you have a computer out in a shed or somewhere without Internet access that you go to periodically, and need to get files to it. Again, your phone could be a carrier. Taking it a step further If you envision a file as a packet, you could, conceivably, do something like tunnel TCP/IP over Syncthing, assuming generous-enough timeouts. It can truly handle communication. But you don t need TCP/IP for this. Consider some other things you could do: You can start to see how there are a lot of possibilities here that extend beyond just file synchronization, though they are built upon a file synchronization tool. Enter NNCP Let s look at a tool that s especially suited for this: NNCP, which I ve been writing about a lot lately. NNCP is designed to handle file exchange and remote execution with remote computers in an asynchronous, store-and-forward manner. NNCP packets are themselves encrypted and authenticated. NNCP traditionally is source-routed (that is, you configure it so that machine A reaches machine D by relaying through B and C), and the packets are onion-routed. NNCP packets can be exchanged by a TCP call, a tar-like stream, copying files to something like a USB stick and physically transporting it to the remote, etc. This works really well and I ve been using it myself. But it gets complicated if the network topology isn t fixed; it is difficult to reroute packets due to the onion routing, for instance. There are various workarounds that could be used but why not just use Syncthing as a transport in those cases? nncp-xfer is the command that exchanges packets by writing them to, and reading them from, a directory. It is what you d use to exchange packets on a USB stick. And what you d use to exchange packets via Syncthing. It writes packets in a RECIPIENT/SENDER/PACKET directory structure, so it is perfectly fine to have multiple systems exchanging packets in a single Syncthing synced folder tree. This structure also allows leaf nodes to only carry the particular packets they re interested in. The packets are all encrypted, so they can be freely synced wherever. Since Syncthing opportunistically syncs a shared folder with any device the folder is shared with, a phone could very easily be the NNCP transport, even if it has no idea what NNCP is. It could carry NNCP packets back and forth between sites, or to the Internet, or whatever. NNCP supports file transmission, file request, and remote execution, all subject to controls, of course. It is easy to integrate with Exim or Postfix to use as a mail transport, Git transport, and so forth. I use it for backups. It would be quite easy to have it send those backups (encrypted zfs send) via nncp-xfer to Syncthing instead of the usual method, and then if I ve shared the Syncthing folder with my phone, all I need to do is bring the phone into Internet range and they get sent. nncp-xfer will normally remove the packets out of the xfer directory as it ingests them, so the space will only be consumed on the phone (and laptop) until we know the packets made it to their destination. Pretty slick, eh?

31 January 2021

Chris Lamb: Free software activities in January 2021

Here is my monthly update covering what I have been doing in the free software world during January 2021 (previous month):

Reproducible Builds One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. This month, I:
I also made the following changes to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, releasing version 164, version 165 and version 166 as well as triaging and merging many contributions from others:

Debian Uploads Debian LTS This month I worked 18 hours on Debian Long Term Support (LTS) and 12 hours on its sister Extended LTS project. You can find out more about the project via the following video:

John Goerzen: The Hidden Drawbacks of P2P (And a Defense of Signal)

Not long ago, I posted a roundup of secure messengers with off-the-grid capabilities. Some conversation followed, which led me to consider some of the problems with P2P protocols. P2P and Privacy Brave adopting IPFS has driven a lot of buzz lately. IPFS is essentially a decentralized, distributed web. This concept has a lot of promise. But take a look at the IPFS privacy document. Some things to highlight: So in this case, you have traded giving information about what you request to specific sites to giving it to potentially hundreds of untrusted peers, some of which may be logging this for nefarious purposes. Worse, you have a durable PeerID that can be used for tracking and tied to your IP address a data collector s dream. This PeerID, combined with DHT requests and the CIDs (Content ID) of the things you host (implying you viewed them in the past), can be used to establish a picture of what you are requesting now and requested recently. Similar can be said from everything like Scuttlebutt to GNU Jami; any service that operates on a P2P basis will likely reveal your IP, and tie your identity to it (and your IP address history). In some cases, as with Jami, this would be limited to friends you add; in others, as with Scuttlebutt and IPFS, it could be revealed to anyone. The advantages of P2P are undeniable and profound, but few are effectively addressing the privacy implications. The one I know of that is, Briar, routes all traffic over Tor; every node is reached by a Tor onion service. Federation: somewhat better In a federated model, every client connects to a server, and there are many servers participating in a federation with each other. Matrix and Mastodon are examples of a federated model. In this scenario, only one server your own homeserver can track you by IP. End-to-end encryption is certainly possible in a federated model, and Matrix supports it. This does give a third party (the specific server you use) knowledge of your IP, but that knowledge can be significantly limited. A downside of this approach is that if your particular homeserver is down, you are unable to communicate. Truly decentralized P2P solutions don t have that problem thought they do have a related one, which is that clients communicating with each other must both be online simultaneously in order for messages to be transmitted, and this can be a real challenge for mobile devices. Centralization and Signal Signal is centralized; it has one central server farm, and if it is down, you can t communicate or choose any other server, either. We saw it go down recently after Elon Musk mentioned it. Still, I recommend Signal for the general public. Here s why. Signal brings encryption and privacy to meet people where they re at, not the other way around. People don t have to choose a server, it can automatically recognize contacts that use Signal, it has emojis, attachments, secure voice and video calling, and (aside from the Musk incident), it all just works. It feels like, and is, a polished, modern experience with the bells and whistles people are used to. I m a huge fan of Matrix (aka Element) and even run my own instance. It has huge promise. But it is Not. There. Yet. Why do I saw this about Matrix? Again, I love MAtrix. I use it every day to interact with Matrix, IRC, Slack, and Discord channels. It has a ton of promise. But would I count on it to carry a my car s broken down and I m stranded message? No. How about some of the other options out there? I mentioned Briar above. It s fantastic and its offline options are novel and promising. But in common usage, it can t deliver a message unless both devices are online simultaneously, and doesn t run on iOS (though both are being worked on). It also can t send photos or do voice or video calling. Some of these same limitations apply to most of the other Signal alternatives also. either that, or they are encryption-optional, or terribly hard to set up and use. I recently mentioned Status, which shows a ton of promise, but has no voice or video calling capabilities. Scuttlebutt is a fantastic protocol with extremely difficult onboarding (lengthy process, error-prone finding a pub, multi-GB initial download, etc.) And many of these leak IP addresses as discussed above. So Signal gives people: If you are going to tell someone, it s so EASY to get your texts away from Facebook and AT&T , then Signal is the thing you ve got to point them to. It may not be in two years, but for now, it is. Do not let the perfect be the enemy of the good. It advances the status quo without harming usability, which nothing else does yet. I am aware of all of the very legitimate criticisms of Signal. They are real and they are why I am excited that there are so many alternatives with promise, some of which I use actively. Let us technical people use, debug, contribute to, and evangelize the alternatives. And while we re doing that, tell Grandma to contact us on Signal.

29 January 2021

Jamie McClelland: So... is Signal good or bad?

After Facebook updated their Whatsapp privacy policy, and a certain rich capitalist who doesn't like Facebook for reasons different then mine told the world to use Signal, Signal's downloads went up by 4,200%. As often happens when something becomes popular, the criticisms start to fly! For the record, I currently think promoting Signal is an important tactical strategy for the left. [I also think we should promote and install federated chat apps like conversations and element and delta chat whereever it is possible.] Here are some of the main criticisms I hear that I think are a distraction: Here are some criticisms that I think are nuanced: These last two are not exactly two sides of the same coin, but they are related. How Signal manages to balance privacy and protection from abuse will be the real test as to whether promoting Signal continues to be a useful strategy for the left.

26 January 2021

Russ Allbery: Review: The City We Became

Review: The City We Became, by N.K. Jemisin
Series: The Great Cities Trilogy #1
Publisher: Orbit
Copyright: March 2020
ISBN: 0-316-50985-X
Format: Kindle
Pages: 449
At an unpredictable point after a city has accumulated enough history, character, and sense of itself, it is born as a living creature. It selects an avatar who both embodies the city and helps it be born. But in that moment of birth, a city is at its most vulnerable, and that is when the Enemy attacks. The birth of cities and the Enemy attacks have happened for millennia, so the cities that have survived the process have developed a system. The most recently born goes to assist the avatar with the birthing process and help fight off Enemy attacks. But the process has become riskier and the last two cities have failed to be born, snuffed out in natural disasters despite that support. Now, it's New York City's turn. It selects its avatar and survives the initial assault with the help of S o Paulo. But something still goes wrong, and it is severely injured in the process. Complicating matters, now there are five more avatars, one for each borough, who will have to unite to fight off the Enemy. And, for the first time, the Enemy has taken human form and is attacking with reason and manipulation and intelligence, not just force. The City We Became has a great premise: take the unique sense of place that defines a city and turn it into a literalized character in a fantasy novel. The avatars are people who retain their own lives and understanding (with one exception that I'll get to in a moment), but gain an awareness of the city they represent. They can fight and repair their city through sympathetic magic and metaphor made real. The prelude that introduces this concept (adapted from Jemisin's earlier short story "The City Born Great") got too gonzo for me, but once Jemisin settles into the main story and introduces avatars with a bit more distance from the city they represent, the premise clicked. The execution, on the other hand, I thought was strained. The biggest problem is that the premise requires an ensemble cast of five borough avatars, the primary avatar, S o Paulo, and the Enemy. That's already a lot, but for the story to work each avatar has to be firmly grounded in their own unique experience of New York, which adds family members, colleagues, and roommates. That's too much to stuff into one novel, which means characters get short shrift. For example, Padmini, the avatar of Queens, gets a great introductory scene and a beautiful bit of characterization that made her one of my favorite characters, but then all but disappears for the remainder of the book. She's in the scenes, but not in a way that matters. Brooklyn and Aislyn get moments of deep characterization, but there's so much else going on that they felt rushed. And what ever happened to Manny's roommate? The bulk of the characterization in this book goes to Broncha, the Bronx avatar, a Lenape woman and a tough-as-nails administrator of a community art museum and maker space. The dynamics between her and her co-workers, her mentorship of Veneza, and her early encounters with the Woman in White are my favorite parts of the book. I thought she and Brooklyn were a useful contrast: two very different ways of finding a base of power in the city without letting go of one's ideals. But before we get to Broncha, we first meet Manny, the Manhattan avatar. Thematically, I thought what Jemisin did here was extremely clever. Manny's past is essentially erased at the start of the book, so he's making sense of this world in the same way the reader is. It makes him a useful reader insert character and parallels the typical tourist experience of arriving in Manhattan and trying to use it to make sense of New York. He's disconnected from the rest of the city because he's the dangerous newcomer with power but not a lot of understanding, which works with my model of the political dynamics of Manhattan. Unfortunately, he's not an interesting person. I appreciated what was happening at the metaphorical layer, but Manny veers between action hero and exposition prompt, and his amnesia meant I never got enough sense of him as a character to care that much about what happened to him. I thought his confrontation with the Woman in White near the start of the book, which establishes the major villain of the book, felt clunky and forced compared to her later encounters with the other characters. The Woman in White, though, is a great villain. It's clear from earlier on that the Enemy is Lovecraftian, but the Woman in White mixes mad scientist glee, manic enthusiasm, and a child-like amusement at the weirdness of humanity into the more typical tropes of tentacles, corruption, and horrific creatures. One of my qualms about reading this book is that I'm not a horror fan and don't enjoy the mental images of unspeakable monsters, but the Woman in White puts such a fascinating spin on them that I enjoyed the scenes in which she appeared. I think the book was at its best when she was trying to psychologically manipulate the characters or attack them with corrupted but pre-existing power structures. I was less interested when it turned into an action-movie fight against animated monsters. The other place Jemisin caught me by surprise is too much of a spoiler to describe in detail (and skip the next paragraph in its entirety if you want to avoid all spoilers): Jemisin didn't take the moral conflict of the book in the direction I was expecting. This book is more interested in supporting the people who are already acting ethically than in redeeming people who make bad choices. That produces a deus ex machina ending that's a bit abrupt, but I appreciated the ethical stance. Overall, I thought the premise was great but the execution was unsteady and a bit overstuffed. There are some great characters and some great scenes, but to me they felt disjointed and occasionally rushed. You also need to enjoy characters taking deep pride in the feel of a specific place and advocating for it with the vigor of a sports rivalry, along with loving descriptions of metaphors turned into magical waves of force. But, if you can roll with that, there are moments of real awe. Jemisin captured for me the joy that comes from a deeply grounded sense of connection to a place. Recommended, albeit with caveats, if you're in the mood for reading about people who love the city they live in. This is the first book of a planned trilogy and doesn't resolve the main conflict, but it reaches a satisfying conclusion. The title of the next book has not yet been announced at the time of this review. Rating: 7 out of 10

25 January 2021

Russ Allbery: Review: Laziness Does Not Exist

Review: Laziness Does Not Exist, by Devon Price
Publisher: Atria Books
Copyright: January 2021
ISBN: 1-9821-4013-5
Format: Kindle
Pages: 216
The premise of Laziness Does Not Exist is in the title: Laziness as a moral failing does not exist. It is a misunderstanding of other problems with physical or psychological causes, a belief system that is used to extract unsustainable amounts of labor, an excuse to withdraw our empathy, and a justification for not solving social problems. Price refers to this as the Laziness Lie, which they define with three main tenets:
  1. Your worth is your productivity.
  2. You cannot trust your own feelings and limits.
  3. There is always more you could be doing.
This book (an expansion of a Medium article) makes the case against all three tenets using the author's own burnout-caused health problems as the starting argument. They then apply that analysis to work, achievements, information overload, relationships, and social pressure. In each case, Price's argument is to prioritize comfort and relaxation, listen to your body and your limits, and learn who you are rather than who the Laziness Lie is trying to push you to be. The reader reaction to a book like this will depend on where the reader is in thinking about the problem. That makes reviewing a challenge, since it's hard to simulate a reader with a different perspective. For myself, I found the content unobjectionable, but largely repetitive of other things I've read. The section on relationships in particular will be very familiar to Captain Awkward readers, just not as pointed. Similarly, the chapter on information overload is ground already covered by Digital Minimalism, among other books. That doesn't make this a bad book, but it's more of a survey, so if you're already well-read on this topic you may not get much out of it. The core assertion is aggressive in part to get the reader to argue with it and thus pay attention, but I still came away convinced that laziness is not a useful word. The symptoms that cause us to call ourselves lazy procrastination, burnout, depression, or executive function problems, for example are better understood without the weight of moral reproach that laziness carries. I do think there is another meaning of laziness that Price doesn't cover, since they are aiming this book exclusively at people who are feeling guilty about possibly being lazy, and we need some term for people who use their social power to get other people to do all their work for them. But given how much the concept of laziness is used to attack and belittle the hard-working or exhausted, I'm happy to replace "laziness" with "exploitation" when talking about that behavior. This is a profoundly kind and gentle book. Price's goal is to help people be less hard on themselves and to take opportunities to relax without guilt. But that also means that it stays in the frame of psychological analysis and self-help, and only rarely strays into political or economic commentary. That means it's more useful for taking apart internalized social programming, but less useful for thinking about the broader political motives of those who try to convince us to work endlessly and treat all problems as personal responsibilities rather than political failures. For that, I think Anne Helen Peterson's Can't Even is the more effective book. Price also doesn't delve much into history, and I now want to read a book on the origin of a work ethic as a defining moral trait. One truly lovely thing about this book is that it's quietly comfortable with human variety of gender and sexuality in a way that's never belabored but that's obvious from the examples that Price uses. Laziness Does Not Exist felt more inclusive in that way, and to some extent on economic class, than Can't Even. I was in the mood for a book that takes apart the political, social, and economic motivations behind convincing people that they have to constantly strive to not be lazy, so the survey nature of this book and its focus on self-help made it not the book for me. It also felt a bit repetitive despite its slim length, and the chapter structure didn't click for me. But it's not a bad book, and I suspect it will be the book that someone else needs to read. Rating: 6 out of 10

22 January 2021

Robert McQueen: Launching Endless OS Foundation

Passion Led Us Here

How our for-profit company became a nonprofit, to better tackle the digital divide. Originally posted on the Endless OS Foundation blog.

An 8-year journey to a nonprofit On the 1st of April 2020, our for-profit Endless Mobile officially became a nonprofit as the Endless OS Foundation. Our launch as a nonprofit just as the global pandemic took hold was, predictably, hardly noticed, but for us the timing was incredible: as the world collectively asked What can we do to help others in need? , we framed our mission statement and launched our .org with the same very important question in mind. Endless always had a social impact mission at its heart, and the challenges related to students, families, and communities falling further into the digital divide during COVID-19 brought new urgency and purpose to our team s decision to officially step in the social welfare space.
On April 1st 2020, our for-profit Endless Mobile officially became a nonprofit as the Endless OS Foundation, focused on the #DigitalDivide.
Our updated status was a long time coming: we began our transformation to a nonprofit organization in late 2019 with the realization that the true charter and passions of our team would be greatly accelerated without the constraints of for-profit goals, investors and sales strategies standing in the way of our mission of digital access and equity for all. But for 8 years we made a go of it commercially, headquartered in Silicon Valley and framing ourselves as a tech startup with access to the venture capital and partnerships on our doorstep. We believed that a successful commercial channel would be the most efficient way to scale the impact of bringing computer devices and access to communities in need. We still believe this we ve just learned through our experience that we don t have the funding to enter the computer and OS marketplace head-on. With the social impact goal first, and the hope of any revenue a secondary goal, we have had many successes in those 8 years bridging the digital divide throughout the world, from Brazil, to Kenya, and the USA. We ve learned a huge amount which will go on to inform our strategy as a nonprofit.
Endless always had a social impact mission at its heart. COVID-19 brought new urgency and purpose to our team s decision to officially step in the social welfare space.
Our unique perspective One thing we learned as a for-profit is that the OS and technology we ve built has some unique properties which are hugely impactful as a working solution to digital equity barriers. And our experience deploying in the field around the world for 8 years has left us uniquely informed via many iterations and incremental improvements.
Endless OS designer in discussion with prospective user
With this knowledge in-hand, we ve been refining our strategy throughout 2020 and now starting to focus on what it really means to become an effective nonprofit and make that impact. In many ways it is liberating to abandon the goals and constraints of being a for-profit entity, and in other ways it s been a challenging journey for me and the team to adjust our way of thinking and let these for-profit notions and models go. Previously we exclusively built and sold a product that defined our success; and any impact we achieved was a secondary consequence of that success and seen through that lens. Now our success is defined purely in terms of social impact, and through our actions, those positive impacts can be made with or without our product . That means that we may develop and introduce technology to solve a problem, but it is equally as valid to find another organization s existing offering and design a way to increase that positive impact and scale.
We develop technology to solve access equity issues, but it s equally as valid to find another organization s offering and partner in a way that increases their positive impact.
The analogy to Free and Open Source Software is very strong while Endless has always used and contributed to a wide variety of FOSS projects, we ve also had a tension where we ve been trying to hold some pieces back and capture value such as our own application or content ecosystem, our own hardware platform necessarily making us competitors to other organisations even though they were hoping to achieve the same things as us. As a nonprofit we can let these ideas go and just pick the best partners and technologies to help the people we re trying to reach.
School kids writing on paper
Digital equity 4 barriers we need to overcome In future, our decisions around which projects to build or engage with will revolve around 4 barriers to digital equity, and how our Endless OS, Endless projects, or our partners offerings can help to solve them. We define these 4 equity barriers as: barriers to devices, barriers to connectivity, barriers to literacy in terms of your ability to use the technology, and barriers to engagement in terms of whether using the system is rewarding and worthwhile.
We define the 4 digital equity barriers we exist to impact as:
1. barriers to devices
2. barriers to connectivity
3. barriers to literacy
4. barriers to engagement
It doesn t matter who makes the solutions that break these barriers; what matters is how we assist in enabling people to use technology to gain access to the education and opportunities these barriers block. Our goal therefore is to simply ensure that solutions exist building them ourselves and with partners such as the FOSS community and other nonprofits proving them with real-world deployments, and sharing our results as widely as possible to allow for better adoption globally. If we define our goal purely in terms of whether people are using Endless OS, we are effectively restricting the reach and scale of our solutions to the audience we can reach directly with Endless OS downloads, installs and propagation. Conversely, partnerships that scale impact are a win-win-win for us, our partners, and the communities we all serve. Engineering impact Our Endless engineering roots and capabilities feed our unique ability to build and deploy all of our solutions, and the practical experience of deploying them gives us evidence and credibility as we advocate for their use. Either activity would be weaker without the other.
Our engineering roots and capabilities feed our unique ability to build and deploy digital divide solutions.
Our partners in various engineering communities will have already seen our change in approach. Particularly, with GNOME we are working hard to invest in upstream and reconcile the long-standing differences between our experience and GNOME. If successful, many more people can benefit from our work than just users of Endless OS. We re working with Learning Equality on Kolibri to build a better app experience for Linux desktop users and bring content publishers into our ecosystem for the first time, and we ve also taken our very own Hack, the immersive and fun destination for kids learning to code, released it for non-Endless systems on Flathub, and made it fully open-source.
Planning tasks with sticky notes on a whiteboard
What s next for our OS? What then is in store for the future of Endless OS, the place where we have invested so much time and planning through years of iterations? For the immediate future, we need the capacity to deploy everything we ve built all at once, to our partners. We built an OS that we feel is very unique and valuable, containing a number of world-firsts: first production OS shipped with OSTree, first Flatpak-only desktop, built-in support for updating OS and apps from USBs, while still providing a great deal of reliability and convenience for deployments in offline and educational-safe environments with great apps and content loaded on every system. However, we need to find a way to deliver this Linux-based experience in a more efficient way, and we d love to talk if you have ideas about how we can do this, perhaps as partners. Can the idea of Endless OS evolve to become a spec that is provided by different platforms in the future, maybe remixes of Debian, Fedora, openSUSE or Ubuntu? Build, Validate, Advocate Beyond the OS, the Endless OS Foundation has identified multiple programs to help underserved communities, and in each case we are adopting our build, validate, advocate strategy. This approach underpins all of our projects: can we build the technology (or assist in the making), will a community in-need validate it by adoption, and can we inspire others by telling the story and advocating for its wider use?
We are adopting a build, validate, advocate strategy.
1. build the technology (or assist in the making)
2. validate by community adoption
3. advocate for its wider use
As examples, we have just launched the Endless Key (link) as an offline solution for students during the COVID-19 at-home distance learning challenges. This project is also establishing a first-ever partnership of well-known online educational brands to reach an underserved offline audience with valuable learning resources. We are developing a pay-as-you-go platform and new partnerships that will allow families to own laptops via micro-payments that are built directly into the operating system, even if they cannot qualify for standard retail financing. And during the pandemic, we ve partnered with Teach For America to focus on very practical digital equity needs in the USA s urban and rural communities. One part of the world-wide digital divide solution We are one solution provider for the complex matrix of issues known collectively as the #DigitalDivide, and these issues will not disappear after the pandemic. Digital equity was an issue long before COVID-19, and we are not so naive to think it can be solved by any single institution, or by the time the pandemic recedes. It will take time and a coalition of partnerships to win. We are in for the long-haul and we are always looking for partners, especially now as we are finding our feet in the nonprofit world. We d love to hear from you, so please feel free to reach out to me I m ramcq on IRC, RocketChat, Twitter, LinkedIn or rob@endlessos.org.

Enrico Zini: Polishing nspawn-runner

This post is part of a series about trying to setup a gitlab runner based on systemd-nspawn. I published the polished result as nspawn-runner on GitHub. gitlab-runner supports adding extra arguments to the custom scripts, and I can take advantage of that to pack all the various scripts that I prototyped so far into an all-in-one nspawn-runner command:
usage: nspawn-runner [-h] [-v] [--debug]
                      chroot-create,chroot-login,prepare,run,cleanup,gitlab-config,toml 
                     ...
Manage systemd-nspawn machines for CI runs.
positional arguments:
   chroot-create,chroot-login,prepare,run,cleanup,gitlab-config,toml 
                        sub-command help
    chroot-create       create a chroot that serves as a base for ephemeral
                        machines
    chroot-login        enter the chroot to perform maintenance
    prepare             start an ephemeral system for a CI run
    run                 run a command inside a CI machine
    cleanup             cleanup a CI machine after it's run
    gitlab-config       configuration step for gitlab-runner
    toml                output the toml configuration for the custom runner
optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         verbose output
  --debug               verbose output
chroot maintenance chroot-create and chroot-login are similar to what pbuilder, cowbuilder, schroot, debspawn and similar tools do. They only take a chroot name, and default the rest of paths to where nspawn-runner expects things to be under /var/lib/nspawn-runner. gitlab-runner setup nspawn-runner toml <chroot-name> outputs a snippet to add to /etc/gitlab-runner/config.toml to configure the CI. For example:
$ ./nspawn-runner toml buster
[[runners]]
  name="buster"
  url="TODO"
  token="TODO"
  executor = "custom"
  builds_dir = "/var/lib/nspawn-runner/.build"
  cache_dir = "/var/lib/nspawn-runner/.cache"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.custom]
    config_exec = "/home/enrico/ /nspawn-runner/nspawn-runner"
    config_args = ["gitlab-config"]
    config_exec_timeout = 200
    prepare_exec = "/home/enrico/ /nspawn-runner/nspawn-runner"
    prepare_args = ["prepare", "buster"]
    prepare_exec_timeout = 200
    run_exec = "/home/enrico/dev/nspawn-runner/nspawn-runner"
    run_args = ["run"]
    cleanup_exec = "/home/enrico/ /nspawn-runner/nspawn-runner"
    cleanup_args = ["cleanup"]
    cleanup_exec_timeout = 200
    graceful_kill_timeout = 200
    force_kill_timeout = 200
One needs to remember to set url and token, and the runner is configured. The end, for now This is it, it works! Time will tell what issues or ideas will come up: for now, it's a pretty decent first version. The various prepare, run, cleanup steps are generic enough that they can be used outside of gitlab-runner: feel free to build on them, and drop me a note if you find this useful! Updated: Issues noticed so far, that could go into a new version:

Next.

Previous.