Search Results: "cts"

4 February 2021

Ulrike Uhlig: On anger, misunderstandings, and hearing with different ears

Anger Anger is a feeling that is mostly taboo in our society. People tend to think that anger and rage are the same thing and reject anger. We are taught to suppress it. But: The suppression of anger can cause a lot of trouble, giving rise to virulent progeny such as malice, passive aggression, hostility, rage, sabotage, hate, blame, guilt, controlling behavior, shame, self-blame, and self-destruction. (Quoted from: Anne Katherine, Where to draw the line ). When we talk about anger, we need to distinguish between acting out anger, and feeling anger. In general, when you hear me talking about anger, I m talking about the feeling named anger in English. Anger is a normal feeling with a super power: it gives us the energy to change a situation that we consider to be unsustainable. We can express the feeling of anger verbally by saying for example: That thing makes me really angry , I can t talk right now, I am very angry about what I just heard . Let s also note that there may be other feelings lying below anger, such as sadness, frustration, grief, fear, etc. Mixing emotions This is a page from the wonderful Making Comics by Scott McCloud. (Yes, emotions are complex! See some more of Scott McCloud s pages about emotion in comics)

Bottling up feelings Most of the time, people don t get angry suddenly, even though it might seem like that from the outside. Instead, they ve been bottling up feelings for a while, and at some point, a small trigger is enough to make the bottle overflow. The reasons for bottling up feelings can be as diverse as people:
  • Low self-esteem: not thinking one has the right and the capacity to express unpleasant feelings
  • Not knowing how to express unpleasant feelings. For example not having learnt to say: I am not comfortable, I ll leave now, I ll get back to you once I know what s going on / if this is about me / if I feel like it.
  • Not trusting one s own feelings, specifically common in people who have been victims of gaslighting
  • Repressing emotions, specifically anger, sometimes to the point of not being able to feel one s own feelings anymore
  • Not being able to express disagreement and instead applying the fawn response: trying to please the other in order to avoid further conflict. See: Fight, flight, freeze, or fawn
  • Not being able to express boundaries: stop , this is enough , I don t accept this , etc.
  • Not being in a situation or a space in which feelings can be expressed freely, for example a workplace or a hierarchical situation where this might be disadvantageous
These are just some examples I can come up with in 3 minutes, the list is non exhaustive. When someone s bottle overflows, their peers can be confused and not know how to react to what they perceive as a sudden outburst while what they are seeing might just be the other person s first time tentative of saying no or stop . Oftentimes, the responsibility for the situation is then put on the shoulders of the person who supposedly exploded : they have been behaving differently than usual or not within the expectations, right? Well, it s not that simple. Conflicts are relational. The other party might have ignored signs, requests, feelings, and needs of the angry person for a while. Maybe the relationship has been deteriorating since some time? Maybe there was a power imbalance, that has never been revisited, updated, questioned? Or maybe their bottle is being filled by something else in their life and getting too small to contain all the drops of suppressed emotions. People change, and relationships change. We cannot assume that because a person has accepted something for months or years, that they always will. I think that anger can be a sign of such change or need thereof.

A message has four sides and that creates misunderstandings Friedemann Schulz von Thun has created a theory, the four-sides model, that establishes four facets of a message:
  • Fact: What is the message about?
  • Self-disclosure: What does the speaker reveal about herself (with or without intention)?
  • Relation: What is the speaker s relationship towards the receiver of the message?
  • Appeal: What does the speaker want to obtain?
Let me adapt Schulz von Thun s example to a situation that happened to me once: I came to a friend s house and I smelled some unknown thing when I entered the apartment. I asked: what s that smell? I actually never bake, so I don t know anything about whatever it is people put into cakes. I wanted to say: I don t know what that smell is, tell me what it is? She replied: You never like anything I do, my furniture gets criticized, my cake s not right, and you criticize me all the time. We ended up in such a big misunderstanding that I left her house 5 minutes after I arrived. With Schulz von Thun s model we can understand what happened here: The speaker s question What s that smell in the kitchen? has four sides:
  1. Factual: There is a smell.
  2. Self-disclosure: I don t know what it is.
  3. Relational: You know what it is.
  4. Appeal: Tell me what it is!
The receiver can hear the question with 4 different ears:
  1. Factual part: There is a smell.
  2. Self-disclosure: I don t like the smell.
  3. Relationship: You re not good at baking cake.
  4. Appeal: Don t bake cake anymore.
At this point, the receiver will probably reply: Bake the cake yourself next time!

Behind the message Sometimes, we hear more strongly with one ear than with the other ears. For example, some people hear more on the relational side and they will always hear You re not good at . Other people hear more on the appeal side and will always try to guess into a message what the other person wants or expects of them.

Form and contents of a message Getting back to anger, I think that it s often not the contents i.e. the factual facet of a message that triggers our bottle to overflow, but the (perceived) intention of the speaker, i.e. the appeal facet. For example, a speaker might have an intention to silence us by using gaslighting, or tone policing. Or a speaker wants to explicitly hurt us because that s how they learnt to deal with their own feelings of hurt. The actual relation between speaker and receiver also seems to play a role in how anger can get triggered, when we hear with the relational ear. There are many nonverbal underlying layers to our communication:
  • The position of both speaker and receiver to each other: are they equals or is there any kind of power imbalance between them? A power imbalance can be a (perceived) dependency: For example, one person in a friendship has a kid and the other one regularly helps them so that the parent has some time to advance their work or career: this can create a feeling of not being good enough by oneself and having to rely on others. Or there is indeed a dependency in which the speaker is a team lead and the receiver a subordinate.
  • The needs of each person: a need to solve problems quickly for one person might conflict with the need to be involved in decision making of the other person. (See Taibi Kahler s drivers)
  • The inner beliefs of each person: in childhood we might have constructed the inner belief: You re okay, I m not okay , or Nobody cares about what I want , or I have to be nice (friendly, hardworking, strong, etc.) all the time otherwise nobody loves me - as some examples.
So, suppose person A offers to help person B, and what person B hears instead is their overprotective mother instead of their friend. Person A might be surprised to see B overreact or disappear for a while to get back their feelings of autonomy and integrity as an adult. Or person C expresses she d like to handle problem X like this while person D might well not hear this as a proposal to handle a problem, but as a decision made without involving her. Which might propel D back to her childhood in which she could also not take decisions autonomously. Then D might react like she would react as a child, slamming a door, shout, run away, freeze, or fawn.

Communication is complex! So, whenever we hear a message, not only do we hear the message with four ears, but also, we hear it with our position, our needs, and our inner beliefs. And sometimes, our bottle is already full, and then it overflows. At that point, I find it important that both sides reflect on the situation and stay in contact. Using empathy and compassion, we can try to better understand what s going on, where we might have hurt the other person, for example, or what was being misunderstood. Did we hear only one side of the message? Do our respective strategies conflict with each other? If we cannot hear each other anymore or always hear only one side of the message, we can try to do a mediation. Obviously, if at that point one side does not actually want to solve the problem, or thinks it s not their problem, then there s not much we can do, and mediation would not help. The examples above might sound familiar to you. I chose them because I ve seen them happen around me often, and I understand them as shared patterns. While all beings on this planets are unique, we share a common humanity, for example through such patterns and common experiences.

31 January 2021

Sean Whitton: gemini:// space

Recently I have become curious about the Gemini Project and the content that people have made available to be retrieved over the gemini:// protocol. I m not convinced by the arguments for not just using http, and mostly it s just that I typically find more things that I am interested in casually reading through on people s gemlogs than I would on, say, reddit, and similar aggregators. But presumably advocates of gemini:// and the text/gemini format would argue that it s various respects in which it differs from the web that makes geminispace conducive to the production of the sort of content you find there. So I m remaining open minded about the possibility that having a completely separate protocol is important, and not just an annoyance because rss2email doesn t work and I had to spend time writing gmi2email. I now have a games console at home for the first time in some years, which I bought in response to the ongoing pandemic, and one thing that I have noticed is that using it feels like being offline in a way that playing games on a regular computer never would. It has a WiFi connection but it doesn t have a web browser, and I am glad that using it provides an opportunity to be disconnected from the usual streams of information. And perhaps something similar ought to be said in favour of how the Gemini project does not just use http. There is, perhaps, a positive psychological effect induced by making the boundary between text/gemini and the web as hard as it is made by using gemini:// rather than http. Something about which I find myself much more sceptical is how the specification for gemini:// and text/gemini is not extensible. Advocates of Gemini have this idea that they can t include, say, a version number in the protocol, because the extensibility of the web is what has led to the problems they think it has, so they want to make it impossible. Now on the one hand perhaps the people behind Gemini are in the best position that anyone is in to come up with a spec which they will finalise and render effectively unchangeable, because a lot of them have been using Gopher for decades, and so they have enough experience to be able to say exactly what Gopher is missing, and be confident that they ve not missed anything. But on the other hand, Gemini is one technological piece in attempts to make a version of the Internet which is healthier for humans the so-called small Internet movement and maybe there will be new ideas about how the small Internet should be which would benefit from a new version of the Gemini specification. So it seems risky to lock-in to one version. Comments on Hacker News.

John Goerzen: The Hidden Drawbacks of P2P (And a Defense of Signal)

Not long ago, I posted a roundup of secure messengers with off-the-grid capabilities. Some conversation followed, which led me to consider some of the problems with P2P protocols. P2P and Privacy Brave adopting IPFS has driven a lot of buzz lately. IPFS is essentially a decentralized, distributed web. This concept has a lot of promise. But take a look at the IPFS privacy document. Some things to highlight: So in this case, you have traded giving information about what you request to specific sites to giving it to potentially hundreds of untrusted peers, some of which may be logging this for nefarious purposes. Worse, you have a durable PeerID that can be used for tracking and tied to your IP address a data collector s dream. This PeerID, combined with DHT requests and the CIDs (Content ID) of the things you host (implying you viewed them in the past), can be used to establish a picture of what you are requesting now and requested recently. Similar can be said from everything like Scuttlebutt to GNU Jami; any service that operates on a P2P basis will likely reveal your IP, and tie your identity to it (and your IP address history). In some cases, as with Jami, this would be limited to friends you add; in others, as with Scuttlebutt and IPFS, it could be revealed to anyone. The advantages of P2P are undeniable and profound, but few are effectively addressing the privacy implications. The one I know of that is, Briar, routes all traffic over Tor; every node is reached by a Tor onion service. Federation: somewhat better In a federated model, every client connects to a server, and there are many servers participating in a federation with each other. Matrix and Mastodon are examples of a federated model. In this scenario, only one server your own homeserver can track you by IP. End-to-end encryption is certainly possible in a federated model, and Matrix supports it. This does give a third party (the specific server you use) knowledge of your IP, but that knowledge can be significantly limited. A downside of this approach is that if your particular homeserver is down, you are unable to communicate. Truly decentralized P2P solutions don t have that problem thought they do have a related one, which is that clients communicating with each other must both be online simultaneously in order for messages to be transmitted, and this can be a real challenge for mobile devices. Centralization and Signal Signal is centralized; it has one central server farm, and if it is down, you can t communicate or choose any other server, either. We saw it go down recently after Elon Musk mentioned it. Still, I recommend Signal for the general public. Here s why. Signal brings encryption and privacy to meet people where they re at, not the other way around. People don t have to choose a server, it can automatically recognize contacts that use Signal, it has emojis, attachments, secure voice and video calling, and (aside from the Musk incident), it all just works. It feels like, and is, a polished, modern experience with the bells and whistles people are used to. I m a huge fan of Matrix (aka Element) and even run my own instance. It has huge promise. But it is Not. There. Yet. Why do I saw this about Matrix? Again, I love MAtrix. I use it every day to interact with Matrix, IRC, Slack, and Discord channels. It has a ton of promise. But would I count on it to carry a my car s broken down and I m stranded message? No. How about some of the other options out there? I mentioned Briar above. It s fantastic and its offline options are novel and promising. But in common usage, it can t deliver a message unless both devices are online simultaneously, and doesn t run on iOS (though both are being worked on). It also can t send photos or do voice or video calling. Some of these same limitations apply to most of the other Signal alternatives also. either that, or they are encryption-optional, or terribly hard to set up and use. I recently mentioned Status, which shows a ton of promise, but has no voice or video calling capabilities. Scuttlebutt is a fantastic protocol with extremely difficult onboarding (lengthy process, error-prone finding a pub, multi-GB initial download, etc.) And many of these leak IP addresses as discussed above. So Signal gives people: If you are going to tell someone, it s so EASY to get your texts away from Facebook and AT&T , then Signal is the thing you ve got to point them to. It may not be in two years, but for now, it is. Do not let the perfect be the enemy of the good. It advances the status quo without harming usability, which nothing else does yet. I am aware of all of the very legitimate criticisms of Signal. They are real and they are why I am excited that there are so many alternatives with promise, some of which I use actively. Let us technical people use, debug, contribute to, and evangelize the alternatives. And while we re doing that, tell Grandma to contact us on Signal.

29 January 2021

Jamie McClelland: So... is Signal good or bad?

After Facebook updated their Whatsapp privacy policy, and a certain rich capitalist who doesn't like Facebook for reasons different then mine told the world to use Signal, Signal's downloads went up by 4,200%. As often happens when something becomes popular, the criticisms start to fly! For the record, I currently think promoting Signal is an important tactical strategy for the left. [I also think we should promote and install federated chat apps like conversations and element and delta chat whereever it is possible.] Here are some of the main criticisms I hear that I think are a distraction: Here are some criticisms that I think are nuanced: These last two are not exactly two sides of the same coin, but they are related. How Signal manages to balance privacy and protection from abuse will be the real test as to whether promoting Signal continues to be a useful strategy for the left.

Russ Allbery: INN 2.6.4

INN 2.6.4 has been released. This is a bug fix and minor feature release over INN 2.6.3, and the upgrade should be painless. You can download the new release from ftp.isc.org or my personal INN pages. The latter also has links to the full changelog and the other INN documentation. As always, thanks to Julien LIE for preparing this release and doing most of the maintenance work on INN! Changes in this release:

27 January 2021

Russ Allbery: Review: A Deadly Education

Review: A Deadly Education, by Naomi Novik
Series: The Scholomance #1
Publisher: Del Rey
Copyright: 2020
ISBN: 0-593-12849-4
Format: Kindle
Pages: 319
Some children are born with magic, which grows as they mature. Magic attracts maleficaria: extremely deadly magical beasts that want to feast on that magic. Having innate magical ability is therefore a recipe for endless attacks from monsters and a death at a young age. This was true even for the enclaves, which are the rich, gated communities of the magical world. Hence, the Scholomance. This is a boarding school for magic users placed in the Void and protected against maleficaria as completely as possible while still letting the students graduate and leave after their senior year. Students are sent there via a teleportation spell with a weight allowance, taught magic by automated systems and magical artifacts, and left on their own to make alliances and survive. Or not survive; protected as well as possible still means that there are maleficaria everywhere, sneaking past the wards of the graduation hall and looking for snacks. The school sends cleansing fire through the halls at certain times; the rest of the time, the students either learn enough magic to defeat maleficaria themselves, form alliances with those who can, or die to feed the magic of the school. Enter Galadriel, or El as she prefers. She's not an enclave kid; she's the grumpy, misfit daughter of a hippie mother whose open-hearted devotion to healing and giving away her abilities make her the opposite of the jealously guarded power structures of the enclaves. El has no resources other than what she can muster on her own. She also has her mother's ethics, which means that although she has an innate talent for malia, drawing magic from the death of other living things, she forces herself to build her mana through rigorously ethical means. Like push-ups. Or, worse, crochet. At the start of the book, El is in her third year of four, and significantly more of her classmates are alive than normally would be. That's because of her classmate, Orion Lake, who has made a full-time hobby of saving everyone from maleficaria. His unique magical ability frees him from the constraints of mana or malia that everyone else is subject to, and he uses that to be a hero, surrounded by adoring fans. And El is thoroughly sick of it. This book is so good in so many different ways that I don't know where to start. Obviously, A Deadly Education is a twist on the boarding school novel, both the traditional and the magical kind. This is not a genre in which I'm that well-read, but even with my lack of familiarity, I noticed so many things Novik does to improve the genre tropes, starting with not making the heroic character with the special powers the protagonist. And getting rid of all the adults, which leaves way more space for rich social dynamics between the kids (complex and interesting ones that are entangled with the social dynamics outside of the school, not some simplistic Lord of the Flies take). Going alone anywhere in the school is dangerous, as is sitting at the bad tables in the cafeteria, so social cliques become a matter of literal life and death. And the students aren't just trying to survive; the ones who aren't part of enclaves are jockeying for invitations or trying to build the power to help their family and allies form their own. El is the first-person narrator of the story and she's wonderful. She's grumpy, cynical, and sarcastic, which is often good for first-person narrators, but she also has a core of ethics from her mother, and from her own decisions, that gives her so much depth. She is the type of person who knows exactly how much an ethical choice will cost her and how objectively stupid it is, and then will make it anyway out of sheer stubbornness and refuse to take credit for it. I will happily read books about characters like El until the end of time. Her mother never appears in this book, and yet she's such a strong presence because El's relationship with her matters, to both El and to the book. El could not be more unlike her mother in both personality and in magical focus, and she's exasperated by the sheer impracticality of some of her mother's ideals. And yet there's a core of love and understanding beneath that, a level at which El completely understands her mother's goals, and El relies on it even when she doesn't realize she's doing so. I don't think I've ever read a portrayal of a mother-daughter relationship this good where one of the parties isn't even present. And I haven't even gotten to the world-building, and the level to which Novik chases down and explores all the implications of this ridiculous murder machine of a school. I will offer this caveat: If you poke at the justification for creating this school in the way it was built, it's going to teeter a lot. That society thought this school was the best solution to its child mortality problem is just something you have to roll with. But once you accept that, the implications are handled so very well. The school is an inhuman character in its own right, with exasperating rules that the students learn and warn each other about. It tries to distract you with rare spellbooks or artifact materials because it's trying to kill you. The language tapes whisper horrific stories of your death. The back wall of your room is a window to the Void, from which you can demand spellbooks. You'll even get them in languages that you understand, for a generous definition of understand that may have involved glancing at one page of text, so be careful not to do that! The school replaces all of the adult teachers in the typical boarding school novel and is so much more interesting than any of them because it adds the science fiction thrill of setting as character. The world-building does mean a lot of infodumping, so be prepared for that. El likes to explain things, tell stories, and over-analyze her life, and reading this book is a bit like reading the journal of a teenage girl. For me, El's voice is so strong, authentic, stubborn, and sarcastically funny that I scarcely noticed the digressions into background material. And the relationships! Some of the turns will be predictable, since of course El's stubborn ethics will be (eventually) rewarded by the story, but the dynamic that develops between El and Orion is something special. It takes a lot to make me have sympathy with the chosen one boy hero, but Novik pulls it off without ever losing sight of the dynamics of class and privilege that are also in play. And the friendships El develops almost accidentally by being stubbornly herself are just wonderful, and the way she navigates them made me respect her even more. The one negative thing I will say about this book is that I don't think Novik quite nailed the climax. Some of this is probably because this is the first book of a series and Novik wanted to hold some social developments in reserve, but I thought El got a bit sidelined and ended up along for the ride in an action-movie sequence. Still, it's a minor quibble, and it's clear from the very end of the book that El is going to get more attention and end up in a different social position in the next book. This was a wholly engrossing and enjoyable story with a satisfying climax and only the barb of a cliffhanger in the very last line. It's the best SFF novel published in 2020 that I've read so far (yes, even better than Network Effect). Highly recommended, and I hope it gets award recognition this year. Followed by The Last Graduate (not yet published at the time of this review). Rating: 9 out of 10

26 January 2021

Russ Allbery: Review: The City We Became

Review: The City We Became, by N.K. Jemisin
Series: The Great Cities Trilogy #1
Publisher: Orbit
Copyright: March 2020
ISBN: 0-316-50985-X
Format: Kindle
Pages: 449
At an unpredictable point after a city has accumulated enough history, character, and sense of itself, it is born as a living creature. It selects an avatar who both embodies the city and helps it be born. But in that moment of birth, a city is at its most vulnerable, and that is when the Enemy attacks. The birth of cities and the Enemy attacks have happened for millennia, so the cities that have survived the process have developed a system. The most recently born goes to assist the avatar with the birthing process and help fight off Enemy attacks. But the process has become riskier and the last two cities have failed to be born, snuffed out in natural disasters despite that support. Now, it's New York City's turn. It selects its avatar and survives the initial assault with the help of S o Paulo. But something still goes wrong, and it is severely injured in the process. Complicating matters, now there are five more avatars, one for each borough, who will have to unite to fight off the Enemy. And, for the first time, the Enemy has taken human form and is attacking with reason and manipulation and intelligence, not just force. The City We Became has a great premise: take the unique sense of place that defines a city and turn it into a literalized character in a fantasy novel. The avatars are people who retain their own lives and understanding (with one exception that I'll get to in a moment), but gain an awareness of the city they represent. They can fight and repair their city through sympathetic magic and metaphor made real. The prelude that introduces this concept (adapted from Jemisin's earlier short story "The City Born Great") got too gonzo for me, but once Jemisin settles into the main story and introduces avatars with a bit more distance from the city they represent, the premise clicked. The execution, on the other hand, I thought was strained. The biggest problem is that the premise requires an ensemble cast of five borough avatars, the primary avatar, S o Paulo, and the Enemy. That's already a lot, but for the story to work each avatar has to be firmly grounded in their own unique experience of New York, which adds family members, colleagues, and roommates. That's too much to stuff into one novel, which means characters get short shrift. For example, Padmini, the avatar of Queens, gets a great introductory scene and a beautiful bit of characterization that made her one of my favorite characters, but then all but disappears for the remainder of the book. She's in the scenes, but not in a way that matters. Brooklyn and Aislyn get moments of deep characterization, but there's so much else going on that they felt rushed. And what ever happened to Manny's roommate? The bulk of the characterization in this book goes to Broncha, the Bronx avatar, a Lenape woman and a tough-as-nails administrator of a community art museum and maker space. The dynamics between her and her co-workers, her mentorship of Veneza, and her early encounters with the Woman in White are my favorite parts of the book. I thought she and Brooklyn were a useful contrast: two very different ways of finding a base of power in the city without letting go of one's ideals. But before we get to Broncha, we first meet Manny, the Manhattan avatar. Thematically, I thought what Jemisin did here was extremely clever. Manny's past is essentially erased at the start of the book, so he's making sense of this world in the same way the reader is. It makes him a useful reader insert character and parallels the typical tourist experience of arriving in Manhattan and trying to use it to make sense of New York. He's disconnected from the rest of the city because he's the dangerous newcomer with power but not a lot of understanding, which works with my model of the political dynamics of Manhattan. Unfortunately, he's not an interesting person. I appreciated what was happening at the metaphorical layer, but Manny veers between action hero and exposition prompt, and his amnesia meant I never got enough sense of him as a character to care that much about what happened to him. I thought his confrontation with the Woman in White near the start of the book, which establishes the major villain of the book, felt clunky and forced compared to her later encounters with the other characters. The Woman in White, though, is a great villain. It's clear from earlier on that the Enemy is Lovecraftian, but the Woman in White mixes mad scientist glee, manic enthusiasm, and a child-like amusement at the weirdness of humanity into the more typical tropes of tentacles, corruption, and horrific creatures. One of my qualms about reading this book is that I'm not a horror fan and don't enjoy the mental images of unspeakable monsters, but the Woman in White puts such a fascinating spin on them that I enjoyed the scenes in which she appeared. I think the book was at its best when she was trying to psychologically manipulate the characters or attack them with corrupted but pre-existing power structures. I was less interested when it turned into an action-movie fight against animated monsters. The other place Jemisin caught me by surprise is too much of a spoiler to describe in detail (and skip the next paragraph in its entirety if you want to avoid all spoilers): Jemisin didn't take the moral conflict of the book in the direction I was expecting. This book is more interested in supporting the people who are already acting ethically than in redeeming people who make bad choices. That produces a deus ex machina ending that's a bit abrupt, but I appreciated the ethical stance. Overall, I thought the premise was great but the execution was unsteady and a bit overstuffed. There are some great characters and some great scenes, but to me they felt disjointed and occasionally rushed. You also need to enjoy characters taking deep pride in the feel of a specific place and advocating for it with the vigor of a sports rivalry, along with loving descriptions of metaphors turned into magical waves of force. But, if you can roll with that, there are moments of real awe. Jemisin captured for me the joy that comes from a deeply grounded sense of connection to a place. Recommended, albeit with caveats, if you're in the mood for reading about people who love the city they live in. This is the first book of a planned trilogy and doesn't resolve the main conflict, but it reaches a satisfying conclusion. The title of the next book has not yet been announced at the time of this review. Rating: 7 out of 10

22 January 2021

Robert McQueen: Launching Endless OS Foundation

Passion Led Us Here

How our for-profit company became a nonprofit, to better tackle the digital divide. Originally posted on the Endless OS Foundation blog.

An 8-year journey to a nonprofit On the 1st of April 2020, our for-profit Endless Mobile officially became a nonprofit as the Endless OS Foundation. Our launch as a nonprofit just as the global pandemic took hold was, predictably, hardly noticed, but for us the timing was incredible: as the world collectively asked What can we do to help others in need? , we framed our mission statement and launched our .org with the same very important question in mind. Endless always had a social impact mission at its heart, and the challenges related to students, families, and communities falling further into the digital divide during COVID-19 brought new urgency and purpose to our team s decision to officially step in the social welfare space.
On April 1st 2020, our for-profit Endless Mobile officially became a nonprofit as the Endless OS Foundation, focused on the #DigitalDivide.
Our updated status was a long time coming: we began our transformation to a nonprofit organization in late 2019 with the realization that the true charter and passions of our team would be greatly accelerated without the constraints of for-profit goals, investors and sales strategies standing in the way of our mission of digital access and equity for all. But for 8 years we made a go of it commercially, headquartered in Silicon Valley and framing ourselves as a tech startup with access to the venture capital and partnerships on our doorstep. We believed that a successful commercial channel would be the most efficient way to scale the impact of bringing computer devices and access to communities in need. We still believe this we ve just learned through our experience that we don t have the funding to enter the computer and OS marketplace head-on. With the social impact goal first, and the hope of any revenue a secondary goal, we have had many successes in those 8 years bridging the digital divide throughout the world, from Brazil, to Kenya, and the USA. We ve learned a huge amount which will go on to inform our strategy as a nonprofit.
Endless always had a social impact mission at its heart. COVID-19 brought new urgency and purpose to our team s decision to officially step in the social welfare space.
Our unique perspective One thing we learned as a for-profit is that the OS and technology we ve built has some unique properties which are hugely impactful as a working solution to digital equity barriers. And our experience deploying in the field around the world for 8 years has left us uniquely informed via many iterations and incremental improvements.
Endless OS designer in discussion with prospective user
With this knowledge in-hand, we ve been refining our strategy throughout 2020 and now starting to focus on what it really means to become an effective nonprofit and make that impact. In many ways it is liberating to abandon the goals and constraints of being a for-profit entity, and in other ways it s been a challenging journey for me and the team to adjust our way of thinking and let these for-profit notions and models go. Previously we exclusively built and sold a product that defined our success; and any impact we achieved was a secondary consequence of that success and seen through that lens. Now our success is defined purely in terms of social impact, and through our actions, those positive impacts can be made with or without our product . That means that we may develop and introduce technology to solve a problem, but it is equally as valid to find another organization s existing offering and design a way to increase that positive impact and scale.
We develop technology to solve access equity issues, but it s equally as valid to find another organization s offering and partner in a way that increases their positive impact.
The analogy to Free and Open Source Software is very strong while Endless has always used and contributed to a wide variety of FOSS projects, we ve also had a tension where we ve been trying to hold some pieces back and capture value such as our own application or content ecosystem, our own hardware platform necessarily making us competitors to other organisations even though they were hoping to achieve the same things as us. As a nonprofit we can let these ideas go and just pick the best partners and technologies to help the people we re trying to reach.
School kids writing on paper
Digital equity 4 barriers we need to overcome In future, our decisions around which projects to build or engage with will revolve around 4 barriers to digital equity, and how our Endless OS, Endless projects, or our partners offerings can help to solve them. We define these 4 equity barriers as: barriers to devices, barriers to connectivity, barriers to literacy in terms of your ability to use the technology, and barriers to engagement in terms of whether using the system is rewarding and worthwhile.
We define the 4 digital equity barriers we exist to impact as:
1. barriers to devices
2. barriers to connectivity
3. barriers to literacy
4. barriers to engagement
It doesn t matter who makes the solutions that break these barriers; what matters is how we assist in enabling people to use technology to gain access to the education and opportunities these barriers block. Our goal therefore is to simply ensure that solutions exist building them ourselves and with partners such as the FOSS community and other nonprofits proving them with real-world deployments, and sharing our results as widely as possible to allow for better adoption globally. If we define our goal purely in terms of whether people are using Endless OS, we are effectively restricting the reach and scale of our solutions to the audience we can reach directly with Endless OS downloads, installs and propagation. Conversely, partnerships that scale impact are a win-win-win for us, our partners, and the communities we all serve. Engineering impact Our Endless engineering roots and capabilities feed our unique ability to build and deploy all of our solutions, and the practical experience of deploying them gives us evidence and credibility as we advocate for their use. Either activity would be weaker without the other.
Our engineering roots and capabilities feed our unique ability to build and deploy digital divide solutions.
Our partners in various engineering communities will have already seen our change in approach. Particularly, with GNOME we are working hard to invest in upstream and reconcile the long-standing differences between our experience and GNOME. If successful, many more people can benefit from our work than just users of Endless OS. We re working with Learning Equality on Kolibri to build a better app experience for Linux desktop users and bring content publishers into our ecosystem for the first time, and we ve also taken our very own Hack, the immersive and fun destination for kids learning to code, released it for non-Endless systems on Flathub, and made it fully open-source.
Planning tasks with sticky notes on a whiteboard
What s next for our OS? What then is in store for the future of Endless OS, the place where we have invested so much time and planning through years of iterations? For the immediate future, we need the capacity to deploy everything we ve built all at once, to our partners. We built an OS that we feel is very unique and valuable, containing a number of world-firsts: first production OS shipped with OSTree, first Flatpak-only desktop, built-in support for updating OS and apps from USBs, while still providing a great deal of reliability and convenience for deployments in offline and educational-safe environments with great apps and content loaded on every system. However, we need to find a way to deliver this Linux-based experience in a more efficient way, and we d love to talk if you have ideas about how we can do this, perhaps as partners. Can the idea of Endless OS evolve to become a spec that is provided by different platforms in the future, maybe remixes of Debian, Fedora, openSUSE or Ubuntu? Build, Validate, Advocate Beyond the OS, the Endless OS Foundation has identified multiple programs to help underserved communities, and in each case we are adopting our build, validate, advocate strategy. This approach underpins all of our projects: can we build the technology (or assist in the making), will a community in-need validate it by adoption, and can we inspire others by telling the story and advocating for its wider use?
We are adopting a build, validate, advocate strategy.
1. build the technology (or assist in the making)
2. validate by community adoption
3. advocate for its wider use
As examples, we have just launched the Endless Key (link) as an offline solution for students during the COVID-19 at-home distance learning challenges. This project is also establishing a first-ever partnership of well-known online educational brands to reach an underserved offline audience with valuable learning resources. We are developing a pay-as-you-go platform and new partnerships that will allow families to own laptops via micro-payments that are built directly into the operating system, even if they cannot qualify for standard retail financing. And during the pandemic, we ve partnered with Teach For America to focus on very practical digital equity needs in the USA s urban and rural communities. One part of the world-wide digital divide solution We are one solution provider for the complex matrix of issues known collectively as the #DigitalDivide, and these issues will not disappear after the pandemic. Digital equity was an issue long before COVID-19, and we are not so naive to think it can be solved by any single institution, or by the time the pandemic recedes. It will take time and a coalition of partnerships to win. We are in for the long-haul and we are always looking for partners, especially now as we are finding our feet in the nonprofit world. We d love to hear from you, so please feel free to reach out to me I m ramcq on IRC, RocketChat, Twitter, LinkedIn or rob@endlessos.org.

Enrico Zini: Polishing nspawn-runner

This post is part of a series about trying to setup a gitlab runner based on systemd-nspawn. I published the polished result as nspawn-runner on GitHub. gitlab-runner supports adding extra arguments to the custom scripts, and I can take advantage of that to pack all the various scripts that I prototyped so far into an all-in-one nspawn-runner command:
usage: nspawn-runner [-h] [-v] [--debug]
                      chroot-create,chroot-login,prepare,run,cleanup,gitlab-config,toml 
                     ...
Manage systemd-nspawn machines for CI runs.
positional arguments:
   chroot-create,chroot-login,prepare,run,cleanup,gitlab-config,toml 
                        sub-command help
    chroot-create       create a chroot that serves as a base for ephemeral
                        machines
    chroot-login        enter the chroot to perform maintenance
    prepare             start an ephemeral system for a CI run
    run                 run a command inside a CI machine
    cleanup             cleanup a CI machine after it's run
    gitlab-config       configuration step for gitlab-runner
    toml                output the toml configuration for the custom runner
optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         verbose output
  --debug               verbose output
chroot maintenance chroot-create and chroot-login are similar to what pbuilder, cowbuilder, schroot, debspawn and similar tools do. They only take a chroot name, and default the rest of paths to where nspawn-runner expects things to be under /var/lib/nspawn-runner. gitlab-runner setup nspawn-runner toml <chroot-name> outputs a snippet to add to /etc/gitlab-runner/config.toml to configure the CI. For example:
$ ./nspawn-runner toml buster
[[runners]]
  name="buster"
  url="TODO"
  token="TODO"
  executor = "custom"
  builds_dir = "/var/lib/nspawn-runner/.build"
  cache_dir = "/var/lib/nspawn-runner/.cache"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.custom]
    config_exec = "/home/enrico/ /nspawn-runner/nspawn-runner"
    config_args = ["gitlab-config"]
    config_exec_timeout = 200
    prepare_exec = "/home/enrico/ /nspawn-runner/nspawn-runner"
    prepare_args = ["prepare", "buster"]
    prepare_exec_timeout = 200
    run_exec = "/home/enrico/dev/nspawn-runner/nspawn-runner"
    run_args = ["run"]
    cleanup_exec = "/home/enrico/ /nspawn-runner/nspawn-runner"
    cleanup_args = ["cleanup"]
    cleanup_exec_timeout = 200
    graceful_kill_timeout = 200
    force_kill_timeout = 200
One needs to remember to set url and token, and the runner is configured. The end, for now This is it, it works! Time will tell what issues or ideas will come up: for now, it's a pretty decent first version. The various prepare, run, cleanup steps are generic enough that they can be used outside of gitlab-runner: feel free to build on them, and drop me a note if you find this useful! Updated: Issues noticed so far, that could go into a new version:

Enrico Zini: Exploring nspawn for CIs

This post is part of a series about trying to setup a gitlab runner based on systemd-nspawn. I published the polished result as nspawn-runner on GitHub. Here I try to figure out possible ways of invoking nspawn for the prepare, run, and cleanup steps of gitlab custom runners. The results might be useful invocations beyond Gitlab's scope of application. I begin with a chroot which will be the base for our build environments:
debootstrap --variant=minbase --include=git,build-essential buster workdir
Fully ephemeral nspawn This would be fantastic: set up a reusable chroot, mount readonly, run the CI in a working directory mounted on tmpfs. It sets up quickly, it cleans up after itself, and it would make prepare and cleanup noops:
mkdir workdir/var/lib/gitlab-runner
systemd-nspawn --read-only --directory workdir --tmpfs /var/lib/gitlab-runner "$@"
However, run gets run multiple times, so I need the side effects of run to persist inside the chroot between runs. Also, if the CI uses a large amount of disk space, tmpfs may get into trouble. nspawn with overlay Federico used --overlay to keep the base chroot readonly while allowing persistent writes on a temporary directory on the filesystem. Note that using --overlay requires systemd and systemd-container from buster-backports because of systemd bug #3847. Example:
mkdir -p tmp-overlay
systemd-nspawn --quiet -D workdir \
  --overlay=" pwd /workdir: pwd /tmp-overlay:/"
I can run this twice, and changes in the file system will persist between systemd-nspawn executions. Great! However, any process will be killed at the end of each execution. machinectl I can give a name to systemd-nspawn invocations using --machine, and it allows me to run multiple commands during the machine lifespan using machinectl and systemd-run. In theory machinectl can also fully manage chroots and disk images in /var/lib/machines, but I haven't found a way with machinectl to start multiple machines sharing the same underlying chroot. It's ok, though: I managed to do that with systemd-nspawn invocations. I can use the --machine=name argument to systemd-nspawn to make it visible to machinectl. I can use the --boot argument to systemd-nspawn to start enough infrastructure inside the container to allow machinectl to interact with it. This gives me any number of persistent and named running systems, that share the same underlying chroot, and can cleanup after themselves. I can run commands in any of those systems as I like, and their side effects persist until a system is stopped. The chroot needs systemd and dbus for machinectl to be able to interact with it:
debootstrap --variant=minbase --include=git,systemd,systemd,build-essential buster workdir
Let's boot the machine:
mkdir -p overlay
systemd-nspawn --quiet -D workdir \
    --overlay=" pwd /workdir: pwd /overlay:/"
    --machine=test --boot
Let's try machinectl:
# machinectl list
MACHINE CLASS     SERVICE        OS     VERSION ADDRESSES
test    container systemd-nspawn debian 10      -
1 machines listed.
# machinectl shell --quiet test /bin/ls -la /
total 60
[ ]
To run commands, rather than machinectl shell, I need to use systemd-run --wait --pipe --machine=name, otherwise machined won't forward the exit code. The result however is pretty good, with working stdin/stdout/stderr redirection and forwarded exit code. Good, I'm getting somewhere. The terminal where I ran systemd-nspawn is currently showing a nice getty for the booted system, which is cute, and not what I want for the setup process of a CI. Spawning machines without needing a terminal machinectl uses /lib/systemd/system/systemd-nspawn@.service to start machines. I suppose there's limited magic in there: start systemd-nspawn as a service, use --machine to give it a name, and machinectl manages it as if it started it itself. What if, instead of installing a unit file for each CI run, I try to do the same thing with systemd-run?
systemd-run \
  -p 'KillMode=mixed' \
  -p 'Type=notify' \
  -p 'RestartForceExitStatus=133' \
  -p 'SuccessExitStatus=133' \
  -p 'Slice=machine.slice' \
  -p 'Delegate=yes' \
  -p 'TasksMax=16384' \
  -p 'WatchdogSec=3min' \
  systemd-nspawn --quiet -D  pwd /workdir \
    --overlay=" pwd /workdir: pwd /overlay:/"
    --machine=test --boot
It works! I can interact with it using machinectl, and fine tune DevicePolicy as needed to lock CI machines down. This setup has a race condition where if I try to run a command inside the machine in the short time window before the machine has finished booting, it fails:
# systemd-run [ ] systemd-nspawn [ ] ; machinectl --quiet shell test /bin/ls -la /
Failed to get shell PTY: Protocol error
# machinectl shell test /bin/ls -la /
Connected to machine test. Press ^] three times within 1s to exit session.
total 60
[ ]
systemd-nspawn has the option --notify-ready=yes that solves exactly this problem:
# systemd-run [ ] systemd-nspawn [ ] --notify-ready=yes ; machinectl --quiet shell test /bin/ls -la /
Running as unit: run-r5a405754f3b740158b3d9dd5e14ff611.service
total 60
[ ]
On nspawn's side, I should now have all I need. Next steps My next step will be wrapping it all together in a gitlab runner.

20 January 2021

Rapha&#235;l Hertzog: Freexian s report about Debian Long Term Support, December 2020

A Debian LTS logo Like each month, have a look at the work funded by Freexian s Debian LTS offering. Debian project funding In December, we put aside 2100 EUR to fund Debian projects. The first project proposal (a tracker.debian.org improvement for the security team) was received and quickly approved by the paid contributors, then we opened a request for bids and the bid winner was announced today (it was easy, we had only one candidate). Hopefully this first project will be completed until our next report. We re looking forward to receive more projects from various Debian teams! Learn more about the rationale behind this initiative in this article. Debian LTS contributors In December, 12 contributors have been paid to work on Debian LTS, their reports are available: Evolution of the situation December was a quiet month as we didn t have a team meeting nor any other unusual activity and we released 43 DLAs. The security tracker currently lists 30 packages with a known CVE and the dla-needed.txt file has 25 packages needing an update. This month we are pleased to welcome Deveryware as new sponsor! Thanks to our sponsors Sponsors that joined recently are in bold.

19 January 2021

John Goerzen: Roundup of Secure Messengers with Off-The-Grid Capabilities (Distributed/Mesh Messengers)

Amid all the conversation about Signal, and the debate over decentralization, one thing has often not been raised: all of these things require an Internet connection. Of course, you might say. Internet is everywhere these days. Well, not so much, and it turns out there are some very good reasons that people might want messengers that work offline. Here are some examples: How do they work? These all use some form of local radio signal. Some, such as Briar, may use short-range Bluetooth and Wifi, while others use radios such as LoRa that can reach several miles with low power. I ve written quite a bit about LoRa before, and its unique low-speed but extreme-distance radio capabilities even on low power. One common thread through these is that most of them are Android-only, though many are compatible with F-Droid and privacy-enhanced Android distributions. Every item on this list uses full end-to-end encryption (E2EE). Let s dive on in. Briar Of all the options mentioned here, Briar is the one that bridges the traditional Internet-based approach with alternative options the best. It offers three ways for distributing data: As far as I can tell, there is no centralized server in Briar at all. Your account , such as it is, lives entirely within your device; if you wipe your device, you will have to make a new account and re-establish contacts. The use of Tor is also neat to see; it ensures that an adversary can t tell, just from that, that you re using Briar at all, though of course timing analysis may still be possible (and Bluetooth and Wifi uses may reval some of who is communicating). Briar features several types of messages (detailed in the manual), which really are just different spins on communication, which they liken to metaphors people are familiar with: By default, Briar raises an audible notification for incoming messages of all types. This is configurable for each type. Blogs have a way to reblog (even a built-in RSS reader to facilitate that), but framed a different way, they are broadcast messages. They could, for instance, be useful for a send help message to everyone (assuming that people haven t all shut off notifications of blogs due to others using them different ways). Briar s how it works page has an illustration specifically of how blogs are distributed. I m unclear on some of the details, and to what extent this applies to other kinds of messages, but one thing that you can notice from this is that a person A could write a broadcast message without Internet access, person B could receive it via Bluetooth or whatever, and then when person B gets Internet access again, the post could be distributed more widely. However, it doesn t appear that Briar is really a full mesh, since only known contacts in the distribution path for the message would repeat it. There are some downsides to Briar. One is that, since an account is fully localized to a device, one must have a separate account for each device. That can lead to contacts having to pick a specific device to send a message to. There is an online indicator, which may help, but it s definitely not the kind of seamless experience you get from Internet-only messengers. Also, it doesn t support migrating to a new phone, live voice/video calls, or attachments, but attachments are in the works. All in all, a solid communicator, and is the only one on this list that works 100% with the hardware everyone already has. While Bluetooth and Wifi have far more limited range than the other entries, there is undeniably convenience in not needing any additional hardware, and it may be particularly helpful when extra bags/pockets aren t available. Also, Briar is fully Open Source. Meshtastic Meshtastic is a radio-first LoRa mesh project. What do I mean by radio-first? Well, basically cell phones are how you interact with Meshtastic, but they are optional. The hardware costs about $30 and the batteries last about 8 days. Range between nodes is a few miles in typical conditions (up to 11km / 7mi in ideal conditions), but nodes act as repeaters, so it is quite conceivable to just drop a node in the middle if you and contacts will be far apart. The project estimates that around 2000 nodes are in operation, and the network is stronger the more nodes are around. The getting started site describes how to build one. Most Meshtastic device builds have a screen and some buttons. They can be used independently from the Android app to display received messages, distance and bearing to other devices (assuming both have a GPS enabled), etc. This video is an introduction showing it off, this one goes over the hardware buttons. So even if your phone is dead, you can at least know where your friends are. Incidentally, the phone links up to the radio board using Bluetooth, and can provide a location source if you didn t include one in your build. There are ideas about solar power for Meshtastic devices, too. Meshtastic doesn t, as far as I know, have an option for routing communication over the Internet, but the devices appear to be very thoughtfully-engineered and easy enough to put together. This one is definitely on my list to try. Ripple-based devices This is based on the LoRa Mesh Radio Instructables project, and is similar in concept to Meshtastic. It uses similar hardware, a similar app, but also has an option with a QWERTY hardware keyboard available, for those that want completely phone-free operation while still being able to send messages. There are a number of related projects posted at Instructables: a GPS tracker, some sensors, etc. These are variations on the same basic concept. These use the Ripple firmware, which is not open source, so I haven t pursued it further. GoTenna For people that want less of a DIY model, and don t mind proprietary solutions, there are two I ll mention. The first is GoTenna Mesh, which is LoRa-based and sells units for $90 each. However, there are significant community concerns about the longevity of the project, as GoTenna has re-focused on government and corporate work. The Android app hasn t been updated in 6 monnths despite a number of reviews citing issues, and the iOS app is also crusty. Beartooth Even more expensive at $125 each is the Beartooth. Also a proprietary option, I haven t looked into it more, but they are specifically targetting backwoods types of markets. Do not use: Bridgefy Bridgefy was briefly prominent since it was used during the Hong Kong protests. However, numerous vulnerabilities have been demonstrated, and the developers have said they are re-working the app to address them. I wouldn t recommend it for now. Alternatives: GMRS handhelds In the USA, GMRS voice handhelds are widely available. Although a license is required, it is simple (no exam) and cheap ($35) and extends to a whole family. GMRS radios also interoperate with FRS radios, which require no license and share some frequencies, but are limited to lower power (though are often sufficient). Handheld GMRS radios that use up to 5W of power are readily available. A voice signal is a lot harder to carry for a long distance than a very low-bandwidth digital one, so even with much more power you will probably not get the same kind of range you will with something like Meshtastic, and they don t come with any kind of security or encryption at all. However, for basic communication, they are often a useful tool.

Russ Allbery: Review: The Secret Barrister

Review: The Secret Barrister, by The Secret Barrister
Publisher: Picador
Copyright: 2018
Printing: 2019
ISBN: 1-5098-4115-6
Format: Kindle
Pages: 344
The Secret Barrister is a survey and critique of the criminal legal system of England and Wales. The author is an anonymous barrister who writes a legal blog of the same name (which I have not read). A brief and simplified primer for those who, like me, are familiar with the US legal system but not the English one: A barrister is a lawyer who argues cases in court, as distinct from a solicitor who does all the other legal work (and may make limited court appearances). If you need criminal legal help in England and Wales, you hire a solicitor, and they are your primary source of legal advise. If your case goes to court, your solicitor will generally (not always) refer the work of arguing your case before a judge and jury to a barrister and "instruct" them in the details of your argument. The job of the barrister is then to handle the courtroom trial, offer trial-specific legal advice, and translate your defense (or the crown's prosecution) into persuasive courtroom arguments. Unlike the United States, with its extremely sharp distinction between prosecutors and criminal defense attorneys, criminal barristers in England and Wales argue both prosecutions and defenses depending on who hires them. (That said, the impression I got from this book is that the creation of the Crown Prosecution Service is moving England closer to the US model and more prosecutions are now handled by barristers employed directly by the CPS, whom I assume do not take defense cases.) Barristers follow the cab-rank rule, which means that, like a taxicab, they are professionally obligated to represent people on a first-come, first-serve basis and are not allowed to pick and choose clients. (Throughout, I'm referencing the legal system of England and Wales because the author restricts his comments to it. Presumably this is because the Scottish and Northern Irish? legal systems are different yet again in ways I do not know.) If details like this sound surprising, you can see the appeal of this book to me. It's easy, in the US, to have a vast ignorance about the legal systems of other countries or even the possibility of different systems, which makes it hard to see how our system could be improved. I had a superficial assumption that since US law started as English common law, the US and English legal systems would be substantially similar. And they are to an extent; they're both adversarial rather than inquisitorial, for example (more on that in a moment). But the current system of criminal prosecution evolved long after US independence and thus evolved differently despite similar legal foundations. Those differences are helpful for this American to ponder the road not taken and the impact of our respective choices. That said, explaining the criminal legal system to Americans isn't the author's purpose. The first fifty pages are that beginner's overview, since apparently even folks who live in England are confused by the ubiquity of US legal dramas (not that those are very accurate representations of the US legal system either). The rest of the book, and its primary purpose, is an examination of the system's failings, starting with the magistrates' courts (which often use lay judges and try what in the US would be called misdemeanors, although as discussed in this book their scope is expanding). Other topics include problems with bail, how prosecution is structured, how victims and witnesses are handled, legal aid, sentencing, and the absurd inadequacy of compensation for erroneous convictions. The most useful part of this book for me, apart from the legal system introduction, was the two chapters the author spends arguing first for and then against replacing an adversarial system with an inquisitorial system (the French criminal justice system, for example). When one is as depressed about the state of one's justice system as both I and the author are, something radically different sounds appealing. The author first makes a solid case for the inquisitorial system and then tries to demolish it, still favoring the adversarial system, and I liked that argument construction. The argument in favor of an adversarial system is solid and convincing, but it's also depressing. It's the argument of someone who has seen the corruption, sloppiness, and political motivations in an adversarial system and fears what would happen if they were able to run rampant under a fig leaf of disinterested objectivity. I can't disagree, particularly when starting from an adversarial system, but this argument feels profoundly cynical. It reminds me of the libertarian argument for capitalism: humans are irredeemably awful, greed and self-interest are the only reliable or universal human motives, and therefore the only economic system that can work is one based on and built to harness greed, because expecting any positive characteristics from humans collectively is hopelessly naive. The author of this book is not quite that negative in their argument for an adversarial system, but it's essentially the same reasoning: the only way a system can be vaguely honest is if it's constantly questioned and attacked. It can never be trusted to be objective on its own terms. I wish the author had spent more time on the obvious counter-argument: when the system is designed for adversarial combat, it normalizes and even valorizes every dirty tactic that might result in a victory. The system reinforces our worst impulses, not to mention grinding up and destroying people who cannot afford their own dirty tricks. The author proposes several explanations for the problems they see in the criminal legal system, including "tough on crime" nonsense from politicians that sounds familiar to this American reader. Most problems, though, they trace back to lack of funding: of the police, of the courts, of the prosecutors, and of legal aid. I don't know enough about English politics to have an independent opinion on this argument, but the stories of outsourcing to the lowest bidder, overworked civil servants, ridiculously low compensation rates, flawed metrics like conviction rates, and headline-driven political posturing that doesn't extend to investing in necessary infrastructure like better case-tracking systems sounds depressingly familiar. This is one of those books where I appreciated the content but not the writing. It's not horrible, but the sentences are ponderous and strained and the author is a bit too fond of two-dollar words. They also have a dramatic and self-deprecating way of describing their own work that I suspect they thought was funny but that I found grating. By the end of this book, I was irritated enough that I can't recommend it. But the content was interesting, even the critique of a political system that isn't mine, and it prompted some new thoughts on the difficulties of creating a fair justice system. If you can deal with the author's writing style, you may also enjoy it. Rating: 6 out of 10

18 January 2021

Evgeni Golov: building a simple KVM switch for 30

Prompted by tweets from Lesley and Dave, I thought about KVM switches again and came up with a rather cheap solution to my individual situation (YMMY, as usual). As I've written last year, my desk has one monitor, keyboard and mouse and two computers. Since writing that post I got a new (bigger) monitor, but also an USB switch again (a DIGITUS USB 3.0 Sharing Switch) - this time one that doesn't freak out my dock \o/ However, having to switch the used computer in two places (USB and monitor) is rather inconvenient, but also getting an KVM switch that can do 4K@60Hz was out of question. Luckily, hackers gonna hack, everything, and not only receipt printers ( ). There is a tool called ddcutil that can talk to your monitor and change various settings. And udev can execute commands when (USB) devices connect You see where this is going? After installing the package (available both in Debian and Fedora), we can inspect our system with ddcutil detect. You might have to load the i2c_dev module (thanks Philip!) before this works -- it seems to be loaded automatically on my Fedora, but you never know .
$ sudo ddcutil detect
Invalid display
   I2C bus:             /dev/i2c-4
   EDID synopsis:
      Mfg id:           BOE
      Model:
      Serial number:
      Manufacture year: 2017
      EDID version:     1.4
   DDC communication failed
   This is an eDP laptop display. Laptop displays do not support DDC/CI.
Invalid display
   I2C bus:             /dev/i2c-5
   EDID synopsis:
      Mfg id:           AOC
      Model:            U2790B
      Serial number:
      Manufacture year: 2020
      EDID version:     1.4
   DDC communication failed
Display 1
   I2C bus:             /dev/i2c-7
   EDID synopsis:
      Mfg id:           AOC
      Model:            U2790B
      Serial number:
      Manufacture year: 2020
      EDID version:     1.4
   VCP version:         2.2
The first detected display is the built-in one in my laptop, and those don't support DDC anyways. The second one is a ghost (see ddcutil#160) which we can ignore. But the third one is the one we can (and will control). As this is the only valid display ddcutil found, we don't need to specify which display to talk to in the following commands. Otherwise we'd have to add something like --display 1 to them. A ddcutil capabilities will show us what the monitor is capable of (or what it thinks, I've heard some give rather buggy output here) -- we're mostly interested in the "Input Source" feature (Virtual Control Panel (VCP) code 0x60):
$ sudo ddcutil capabilities
 
   Feature: 60 (Input Source)
      Values:
         0f: DisplayPort-1
         11: HDMI-1
         12: HDMI-2
 
Seems mine supports it, and I should be able to switch the inputs by jumping between 0x0f, 0x11 and 0x12. You can see other values defined by the spec in ddcutil vcpinfo 60 --verbose, some monitors are using wrong values for their inputs . Let's see if ddcutil getvcp agrees that I'm using DisplayPort now:
$ sudo ddcutil getvcp 0x60
VCP code 0x60 (Input Source                  ): DisplayPort-1 (sl=0x0f)
And try switching to HDMI-1 using ddcutil setvcp:
$ sudo ddcutil setvcp 0x60 0x11
Cool, cool. So now we just need a way to trigger input source switching based on some event There are three devices connected to my USB switch: my keyboard, my mouse and my Yubikey. I do use the mouse and the Yubikey while the laptop is not docked too, so these are not good indicators that the switch has been turned to the laptop. But the keyboard is! Let's see what vendor and product IDs it has, so we can write an udev rule for it:
$ lsusb
 
Bus 005 Device 006: ID 17ef:6047 Lenovo ThinkPad Compact Keyboard with TrackPoint
 
Okay, so let's call ddcutil setvcp 0x60 0x0f when the USB device 0x17ef:0x6047 is added to the system:
ACTION=="add", SUBSYSTEM=="usb", ATTR idVendor =="17ef", ATTR idProduct =="6047", RUN+="/usr/bin/ddcutil setvcp 0x60 0x0f"
$ sudo vim /etc/udev/rules.d/99-ddcutil.rules
$ sudo udevadm control --reload
And done! Whenever I connect my keyboard now, it will force my screen to use DisplayPort-1. On my workstation, I deployed the same rule, but with ddcutil setvcp 0x60 0x11 to switch to HDMI-1 and my cheap not-really-KVM-but-in-the-end-KVM-USB-switch is done, for the price of one USB switch (~30 ). Note: if you want to use ddcutil with a Lenovo Thunderbolt 3 Dock (or any other dock using Displayport Multi-Stream Transport (MST)), you'll need kernel 5.10 or newer, which fixes a bug that prevents ddcutil from talking to the monitor using I C.

17 January 2021

Wouter Verhelst: On Statements, Facts, Hypotheses, Science, Religion, and Opinions

The other day, we went to a designer's fashion shop whose owner was rather adamant that he was never ever going to wear a face mask, and that he didn't believe the COVID-19 thing was real. When I argued for the opposing position, he pretty much dismissed what I said out of hand, claiming that "the hospitals are empty dude" and "it's all a lie". When I told him that this really isn't true, he went like "well, that's just your opinion". Well, no -- certain things are facts, not opinions. Even if you don't believe that this disease kills people, the idea that this is a matter of opinion is missing the ball by so much that I was pretty much stunned by the level of ignorance. His whole demeanor pissed me off rather quickly. While I disagree with the position that it should be your decision whether or not to wear a mask, it's certainly possible to have that opinion. However, whether or not people need to go to hospitals is not an opinion -- it's something else entirely. After calming down, the encounter got me thinking, and made me focus on something I'd been thinking about before but hadn't fully forumlated: the fact that some people in this world seem to misunderstand the nature of what it is to do science, and end up, under the claim of being "sceptical", with various nonsense things -- see scientology, flat earth societies, conspiracy theories, and whathaveyou. So, here's something that might (but probably won't) help some people figuring out stuff. Even if it doesn't, it's been bothering me and I want to write it down so it won't bother me again. If you know all this stuff, it might be boring and you might want to skip this post. Otherwise, take a deep breath and read on... Statements are things people say. They can be true or false; "the sun is blue" is an example of a statement that is trivially false. "The sun produces light" is another one that is trivially true. "The sun produces light through a process that includes hydrogen fusion" is another statement, one that is a bit more difficult to prove true or false. Another example is "Wouter Verhelst does not have a favourite color". That happens to be a true statement, but it's fairly difficult for anyone that isn't me (or any one of the other Wouters Verhelst out there) to validate as true. While statements can be true or false, combining statements without more context is not always possible. As an example, the statement "Wouter Verhelst is a Debian Developer" is a true statement, as is the statement "Wouter Verhelst is a professional Volleybal player"; but the statement "Wouter Verhelst is a professional Volleybal player and a Debian Developer" is not, because while I am a Debian Developer, I am not a professional Volleybal player -- I just happen to share a name with someone who is. A statement is never a fact, but it can describe a fact. When a statement is a true statement, either because we trivially know what it states to be true or because we have performed an experiment that proved beyond any possible doubt that the statement is true, then what the statement describes is a fact. For example, "Red is a color" is a statement that describes a fact (because, yes, red is definitely a color, that is a fact). Such statements are called statements of fact. There are other possible statements. "Grass is purple" is a statement, but it is not a statement of fact; because as everyone knows, grass is (usually) green. A statement can also describe an opinion. "The Porsche 911 is a nice car" is a statement of opinion. It is one I happen to agree with, but it is certainly valid for someone else to make a statement that conflicts with this position, and there is nothing wrong with that. As the saying goes, "opinions are like assholes: everyone has one". Statements describing opinions are known as statements of opinion. The differentiating factor between facts and opinions is that facts are universally true, whereas opinions only hold for the people who state the opinion and anyone who agrees with them. Sometimes it's difficult or even impossible to determine whether a statement is true or not. The statement "The numbers that win the South African Powerball lottery on the 31st of July 2020 are 2, 3, 5, 19, 35, and powerball 14" is not a statement of fact, because at the time of writing, the 31st of July 2020 is in the future, which at this point gives it a 1 in 24,435,180 chance to be true). However, that does not make it a statement of opinion; it is not my opinion that the above numbers will win the South African powerball; instead, it is my guess that those numbers will be correct. Another word for "guess" is hypothesis: a hypothesis is a statement that may be universally true or universally false, but for which the truth -- or its lack thereof -- cannot currently be proven beyond doubt. On Saturday, August 1st, 2020 the above statement about the South African Powerball may become a statement of fact; most likely however, it will instead become a false statement. An unproven hypothesis may be expressed as a matter of belief. The statement "There is a God who rules the heavens and the Earth" cannot currently (or ever) be proven beyond doubt to be either true or false, which by definition makes it a hypothesis; however, for matters of religion this is entirely unimportant, as for believers the belief that the statement is correct is all that matters, whereas for nonbelievers the truth of that statement is not at all relevant. A belief is not an opinion; an opinion is not a belief. Scientists do not deal with unproven hypotheses, except insofar that they attempt to prove, through direct observation of nature (either out in the field or in a controlled laboratory setting) that the hypothesis is, in fact, a statement of fact. This makes unprovable hypotheses unscientific -- but that does not mean that they are false, or even that they are uninteresting statements. Unscientific statements are merely statements that science cannot either prove or disprove, and that therefore lie outside of the realm of what science deals with. Given that background, I have always found the so-called "conflict" between science and religion to be a non-sequitur. Religion deals in one type of statements; science deals in another. The do not overlap, since a statement can either be proven or it cannot, and religious statements by their very nature focus on unprovable belief rather than universal truth. Sure, the range of things that science has figured out the facts about has grown over time, which implies that religious statements have sometimes been proven false; but is it heresy to say that "animals exist that can run 120 kph" if that is the truth, even if such animals don't exist in, say, Rome? Something very similar can be said about conspiracy theories. Yes, it is possible to hypothesize that NASA did not send men to the moon, and that all the proof contrary to that statement was somehow fabricated. However, by its very nature such a hypothesis cannot be proven or disproven (because the statement states that all proof was fabricated), which therefore implies that it is an unscientific statement. It is good to be sceptical about what is being said to you. People can have various ideas about how the world works, but only one of those ideas -- one of the possible hypotheses -- can be true. As long as a hypothesis remains unproven, scientists love to be sceptical themselves. In fact, if you can somehow prove beyond doubt that a scientific hypothesis is false, scientists will love you -- it means they now know something more about the world and that they'll have to come up with something else, which is a lot of fun. When a scientific experiment or observation proves that a certain hypothesis is true, then this probably turns the hypothesis into a statement of fact. That is, it is of course possible that there's a flaw in the proof, or that the experiment failed (but that the failure was somehow missed), or that no observance of a particular event happened when a scientist tried to observe something, but that this was only because the scientist missed it. If you can show that any of those possibilities hold for a scientific proof, then you'll have turned a statement of fact back into a hypothesis, or even (depending on the exact nature of the flaw) into a false statement. There's more. It's human nature to want to be rich and famous, sometimes no matter what the cost. As such, there have been scientists who have falsified experimental results, or who have claimed to have observed something when this was not the case. For that reason, a scientific paper that gets written after an experiment turned a hypothesis into fact describes not only the results of the experiment and the observed behavior, but also the methodology: the way in which the experiment was run, with enough details so that anyone can retry the experiment. Sometimes that may mean spending a large amount of money just to be able to run the experiment (most people don't have an LHC in their backyard, say), and in some cases some of the required materials won't be available (the latter is expecially true for, e.g., certain chemical experiments that involve highly explosive things); but the information is always there, and if you spend enough time and money reading through the available papers, you will be able to independently prove the hypothesis yourself. Scientists tend to do just that; when the results of a new experiment are published, they will try to rerun the experiment, partially because they want to see things with their own eyes; but partially also because if they can find fault in the experiment or the observed behavior, they'll have reason to write a paper of their own, which will make them a bit more rich and famous. I guess you could say that there's three types of people who deal with statements: scientists, who deal with provable hypotheses and statements of fact (but who have no use for unprovable hypotheses and statements of opinion); religious people and conspiracy theorists, who deal with unprovable hypotheses (where the religious people deal with these to serve a large cause, while conspiracy theorists only care about the unprovable hypotheses); and politicians, who should care about proven statements of fact and produce statements of opinion, but who usually attempt the reverse of those two these days :-/ Anyway... [[!img Error: Image::Magick is not installed]]

16 January 2021

Christian Kastner: Keeping your Workstation Silent

I require moderately powerful boxes for my work. Moderately powerful boxes generate a non-negligible amount of heat, and therefore require appropriate cooling. Most cooling solutions involve a certain degree of noise, which I dislike. Water-cooling is supposed to be the most effective solution for noise reduction, but it always felt like too much of a hassle to try to me. Considering the various solutions I've tried in the past years, two observations I've made really stick out:
  1. Noctua's reputation for CPU cooling is well-earned.
  2. A "silent" (meaning sound-proofed) case, isn't. At least, not anymore.
I've tried numerous coolers in the past, some of monstrous proportions (always thinking that more mass must be better, and reputable brands are equally good), but I was never really satisfied; hence, I was doubtful that trying yet another cooler would make a difference. I'm glad I tried the Noctua NH-D15 anyway. With some tweaking to the fan profile in the BIOS, it's totally inaudible at normal to medium workloads, and just a very gentle hum at full load subtle enough to disappear in the background. For the past decade, I've also regularly purchased sound-proofed cases, but this habit appears anachronistic now. Years ago, sound-proofed cases helped contain the noise of a few HDDs. However, all of my boxes now contain NVMe drives (which, to me, are the biggest improvement to computing since CPUs going multi-core). On the other hand, some of my boxes now contain powerful GPUs used for GPGPU computing, and with the recent higher-end Nvidia and AMD cards all pulling in over 300W, there is a lot of heat to manage. The best way to quickly dump heat is with good airflow. Sound-proofing works against that. Its insulation restricts airflow, which ultimately causes even more noise, as the GPU's fans need to spin at very high RPMs. This is, of course, totally obvious in hindsight. I removed the front cover from a new GPU box, and the noise went down from almost-hair-dryer to acceptable-whoosh levels. I'm wondering whether a completely open box might not be even more effective.

15 January 2021

Dirk Eddelbuettel: Rcpp 1.0.6: Some Updates

rcpp logo The Rcpp team is proud to announce release 1.0.6 of Rcpp which arrived at CRAN earlier today, and has been uploaded to Debian too. Windows and macOS builds should appear at CRAN in the next few days. This marks the first release on the new six-months cycle announced with release 1.0.5 in July. As reminder, interim dev or rc releases will often be available in the Rcpp drat repo; this cycle there were four. Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 2174 packages on CRAN depend on Rcpp for making analytical code go faster and further (which is an 8.5% increase just since the last release), along with 207 in BioConductor. This release features six different pull requests from five different contributors, mostly fixing fairly small corner cases, plus some minor polish on documentation and continuous integration. Before releasing we once again made numerous reverse dependency checks none of which revealed any issues. So the passage at CRAN was pretty quick despite the large dependency footprint, and we are once again grateful for all the work the CRAN maintainers do.

Changes in Rcpp patch release version 1.0.6 (2021-01-14)
  • Changes in Rcpp API:
    • Replace remaining few uses of EXTPTR_PTR with R_ExternalPtrAddr (Kevin in #1098 fixing #1097).
    • Add push_back and push_front for DataFrame (Walter Somerville in #1099 fixing #1094).
    • Remove a misleading-to-wrong comment (Mattias Ellert in #1109 cleaning up after #1049).
    • Address a sanitizer report by initializing two private bool variables (Benjamin Christoffersen in #1113).
    • External pointer finalizer toggle default values were corrected to true (Dirk in #1115).
  • Changes in Rcpp Documentation:
    • Several URLs were updated to https and/or new addresses (Dirk).
  • Changes in Rcpp Deployment:
    • Added GitHub Actions CI using the same container-based setup used previously, and also carried code coverage over (Dirk in #1128).
  • Changes in Rcpp support functions:
    • Rcpp.package.skeleton() avoids warning from R. (Dirk)

Thanks to my CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2616 previous questions. If you like this or other open-source work I do, you can sponsor me at GitHub. My sincere thanks to my current sponsors for me keeping me caffeinated.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Mike Gabriel: UBports: Packaging of Lomiri Operating Environment for Debian (part 04)

Before and during FOSDEM 2020, I agreed with the people (developers, supporters, managers) of the UBports Foundation to package the Unity8 Operating Environment for Debian. Since 27th Feb 2020, Unity8 has now become Lomiri. Things got delayed a little recently as my main developer contact on the upstream side was on sick leave for a while. Fortunately, he has now fully recovered and work is getting back on track. Recent Uploads to Debian related to Lomiri Over the past 3 months I worked on the following bits and pieces regarding Lomiri in Debian: The next projects / packages ahead are lomiri-ui-toolkit, integrating changes required for Lomiri into Ayatana Indicators and then moving on with several other, smaller packages. Credits Many big thanks go to Marius and Dalton for their work on the UBports project and being always available for questions, feedback, etc. Thanks to Florian Leeber for being my point of contact for topcis regarding my cooperation with the UBports Foundation.

13 January 2021

Antoine Beaupr : New phone: Pixel 4a

I'm sorry to announce that I gave up on the Fairphone series and switched to a Google Phone (Pixel 4a) running CalyxOS.

Problems in fairy land My fairphone2, even if it is less than two years old, is having major problems:
  • from time to time, the screen flickers and loses "touch" until I squeeze it back together
  • the camera similarly disconnects regularly
  • even when it works, the camera is... pretty bad: low light is basically unusable, it's slow and grainy
  • the battery can barely keep up for one day
  • the cellular coverage is very poor, in Canada: I lose signal at the grocery store and in the middle of my house...
Some of those problems are known: the Fairphone 2 is old now. It was probably old even when I got it. But I can't help but feel a little sad to let it go: the entire point of that device was to make it easy to fix. But alas, because it's sold only in Europe, local stores don't carry replacement parts. To be fair, Fairphone did offer to fix the device, but with a 2 weeks turnaround, I had to get another phone anyways. I did actually try to buy a fairphone3, from Clove. But they did some crazy validation routine. By email, they asked me to provide a photo copy of a driver's license and the credit card, arguing they need to do this to combat fraud. I found that totally unacceptable and asked them to cancel my order. And because I'm not sure the FP3 will fix the coverage issues, I decided to just give up on Fairphone until they officially ship to the Americas.

Do no evil, do not pass go, do not collect 200$ So I got a Google phone, specifically a Pixel 4a. It's a nice device, all small and shiny, but it's "plasticky" - I would have prefered metal, but it seems you need to pay much, much more to get that (in the Pixel 5). In any case, it's certainly a better form factor than the Fairphone 2: even though the screen is bigger, the device itself is actually smaller and thinner, which feels great. The OLED screen is beautiful, awesome contrast and everything, and preliminary tests show that the camera is much better than the one on the Fairphone 2. (The be fair, again, that is another thing the FP3 improved significantly. And that is with the stock Camera app from CalyxOS/AOSP, so not as good as the Google Camera app, which does AI stuff.)

CalyxOS: success The Pixel 4a not not supported by LineageOS: it seems every time I pick a device in that list, I manage to miss the right device by one (I bought a Samsung S9 before, which is also unsupported, even though the S8 is). But thankfully, it is supported by CalyxOS. That install was a breeze: I was hesitant in playing again with installing a custom Android firmware on a phone after fighting with this quite a bit in the past (e.g. htc-one-s, lg-g3-d852). But it turns out their install instructions, mostly using a AOSP alliance device-flasher works absolutely great. It assumes you know about the commandline, and it does require to basically curl sudo (because you need to download their binary and run it as root), but it Just. Works. It reminded me of how great it was to get the Fairphone with TWRP preinstalled... Oh, and kudos to the people in #calyxos on Freenode: awesome tech support, super nice folks. An amazing improvement over the ambiance in #lineageos! :)

Migrating data Unfortunately, migrating the data was the usual pain in the back. This should improve the next time I do this: CalyxOS ships with seedvault, a secure backup system for Android 10 (or 9?) and later which backs up everything (including settings!) with encryption. Apparently it works great, and CalyxOS is also working on a migration system to switch phones. But, obviously, I couldn't use that on the Fairphone 2 running Android 7... So I had to, again, improvised. The first step was to install Syncthing, to have an easy way to copy data around. That's easily done through F-Droid, already bundled with CalyxOS (including the privileged extension!). Pair the devices and boom, a magic portal to copy stuff over. The other early step I took was to copy apps over using the F-Droid "find nearby" functionality. It's a bit quirky, but really helps in copying a bunch of APKs over. Then I setup a temporary keepassxc password vault on the Syncthing share so that I could easily copy-paste passwords into apps. I used to do this in a text file in Syncthing, but copy-pasting in the text file is much harder than in KeePassDX. (I just picked one, maybe KeePassDroid is better? I don't know.) Do keep a copy of the URL of the service to reduce typing as well. Then the following apps required special tweaks:
  • AntennaPod has an import/export feature: export on one end, into the Syncthing share, then import on the other. then go to the queue and select all episodes and download
  • the Signal "chat backup" does copy the secret key around, so you don't get the "security number change" warning (even if it prompts you to re-register) - external devices need to be relinked though
  • AnkiDroid, DSub, Nextcloud, and Wallabag required copy-pasting passwords
I tried to sync contacts with DAVx5 but that didn't work so well: the account was setup correctly, but contacts didn't show up. There's probably just this one thing I need to do to fix this, but since I don't really need sync'd contact, it was easier to export a VCF file to Syncthing and import again.

Known problems One problem with CalyxOS I found is that the fragile little microg tweaks didn't seem to work well enough for Signal. That was unexpected so they encouraged me to file that as a bug. The other "issue" is that the bootloader is locked, which makes it impossible to have "root" on the device. That's rather unfortunate: I often need root to debug things on Android. In particular, it made it difficult to restore data from OSMand (see below). But I guess that most things just work out of the box now, so I don't really need it and appreciate the extra security. Locking the bootloader means full cryptographic verification of the phone, so that's a good feature to have! OSMand still doesn't have a good import/export story. I ended up sharing the Android/data/net.osmand.plus/files directory and importing waypoints, favorites and tracks by hand. Even though maps are actually in there, it's not possible for Syncthing to write directly to the same directory on the new phone, "thanks" to the new permission system in Android which forbids this kind of inter-app messing around. Tracks are particularly a problem: my older OSMand setup had all those folders neatly sorting those tracks by month. This makes it really annoying to track every file manually and copy it over. I have mostly given up on that for now, unfortunately. And I'll still need to reconfigure profiles and maps and everything by hand. Sigh. I guess that's a good clearinghouse for my old tracks I never use... Update: turns out setting storage to "shared" fixed the issue, see comments below!

Conclusion Overall, CalyxOS seems like a good Android firmware. The install is smooth and the resulting install seems solid. The above problems are mostly annoyances and I'm very happy with the experience so far, although I've only been using it for a few hours so this is very preliminary.

Vincent Fourmond: Taking advantage of Ruby in QSoas

First of all, let me all wish you a happy new year, with all my wishes of health and succes. I sincerely hope this year will be simpler for most people as last year ! For the first post of the year, I wanted to show you how to take advantage of Ruby, the programming language embedded in QSoas, to make various things, like:

Summing the values in a column When using commands that take formulas (Ruby code), like apply-formula, the code is run for every single point, for which all the values are updated. In particulier, the state of the previous point is not known. However, it is possible to store values in what is called global variables, whose name start with an $ sign. Using this, we can keep track of the previous values. For instance, to create a new column with the sum of the y values, one can use the following approach:
QSoas> eval $sum=0
QSoas> apply-formula /extra-columns=1 $sum+=y;y2=$sum
The first line initializes the variable to 0, before we start summing, and the code in the second line is run for each dataset row, in order. For the first row, for instance, $sum is initially 0 (from the eval line); after the execution of the code, it is now the first value of y. After the second row, the second value of y is added, and so on. The image below shows the resulting y2 when used on:
QSoas> generate-dataset -1 1 x


Extending values in a column Another use of the global variables is to add "missing" data. For instance, let's imagine that a files given the variation of current over time as the potential is changed, but the potential is only changed stepwise and only indicated when it changes:
## time	current	potential
0	0.1	0.5
1	0.2
2	0.3
3	0.2
4	1.2	0.6
5	1.3
...
If you need to have the values everywhere, for instance if you need to split on their values, you could also use a global variable, taking advantage of the fact that missing values are represented by QSoas using "Not A Number" values, which can be detected using the Ruby function nan?:
QSoas> apply-formula "if y2.nan?; then y2=$value; else $value=y2;end"
Note the need of quotes because there are spaces in the ruby code. If the value of y2 is NaN, that is it is missing, then it is taken from the global variable $value else $value is set the current value of y2. Hence, the values are propagated down:
## time	current	potential
0	0.1	0.5
1	0.2	0.5
2	0.3	0.5
3	0.2	0.5
4	1.2	0.6
5	1.3	0.6
...
Of course, this doesn't work if the first value of y2 is missing.

Renaming using a pattern The command save-datasets can be used to save a whole series of datasets to the disk. It can also rename them on the fly, and, using the /mode=rename option, does only the renaming part, without saving. You can make full use of meta-data (see also a first post here)for renaming. The full power is unlocked using the /expression= option. For instance, for renaming the last 5 datasets (so numbers 0 to 4) using a scheme based on the value of their pH meta-data, you can use the following code:
QSoas> save-datasets /mode=rename /expression='"dataset-# $meta.pH "' 0..4
The double quotes are cumbersome but necessary, since the outer quotes (') prevent the inner ones (") to be removed and the inner quotes are here to indicate to Ruby that we are dealing with text. The bit inside # ... is interpreted by Ruby as Ruby code; here it is $meta.pH, the value of the "pH" meta-data. Finally the 0..4 specifies the datasets to work with. So theses datasets will change name to become dataset-7 for pH 7, etc...

About QSoas QSoas is a powerful open source data analysis program that focuses on flexibility and powerful fitting capacities. It is released under the GNU General Public License. It is described in Fourmond, Anal. Chem., 2016, 88 (10), pp 5050 5052. Current version is 3.0. You can download its source code there (or clone from the GitHub repository) and compile it yourself, or buy precompiled versions for MacOS and Windows there.

Next.

Previous.