(Edit 2023-05-10: This has now
launched for a subset of Twitter users. The code that existed to notify users that device identities had changed does not appear to have been enabled - as a result, in its current form, Twitter can absolutely MITM conversations and read your messages)
Elon Musk appeared on an interview with Tucker Carlson last month, with one of the topics being the fact that Twitter could be legally compelled to hand over users' direct messages to government agencies since they're held on Twitter's servers and aren't encrypted. Elon talked about how they were in the process of implementing proper encryption for DMs that would prevent this - "You could put a gun to my head and I couldn't tell you. That's how it should be."
tl;dr - in the current implementation, while Twitter
could subvert the end-to-end nature of the encryption, it could not do so without users being notified. If any user involved in a conversation were to ignore that notification,
all messages in that conversation (including ones sent in the past) could then be decrypted. This isn't ideal, but it still seems like an improvement over having no encryption at all. More technical discussion follows.
For context: all information about Twitter's implementation here has been derived from reverse engineering version 9.86.0 of the Android client and 9.56.1 of the iOS client (the current versions at time of writing), and the feature hasn't yet launched. While it's certainly possible that there could be major changes in the protocol between now launch, Elon has asserted that they plan to launch the feature this week so it's plausible that this reflects what'll ship.
For it to be impossible for Twitter to read DMs, they need to not only be encrypted, they need to be encrypted with a key that's not available to Twitter. This is what's referred to as "end-to-end encryption", or e2ee - it means that the only components in the communication chain that have access to the unencrypted data are the endpoints. Even if the message passes through other systems (and even if it's
stored on other systems), those systems do not have access to the keys that would be needed to decrypt the data.
End-to-end encrypted messengers were initially popularised by
Signal, but the Signal protocol has since been incorporated into
WhatsApp and is probably much more widely used there. Millions of people per day are sending messages to each other that pass through servers controlled by third parties, but those third parties are completely unable to read the contents of those messages. This is the scenario that Elon described, where there's no degree of compulsion that could cause the people relaying messages to and from people to decrypt those messages afterwards.
But for this to be possible, both ends of the communication need to be able to encrypt messages in a way the other end can decrypt. This is usually performed using
AES, a well-studied encryption algorithm with no known significant weaknesses. AES is a form of what's referred to as a symmetric encryption, one where encryption and decryption are performed with the same key. This means that both ends need access to that key, which presents us with a bootstrapping problem. Until a shared secret is obtained, there's no way to communicate securely, so how do we generate that shared secret? A common mechanism for this is something called
Diffie Hellman key exchange, which makes use of asymmetric encryption. In asymmetric encryption, an encryption key can be split into two components - a public key and a private key. Both devices involved in the communication combine their private key and the other party's public key to generate a secret that can only be decoded with access to the private key. As long as you know the other party's public key, you can now securely generate a shared secret with them. Even a third party with access to all the public keys won't be able to identify this secret. Signal makes use of a variation of Diffie-Hellman called
Extended Triple Diffie-Hellman that has some desirable properties, but it's not
strictly necessary for the implementation of something that's end-to-end encrypted.
Although it was
rumoured that Twitter would make use of the Signal protocol, and in fact there are vestiges of code in the Twitter client that still reference Signal, recent versions of the app have shipped with an entirely different approach that appears to have been written from scratch. It seems simple enough. Each device generates an asymmetric keypair using the NIST P-256 elliptic curve, along with a device identifier. The device identifier and the public half of the key are uploaded to Twitter using a new API endpoint called
/1.1/keyregistry/register. When you want to send an encrypted DM to someone, the app calls
/1.1/keyregistry/extract_public_keys with the IDs of the users you want to communicate with, and gets back a list of their public keys. It then looks up the conversation ID (a numeric identifier that corresponds to a given DM exchange - for a 1:1 conversation between two people it doesn't appear that this ever changes, so if you DMed an account 5 years ago and then DM them again now from the same account, the conversation ID will be the same) in a local database to retrieve a conversation key. If that key doesn't exist yet, the sender generates a random one. The message is then encrypted with the conversation key using AES in GCM mode, and the conversation key is then put through Diffie-Hellman with each of the recipients' public device keys. The encrypted message is then sent to Twitter along with the list of encrypted conversation keys. When each of the recipients' devices receives the message it checks whether it already has a copy of the conversation key, and if not performs its half of the Diffie-Hellman negotiation to decrypt the encrypted conversation key. One it has the conversation key it decrypts it and shows it to the user.
What would happen if Twitter changed the registered public key associated with a device to one where they held the private key, or added an entirely new device to a user's account? If the app were to just happily send a message with the conversation key encrypted with that new key, Twitter would be able to decrypt that and obtain the conversation key. Since the conversation key is tied to the conversation, not any given pair of devices, obtaining the conversation key means you can then decrypt every message in that conversation, including ones sent before the key was obtained.
(An aside: Signal and WhatsApp make use of a protocol called
Sesame which involves additional secret material that's shared between every device a user owns, hence why you have to do that QR code dance whenever you add a new device to your account. I'm grossly over-simplifying how clever the Signal approach is here, largely because I don't understand the details of it myself. The Signal protocol uses something called the
Double Ratchet Algorithm to implement the actual message encryption keys in such a way that even if someone were able to successfully impersonate a device they'd only be able to decrypt messages sent after that point even if they had encrypted copies of every previous message in the conversation)
How's this avoided? Based on the UI that exists in the iOS version of the app, in a fairly straightforward way - each user can only have a single device that supports encrypted messages. If the user (or, in our hypothetical, a malicious Twitter) replaces the device key, the client will generate a notification. If the user pays attention to that notification and verifies with the recipient through some out of band mechanism that the device has actually been replaced, then everything is fine. But, if
any participant in the conversation ignores this warning, the holder of the subverted key can obtain the conversation key and decrypt the entire history of the conversation. That's strictly worse than anything based on Signal, where such impersonation would simply not work, but even in the Twitter case it's not possible for someone to silently subvert the security.
So when Elon says Twitter wouldn't be able to decrypt these messages even if someone held a gun to his head, there's a condition applied to that - it's true as long as nobody fucks up. This is clearly better than the messages just not being encrypted at all in the first place, but overall it's a weaker solution than Signal. If you're currently using Twitter DMs, should you turn on encryption? As long as the limitations aren't too limiting, definitely! Should you use this in preference to Signal or WhatsApp? Almost certainly not. This seems like a genuine incremental improvement, but it'd be easy to interpret what Elon says as providing stronger guarantees than actually exist.
comments
As we get closer to Debian Bookworm's release, I thought I'd share
one change in Python 3.11 that will surely affect many people.
Python 3.11 implements the new PEP 668, Marking Python base environments
as externally managed 1. If you use 
I write letters. The kind that are written on paper with a dip pen 
So I headed over to the obvious program anybody would use for these things (XeLaTeX, of course) and quickly designed a page with fields for the basic thinks I want to record; it was a bit hurried, and I may improve on it the next time I make one, but I expect this one to last me two or three years, and it is good enough.
I ve decided to make it A6 sized, so that it doesn t require a lot of space on my busy desktop, and it could be carried inside a portable desktop, if I ever decide to finish the one for which I ve made a mockup years ago :)
I ve also added a few pages for the addresses of my correspondents (and an index of the letters I ve exchanged with them), and a few empty pages for other notes.
Then I ve used my 
After printing (an older version where some of the pages are repeated. whoops, but it only happened 4 times, and it s not a big deal), it was time for binding this into a book.
I ve opted for Coptic stitch, so that the book will open completely flat and writing on it will be easier and the covers are 2 mm cardboard covered in linen-look bookbinding paper (sadly I no longer have a source for bookbinding cloth made from actual cloth).
I tried to screenprint a simple design on the cover: the first attempt was unusable (the paper was smaller than the screen, so I couldn t keep it in the right place and moved as I was screenprinting); on the second attempt I used some masking tape to keep the paper in place, and they were a bit better, but I need more practice with the technique.
Finally, I decided that for such a Victorian thing I will use an Iron-gall ink, but it s Rohrer & Knlingner Scabiosa, with a purple undertone, because life s too short to use blue-black ink :D
And now, I m off to write an actual letter, rather than writing online about things that are related to letter writing.
Kweather Snap
Hello folks!
We recently received a case letting us know that Dataproc 2.1.1 was unable to write to a BigQuery table with a column of type JSON. Although the BigQuery connector for Spark has had support for JSON columns since 0.28.0, the Dataproc images on the 2.1 line still cannot create tables with JSON columns or write to existing tables with JSON columns.
The customer has graciously granted permission to share the code we developed to allow this operation. So if you are interested in working with JSON column tables on Dataproc 2.1 please continue reading!
Use the following gcloud command to create your single-node dataproc cluster:
I do try not to report because right now every other day we see somewhere or the other the Govt. curtailing our rights and most people are mute 
When I travel for a few days I don t usually
For the soles I may have gone a bit overboard with the vibram claw, but:
As for the finished weight, at 235 g for the pair I thought I could do better, but apparently shoes are considered ultralight if they are around 500 g? Using just one layer of mesh rather than two would probably help, but it would have required a few changes to the pattern, and anyway I don t really to carry them around all day.
I ve also added a loop of fabric (polycotton) to the centre back to be able to hang the slippers to the backpack when wet or dirty; a bit of narrow webbing may have been better, but I didn t have any in my stash.
The 
This isn't a book review, although the reason that I am typing this now is
because of a book, You Are Not Alone: from the creator and host of
Griefcast, Cariad Lloyd, ISBN: 978-1526621870 and I include a handful of
quotes from Cariad where there is really no better way of describing things.
Many people experience death for the first time as a child, often relating to a
family pet. Death is universal but every experience of death is unique. One of
the myths of grief is the idea of the Five Stages but this is a
misinterpretation. Denial, Anger, Bargaining, Depression and Acceptance
represent the five stage model of death and have nothing to do with
grief. The five stages were developed from studying those who are
terminally ill, the dying, not those who then grieve for the dead person and
have to go on living without them. Grief is for those who loved the person who
has died and it varies between each of those people just as people vary in how
they love someone. The Five Stages end at the moment of death, grief is what
comes next and most people do not grieve in stages, it can be more like a
tangled knot.
Death has a date and time, so that is why the last stage of the model is
Acceptance. Grief has no timetable, those who grieve will carry that grief for
the rest of their lives. Death starts the process of grief in those who go on
living just as it ends the life of the person who is loved. "Grief eases and
changes and returns but it never disappears.".
I suspect many will have already stopped reading by this point. People do not
talk about death and grief enough and this only adds to the burden of those who
carry their grief. It can be of enormous comfort to those who have carried
grief for some time to talk directly about the dead, not in vague pleasantries
but with specific and strong memories. Find a safe place without distractions
and talk with the person grieving face to face. Name the dead person. Go to
places with strong memories and be there alongside. Talk about the times with
that person before their death. Early on, everything about grief is painful and
sad. It does ease but it remains unpredictable. Closing it away in a box inside
your head (as I did at one point) is like cutting off a damaged limb but
keeping the pain in a box on the shelf. You still miss the limb and eventually,
the box starts leaking.
For me, there were family pets which died but my first job out of university
was to work in hospitals, helping the nurses manage the medication regimen and
providing specialist advice as a pharmacist. It will not be long in that
environment before everyone on the ward gets direct experience of the death of
a person. In some ways, this helped me to separate the process of death from
the process of grief. I cared for these people as patients but these were not
my loved ones. Later, I worked in specialist terminal care units, including
providing potential treatments as part of clinical trials. Here, it was not
expected for any patient to be discharged alive. The more aggressive
chemotherapies had already been tried and had failed, this was about pain
relief, symptom management and helping the loved ones. Palliative care is not
just about the patient, it involves helping the loved ones to accept what is
happening as this provides comfort to the patient by closing the loop.
Grief is stressful. One of the most common