This post is the second and final part of my Malaysia-Thailand trip. Feel free to check out the Malaysia part here if you haven t already. Kuala Lumpur to Bangkok is around 1500 km by road, and so I took a Malaysian Airlines flight to travel to Bangkok. The flight staff at the Kuala Lumpur only asked me for a return/onward flight and Thailand immigration asked a few questions but did not check any documents (obviously they checked and stamped my passport ;)). The currency of Thailand is the Thai baht, and 1 Thai baht = 2.5 Indian Rupees. The Thailand time is 1.5 hours ahead of Indian time (For example, if it is 12 noon in India, it will be 13:30 in Thailand).
I landed in Bangkok at around 3 PM local time. Fletcher was in Bangkok that time, leaving for Pattaya and we had booked the same hostel. So I took a bus to Pattaya from the airport. The next bus for which the tickets were available was at 7 PM, so I took tickets for that one. The bus ticket cost was 143 Thai Baht. I didn t buy SIM at the airport, thinking there must be better deals in the city. As a consequence, there was no way to contact Fletcher through internet. Although I had a few minutes call remaining out of my international roaming pack.
A welcome sign at Bangkok's Suvarnabhumi airport.
Bus from Suvarnabhumi Airport to Jomtien Beach in Pattaya.
Our accommodation was near Jomtien beach, so I got off at the last stop, as the bus terminates at the Jomtien beach. Then I decided to walk towards my accommodation. I was using OsmAnd for navigation. However, the place was not marked on OpenStreetMap, and it turned out I missed the street my hostel was on and walked around 1 km further as I was chasing a similarly named incorrect hostel on OpenStreetMap. Then I asked for help from two men sitting at a caf . One of them said he will help me find the street my hostel is on. So, I walked with him, and he told me he lives in Thailand for many years, but he is from Kuwait. He also gave me valuable information. Like, he told me about shared hail-and-ride songthaews which run along the Jomtien Second Road and charge 10 Baht for any distance on their route. This tip significantly reduced our expenses. Further, he suggested me 7-Eleven shops for buying a local SIM. Like Malaysia, Thailand has 24/7 7-Eleven convenience stores, a lot of them not even 100 m apart.
The Kuwaiti person dropped me at the address where my hostel was. I tried searching for a person in-charge of that hostel, and soon I realized there was no reception. After asking for help from locals for some time, I bumped into Fletcher, who also came to this address and was searching for the same. After finding a friend, I felt a sigh of relief. Adjacent to the property, there was a hairdresser shop. We went there and asked about this property. The woman called the owner, and she also told us the required passcodes to go inside. Our accommodation was in a room on the second floor, which required us to put a passcode for opening. We entered the passcode and entered the room. So, we stayed at this hostel which had no reception. Due to this, it took 2 hours to find our room and enter. It reminded me of a difficult experience I had in Albania, where me and Akshat were not able to find our apartment in one of the hottest days and the owner didn t know our language.
Traveling from the place where the bus dropped me to the hostel, I saw streets were filled with bars and massage parlors, which was expected. Prostitutes were everywhere. We went out at night towards the beach and also roamed around in 7-Elevens to buy a SIM card for myself. I got a SIM for 7 day unlimited internet for 399 baht. Turns out that the rates of SIM cards at the airport were not so different from inside the city.
Road near Jomtien beach in Pattaya
Photo of a songthaew in Pattaya. There are shared songthaews which run along Jomtien Second road and takes 10 bath to anywhere on the route.
Jomtien Beach in Pattaya.
In terms of speaking English, locals didn t know English at all in both Pattaya and Bangkok. I normally don t expect locals to know English in a non-English speaking country, but the fact that Bangkok is one of the most visited places by tourists made me expect locals to know some English. Talking to locals is an integral part of travel for me, which I couldn t do a lot in Thailand. This aspect is much more important for me than going to touristy places.
So, we were in Pattaya. Next morning, Fletcher and I went to Tiger park using shared songthaew. After that, we planned to visit Pattaya Floating market which is near the Tiger Park, but we felt the ticket prices were higher than it was worth. Fletcher had to leave for Bangkok on that day. I suggested him to go to Suvarnabhumi Airport from the Jomtien beach bus terminal (this was the route I took the last day in opposite direction) to avoid traffic congestion inside Bangkok, as he can follow up with metro once he reaches the airport. From the floating market, we were walking in sweltering heat to reach the Jomtien beach. I tried asking for a lift and eventually got successful as a scooty stopped, and surprisingly the person gave a ride to both of us. He was from Delhi, so maybe that s the reason he stopped for us. Then we took a songthaew to the bus terminal and after having lunch, Fletcher left for Bangkok.
A welcome sign at Pattaya Floating market.
This Korean Vegetasty noodles pack was yummy and was available at many 7-Eleven stores.
Next day I went to Bangkok, but Fletcher already left for Kuala Lumpur. Here I had booked a private room in a hotel (instead of a hostel) for four nights, mainly because of my luggage. This costed 5600 INR for four nights. It was 2 km from the metro station, which I used to walk both sides. In Bangkok, I visited Sukhumvit and Siam by metro. Going to some areas require crossing the Chao Phraya river. For this, I took Chao Phraya Express Boat for going to places like Khao San road and Wat Arun. I would recommend taking the boat ride as it had very good views. In Bangkok, I met a person from Pakistan staying in my hotel and so here also I got some company. But by the time I met him, my days were almost over. So, we went to a random restaurant selling Indian food where we ate some paneer dish with naan and that restaurant person was from Myanmar.
Wat Arun temple stamps your hand upon entry
Wat Arun temple
Khao San Road
A food stall at Khao San Road
Chao Phraya Express Boat
For eating, I mainly relied on fruits and convenience stores. Bananas were very tasty. This was the first time I saw banana flesh being yellow. Mangoes were delicious and pineapples were smaller and flavorful. I also ate Rose Apple, which I never had before. I had Chhole Kulche once in Sukhumvit. That was a little expensive as it costed 164 baht. I also used to buy premix coffee packets from 7-Eleven convenience stores and prepare them inside the stores.
Banana with yellow flesh
Fruits at a stall in Bangkok
Trimmed pineapples from Thailand.
Corn in Bangkok.
A board showing coffee menu at a 7-Eleven store along with rates in Pattaya.
In this section of 7-Eleven, you can buy a premix coffee and mix it with hot water provided at the store to prepare.
My booking from Bangkok to Delhi was in Air India flight, and they were serving alcohol in the flight. I chose red wine, and this was my first time having alcohol in a flight.
Red wine being served in Air India
Notes
In this whole trip spanning two weeks, I did not pay for drinking water (except for once in Pattaya which was 9 baht) and toilets. Bangkok and Kuala Lumpur have plenty of malls where you should find a free-of-cost toilet nearby. For drinking water, I relied mainly on my accommodation providing refillable water for my bottle.
Thailand seemed more expensive than Malaysia on average. Malaysia had discounted price due to the Chinese New year.
I liked Pattaya more than Bangkok. Maybe because Pattaya has beach and Bangkok doesn t. Pattaya seemed more lively, and I could meet and talk to a few people as opposed to Bangkok.
Chao Phraya River express boat costs 150 baht for one day where you can hop on and off to any boat.
I started looking at aerc, a new Terminal mail client, in
around 2019. At that time it was promising, but ultimately not ready yet for me, so
I put it away and went back to neomutt which I have been
using (in one form or another) all century.
These days, I use neomutt as an IMAP client which is perhaps what it's worst
at: prior to that, and in common with most users (I think), I used it to read
local mail, either fetched via offlineimap or
directly on my mail server. I switched to using it as a (slow, blocking) IMAP
client because I got sick of maintaining offlineimap (or
mbsync), and I started to use neomutt to
read my work mail, which was too large (and rate limited) for local
syncing.
This year I noticed that aerc had a new maintainer who was presenting about
it at FOSDEM, so I thought I'd take another look. It's
come a long way: far enough to actually displace neomutt for my day-to-day
mail use. In particular, it's a much better IMAP client.
I still reach for neomutt for some tasks, but I'm now using aerc for most
things.
aerc is available in Debian, but I recommending building from upstream source
at the moment as the project is quite fast-moving.
On Mastodon, the
question came up of
how Ubuntu would deal with something like the npm install
everything situation. I replied:
Ubuntu is curated, so it probably wouldn t get this far. If it did, then
the worst case is that it would get in the way of CI allowing other
packages to be removed (again from a curated system, so people are used to
removal not being self-service); but the release team would have no
hesitation in removing a package like this to fix that, and it certainly
wouldn t cause this amount of angst.
If you did this in a PPA, then
I can t think of any particular negative effects.
OK, if you added lots of build-dependencies (as well as run-time
dependencies) then you might be able to take out a builder. But Launchpad
builders already run arbitrary user-submitted code by design and are
therefore very carefully sandboxed and treated as ephemeral, so this is
hardly novel.
There s a lot to be said for the arrangement of having a curated system for
the stuff people actually care about plus an ecosystem of add-on
repositories. PPAs cover a wide range of levels of developer activity, from
throwaway experiments to quasi-official distribution methods; there are
certainly problems that arise from it being difficult to tell the difference
between those extremes and from there being no systematic confinement, but
for this particular kind of problem they re very nearly ideal. (Canonical
has tried various other approaches to software distribution, and while they
address some of the problems, they aren t obviously
better at helping
people make reliable social judgements about code they don t know.)
For a hypothetical package with a huge number of dependencies, to even try
to upload it directly to Ubuntu you d need to be an Ubuntu developer with
upload rights (or to go via Debian, where you d have to clear a similar
hurdle). If you have those, then the first upload has to pass manual review
by an archive administrator. If your package passes that, then it still has
to build and get through
proposed-migrationCI before it
reaches anything that humans typically care about.
On the other hand, if you were inclined to try this sort of experiment,
you d almost certainly try it in a PPA, and that would trouble nobody but yourself.
My effort to improve transparency and confidence of public apt archives continues. I started to work on this in Apt Archive Transparency in which I mention the debdistget project in passing. Debdistget is responsible for mirroring index files for some public apt archives. I ve realized that having a publicly auditable and preserved mirror of the apt repositories is central to being able to do apt transparency work, so the debdistget project has become more central to my project than I thought. Currently I track Trisquel, PureOS, Gnuinos and their upstreams Ubuntu, Debian and Devuan.
Debdistget download Release/Package/Sources files and store them in a git repository published on GitLab. Due to size constraints, it uses two repositories: one for the Release/InRelease files (which are small) and one that also include the Package/Sources files (which are large). See for example the repository for Trisquel release files and the Trisquel package/sources files. Repositories for all distributions can be found in debdistutils archives GitLab sub-group.
The reason for splitting into two repositories was that the git repository for the combined files become large, and that some of my use-cases only needed the release files. Currently the repositories with packages (which contain a couple of months worth of data now) are 9GB for Ubuntu, 2.5GB for Trisquel/Debian/PureOS, 970MB for Devuan and 450MB for Gnuinos. The repository size is correlated to the size of the archive (for the initial import) plus the frequency and size of updates. Ubuntu s use of Apt Phased Updates (which triggers a higher churn of Packages file modifications) appears to be the primary reason for its larger size.
Working with large Git repositories is inefficient and the GitLab CI/CD jobs generate quite some network traffic downloading the git repository over and over again. The most heavy user is the debdistdiff project that download all distribution package repositories to do diff operations on the package lists between distributions. The daily job takes around 80 minutes to run, with the majority of time is spent on downloading the archives. Yes I know I could look into runner-side caching but I dislike complexity caused by caching.
Fortunately not all use-cases requires the package files. The debdistcanary project only needs the Release/InRelease files, in order to commit signatures to the Sigstore and Sigsum transparency logs. These jobs still run fairly quickly, but watching the repository size growth worries me. Currently these repositories are at Debian 440MB, PureOS 130MB, Ubuntu/Devuan 90MB, Trisquel 12MB, Gnuinos 2MB. Here I believe the main size correlation is update frequency, and Debian is large because I track the volatile unstable.
So I hit a scalability end with my first approach. A couple of months ago I solved this by discarding and resetting these archival repositories. The GitLab CI/CD jobs were fast again and all was well. However this meant discarding precious historic information. A couple of days ago I was reaching the limits of practicality again, and started to explore ways to fix this. I like having data stored in git (it allows easy integration with software integrity tools such as GnuPG and Sigstore, and the git log provides a kind of temporal ordering of data), so it felt like giving up on nice properties to use a traditional database with on-disk approach. So I started to learn about Git-LFS and understanding that it was able to handle multi-GB worth of data that looked promising.
Fairly quickly I scripted up a GitLab CI/CD job that incrementally update the Release/Package/Sources files in a git repository that uses Git-LFS to store all the files. The repository size is now at Ubuntu 650kb, Debian 300kb, Trisquel 50kb, Devuan 250kb, PureOS 172kb and Gnuinos 17kb. As can be expected, jobs are quick to clone the git archives: debdistdiff pipelines went from a run-time of 80 minutes down to 10 minutes which more reasonable correlate with the archive size and CPU run-time.
The LFS storage size for those repositories are at Ubuntu 15GB, Debian 8GB, Trisquel 1.7GB, Devuan 1.1GB, PureOS/Gnuinos 420MB. This is for a couple of days worth of data. It seems native Git is better at compressing/deduplicating data than Git-LFS is: the combined size for Ubuntu is already 15GB for a couple of days data compared to 8GB for a couple of months worth of data with pure Git. This may be a sub-optimal implementation of Git-LFS in GitLab but it does worry me that this new approach will be difficult to scale too. At some level the difference is understandable, Git-LFS probably store two different Packages files around 90MB each for Trisquel as two 90MB files, but native Git would store it as one compressed version of the 90MB file and one relatively small patch to turn the old files into the next file. So the Git-LFS approach surprisingly scale less well for overall storage-size. Still, the original repository is much smaller, and you usually don t have to pull all LFS files anyway. So it is net win.
Throughout this work, I kept thinking about how my approach relates to Debian s snapshot service. Ultimately what I would want is a combination of these two services. To have a good foundation to do transparency work I would want to have a collection of all Release/Packages/Sources files ever published, and ultimately also the source code and binaries. While it makes sense to start on the latest stable releases of distributions, this effort should scale backwards in time as well. For reproducing binaries from source code, I need to be able to securely find earlier versions of binary packages used for rebuilds. So I need to import all the Release/Packages/Sources packages from snapshot into my repositories. The latency to retrieve files from that server is slow so I haven t been able to find an efficient/parallelized way to download the files. If I m able to finish this, I would have confidence that my new Git-LFS based approach to store these files will scale over many years to come. This remains to be seen. Perhaps the repository has to be split up per release or per architecture or similar.
Another factor is storage costs. While the git repository size for a Git-LFS based repository with files from several years may be possible to sustain, the Git-LFS storage size surely won t be. It seems GitLab charges the same for files in repositories and in Git-LFS, and it is around $500 per 100GB per year. It may be possible to setup a separate Git-LFS backend not hosted at GitLab to serve the LFS files. Does anyone know of a suitable server implementation for this? I had a quick look at the Git-LFS implementation list and it seems the closest reasonable approach would be to setup the Gitea-clone Forgejo as a self-hosted server. Perhaps a cloud storage approach a la S3 is the way to go? The cost to host this on GitLab will be manageable for up to ~1TB ($5000/year) but scaling it to storing say 500TB of data would mean an yearly fee of $2.5M which seems like poor value for the money.
I realized that ultimately I would want a git repository locally with the entire content of all apt archives, including their binary and source packages, ever published. The storage requirements for a service like snapshot (~300TB of data?) is today not prohibitly expensive: 20TB disks are $500 a piece, so a storage enclosure with 36 disks would be around $18.000 for 720TB and using RAID1 means 360TB which is a good start. While I have heard about ~TB-sized Git-LFS repositories, would Git-LFS scale to 1PB? Perhaps the size of a git repository with multi-millions number of Git-LFS pointer files will become unmanageable? To get started on this approach, I decided to import a mirror of Debian s bookworm for amd64 into a Git-LFS repository. That is around 175GB so reasonable cheap to host even on GitLab ($1000/year for 200GB). Having this repository publicly available will make it possible to write software that uses this approach (e.g., porting debdistreproduce), to find out if this is useful and if it could scale. Distributing the apt repository via Git-LFS would also enable other interesting ideas to protecting the data. Consider configuring apt to use a local file:// URL to this git repository, and verifying the git checkout using some method similar to Guix s approach to trusting git content or Sigstore s gitsign.
A naive push of the 175GB archive in a single git commit ran into pack size limitations:
remote: fatal: pack exceeds maximum allowed size (4.88 GiB)
however breaking up the commit into smaller commits for parts of the archive made it possible to push the entire archive. Here are the commands to create this repository:
git init git lfs install git lfs track 'dists/**' 'pool/**' git add .gitattributes git commit -m"Add Git-LFS track attributes." .gitattributes time debmirror --method=rsync --host ftp.se.debian.org --root :debian --arch=amd64 --source --dist=bookworm,bookworm-updates --section=main --verbose --diff=none --keyring /usr/share/keyrings/debian-archive-keyring.gpg --ignore .git . git add dists project git commit -m"Add." -a git remote add origin git@gitlab.com:debdistutils/archives/debian/mirror.git git push --set-upstream origin --all for d in pool//; do echo $d; time git add $d; git commit -m"Add $d." -a git push done
The resulting repository size is around 27MB with Git LFS object storage around 174GB. I think this approach would scale to handle all architectures for one release, but working with a single git repository for all releases for all architectures may lead to a too large git repository (>1GB). So maybe one repository per release? These repositories could also be split up on a subset of pool/ files, or there could be one repository per release per architecture or sources.
Finally, I have concerns about using SHA1 for identifying objects. It seems both Git and Debian s snapshot service is currently using SHA1. For Git there is SHA-256 transition and it seems GitLab is working on support for SHA256-based repositories. For serious long-term deployment of these concepts, it would be nice to go for SHA256 identifiers directly. Git-LFS already uses SHA256 but Git internally uses SHA1 as does the Debian snapshot service.
What do you think? Happy Hacking!
Last week we held our promised miniDebConf in Santa Fe City, Santa Fe province,
Argentina just across the river from Paran , where I have spent almost six
beautiful months I will never forget.
Around 500 Kilometers North from Buenos Aires, Santa Fe and Paran are separated
by the beautiful and majestic Paran river, which flows from Brazil, marks the
Eastern border of Paraguay, and continues within Argentina as the heart of the
litoral region of the country, until it merges with the Uruguay river (you
guessed right the river marking the Eastern border of Argentina, first with
Brazil and then with Uruguay), and they become the R o de la Plata.
This was a short miniDebConf: we were lent the APUL union s building for the
weekend (thank you very much!); during Saturday, we had a cycle of talks, and on
sunday we had more of a hacklab logic, having some unstructured time to work
each on their own projects, and to talk and have a good time together.
We were five Debian people attending:
santiago debacle eamanu dererk gwolf @debian.org. My main contact to
kickstart organization was Mart n Bayo. Mart n was for many years the leader of
the Technical Degree on Free Software at Universidad Nacional del
Litoral,
where I was also a teacher for several years. Together with Leo Mart nez, also a
teacher at the tecnicatura, they contacted us with Guillermo and Gabriela,
from the APUL non-teaching-staff union of said university.
We had the following set of talks (for which there is a promise to get
electronic record, as APUL was kind enough to record them! of course, I will
push them to our usual conference video archiving service as soon as I get them)
Hour
Title (Spanish)
Title (English)
Presented by
10:00-10:25
Introducci n al Software Libre
Introduction to Free Software
Mart n Bayo
10:30-10:55
Debian y su comunidad
Debian and its community
Emanuel Arias
11:00-11:25
Por qu sigo contribuyendo a Debian despu s de 20 a os?
Why am I still contributing to Debian after 20 years?
Santiago Ruano
11:30-11:55
Mi identidad y el proyecto Debian: Qu es el llavero OpenPGP y por qu ?
My identity and the Debian project: What is the OpenPGP keyring and why?
Gunnar Wolf
12:00-13:00
Explorando las masculinidades en el contexto del Software Libre
Exploring masculinities in the context of Free Software
Gora Ortiz Fuentes - Jos Francisco Ferro
13:00-14:30
Lunch
14:30-14:55
Debian para el d a a d a
Debian for our every day
Leonardo Mart nez
15:00-15:25
Debian en las Raspberry Pi
Debian in the Raspberry Pi
Gunnar Wolf
15:30-15:55
Device Trees
Device Trees
Lisandro Dami n Nicanor Perez Meyer (videoconferencia)
16:00-16:25
Python en Debian
Python in Debian
Emmanuel Arias
16:30-16:55
Debian y XMPP en la medici n de viento para la energ a e lica
Debian and XMPP for wind measuring for eolic energy
Martin Borgert
As it always happens DebConf, miniDebConf and other Debian-related activities
are always fun, always productive, always a great opportunity to meet again our
decades-long friends. Lets see what comes next!
teamwork, or: why I love the Debian Perl Group:
elbrus has introduced a (very untypical) package into the
Debian Perl Group in 2022.
after changes of the default compiler options
(-Werror=implicit-function-declaration) in debian, it didn't
build any more & received an RC bug.
because I sometimes like challenges, I had a look at it & cobbled together
a patch. as I hardly speak any C, I sent my notes to the bug report
& (implictly) asked for help. & went out to meet a
friend.
when I came home, I found an email from ntyni, sent less than 2 hours
after my mail, where he friendly pointed out the issues with my patch
& sent a corrected version.
all I needed to do was to adjust the patch & upload the package. one
more bug fixed, one less task for us, & elbrus can concentrate on more
important tasks :) thanks again, niko!
Closing arguments in the trial between various people and Craig Wright over whether he's Satoshi Nakamoto are wrapping up today, amongst a bewildering array of presented evidence. But one utterly astonishing aspect of this lawsuit is that expert witnesses for both sides agreed that much of the digital evidence provided by Craig Wright was unreliable in one way or another, generally including indications that it wasn't produced at the point in time it claimed to be. And it's fascinating reading through the subtle (and, in some cases, not so subtle) ways that that's revealed.
One of the pieces of evidence entered is screenshots of data from Mind Your Own Business, a business management product that's been around for some time. Craig Wright relied on screenshots of various entries from this product to support his claims around having controlled meaningful number of bitcoin before he was publicly linked to being Satoshi. If these were authentic then they'd be strong evidence linking him to the mining of coins before Bitcoin's public availability. Unfortunately the screenshots themselves weren't contemporary - the metadata shows them being created in 2020. This wouldn't fundamentally be a problem (it's entirely reasonable to create new screenshots of old material), as long as it's possible to establish that the material shown in the screenshots was created at that point. Sadly, well.
One part of the disclosed information was an email that contained a zip file that contained a raw database in the format used by MYOB. Importing that into the tool allowed an audit record to be extracted - this record showed that the relevant entries had been added to the database in 2020, shortly before the screenshots were created. This was, obviously, not strong evidence that Craig had held Bitcoin in 2009. This evidence was reported, and was responded to with a couple of additional databases that had an audit trail that was consistent with the dates in the records in question. Well, partially. The audit record included session data, showing an administrator logging into the data base in 2011 and then, uh, logging out in 2023, which is rather more consistent with someone changing their system clock to 2011 to create an entry, and switching it back to present day before logging out. In addition, the audit log included fields that didn't exist in versions of the product released before 2016, strongly suggesting that the entries dated 2009-2011 were created in software released after 2016. And even worse, the order of insertions into the database didn't line up with calendar time - an entry dated before another entry may appear in the database afterwards, indicating that it was created later. But even more obvious? The database schema used for these old entries corresponded to a version of the software released in 2023.
This is all consistent with the idea that these records were created after the fact and backdated to 2009-2011, and that after this evidence was made available further evidence was created and backdated to obfuscate that. In an unusual turn of events, during the trial Craig Wright introduced further evidence in the form of a chain of emails to his former lawyers that indicated he had provided them with login details to his MYOB instance in 2019 - before the metadata associated with the screenshots. The implication isn't entirely clear, but it suggests that either they had an opportunity to examine this data before the metadata suggests it was created, or that they faked the data? So, well, the obvious thing happened, and his former lawyers were asked whether they received these emails. The chain consisted of three emails, two of which they confirmed they'd received. And they received a third email in the chain, but it was different to the one entered in evidence. And, uh, weirdly, they'd received a copy of the email that was submitted - but they'd received it a few days earlier. In 2024.
And again, the forensic evidence is helpful here! It turns out that the email client used associates a timestamp with any attachments, which in this case included an image in the email footer - and the mysterious time travelling email had a timestamp in 2024, not 2019. This was created by the client, so was consistent with the email having been sent in 2024, not being sent in 2019 and somehow getting stuck somewhere before delivery. The date header indicates 2019, as do encoded timestamps in the MIME headers - consistent with the mail being sent by a computer with the clock set to 2019.
But there's a very weird difference between the copy of the email that was submitted in evidence and the copy that was located afterwards! The first included a header inserted by gmail that included a 2019 timestamp, while the latter had a 2024 timestamp. Is there a way to determine which of these could be the truth? It turns out there is! The format of that header changed in 2022, and the version in the email is the new version. The version with the 2019 timestamp is anachronistic - the format simply doesn't match the header that gmail would have introduced in 2019, suggesting that an email sent in 2022 or later was modified to include a timestamp of 2019.
This is by no means the only indication that Craig Wright's evidence may be misleading (there's the whole argument that the Bitcoin white paper was written in LaTeX when general consensus is that it's written in OpenOffice, given that's what the metadata claims), but it's a lovely example of a more general issue.
Our technology chains are complicated. So many moving parts end up influencing the content of the data we generate, and those parts develop over time. It's fantastically difficult to generate an artifact now that precisely corresponds to how it would look in the past, even if we go to the effort of installing an old OS on an old PC and setting the clock appropriately (are you sure you're going to be able to mimic an entirely period appropriate patch level?). Even the version of the font you use in a document may indicate it's anachronistic. I'm pretty good at computers and I no longer have any belief I could fake an old document.
(References: this Dropbox, under "Expert reports", "Patrick Madden". Initial MYOB data is in "Appendix PM7", further analysis is in "Appendix PM42", email analysis is "Sixth Expert Report of Mr Patrick Madden")
Utkarsh Gupta
did 11.25h (out of 26.75h assigned and 33.25h from previous period), thus carrying over 48.75 to the next month.
Evolution of the situation
In February, we have released 17 DLAs.
The number of DLAs published during February was a bit lower than usual, as there was much work going on in the area of triaging CVEs (a number of which turned out to not affect Debia buster, and others which ended up being duplicates, or otherwise determined to be invalid). Of the packages which did receive updates, notable were sudo (to fix a privilege management issue), and iwd and wpa (both of which suffered from authentication bypass vulnerabilities).
While this has already been already announced in the Freexian blog, we would like to mention here the start of the Long Term Support project for Samba 4.17. You can find all the important details in that post, but we would like to highlight that it is thanks to our LTS sponsors that we are able to fund the work from our partner, Catalyst, towards improving the security support of Samba in Debian 12 (Bookworm).
Thanks to our sponsors
Sponsors that joined recently are in bold.
Introduction
There have been many experiments with the sizes of computers, some of which have stayed around and some have gone away. The trend has been to make computers smaller, the early computers had buildings for them. Recently for come classes computers have started becoming as small as could be reasonably desired. For example phones are thin enough that they can blow away in a strong breeze, smart watches are much the same size as the old fashioned watches they replace, and NUC type computers are as small as they need to be given the size of monitors etc that they connect to.
This means that further development in the size and shape of computers will largely be determined by human factors.
I think we need to consider how computers might be developed to better suit humans and how to write free software to make such computers usable without being constrained by corporate interests.
Those of us who are involved in developing OSs and applications need to consider how to adjust to the changes and ideally anticipate changes. While we can t anticipate the details of future devices we can easily predict general trends such as being smaller, higher resolution, etc.
Desktop/Laptop PCs
When home computers first came out it was standard to have the keyboard in the main box, the Apple ][ being the most well known example. This has lost popularity due to the demand to have multiple options for a light keyboard that can be moved for convenience combined with multiple options for the box part. But it still pops up occasionally such as the Raspberry Pi 400 [1] which succeeds due to having the computer part being small and light. I think this type of computer will remain a niche product. It could be used in a add a screen to make a laptop as opposed to the add a keyboard to a tablet to make a laptop model but a tablet without a keyboard is more useful than a non-server PC without a display.
The PC as box with connections for keyboard, display, etc has a long future ahead of it. But the sizes will probably decrease (they should have stopped making PC cases to fit CD/DVD drives at least 10 years ago). The NUC size is a useful option and I think that DVD drives will stop being used for software soon which will allow a range of smaller form factors.
The regular laptop is something that will remain useful, but the tablet with detachable keyboard devices could take a lot of that market. Full functionality for all tasks requires a keyboard because at the moment text editing with a touch screen is an unsolved problem in computer science [2].
The Lenovo Thinkpad X1 Fold [3] and related Lenovo products are very interesting. Advances in materials allow laptops to be thinner and lighter which leaves the screen size as a major limitation to portability. There is a conflict between desiring a large screen to see lots of content and wanting a small size to carry and making a device foldable is an obvious solution that has recently become possible. Making a foldable laptop drives a desire for not having a permanently attached keyboard which then makes a touch screen keyboard a requirement. So this means that user interfaces for PCs have to be adapted to work well on touch screens. The Think line seems to be continuing the history of innovation that it had when owned by IBM. There are also a range of other laptops that have two regular screens so they are essentially the same as the Thinkpad X1 Fold but with two separate screens instead of one folding one, prices are as low as $600US.
I think that the typical interfaces for desktop PCs (EG MS-Windows and KDE) don t work well for small devices and touch devices and the Android interface generally isn t a good match for desktop systems. We need to invent more options for this. This is not a criticism of KDE, I use it every day and it works well. But it s designed for use cases that don t match new hardware that is on sale. As an aside it would be nice if Lenovo gave samples of their newest gear to people who make significant contributions to GUIs. Give a few Thinkpad Fold devices to KDE people, a few to GNOME people, and a few others to people involved in Wayland development and see how that promotes software development and future sales.
We also need to adopt features from laptops and phones into desktop PCs. When voice recognition software was first released in the 90s it was for desktop PCs, it didn t take off largely because it wasn t very accurate (none of them recognised my voice). Now voice recognition in phones is very accurate and it s very common for desktop PCs to have a webcam or headset with a microphone so it s time for this to be re-visited. GPS support in laptops is obviously useful and can work via Wifi location, via a USB GPS device, or via wwan mobile phone hardware (even if not used for wwan networking). Another possibility is using the same software interfaces as used for GPS on laptops for a static definition of location for a desktop PC or server.
The Interesting New Things
Watch Like
The wrist-watch [4] has been a standard format for easy access to data when on the go since it s military use at the end of the 19th century when the practical benefits beat the supposed femininity of the watch. So it seems most likely that they will continue to be in widespread use in computerised form for the forseeable future. For comparison smart phones have been in widespread use as pocket watches for about 10 years.
The question is how will watch computers end up? Will we have Dick Tracy style watch phones that you speak into? Will it be the current smart watch functionality of using the watch to answer a call which goes to a bluetooth headset? Will smart watches end up taking over the functionality of the calculator watch [5] which was popular in the 80 s? With today s technology you could easily have a fully capable PC strapped to your forearm, would that be useful?
Phone Like
Folding phones (originally popularised as Star Trek Tricorders) seem likely to have a long future ahead of them. Engineering technology has only recently developed to the stage of allowing them to work the way people would hope them to work (a folding screen with no gaps). Phones and tablets with multiple folds are coming out now [6]. This will allow phones to take much of the market share that tablets used to have while tablets and laptops merge at the high end. I ve previously written about Convergence between phones and desktop computers [7], the increased capabilities of phones adds to the case for Convergence.
Folding phones also provide new possibilities for the OS. The Oppo OnePlus Open and the Google Pixel Fold both have a UI based around using the two halves of the folding screen for separate data at some times. I think that the current user interfaces for desktop PCs don t properly take advantage of multiple monitors and the possibilities raised by folding phones only adds to the lack. My pet peeve with multiple monitor setups is when they don t make it obvious which monitor has keyboard focus so you send a CTRL-W or ALT-F4 to the wrong screen by mistake, it s a problem that also happens on a single screen but is worse with multiple screens. There are rumours of phones described as three fold (where three means the number of segments with two folds between them), it will be interesting to see how that goes.
Will phones go the same way as PCs in terms of having a separation between the compute bit and the input device? It s quite possible to have a compute device in the phone form factor inside a secure pocket which talks via Bluetooth to another device with a display and speakers. Then you could change your phone between a phone-size display and a tablet sized display easily and when using your phone a thief would not be able to easily steal the compute bit (which has passwords etc). Could the watch part of the phone (strapped to your wrist and difficult to steal) be the active part and have a tablet size device as an external display? There are already announcements of smart watches with up to 1GB of RAM (same as the Samsung Galaxy S3), that s enough for a lot of phone functionality.
The Rabbit R1 [8] and the Humane AI Pin [9] have some interesting possibilities for AI speech interfaces. Could that take over some of the current phone use? It seems that visually impaired people have been doing badly in the trend towards touch screen phones so an option of a voice interface phone would be a good option for them. As an aside I hope some people are working on AI stuff for FOSS devices.
Laptop Like
One interesting PC variant I just discovered is the Higole 2 Pro portable battery operated Windows PC with 5.5 touch screen [10]. It looks too thick to fit in the same pockets as current phones but is still very portable. The version with built in battery is $AU423 which is in the usual price range for low end laptops and tablets. I don t think this is the future of computing, but it is something that is usable today while we wait for foldable devices to take over.
The recent release of the Apple Vision Pro [11] has driven interest in 3D and head mounted computers. I think this could be a useful peripheral for a laptop or phone but it won t be part of a primary computing environment. In 2011 I wrote about the possibility of using augmented reality technology for providing a desktop computing environment [12]. I wonder how a Vision Pro would work for that on a train or passenger jet.
Another interesting thing that s on offer is a laptop with 7 touch screen beside the keyboard [13]. It seems that someone just looked at what parts are available cheaply in China (due to being parts of more popular devices) and what could fit together. I think a keyboard should be central to the monitor for serious typing, but there may be useful corner cases where typing isn t that common and a touch-screen display is of use. Developing a range of strange hardware and then seeing which ones get adopted is a good thing and an advantage of Ali Express and Temu.
Useful Hardware for Developing These Things
I recently bought a second hand Thinkpad X1 Yoga Gen3 for $359 which has stylus support [14], and it s generally a great little laptop in every other way. There s a common failure case of that model where touch support for fingers breaks but the stylus still works which allows it to be used for testing touch screen functionality while making it cheap.
The PineTime is a nice smart watch from Pine64 which is designed to be open [15]. I am quite happy with it but haven t done much with it yet (apart from wearing it every day and getting alerts etc from Android). At $50 when delivered to Australia it s significantly more expensive than most smart watches with similar features but still a lot cheaper than the high end ones. Also the Raspberry Pi Watch [16] is interesting too.
The PinePhonePro is an OK phone made to open standards but it s hardware isn t as good as Android phones released in the same year [17]. I ve got some useful stuff done on mine, but the battery life is a major issue and the screen resolution is low. The Librem 5 phone from Purism has a better hardware design for security with switches to disable functionality [18], but it s even slower than the PinePhonePro. These are good devices for test and development but not ones that many people would be excited to use every day.
Wwan hardware (for accessing the phone network) in M.2 form factor can be obtained for free if you have access to old/broken laptops. Such devices start at about $35 if you want to buy one. USB GPS devices also start at about $35 so probably not worth getting if you can get a wwan device that does GPS as well.
What We Must Do
Debian appears to have some voice input software in the pocketsphinx package but no documentation on how it s to be used. This would be a good thing to document, I spent 15 mins looking at it and couldn t get it going.
To take advantage of the hardware features in phones we need software support and we ideally don t want free software to lag too far behind proprietary software which IMHO means the typical Android setup for phones/tablets.
Support for changing screen resolution is already there as is support for touch screens. Support for adapting the GUI to changed screen size is something that needs to be done even today s hardware of connecting a small laptop to an external monitor doesn t have the ideal functionality for changing the UI. There also seem to be some limitations in touch screen support with multiple screens, I haven t investigated this properly yet, it definitely doesn t work in an expected manner in Ubuntu 22.04 and I haven t yet tested the combinations on Debian/Unstable.
ML is becoming a big thing and it has some interesting use cases for small devices where a smart device can compensate for limited input options. There s a lot of work that needs to be done in this area and we are limited by the fact that we can t just rip off the work of other people for use as training data in the way that corporations do.
Security is more important for devices that are at high risk of theft. The vast majority of free software installations are way behind Android in terms of security and we need to address that. I have some ideas for improvement but there is always a conflict between security and usability and while Android is usable for it s own special apps it s not usable in a I want to run applications that use any files from any other applicationsin any way I want sense. My post about Sandboxing Phone apps is relevant for people who are interested in this [19]. We also need to extend security models to cope with things like ok google type functionality which has the potential to be a bug and the emerging class of LLM based attacks.
I will write more posts about these thing.
Please write comments mentioning FOSS hardware and software projects that address these issues and also documentation for such things.
/usr-move, by Helmut Grohne
Much of the work was spent on handling interaction with time time64 transition
and sending patches for mitigating fallout. The set of packages relevant to
debootstrap is mostly converted and the patches for glibc and base-files
have been refined due to feedback from the upload to Ubuntu noble. Beyond this,
he sent patches for all remaining packages that cannot move their files with
dh-sequence-movetousr and packages using dpkg-divert in ways that dumat
would not recognize.
Upcoming improvements to Salsa CI, by Santiago Ruano Rinc n
Last month, Santiago Ruano Rinc n started the work on integrating sbuild into
the Salsa CI pipeline. Initially, Santiago used sbuild with the unshare
chroot mode. However, after discussion with josch, jochensp and helmut (thanks
to them!), it turns out that the unshare mode is not the most suitable for the
pipeline, since the level of isolation it provides is not needed, and some test
suites would fail (eg: krb5). Additionally, one of the requirements of the
build job is the use of ccache, since it is needed by some C/C++ large projects
to reduce the compilation time. In the preliminary work with unshare last
month, it was not possible to make ccache to work.
Finally, Santiago changed the chroot mode, and now has a couple of POC (cf:
1
and 2)
that rely on the schroot and sudo, respectively. And the good news is that
ccache is successfully used by sbuild with schroot!
The image here comes from an example of building grep. At the end of the
build, ccache -s shows the statistics of the cache that it used, and so a
little more than half of the calls of that job were cacheable. The most
important pieces are in place to finish the integration of sbuild into the
pipeline.
Other than that, Santiago also reviewed the very useful
merge request !346,
made by IOhannes zm lnig to autodetect the release from debian/changelog. As
agreed with IOhannes, Santiago is preparing a merge request to include the
release autodetection use case in the very own Salsa CI s CI.
Packaging simplemonitor, by Carles Pina i Estany
Carles started using simplemonitor in
2017, opened a
WNPP bug in 2022
and started packaging simplemonitor dependencies in October 2023. After
packaging five direct and indirect dependencies, Carles finally uploaded
simplemonitor to unstable in February.
During the packaging of simplemonitor, Carles reported
a few issues
to upstream. Some of these were to make the simplemonitor package build and run
tests reproducibly. A reproducibility issue was reprotest overriding the
timezone, which broke simplemonitor s tests. There have been discussions on
resolving this upstream in simplemonitor and
in reprotest,
too.
Carles also started upgrading or improving some of simplemonitor s dependencies.
Miscellaneous contributions
Stefano Rivera spent some time doing admin on debian.social infrastructure.
Including dealing with a spike of abuse on the Jitsi server.
Stefano started to prepare a new release of dh-python, including cleaning out
a lot of old Python 2.x related code. Thanks to Niels Thykier (outside
Freexian) for spear-heading this work.
DebConf 24 planning is beginning. Stefano discussed venues and finances with
the local team and remotely supported a site-visit by Nattie (outside
Freexian).
Also in the DebConf 24 context, Santiago took part in discussions and
preparations related to the Content Team.
A JIT bug was
reported against pypy3 in Debian Bookworm. Stefano bisected the upstream
history to find the patch (it was already resolved upstream) and released an
update to pypy3 in bookworm.
Enrico participated in /usr-merge discussions with Helmut.
Colin dug into a cluster of celery build failures and tracked the hardest bit
down to a Python 3.12 regression, now
fixed in unstable. celery should be back in testing once the 64-bit time_t
migration is out of the way.
Thorsten Alteholz uploaded a new upstream version of cpdb-libs. Unfortunately
upstream changed the naming of their release tags, so updating the watch file
was a bit demanding. Anyway this version 2.0 is a huge step towards
introduction of the new Common Print Dialog Backends.
Helmut send patches for 48 cross build failures.
Helmut changed debvm to use mkfs.ext4 instead of genext2fs.
Helmut sent a
debci MR
for improving collector robustness.
In preparation for DebConf 25, Santiago worked on the Brest Bid.
I had reported this to Ansible a year ago (2023-02-23), but it seems this is considered expected behavior, so I am posting it here now.
TL;DR
Don't ever consume any data you got from an inventory if there is a chance somebody untrusted touched it.
Inventory plugins
Inventory plugins allow Ansible to pull inventory data from a variety of sources.
The most common ones are probably the ones fetching instances from clouds like Amazon EC2
and Hetzner Cloud or the ones talking to tools like Foreman.
For Ansible to function, an inventory needs to tell Ansible how to connect to a host (so e.g. a network address) and which groups the host belongs to (if any).
But it can also set any arbitrary variable for that host, which is often used to provide additional information about it.
These can be tags in EC2, parameters in Foreman, and other arbitrary data someone thought would be good to attach to that object.
And this is where things are getting interesting.
Somebody could add a comment to a host and that comment would be visible to you when you use the inventory with that host.
And if that comment contains a Jinja expression, it might get executed.
And if that Jinja expression is using the pipe lookup, it might get executed in your shell.
Let that sink in for a moment, and then we'll look at an example.
Example inventory plugin
defines the plugin name as evgeni.inventoryrce.inventory
accepts any config that ends with evgeni.yml (we'll need that to trigger the use of this inventory later)
adds an imaginary host exploit.example.com with local connection type and something_funny variable to the inventory
In reality this would be talking to some API, iterating over hosts known to it, fetching their data, etc.
But the structure of the code would be very similar.
The crucial part is that if we have a string with a Jinja expression, we can set it as a variable for a host.
Using the example inventory plugin
Now we install the collection containing this inventory plugin,
or rather write the code to ~/.ansible/collections/ansible_collections/evgeni/inventoryrce/plugins/inventory/inventory.py
(or wherever your Ansible loads its collections from).
And we create a configuration file.
As there is nothing to configure, it can be empty and only needs to have the right filename: touch inventory.evgeni.yml is all you need.
If we now call ansible-inventory, we'll see our host and our variable present:
(ANSIBLE_INVENTORY_ENABLED=evgeni.inventoryrce.inventory is required to allow the use of our inventory plugin, as it's not in the default list.)
So far, nothing dangerous has happened.
The inventory got generated, the host is present, the funny variable is set, but it's still only a string.
Executing a playbook, interpreting Jinja
To execute the code we'd need to use the variable in a context where Jinja is used.
This could be a template where you actually use this variable, like a report where you print the comment the creator has added to a VM.
Or a debug task where you dump all variables of a host to analyze what's set.
Let's use that!
-hosts:alltasks:-name:Display all variables/facts known for a hostansible.builtin.debug:var:hostvars[inventory_hostname]
This playbook looks totally innocent: run against all hosts and dump their hostvars using debug.
No mention of our funny variable.
Yet, when we execute it, we see:
% ANSIBLE_INVENTORY_ENABLED=evgeni.inventoryrce.inventoryansible-playbook-iinventory.evgeni.ymltest.yml
PLAY [all] ************************************************************************************************TASK [Gathering Facts] ************************************************************************************ok: [exploit.example.com]TASK [Display all variables/facts known for a host] *******************************************************ok: [exploit.example.com] => "hostvars[inventory_hostname]": "ansible_all_ipv4_addresses": [ "192.168.122.1" ], "something_funny": ""PLAY RECAP *************************************************************************************************exploit.example.com : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
We got all variables dumped, that was expected, but now something_funny is an empty string?
Jinja got executed, and the expression was lookup("pipe", "touch /tmp/hacked" ) and touch does not return anything.
But it did create the file!
% ls-alh/tmp/hacked-rw-r--r--. 1 evgeni evgeni 0 Mar 10 17:18 /tmp/hacked
We just "hacked" the Ansible control node (aka: your laptop),
as that's where lookup is executed.
It could also have used the url lookup to send the contents of your Ansible vault to some internet host.
Or connect to some VPN-secured system that should not be reachable from EC2/Hetzner/ .
Why is this possible?
This happens because set_variable(entity, varname, value) doesn't mark the values as unsafe and Ansible processes everything with Jinja in it.
In this very specific example, a possible fix would be to explicitly wrap the string in AnsibleUnsafeText by using wrap_var:
fork,vinhostvars.items():try:self.inventory.set_variable(host_name,k,v)exceptValueErrorase:self.display.warning("Could not set host info hostvar for %s, skipping %s: %s"%(host,k,to_text(e)))
(foreman.py)
And honestly, I can totally understand that.
When developing an inventory, you do not expect to handle insecure input data.
You also expect the API to handle the data in a secure way by default.
But set_variable doesn't allow you to tag data as "safe" or "unsafe" easily and data in Ansible defaults to "safe".
Can something similar happen in other parts of Ansible?
It certainly happened in the past that Jinja was abused in Ansible: CVE-2016-9587, CVE-2017-7466, CVE-2017-7481
But even if we only look at inventories, add_host(host) can be abused in a similar way:
% ANSIBLE_INVENTORY_ENABLED=evgeni.inventoryrce.inventoryansible-playbook-iinventory.evgeni.ymltest.yml
PLAY [all] ************************************************************************************************TASK [Gathering Facts] ************************************************************************************fatal: [lol lookup("pipe", "touch /tmp/hacked-host" ) ]: UNREACHABLE! => "changed": false, "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname lol: No address associated with hostname", "unreachable": true PLAY RECAP ************************************************************************************************lol lookup("pipe", "touch /tmp/hacked-host" ) : ok=0 changed=0 unreachable=1 failed=0 skipped=0 rescued=0 ignored=0% ls-alh/tmp/hacked-host
-rw-r--r--. 1 evgeni evgeni 0 Mar 13 08:44 /tmp/hacked-host
Affected versions
I've tried this on Ansible (core) 2.13.13 and 2.16.4.
I'd totally expect older versions to be affected too, but I have not verified that.
FTP master
This month I accepted 242 and rejected 42 packages. The overall number of packages that got accepted was 251.
This was just a short month and the weather outside was not really motivating. I hope it will be better in March.
Debian LTS
This was my hundred-sixteenth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.
During my allocated time I uploaded:
[DLA 3739-1] libjwt security update for one CVE to fix some constant-time-for-execution-issue
[#1064551] Bookworm PU bug for libjwt; upload after approval
[DLA 3741-1] engrampa security update for one CVE to fix a path traversal issue with CPIO archives
[#1060186] Bookworm PU-bug for libde265 was flagged for acceptance
[#1056935] Bullseye PU-bug for libde265 was flagged for acceptance
I also started to work on qtbase-opensource-src (an update is needed for ELTS, so an LTS update seems to be appropriate as well, especially as there are postponed CVE).
Debian ELTS
This month was the sixty-seventth ELTS month. During my allocated time I uploaded:
[ELA-1047-1]bind9 security update for one CVE to fix an stack exhaustion issue in Jessie and Stretch
The upload of bind9 was a bit exciting, but all occuring issues with the new upload workflow could be quickly fixed by Helmut and the packages finally reached their destination. I wonder why it is always me who stumbles upon special cases? This month I also worked on the Jessie and Stretch updates for exim4. I also started to work on an update for qtbase-opensource-src in Stretch (and LTS and other releases as well).
Debian Printing
This month I uploaded new upstream versions of:
This work is generously funded by Freexian!
Debian Matomo
I started a new team debian-matomo-maintainers. Within this team all matomo related packages should be handled. PHP PEAR or PECL packages shall be still maintained in their corresponding teams.
This month I uploaded:
Welcome to the February 2024 report from the Reproducible Builds project! In our reports, we try to outline what we have been up to over the past month as well as mentioning some of the important things happening in software supply-chain security.
Reproducible Builds at FOSDEM 2024
Core Reproducible Builds developer Holger Levsen presented at the main track at FOSDEM on Saturday 3rd February this year in Brussels, Belgium. However, that wasn t the only talk related to Reproducible Builds.
However, please see our comprehensive FOSDEM 2024 news post for the full details and links.
Three new reproducibility-related academic papers
A total of three separate scholarly papers related to Reproducible Builds have appeared this month:
Signing in Four Public Software Package Registries: Quantity, Quality, and Influencing Factors by Taylor R. Schorlemmer, Kelechi G. Kalu, Luke Chigges, Kyung Myung Ko, Eman Abdul-Muhd, Abu Ishgair, Saurabh Bagchi, Santiago Torres-Arias and James C. Davis (Purdue University, Indiana, USA) is concerned with the problem that:
Package maintainers can guarantee package authorship through software signing [but] it is unclear how common this practice is, and whether the resulting signatures are created properly. Prior work has provided raw data on signing practices, but measured single platforms, did not consider time, and did not provide insight on factors that may influence signing. We lack a comprehensive, multi-platform understanding of signing adoption and relevant factors. This study addresses this gap. (arXiv, full PDF)
[The] principle of reusability [ ] makes it harder to reproduce projects build environments, even though reproducibility of build environments is essential for collaboration, maintenance and component lifetime. In this work, we argue that functional package managers provide the tooling to make build environments reproducible in space and time, and we produce a preliminary evaluation to justify this claim.
This paper thus proposes an approach to automatically identify configuration options causing non-reproducibility of builds. It begins by building a set of builds in order to detect non-reproducible ones through binary comparison. We then develop automated techniques that combine statistical learning with symbolic reasoning to analyze over 20,000 configuration options. Our methods are designed to both detect options causing non-reproducibility, and remedy non-reproducible configurations, two tasks that are challenging and costly to perform manually. (HAL Portal, full PDF)
Distribution work
In Debian this month, 5 reviews of Debian packages were added, 22 were updated and 8 were removed this month adding to Debian s knowledge about identified issues. A number of issue types were updated as well. [ ][ ][ ][ ] In addition, Roland Clobus posted his 23rd update of the status of reproducible ISO images on our mailing list. In particular, Roland helpfully summarised that all major desktops build reproducibly with bullseye, bookworm, trixie and sid provided they are built for a second time within the same DAK run (i.e. [within] 6 hours) and that there will likely be further work at a MiniDebCamp in Hamburg. Furthermore, Roland also responded in-depth to a query about a previous report Fedora developer Zbigniew J drzejewski-Szmek announced a work-in-progress script called fedora-repro-build that attempts to reproduce an existing package within a koji build environment. Although the projects README file lists a number of fields will always or almost always vary and there is a non-zero list of other known issues, this is an excellent first step towards full Fedora reproducibility.
Jelle van der Waa introduced a new linter rule for Arch Linux packages in order to detect cache files leftover by the Sphinx documentation generator which are unreproducible by nature and should not be packaged. At the time of writing, 7 packages in the Arch repository are affected by this.
Elsewhere, Bernhard M. Wiedemann posted another monthly update for his work elsewhere in openSUSE.
diffoscopediffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes such as uploading versions 256, 257 and 258 to Debian and made the following additional changes:
Use a deterministic name instead of trusting gpg s use-embedded-filenames. Many thanks to Daniel Kahn Gillmor dkg@debian.org for reporting this issue and providing feedback. [][]
Don t error-out with a traceback if we encounter struct.unpack-related errors when parsing Python .pyc files. (#1064973). []
Don t try and compare rdb_expected_diff on non-GNU systems as %p formatting can vary, especially with respect to MacOS. []
Expand an older changelog entry with a CVE reference. []
Make test_zip black clean. []
In addition, James Addison contributed a patch to parse the headers from the diff(1) correctly [][] thanks! And lastly, Vagrant Cascadian pushed updates in GNU Guix for diffoscope to version 255, 256, and 258, and updated trydiffoscope to 67.0.6.
reprotestreprotest is our tool for building the same source code twice in different environments and then checking the binaries produced by each build for any differences. This month, Vagrant Cascadian made a number of changes, including:
Create a (working) proof of concept for enabling a specific number of CPUs. [][]
Consistently use 398 days for time variation rather than choosing randomly and update README.rst to match. [][]
Support a new --vary=build_path.path option. [][][][]
Website updates
There were made a number of improvements to our website this month, including:
Chris Lamb:
Improve the relative sizing of headers. []
Re-order and punch up the introduction and documentation on the SOURCE_DATE_EPOCH page. []
Update SOURCE_DATE_EPOCH documentation re. datetime.datetime.fromtimestamp. Thanks, James Addison. []
Reproducibility testing framework
The Reproducible Builds project operates a comprehensive testing framework (available at tests.reproducible-builds.org) in order to check packages and other artifacts for reproducibility. In February, a number of changes were made by Holger Levsen:
Grant Jan-Benedict Glaw shell access to the Jenkins node. []
Enable debugging for NetBSD reproducibility testing. []
Use /usr/bin/du --apparent-size in the Jenkins shell monitor. []
Revert reproducible nodes: mark osuosl2 as down . []
Thanks again to Codethink, for they have doubled the RAM on our arm64 nodes. []
Only set /proc/$pid/oom_score_adj to -1000 if it has not already been done. []
Add the opemwrt-target-tegra and jtx task to the list of zombie jobs. [][]
Vagrant Cascadian also made the following changes:
Overhaul the handling of OpenSSH configuration files after updating from Debian bookworm. [][][]
Add two new armhf architecture build nodes, virt32z and virt64z, and insert them into the Munin monitoring. [][] [][]
In addition, Alexander Couzens updated the OpenWrt configuration in order to replace the tegra target with mpc85xx [], Jan-Benedict Glaw updated the NetBSD build script to use a separate $TMPDIR to mitigate out of space issues on a tmpfs-backed /tmp [] and Zheng Junjie added a link to the GNU Guix tests [].
Lastly, node maintenance was performed by Holger Levsen [][][][][][] and Vagrant Cascadian [][][][].
Upstream patches
The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:
Thrilled to share that a new version of prrd arrived at
CRAN yesterday in a first
update in two and a half years. prrd facilitates
the parallel running [of] reverse dependency [checks] when
preparing R packages. It is used extensively for releases I make of Rcpp, RcppArmadillo,
RcppEigen,
BH, and
others.
The key idea of prrd is simple,
and described in some more detail on its webpage and
its GitHub repo.
Reverse dependency checks are an important part of package development
that is easily done in a (serial) loop. But these checks are also
generally embarassingly parallel as there is no or little
interdependency between them (besides maybe shared build depedencies).
See the (dated) screenshot (running six parallel workers, arranged in a
split byobu session).
This release, the first since 2021, brings a number of enhancments.
In particular, the summary function is now improved in several ways. Josh also put in a nice PR
that generalizes some setup defaults and values.
The release is summarised in the NEWS entry:
Changes in prrd
version 0.0.6 (2024-03-06)
The summary function has received several enhancements:
Extended summary is only running when failures are seen.
The summariseQueue function now displays an
anticipated completion time and remaining duration.
The use of optional package foghorn has
been refined, and refactored, when running summaries.
The dequeueJobs.r scripts can receive a date
argument, the date can be parse via anydate if anytime ins present.
The enqueeJobs.r now considers skipped package when
running 'addfailed' while ensuring selecting packages are still on
CRAN.
The CI setup has been updated (twice),
Enqueing and dequing functions and scripts now support relative
directories, updated documentation (#18 by Joshua
Ulrich).
Tightening the feedback loop Link to heading One thing we notice ever so often is that although Phosh s source code is publicly available and upcoming changes are open for review the feedback loop between changes being made to the development branch and users noticing the change can still be quiet long.
This can be problematic as we ideally want to catch a regression or broken use case triggered by a change on the development branch (aka main) before the general availability of a new version.
Last month, I had a trip to Malaysia and Thailand. I stayed for six days in each of the countries. The selection of these countries was due to both of them granting visa-free entry to Indian tourists for some time window. This post covers the Malaysia part and Thailand part will be covered in the next post. If you want to travel to any of these countries in the visa-free time period, I have written all the questions asked during immigration and at airports during this trip here which might be of help.
I mostly stayed in Kuala Lumpur and went to places around it. Although before the trip, I planned to visit Ipoh and Cameron Highlands too, but could not cover it during the trip. I found planning a trip to Malaysia a little difficult. The country is divided into two main islands - Peninsular Malaysia and Borneo. Then there are more islands - Langkawi, Penang island, Perhentian and Redang Islands. Reaching those islands seemed a little difficult to plan and I wish to visit more places in my next Malaysia trip.
My first day hostel was booked in Chinatown part of Kuala Lumpur, near Pasar Seni LRT station. As soon as I checked-in and entered my room, I met another Indian named Fletcher, and after that we accompanied each other in the trip. That day, we went to Muzium Negara and Little India. I realized that if you know the right places to buy what you want, Malaysia could be quite cheap. Malaysian currency is Malaysian Ringgit (MYR). 1 MYR is equal to 18 INR. For 2 MYR, you can get a good masala tea in Little India and it costs like 4-5 MYR for a masala dosa. The vegetarian food has good availability in Kuala Lumpur, thanks to the Tamil community. I also tried Mee Goreng, which was vegetarian, and I found it fine in terms of taste. When I checked about Mee Goreng on Wikipedia, I found out that it is unique to Indian immigrants in Malaysia (and neighboring countries) but you don t get it in India!
Mee Goreng, a dish made of noodles in Malaysia.
For the next day, Fletcher had planned a trip to Genting Highlands and pre booked everything. I also planned to join him but when we went to KL Sentral to take the bus, his bus tickets were sold out. I could take a bus at a different time, but decided to visit some other place for the day and cover Genting Highlands later. At the ticket counter, I met a family from Delhi and they wanted to go to Genting Highlands but due to not getting bus tickets for that day, they decided to buy a ticket for the next day and instead planned for Batu Caves that day. I joined them and went to Batu Caves.
After returning from Batu Caves, we went our separate ways. I went back and took rest at my hostel and later went to Petronas Towers at night. Petronas Towers is the icon of Kuala Lumpur. Having a photo there was a must. I was at Petronas Towers at around 9 PM. Around that time, Fletcher came back from Genting Highlands and we planned to meet at KL Sentral to head for dinner.
Me at Petronas Towers.
We went back to the same place as the day before where I had Mee Goreng. This time we had dosa and a masala tea. Their masala tea from the last day was tasty and that s why I was looking for them in the first place. We also met a Malaysian family having Indian ancestry dining there and had a nice conversation. Then we went to a place to eat roti canai in Pasar Seni market. Roti canai is a popular non-vegetarian dish in Malaysia but I took the vegetarian version.
Photo with Malaysians.
The next day, we went to Berjaya Time Square shopping place which sells pretty cheap items for daily use and souveniers too. However, I bought souveniers from Petaling Street, which is in Chinatown. At night, we explored Bukit Bintang, which is the heart of Kuala Lumpur and is famous for its nightlife.
After that, Fletcher went to Bangkok and I was in Malaysia for two more days. Next day, I went to Genting Highlands and took the cable car, which had awesome views. I came back to Kuala Lumpur by the night. The remaining day I just roamed around in Bukit Bintang. Then I took a flight for Bangkok on 7th Feb, which I will cover in the next post.
In Malaysia, I met so many people from different countries - apart from people from Indian subcontinent, I met Syrians, Indonesians (Malaysia seems to be a popular destination for Indonesian tourists) and Burmese people. Meeting people from other cultures is an integral part of travel for me.
My expenses for Food + Accommodation + Travel added to 10,000 INR for a week in Malaysia, while flight costs were: 13,000 INR (Delhi to Kuala Lumpur) + 10,000 INR (Kuala Lumpur to Bangkok) + 12,000 INR (Bangkok to Delhi).
For OpenStreetMap users, good news is Kuala Lumpur is fairly well-mapped on OpenStreetMap.
Tips
I bought local SIM from a shop at KL Sentral station complex which had news in their name (I forgot the exact name and there are two shops having news in their name) and it was the cheapest option I could find. The SIM was 10 MYR for 5 GB data for a week. If you want to make calls too, then you need to spend extra 5 MYR.
7-Eleven and KK Mart convenience stores are everywhere in the city and they are open all the time (24 hours a day). If you are a vegetarian, you can at least get some bread and cheese from there to eat.
A lot of people know English (and many - Indians, Pakistanis, Nepalis - know Hindi) in Kuala Lumpur, so I had no language problems most of the time.
For shopping on budget, you can go to Petaling Street, Berjaya Time Square or Bukit Bintang. In particular, there is a shop named I Love KL Gifts in Bukit Bintang which had very good prices. just near the metro/monorail stattion. Check out location of the shop on OpenStreetMap.
We are thrilled to share that RcppEigen has
now upgraded to Eigen release
3.4.0! The new release 0.3.4.0.0 arrived on CRAN earlier today, and has been
shipped to Debian as well. Eigen is a C++ template library
for linear algebra: matrices, vectors, numerical solvers, and related
algorithms.
This update has been in the works for a full two and a half years! It
all started with a PR #102 by Yixuan bringing the package-local
changes for R integration forward to usptream release 3.4.0. We opened
issue
#103 to steer possible changes from reverse-dependency checking
through. Lo and behold, this just stalled because a few substantial
changes were needed and not coming. But after a long wait, and like a
bolt out of a perfectly blue sky, Andrew revived it in January
with a reverse depends run of his own along with a set of PRs. That was
the push that was needed, and I steered it along with a number of
reverse dependency checks, and occassional emails to maintainers. We
managed to bring it down to only three packages having a hickup, and all
three had received PRs thanks to Andrew and even merged them.
So the plan became to release today following a final fourteen day
window. And CRAN was convinced
by our arguments that we followed due process. So there it is! Big big
thanks to all who helped it along, especially Yixuan and Andrew but also Mikael who updated another patch
set he had prepared for the previous release series.
The complete NEWS file entry follows.
Changes in
RcppEigen version 0.3.4.0.0 (2024-02-28)
The Eigen version has been upgrade to release 3.4.0
(Yixuan)
Extensive reverse-dependency checks ensure only three out of over
400 packages at CRAN are affected; PRs and patches helped other
packages
The long-running branch also contains substantial contributions
from Mikael Jagan (for the lme4 interface) and Andrew Johnson (revdep
PRs)
Update 28.02.2024 19:45 CET: There is now a blog entry at https://blog.opencollective.com/open-collective-official-statement-ocf-dissolution/ trying to discern the legal entities in the Open Collective ecosystem and recommending potential ways forward.
Gee, there is nothing on their blog yet, but I just [28.02.2023 00:07 CET] received this email from Mike Strode, Program Officer at the Open Collective Foundation:
Dear Daniel Lange,
It is with a heavy heart that I'm writing today to inform you that the
Board of Directors of the Open Collective Foundation (OCF) has made the
difficult decision to dissolve OCF, effective December 31, 2024.
We are proud of the work we have been able to do together. We have been
honored to build community with you and the hundreds of other collectives
hosted at the Open Collective Foundation.
What you need to know:
We are beginning a staged dissolution process that will allow our over
600 collectives the time to close or transition their work. Dissolving OCF
will take many months, and involves settling all liabilities while spending
down all funds in a legally compliant manner.
Our priority is to support our collectives in navigating this change. We
want to provide collectives the longest possible runway to wind down or
transition their operations while we focus on the many legal and financial
tasks associated with dissolving a nonprofit.
March 15 is the last day to accept donations. You will have until September 30
to work with us to develop and implement a plan to spend down the money
in your fund. Key dates are included at the bottom of this email.
We know this is going to be difficult, and we will do everything we can to
ease the transition for you.
How we will support collectives:
It remains our fiduciary responsibility to safeguard each collective's
charitable assets and ensure funds are used solely for specified charitable
purposes.
We will be providing assistance and support to you, whether you choose to
spend out and close down your collective or continue your work through
another 501(c)(3) organization or fiscal sponsor.
Unfortunately, we had to say goodbye to several of our colleagues today as
we pare down our core staff to reduce costs. I will be staying on staff to
support collectives through this transition, along with Wayne Kleppe, our
Finance Administrator.
What led to this decision:
From day one, OCF was committed to experimentation and innovation. We were
dedicated to finding new ways to open up the nonprofit space, making it
easier for people to raise and access funding so they can do good in their
communities.
OCF was created by Open Collective Inc. (OCI), a company formed in 2015
with the goal of "enabling groups to quickly set up a collective, raise
funds and manage them transparently." Soon after being founded by OCI, OCF
went through a period of rapid growth. We responded to increased demand
arising from the COVID-19 pandemic without taking the time to establish the
appropriate systems and infrastructure to sustain that growth.
Unfortunately, over the past year, we have learned that Open Collective
Foundation's business model is not sustainable with the number of complex
services we have offered and the fees we pay to the Open Collective Inc.
tech platform.
In late 2023, we made the decision to pause accepting new collectives in
order to create space for us to address the issues. Unfortunately, it
became clear that it would not be financially feasible to make the
necessary corrections, and we determined that OCF is not viable.
What's next:
We know this news will raise questions for many of our collectives. We will
be making space for questions and reactions in the coming weeks.
In the meantime, we have developed this FAQ which we will keep updated as
more questions come in.
What you need to do next:
Review the FAQ
Discuss your options within your collective. Your options are:
spend down and close out your collective
spend down and transfer your collective to another fiscal sponsor,
or
transfer your collective and funds to another charitable
organization.
Reply-all to this email with any questions, requests, or to set up a
time to talk. Please make sure generalinquiries@opencollective.org is
copied on your email.
Dates to know:
Last day to accept funds/receive donations: March 15, 2024
Last day collectives can have employees: June 30, 2024
Last day to spend or transfer funds: September 30, 2024
In Care & Accompaniment,
Mike Strode Program Officer Open Collective FoundationOur mailing address has changed! We are now located at 440 N. Barranca
Avenue #3717, Covina, CA 91723, USA
I first became aware of Ray Dalio when either he or his publisher
plastered advertisements for The Principles all over the San
Francisco 4th and King Caltrain station. If I recall correctly, there
were also constant radio commercials; it was a whole thing in 2017. My
brain is very good at tuning out advertisements, so my only thought at the
time was "some business guy wrote a self-help book." I think I vaguely
assumed he was a CEO of some traditional business, since that's usually
who writes heavily marketed books like this. I did not connect him with
hedge funds or Bridgewater, which I have a bad habit of confusing with
Blackwater.
The Principles turns out to be more of a laundered cult manual than
a self-help book. And therein lies a story.
Rob Copeland is currently with The New York Times, but for many
years he was the hedge fund reporter for The Wall Street Journal.
He covered, among other things, Bridgewater Associates, the enormous hedge
fund founded by Ray Dalio. The Fund is a biography of Ray Dalio
and a history of Bridgewater from its founding as a vehicle for Dalio's
advising business until 2022 when Dalio, after multiple false starts and
title shuffles, finally retired from running the company. (Maybe. Based
on the history recounted here, it wouldn't surprise me if he was back at
the helm by the time you read this.)
It is one of the wildest, creepiest, and most abusive business histories
that I have ever read.
It's probably worth mentioning, as Copeland does explicitly, that Ray
Dalio and Bridgewater hate this book and claim it's a pack of lies.
Copeland includes some of their denials (and many non-denials that sound
as good as confirmations to me) in footnotes that I found increasingly
amusing.
A lawyer for Dalio said he "treated all employees equally, giving
people at all levels the same respect and extending them the same
perks."
Uh-huh.
Anyway, I personally know nothing about Bridgewater other than what I
learned here and the occasional mention in Matt Levine's newsletter (which
is where I got the recommendation for this book). I have no independent
information whether anything Copeland describes here is true, but Copeland
provides the typical extensive list of notes and sourcing one expects in a
book like this, and Levine's comments indicated it's generally consistent
with Bridgewater's industry reputation. I think this book is true, but
since the clear implication is that the world's largest hedge fund was
primarily a deranged cult whose employees mostly spied on and rated each
other rather than doing any real investment work, I also have questions,
not all of which Copeland answers to my satisfaction. But more on that
later.
The center of this book are the Principles. These were an ever-changing
list of rules and maxims for how people should conduct themselves within
Bridgewater. Per Copeland, although Dalio later published a book by that
name, the version of the Principles that made it into the book was
sanitized and significantly edited down from the version used inside the
company. Dalio was constantly adding new ones and sometimes changing
them, but the common theme was radical, confrontational "honesty": never
being silent about problems, confronting people directly about anything
that they did wrong, and telling people all of their faults so that they
could "know themselves better."
If this sounds like textbook abusive behavior, you have the right idea.
This part Dalio admits to openly, describing Bridgewater as a firm that
isn't for everyone but that achieves great results because of this
culture. But the uncomfortably confrontational vibes are only the tip of
the iceberg of dysfunction. Here are just a few of the ways this played
out according to Copeland:
Dalio decided that everyone's opinions should be weighted by the
accuracy of their previous decisions, to create a "meritocracy," and
therefore hired people to build a social credit system in which people
could use an app to constantly rate all of their co-workers. This
almost immediately devolved into out-group bullying worthy of a high
school, with employees hurriedly down-rating and ostracizing any
co-worker that Dalio down-rated.
When an early version of the system uncovered two employees at
Bridgewater with more credibility than Dalio, Dalio had the system
rigged to ensure that he always had the highest ratings and was not
affected by other people's ratings.
Dalio became so obsessed with the principle of confronting problems
that he created a centralized log of problems at Bridgewater and
required employees find and report a quota of ten or twenty new issues
every week or have their bonus docked. He would then regularly pick
some issue out of the issue log, no matter how petty, and treat it
like a referendum on the worth of the person responsible for the
issue.
Dalio's favorite way of dealing with a problem was to put someone on
trial. This involved extensive investigations followed by a meeting
where Dalio would berate the person and harshly catalog their flaws,
often reducing them to tears or panic attacks, while smugly insisting
that having an emotional reaction to criticism was a personality flaw.
These meetings were then filmed and added to a library available to
all Bridgewater employees, often edited to remove Dalio's personal
abuse and to make the emotional reaction of the target look
disproportionate. The ones Dalio liked the best were shown to all new
employees as part of their training in the Principles.
One of the best ways to gain institutional power in Bridgewater was to
become sycophantically obsessed with the Principles and to be an eager
participant in Dalio's trials. The highest levels of Bridgewater
featured constant jockeying for power, often by trying to catch rivals
in violations of the Principles so that they would be put on trial.
In one of the common and all-too-disturbing connections between Wall
Street finance and the United States' dysfunctional government, James
Comey (yes, that James
Comey) ran internal security for Bridgewater for three years, meaning
that he was the one who pulled evidence from surveillance cameras for
Dalio to use to confront employees during his trials.
In case the cult vibes weren't strong enough already, Bridgewater
developed its own idiosyncratic language worthy of Scientology. The
trials were called "probings," firing someone was called "sorting" them,
and rating them was called "dotting," among many other
Bridgewater-specific terms. Needless to say, no one ever probed Dalio
himself. You will also be completely unsurprised to learn that Copeland
documents instances of sexual harassment and discrimination at
Bridgewater, including some by Dalio himself, although that seems to be a
relatively small part of the overall dysfunction. Dalio was happy to
publicly humiliate anyone regardless of gender.
If you're like me, at this point you're probably wondering how Bridgewater
continued operating for so long in this environment. (Per Copeland, since
Dalio's retirement in 2022, Bridgewater has drastically reduced the
cult-like behaviors, deleted its archive of probings, and de-emphasized the
Principles.) It was not actually a religious cult; it was a hedge fund
that has to provide investment services to huge, sophisticated clients,
and by all accounts it's a very successful one. Why did this bizarre
nightmare of a workplace not interfere with Bridgewater's business?
This, I think, is the weakest part of this book. Copeland makes a few
gestures at answering this question, but none of them are very satisfying.
First, it's clear from Copeland's account that almost none of the
employees of Bridgewater had any control over Bridgewater's investments.
Nearly everyone was working on other parts of the business (sales,
investor relations) or on cult-related obsessions. Investment decisions
(largely incorporated into algorithms) were made by a tiny core of people
and often by Dalio himself. Bridgewater also appears to not trade
frequently, unlike some other hedge funds, meaning that they probably stay
clear of the more labor-intensive high-frequency parts of the business.
Second, Bridgewater took off as a hedge fund just before the hedge fund
boom in the 1990s. It transformed from Dalio's personal consulting
business and investment newsletter to a hedge fund in 1990 (with an
earlier investment from the World Bank in 1987), and the 1990s were a very
good decade for hedge funds. Bridgewater, in part due to Dalio's
connections and effective marketing via his newsletter, became one of the
largest hedge funds in the world, which gave it a sort of institutional
momentum. No one was questioned for putting money into Bridgewater even
in years when it did poorly compared to its rivals.
Third, Dalio used the tried and true method of getting free publicity from
the financial press: constantly predict an upcoming downturn, and
aggressively take credit whenever you were right. From nearly the start
of his career, Dalio predicted economic downturns year after year.
Bridgewater did very well in the 2000 to 2003 downturn, and again during
the 2008 financial crisis. Dalio aggressively takes credit for predicting
both of those downturns and positioning Bridgewater correctly going into
them. This is correct; what he avoids mentioning is that he also
predicted downturns in every other year, the majority of which never
happened.
These points together create a bit of an answer, but they don't feel like
the whole picture and Copeland doesn't connect the pieces. It seems
possible that Dalio may simply be good at investing; he reads obsessively
and clearly enjoys thinking about markets, and being an abusive cult
leader doesn't take up all of his time. It's also true that to some
extent hedge funds are semi-free money machines, in that once you have a
sufficient quantity of money and political connections you gain access to
investment opportunities and mechanisms that are very likely to make money
and that the typical investor simply cannot access. Dalio is clearly good
at making personal connections, and invested a lot of effort into forming
close ties with tricky clients such as pools of Chinese money.
Perhaps the most compelling explanation isn't mentioned directly in this
book but instead comes from Matt Levine. Bridgewater touts its
algorithmic trading over humans making individual trades, and there is
some reason to believe that consistently applying an algorithm without
regard to human emotion is a solid trading strategy in at least some
investment areas. Levine has asked in his newsletter, tongue firmly in
cheek, whether the bizarre cult-like behavior and constant infighting is a
strategy to distract all the humans and keep them from messing with the
algorithm and thus making bad decisions.
Copeland leaves this question unsettled. Instead, one comes away from
this book with a clear vision of the most dysfunctional workplace I have
ever heard of, and an endless litany of bizarre events each more
astonishing than the last. If you like watching train wrecks, this is the
book for you. The only drawback is that, unlike other entries in this
genre such as Bad Blood or
Billion Dollar Loser, Bridgewater is a
wildly successful company, so you don't get the schadenfreude of seeing a
house of cards collapse. You do, however, get a helpful mental model to
apply to the next person who tries to talk to you about "radical honesty"
and "idea meritocracy."
The flaw in this book is that the existence of an organization like
Bridgewater is pointing to systematic flaws in how our society works,
which Copeland is largely uninterested in interrogating. "How could this
have happened?" is a rather large question to leave unanswered. The sheer
outrageousness of Dalio's behavior also gets a bit tiring by the end of
the book, when you've seen the patterns and are hearing about the fourth
variation. But this is still an astonishing book, and a worthy entry in
the genre of capitalism disasters.
Rating: 7 out of 10
This is my third update on writing a language server for Debian packaging files, which
aims at providing a better developer experience for Debian packagers.
Lets go over what have done since the last report.
Semantic token support
I have added support for what the Language Server Protocol (LSP) call semantic tokens. These
are used to provide the editor insights into tokens of interest for users. Allegedly,
this is what editors would use for syntax highlighting as well.
Unfortunately, eglot (emacs) does not support semantic tokens, so I was not able to test
this. There is a 3-year old PR for supporting with the last update being ~3 month basically
saying "Please sign the Copyright Assignment". I pinged the GitHub issue in the hopes it will
get unstuck.
For good measure, I also checked if I could try it via neovim. Before installing, I read
the neovim docs, which helpfully listed the features supported. Sadly, I did not spot
semantic tokens among those and parked from there.
That was a bit of a bummer, but I left the feature in for now. If you have an LSP capable
editor that supports semantic tokens, let me know how it works for you! :)
Spellchecking
Finally, I implemented something Otto was missing! :)
This stared with Paul Wise reminding me that there were Python binding for the hunspell
spellchecker. This enabled me to get started with a quick prototype that spellchecked the
Description fields in debian/control. I also added spellchecking of comments while
I was add it.
The spellchecker runs with the standard en_US dictionary from hunspell-en-us, which
does not have a lot of technical terms in it. Much less any of the Debian specific slang.
I spend considerable time providing a "built-in" wordlist for technical and Debian specific
slang to overcome this. I also made a "wordlist" for known Debian people that the
spellchecker did not recognise. Said wordlist is fairly short as a proof of concept, and
I fully expect it to be community maintained if the language server becomes a success.
My second problem was performance. As I had suspected that spellchecking was not the
fastest thing in the world. Therefore, I added a very small language server for the
debian/changelog, which only supports spellchecking the textual part. Even for a
small changelog of a 1000 lines, the spellchecking takes about 5 seconds, which
confirmed my suspicion. With every change you do, the existing diagnostics hangs around
for 5 seconds before being updated. Notably, in emacs, it seems that diagnostics
gets translated into an absolute character offset, so all diagnostics after the change
gets misplaced for every character you type.
Now, there is little I could do to speed up hunspell. But I can, as always, cheat.
The way diagnostics work in the LSP is that the server listens to a set of notifications
like "document opened" or "document changed". In a response to that, the LSP can start
its diagnostics scanning of the document and eventually publish all the diagnostics to
the editor. The spec is quite clear that the server owns the diagnostics and the
diagnostics are sent as a "notification" (that is, fire-and-forgot). Accordingly, there
is nothing that prevents the server from publishing diagnostics multiple times for a
single trigger. The only requirement is that the server publishes the accumulated
diagnostics in every publish (that is, no delta updating).
Leveraging this, I had the language server for debian/changelog scan the document and
publish once for approximately every 25 typos (diagnostics) spotted. This means you quickly
get your first result and that clears the obsolete diagnostics. Thereafter, you get
frequent updates to the remainder of the document if you do not perform any further changes.
That is, up to a predefined max of typos, so we do not overload the client for longer
changelogs. If you do any changes, it resets and starts over.
The only bit missing was dealing with concurrency. By default, a pygls language server
is single threaded. It is not great if the language server hangs for 5 seconds everytime
you type anything. Fortunately, pygls has builtin support for asyncio and threaded
handlers. For now, I did an async handler that await after each line and setup some
manual detection to stop an obsolete diagnostics run. This means the server will fairly
quickly abandon an obsolete run.
Also, as a side-effect of working on the spellchecking, I fixed multiple typos in the
changelog of debputy. :)
Follow up on the "What next?" from my previous update
In my previous update, I mentioned I had to finish up my python-debian changes to
support getting the location of a token in a deb822 file. That was done, the MR
is now filed, and is pending review. Hopefully, it will be merged and uploaded soon. :)
I also submitted my proposal for a different way of handling relationship substvars to
debian-devel. So far, it seems to have received only positive feedback. I hope it stays
that way and we will have this feature soon. Guillem proposed to move some of this into
dpkg, which might delay my plans a bit. However, it might be for the better in the
long run, so I will wait a bit to see what happens on that front. :)
As noted above, I managed to add debian/changelog as a support format for the
language server. Even if it only does spellchecking and trimming of trailing newlines
on save, it technically is a new format and therefore cross that item off my list. :D
Unfortunately, I did not manage to write a linter variant that does not involve using
an LSP-capable editor. So that is still pending. Instead, I submitted an MR against
elpa-dpkg-dev-el to have it recognize all the fields that the debian/control
LSP knows about at this time to offset the lack of semantic token support in
eglot.
From here...
My sprinting on this topic will soon come to an end, so I have to a bit more careful
now with what tasks I open!
I think I will narrow my focus to providing a batch linting interface. Ideally, with
an auto-fix for some of the more mechanical issues, where this is little doubt about
the answer.
Additionally, I think the spellchecking will need a bit more maturing. My current
code still trips on naming patterns that are "clearly" verbatim or code references
like things written in CamelCase or SCREAMING_SNAKE_CASE. That gets annoying
really quickly. It also trips on a lot of commands like dpkg-gencontrol, but that
is harder to fix since it could have been a real word. I think those will have to be
fixed people using quotes around the commands. Maybe the most popular ones will end
up in the wordlist.
Beyond that, I will play it by ear if I have any time left. :)
A welcome sign at Bangkok's Suvarnabhumi airport.
Bus from Suvarnabhumi Airport to Jomtien Beach in Pattaya.
Road near Jomtien beach in Pattaya
Photo of a songthaew in Pattaya. There are shared songthaews which run along Jomtien Second road and takes 10 bath to anywhere on the route.
Jomtien Beach in Pattaya.
A welcome sign at Pattaya Floating market.
This Korean Vegetasty noodles pack was yummy and was available at many 7-Eleven stores.
Wat Arun temple stamps your hand upon entry
Wat Arun temple
Khao San Road
A food stall at Khao San Road
Chao Phraya Express Boat
Banana with yellow flesh
Fruits at a stall in Bangkok
Trimmed pineapples from Thailand.
Corn in Bangkok.
A board showing coffee menu at a 7-Eleven store along with rates in Pattaya.
In this section of 7-Eleven, you can buy a premix coffee and mix it with hot water provided at the store to prepare.
Red wine being served in Air India
On Mastodon, the
Last week we held our promised miniDebConf in Santa Fe City, Santa Fe province,
Argentina just across the river from Paran , where I have spent almost six
beautiful months I will never forget.
teamwork, or: why I love the Debian Perl Group:
elbrus has introduced a (very untypical) 
Closing arguments in the trial between various people and 
Like each month, have a look at the work funded by 
The image here comes from an example of building 
I had reported this to Ansible a year ago (2023-02-23), but it seems this is considered expected behavior, so I am posting it here now.
TL;DR
Don't ever consume any data you got from an inventory if there is a chance somebody untrusted touched it.
Inventory plugins
Thrilled to share that a new version of 
The key idea of 
Tightening the feedback loop Link to heading One thing we notice ever so often is that although Phosh s source code is publicly available and upcoming changes are open for review the feedback loop between changes being made to the development branch and users noticing the change can still be quiet long.
This can be problematic as we ideally want to catch a regression or broken use case triggered by a change on the development branch (aka main) before the general availability of a new version.
Mee Goreng, a dish made of noodles in Malaysia.
Me at Petronas Towers.
Photo with Malaysians.
Update 28.02.2024 19:45 CET: There is now a blog entry at