My work computer runs Debian 11 bullseye (the current stable release) in a mechanical 500GB disk, and I was provided with a new SDD disk but its size was 480 GB. So I had to shrink my partitions before copying the data to the new disk. It turned out to be a bit difficult because my main partition was encrypted.
I write here how I did, maybe there are other simpler ways but I couldn t find them.
References:
sudo lvremove /dev/larjona-pc-vg/swap_1
sudo pvresize --setphysicalvolumesize 380G /dev/mapper/cryptdisk
sudo pvchange -x y /dev/mapper/cryptdisk
sudo lvcreate -L 4G -n swap_1 larjona-pc-vg
sudo mkswap -L swap_1 /dev/larjona-pc-vg/swap_1
Display information about the physical volume in order to shrink it:
sudo pvs -v --segments --units s /dev/mapper/cryptdisk
sudo cryptsetup -b 838860800 resize cryptdisk
sudo cryptsetup status cryptdisk
sudo vgchange -a n vgroup
sudo vgchange -an
sudo cryptsetup luksClose cryptdisk
Then reduced the sda3 partition with the KDE partition manager (it took a while), and copy it to the new disk.
Turned off the computer and unplugged the old disk. Started the computer with the Debian 11 Live USB again, UEFI boot.
Now, to make my system boot:
sudo cryptsetup luksOpen /dev/sda3 crypdisk
sudo vgscan --mknodes
sudo vgchange -ay
sudo mount /dev/mapper/larjona--pc--vg-root /mnt
sudo mount /dev/sda2 /mnt/boot
sudo mount /dev/sda1 /mnt/boot/efi
mount --rbind /sys /media/linux/sys
mount -t efivarfs none /sys/firmware/efi/efivars
for i in /dev /dev/pts /proc /run; do sudo mount -B $i /mnt$i; done
sudo chroot /mnt
Then edited /mnt/etc/crypttab to reflect the name of the new encrypted partition, edited /mnt/etc/fstab to paste the UUIDs of the new partitions.grub-install and reinstalled the kernels as noted in the reference, rebooted and logged in my Plasma desktop
(Well, the actual process was not so smooth but after several tries and errors and searching for help I managed to get the needed commands to make my system boot from the new disk).
Needs to be able to create two copies always. Can get stuck in irreversible read-only mode if only one copy can be made.Even as of now, RAID-1 and RAID-10 has this note:
The simple redundancy RAID levels utilize different mirrors in a way that does not achieve the maximum performance. The logic can be improved so the reads will spread over the mirrors evenly or based on device congestion.Granted, that's not a stability concern anymore, just performance. A reviewer of a draft of this article actually claimed that BTRFS only reads from one of the drives, which hopefully is inaccurate, but goes to show how confusing all this is. There are other warnings in the Debian wiki that are quite scary. Even the legendary Arch wiki has a warning on top of their BTRFS page, still. Even if those issues are now fixed, it can be hard to tell when they were fixed. There is a changelog by feature but it explicitly warns that it doesn't know "which kernel version it is considered mature enough for production use", so it's also useless for this. It would have been much better if BTRFS was released into the world only when those bugs were being completely fixed. Or that, at least, features were announced when they were stable, not just "we merged to mainline, good luck". Even now, we get mixed messages even in the official BTRFS documentation which says "The Btrfs code base is stable" (main page) while at the same time clearly stating unstable parts in the status page (currently RAID56). There are much harsher BTRFS critics than me out there so I will stop here, but let's just say that I feel a little uncomfortable trusting server data with full RAID arrays to BTRFS. But surely, for a workstation, things should just work smoothly... Right? Well, let's see the snags I hit.
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 931,5G 0 disk
sda1 8:1 0 200M 0 part /boot/efi
sda2 8:2 0 1G 0 part /boot
sda3 8:3 0 7,8G 0 part
fedora_swap 253:5 0 7.8G 0 crypt [SWAP]
sda4 8:4 0 922,5G 0 part
fedora_crypt 253:4 0 922,5G 0 crypt /
(This might not entirely be accurate: I rebuilt this from the Debian
side of things.)
This is pretty straightforward, except for the swap partition:
normally, I just treat swap like any other logical volume and create
it in a logical volume. This is now just speculation, but I bet it was
setup this way because "swap" support was only added in BTRFS 5.0.
I fully expect BTRFS experts to yell at me now because this is an old
setup and BTRFS is so much better now, but that's exactly the point
here. That setup is not that old (2018? old? really?), and migrating
to a new partition scheme isn't exactly practical right now. But let's
move on to more practical considerations.
/dev/nvme0n1 and nvme1n1/dev/md1vg_tbbuild05 (multiple PVs can be added to a single VG which is
why there is that abstraction)NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259:0 0 1.7T 0 disk
nvme0n1p1 259:1 0 8M 0 part
nvme0n1p2 259:2 0 512M 0 part
md0 9:0 0 511M 0 raid1 /boot
nvme0n1p3 259:3 0 1.7T 0 part
md1 9:1 0 1.7T 0 raid1
crypt_dev_md1 253:0 0 1.7T 0 crypt
vg_tbbuild05-root 253:1 0 30G 0 lvm /
vg_tbbuild05-swap 253:2 0 125.7G 0 lvm [SWAP]
vg_tbbuild05-srv 253:3 0 1.5T 0 lvm /srv
nvme0n1p4 259:4 0 1M 0 part
I stripped the other nvme1n1 disk because it's basically the same.
Now, if we look at my BTRFS-enabled workstation, which doesn't even
have RAID, we have the following:
/dev/sda with, again, /dev/sda4 being where BTRFS livesfedora_crypt, which is, confusingly, kind of like a
volume group. it's where everything lives. i think.home, root, /, etc. those are actually the things
that get mounted. you'd think you'd mount a filesystem, but no, you
mount a subvolume. that is backwards.lsblk:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 931,5G 0 disk
sda1 8:1 0 200M 0 part /boot/efi
sda2 8:2 0 1G 0 part /boot
sda3 8:3 0 7,8G 0 part [SWAP]
sda4 8:4 0 922,5G 0 part
fedora_crypt 253:4 0 922,5G 0 crypt /srv
Notice how we don't see all the BTRFS volumes here? Maybe it's because
I'm mounting this from the Debian side, but lsblk definitely gets
confused here. I frankly don't quite understand what's going on, even
after repeatedly looking around the rather dismal
documentation. But that's what I gather from the following
commands:
root@curie:/home/anarcat# btrfs filesystem show
Label: 'fedora' uuid: 5abb9def-c725-44ef-a45e-d72657803f37
Total devices 1 FS bytes used 883.29GiB
devid 1 size 922.47GiB used 916.47GiB path /dev/mapper/fedora_crypt
root@curie:/home/anarcat# btrfs subvolume list /srv
ID 257 gen 108092 top level 5 path home
ID 258 gen 108094 top level 5 path root
ID 263 gen 108020 top level 258 path root/var/lib/machines
I only got to that point through trial and error. Notice how I use an
existing mountpoint to list the related subvolumes. If I try to use
the filesystem path, the one that's listed in filesystem show, I
fail:
root@curie:/home/anarcat# btrfs subvolume list /dev/mapper/fedora_crypt
ERROR: not a btrfs filesystem: /dev/mapper/fedora_crypt
ERROR: can't access '/dev/mapper/fedora_crypt'
Maybe I just need to use the label? Nope:
root@curie:/home/anarcat# btrfs subvolume list fedora
ERROR: cannot access 'fedora': No such file or directory
ERROR: can't access 'fedora'
This is really confusing. I don't even know if I understand this
right, and I've been staring at this all afternoon. Hopefully, the
lazyweb will correct me eventually.
(As an aside, why are they called "subvolumes"? If something is a
"sub" of "something else", that "something else" must exist
right? But no, BTRFS doesn't have "volumes", it only has
"subvolumes". Go figure. Presumably the filesystem still holds "files"
though, at least empirically it doesn't seem like it lost anything so
far.
In any case, at least I can refer to this section in the future, the
next time I fumble around the btrfs commandline, as I surely will. I
will possibly even update this section as I get better at it, or based
on my reader's judicious feedback.
/etc/fstab,
on the Debian side of things:
UUID=5abb9def-c725-44ef-a45e-d72657803f37 /srv btrfs defaults 0 2
This thankfully ignores all the subvolume nonsense because it relies
on the UUID. mount tells me that's actually the "root" (? /?)
subvolume:
root@curie:/home/anarcat# mount grep /srv
/dev/mapper/fedora_crypt on /srv type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)
Let's see if I can mount the other volumes I have on there. Remember
that subvolume list showed I had home, root, and
var/lib/machines. Let's try root:
mount -o subvol=root /dev/mapper/fedora_crypt /mnt
Interestingly, root is not the same as /, it's a different
subvolume! It seems to be the Fedora root (/, really) filesystem. No
idea what is happening here. I also have a home subvolume, let's
mount it too, for good measure:
mount -o subvol=home /dev/mapper/fedora_crypt /mnt/home
Note that lsblk doesn't notice those two new mountpoints, and that's
normal: it only lists block devices and subvolumes (rather
inconveniently, I'd say) do not show up as devices:
root@curie:/home/anarcat# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 931,5G 0 disk
sda1 8:1 0 200M 0 part
sda2 8:2 0 1G 0 part
sda3 8:3 0 7,8G 0 part
sda4 8:4 0 922,5G 0 part
fedora_crypt 253:4 0 922,5G 0 crypt /srv
This is really, really confusing. Maybe I did something wrong in the
setup. Maybe it's because I'm mounting it from outside Fedora. Either
way, it just doesn't feel right.
root@curie:/home/anarcat# df -h /srv /mnt /mnt/home
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/fedora_crypt 923G 886G 31G 97% /srv
/dev/mapper/fedora_crypt 923G 886G 31G 97% /mnt
/dev/mapper/fedora_crypt 923G 886G 31G 97% /mnt/home
(Notice, in passing, that it looks like the same filesystem is mounted
in different places. In that sense, you'd expect /srv and /mnt
(and /mnt/home?!) to be exactly the same, but no: they are entirely
different directory structures, which I will not call "filesystems"
here because everyone's head will explode in sparks of confusion.)
Yes, disk space is shared (that's the Size and Avail columns,
makes sense). But nope, no cookie for you: they all have the same
Used columns, so you need to actually walk the entire filesystem to
figure out what each disk takes.
(For future reference, that's basically:
root@curie:/home/anarcat# time du -schx /mnt/home /mnt /srv
124M /mnt/home
7.5G /mnt
875G /srv
883G total
real 2m49.080s
user 0m3.664s
sys 0m19.013s
And yes, that was painfully slow.)
ZFS actually has some oddities in that regard, but at least it tells
me how much disk each volume (and snapshot) takes:
root@tubman:~# time df -t zfs -h
Filesystem Size Used Avail Use% Mounted on
rpool/ROOT/debian 3.5T 1.4G 3.5T 1% /
rpool/var/tmp 3.5T 384K 3.5T 1% /var/tmp
rpool/var/spool 3.5T 256K 3.5T 1% /var/spool
rpool/var/log 3.5T 2.0G 3.5T 1% /var/log
rpool/home/root 3.5T 2.2G 3.5T 1% /root
rpool/home 3.5T 256K 3.5T 1% /home
rpool/srv 3.5T 80G 3.5T 3% /srv
rpool/var/cache 3.5T 114M 3.5T 1% /var/cache
bpool/BOOT/debian 571M 90M 481M 16% /boot
real 0m0.003s
user 0m0.002s
sys 0m0.000s
That's 56360 times faster, by the way.
But yes, that's not fair: those in the know will know there's a
different command to do what df does with BTRFS filesystems, the
btrfs filesystem usage command:
root@curie:/home/anarcat# time btrfs filesystem usage /srv
Overall:
Device size: 922.47GiB
Device allocated: 916.47GiB
Device unallocated: 6.00GiB
Device missing: 0.00B
Used: 884.97GiB
Free (estimated): 30.84GiB (min: 27.84GiB)
Free (statfs, df): 30.84GiB
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 512.00MiB (used: 0.00B)
Multiple profiles: no
Data,single: Size:906.45GiB, Used:881.61GiB (97.26%)
/dev/mapper/fedora_crypt 906.45GiB
Metadata,DUP: Size:5.00GiB, Used:1.68GiB (33.58%)
/dev/mapper/fedora_crypt 10.00GiB
System,DUP: Size:8.00MiB, Used:128.00KiB (1.56%)
/dev/mapper/fedora_crypt 16.00MiB
Unallocated:
/dev/mapper/fedora_crypt 6.00GiB
real 0m0,004s
user 0m0,000s
sys 0m0,004s
Almost as fast as ZFS's df! Good job. But wait. That doesn't actually
tell me usage per subvolume. Notice it's filesystem usage, not
subvolume usage, which unhelpfully refuses to exist. That command
only shows that one "filesystem" internal statistics that are pretty
opaque.. You can also appreciate that it's wasting 6GB of
"unallocated" disk space there: I probably did something Very Wrong
and should be punished by Hacker News. I also wonder why it has 1.68GB
of "metadata" used...
At this point, I just really want to throw that thing out of the
window and restart from scratch. I don't really feel like learning the
BTRFS internals, as they seem oblique and completely bizarre to me. It
feels a little like the state of PHP now: it's actually pretty solid,
but built upon so many layers of cruft that I still feel it corrupts
my brain every time I have to deal with it (needle or haystack first?
anyone?)...
Welcome to the March 2022 report from the Reproducible Builds project! In our monthly reports we outline the most important things that we have been up to over the past month.
The in-toto project was accepted as an incubating project within the Cloud Native Computing Foundation (CNCF). in-toto is a framework that protects the software supply chain by collecting and verifying relevant data. It does so by enabling libraries to collect information about software supply chain actions and then allowing software users and/or project managers to publish policies about software supply chain practices that can be verified before deploying or installing software. CNCF foundations hosts a number of critical components of the global technology infrastructure under the auspices of the Linux Foundation. (View full announcement.)
Herv Boutemy posted to our mailing list with an announcement that the Java Reproducible Central has hit the milestone of 500 fully reproduced builds of upstream projects . Indeed, at the time of writing, according to the nightly rebuild results, 530 releases were found to be fully reproducible, with 100% reproducible artifacts.
GitBOM is relatively new project to enable build tools trace every source file that is incorporated into build artifacts. As an experiment and/or proof-of-concept, the GitBOM developers are rebuilding Debian to generate side-channel build metadata for versions of Debian that have already been released. This only works because Debian is (partial) reproducible, so one can be sure that that, if the case where build artifacts are identical, any metadata generated during these instrumented builds applies to the binaries that were built and released in the past. More information on their approach is available in README file in the bomsh repository.
[ ] implemented what we call package multi-versioning for C/C++ software that lacks function multi-versioning and run-time dispatch [ ]. It is another way to ensure that users do not have to trade reproducibility for performance. (full PDF)
Kit Martin posted to the FOSSA blog a post titled The Three Pillars of Reproducible Builds. Inspired by the shock of infiltrated or intentionally broken NPM packages, supply chain attacks, long-unnoticed backdoors , the post goes on to outline the high-level steps that lead to a reproducible build:
It is one thing to talk about reproducible builds and how they strengthen software supply chain security, but it s quite another to effectively configure a reproducible build. Concrete steps for specific languages are a far larger topic than can be covered in a single blog post, but today we ll be talking about some guiding principles when designing reproducible builds. [ ]The article was discussed on Hacker News.
Finally, Bernhard M. Wiedemann noticed that the GNU Helloworld project varies depending on whether it is being built during a full moon! (Reddit announcement, openSUSE bug report)
There will be an in-person Debian Reunion in Hamburg, Germany later this year, taking place from 23 30 May. Although this is a Debian event, there will be some folks from the broader Reproducible Builds community and, of course, everyone is welcome. Please see the event page on the Debian wiki for more information.
Bernhard M. Wiedemann posted to our mailing list about a meetup for Reproducible Builds folks at the openSUSE conference in Nuremberg, Germany.
It was also recently announced that DebConf22 will take place this year as an in-person conference in Prizren, Kosovo. The pre-conference meeting (or Debcamp ) will take place from 10 16 July, and the main talks, workshops, etc. will take place from 17 24 July.
SOURCE_DATE_EPOCH environment variable, both by expanding parts of the existing text [ ][ ] as well as clarifying meaning by removing text in other places [ ]. In addition, Chris Lamb added a Twitter Card to our website s metadata too [ ][ ][ ].
On our mailing list this month:
So now we have 364 source packages for which we have a patch and for which we can show that this patch does not change the build output. Do you agree that with those two properties, the advantages of the 3.0 (quilt) format are sufficient such that the change shall be implemented at least for those 364? [ ]
buildpath_in_postgres_opcodes, captures_kernel_version_via_CMAKE_SYSTEM, build_id_differences_only, etc.
python-securesystemslib package to version 0.22.0-1 and the in-toto package to 1.2.0-1.
In openSUSE, Bernhard M. Wiedemann posted his usual monthly reproducible builds status report.
207, 208 and 209 to Debian unstable, as well as made the following changes to the code itself:
--append-build-command option [ ], which was subsequently uploaded to Debian unstable by Holger Levsen.
python-ara.nbformat.chemical-structures.fiat.intel-mediasdk.libao.pcp.tevent.pcp.apr-util.liggghts.kristall.lmod.libranlip.xrt.btrfsmaintenance.
The Reproducible Builds project runs a significant testing framework at tests.reproducible-builds.org, to check packages and other artifacts for reproducibility. This month, the following changes were made:
dsa-check-running-kernel script with a packaged version. [ ]sources.lst file for our mail server as its still running Debian buster. [ ]debsecan package everywhere; it got installed accidentally via the Recommends relation. [ ]#reproducible-builds on irc.oftc.net.
rb-general@lists.reproducible-builds.org
| Publisher: | New American Library |
| Copyright: | 2016 |
| ISBN: | 0-698-18327-4 |
| Format: | Kindle |
| Pages: | 572 |
The following contributors got their Debian Developer accounts in the last two months:
Just as I did for 2020, I won't publically disclose exactly how many books I read in 2021, but they evidently provoked enough thoughts that felt it worth splitting my yearly writeup into separate posts. I will reveal, however, that I got through more books than the previous year, and, like before, I enjoyed the books I read this year even more in comparison as well.
How much of this is due to refining my own preferences over time, and how much can be ascribed to feeling less pressure to read particular books? It s impossible to say, and the question is complicated further by the fact I found many of the classics I read well worth of their entry into the dreaded canon.
But enough of the throat-clearing. In today's post I'll be looking at my favourite books filed under memoir and biography, in no particular order.
Books that just missed the cut here include: Bernard Crick's celebrated 1980 biography of George Orwell, if nothing else because it was a pleasure to read; Hilary Mantel's exhilaratingly bitter early memoir, Giving up the Ghost (2003); and Patricia Lockwood's hilarious Priestdaddy (2017). I also had a soft spot for Tim Kreider's We Learn Nothing (2012) as well, despite not knowing anything about the author in advance, likely a sign of good writing. The strangest book in this category I read was definitely Michelle Zauner's Crying in H Mart. Based on a highly-recommended 2018 essay in the New Yorker, its rich broth of genuine yearning for a departed mother made my eyebrows raise numerous times when I encountered inadvertent extra details about Zauner's relationships.
Beethoven: A Life in Nine Pieces (2020) Laura Tunbridge Whilst it might immediately present itself as a clickbait conceit, organising an overarching narrative around just nine compositions by Beethoven turns out to be an elegant way of saying something fresh about this grizzled old bear. Some of Beethoven's most famous compositions are naturally included in the nine (eg. the Eroica and the Hammerklavier piano sonata), but the book raises itself above conventional Beethoven fare when it highlights, for instance, his Septet, Op. 20, an early work that is virtually nobody's favourite Beethoven piece today. The insight here is that it was widely popular in its time, played again and again around Vienna for the rest of his life. No doubt many contemporary authors can relate to this inability to escape being artistically haunted by an earlier runaway success. The easiest way to say something interesting about Beethoven in the twenty-first century is to talk about the myth of Beethoven instead. Or, as Tunbridge implies, perhaps that should really be 'Beethoven' in leaden quotation marks, given so much about what we think we know about the man is a quasi-fictional construction. Take Anton Schindler, Beethoven's first biographer and occasional amanuensis, who destroyed and fabricated details about Beethoven's life, casting himself in a favourable light and exaggerating his influence with the composer. Only a few decades later, the idea of a 'heroic' German was to be politically useful as well; the Anglosphere often need reminding that Germany did not exist as a nation-state prior to 1871, so it should be unsurprising to us that the late nineteenth-century saw a determined attempt to create a uniquely 'German' culture ex nihilo. (And the less we say about Immortal Beloved the better, even though I treasure that film.) Nevertheless, Tunbridge cuts through Beethoven's substantial legacy using surgical precision that not only avoids feeling like it is settling a score, but it also does so in a way that is unlikely to completely alienate anyone emotionally dedicated to some already-established idea of the man to bring forth the tediously predictable sentiment that Beethoven has 'gone woke'. With Alex Ross on the cult of Wagner, it seems that books about the 'myth of X' are somewhat in vogue right now. And this pattern within classical music might fit into some broader trend of deconstruction in popular non-fiction too, especially when we consider the numerous contemporary books on the long hangover of the Civil Rights era (Robin DiAngelo's White Fragility, etc.), the multifarious ghosts of Empire (Akala's Natives, Sathnam Sanghera's Empireland, etc.) or even the 'transmogrification' of George Orwell into myth. But regardless of its place in some wider canon, A Life in Nine Pieces is beautifully printed in hardback form (worth acquiring for that very reason alone), and it is one of the rare good books about classical music that can be recommended to both the connoisseur and the layperson alike.
Sea State (2021) Tabitha Lasley In her mid-30s and jerking herself out of a terrible relationship, Tabitha Lasley left London and put all her savings into a six-month lease on a flat within a questionable neighbourhood in Aberdeen, Scotland. She left to make good on a lukewarm idea for a book about oil rigs and the kinds of men who work on them: I wanted to see what men were like with no women around, she claims. The result is Sea State, a forthright examination of the life of North Sea oil riggers, and an unsparing portrayal of loneliness, masculinity, female desire and the decline of industry in Britain. (It might almost be said that Sea State is an update of a sort to George Orwell's visit to the mines in the North of England.) As bracing as the North Sea air, Sea State spoke to me on multiple levels but I found it additionally interesting to compare and contrast with Julian Barnes' The Man with Red Coat (see below). Women writers are rarely thought to be using fiction for higher purposes: it is assumed that, unlike men, whatever women commit to paper is confessional without any hint of artfulness. Indeed, it seems to me that the reaction against the decades-old genre of autofiction only really took hold when it became the domain of millennial women. (By contrast, as a 75-year-old male writer with a firmly established reputation in the literary establishment, Julian Barnes is allowed wide latitude in what he does with his sources and his writing can be imbued with supremely confident airs as a result.) Furthermore, women are rarely allowed metaphor or exaggeration for dramatic effect, and they certainly aren t permitted to emphasise darker parts in order to explore them... hence some of the transgressive gratification of reading Sea State. Sea State is admittedly not a work of autofiction, but the sense that you are reading about an author writing a book is pleasantly unavoidable throughout. It frequently returns to the topic of oil workers who live multiple lives, and Lasley admits to living two lives herself: she may be in love but she's also on assignment, and a lot of the pleasure in this candid and remarkably accessible book lies in the way these states become slowly inseparable.
Twilight of Democracy (2020) Anne Applebaum For the uninitiated, Anne Applebaum is a staff writer for The Atlantic magazine who won a Pulitzer-prize for her 2004 book on the Soviet Gulag system. Her latest book, however, Twilight of Democracy is part memoir and part political analysis and discusses the democratic decline and the rise of right-wing populism. This, according to Applebaum, displays distinctly authoritarian tendencies, and who am I to disagree? Applebaum does this through three main case studies (Poland, the United Kingdom and the United States), but the book also touches on Hungary as well. The strongest feature of this engaging book is that Appelbaum's analysis focuses on the intellectual classes and how they provide significant justification for a descent into authoritarianism. This is always an important point to be remembered, especially as much of the folk understanding of the rise of authoritarian regimes tends to place exaggerated responsibility on the ordinary and everyday citizen: the blame placed on the working-class in the Weimar Republic or the scorn heaped upon 'white trash' of the contemporary Rust Belt, for example. Applebaum is uniquely poised to discuss these intellectuals because, well, she actually knows a lot of them personally. Or at least, she used to know them. Indeed, the narrative of the book revolves around two parties she hosted, both in the same house in northwest Poland. The first party, on 31 December 1999, was attended by friends from around the Western world, but most of the guests were Poles from the broad anti-communist alliance. They all agreed about democracy, the rule of law and the route to prosperity whilst toasting in the new millennium. (I found it amusing to realise that War and Peace also starts with a party.) But nearly two decades later, many of the attendees have ended up as supporters of the problematic 'Law and Justice' party which currently governs the country. Applebaum would now cross the road to avoid them, and they would do the same to her, let alone behave themselves at a cordial reception. The result of this autobiographical detail is that by personalising the argument, Applebaum avoids the trap of making too much of high-minded abstract argument for 'democracy', and additionally makes her book compellingly spicy too. Yet the strongest part of this book is also its weakest. By individualising the argument, it often feels that Applebaum is settling a number of personal scores. She might be very well justified in doing this, but at times it feels like the reader has walked in halfway through some personal argument and is being asked to judge who is in the right. Furthermore, Applebaum's account of contemporary British politics sometimes deviates into the cartoonish: nothing was egregiously incorrect in any of her summations, but her explanation of the Brexit referendum result didn't read as completely sound. Nevertheless, this lively and entertaining book that can be read with profit, even if you disagree with significant portions of it, and its highly-personal approach makes it a refreshing change from similar contemporary political analysis (eg. David Runciman's How Democracy Ends) which reaches for that more 'objective' line.
The Man in the Red Coat (2019) Julian Barnes As rich as the eponymous red coat that adorns his cover, Julian Barnes quasi-biography of French gynaecologist Samuel-Jean Pozzi (1846 1918) is at once illuminating, perplexing and downright hilarious. Yet even that short description is rather misleading, for this book evades classification all manner number of ways. For instance, it is unclear that, with the biographer's narrative voice so obviously manifest, it is even a biography in the useful sense of the word. After all, doesn't the implied pact between author and reader require the biographer to at least pretend that they are hiding from the reader? Perhaps this is just what happens when an author of very fine fiction turns his hand to non-fiction history, and, if so, it represents a deeper incursion into enemy territory after his 1984 metafictional Flaubert's Parrot. Indeed, upon encountering an intriguing mystery in Pozzi's life crying out for a solution, Barnes baldly turns to the reader, winks and states: These matters could, of course, be solved in a novel. Well, quite. Perhaps Barnes' broader point is that, given that's impossible for the author to completely melt into air, why not simply put down your cards and have a bit of fun whilst you're at it? If there's any biography that makes the case for a rambling and lightly polemical treatment, then it is this one. Speaking of having fun, however, two qualities you do not expect in a typical biography is simply how witty they can be, as well as it having something of the whiff of the thriller about it. A bullet might be mentioned in an early chapter, but given the name and history of Monsieur Pozzi is not widely known, one is unlikely to learn how he lived his final years until the closing chapters. (Or what happened to that turtle.) Humour is primarily incorporated into the book in two main ways: first, by explicitly citing the various wits of the day ( What is a vice? Merely a taste you don t share. etc.), but perhaps more powerful is the gentle ironies, bon mots and observations in Barnes' entirely unflappable prose style, along with the satire implicit in him writing this moreish pseudo-biography to begin with. The opening page, with its steadfast refusal to even choose where to begin, is somewhat characteristic of Barnes' method, so if you don't enjoy the first few pages then you are unlikely to like the rest. (Indeed, the whole enterprise may be something of an acquired taste. Like Campari.) For me, though, I was left wryly grinning and often couldn't wait to turn the page. Indeed, at times it reminded me of a being at a dinner party with an extremely charming guest at the very peak of his form as a wit and raconteur, delighting the party with his rambling yet well-informed discursive on his topic de jour. A significant book, and a book of significance.
Here's a description of how my monthly external backups are managed at a
technical level. I didn't realise I hadn't written this all down anywhere yet.
What
I plug in one of two (prepared) external hard drives into my headless
NAS. The NAS contains my primary data backup. A job
automatically decrypts the encrypted filesystem on the drive, mounts it and
synchronises the copy of my backup data on the drive from that on the NAS.
Whilst this is going on, the blinkstick LED on the NAS
switches to a colour to signal "in progress". When it's done, the light changes
to green to signal "done" and I can remove it. If something goes wrong, it
turns red and I get mail.
Why
I want a third-strand, off-site backup of my and my family's data in case
of a disaster in our house. For it to be useful it has to be regular, so
I needed to remove as much of the friction of performing the backup as
possible.
I use two drives alternately so that I don't have all my eggs in one basket
in the window when I bring one of them home and perform the backup.
How
As much as possible I lean on systemd and its ability to trigger actions
based on events.
systemd instantiates a corresponding
device unit, named
dev-disk-by\x2duuid-aaaaaaaa\x2daaaa\x2daaaa\x2daaaa\x2daaaaaaaaaaaa.device,
where aaaa is the UUID of a partition on the devicesystemd service which has a WantedBy relationship
on the device unit, so when the device appears, systemd starts the
backup service.Requires and
After relationships on systemd-cryptsetup@extbackup.service, a
service created by systemd's cryptsetup generator on start-up (but
slightly customised, see below). The encrypted device is therefore
unlocked.ExecStart and ExecStop. These are used to:
systemd-cryptsetup@extbackup.service unit realises it is
not required any more. It has been customised with StopWhenUnneeded=true1,
so the encrypted filesystem is closed, ready for the drive to be removed.systemd units have,
as a matter of course,
OnFailure=status-email-user@%n.service blinkstick-fail.service
Preparing a new backup disk
This is mostly just a standard dm-crypt/cryptsetup/LUKS encrypted
device, on top of a standard partition on the underlying disk, with
a normal filesystem sitting on top: Basically, the most common way
to encrypt a drive in Linux.
See places like the
cryptsetup docs for how
to set something like that up. The key things here are
WantedBy line in the backup service file.
(look in /dev/disk/by-uuid before and after inserting it and see
what was added)/etc/crypttab line with all the info needed to decrypt/etc/fstab line with all the info needed to mount
(yes, really; see "Issues" below) [Unit]
OnFailure=status-email-user@%n.service blinkstick-fail.service
Requires=systemd-cryptsetup@extbackup.service backup.mount
After=systemd-cryptsetup@extbackup.service backup.mount
[Service]
Type=oneshot
ExecStart=/usr/local/bin/blinkstick --index 1 --limit 10 --set-color 33c280
ExecStart=/bin/mount /extbackup
ExecStart=/home/jon/bin/phobos-backup-monthly
ExecStop=/bin/umount /extbackup
ExecStop=/usr/local/bin/blinkstick --index 1 --limit 10 --set-color green
[Install]
WantedBy=dev-disk-by\x2duuid-aaaaaaaa\x2daaaa\x2daaaa\x2daaaa\x2daaaaaaaaaaaa.device
The dashes in the UUID in WantedBy= need to be encoded as \x2d and then
the slashes from the path bit as dashes. Using dashes to encode
slashes is possibly the single most frustrating systemd design decision.
Issues
Sadly (as detailed in Blinkenlights, part 2) there are
2 some frustrating limitations with trying to handle the mount
(and unmount) of the filesystem in systemd-land, so instead, it's done
using the traditional mount, umount and fstab.
If you can point out any improvements to this approach, please let me know!
systemd edit
systemd-cryptsetup@extbackup.service to add the StopWhenUnneeded to
an override file and not need the rest. | Series: | Seraphina #2 |
| Publisher: | Ember |
| Copyright: | 2015 |
| ISBN: | 0-375-89659-7 |
| Format: | Kindle |
| Pages: | 458 |
The Rcpp team is pleased to announce release 1.0.7 of Rcpp which arrived at CRAN earlier today, and will be uploaded to Debian shortly. Windows and macOS builds should appear at CRAN in the next few days. This release continues with the six-months cycle started with release 1.0.5 last July. As a reminder, interim dev or rc releases will alwasys be available in the Rcpp drat repo; this cycle there were seven (!!). These rolling release tend to work just as well, and are also fully tested against all reverse-dependencies.
Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 2323 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 227 in BioConductor.
This release contains a change which Luke Tierney urged us to make a good year ago in #1081) (and which we had looked at earlier in #382). Implementing the change in a regular update proved a little tricky, and my initial branch lay dormant until I aki revived it, and finished the transition (which we then did in two PRs). The change concerns how Rcpp grows internal objects, and the new approach (thanks to the hint by Luke) closer to what R does guaranteeing linear behaviour. It turns out that we overlooked one aspect (of coping with Modules built under earlier Rcpp releases) so the initial upload to CRAN on Saturday revealed that we needed a small adjustment that we made for the final release. This version should now be more performant, and rest on a stable API. Based on the reverse depends checks by both us and CRAN (using the updated version), we expect no issues with existing code. However, it something does act up a fresh compilation of the packages using Rcpp may help.
We also made a few other minor changes in the API such as silencing a silly compiler warning, ensuring global Rcout and Rcerr objects, adding support for a new Rcpp::message() call, completing a switch to uint32_t instead of unsigned int and including the cfloat header (which relates to STRICT_R_HEADERS discussed below). Similarly, the Rcpp Attributes feature was enhanced by coping better with packages with a dot in their name and their for per-package include files, along with support for more quiet compilation if desired.
As some Rcpp users may have seen, we plan to enable STRICT_R_HEADERS by the next release (expected in January 2022). A long issue tick #1158 is laying the ground work. Maintainers of 81 packages which are affected and need a small change (such as for example switching from PI to M_PI); of these 56 have already responded. We plan to be in touch in the fall. Adding the cfloat header is one help in this transition (as the corresponding C header was pulled in when STRICT_R_HEADERS is not defined) as it ensures DBL_EPSILON and alike are defined.
Last but not least this is also the first relase in which we welcome I aki as a new member of the Rcpp Core team. Yay!
The NEWS file entries follow summarizing the nine key PRs in this release.
Thanks to my CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome underChanges in Rcpp release version 1.0.7 (2021-07-06)
- Changes in Rcpp API:
- Refactored
Rcpp_PreserveObjectandRcpp_ReleaseObjectwhich are now O(1) (Dirk and I aki in #1133 and #1135 fixing #382 and #1081).- A spuriously assigned variable was removed (Dirk in #1138 fixing #1137).
- Global
RcoutandRcerrobjects are supported via a compiler directive (I aki in #1139 fixing #928)- Add support for
Rcpp::message(Dirk in #1146 fixing #1145).- The
uint32_ttype is used throughout instead ofunsigned int(Dirk in #1153 fixing #1152).- The
cfloatheader for floating point limits is now included (Dirk in #1162 fixing #1161).- Changes in Rcpp Attributes:
- Forthcoming Changes in Rcpp API:
- Starting with Rcpp 1.0.8 anticipated in January 2022,
STRICT_R_HEADERSwill be enabled by default, see #1126.
rcpp tag at StackOverflow which also allows searching among the (currently) 2616 previous questions.
If you like this or other open-source work I do, you can sponsor me at GitHub.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
Ten years ago I began
the olduse.net exhibit,
spooling out Usenet history in real time with a 30 year delay.
My archive has reached its end, and ten years is
more than long enough to keep running something you cobbled together
overnight way back when. So, this is the end for olduse.net.
The site will continue running for another week or so, to give you time to
read the last posts. Find the very last one, if you can!
The source code used to run it, and the content of the website have
themselves been archived up for posterity at
The Internet Archive.
Sometime in 2022, a spammer will purchase the domain, but not find it to be
of much value.
The Utzoo archives that underlay it have currently sadly
been censored off the Internet
by someone. This will be unsuccessful; by now they have spread and many
copies will live on.
You can post to olduse.net, but it won't show up for at least 30 years.Actually, those posts drop right now! Here are the followups to 30-year-old Usenet posts that I've accumulated over the past decade. Mike replied in 2011 to JPM's post in 1981 on fa.arms-d "Re: CBS Reports"
A greeting from the future: I actually watched this yesterday (2011-06-10) after reading about it here.Christian Brandt replied in 2011 to schrieb phyllis's post in 1981 on the "comments" newsgroup "Re: thank you rrg"
Funny, it will be four years until you post the first subnet post i ever read and another eight years until my own first subnet post shows up.Bernard Peek replied in 2012 to mark's post in 1982 on net.sf-lovers "Re: luke - vader relationship"
Martijn Dekker replied in 2012 to henry's post in 1982 on the "test" newsgroup "Re: another boring test message" trentbuck replied in 2012 to dwl's post in 1982 on the "net.jokes" newsgroup "Re: A child hood poem" Eveline replied in 2013 to a post in 1983 on net.jokes.q "Re: A couple"i suggest that darth vader is luke skywalker's mother.You may be on to something there.
Ha!Bill Leary replied in 2015 to Darin Johnson's post in 1985 on net.games.frp "Re: frp & artwork" Frederick Smith replied in 2021 to David Hoopes's post in 1990 on trial.rec.metalworking "Re: Is this group still active?"
One of my goals for Consfigurator is to make it
capable of installing Debian to my laptop, so that I can stop booting to GRML
and manually partitioning and debootstrapping a basic system, only to then
turn to configuration management to set everything else up. My configuration
management should be able to handle the partitioning and debootstrapping, too.
The first stage was to make Consfigurator capable of debootstrapping a basic
system, chrooting into it, and applying other arbitrary configuration, such as
installing packages. That s been in place for some weeks now. It s
sophisticated enough to avoid starting up newly installed services, but I
still need to add some bind mounting.
Another significant piece is teaching Consfigurator how to partition block
devices. That s quite tricky to do in a sufficiently general way I want to
cleanly support various combinations of LUKS, LVM and regular partitions,
including populating /etc/crypttab and /etc/fstab. I have some ideas about
how to do it, but it ll probably take a few tries to get the abstractions
right.
Let s imagine that code is all in place, such that Consfigurator can be
pointed at a block device and it will install a bootable Debian system to it.
Then to install Debian to my laptop I d just need to take my laptop s disk
drive out and plug it into another system, and run Consfigurator on that
system, as root, pointed at the block device representing my laptop s disk
drive. For virtual machines, it would be easy to write code which loop-mounts
an empty disk image, and then Consfigurator could be pointed at the
loop-mounted block device, thereby making the disk image file bootable.
This is adequate for virtual machines, or small single-board computers with
tiny storage devices (not that I actually use any of those, but I want
Consfigurator to be able to make disk images for them!). But it s not much
good for my laptop. I casually referred to taking out my laptop s disk drive
and connecting it to another computer, but this would void my laptop s
warranty. And Consfigurator would not be able to update my laptop s NVRAM, as
is needed on UEFI systems.
What s wanted here is a live system which can run Consfigurator directly on
the laptop, pointed at the block device representing its physical disk drive.
Ideally this live system comes with a chroot with the root filesystem for the
new Debian install already built, so that network access is not required, and
all Consfigurator has to do is partition the drive and copy in the contents of
the chroot. The live system could be set up to automatically start doing that
upon boot, but another option is to just make Consfigurator itself available
to be used interactively. The user boots the live system, starts up Emacs,
starts up Lisp, and executes a Consfigurator deployment, supplying the block
device representing the laptop s disk drive as an argument to the deployment.
Consfigurator goes off and partitions that drive, copies in the contents of
the chroot, and executes grub-install to make the laptop bootable. This is
also much easier to debug than a live system which tries to start partitioning
upon boot. It would look something like this:
;; melete.silentflame.com is a Consfigurator host object representing the
;; laptop, including information about the partitions it should have
(deploy-these :local ...
(chroot:partitioned-and-installed
melete.silentflame.com "/srv/chroot/melete" "/dev/nvme0n1"))
Now, building live systems is a fair bit more involved than installing Debian
to a disk drive and making it bootable, it turns out. While I want
Consfigurator to be able to completely replace the Debian Installer, I decided
that it is not worth trying to reimplement the relevant parts of the Debian
Live tool suite, because I do not need to make arbitrary customisations to any
live systems. I just need to have some packages installed and some files in
place. Nevertheless, it is worth teaching Consfigurator how to invoke Debian
Live, so that the customisation of the chroot which isn t just a matter of
passing options to lb_config(1) can be done with Consfigurator. This is what
I ve ended up with in Consfigurator s source code:
(defpropspec image-built :lisp (config dir properties)
"Build an image under DIR using live-build(7), where the resulting live
system has PROPERTIES, which should contain, at a minimum, a property from
CONSFIGURATOR.PROPERTY.OS setting the Debian suite and architecture. CONFIG
is a list of arguments to pass to lb_config(1), not including the '-a' and
'-d' options, which Consfigurator will supply based on PROPERTIES.
This property runs the lb_config(1), lb_bootstrap(1), lb_chroot(1) and
lb_binary(1) commands to build or rebuild the image. Rebuilding occurs only
when changes to CONFIG or PROPERTIES mean that the image is potentially
out-of-date; e.g. if you just add some new items to PROPERTIES then in most
cases only lb_chroot(1) and lb_binary(1) will be re-run.
Note that lb_chroot(1) and lb_binary(1) both run after applying PROPERTIES,
and might undo some of their effects. For example, to configure
/etc/apt/sources.list, you will need to use CONFIG not PROPERTIES."
(:desc (declare (ignore config properties))
#?"Debian Live image built in $ dir ")
(let* (...)
;; ...
(eseqprops
;; ...
(on-change
(eseqprops
(on-change
(file:has-content ,auto/config ,(auto/config config) :mode #o755)
(file:does-not-exist ,@clean)
(%lbconfig ,dir)
(%lbbootstrap t ,dir))
(%lbbootstrap nil ,dir)
(deploys ((:chroot :into ,chroot)) ,host))
(%lbchroot ,dir)
(%lbbinary ,dir)))))
Here, %lbconfig is a property running lb_config(1), %lbbootstrap one which
runs lb_bootstrap(1), etc. Those properties all just change directory to the
right place and run the command, essentially, with a little extra code to
handle failed debootstraps and the like.
The ON-CHANGE and ESEQPROPS combinators work together to sequence the
interaction of the Debian Live suite and Consfigurator.
(try-register-data-source
:git-snapshot :name "consfig" :repo #P"src/cl/consfig/" ...)
(defproplist hybrid-live-iso-built :lisp ()
"Build a Debian Live system in /srv/live/spw.
Typically this property is not applied in a DEFHOST form, but rather run as
needed at the REPL. The reason for this is that otherwise the whole image will
get rebuilt each time a commit is made to my dotfiles repo or to my consfig."
(:desc "Sean's Debian Live system image built")
(live-build:image-built.
'("--archive-areas" "main contrib non-free" ...)
"/srv/live/spw"
(os:debian-stable "buster" :amd64)
(basic-props)
(apt:installed "whatever" "you" "want")
(git:snapshot-extracted "/etc/skel/src" "dotfiles")
(file:is-copy-of "/etc/skel/.bashrc" "/etc/skel/src/dotfiles/.bashrc")
(git:snapshot-extracted "/root/src/cl" "consfig")))
The first argument to LIVE-BUILD:IMAGE-BUILT. is additional arguments to
lb_config(1). The third argument onwards are the properties for the live
system. The cool thing is GIT:SNAPSHOT-EXTRACTED the calls to this ensure
that a copy of my Emacs configuration and my consfig end up in the live image,
ready to be used interactively to install Debian, as described above. I ll
need to add something like (chroot:host-chroot-bootstrapped
melete.silentflame.com "/srv/chroot/melete") too.
As with everything Consfigurator-related, Joey Hess s
Propellor
is the giant upon whose shoulders I m standing.
Last weekend Debian Quebec held a Bug Squashing Party to try to fix some
bugs in the upcoming Debian Bullseye.
I wasn't convinced at first, but Tassia's contagious energy and willingness to
help organise the event eventually won me over. And shockers! it was
really fun.
We fixed a couple of RC bugs, held lightning talks and had a
virtual pizza party!
My lightning talk on autopkgtests was well received and a few people decided to
migrate to sbuild and enable autopkgtests by default.
Sergio's talk on debuginfod was incredibly interesting. I'm not a C
programmer and the live demo made me understand how this service can help
making debugging C easier.
Jerome's talk on using Yubikeys to unlock LUKS encrypted drives was also very
good! It also served as a reminder that Yubico's product are much more
featureful and convenient to use than other Open Hardware/ Free Software
hardware tokens. Hopefully that will change as enterprises like Nitrokey and
Solokey mature.
This was my third BSP, crazy how time flies... With the Bullseye release closing
in, you should try to join or organise one!
There are ton of articles out there to configure mutt and all related
things to it. But I find certain configs very hard to get. So I will
write down those.
set date_format="%d/%m/%y %I:%M %p"
set index_format="%3C %Z %?X?@& ? %-22.17L %-5.50s %> %-22.20D"
set editor="nano --syntax=email -r 70 -b"
new_mail_command can be used to execute
a custom script upon new
mail, such as running notify-send. There are many standalone
mail notifier such as mailnag,
mail notification
etc. It all just felt bulky for me. So I ended up making
these.
#!/bin/sh
accounts="$(awk '/^Channel/ print $2 ' "$MBSYNCRC")"
for account in $accounts; do
acc="$(echo "$account" sed "s/.*\///")"
new=$(find "$MAILBOX/$acc/Inbox/new/" -type f -newer "$MUTT/.mailsynclastrun" 2> /dev/null)
newcount=$(echo "$new" sed '/^\s*$/d' wc -l)
if [ "$newcount" -gt "0" ]; then
for file in $new; do
# Extract subject and sender from mail.
from=$(awk '/^From: / && ++n ==1,/^\<.*\>:/' "$file" perl -CS -MEncode -ne 'print decode("MIME-Header", $_)' awk ' $1=""; if (NF>=3)$NF=""; print $0 ' sed 's/^[[:blank:]]*[\"'\''\<]*//;s/[\"'\''\>]*[[:blank:]]*$//')
subject=$(awk '/^Subject: / && ++n == 1,/^\<.*\>: / && ++i == 2' "$file" head -n 1 perl -CS -MEncode -ne 'print decode("MIME-Header", $_)' sed 's/^Subject: //' sed 's/^ [[:blank:]]*[\"'\''\<]*//;s/[\"'\''\>]*[[:blank:]]*$//' tr -d '\n')
displays="$(pgrep -a Xorg grep -wo "[0-9]*:[0-9]\+")"
for x in $displays; do
export DISPLAY=$x
notify-send -i $MUTT/mutt.png -t 5000 "$account received new message:" "$from: <i>$subject</i>"
done
done
fi
done
touch "$MUTT/.mailsynclastrun"
EOF and secret key not found error.
mod_passenger inside Apache httpd) to Puma (running standalone and Apache just being a reverse proxy). While doing so, we initially deployed Puma listening on localhost:3000 and while localhost is pretty safe, a local user could still turn out evil and talk directly to Puma, pretending to be authenticated by Apache (think Kerberos or X.509 cert auth).
Obviously, this is not optimal, so the next task was to switch Puma to listen on an UNIX socket and only allow Apache to talk to said socket.
This doesn't sound overly complicated, and indeed it wasn't. The most time/thought was spent on doing that in a way that doesn't break existing setups and still allows binding to a TCP socket for setups where users explicitly want that.
We also made a change to the SELinux policy to properly label the newly created socket and allow httpd to access it.
The whole change was carefully tested on CentOS 7 and worked like a charm. So we merged it, and it broke. Only on CentOS 8, but broken is broken, right?
This is the start of my Thanksgiving story "learn how to debug SELinux issues" ;)
From the logs of our integration test I knew the issue was Apache not being able to talk to that new socket (we archive sos reports as part of the tests, and those clearly had it in the auditd logs). But I also knew we did prepare our policy for that change, so either our preparation was not sufficient or the policy wasn't properly loaded.
The same sos report also contained the output of semanage fcontext --list which stated that all regular files called /run/foreman.sock would get the foreman_var_run_t type assigned. Wait a moment, all regular files?! A socket is not a regular file! Let's quickly make that truly all files. That clearly changed the semanage fcontext --list output, but the socket was still created as var_run_t?!
It was time to actually boot a CentOS 8 VM and try more things out. Interestingly, you actually can't add a rule for /run/something, as /run is an alias (equivalency in SELinux speak) for /var/run:
# semanage fcontext --add -t foreman_var_run_t /run/foreman.sock ValueError: File spec /run/foreman.sock conflicts with equivalency rule '/run /var/run'; Try adding '/var/run/foreman.sock' insteadI have no idea how the list output in the report got that
/run rule, but okay, let's match /var/run/foreman.sock.
Did that solve the issue? Of course not! And you knew it, as I didn't get to the juciest part of the headline yet: systemd!
We use systemd to create the socket, as it is both convenient and useful (no more clients connecting before Rails has finished booting). But why is it wrongly labeling our freshly created socket?! A quick check with touch shows that the policy is correct now, the touched file gets the right type assigned. So it must be something with systemd
A bit of poking (and good guesswork based on prior experience with a similar issue in Puppet: PUP-2169 and PUP-10548) led to the realization that a systemctl daemon-reexec after adding the file context rule "fixes" the issue. Moving the poking to Google, you quickly end up at systemd issue #9997 which is fixed in v245, but that's in no EL release yet. And indeed, the issue seems fixed on my Fedora 33 with systemd 246, but I still need it to work on CentOS 7 and 8
Well, maybe that reexec isn't that bad after all? At least the socket is now properly labeled and httpd can connect to it on CentOS 8.
Btw, no idea why the connection worked on CentOS 7, as there the socket was also wrongly labeled, but SELinux didn't deny httpd to open it.
Big shout out to lzap and ewoud for helping me with this beast!
Welcome to the September 2020 report from the Reproducible Builds project. In our monthly reports, we attempt to summarise the things that we have been up to over the past month, but if you are interested in contributing to the project, please visit our main website.
This month, the Reproducible Builds project was pleased to announce a donation from Amateur Radio Digital Communications (ARDC) in support of its goals. ARDC s contribution will propel the Reproducible Builds project s efforts in ensuring the future health, security and sustainability of our increasingly digital society. Amateur Radio Digital Communications (ARDC) is a non-profit which was formed to further research and experimentation with digital communications using radio, with a goal of advancing the state of the art of amateur radio and to educate radio operators in these techniques. You can view the full announcement as well as more information about ARDC on their website.
The Threema privacy and security-oriented messaging application announced that within the next months , their apps will become fully open source, supporting reproducible builds :
This is to say that anyone will be able to independently review Threema s security and verify that the published source code corresponds to the downloaded app.You can view the full announcement on Threema s website.
On 18th September, Bernhard M. Wiedemann gave a presentation in German titled Wie reproducible builds Software sicherer machen ( How reproducible builds make software more secure ) at the Internet Security Digital Days 2020 conference. (View video.)
On Saturday 10th October, Morten Linderud will give a talk at Arch Conf Online 2020 on The State of Reproducible Builds in the Arch Linux distribution:
The previous year has seen great progress in Arch Linux to get reproducible builds in the hands of the users and developers. In this talk we will explore the current tooling that allows users to reproduce packages, the rebuilder software that has been written to check packages and the current issues in this space.During the Reproducible Builds summit in Marrakesh, GNU Guix, NixOS and Debian were able to produce a bit-for-bit identical binary when building GNU Mes, despite using three different major versions of GCC. Since the summit, additional work resulted in a bit-for-bit identical Mes binary using
tcc and this month, a fuller update was posted by the individuals involved.
Chris Lamb uploaded a number of Debian packages to address reproducibility issues that he had previously provided patches for, including cfingerd (#831021), grap (#870573), splint (#924003) & schroot (#902804)
Last month, an issue was identified where a large number of Debian .buildinfo build certificates had been tainted on the official Debian build servers, as these environments had files underneath the /usr/local/sbin directory to prevent the execution of system services during package builds. However, this month, Aurelien Jarno and Wouter Verhelst fixed this issue in varying ways, resulting in a special policy-rcd-declarative-deny-all package.
Building on Chris Lamb s previous work on reproducible builds for Debian .ISO images, Roland Clobus announced his work in progress on making the Debian Live images reproducible. [ ]
Lucas Nussbaum performed an archive-wide rebuild of packages to test enabling the reproducible=+fixfilepath Debian build flag by default. Enabling the fixfilepath feature will likely fix reproducibility issues in an estimated 500-700 packages. The test revealed only 33 packages (out of 30,000 in the archive) that fail to build with fixfilepath. Many of those will be fixed when the default LLVM/Clang version is upgraded.
79 reviews of Debian packages were added, 23 were updated and 17 were removed this month adding to our knowledge about identified issues. Chris Lamb added and categorised a number of new issue types, including packages that captures their build path via quicktest.h and absolute build directories in documentation generated by Doxygen , etc.
Lastly, Lukas Puehringer s uploaded a new version of the in-toto to Debian which was sponsored by Holger Levsen. [ ]
159 and 160 to Debian:
pgpdump, and check that the associated binary is actually installed before attempting to run it. (#969753)guestfs cleanup failure. [ ]FALLBACK_FILE_EXTENSION_SUFFIX, otherwise we run pgpdump against all files that are recognised by file(1) as data. [ ]
This month, a number of updates to the main Reproducible Builds website and related documentation. Chris Lamb made the following changes:
jekyll-polyglot package is required [ ]. Lastly, diffoscope.org and reproducible-builds.org were transferred to Software Freedom Conservancy. Many thanks to Brett Smith from Conservancy, J r my Bobbio (lunar) and Holger Levsen for their help with transferring and to Mattia Rizzolo for initiating this.
cfn-python-lint (build failure)clutter (avoid a random ID in HTML from xsltproc)kubernetes (1-bit order in manual page)libint (merged, filesystem order)libmysofa (disable -fprofile-arcs and code coverage)libnet (merged, date)libqb (date / copyright)libsemigroups (CPU detection)nauty (CPU type detection)git2-rs (sort return ordering of readdir(3))git2-rs, pyftpdlib, python-nbclient, python-pyzmq & python-sidpy.
The Reproducible Builds project operates a Jenkins-based testing framework to power tests.reproducible-builds.org. This month, Holger Levsen made the following changes:
usrmerge. [ ]systemctl status [ ] and the number of diffoscope processes [ ].
xz compression format. [ ][ ][ ]schroot sessions after 2 days, not 3. [ ]schroot sessions. [ ]#reproducible-builds on irc.oftc.net.
rb-general@lists.reproducible-builds.org
In our monthly reports, we summarise the things that we have been up to over the past month. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced from the original free software source code to the pre-compiled binaries we install on our systems. If you re interested in contributing to the project, please visit our main website.
One hiccup we ve encountered in SecureDrop development is that not all Python wheels can be built reproducibly. We ship multiple (Python) projects in Debian packages, with Python dependencies included in those packages as wheels. In order for our Debian packages to be reproducible, we need that wheel build process to also be reproducibleParallel to this, transparencylog.com was also launched, a service that verifies the contents of URLs against a publicly recorded cryptographic log. It keeps an append-only log of the cryptographic digests of all URLs it has seen. (GitHub repo)
On 18th September, Bernhard M. Wiedemann will give a presentation in German, titled Wie reproducible builds Software sicherer machen ( How reproducible builds make software more secure ) at the Internet Security Digital Days 2020 conference.
Holger gave a talk titled Reproducing Bullseye in practice , focusing on independently verifying that the binaries distributed from ftp.debian.org are made from their claimed sources. It also served as a general update on the status of reproducible builds within Debian. The video (145 MB) and slides are available.
There were also a number of other talks that involved Reproducible Builds too. For example, the Malayalam language mini-conference had a talk titled , ? ( I want to join Debian, what should I do? ) presented by Praveen Arimbrathodiyil, the Clojure Packaging Team BoF session led by Elana Hashman, as well as Where is Salsa CI right now? that was on the topic of Salsa, the collaborative development server that Debian uses to provide the necessary tools for package maintainers, packaging teams and so on.
Jonathan Bustillos (Jathan) also gave a talk in Spanish titled Un camino verificable desde el origen hasta el binario ( A verifiable path from source to binary ). (Video, 88MB)
After many years of development work, the compiler for the Rust programming language now generates reproducible binary code. This generated some general discussion on Reddit on the topic of reproducibility in general.
Paul Spooren posted a request for comments to OpenWrt s openwrt-devel mailing list asking for clarification on when to raise the PKG_RELEASE identifier of a package. This is needed in order to successfully perform rebuilds in a reproducible builds context.
In openSUSE, Bernhard M. Wiedemann published his monthly Reproducible Builds status update.
Chris Lamb provided some comments and pointers on an upstream issue regarding the reproducibility of a Snap / SquashFS archive file. [ ]
.buildinfo build certificates have been tainted on the official Debian build servers, as these environments have files underneath the /usr/local/sbin directory [ ]. He also filed against bug for debrebuild after spotting that it can fail to download packages from snapshot.debian.org [ ].
This month, several issues were uncovered (or assisted) due to the efforts of reproducible builds.
For instance, Debian bug #968710 was filed by Simon McVittie, which describes a problem with detached debug symbol files (required to generate a traceback) that is unlikely to have been discovered without reproducible builds. In addition, Jelmer Vernooij called attention that the new Debian Janitor tool is using the property of reproducibility (as well as diffoscope when applying archive-wide changes to Debian:
New merge proposals also include a link to the diffoscope diff between a vanilla build and the build with changes. Unfortunately these can be a bit noisy for packages that are not reproducible yet, due to the difference in build environment between the two builds. [ ]56 reviews of Debian packages were added, 38 were updated and 24 were removed this month adding to our knowledge about identified issues. Specifically, Chris Lamb added and categorised the
nondeterministic_version_generated_by_python_param and the lessc_nondeterministic_keys toolchain issues. [ ][ ]
Holger Levsen sponsored Lukas Puehringer s upload of the python-securesystemslib pacage, which is a dependency of in-toto, a framework to secure the integrity of software supply chains. [ ]
Lastly, Chris Lamb further refined his merge request against the debian-installer component to allow all arguments from sources.list files (such as [check-valid-until=no]) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure and sent a ping to the team that maintains that code.
asymptote (shell/Perl date)getfem (embeds datetime and user, submitted via email)getdp (hostname and user)getdp (user)guix (disable parallelism)httpcomponents-client (Java documentation generator readdir order)kuberlr (date)lal (date and time issue, submitted via email)libmesh (host)OBS (discuss how to track old build prjconf metadata in buildinfo)openblas (disable CPU detection)openfoam-selector (date)perl (toolchain, date)python-blosc (CPU detection)python-eventlet (fails to build far in the future)rna-star (date and hostname)trilinos (date)xz/b4 (workaround CPU count influencing output, reported upstream)json-c (forwarded upstream).nmh.golang-gonum-v1-plot.chirp.pixelmed-codec.debhelper.muroar.serd.pencil2d.tpot.evolution.aflplusplus.plexus-archiver (timezone/DST issue)libjpeg-turbo.jack-audio-connection-kit.glusterfs.155, 156, 157 and 158 to Debian:
<!ENTITY> declarations inside the Document Type Definition (DTD), or when a DTD or entity references an external resource. (#212)pgpdump(1) can successfully parse some binary files, so check that the parsed output contains something sensible before accepting it. [ ]gnumeric from the Debian build-dependencies as it has been removed from the testing distribution. (#968742)fallback_recognises to prevent matching .xsb binary XML files.file(1) returns data . (#211)ppudump version does not match our file header. [ ]repr(object) output in Calling external command messages. [ ].ppu files requires ppudump version 3.2.0 or higher. [ ]setup.py that diffoscope works with Python version 3.8 [ ] and Frazer Clews applied some Pylint suggestions [ ] and removed some deprecated methods [ ].
This month, Chris Lamb updated the main Reproducible Builds website and documentation to:
SOURCE_DATE_EPOCH age. [ ]
The Reproducible Builds project operate a Jenkins-based testing framework to power tests.reproducible-builds.org. This month, Holger Levsen made the following changes:
arm64 architecture. [ ]armhf. [ ][ ][ ]ruby-jekyll-polyglot package to needed for the recently-added internationalisation and translation support on the Reproducible Builds website. [ ]jenkins.debian.net, www.reproducible-builds.org, diffoscope.org, buildinfos.debian.net, etc.). [ ][ ][ ][ ][ ]arm64 architecture anymore. [ ]If you think you know how to spread the word about reproducibility in the context of Bitcoin wallets through WalletScrutiny, your contributions are highly welcome on this PR [ ]Julien Lepiller posted to the list linking to a blog post by Tavis Ormandy titled You don t need reproducible builds. Morten Linderud (foxboron) responded with a clear rebuttal that Tavis was only considering the narrow use-case of proprietary vendors and closed-source software. He additionally noted that the criticism that reproducible builds cannot prevent against backdoors being deliberately introduced into the upstream source ( bugdoors ) are decidedly (and deliberately) outside the scope of reproducible builds to begin with. Chris Lamb included the Reproducible Builds mailing list in a wider discussion regarding a tentative proposal to include
.buildinfo files in .deb packages, adding his remarks regarding requiring a custom tool in order to determine whether generated build artifacts are identical in a reproducible context. [ ]
Jonathan Bustillos (Jathan) posted a quick email to the list requesting whether there was a list of To do tasks in Reproducible Builds.
Lastly, Chris Lamb responded at length to a query regarding the status of reproducible builds for Debian ISO or installation images. He noted that most of the technical work has been performed but there are at least four issues until they can be generally advertised as such . He pointed that the privacy-oriented Tails operation system, which is based directly on Debian, has had reproducible builds for a number of years now. [ ]
#reproducible-builds on irc.oftc.net.
rb-general@lists.reproducible-builds.org
Next.