Russ Allbery: Review: Embers of War
| Series: | Embers of War #1 |
| Publisher: | Titan Books |
| Copyright: | February 2018 |
| ISBN: | 1-78565-519-1 |
| Format: | Kindle |
| Pages: | 312 |
| Series: | Embers of War #1 |
| Publisher: | Titan Books |
| Copyright: | February 2018 |
| ISBN: | 1-78565-519-1 |
| Format: | Kindle |
| Pages: | 312 |
sbuild-qemu utilities as part of
sbuild. Notable changes are support for new architectures, and a new
sbuild-qemu-boot utility to start a VM and interact with its console
directly through the terminal.
As a reminder, these utilities leverage functionality in vmdb2,
autopkgtest, and sbuild to use QEMU VM images for building and
testing packages in strongly isolated environments and/or on foreign
architectures and/or that might break the system. For example, the
following would create an image for sid on the ppc64el architecture,
and then use that image to build a package:
# Creating an image still requires root, unfortunately $ sudo sbuild-qemu-create --arch ppc64el -o simple.img unstable http://deb.debian.org/debian $ sudo chown <user>:<group> simple.img # Building a package doesn't $ sbuild-qemu --arch ppc64el --image simple.img -d unstable [<other sbuild options>] FOO.dsc # Neither does updating the image (runs apt-get update && dist-upgrade in the VM) $ sbuild-qemu-update --arch ppc64el simple.imgSince the
sbuild-qemu utilities build on top of the autopkgtest
utilities, the created images can also be used for running autopkgtests:
$ autopkgtest <test params> -- qemu --dpkg-architecture ppc64el simple.img
sbuild-qemu-boot which boots a VM
directly to a console, where it can be used as a porterbox, or to evaluate
changes that might break the system, etc. By default, the VM is started in
read-only mode, so changes are not saved when the VM is shut down. This
can be changed with the --read-write option. With the --shared-dir
option, a directory can be shared with the host. It will be mounted at
/shared within the VM.
$ sbuild-qemu-boot --arch ppc64el --shared-dir /path/to/folder simple.imgNote that the console in the VM is
24x80 by default. This can be changed
within the VM with stty to match your host, e.g.:
$ stty rows M cols NI'm still pondering whether to automate this in
sbuild-qemu-create.
vmdb2 and autopkgtest, the sbuild-qemu
utilities now also support new architectures. Currently, amd64,
i386, arm64, armhf, and ppc64el are supported, so most of the
official architectures.
However, there's an autopkgtest issue
(#10031002)
with the host-guest communication bridge on armhf and on multi-core
arm64 (single-core seems to work fine) that currently makes images
unusable for sbuild and autopkgtest. I haven't yet figured out what
the problem seems to be, but for the time being, the images can still be
used with sbuild-qemu-boot.
Welcome to the January 2022 report from the Reproducible Builds project. In our reports, we try outline the most important things that have been happening in the past month. As ever, if you are interested in contributing to the project, please visit our Contribute page on our website.
An interesting blog post was published by Paragon Initiative Enterprises about Gossamer, a proposal for securing the PHP software supply-chain. Utilising code-signing and third-party attestations, Gossamer aims to mitigate the risks within the notorious PHP world via publishing attestations to a transparency log. Their post, titled Solving Open Source Supply Chain Security for the PHP Ecosystem goes into some detail regarding the design, scope and implementation of the system.
This month, the Linux Foundation announced SupplyChainSecurityCon, a conference focused on exploring the security threats affecting the software supply chain, sharing best practices and mitigation tactics. The conference is part of the Linux Foundation s Open Source Summit North America and will take place June 21st 24th 2022, both virtually and in Austin, Texas.
There was a significant progress made in the Debian Linux distribution this month, including:
mmdebstrap when SOURCE_DATE_EPOCH is set. As of Debian 11 Bullseye, this works for all variants, except for the variant that includes all so-called Priority: standard packages where fontconfig caches, *.pyc files and man-db index.db are unreproducible. (These issues have been addressed in Tails and live-build by removing *.pyc files, in addition to removing index.db files as well. (Whilst index.db files can be regenerated, there is no straightforward method to re-creating *.pyc files, and the resulting installation will suffer from reduced performance of Python scripts. Ideally, no removal would be necessary as all files would be created reproducibly in the first place.)
beta.tests.reproducible-builds.org is now aware/capable of building packages for Debian bookworm.
fontconfig s cache files being unreproducible saw some progress this month. Debian bug number #864082 (originally filed in June 2017) was NMU d by josch, although this resulted in a small number of minor side-effects which have already been addressed with a follow-up patch.
.buildinfo files in a post entitled, Reproducible Builds: Debian and the case of the missing version string.
kpcyrd reported on Twitter about the release of version 0.2.0 of pacman-bintrans, an experiment with binary transparency for the Arch Linux package manager, pacman. This new version is now able to query rebuilderd to check if a package was independently reproduced.
199, 200, 201 and 202 to Debian unstable (that were later backported to Debian bullseye-backports by Mattia Rizzolo), as well as made the following changes to the code itself:
--timeout=60 will mean that diffoscope will not recurse into any sub-archives after 60 seconds total execution time has elapsed. Note that this is not a fixed/strict timeout due to implementation issues. [ ][ ]odt2txt, including the one provided by the unoconv package. [ ].pyc file (and add test). [ ][ ]debian.deb822 module raises any exception on import, re-raise it as an ImportError. This should fix diffoscope on some Fedora systems. [ ].inv inventory file is labelled The remainder of this file is compressed using zlib, it might not actually be. In this case, don t traceback and simply return the original content. [ ]--timeout option due to a few misconceptions. [ ]NT_GNU_BUILD_ID has been applied messages to, for example, not duplicating the full filename in the diffoscope output. [ ]
The Reproducible Builds project runs a significant testing framework at tests.reproducible-builds.org, to check packages and other artifacts for reproducibility. This month, the following changes were made:
po4a package where appropriate, as it is needed for the Reproducible Builds website job [ ]. In addition, also run the i18n.sh and contributors.sh scripts [ ].devscripts package from bullseye-backports on Debian nodes. [ ]arm64 architecture nodes hosted by Codethink. [ ]freeplane (timestamps, file order)opensearch (timestamps, file order)cosign (date)gnome-todo (discover fix for single-CPU build failure)gstreamer-plugins-good (build fails without debuginfo)jogl2 (parallelism-related issue)libkolabxml (ASLR)monero-gui (CPU)python-pyudev (date)rekor (date)rivet (FTBFS-j1 has unreleased fix upstream)skaffold (date)dh-raku.libgtkdatabox.qcelemental.node-ramda.ncurses.python-fluids (forwarded upstream).node-istanbul.texlive-base.python-cooler.insighttoolkit5.dino-im.libxtrxll.libavif.go-for-it.clfft.last-align.gkrellm-leds.last-align.binutils-xtensa-lx106.gcc-xtensa-lx106.gcc-sh-elf.pesign.upb.paho.mqtt.c.imath.libvcflib.pkg-js-tools.vdeplug4.kget.mathgl.apulse.akonadi.recastnavigation.soundkonverter.indi.akonadi-mime.akonadi-import-wizard.akonadi-contacts.cryptominisat.hipercontracer.libksba.leatherman.segyio.darkradiant.openmm.bagel.kallisto.evolution-ews.#reproducible-builds on irc.oftc.net.
rb-general@lists.reproducible-builds.org
I first started using ledger the original plain-text accounting
software in 2017. Since then, I had been pretty happy with my
accounting routine, but grew a little annoyed by the repetitive manual work I
had to do to assign recurring transactions to the right account.
To make things easier, I had a collection of bash scripts to parse and convert
the CSV files from my bank's website1 into ledger entries. They were of
course ugly, unreadable piles of sed-grep-regex and did not let met achieve the
automation complexity I longed for.
Dissatisfied with ledger's features, I decided to migrate to hledger. Contrary
to ledger, hledger comes with a CSV parser one can use to import
and classify recurring transactions automagically.
Having a proper DSL for this makes all the difference: I can easily add new
rules and understand the old ones. In the end, I get a very consistent result,
something my old bash scripts weren't great at.
Here is what my debit.csv.rules file looks like. It is used to tell hledger
how CSV data from my debit card should be parsed:
# skip the headings line:
skip 1
# assign fields
fields , , , date, , description, , amount2-in, amount2-out
# assign account where the money comes from
# 99 to get it at the bottom of the transaction
account99 assets:checking
include matchers.rules
matchers.rules. I'm doing this since I want to be
able to use the same set of rules for debit and credit and use the cards
interchangeably.
Here's a snippet of my matchers.rules file:
# house
if Loyer/bail
account2 expenses:rent
# leisure
if PAYPAL .*STEAM GAMES
account2 expenses:leisure:videogame
if PAYPAL .*BANDCAMP
account2 expenses:leisure:music
"SOME ACCOUNT DESC","111111","EOP","2022/01/03",00002,"Loyer/bail","",521.00,"","","","",""
2022-01-03 Loyer/bail
expenses:rent 521.00
assets:checking
= expenses:rent
assets:receivable:rent *0.5
assets:checking
2022-01-03 Loyer/bail ; modified:
expenses:rent 521.00
assets:receivable:rent 260.50 ; generated-posting: = expenses:rent
assets:checking
#!/bin/bash
#
# Convert CSV to ledger using hledger
declare -a assets=("credit" "debit")
# Read the array values with space
for i in "$ assets[@] "
do
if test -f "$i.csv"
then
# convert to ledger
LEDGER=$(hledger -f "$i.csv" --rules-file rules/"$i.csv".rules print)
# add auto_postings
LEDGER=$(printf "include rules/auto_postings.ledger\n\n$LEDGER\n" hledger print -f- --auto)
# remove superfluous assets:checking lines
LEDGER=$(printf "$LEDGER" sed '/assets:checking \+; generated.\+/d')
printf "$LEDGER" > "$i.ledger"
else
printf "File $i.csv does not exist\n"
fi
done
#hledger @irc.libera.chat for the help!
Welcome to the 35th post in the ravishingly rabiant R recommendations, or R4. Today s post is about apt and R tools.
Many of us have been running RStudio off our local machines for as long as binaries have been provided. Which is by now probably a bit over a decade. Time flies.
And as nice it is to have matching binaries, in my case in the .deb format used on Debian or Ubuntu, it is wee bit a painful to manually download a file and then install it. Twice the pain if you are lucky enough to be on a system where you can also run RStudio Server. And now three times as painful as you may need a matching quarto-cli binary for the nice quarto service.
So wouldn t it be nice to have an apt-getable repo? And to autoMAGICall get updated versions when they are available? Oh yes. And I had been bugging JJ from day one. And JJ would almost listen intendly, nod briefly and firmly, and issue an assured we will look into it. Well, they are still looking
Luckily, years ago, Carl wrote a helper script for our use in Rocker. I promptly adopted these and kept them in the littler examples directory as a pair of script getRStudioDesktop.r and getRStudioServer.r, later complemented by getQuartoCli.r. And I used these for years, somewhere between weekly and monthly.
But it is still very manual: three script calls, one sudo dpkg -i call. And as our good friends at RStudio don t seem to be coming forward with a repo, I created one at GitHub thinking I could serve the files via GitHub Pages. Which of course I cannot as the .deb file for rstudio is well above the 100mb limit. So that seemed to be a bit of a setback. But after a bit of pondering, and recognizing that I am now in the fortunate position to have symmetric broadband access at home, I reckoned that until the bandwidth use gets excessive I will serve this as truly personal package archive (or tPPA) from here. Note that this is calibrated for my use so Ubuntu amd64 it is. Nothing else. And that it installs dailies . Which may cause issues for some people. You have warned. Reading tis paragraphs signifies agreement with the terms and limitations. Just kidding.
A quick screenshot from an update earlier is here. Note that I use the fabulous wajig wrapper by Graham Williams here as my frontend to apt, dpkg and more as I have for even longer than I have use RStudio. Its use is tangential here; sudo apt upgrade would have done the same (and is essentially being called). And it demonstrates the main benefit: we are now automated as the cron scheduler launches an update of the PPA at which ever frequency you chose (currently twice a week for me) and after that it becomes part of the normal apt updates we do anyway (and which I do about daily). So that s main gist: automated apt upgrades of rstudio, rstudio-server, and quarto-cli.
And you can find the underlying code in the GitHub repo ppa-rstudio which I put together a good week ago. I am currently updating the tPPA twice a week from crontab and have had two full upgrades already.
And who knows, maybe with a bid of prodding RStudio may come around. One can always hope.
If you like this or other open-source work I do, you can sponsor me at GitHub.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
unstable, they are installed into the testing directory. This directory contains packages that have not yet been accepted into the stable release but are on their way there.
Stable:
The stable distribution includes Debian s most recent officially released distribution. The current stable release which is Debian 11 is codenamed Bullseye.
Also we have the oldstable which is the previous stable release. The Debian 10 is now old stable which was codenamed Buster.
Architectures: These are known as the CPUs achitecture and there are various ports like amd64, arm64, i386 et.c.
An scenerio for example is if a user wants to test a package such as acorn in Testing on arm64 along with a package X from Unstable this would be a pin-package (Pin packages are packages that need to be obtained from a different suite than the main suite that selected.), which means the package the user wants to test with the initial Package selected.Finally, trigger can be described as the name of the test job which is optional. This test is done to check if those packages in unstable can be migrated to Testing.
This is a break down of Debci and I hope you enjoyed learning about what my internship entails. Till next time!
references:
Debian releases.
Ports
stty (change and print terminal line settings) as a program.
Thanks to some heroes, basenc, pr, chcon and runcon have been implemented. For example, for the two last programs, Koutheir Attouchi wrote new crates to manage SELinux properly. This crate has been used for some other utilities like cp, ls or id.
Leveraging the GNU testsuite to test this implementation
Because the GNU testsuite is excellent, we now have a proper CI using it to run the tests. It is pretty long on the Github action CI (almost two hours to run it) but it is an amazing improvement to the way we work. It was a joint work from a bunch of folks (James Robson, Roy Ivy III, etc). To achieve this, we also made it easier to run the GNU testsuite locally with the Rust implementation but also to ignore some tests or adjust some error messages (see build-gnu.sh and run-gnu-test.sh).
Following a suggestion of Brian G, a colleague at Mozilla (he did the same for some Firefox major change), we are now collecting the history of fail/pass/error into a separate repository and generating a daily graph showing the evolution of regression. | Total | 611 tests |
| Pass | 214 |
| Skip | 84 |
| Fail | 298 |
| Error | 15 |
Warning: Congrats! The gnu test tests/chmod/c-option is now passing!
<br />Warning: Congrats! The gnu test tests/chmod/silent is now passing!
<br />Warning: Congrats! The gnu test tests/chmod/umask-x is now passing!
<br />Error: GNU test failed: tests/du/long-from-unreadable. tests/du/long-from-unreadable is passing on 'master'. Maybe you have to rebase?
[...]
<br />Warning: Changes from master: PASS +4 / FAIL +0 / ERROR -4 / SKIP +0
This is also beneficial to GNU as, by implementing some options, Michael Debertol noticed some incorrect behaviors (with sort and cat) or an uninitialized variable (with chmod).
Documentations
Every day, we are generating the user documentation and of the internal coreutils.
User documentation: https://uutils.github.io/coreutils-docs/user/ Example: ls or cp
The internal documentation can be seen on: https://uutils.github.io/coreutils-docs/dev/uucore/ 202. This version includes the following changes:
[ Chris Lamb ]
* Don't fail if comparing a nonexistent file with a .pyc file (and add test).
(Closes: #1004312)
* Drop a reference in the manual page which claims the ability to compare
non-existent files on the command-line. This has not been possible since
version 32 which was released in September 2015. (Closes: #1004182)
* Add experimental support for incremental output support with a timeout.
Passing, for example, --timeout=60 will mean that diffoscope will not
recurse into any sub-archives after 60 seconds total execution time has
elapsed and mark the diff as being incomplete. (Note that this is not a
fixed/strict timeout due to implementation issues.)
(Closes: reproducible-builds/diffoscope#301)
* Don't return with an exit code of 0 if we encounter device file such as
/dev/stdin with human-readable metadata that matches literal, non-device,
file contents. (Closes: #1004198)
* Correct a "recompile" typo.
[ Sergei Trofimovich ]
* Fix/update whitespace for Black 21.12.
I've found that using a laptop during conference talks means you either
end up doing something else and missing important bits of the talk, or
at least look like you're doing something else. But it's extremely
helpful to be able to look up the person who is talking, or their project,
or expand an acronym that's mentioned or read around the subject.
At December's uksystems21 conference, I experimented with using an
iPad as a kind of compromise. Modern iOS versions let you split the
display between two apps1, so I put the built-in Notes app on
one side and a web browser on the other. I took notes using the Apple
Pencil. I've got a "paper-like" rough surface display protector on the
front which vastly improves the experience of using the Apple Pencil
for writing2.
I mostly took notes on the active talk, but I also tweaked my own slides3, looked
up supplementary information about the topic and the talker, and things like
that. It worked really well: much better than I expected.
Apple's split-screen implementation is clunky but essential to make this
work. The textured surface protector is a serious improvement over
the normal surface for writing. But most importantly I didn't lose focus
over the talks, and I don't think I looked like I did either.
| Publisher: | Avery |
| Copyright: | 2022 |
| ISBN: | 0-593-41914-6 |
| Format: | Kindle |
| Pages: | 291 |
Progvis finally made it into
Debian! What is it, you
ask? It is a great tool to teach about memory management and
concurrency.
I first saw progvis in the poster presentation his author, Filip
Str mb ck, did last year at the 52nd ACM Technical Sympossium on
Computer Science Education (SIGCSE),
immediately recognizing it as a tool I wanted to use at my classes,
and being it free software, make it available for all interested
Debian users. Quoting from
Progvis Web page:
This is a program visualization tool aimed at concurrent programs and related issues. The tool itself is mostly language agnostic, and relies on Storm to compile the provided code and provide basic debug information. The generated code is then inspected and instrumented to provide an experience similar to a basic debugger. The tool emphasizes a visual representation of the object hierarchy that is manipulated by the executed program to make it easy to understand how it looks. In particular, a visual representation is beneficial over a text representation since it makes it easier to find shared data that might need to be synchronized in a concurrent program. As mentioned, the tool is aimed at concurrent programs. Therefore, it allows spawning multiple threads running the same program to see if that affects the program s execution (this is mostly interesting if global variables are used). Furthermore, any spawned threads also appear in the tool, and the user may control them independently to explore possible race conditions or other synchronization errors. If enabled from the menu bar, the tool keeps track of reads and writes to the data structure in order to highlight basic race conditions in addition to deadlocks.
So, what is this Storm thing? Filip promptly informed me that
Progvis is not just a pedagogical tool Or rather, that it is part
of something bigger. Progvis is a program built using the Storm
programming language platform is more than a
compiler; it presents as a framework for creating languages,
designed to make easy to implement languages that can be extended
with new syntax and semantics. Storm is much more than what I have
explored, and can be used as an interactive compiler, a language
server used as a service for highlighting and completing in IDEs. But
I won t dig much more into Storm (which is, of course, now also
available in Debian as
well as the libraries built from the same source).
Back to progvis: It implements a very-close-to-C++ language, with some
details to better suit its purpose (i.e. instead of using the usual
pthread implementation, an own thread model is used; i.e. thread
creation is handled via int thread_id = thread_name(funcname,
¶ms) instead of the more complex
pthread_create()
function (including details such as the thread object being passed as
by reference as a parameter)
All in all, while I have not yet taken full advantage of this tool in
my teaching, it has helped me show somewhat hard to grasp concepts
such as:
progvis-examples package to get
started. You will find some dozens of sample programs in
/usr/share/doc/progvis-examples/examples; playing with them will
help you better understand the tool and be able to better write your
own programs.
print-based debugging was used in lieu of print()-based
debugging, and str were bytes, not unicode.
And things were working just fine. One day python2 EOL was announced, with a
date far in the future. Far enough to procrastinate for a long time. Combine
this with a codebase that is stable enough to not see many commits, and the fact
that Debsources is a volunteer-based project that happens at best on week-ends,
and you end up with a dormant software and a missed deadline.
But, as dormant as the codebase is, the instance hosted at
sources.debian.org is very popular and gets 200k
to 500k hits per day. Largely enough to be worth a proper maintenance and a
transition to python3.
Funky file names
While transitioning to python3 and juggling left and right with str, bytes
and unicode for internal objects, files, database entries and HTTP content, I
stumbled upon a bug that has been there since day 1.
Quick recap if you're unfamiliar with this tool: Debsources displays the content
of the source packages in the Debian archive. In other words, it's a bit like
GitHub, but for the Debian source code.
And some pieces of software out there, that ended up in Debian packages, happen
to contain files whose names can't be decoded to UTF-8. Interestingly enough,
there's no such thing as a standard for file names: with a few exceptions that
vary by operating system, any sequence of bytes can be a legit file name. And
some sequences of bytes are not valid UTF-8.
Of course those files are rare, and using ASCII characters to name a file is a
much more common practice than using bytes in a non-UTF-8 character
encoding. But when you deal with almost 100 million files on which you have no
control (those files come from free software projects, and make their way into
Debian without any renaming), it happens.
Now back to the bug: when trying to display such a file through the web
interface, it would crash because it can't convert the file name to UTF-8, which
is needed for the HTML representation of the page.
Bugfix
An often valid approach when trying to represent invalid UTF-8 content is to
ignore errors, and replace them with ? or . This is what Debsources
actually does to display non-UTF-8 file content.
Unfortunately, this best-effort approach is not suitable for file names, as file
names are also identifiers in Debsources: among other places, they are part of
URLs. If an URL were to use placeholder characters to replace those bytes, there
would be no deterministic way to match it with a file on disk anymore.
The representation of binary data into text is a known problem. Multiple
lossless solutions exist, such as base64 and its variants, but URLs looking like
https://sources.debian.org/src/Y293c2F5LzMuMDMtOS4yL2Nvd3NheS8= are not
readable at all compared to
https://sources.debian.org/src/cowsay/3.03-9.2/cowsay/. Plus, not
backwards-compatible with all existing links.
The solution I chose is to use double-percent encoding: this allows the representation of any byte in an
URL, while keeping allowed characters unchanged - and preventing CGI gateways
from trying to decode non-UTF-8 bytes. This is the best of both worlds: regular
file names get to appear normally and are human-readable, and funky file names
only have percent signs and hex numbers where needed.
Here is an example of such an URL:
https://sources.debian.org/src/aspell-is/0.51-0-4/%25EDslenska.alias/. Notice
the %25ED to represent the percentage symbol itself (%25) followed by an
invalid UTF-8 byte (%ED).
Transitioning to this was quite a challenge, as those file names don't only
appear in URLs, but also in web pages themselves, log files, database tables,
etc. And everything was done with str: made sense in python2 when str were
bytes, but not much in python3.
What are those files? What's their network?
I was wondering too. Let's list them!
import os
with open('non-utf-8-paths.bin', 'wb') as f:
for root, folders, files in os.walk(b'/srv/sources.debian.org/sources/'):
for path in folders + files:
try:
path.decode('utf-8')
except UnicodeDecodeError:
f.write(root + b'/' + path + b'\n')
It can synchronise the system clock with NTP servers, reference clocks (e.g. GPS receiver), and manual input using wristwatch and keyboard. It can also operate as an NTPv4 (RFC 5905) server and peer to provide a time service to other computers in the network. It is designed to perform well in a wide range of conditions, including intermittent network connections, heavily congested networks, changing temperatures (ordinary computer clocks are sensitive to temperature), and systems that do not run continuosly, or run on a virtual machine. Typical accuracy between two machines synchronised over the Internet is within a few milliseconds; on a LAN, accuracy is typically in tens of microseconds. With hardware timestamping, or a hardware reference clock, sub-microsecond accuracy may be possible.Now that's already great documentation right there. What it is, why it's good, and what to expect from it. I want more. They have a very handy comparison table between
chrony, ntp and
openntpd.
ntp program, I have, a long time ago, switched to using
openntpd on all my computers. I hadn't thought about it until I
recently noticed a lot of noise on one of my servers:
jan 18 10:09:49 curie ntpd[1069]: adjusting local clock by -1.604366s
jan 18 10:08:18 curie ntpd[1069]: adjusting local clock by -1.577608s
jan 18 10:05:02 curie ntpd[1069]: adjusting local clock by -1.574683s
jan 18 10:04:00 curie ntpd[1069]: adjusting local clock by -1.573240s
jan 18 10:02:26 curie ntpd[1069]: adjusting local clock by -1.569592s
You read that right, openntpd was constantly rewinding the clock,
sometimes in less than two minutes. The above log was taken while
doing diagnostics, looking at the last 30 minutes of logs. So, on
average, one 1.5 seconds rewind per 6 minutes!
That might be due to a dying real time clock (RTC) or some other
hardware problem. I know for a fact that the CMOS battery on that
computer (curie) died and I wasn't able to replace
it (!). So that's partly garbage-in, garbage-out here. But still, I
was curious to see how chrony would behave... (Spoiler: much better.)
But I also had trouble on another workstation, that one a much more
recent machine (angela). First, it seems OpenNTPd
would just fail at boot time:
anarcat@angela:~(main)$ sudo systemctl status openntpd
openntpd.service - OpenNTPd Network Time Protocol
Loaded: loaded (/lib/systemd/system/openntpd.service; enabled; vendor pres>
Active: inactive (dead) since Sun 2022-01-23 09:54:03 EST; 6h ago
Docs: man:openntpd(8)
Process: 3291 ExecStartPre=/usr/sbin/ntpd -n $DAEMON_OPTS (code=exited, sta>
Process: 3294 ExecStart=/usr/sbin/ntpd $DAEMON_OPTS (code=exited, status=0/>
Main PID: 3298 (code=exited, status=0/SUCCESS)
CPU: 34ms
jan 23 09:54:03 angela systemd[1]: Starting OpenNTPd Network Time Protocol...
jan 23 09:54:03 angela ntpd[3291]: configuration OK
jan 23 09:54:03 angela ntpd[3297]: ntp engine ready
jan 23 09:54:03 angela ntpd[3297]: ntp: recvfrom: Permission denied
jan 23 09:54:03 angela ntpd[3294]: Terminating
jan 23 09:54:03 angela systemd[1]: Started OpenNTPd Network Time Protocol.
jan 23 09:54:03 angela systemd[1]: openntpd.service: Succeeded.
After a restart, somehow it worked, but it took a long time to sync
the clock. At first, it would just not consider any peer at all:
anarcat@angela:~(main)$ sudo ntpctl -s all
0/20 peers valid, clock unsynced
peer
wt tl st next poll offset delay jitter
159.203.8.72 from pool 0.debian.pool.ntp.org
1 5 2 6s 6s ---- peer not valid ----
138.197.135.239 from pool 0.debian.pool.ntp.org
1 5 2 6s 7s ---- peer not valid ----
216.197.156.83 from pool 0.debian.pool.ntp.org
1 4 1 2s 9s ---- peer not valid ----
142.114.187.107 from pool 0.debian.pool.ntp.org
1 5 2 5s 6s ---- peer not valid ----
216.6.2.70 from pool 1.debian.pool.ntp.org
1 4 2 2s 8s ---- peer not valid ----
207.34.49.172 from pool 1.debian.pool.ntp.org
1 4 2 0s 5s ---- peer not valid ----
198.27.76.102 from pool 1.debian.pool.ntp.org
1 5 2 5s 5s ---- peer not valid ----
158.69.254.196 from pool 1.debian.pool.ntp.org
1 4 3 1s 6s ---- peer not valid ----
149.56.121.16 from pool 2.debian.pool.ntp.org
1 4 2 5s 9s ---- peer not valid ----
162.159.200.123 from pool 2.debian.pool.ntp.org
1 4 3 1s 6s ---- peer not valid ----
206.108.0.131 from pool 2.debian.pool.ntp.org
1 4 1 6s 9s ---- peer not valid ----
205.206.70.40 from pool 2.debian.pool.ntp.org
1 5 2 8s 9s ---- peer not valid ----
2001:678:8::123 from pool 2.debian.pool.ntp.org
1 4 2 5s 9s ---- peer not valid ----
2606:4700:f1::1 from pool 2.debian.pool.ntp.org
1 4 3 2s 6s ---- peer not valid ----
2607:5300:205:200::1991 from pool 2.debian.pool.ntp.org
1 4 2 5s 9s ---- peer not valid ----
2607:5300:201:3100::345c from pool 2.debian.pool.ntp.org
1 4 4 1s 6s ---- peer not valid ----
209.115.181.110 from pool 3.debian.pool.ntp.org
1 5 2 5s 6s ---- peer not valid ----
205.206.70.42 from pool 3.debian.pool.ntp.org
1 4 2 0s 6s ---- peer not valid ----
68.69.221.61 from pool 3.debian.pool.ntp.org
1 4 1 2s 9s ---- peer not valid ----
162.159.200.1 from pool 3.debian.pool.ntp.org
1 4 3 4s 7s ---- peer not valid ----
Then it would accept them, but still wouldn't sync the clock:
anarcat@angela:~(main)$ sudo ntpctl -s all
20/20 peers valid, clock unsynced
peer
wt tl st next poll offset delay jitter
159.203.8.72 from pool 0.debian.pool.ntp.org
1 8 2 5s 6s 0.672ms 13.507ms 0.442ms
138.197.135.239 from pool 0.debian.pool.ntp.org
1 7 2 4s 8s 1.260ms 13.388ms 0.494ms
216.197.156.83 from pool 0.debian.pool.ntp.org
1 7 1 3s 5s -0.390ms 47.641ms 1.537ms
142.114.187.107 from pool 0.debian.pool.ntp.org
1 7 2 1s 6s -0.573ms 15.012ms 1.845ms
216.6.2.70 from pool 1.debian.pool.ntp.org
1 7 2 3s 8s -0.178ms 21.691ms 1.807ms
207.34.49.172 from pool 1.debian.pool.ntp.org
1 7 2 4s 8s -5.742ms 70.040ms 1.656ms
198.27.76.102 from pool 1.debian.pool.ntp.org
1 7 2 0s 7s 0.170ms 21.035ms 1.914ms
158.69.254.196 from pool 1.debian.pool.ntp.org
1 7 3 5s 8s -2.626ms 20.862ms 2.032ms
149.56.121.16 from pool 2.debian.pool.ntp.org
1 7 2 6s 8s 0.123ms 20.758ms 2.248ms
162.159.200.123 from pool 2.debian.pool.ntp.org
1 8 3 4s 5s 2.043ms 14.138ms 1.675ms
206.108.0.131 from pool 2.debian.pool.ntp.org
1 6 1 0s 7s -0.027ms 14.189ms 2.206ms
205.206.70.40 from pool 2.debian.pool.ntp.org
1 7 2 1s 5s -1.777ms 53.459ms 1.865ms
2001:678:8::123 from pool 2.debian.pool.ntp.org
1 6 2 1s 8s 0.195ms 14.572ms 2.624ms
2606:4700:f1::1 from pool 2.debian.pool.ntp.org
1 7 3 6s 9s 2.068ms 14.102ms 1.767ms
2607:5300:205:200::1991 from pool 2.debian.pool.ntp.org
1 6 2 4s 9s 0.254ms 21.471ms 2.120ms
2607:5300:201:3100::345c from pool 2.debian.pool.ntp.org
1 7 4 5s 9s -1.706ms 21.030ms 1.849ms
209.115.181.110 from pool 3.debian.pool.ntp.org
1 7 2 0s 7s 8.907ms 75.070ms 2.095ms
205.206.70.42 from pool 3.debian.pool.ntp.org
1 7 2 6s 9s -1.729ms 53.823ms 2.193ms
68.69.221.61 from pool 3.debian.pool.ntp.org
1 7 1 1s 7s -1.265ms 46.355ms 4.171ms
162.159.200.1 from pool 3.debian.pool.ntp.org
1 7 3 4s 8s 1.732ms 35.792ms 2.228ms
It took a solid five minutes to sync the clock, even though the
peers were considered valid within a few seconds:
jan 23 15:58:41 angela systemd[1]: Started OpenNTPd Network Time Protocol.
jan 23 15:58:58 angela ntpd[84086]: peer 142.114.187.107 now valid
jan 23 15:58:58 angela ntpd[84086]: peer 198.27.76.102 now valid
jan 23 15:58:58 angela ntpd[84086]: peer 207.34.49.172 now valid
jan 23 15:58:58 angela ntpd[84086]: peer 209.115.181.110 now valid
jan 23 15:58:59 angela ntpd[84086]: peer 159.203.8.72 now valid
jan 23 15:58:59 angela ntpd[84086]: peer 138.197.135.239 now valid
jan 23 15:58:59 angela ntpd[84086]: peer 162.159.200.123 now valid
jan 23 15:58:59 angela ntpd[84086]: peer 2607:5300:201:3100::345c now valid
jan 23 15:59:00 angela ntpd[84086]: peer 2606:4700:f1::1 now valid
jan 23 15:59:00 angela ntpd[84086]: peer 158.69.254.196 now valid
jan 23 15:59:01 angela ntpd[84086]: peer 216.6.2.70 now valid
jan 23 15:59:01 angela ntpd[84086]: peer 68.69.221.61 now valid
jan 23 15:59:01 angela ntpd[84086]: peer 205.206.70.40 now valid
jan 23 15:59:01 angela ntpd[84086]: peer 205.206.70.42 now valid
jan 23 15:59:02 angela ntpd[84086]: peer 162.159.200.1 now valid
jan 23 15:59:04 angela ntpd[84086]: peer 216.197.156.83 now valid
jan 23 15:59:05 angela ntpd[84086]: peer 206.108.0.131 now valid
jan 23 15:59:05 angela ntpd[84086]: peer 2001:678:8::123 now valid
jan 23 15:59:05 angela ntpd[84086]: peer 149.56.121.16 now valid
jan 23 15:59:07 angela ntpd[84086]: peer 2607:5300:205:200::1991 now valid
jan 23 16:03:47 angela ntpd[84086]: clock is now synced
That seems kind of odd. It was also frustrating to have very little
information from ntpctl about the state of the daemon. I understand
it's designed to be minimal, but it could inform me on his known
offset, for example. It does tell me about the offset with the
different peers, but not as clearly as one would expect. It's also
unclear how it disciplines the RTC at all.
chrony:
jan 23 16:07:16 angela systemd[1]: Starting chrony, an NTP client/server...
jan 23 16:07:16 angela chronyd[87765]: chronyd version 4.0 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 -DEBUG)
jan 23 16:07:16 angela chronyd[87765]: Initial frequency 3.814 ppm
jan 23 16:07:16 angela chronyd[87765]: Using right/UTC timezone to obtain leap second data
jan 23 16:07:16 angela chronyd[87765]: Loaded seccomp filter
jan 23 16:07:16 angela systemd[1]: Started chrony, an NTP client/server.
jan 23 16:07:21 angela chronyd[87765]: Selected source 206.108.0.131 (2.debian.pool.ntp.org)
jan 23 16:07:21 angela chronyd[87765]: System clock TAI offset set to 37 seconds
First, you'll notice there's none of that "clock synced" nonsense, it
picks a source, and then... it's just done. Because the clock on this
computer is not drifting that much, and openntpd had (presumably) just
sync'd it anyways. And indeed, if we look at detailed stats from the
powerful chronyc client:
anarcat@angela:~(main)$ sudo chronyc tracking
Reference ID : CE6C0083 (ntp1.torix.ca)
Stratum : 2
Ref time (UTC) : Sun Jan 23 21:07:21 2022
System time : 0.000000311 seconds slow of NTP time
Last offset : +0.000807989 seconds
RMS offset : 0.000807989 seconds
Frequency : 3.814 ppm fast
Residual freq : -24.434 ppm
Skew : 1000000.000 ppm
Root delay : 0.013200894 seconds
Root dispersion : 65.357254028 seconds
Update interval : 1.4 seconds
Leap status : Normal
We see that we are nanoseconds away from NTP time. That was ran very
quickly after starting the server (literally in the same second as
chrony picked a source), so stats are a bit weird (e.g. the Skew is
huge). After a minute or two, it looks more reasonable:
Reference ID : CE6C0083 (ntp1.torix.ca)
Stratum : 2
Ref time (UTC) : Sun Jan 23 21:09:32 2022
System time : 0.000487002 seconds slow of NTP time
Last offset : -0.000332960 seconds
RMS offset : 0.000751204 seconds
Frequency : 3.536 ppm fast
Residual freq : +0.016 ppm
Skew : 3.707 ppm
Root delay : 0.013363549 seconds
Root dispersion : 0.000324015 seconds
Update interval : 65.0 seconds
Leap status : Normal
Now it's learning how good or bad the RTC clock is ("Frequency"), and
is smoothly adjusting the System time to follow the average offset
(RMS offset, more or less). You'll also notice the Update interval
has risen, and will keep expanding as chrony learns more about the
internal clock, so it doesn't need to constantly poll the NTP servers
to sync the clock. In the above, we're 487 micro seconds (less than a
milisecond!) away from NTP time.
(People interested in the explanation of every single one of those
fields can read the excellent chronyc manpage. That thing made me
want to nerd out on NTP again!)
On the machine with the bad clock, chrony also did a 1.5 second
adjustment, but just once, at startup:
jan 18 11:54:33 curie chronyd[2148399]: Selected source 206.108.0.133 (2.debian.pool.ntp.org)
jan 18 11:54:33 curie chronyd[2148399]: System clock wrong by -1.606546 seconds
jan 18 11:54:31 curie chronyd[2148399]: System clock was stepped by -1.606546 seconds
jan 18 11:54:31 curie chronyd[2148399]: System clock TAI offset set to 37 seconds
Then it would still struggle to keep the clock in sync, but not as
badly as openntpd. Here's the offset a few minutes after that above
startup:
System time : 0.000375352 seconds slow of NTP time
And again a few seconds later:
System time : 0.001793046 seconds slow of NTP time
I don't currently have access to that machine, and will update this
post with the latest status, but so far I've had a very good
experience with chrony on that machine, which is a testament to its
resilience, and it also just works on my other machines as well.
chrony's feature set is so much superior... Here's an excerpt of the
extras in chrony, taken from the comparison table:
chrony folks, I'm switching.
chrony does as bad of a job as openntpd on that old
machine, and just doesn't tell me about it. For example, here's
another log sample from another server
(marcos):
jan 23 11:13:25 marcos ntpd[1976694]: adjusting clock frequency by 0.451035 to -16.420273ppm
I get those basically every day, which seems to show that it's at
least trying to keep track of the hardware clock.
In other words, it's quite possible I have no idea what I'm talking
about and you definitely need to take this article with a grain of
salt. I'm not an NTP expert.
Update: I should also mentioned that I haven't evaluated
systemd-timesyncd, for a few reasons:
chrony (at least as shipped in
Debian) is sane (good default peers, no open network by default),
installing it is as simple as:
apt install chrony
And because it somehow conflicts with openntpd, that also takes care
of removing that cruft as well.
systemd-timesyncd was not only
shipped, but it was installed by default, as it was marked as a hard
dependency of systemd. That was "fixed" in systemd-247.9-2 (see
bug 986651), but only by making the dependency a Recommends
and marking it as Priority: important.
So in effect, systemd-timesyncd became the default NTP daemon in
Debian in bookworm, which I find somewhat surprising. timesyncd has
many things going for it (as mentioned above), but I do find it a bit
annoying that systemd is replacing all those utilities in such a
way. I also wonder what is going to happen on upgrades. This is all a
little frustrating too because there is no good comparison between the
other NTP daemons and timesyncd anywhere. The chrony comparison
table doesn't mention it, and an audit by the Core Infrastructure
Initiative from 2017 doesn't mention it either, even though
timesyncd was announced in 2014. (Same with this blog post from Facebook.)
The second release of the still fairly new qlcal package arrivied at CRAN today.
qlcal is based on the calendaring subset of QuantLib. It is provided (for the R package) as a set of included files, so the package is self-contained and does not depend on an external QuantLib library (which can be demanding to build). qlcal covers over sixty country / market calendars and can compute holiday lists, its complement (i.e. business day lists) and much more.
This release brings a further package simplification from removing a few more files not needed for just calendaring, as well as an update 2022 calendar for China from the just-release 1.25 version of QuantLib.
See the project page and package documentation for more details, and more examples. If you like this or other open-source work I do, you can now sponsor me at GitHub.Changes in version 0.0.2 (2022-01-21)
- Further minimize set of files needed for calendaring
- Update China calendar from QuantLib 1.25 release
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
In my four most recent posts, I went over the memoirs and biographies, the non-fiction, the fiction and the 'classic' novels that I enjoyed reading the most in 2021. But in the very last of my 2021 roundup posts, I'll be going over some of my favourite movies. (Saying that, these are perhaps less of my 'favourite films' than the ones worth remarking on after all, nobody needs to hear that The Godfather is a good movie.)
It's probably helpful to remark you that I took a self-directed course in film history in 2021, based around the first volume of Roger Ebert's The Great Movies. This collection of 100-odd movie essays aims to make a tour of the landmarks of the first century of cinema, and I watched all but a handul before the year was out. I am slowly making my way through volume two in 2022. This tome was tremendously useful, and not simply due to the background context that Ebert added to each film: it also brought me into contact with films I would have hardly come through some other means. Would I have ever discovered the sly comedy of Trouble in Paradise (1932) or the touching proto-realism of L'Atalante (1934) any other way? It also helped me to 'get around' to watching films I may have put off watching forever the influential Battleship Potemkin (1925), for instance, and the ur-epic Lawrence of Arabia (1962) spring to mind here.
Choosing a 'worst' film is perhaps more difficult than choosing the best. There are first those that left me completely dry (Ready or Not, Written on the Wind, etc.), and those that were simply poorly executed. And there are those that failed to meet their own high opinions of themselves, such as the 'made for Reddit' Tenet (2020) or the inscrutable Vanilla Sky (2001) the latter being an almost perfect example of late-20th century cultural exhaustion.
But I must save my most severe judgement for those films where I took a visceral dislike how their subjects were portrayed. The sexually problematic Sixteen Candles (1984) and the pseudo-Catholic vigilantism of The Boondock Saints (1999) both spring to mind here, the latter of which combines so many things I dislike into such a short running time I'd need an entire essay to adequately express how much I disliked it.
Dogtooth (2009) A father, a mother, a brother and two sisters live in a large and affluent house behind a very high wall and an always-locked gate. Only the father ever leaves the property, driving to the factory that he happens to own. Dogtooth goes far beyond any allusion to Josef Fritzl's cellar, though, as the children's education is a grotesque parody of home-schooling. Here, the parents deliberately teach their children the wrong meaning of words (e.g. a yellow flower is called a 'zombie'), all of which renders the outside world utterly meaningless and unreadable, and completely mystifying its very existence. It is this creepy strangeness within a 'regular' family unit in Dogtooth that is both socially and epistemically horrific, and I'll say nothing here of its sexual elements as well. Despite its cold, inscrutable and deadpan surreality, Dogtooth invites all manner of potential interpretations. Is this film about the artificiality of the nuclear family that the West insists is the benchmark of normality? Or is it, as I prefer to believe, something more visceral altogether: an allegory for the various forms of ontological violence wrought by fascism, as well a sobering nod towards some of fascism's inherent appeals? (Perhaps it is both. In 1972, French poststructuralists Gilles and F lix Guattari wrote Anti-Oedipus, which plays with the idea of the family unit as a metaphor for the authoritarian state.) The Greek-language Dogtooth, elegantly shot, thankfully provides no easy answers.
Holy Motors (2012) There is an infamous scene in Un Chien Andalou, the 1929 film collaboration between Luis Bu uel and famed artist Salvador Dal . A young woman is cornered in her own apartment by a threatening man, and she reaches for a tennis racquet in self-defence. But the man suddenly picks up two nearby ropes and drags into the frame two large grand pianos... each leaden with a dead donkey, a stone tablet, a pumpkin and a bewildered priest. This bizarre sketch serves as a better introduction to Leos Carax's Holy Motors than any elementary outline of its plot, which ostensibly follows 24 hours in the life of a man who must play a number of extremely diverse roles around Paris... all for no apparent reason. (And is he even a man?) Surrealism as an art movement gets a pretty bad wrap these days, and perhaps justifiably so. But Holy Motors and Un Chien Andalou serve as a good reminder that surrealism can be, well, 'good, actually'. And if not quite high art, Holy Motors at least demonstrates that surrealism can still unnerving and hilariously funny. Indeed, recalling the whimsy of the plot to a close friend, the tears of laughter came unbidden to my eyes once again. ("And then the limousines...!") Still, it is unclear how Holy Motors truly refreshes surrealism for the twenty-first century. Surrealism was, in part, a reaction to the mechanical and unfeeling brutality of World War I and ultimately sought to release the creative potential of the unconscious mind. Holy Motors cannot be responding to another continental conflagration, and so it appears to me to be some kind of commentary on the roles we exhibit in an era of 'post-postmodernity': a sketch on our age of performative authenticity, perhaps, or an idle doodle on the function and psychosocial function of work. Or perhaps not. After all, this film was produced in a time that offers the near-universal availability of mind-altering substances, and this certainly changes the context in which this film was both created. And, how can I put it, was intended to be watched.
Manchester by the Sea (2016) An absolutely devastating portrayal of a character who is unable to forgive himself and is hesitant to engage with anyone ever again. It features a near-ideal balance between portraying unrecoverable anguish and tender warmth, and is paradoxically grandiose in its subtle intimacy. The mechanics of life led me to watch this lying on a bed in a chain hotel by Heathrow Airport, and if this colourless circumstance blunted the film's emotional impact on me, I am probably thankful for it. Indeed, I find myself reduced in this review to fatuously recalling my favourite interactions instead of providing any real commentary. You could write a whole essay about one particular incident: its surfaces, subtexts and angles... all despite nothing of any substance ever being communicated. Truly stunning.
McCabe & Mrs. Miller (1971) Roger Ebert called this movie one of the saddest films I have ever seen, filled with a yearning for love and home that will not ever come. But whilst it is difficult to disagree with his sentiment, Ebert's choice of sad is somehow not quite the right word. Indeed, I've long regretted that our dictionaries don't have more nuanced blends of tragedy and sadness; perhaps the Ancient Greeks can loan us some. Nevertheless, the plot of this film is of a gambler and a prostitute who become business partners in a new and remote mining town called Presbyterian Church. However, as their town and enterprise booms, it comes to the attention of a large mining corporation who want to bully or buy their way into the action. What makes this film stand out is not the plot itself, however, but its mood and tone the town and its inhabitants seem to be thrown together out of raw lumber, covered alternatively in mud or frozen ice, and their days (and their personalities) are both short and dark in equal measure. As a brief aside, if you haven't seen a Roger Altman film before, this has all the trappings of being a good introduction. As Ebert went on to observe: This is not the kind of movie where the characters are introduced. They are all already here. Furthermore, we can see some of Altman's trademark conversations that overlap, a superb handling of ensemble casts, and a quietly subversive view of the tyranny of 'genre'... and the latter in a time when the appetite for revisionist portrays of the West was not very strong. All of these 'Altmanian' trademarks can be ordered in much stronger measures in his later films: in particular, his comedy-drama Nashville (1975) has 24 main characters, and my jejune interpretation of Gosford Park (2001) is that it is purposefully designed to poke fun those who take a reductionist view of 'genre', or at least on the audience's expectations. (In this case, an Edwardian-era English murder mystery in the style of Agatha Christie, but where no real murder or detection really takes place.) On the other hand, McCabe & Mrs. Miller is actually a poor introduction to Altman. The story is told in a suitable deliberate and slow tempo, and the two stars of the film are shown thoroughly defrocked of any 'star status', in both the visual and moral dimensions. All of these traits are, however, this film's strength, adding up to a credible, fascinating and riveting portrayal of the old West.
Detour (1945) Detour was filmed in less than a week, and it's difficult to decide out of the actors and the screenplay which is its weakest point.... Yet it still somehow seemed to drag me in. The plot revolves around luckless Al who is hitchhiking to California. Al gets a lift from a man called Haskell who quickly falls down dead from a heart attack. Al quickly buries the body and takes Haskell's money, car and identification, believing that the police will believe Al murdered him. An unstable element is soon introduced in the guise of Vera, who, through a set of coincidences that stretches credulity, knows that this 'new' Haskell (ie. Al pretending to be him) is not who he seems. Vera then attaches herself to Al in order to blackmail him, and the world starts to spin out of his control. It must be understood that none of this is executed very well. Rather, what makes Detour so interesting to watch is that its 'errors' lend a distinctively creepy and unnatural hue to the film. Indeed, in the early twentieth century, Sigmund Freud used the word unheimlich to describe the experience of something that is not simply mysterious, but something creepy in a strangely familiar way. This is almost the perfect description of watching Detour its eerie nature means that we are not only frequently second-guessed about where the film is going, but are often uncertain whether we are watching the usual objective perspective offered by cinema. In particular, are all the ham-fisted segues, stilted dialogue and inscrutable character motivations actually a product of Al inventing a story for the viewer? Did he murder Haskell after all, despite the film 'showing' us that Haskell died of natural causes? In other words, are we watching what Al wants us to believe? Regardless of the answers to these questions, the film succeeds precisely because of its accidental or inadvertent choices, so it is an implicit reminder that seeking the director's original intention in any piece of art is a complete mirage. Detour is certainly not a good film, but it just might be a great one. (It is a short film too, and, out of copyright, it is available online for free.)
Safe (1995) Safe is a subtly disturbing film about an upper-middle-class housewife who begins to complain about vague symptoms of illness. Initially claiming that she doesn't feel right, Carol starts to have unexplained headaches, a dry cough and nosebleeds, and eventually begins to have trouble breathing. Carol's family doctor treats her concerns with little care, and suggests to her husband that she sees a psychiatrist. Yet Carol's episodes soon escalate. For example, as a 'homemaker' and with nothing else to occupy her, Carol's orders a new couch for a party. But when the store delivers the wrong one (although it is not altogether clear that they did), Carol has a near breakdown. Unsure where to turn, an 'allergist' tells Carol she has "Environmental Illness," and so Carol eventually checks herself into a new-age commune filled with alternative therapies. On the surface, Safe is thus a film about the increasing about of pesticides and chemicals in our lives, something that was clearly felt far more viscerally in the 1990s. But it is also a film about how lack of genuine healthcare for women must be seen as a critical factor in the rise of crank medicine. (Indeed, it made for something of an uncomfortable watch during the coronavirus lockdown.) More interestingly, however, Safe gently-yet-critically examines the psychosocial causes that may be aggravating Carol's illnesses, including her vacant marriage, her hollow friends and the 'empty calorie' stimulus of suburbia. None of this should be especially new to anyone: the gendered Victorian term 'hysterical' is often all but spoken throughout this film, and perhaps from the very invention of modern medicine, women's symptoms have often regularly minimised or outright dismissed. (Hilary Mantel's 2003 memoir, Giving Up the Ghost is especially harrowing on this.) As I opened this review, the film is subtle in its messaging. Just to take one example from many, the sound of the cars is always just a fraction too loud: there's a scene where a group is eating dinner with a road in the background, and the total effect can be seen as representing the toxic fumes of modernity invading our social lives and health. I won't spoiler the conclusion of this quietly devasting film, but don't expect a happy ending.
The Driver (1978) Critics grossly misunderstood The Driver when it was first released. They interpreted the cold and unemotional affect of the characters with the lack of developmental depth, instead of representing their dissociation from the society around them. This reading was encouraged by the fact that the principal actors aren't given real names and are instead known simply by their archetypes instead: 'The Driver', 'The Detective', 'The Player' and so on. This sort of quasi-Jungian erudition is common in many crime films today (Reservoir Dogs, Kill Bill, Layer Cake, Fight Club), so the critics' misconceptions were entirely reasonable in 1978. The plot of The Driver involves the eponymous Driver, a noted getaway driver for robberies in Los Angeles. His exceptional talent has far prevented him from being captured thus far, so the Detective attempts to catch the Driver by pardoning another gang if they help convict the Driver via a set-up robbery. To give himself an edge, however, The Driver seeks help from the femme fatale 'Player' in order to mislead the Detective. If this all sounds eerily familiar, you would not be far wrong. The film was essentially remade by Nicolas Winding Refn as Drive (2011) and in Edgar Wright's 2017 Baby Driver. Yet The Driver offers something that these neon-noir variants do not. In particular, the car chases around Los Angeles are some of the most captivating I've seen: they aren't thrilling in the sense of tyre squeals, explosions and flying boxes, but rather the vehicles come across like wild animals hunting one another. This feels especially so when the police are hunting The Driver, which feels less like a low-stakes game of cat and mouse than a pack of feral animals working together a gang who will tear apart their prey if they find him. In contrast to the undercar neon glow of the Fast & Furious franchise, the urban realism backdrop of the The Driver's LA metropolis contributes to a sincere feeling of artistic fidelity as well. To be sure, most of this is present in the truly-excellent Drive, where the chase scenes do really communicate a credible sense of stakes. But the substitution of The Driver's grit with Drive's soft neon tilts it slightly towards that common affliction of crime movies: style over substance. Nevertheless, I can highly recommend watching The Driver and Drive together, as it can tell you a lot about the disconnected socioeconomic practices of the 1980s compared to the 2010s. More than that, however, the pseudo-1980s synthwave soundtrack of Drive captures something crucial to analysing the world of today. In particular, these 'sounds from the past filtered through the present' bring to mind the increasing role of nostalgia for lost futures in the culture of today, where temporality and pop culture references are almost-exclusively citational and commemorational.
The Souvenir (2019) The ostensible outline of this quietly understated film follows a shy but ambitious film student who falls into an emotionally fraught relationship with a charismatic but untrustworthy older man. But that doesn't quite cover the plot at all, for not only is The Souvenir a film about a young artist who is inspired, derailed and ultimately strengthened by a toxic relationship, it is also partly a coming-of-age drama, a subtle portrait of class and, finally, a film about the making of a film. Still, one of the geniuses of this truly heartbreaking movie is that none of these many elements crowds out the other. It never, ever feels rushed. Indeed, there are many scenes where the camera simply 'sits there' and quietly observes what is going on. Other films might smother themselves through references to 18th-century oil paintings, but The Souvenir somehow evades this too. And there's a certain ring of credibility to the story as well, no doubt in part due to the fact it is based on director Joanna Hogg's own experiences at film school. A beautifully observed and multi-layered film; I'll be happy if the sequel is one-half as good.
The Wrestler (2008) Randy 'The Ram' Robinson is long past his prime, but he is still rarin' to go in the local pro-wrestling circuit. Yet after a brutal beating that seriously threatens his health, Randy hangs up his tights and pursues a serious relationship... and even tries to reconnect with his estranged daughter. But Randy can't resist the lure of the ring, and readies himself for a comeback. The stage is thus set for Darren Aronofsky's The Wrestler, which is essentially about what drives Randy back to the ring. To be sure, Randy derives much of his money from wrestling as well as his 'fitness', self-image, self-esteem and self-worth. Oh, it's no use insisting that wrestling is fake, for the sport is, needless to say, Randy's identity; it's not for nothing that this film is called The Wrestler. In a number of ways, The Sound of Metal (2019) is both a reaction to (and a quiet remake of) The Wrestler, if only because both movies utilise 'cool' professions to explore such questions of identity. But perhaps simply when The Wrestler was produced makes it the superior film. Indeed, the role of time feels very important for the Wrestler. In the first instance, time is clearly taking its toll on Randy's body, but I felt it more strongly in the sense this was very much a pre-2008 film, released on the cliff-edge of the global financial crisis, and the concomitant precarity of the 2010s. Indeed, it is curious to consider that you couldn't make The Wrestler today, although not because the relationship to work has changed in any fundamentalway. (Indeed, isn't it somewhat depressing the realise that, since the start of the pandemic and the 'work from home' trend to one side, we now require even more people to wreck their bodies and mental health to cover their bills?) No, what I mean to say here is that, post-2016, you cannot portray wrestling on-screen without, how can I put it, unwelcome connotations. All of which then reminds me of Minari's notorious red hat... But I digress. The Wrestler is a grittily stark darkly humorous look into the life of a desperate man and a sorrowful world, all through one tragic profession.
Thief (1981) Frank is an expert professional safecracker and specialises in high-profile diamond heists. He plans to use his ill-gotten gains to retire from crime and build a life for himself with a wife and kids, so he signs on with a top gangster for one last big score. This, of course, could be the plot to any number of heist movies, but Thief does something different. Similar to The Wrestler and The Driver (see above) and a number of other films that I watched this year, Thief seems to be saying about our relationship to work and family in modernity and postmodernity. Indeed, the 'heist film', we are told, is an understudied genre, but part of the pleasure of watching these films is said to arise from how they portray our desired relationship to work. In particular, Frank's desire to pull off that last big job feels less about the money it would bring him, but a displacement from (or proxy for) fulfilling some deep-down desire to have a family or indeed any relationship at all. Because in theory, of course, Frank could enter into a fulfilling long-term relationship right away, without stealing millions of dollars in diamonds... but that's kinda the entire point: Frank needing just one more theft is an excuse to not pursue a relationship and put it off indefinitely in favour of 'work'. (And being Federal crimes, it also means Frank cannot put down meaningful roots in a community.) All this is communicated extremely subtly in the justly-lauded lowkey diner scene, by far the best scene in the movie. The visual aesthetic of Thief is as if you set The Warriors (1979) in a similarly-filthy Chicago, with the Xenophon-inspired plot of The Warriors replaced with an almost deliberate lack of plot development... and the allure of The Warriors' fantastical criminal gangs (with their alluringly well-defined social identities) substituted by a bunch of amoral individuals with no solidarity beyond the immediate moment. A tale of our time, perhaps. I should warn you that the ending of Thief is famously weak, but this is a gritty, intelligent and strangely credible heist movie before you get there.
Uncut Gems (2019) The most exhausting film I've seen in years; the cinematic equivalent of four cups of double espresso, I didn't even bother even trying to sleep after downing Uncut Gems late one night. Directed by the two Safdie Brothers, it often felt like I was watching two films that had been made at the same time. (Or do I mean two films at 2X speed?) No, whatever clumsy metaphor you choose to adopt, the unavoidable effect of this film's finely-tuned chaos is an uncompromising and anxiety-inducing piece of cinema. The plot follows Howard as a man lost to his countless vices mostly gambling with a significant side hustle in adultery, but you get the distinct impression he would be happy with anything that will give him another high. A true junkie's junkie, you might say. You know right from the beginning it's going to end in some kind of disaster, the only question remaining is precisely how and what. Portrayed by an (almost unrecognisable) Adam Sandler, there's an uncanny sense of distance in the emotional chasm between 'Sandler-as-junkie' and 'Sandler-as-regular-star-of-goofy-comedies'. Yet instead of being distracting and reducing the film's affect, this possibly-deliberate intertextuality somehow adds to the masterfully-controlled mayhem. My heart races just at the memory. Oof.
Woman in the Dunes (1964) I ended up watching three films that feature sand this year: Denis Villeneuve's Dune (2021), Lawrence of Arabia (1962) and Woman in the Dunes. But it is this last 1964 film by Hiroshi Teshigahara that will stick in my mind in the years to come. Sure, there is none of the Medician intrigue of Dune or the Super Panavision-70 of Lawrence of Arabia (or its quasi-orientalist score, itself likely stolen from Anton Bruckner's 6th Symphony), but Woman in the Dunes doesn't have to assert its confidence so boldly, and it reveals the enormity of its plot slowly and deliberately instead. Woman in the Dunes never rushes to get to the film's central dilemma, and it uncovers its terror in little hints and insights, all whilst establishing the daily rhythm of life. Woman in the Dunes has something of the uncanny horror as Dogtooth (see above), as well as its broad range of potential interpretations. Both films permit a wide array of readings, without resorting to being deliberately obscurantist or being just plain random it is perhaps this reason why I enjoyed them so much. It is true that asking 'So what does the sand mean?' sounds tediously sophomoric shorn of any context, but it somehow applies to this thoughtfully self-contained piece of cinema.
A Quiet Place (2018) Although A Quiet Place was not actually one of the best films I saw this year, I'm including it here as it is certainly one of the better 'mainstream' Hollywood franchises I came across. Not only is the film very ably constructed and engages on a visceral level, I should point out that it is rare that I can empathise with the peril of conventional horror movies (and perhaps prefer to focus on its cultural and political aesthetics), but I did here. The conceit of this particular post-apocalyptic world is that a family is forced to live in almost complete silence while hiding from creatures that hunt by sound alone. Still, A Quiet Place engages on an intellectual level too, and this probably works in tandem with the pure 'horrorific' elements and make it stick into your mind. In particular, and to my mind at least, A Quiet Place a deeply American conservative film below the surface: it exalts the family structure and a certain kind of sacrifice for your family. (The music often had a passacaglia-like strain too, forming a tombeau for America.) Moreover, you survive in this dystopia by staying quiet that is to say, by staying stoic suggesting that in the wake of any conflict that might beset the world, the best thing to do is to keep quiet. Even communicating with your loved ones can be deadly to both of you, so not emote, acquiesce quietly to your fate, and don't, whatever you do, speak up. (Or join a union.) I could go on, but The Quiet Place is more than this. It's taut and brief, and despite cinema being an increasingly visual medium, it encourages its audience to develop a new relationship with sound.
The Rcpp team is thrilled to share the news of the newest release 1.0.8 of Rcpp which hit CRAN today, and has already been uploaded to Debian as well. Windows and macOS builds should appear at CRAN in the next few days. This release continues with the six-months cycle started with release 1.0.5 in July 2020. As a reminder, interim dev or rc releases will alwasys be available in the Rcpp drat repo; this cycle there were once again seven (!!) times two as we also tested the modified header (more below). These rolling release tend to work just as well, and are also fully tested against all reverse-dependencies.
Rcpp has become the most popular way of enhancing R with C or C++ code. Right now, around 2478 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 242 in BioConductor.
This release finally brings a change we have worked on quite a bit over the last few months. The idea of enforcing the setting of STRICT_R_HEADERS was prososed years ago in 2016 and again in 2018. But making such a chance against a widely-deployed code base has repurcussions, and we were not ready then. Last April, this was revisited in issue #1158. Over the course of numerous lengthy runs of tests of a changed Rcpp package against (essentially) all reverse-dependencies (i.e. packages which use Rcpp) we identified ninetyfour packages in total which needed a change. We provided either a patch we emailed, or a GitHub pull request, to all ninetyfour. And we are happy to say that eighty cases were resolved via a new CRAN upload, with a seven more having merged the pull request but not yet uploaded.
Hence, we could make the case to CRAN (who were always CC ed on the monthly nag emails we sent to maintainers of packages needing a change) that an upload was warranted. And after a brief period for their checks and inspection, our January 11 release of Rcpp 1.0.8 arrived on CRAN on January 13.
So with that, a big and heartfelt Thank You! to all eighty maintainers for updating their packages to permit this change at the Rcpp end, to CRAN for the extra checking, and to everybody else who I bugged with the numerous emails and updated to the seemingly never-ending issue #1158. We all got this done, and that is a Good Thing (TM).
Other than the aforementioned change which will not automatically set STRICT_R_HEADERS (unless opted out which one can), a number of nice pull request by a number of contributors are included in this release:
Thanks to my CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome underChanges in Rcpp release version 1.0.8 (2022-01-11)
- Changes in Rcpp API:
STRICT_R_HEADERSis now enabled by default, see extensive discussion in #1158 closing #898.- A new
#defineallows default setting of finalizer calls for external pointers (I aki in #1180 closing #1108).Rcpp:::CxxFlags()now quotes the include path generated, (Kevin in #1189 closing #1188).- New header files
Rcpp/Light,Rcpp/Lighter,Rcpp/Lightestand defaultRcpp/Rcppfor fine-grained access to features (and compilation time) (Dirk #1191 addressing #1168).- Changes in Rcpp Attributes:
- Changes in Rcpp Documentation:
- Changes in Rcpp Deployment:
rcpp tag at StackOverflow which also allows searching among the (currently) 2822 previous questions.
If you like this or other open-source work I do, you can sponsor me at GitHub.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
At CES this week, Lenovo announced that their new Z-series laptops would ship with AMD processors that incorporate Microsoft's Pluton security chip. There's a fair degree of cynicism around whether Microsoft have the interests of the industry as a whole at heart or not, so unsurprisingly people have voiced concerns about Pluton allowing for platform lock-in and future devices no longer booting non-Windows operating systems. Based on what we currently know, I think those concerns are understandable but misplaced.as the Trusted Platform Module; as a security processor used for non-TPM scenarios like platform resiliency; or OEMs can choose to ship with Pluton turned off. What we're likely to see to begin with is the former - Pluton will run firmware that exposes a Trusted Computing Group compatible TPM interface. This is almost identical to the status quo. Microsoft have required that all Windows certified hardware ship with a TPM for years now, but for cost reasons this is often not in the form of a separate hardware component. Instead, both Intel and AMD provide support for running the TPM stack on a component separate from the main execution cores on the system - for Intel, this TPM code runs on the Management Engine integrated into the chipset, and for AMD on the Platform Security Processor that's integrated into the CPU package itself.
Next.