We are happy to share that the RcppSimdJson
package has been updated to release 0.1.10.
RcppSimdJson
wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via
very clever algorithmic engineering to obtain largely branch-free code,
coupled with modern C++ and newer compiler instructions, it results in
parsing gigabytes of JSON parsed per second which is quite
mindboggling. The best-case performance is faster than CPU speed as
use of parallel SIMD instructions and careful branch avoidance can lead
to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire
at QCon.
This release updates the underlying simdjson library to
version 3.1.8 (also made today). Otherwise we only made a minor edit to
the README and adjusted one tweek for code coverage.
The (very short) NEWS entry for this release follows.
Changes in version 0.1.10
(2023-05-14)
simdjson was upgraded to version 3.1.8
(Dirk in #85).
A first follow-up to the initial
announcement just days ago of the new crc32c
package. The package offers cyclical checksum with parity in
hardware-accelerated form on (recent enough) intel cpus as well as on
arm64.
This follow-up was needed because I missed, when switching to a
default static library build, that newest compilers would
complain if -fPIC was not set. gcc-12 on my
box was happy, gcc-13 on recent Fedora as used at CRAN was
not. A second error was assuming that saying
SystemRequirements: cmake would suffice. But hold on
whippersnapper: macOS always has a surprise for you! As
described at the end of the appropriate section in Writing R
Extensions, on that OS you have to go the basement, open
four cupboards, rearrange three shelves and then you get to use it. And
then in doing so (from an added configure script) I failed
to realize Windows needed a fallback. Gee.
The NEWS entry for this (as well the initial release) follows.
Changes in version 0.0.2
(2023-05-11)
Explicitly set cmake property for
position-independent code
Help macOS find its cmake binary as
detailed also in WRE
Help Windows with a non-conditional Makevars.win pointing at
cmake
Add more badges to README.md
Changes in version 0.0.1
(2023-05-07)
Initial release version and CRAN upload
If you like this or other open-source work I do, you can now sponsor me at
GitHub.
Happy to announce a new package: crc32c. This
arose out of a user request to add crc32c (which is related
to but differnt from crc32 without the trailing c) to my digest
package. Which I did (for now in a branch), using the software-fallback
version of crc32c from the reference implementation by
Google at their crc32c
repo.
However, the Google repo also offers hardware-accelerated versions
and switches at run-time. So I pondered a little about how to offer the
additional performance without placing a dependency burden on all users
of digest.
Lo and behold, I think I found a solution by reusing what R offers.
First off, the crc32c package
wraps the Google repo cleanly and directly. We include all the repo code
but not the logging or benchmarking code. This keeps external
dependencies down to just cmake. Which while still rare in
the CRAN world is at least not entirely uncommon. So now each time you
build the crc32c R
package, the upstream hardware detection is added transparently
thanks in part to cmake. We build libcrc32c.a
as a static library and include it in the R package and its own shared
library.
And we added exporting of three key functions, directly at the C
level, along with exporting one C level function of package that other
packages can call. The distinction matters. The second step of offering
a function R can call (also from other packages) is used and documented.
By means of an Imports: statement to instantiate the
package providing the functionality, the client package obtains access
to a compiled functions its R code can then call. A number of other R
packages use this.
But what is less well known is that we can do the same with C level
functions. Again, it takes an imports statement but once that is done we
can call C to C . Which is quite nice. I am working currently on the
branch in digest which
motivated this, and it can import the automagically
hardware-accelerated functionality on the suitable hardware. Stay
tuned for a change in digest.
I also won and lost the CRAN lottery for once: the package made it
through the new package checks without any revisions. Only to then
immediately fail on the CRAN machines using gcc-13 as a
-fPIC was seen as missing when making the shared library.
Even though both my
CI and the r-universe builds
were all green. Ah well. So a 0.0.2 release will be coming in a day or
two.
If you like this or other open-source work I do, you can now sponsor me at
GitHub.
A new release 0.4.18 of RQuantLib
arrived at CRAN earlier today,
and will be uploaded to Debian as
well.
QuantLib is a very
comprehensice free/open-source library for quantitative
finance; RQuantLib
connects it to the R environment and language.
This release of RQuantLib
comes about six months after the previous maintenance release. It brings
a few small updates triggered by small changes in the QuantLib releases
1.29 and 1.30. It also contains updates reflecting changes in the
rgl package kindly contributed by Duncan Murdoch. Last but
not least, Jeroen Ooms helped with two pull requests updating builds on,
repspectively, macOS and Windows by updating the pre-made libraries of
QuantLib.
Changes in RQuantLib version 0.4.18 (2023-05-01)
Use of several rgl functions was updated to a new
naming scheme in the package (kindly contributed by Duncan Murdoch in #174)
A default argument is now given for option surface plots
Changed call from SwaptionVolCube1 to SabrSwaptionVolatilityCube
(conditional on using QuantLib 1.30 or later)
Some other deprecation warnings were tweaked as in QL test
file
Builds for macOS and Windows were updated with more library build
(changes kindly contributed by Jeron Ooms in #176 and
#175)
Some remaining throw calls were replace by
Rcpp::stop
Rust is a hugely popular compiled programming language and accelerating it was an important goal for Firebuild for some time.
Firebuild s v0.8.0 release finally added Rust support in addition to numerous other improvements including support for Doxygen, Intel s Fortran compiler and restored javac and javadoc acceleration.
Firebuild s Rust + Cargo support
Firebuild treats programs as black boxes intercepting C standard library calls and system calls. It shortcuts the program invocations that predictably generate the same outputs because the program itself is known to be deterministic and all inputs are known in advance. Rust s compiler, rustc is deterministic in itself and simple rustc invocations were already accelerated, but parallel builds driven by Cargo needed a few enhancements in Firebuild.
Cargo s jobserver
Cargo uses the Rust variant of the GNU Make s jobserver to control the parallelism in a build. The jobserver creates a file descriptor from which descendant processes can read tokens and are allowed to run one extra thread or parallel process per token received. After the extra threads or processes are finished the tokens must be returned by writing to the other file descriptor the jobserver created. The jobserver s file descriptors are shared with the descendant processes via environment variables:
Since getting tokens from the jobserver involves reading them as nondeterministic bytes from an inherited file descriptor this is clearly an operation that would depend on input not known in advance. Firebuild needs to make an exception and ignore jobserver usage related reads and writes since they are not meant to change the build results. However, there are programs not caring at all about jobservers and their file descriptors. They happily close the inherited file descriptors and open new ones with the same id, to use them for entirely different purposes. One such program is the widely used ./configure script, thus the case is far from being theoretical.
To stay on the safe side firebuild ignores jobserver fd usage only in programs which are known to use the jobserver properly. The list of the programs is now configurable in /etc/firebuild.conf and since rustc is on the list by default parallel Rust builds are accelerated out of the box!
Writable dependency dir
The other issue that prevented highly accelerated Rust builds was rustc s -L dependency=<dir> parameter. This directory is populated in a not fully deterministic order in parallel builds. Firebuild on the other hand hashes directory listings of open()-ed directories treating them as inputs assuming that the directory content will influence the intercepted programs outputs. As rustc programs started in parallel scanned the dependency directory in different states depending on what other Rust compilations finished already Firebuild had to store the full directory content as an input for each rustc cache entry resulting low hit rate when rustc was started again with otherwise identical inputs.
The solution here is ignoring rustc scanning the dependency directory, because the dependencies actually used are still treated as input and are checked when shortcutting rustc. With that implemented in firebuild, too, librsvg s build that uses Rust and Cargo can be accelerated by more than 90%, even on a system having 12 cores/24 threads!:
Firebuild accelerating librsvg s Rust + Cargo build from 38s to 2.8s on a Ryzen 5900X (12C/24T) system
On the way to accelerate anything
Firebuild s latest release incorporated more than 100 changes just from the last two months. They unlocked acceleration of Rust builds with Cargo, fixed Firebuild to work with the latest Java update that slightly changed its behavior, started accelerating Intel s Fortran compiler in addition to accelerating gfortran that was already supported and included many smaller changes improving the acceleration of other compilers and tools. If your favorite toolchain is not mentioned, there is still a good chance that it is already supported. Give Firebuild a try and tell us about your experience!
Update 1: Comparison to sccache came up in the reddit topic about Firebuild s Rust acceleration , thus by popular demand this is how sccache performs on the same project:
Firebuild 0.8.0 vs. sccache 0.4.2 accelerating librsvg s Rust + Cargo build
All builds took place on the same Ryzen 5900X system with 12 cores / 24 threads in LXC containers limited to using 1-12 virtual CPUs. A warm-up build took place before the vanilla (without any instrumentation) build to download and compile the dependency crates to measure only the project s build time. A git clean command cleared all the build artifacts from the project directory before each build and ./autogen.sh was run to measure only clean rebuilds (without autotools). See test configuration in the Firebuild performance test repository for more details and easy reproduction.
Firebuild had lower overhead than sccache (2.83% vs. 6.10% on 1 CPU and 7.71% vs. 22.05% on 12 CPUs) and made the accelerated build finish much faster (2.26% vs. 19.41% of vanilla build s time on 1 CPU and 7.5% vs. 27.4% of vanilla build s time on 12 CPUs).
The fifth release of the still new-ish qlcal package
arrivied at CRAN just now.
qlcal
delivers the calendaring parts of QuantLib. It is provided (for the R
package) as a set of included files, so the package is self-contained
and does not depend on an external QuantLib library (which can be
demanding to build). qlcal covers
over sixty country / market calendars and can compute holiday lists, its
complement (i.e. business day lists) and much more.
This release brings updates to five calendars from the QuantLib 1.30
release from this week.
Changes in version 0.0.5
(2023-04-19)
Calendars routines for Australia, Denmark, New Zealand, Turkey
and the US have been updated from QuantLib 1.30.
Support for 'Australia/ASX' has been added.
Added demo showing all US holidays in current year
We also added a quick little demo using xts to
column-bind calendars produced from each of the different US
sub-calendars. This is a slightly updated version of the sketch we
tooted a few days ago. The output now is
(and we just discovered a tiny h -> hols
bug in the demo so see the git repo sorry!).
The release was finalized uploaded yesterday morning. But because we
have to set CXX_STD=CXX14 as satisfy requirements of some
of the Boost headers, we get ourselves a NOTE and with that a manual
inspection and a delay of 1 1/2 days. Not really all that meaningful
in the grand scheme of things but still suboptimal relative to the fully
automated passage this release should have gotten. Oh well.
Courtesy of my CRANberries, there
is a diffstat report for this
release. See the project page
and package documentation for more details, and more examples.
If you like this or other open-source work I do, you can now sponsor me at
GitHub.
While I am not the biggest fan of Docker, I must admit it has quite some reach across various service providers and can often be seen as an API for running things in isolated environments.
One such service provider is GitHub when it comes to their Actions service.
I have no idea what isolation technology GitHub uses on the outside of Actions, but inside you just get an Ubuntu system and can run whatever you want via Docker as that comes pre-installed and pre-configured. This especially means you can run things inside vanilla Debian containers, that are free from any GitHub or Canonical modifications one might not want ;-)
So, if you want to run, say, lintian from sid, you can define a job to do so:
lintian:runs-on:ubuntu-latestcontainer:debian:sidsteps:-[do something to get a package to run lintian on]-run:apt-get update-run:apt-get install -y --no-install-recommends lintian-run:lintian --info --display-info *.changes
This will run on Ubuntu (latest right now means 22.04 for GitHub), but then use Docker to run the debian:sid container and execute all further steps inside it.
Pretty short and straight forward, right?
Now lintian does static analysis of the package, it doesn't need to install it.
What if we want to run autopkgtest that performs tests on an actually installed package?
autopkgtest comes with various "virt servers", which are providing isolation of the testbed, so that it does not interfere with the host system.
The simplest available virt server, autopkgtest-virt-null doesn't actually provide any isolation, as it runs things directly on the host system.
This might seem fine when executed inside an ephemeral container in an CI environment, but it also means that multiple tests have the potential to influence each other as there is no way to revert the testbed to a clean state.
For that, there are other, "real", virt servers available: chroot, lxc, qemu, docker and many more.
They all have one in common: to use them, one needs to somehow provide an "image" (a prepared chroot, a tarball of a chroot, a vm disk, a container, , you get it) to operate on and most either bring a tool to create such an "image" or rely on a "registry" (online repository) to provide them.
Most users of autopkgtest on GitHub (that I could find with their terrible search) are using either the null or the lxd virt servers. Probably because these are dead simple to set up (null) or the most "native" (lxd) in the Ubuntu environment.
As I wanted to execute multiple tests that for sure would interfere with each other, the null virt server was out of the discussion pretty quickly.
The lxd one also felt odd, as that meant I'd need to set up lxd (can be done in a few commands, but still) and it would need to download stuff from Canonical, incurring costs (which I couldn't care less about) and taking time which I do care about!).
Enter autopkgtest-virt-docker, which recentlywas added to autopkgtest! No need to set things up, as GitHub already did all the Docker setup for me, and almost no waiting time to download the containers, as GitHub does heavy caching of stuff coming from Docker Hub (or at least it feels like that).
The only drawback? It was added in autopkgtest 5.23, which Ubuntu 22.04 doesn't have.
"We need to go deeper" and run autopkgtest from a sid container!
With this idea, our current job definition would look like this:
autopkgtest:runs-on:ubuntu-latestcontainer:debian:sidsteps:-[do something to get a package to run autopkgtest on]-run:apt-get update-run:apt-get install -y --no-install-recommends autopkgtest autodep8 docker.io-run:autopkgtest *.changes --setup-commands="apt-get update" -- docker debian:sid
(--setup-commands="apt-get update" is needed as the container comes with an empty apt cache and wouldn't be able to find dependencies of the tested package)
However, this will fail:
# autopkgtest*.changes--setup-commands="apt-get update"--dockerdebian:sid
autopkgtest [10:20:54]: starting date and time: 2023-04-07 10:20:54+0000autopkgtest [10:20:54]: version 5.28autopkgtest [10:20:54]: host a82a11789c0d; command line: /usr/bin/autopkgtest bley_2.0.0-1_amd64.changes '--setup-commands=apt-get update' -- docker debian:sidUnexpected error:Traceback (most recent call last): File "/usr/share/autopkgtest/lib/VirtSubproc.py", line 829, in mainloop command() File "/usr/share/autopkgtest/lib/VirtSubproc.py", line 758, in command r = f(c, ce) ^^^^^^^^ File "/usr/share/autopkgtest/lib/VirtSubproc.py", line 692, in cmd_copydown copyupdown(c, ce, False) File "/usr/share/autopkgtest/lib/VirtSubproc.py", line 580, in copyupdown copyupdown_internal(ce[0], c[1:], upp) File "/usr/share/autopkgtest/lib/VirtSubproc.py", line 607, in copyupdown_internal copydown_shareddir(sd[0], sd[1], dirsp, downtmp_host) File "/usr/share/autopkgtest/lib/VirtSubproc.py", line 562, in copydown_shareddir shutil.copy(host, host_tmp) File "/usr/lib/python3.11/shutil.py", line 419, in copy copyfile(src, dst, follow_symlinks=follow_symlinks) File "/usr/lib/python3.11/shutil.py", line 258, in copyfile with open(dst, 'wb') as fdst: ^^^^^^^^^^^^^^^FileNotFoundError: [Errno 2] No such file or directory: '/tmp/autopkgtest-virt-docker.shared.kn7n9ioe/downtmp/wrapper.sh'autopkgtest [10:21:07]: ERROR: testbed failure: unexpected eof from the testbed
Running the same thing locally of course works, so there has to be something special about the setup at GitHub. But what?!
A bit of digging revealed that autopkgtest-virt-docker tries to use a shared directory (using Dockers --volume) to exchange things with the testbed (for the downtmp-host capability). As my autopkgtest is running inside a container itself, nothing it tells the Docker deamon to mount will be actually visible to it.
In retrospect this makes total sense and autopkgtest-virt-docker has a switch to "fix" the issue: --remote as the Docker deamon is technically remote when viewed from the place autopkgtest runs at.
I'd argue this is not a bug in autopkgtest(-virt-docker), as the situation is actually cared for. There is even some auto-detection of "remote" daemons in the code, but that doesn't "know" how to detect the case where the daemon socket is mounted (vs being set as an environment variable). I've opened an MR (assume remote docker when running inside docker) which should detect the case of running inside a Docker container which kind of implies the daemon is remote.
Not sure the patch will be accepted (it is a band-aid after all), but in the meantime I am quite happy with using --remote and so could you ;-)
Armadillo is a powerful
and expressive C++ template library for linear algebra and scientific
computing. It aims towards a good balance between speed and ease of use,
has a syntax deliberately close to Matlab, and is useful for algorithm
development directly in C++, or quick conversion of research code into
production environments. RcppArmadillo
integrates this library with the R environment and language and is
widely used by (currently) 1052 other packages on CRAN, downloaded 28.6 million
times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint
/ vignette) by Conrad and myself has been cited 522 times according
to Google Scholar.
This release brings a new upstream release 12.2.0 made by Conrad a
day or so ago. We prepared the usual release candidate, tested on the
over 1000 reverse depends, found no issues and sent it to CRAN. Where it
got tested again and was auto-processed smoothly by CRAN.
The releases actually has a relatively small set of changes as a
first follow-up release in the 12.2.* series.
Changes
in RcppArmadillo version 0.12.2.0.0 (2023-04-04)
Upgraded to Armadillo release 12.2.0 (Cortisol Profusion
Deluxe)
more efficient use of FFTW3 by fft() and
ifft()
faster in-place element-wise multiplication of sparse matrices by
dense matrices
added spsolve_factoriser class to allow reuse of sparse matrix
factorisation for solving systems of linear equations
Review: The Nordic Theory of Everything, by Anu Partanen
Publisher:
Harper
Copyright:
2016
Printing:
June 2017
ISBN:
0-06-231656-7
Format:
Kindle
Pages:
338
Anu Partanen is a Finnish journalist who immigrated to the United States.
The Nordic Theory of Everything, subtitled In Search of a
Better Life, is an attempt to explain the merits of Finnish approaches to
government and society to a US audience. It was her first book.
If you follow US policy discussion at all, you have probably been exposed
to many of the ideas in this book. There was a time when the US left was
obsessed with comparisons between the US and Nordic countries, and while
that obsession has faded somewhat, Nordic social systems are still
discussed with envy and treated as a potential model. Many of the topics
of this book are therefore predictable: parental leave, vacation, health
care, education, happiness, life expectancy, all the things that are far
superior in Nordic countries than in the United States by essentially
every statistical measure available, and which have been much-discussed.
Partanen brings two twists to this standard analysis. The first is that
this book is part memoir: she fell in love with a US writer and made the
decision to move to the US rather than asking him to move to Finland. She
therefore experienced the transition between social and government systems
first-hand and writes memorably on the resulting surprise, trade-offs,
anxiety, and bafflement. The second, which I've not seen previously in
this policy debate, is a fascinating argument that Finland is a far more
individualistic country than the United States precisely because of its
policy differences.
Most people, including myself, assumed that part of what made the
United States a great country, and such an exceptional one, was that
you could live your life relatively unencumbered by the downside of a
traditional, old-fashioned society: dependency on the people you
happened to be stuck with. In America you had the liberty to express
your individuality and choose your own community. This would allow
you to interact with family, neighbors, and fellow citizens on the
basis of who you were, rather than on what you were obligated to do or
expected to be according to old-fashioned thinking.
The longer I lived in America, therefore, and the more places I
visited and the more people I met and the more American I myself
became the more puzzled I grew. For it was exactly those key
benefits of modernity freedom, personal independence, and
opportunity that seemed, from my outsider s perspective, in a
thousand small ways to be surprisingly missing from American life
today. Amid the anxiety and stress of people s daily lives, those
grand ideals were looking more theoretical than actual.
The core of this argument is that the structure of life in the United
States essentially coerces dependency on other people: employers, spouses,
parents, children, and extended family. Because there is no universally
available social support system, those relationships become essential for
any hope of a good life, and often for survival. If parents do not
heavily manage their children's education, there is a substantial risk of
long-lasting damage to the stability and happiness of their life. If
children do not care for their elderly parents, they may receive no care
at all. Choosing not to get married often means choosing precarity and
exhaustion because navigating society without pooling resources with
someone else is incredibly difficult.
It was as if America, land of the Hollywood romance, was in practice
mired in a premodern time when marriage was, first and foremost, not
an expression of love, but rather a logistical and financial pact to
help families survive by joining resources.
Partanen contrasts this with what she calls the Nordic theory of love:
What Lars Tr g rdh came to understand during his years in the United
States was that the overarching ambition of Nordic societies during
the course of the twentieth century, and into the twenty-first, has
not been to socialize the economy at all, as is often mistakenly
assumed. Rather the goal has been to free the individual from all
forms of dependency within the family and in civil society: the poor
from charity, wives from husbands, adult children from parents, and
elderly parents from their children. The express purpose of this
freedom is to allow all those human relationships to be unencumbered
by ulterior motives and needs, and thus to be entirely free,
completely authentic, and driven purely by love.
She sees this as the common theme through most of the policy differences
discussed in this book. The Finnish approach is to provide neutral and
universal logistical support for most of life's expected challenges:
birth, child-rearing, education, health, unemployment, and aging. This
relieves other social relations family, employer, church of the
corrosive strain of dependency and obligation. It also ensures people's
basic well-being isn't reliant on accidents of association.
If the United States is so worried about crushing entrepreneurship and
innovation, a good place to start would be freeing start-ups and
companies from the burdens of babysitting the nation s citizens.
I found this fascinating as a persuasive technique. Partanen embraces the
US ideal of individualism and points out that, rather than being
collectivist as the US right tends to assume, Finland is better at
fostering individualism and independence because the government works to
removes unnecessary premodern constraints on individual lives. The reason
why so many Americans are anxious and frantic is not a personal failing or
bad luck. It's because the US social system is deeply hostile to healthy
relationships and individual independence. It demands a constant level of
daily problem-solving and crisis management that is profoundly exhausting,
nearly impossible to navigate alone, and damaging to the ideal of equal
relationships.
Whether this line of argument will work is another question, and I'm
dubious for reasons that Partanen (probably wisely) avoids. She presents
the Finnish approach as a discovery that the US would benefit from, and
the US approach as a well-intentioned mistake. I think this is
superficially appealing; almost all corners of US political belief at
least give lip service to individualism and independence. However,
advocates of political change will eventually need to address the fact
that many US conservatives see this type of social coercion as an intended
feature of society rather than a flaw.
This is most obvious when one looks at family relationships. Partanen
treats the idea that marriage should be a free choice between equals
rather than an economic necessity as self-evident, but there is a
significant strain of US political thought that embraces punishing people
for not staying within the bounds of a conservative ideal of family. One
will often find, primarily but not exclusively among the more religious, a
contention that the basic unit of society is the (heterosexual,
patriarchal) family, not the individual, and that the suffering of anyone
outside that structure is their own fault. Not wanting to get married, be
the primary caregiver for one's parents, or abandon a career in order to
raise children is treated as malignant selfishness and immorality rather
than a personal choice that can be enabled by a modern social system.
Here, I think Partanen is accurate to identify the Finnish social system
as more modern. It embraces the philosophical concept of modernity,
namely that social systems can be improved and social structures are not
timeless. This is going to be a hard argument to swallow for those who
see the pressure towards forming dependency ties within families as
natural, and societal efforts to relieve those pressures as government
meddling. In that intellectual framework, rather than an attempt to
improve the quality of life, government logistical support is perceived as
hostility to traditional family obligations and an attempt to replace
"natural" human ties with "artificial" dependence on government services.
Partanen doesn't attempt to have that debate.
Two other things struck me in this book. The first is that, in Partanen's
presentation, Finns expect high-quality services from their government and
work to improve it when it falls short. This sounds like an obvious
statement, but I don't think it is in the context of US politics, and
neither does Partanen. She devotes a chapter to the topic, subtitled "Go
ahead: ask what your country can do for you."
This is, to me, one of the most frustrating aspects of US political
debate. Our attitude towards government is almost entirely hostile and
negative even among the political corners that would like to see
government do more. Failures of government programs are treated as
malice, malfeasance, or inherent incompetence: in short, signs the program
should never have been attempted, rather than opportunities to learn and
improve. Finland had mediocre public schools, decided to make them
better, and succeeded. The moment US public schools start deteriorating,
we throw much of our effort into encouraging private competition and
dismantling the public school system.
Partanen doesn't draw this connection, but I see a link between the US
desire for market solutions to societal problems and the level of
exhaustion and anxiety that is so common in US life. Solving problems by
throwing them open to competition is a way of giving up, of saying we have
no idea how to improve something and are hoping someone else will figure
it out for a profit. Analyzing the failures of an existing system and
designing incremental improvements is hard and slow work. Throwing out
the system and hoping some corporation will come up with something better
is disruptive but easy.
When everyone is already overwhelmed by life and devoid of energy to
work on complex social problems, it's tempting to give up on compromise
and coalition-building and let everyone go their separate ways on their
own dime. We cede the essential work of designing a good society to
start-ups. This creates a vicious cycle: the resulting market solutions
are inevitably gated by wealth and thus precarious and artificially
scarce, which in turn creates more anxiety and stress. The short-term
energy savings from not having to wrestle with a hard problem is
overwhelmed by the long-term cost of having to navigate a complex and
adversarial economic relationship.
That leads into the last point: schools. There's a lot of discussion here
about school quality and design, which I won't review in detail but which
is worth reading. What struck me about Partanen's discussion, though, is
how easy the Finnish system is to use. Finnish parents just send their
kids to the most convenient school and rarely give that a second thought.
The critical property is that all the schools are basically fine, and
therefore there is no need to place one's child in an exceptional school
to ensure they have a good life.
It's axiomatic in the US that more choice is better. This is a constant
refrain in our political discussion around schools: parental choice,
parental control, options, decisions, permission, matching children to
schools tailored for their needs. Those choices are almost entirely
absent in Finland, at least in Partanen's description, and the amount of
mental and emotional energy this saves is astonishing. Parents simply
don't think about this, and everything is fine.
I think we dramatically underestimate the negative effects of constantly
having to make difficult decisions with significant consequences, and
drastically overstate the benefits of having every aspect of life be full
of major decision points. To let go of that attempt at control, however
illusory, people have to believe in a baseline of quality that makes the
choice less fraught. That's precisely what Finland provides by expecting
high-quality social services and working to fix them when they fall short,
an effort that the United States has by and large abandoned.
A lot of non-fiction books could be turned into long articles without
losing much substance, and I think The Nordic Theory of Everything
falls partly into that trap. Partanen repeats the same ideas from several
different angles, and the book felt a bit padded towards the end. If
you're already familiar with the policy comparisons between the US and
Nordic countries, you will have seen a lot of this before, and the book
bogs down when Partanen strays too far from memoir and personal reactions.
But the focus on individualism and eliminating dependency is new, at least
to me, and is such an illuminating way to look at the contrast that I
think the book is worth reading just for that.
Rating: 7 out of 10
I ve used hardware-backed OpenPGP keys since 2006 when I imported newly generated rsa1024 subkeys to a FSFE Fellowship card. This worked well for several years, and I recall buying more ZeitControl cards for multi-machine usage and backup purposes. As a side note, I recall being unsatisfied with the weak 1024-bit RSA subkeys at the time my primary key was a somewhat stronger 1280-bit RSA key created back in 2002 but OpenPGP cards at the time didn t support more than 1024 bit RSA, and were (and still often are) also limited to power-of-two RSA key sizes which I dislike.
I had my master key on disk with a strong password for a while, mostly to refresh expiration time of the subkeys and to sign other s OpenPGP keys. At some point I stopped carrying around encrypted copies of my master key. That was my main setup when I migrated to a new stronger RSA 3744 bit key with rsa2048 subkeys on a YubiKey NEO back in 2014. At that point, signing other s OpenPGP keys was a rare enough occurrence that I settled with bringing out my offline machine to perform this operation, transferring the public key to sign on USB sticks. In 2019 I re-evaluated my OpenPGP setup and ended up creating a offline Ed25519 key with subkeys on a FST-01G running Gnuk. My approach for signing other s OpenPGP keys were still to bring out my offline machine and sign things using the master secret using USB sticks for storage and transport. Which meant I almost never did that, because it took too much effort. So my 2019-era Ed25519 key still only has a handful of signatures on it, since I had essentially stopped signing other s keys which is the traditional way of getting signatures in return.
None of this caused any critical problem for me because I continued to use my old 2014-era RSA3744 key in parallel with my new 2019-era Ed25519 key, since too many systems didn t handle Ed25519. However, during 2022 this changed, and the only remaining environment that I still used my RSA3744 key for was in Debian and they require OpenPGP signatures on the new key to allow it to replace an older key. I was in denial about this sub-optimal solution during 2022 and endured its practical consequences, having to use the YubiKey NEO (which I had replaced with a permanently inserted YubiKey Nano at some point) for Debian-related purposes alone.
In December 2022 I bought a new laptop and setup a FST-01SZ with my Ed25519 key, and while I have taken a vacation from Debian, I continue to extend the expiration period on the old RSA3744-key in case I will ever have to use it again, so the overall OpenPGP setup was still sub-optimal. Having two valid OpenPGP keys at the same time causes people to use both for email encryption (leading me to have to use both devices), and the WKD Key Discovery protocol doesn t like two valid keys either. At FOSDEM 23 I ran into Andre Heinecke at GnuPG and I couldn t help complain about how complex and unsatisfying all OpenPGP-related matters were, and he mildly ignored my rant and asked why I didn t put the master key on another smartcard. The comment sunk in when I came home, and recently I connected all the dots and this post is a summary of what I did to move my offline OpenPGP master key to a Nitrokey Start.
First a word about device choice, I still prefer to use hardware devices that are as compatible with free software as possible, but the FST-01G or FST-01SZ are no longer easily available for purchase. I got a comment about Nitrokey start in my last post, and had two of them available to experiment with. There are things to dislike with the Nitrokey Start compared to the YubiKey (e.g., relative insecure chip architecture, the bulkier form factor and lack of FIDO/U2F/OATH support) but as far as I know there is no more widely available owner-controlled device that is manufactured for an intended purpose of implementing an OpenPGP card. Thus it hits the sweet spot for me.
Nitrokey Start
The first step is to run latest firmware on the Nitrokey Start for bug-fixes and important OpenSSH 9.0 compatibility and there are reproducible-built firmware published that you can install using pynitrokey. I run Trisquel 11 aramo on my laptop, which does not include the Python Pip package (likely because it promotes installing non-free software) so that was a slight complication. Building the firmware locally may have worked, and I would like to do that eventually to confirm the published firmware, however to save time I settled with installing the Ubuntu 22.04 packages on my machine:
$ sha256sum python3-pip*
ded6b3867a4a4cbaff0940cab366975d6aeecc76b9f2d2efa3deceb062668b1c python3-pip_22.0.2+dfsg-1ubuntu0.2_all.deb
e1561575130c41dc3309023a345de337e84b4b04c21c74db57f599e267114325 python3-pip-whl_22.0.2+dfsg-1ubuntu0.2_all.deb
$ doas dpkg -i python3-pip*
...
$ doas apt install -f
...
$
Installing pynitrokey downloaded a bunch of dependencies, and it would be nice to audit the license and security vulnerabilities for each of them. (Verbose output below slightly redacted.)
jas@kaka:~$ pip3 install --user pynitrokey
Collecting pynitrokey
Downloading pynitrokey-0.4.34-py3-none-any.whl (572 kB)
Collecting frozendict~=2.3.4
Downloading frozendict-2.3.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (113 kB)
Requirement already satisfied: click<9,>=8.0.0 in /usr/lib/python3/dist-packages (from pynitrokey) (8.0.3)
Collecting ecdsa
Downloading ecdsa-0.18.0-py2.py3-none-any.whl (142 kB)
Collecting python-dateutil~=2.7.0
Downloading python_dateutil-2.7.5-py2.py3-none-any.whl (225 kB)
Collecting fido2<2,>=1.1.0
Downloading fido2-1.1.0-py3-none-any.whl (201 kB)
Collecting tlv8
Downloading tlv8-0.10.0.tar.gz (16 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: certifi>=14.5.14 in /usr/lib/python3/dist-packages (from pynitrokey) (2020.6.20)
Requirement already satisfied: pyusb in /usr/lib/python3/dist-packages (from pynitrokey) (1.2.1.post1)
Collecting urllib3~=1.26.7
Downloading urllib3-1.26.15-py2.py3-none-any.whl (140 kB)
Collecting spsdk<1.8.0,>=1.7.0
Downloading spsdk-1.7.1-py3-none-any.whl (684 kB)
Collecting typing_extensions~=4.3.0
Downloading typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Requirement already satisfied: cryptography<37,>=3.4.4 in /usr/lib/python3/dist-packages (from pynitrokey) (3.4.8)
Collecting intelhex
Downloading intelhex-2.3.0-py2.py3-none-any.whl (50 kB)
Collecting nkdfu
Downloading nkdfu-0.2-py3-none-any.whl (16 kB)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from pynitrokey) (2.25.1)
Collecting tqdm
Downloading tqdm-4.65.0-py3-none-any.whl (77 kB)
Collecting nrfutil<7,>=6.1.4
Downloading nrfutil-6.1.7.tar.gz (845 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: cffi in /usr/lib/python3/dist-packages (from pynitrokey) (1.15.0)
Collecting crcmod
Downloading crcmod-1.7.tar.gz (89 kB)
Preparing metadata (setup.py) ... done
Collecting libusb1==1.9.3
Downloading libusb1-1.9.3-py3-none-any.whl (60 kB)
Collecting pc_ble_driver_py>=0.16.4
Downloading pc_ble_driver_py-0.17.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB)
Collecting piccata
Downloading piccata-2.0.3-py3-none-any.whl (21 kB)
Collecting protobuf<4.0.0,>=3.17.3
Downloading protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)
Collecting pyserial
Downloading pyserial-3.5-py2.py3-none-any.whl (90 kB)
Collecting pyspinel>=1.0.0a3
Downloading pyspinel-1.0.3.tar.gz (58 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (from nrfutil<7,>=6.1.4->pynitrokey) (5.4.1)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil~=2.7.0->pynitrokey) (1.16.0)
Collecting pylink-square<0.11.9,>=0.8.2
Downloading pylink_square-0.11.1-py2.py3-none-any.whl (78 kB)
Collecting jinja2<3.1,>=2.11
Downloading Jinja2-3.0.3-py3-none-any.whl (133 kB)
Collecting bincopy<17.11,>=17.10.2
Downloading bincopy-17.10.3-py3-none-any.whl (17 kB)
Collecting fastjsonschema>=2.15.1
Downloading fastjsonschema-2.16.3-py3-none-any.whl (23 kB)
Collecting astunparse<2,>=1.6
Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting oscrypto~=1.2
Downloading oscrypto-1.3.0-py2.py3-none-any.whl (194 kB)
Collecting deepmerge==0.3.0
Downloading deepmerge-0.3.0-py2.py3-none-any.whl (7.6 kB)
Collecting pyocd<=0.31.0,>=0.28.3
Downloading pyocd-0.31.0-py3-none-any.whl (12.5 MB)
Collecting click-option-group<0.6,>=0.3.0
Downloading click_option_group-0.5.5-py3-none-any.whl (12 kB)
Collecting pycryptodome<4,>=3.9.3
Downloading pycryptodome-3.17-cp35-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
Collecting pyocd-pemicro<1.2.0,>=1.1.1
Downloading pyocd_pemicro-1.1.5-py3-none-any.whl (9.0 kB)
Requirement already satisfied: colorama<1,>=0.4.4 in /usr/lib/python3/dist-packages (from spsdk<1.8.0,>=1.7.0->pynitrokey) (0.4.4)
Collecting commentjson<1,>=0.9
Downloading commentjson-0.9.0.tar.gz (8.7 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: asn1crypto<2,>=1.2 in /usr/lib/python3/dist-packages (from spsdk<1.8.0,>=1.7.0->pynitrokey) (1.4.0)
Collecting pypemicro<0.2.0,>=0.1.9
Downloading pypemicro-0.1.11-py3-none-any.whl (5.7 MB)
Collecting libusbsio>=2.1.11
Downloading libusbsio-2.1.11-py3-none-any.whl (247 kB)
Collecting sly==0.4
Downloading sly-0.4.tar.gz (60 kB)
Preparing metadata (setup.py) ... done
Collecting ruamel.yaml<0.18.0,>=0.17
Downloading ruamel.yaml-0.17.21-py3-none-any.whl (109 kB)
Collecting cmsis-pack-manager<0.3.0
Downloading cmsis_pack_manager-0.2.10-py2.py3-none-manylinux1_x86_64.whl (25.1 MB)
Collecting click-command-tree==1.1.0
Downloading click_command_tree-1.1.0-py3-none-any.whl (3.6 kB)
Requirement already satisfied: bitstring<3.2,>=3.1 in /usr/lib/python3/dist-packages (from spsdk<1.8.0,>=1.7.0->pynitrokey) (3.1.7)
Collecting hexdump~=3.3
Downloading hexdump-3.3.zip (12 kB)
Preparing metadata (setup.py) ... done
Collecting fire
Downloading fire-0.5.0.tar.gz (88 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/lib/python3/dist-packages (from astunparse<2,>=1.6->spsdk<1.8.0,>=1.7.0->pynitrokey) (0.37.1)
Collecting humanfriendly
Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
Collecting argparse-addons>=0.4.0
Downloading argparse_addons-0.12.0-py3-none-any.whl (3.3 kB)
Collecting pyelftools
Downloading pyelftools-0.29-py2.py3-none-any.whl (174 kB)
Collecting milksnake>=0.1.2
Downloading milksnake-0.1.5-py2.py3-none-any.whl (9.6 kB)
Requirement already satisfied: appdirs>=1.4 in /usr/lib/python3/dist-packages (from cmsis-pack-manager<0.3.0->spsdk<1.8.0,>=1.7.0->pynitrokey) (1.4.4)
Collecting lark-parser<0.8.0,>=0.7.1
Downloading lark-parser-0.7.8.tar.gz (276 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib/python3/dist-packages (from jinja2<3.1,>=2.11->spsdk<1.8.0,>=1.7.0->pynitrokey) (2.0.1)
Collecting asn1crypto<2,>=1.2
Downloading asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)
Collecting wrapt
Downloading wrapt-1.15.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB)
Collecting future
Downloading future-0.18.3.tar.gz (840 kB)
Preparing metadata (setup.py) ... done
Collecting psutil>=5.2.2
Downloading psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB)
Collecting capstone<5.0,>=4.0
Downloading capstone-4.0.2-py2.py3-none-manylinux1_x86_64.whl (2.1 MB)
Collecting naturalsort<2.0,>=1.5
Downloading naturalsort-1.5.1.tar.gz (7.4 kB)
Preparing metadata (setup.py) ... done
Collecting prettytable<3.0,>=2.0
Downloading prettytable-2.5.0-py3-none-any.whl (24 kB)
Collecting intervaltree<4.0,>=3.0.2
Downloading intervaltree-3.1.0.tar.gz (32 kB)
Preparing metadata (setup.py) ... done
Collecting ruamel.yaml.clib>=0.2.6
Downloading ruamel.yaml.clib-0.2.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (485 kB)
Collecting termcolor
Downloading termcolor-2.2.0-py3-none-any.whl (6.6 kB)
Collecting sortedcontainers<3.0,>=2.0
Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Requirement already satisfied: wcwidth in /usr/lib/python3/dist-packages (from prettytable<3.0,>=2.0->pyocd<=0.31.0,>=0.28.3->spsdk<1.8.0,>=1.7.0->pynitrokey) (0.2.5)
Building wheels for collected packages: nrfutil, crcmod, sly, tlv8, commentjson, hexdump, pyspinel, fire, intervaltree, lark-parser, naturalsort, future
Building wheel for nrfutil (setup.py) ... done
Created wheel for nrfutil: filename=nrfutil-6.1.7-py3-none-any.whl size=898520 sha256=de6f8803f51d6c26d24dc7df6292064a468ff3f389d73370433fde5582b84a10
Stored in directory: /home/jas/.cache/pip/wheels/39/2b/9b/98ab2dd716da746290e6728bdb557b14c1c9a54cb9ed86e13b
Building wheel for crcmod (setup.py) ... done
Created wheel for crcmod: filename=crcmod-1.7-cp310-cp310-linux_x86_64.whl size=31422 sha256=5149ac56fcbfa0606760eef5220fcedc66be560adf68cf38c604af3ad0e4a8b0
Stored in directory: /home/jas/.cache/pip/wheels/85/4c/07/72215c529bd59d67e3dac29711d7aba1b692f543c808ba9e86
Building wheel for sly (setup.py) ... done
Created wheel for sly: filename=sly-0.4-py3-none-any.whl size=27352 sha256=f614e413918de45c73d1e9a8dca61ca07dc760d9740553400efc234c891f7fde
Stored in directory: /home/jas/.cache/pip/wheels/a2/23/4a/6a84282a0d2c29f003012dc565b3126e427972e8b8157ea51f
Building wheel for tlv8 (setup.py) ... done
Created wheel for tlv8: filename=tlv8-0.10.0-py3-none-any.whl size=11266 sha256=3ec8b3c45977a3addbc66b7b99e1d81b146607c3a269502b9b5651900a0e2d08
Stored in directory: /home/jas/.cache/pip/wheels/e9/35/86/66a473cc2abb0c7f21ed39c30a3b2219b16bd2cdb4b33cfc2c
Building wheel for commentjson (setup.py) ... done
Created wheel for commentjson: filename=commentjson-0.9.0-py3-none-any.whl size=12092 sha256=28b6413132d6d7798a18cf8c76885dc69f676ea763ffcb08775a3c2c43444f4a
Stored in directory: /home/jas/.cache/pip/wheels/7d/90/23/6358a234ca5b4ec0866d447079b97fedf9883387d1d7d074e5
Building wheel for hexdump (setup.py) ... done
Created wheel for hexdump: filename=hexdump-3.3-py3-none-any.whl size=8913 sha256=79dfadd42edbc9acaeac1987464f2df4053784fff18b96408c1309b74fd09f50
Stored in directory: /home/jas/.cache/pip/wheels/26/28/f7/f47d7ecd9ae44c4457e72c8bb617ef18ab332ee2b2a1047e87
Building wheel for pyspinel (setup.py) ... done
Created wheel for pyspinel: filename=pyspinel-1.0.3-py3-none-any.whl size=65033 sha256=01dc27f81f28b4830a0cf2336dc737ef309a1287fcf33f57a8a4c5bed3b5f0a6
Stored in directory: /home/jas/.cache/pip/wheels/95/ec/4b/6e3e2ee18e7292d26a65659f75d07411a6e69158bb05507590
Building wheel for fire (setup.py) ... done
Created wheel for fire: filename=fire-0.5.0-py2.py3-none-any.whl size=116951 sha256=3d288585478c91a6914629eb739ea789828eb2d0267febc7c5390cb24ba153e8
Stored in directory: /home/jas/.cache/pip/wheels/90/d4/f7/9404e5db0116bd4d43e5666eaa3e70ab53723e1e3ea40c9a95
Building wheel for intervaltree (setup.py) ... done
Created wheel for intervaltree: filename=intervaltree-3.1.0-py2.py3-none-any.whl size=26119 sha256=5ff1def22ba883af25c90d90ef7c6518496fcd47dd2cbc53a57ec04cd60dc21d
Stored in directory: /home/jas/.cache/pip/wheels/fa/80/8c/43488a924a046b733b64de3fac99252674c892a4c3801c0a61
Building wheel for lark-parser (setup.py) ... done
Created wheel for lark-parser: filename=lark_parser-0.7.8-py2.py3-none-any.whl size=62527 sha256=3d2ec1d0f926fc2688d40777f7ef93c9986f874169132b1af590b6afc038f4be
Stored in directory: /home/jas/.cache/pip/wheels/29/30/94/33e8b58318aa05cb1842b365843036e0280af5983abb966b83
Building wheel for naturalsort (setup.py) ... done
Created wheel for naturalsort: filename=naturalsort-1.5.1-py3-none-any.whl size=7526 sha256=bdecac4a49f2416924548cae6c124c85d5333e9e61c563232678ed182969d453
Stored in directory: /home/jas/.cache/pip/wheels/a6/8e/c9/98cfa614fff2979b457fa2d9ad45ec85fa417e7e3e2e43be51
Building wheel for future (setup.py) ... done
Created wheel for future: filename=future-0.18.3-py3-none-any.whl size=492037 sha256=57a01e68feca2b5563f5f624141267f399082d2f05f55886f71b5d6e6cf2b02c
Stored in directory: /home/jas/.cache/pip/wheels/5e/a9/47/f118e66afd12240e4662752cc22cefae5d97275623aa8ef57d
Successfully built nrfutil crcmod sly tlv8 commentjson hexdump pyspinel fire intervaltree lark-parser naturalsort future
Installing collected packages: tlv8, sortedcontainers, sly, pyserial, pyelftools, piccata, naturalsort, libusb1, lark-parser, intelhex, hexdump, fastjsonschema, crcmod, asn1crypto, wrapt, urllib3, typing_extensions, tqdm, termcolor, ruamel.yaml.clib, python-dateutil, pyspinel, pypemicro, pycryptodome, psutil, protobuf, prettytable, oscrypto, milksnake, libusbsio, jinja2, intervaltree, humanfriendly, future, frozendict, fido2, ecdsa, deepmerge, commentjson, click-option-group, click-command-tree, capstone, astunparse, argparse-addons, ruamel.yaml, pyocd-pemicro, pylink-square, pc_ble_driver_py, fire, cmsis-pack-manager, bincopy, pyocd, nrfutil, nkdfu, spsdk, pynitrokey
WARNING: The script nitropy is installed in '/home/jas/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed argparse-addons-0.12.0 asn1crypto-1.5.1 astunparse-1.6.3 bincopy-17.10.3 capstone-4.0.2 click-command-tree-1.1.0 click-option-group-0.5.5 cmsis-pack-manager-0.2.10 commentjson-0.9.0 crcmod-1.7 deepmerge-0.3.0 ecdsa-0.18.0 fastjsonschema-2.16.3 fido2-1.1.0 fire-0.5.0 frozendict-2.3.5 future-0.18.3 hexdump-3.3 humanfriendly-10.0 intelhex-2.3.0 intervaltree-3.1.0 jinja2-3.0.3 lark-parser-0.7.8 libusb1-1.9.3 libusbsio-2.1.11 milksnake-0.1.5 naturalsort-1.5.1 nkdfu-0.2 nrfutil-6.1.7 oscrypto-1.3.0 pc_ble_driver_py-0.17.0 piccata-2.0.3 prettytable-2.5.0 protobuf-3.20.3 psutil-5.9.4 pycryptodome-3.17 pyelftools-0.29 pylink-square-0.11.1 pynitrokey-0.4.34 pyocd-0.31.0 pyocd-pemicro-1.1.5 pypemicro-0.1.11 pyserial-3.5 pyspinel-1.0.3 python-dateutil-2.7.5 ruamel.yaml-0.17.21 ruamel.yaml.clib-0.2.7 sly-0.4 sortedcontainers-2.4.0 spsdk-1.7.1 termcolor-2.2.0 tlv8-0.10.0 tqdm-4.65.0 typing_extensions-4.3.0 urllib3-1.26.15 wrapt-1.15.0
jas@kaka:~$
Then upgrading the device worked remarkable well, although I wish that the tool would have printed URLs and checksums for the firmware files to allow easy confirmation.
jas@kaka:~$ PATH=$PATH:/home/jas/.local/bin
jas@kaka:~$ nitropy start list
Command line tool to interact with Nitrokey devices 0.4.34
:: 'Nitrokey Start' keys:
FSIJ-1.2.15-5D271572: Nitrokey Nitrokey Start (RTM.12.1-RC2-modified)
jas@kaka:~$ nitropy start update
Command line tool to interact with Nitrokey devices 0.4.34
Nitrokey Start firmware update tool
Platform: Linux-5.15.0-67-generic-x86_64-with-glibc2.35
System: Linux, is_linux: True
Python: 3.10.6
Saving run log to: /tmp/nitropy.log.gc5753a8
Admin PIN:
Firmware data to be used:
- FirmwareType.REGNUAL: 4408, hash: ...b'72a30389' valid (from ...built/RTM.13/regnual.bin)
- FirmwareType.GNUK: 129024, hash: ...b'25a4289b' valid (from ...prebuilt/RTM.13/gnuk.bin)
Currently connected device strings:
Device:
Vendor: Nitrokey
Product: Nitrokey Start
Serial: FSIJ-1.2.15-5D271572
Revision: RTM.12.1-RC2-modified
Config: *:*:8e82
Sys: 3.0
Board: NITROKEY-START-G
initial device strings: [ 'name': '', 'Vendor': 'Nitrokey', 'Product': 'Nitrokey Start', 'Serial': 'FSIJ-1.2.15-5D271572', 'Revision': 'RTM.12.1-RC2-modified', 'Config': '*:*:8e82', 'Sys': '3.0', 'Board': 'NITROKEY-START-G' ]
Please note:
- Latest firmware available is:
RTM.13 (published: 2022-12-08T10:59:11Z)
- provided firmware: None
- all data will be removed from the device!
- do not interrupt update process - the device may not run properly!
- the process should not take more than 1 minute
Do you want to continue? [yes/no]: yes
...
Starting bootloader upload procedure
Device: Nitrokey Start FSIJ-1.2.15-5D271572
Connected to the device
Running update!
Do NOT remove the device from the USB slot, until further notice
Downloading flash upgrade program...
Executing flash upgrade...
Waiting for device to appear:
Wait 20 seconds.....
Downloading the program
Protecting device
Finish flashing
Resetting device
Update procedure finished. Device could be removed from USB slot.
Currently connected device strings (after upgrade):
Device:
Vendor: Nitrokey
Product: Nitrokey Start
Serial: FSIJ-1.2.19-5D271572
Revision: RTM.13
Config: *:*:8e82
Sys: 3.0
Board: NITROKEY-START-G
device can now be safely removed from the USB slot
final device strings: [ 'name': '', 'Vendor': 'Nitrokey', 'Product': 'Nitrokey Start', 'Serial': 'FSIJ-1.2.19-5D271572', 'Revision': 'RTM.13', 'Config': '*:*:8e82', 'Sys': '3.0', 'Board': 'NITROKEY-START-G' ]
finishing session 2023-03-16 21:49:07.371291
Log saved to: /tmp/nitropy.log.gc5753a8
jas@kaka:~$
jas@kaka:~$ nitropy start list
Command line tool to interact with Nitrokey devices 0.4.34
:: 'Nitrokey Start' keys:
FSIJ-1.2.19-5D271572: Nitrokey Nitrokey Start (RTM.13)
jas@kaka:~$
Before importing the master key to this device, it should be configured. Note the commands in the beginning to make sure scdaemon/pcscd is not running because they may have cached state from earlier cards. Change PIN code as you like after this, my experience with Gnuk was that the Admin PIN had to be changed first, then you import the key, and then you change the PIN.
jas@kaka:~$ gpg-connect-agent "SCD KILLSCD" "SCD BYE" /bye
OK
ERR 67125247 Slut p fil <GPG Agent>
jas@kaka:~$ ps auxww grep -e pcsc -e scd
jas 11651 0.0 0.0 3468 1672 pts/0 R+ 21:54 0:00 grep --color=auto -e pcsc -e scd
jas@kaka:~$ gpg --card-edit
Reader ...........: 20A0:4211:FSIJ-1.2.19-5D271572:0
Application ID ...: D276000124010200FFFE5D2715720000
Application type .: OpenPGP
Version ..........: 2.0
Manufacturer .....: unmanaged S/N range
Serial number ....: 5D271572
Name of cardholder: [not set]
Language prefs ...: [not set]
Salutation .......:
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: rsa2048 rsa2048 rsa2048
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 0
KDF setting ......: off
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]
gpg/card> admin
Admin commands are allowed
gpg/card> kdf-setup
gpg/card> passwd
gpg: OpenPGP card no. D276000124010200FFFE5D2715720000 detected
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection? 3
PIN changed.
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection? q
gpg/card> name
Cardholder's surname: Josefsson
Cardholder's given name: Simon
gpg/card> lang
Language preferences: sv
gpg/card> sex
Salutation (M = Mr., F = Ms., or space): m
gpg/card> login
Login data (account name): jas
gpg/card> url
URL to retrieve public key: https://josefsson.org/key-20190320.txt
gpg/card> forcesig
gpg/card> key-attr
Changing card key attribute for: Signature key
Please select what kind of key you want:
(1) RSA
(2) ECC
Your selection? 2
Please select which elliptic curve you want:
(1) Curve 25519
(4) NIST P-384
Your selection? 1
The card will now be re-configured to generate a key of type: ed25519
Note: There is no guarantee that the card supports the requested size.
If the key generation does not succeed, please check the
documentation of your card to see what sizes are allowed.
Changing card key attribute for: Encryption key
Please select what kind of key you want:
(1) RSA
(2) ECC
Your selection? 2
Please select which elliptic curve you want:
(1) Curve 25519
(4) NIST P-384
Your selection? 1
The card will now be re-configured to generate a key of type: cv25519
Changing card key attribute for: Authentication key
Please select what kind of key you want:
(1) RSA
(2) ECC
Your selection? 2
Please select which elliptic curve you want:
(1) Curve 25519
(4) NIST P-384
Your selection? 1
The card will now be re-configured to generate a key of type: ed25519
gpg/card>
jas@kaka:~$ gpg --card-edit
Reader ...........: 20A0:4211:FSIJ-1.2.19-5D271572:0
Application ID ...: D276000124010200FFFE5D2715720000
Application type .: OpenPGP
Version ..........: 2.0
Manufacturer .....: unmanaged S/N range
Serial number ....: 5D271572
Name of cardholder: Simon Josefsson
Language prefs ...: sv
Salutation .......: Mr.
URL of public key : https://josefsson.org/key-20190320.txt
Login data .......: jas
Signature PIN ....: not forced
Key attributes ...: ed25519 cv25519 ed25519
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 0
KDF setting ......: on
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]
jas@kaka:~$
Once setup, bring out your offline machine and boot it and mount your USB stick with the offline key. The paths below will be different, and this is using a somewhat unorthodox approach of working with fresh GnuPG configuration paths that I chose for the USB stick.
jas@kaka:/media/jas/2c699cbd-b77e-4434-a0d6-0c4965864296$ cp -a gnupghome-backup-masterkey gnupghome-import-nitrokey-5D271572
jas@kaka:/media/jas/2c699cbd-b77e-4434-a0d6-0c4965864296$ gpg --homedir $PWD/gnupghome-import-nitrokey-5D271572 --edit-key B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE
gpg (GnuPG) 2.2.27; Copyright (C) 2021 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> keytocard
Really move the primary key? (y/N) y
Please select where to store the key:
(1) Signature key
(3) Authentication key
Your selection? 1
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg>
Save changes? (y/N) y
jas@kaka:/media/jas/2c699cbd-b77e-4434-a0d6-0c4965864296$
At this point it is useful to confirm that the Nitrokey has the master key available and that is possible to sign statements with it, back on your regular machine:
jas@kaka:~$ gpg --card-status
Reader ...........: 20A0:4211:FSIJ-1.2.19-5D271572:0
Application ID ...: D276000124010200FFFE5D2715720000
Application type .: OpenPGP
Version ..........: 2.0
Manufacturer .....: unmanaged S/N range
Serial number ....: 5D271572
Name of cardholder: Simon Josefsson
Language prefs ...: sv
Salutation .......: Mr.
URL of public key : https://josefsson.org/key-20190320.txt
Login data .......: jas
Signature PIN ....: not forced
Key attributes ...: ed25519 cv25519 ed25519
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 1
KDF setting ......: on
Signature key ....: B1D2 BD13 75BE CB78 4CF4 F8C4 D73C F638 C53C 06BE
created ....: 2019-03-20 23:37:24
Encryption key....: [none]
Authentication key: [none]
General key info..: pub ed25519/D73CF638C53C06BE 2019-03-20 Simon Josefsson <simon@josefsson.org>
sec> ed25519/D73CF638C53C06BE created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 5D271572
ssb> ed25519/80260EE8A9B92B2B created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 42315277
ssb> ed25519/51722B08FE4745A2 created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 42315277
ssb> cv25519/02923D7EE76EBD60 created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 42315277
jas@kaka:~$ echo foo gpg -a --sign gpg --verify
gpg: Signature made Thu Mar 16 22:11:02 2023 CET
gpg: using EDDSA key B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE
gpg: Good signature from "Simon Josefsson <simon@josefsson.org>" [ultimate]
jas@kaka:~$
Finally to retrieve and sign a key, for example Andre Heinecke s that I could confirm the OpenPGP key identifier from his business card.
jas@kaka:~$ gpg --locate-external-keys aheinecke@gnupg.com
gpg: key 1FDF723CF462B6B1: public key "Andre Heinecke <aheinecke@gnupg.com>" imported
gpg: Total number processed: 1
gpg: imported: 1
gpg: marginals needed: 3 completes needed: 1 trust model: pgp
gpg: depth: 0 valid: 2 signed: 7 trust: 0-, 0q, 0n, 0m, 0f, 2u
gpg: depth: 1 valid: 7 signed: 64 trust: 7-, 0q, 0n, 0m, 0f, 0u
gpg: next trustdb check due at 2023-05-26
pub rsa3072 2015-12-08 [SC] [expires: 2025-12-05]
94A5C9A03C2FE5CA3B095D8E1FDF723CF462B6B1
uid [ unknown] Andre Heinecke <aheinecke@gnupg.com>
sub ed25519 2017-02-13 [S]
sub ed25519 2017-02-13 [A]
sub rsa3072 2015-12-08 [E] [expires: 2025-12-05]
sub rsa3072 2015-12-08 [A] [expires: 2025-12-05]
jas@kaka:~$ gpg --edit-key "94A5C9A03C2FE5CA3B095D8E1FDF723CF462B6B1"
gpg (GnuPG) 2.2.27; Copyright (C) 2021 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
pub rsa3072/1FDF723CF462B6B1
created: 2015-12-08 expires: 2025-12-05 usage: SC
trust: unknown validity: unknown
sub ed25519/2978E9D40CBABA5C
created: 2017-02-13 expires: never usage: S
sub ed25519/DC74D901C8E2DD47
created: 2017-02-13 expires: never usage: A
The following key was revoked on 2017-02-23 by RSA key 1FDF723CF462B6B1 Andre Heinecke <aheinecke@gnupg.com>
sub cv25519/1FFE3151683260AB
created: 2017-02-13 revoked: 2017-02-23 usage: E
sub rsa3072/8CC999BDAA45C71F
created: 2015-12-08 expires: 2025-12-05 usage: E
sub rsa3072/6304A4B539CE444A
created: 2015-12-08 expires: 2025-12-05 usage: A
[ unknown] (1). Andre Heinecke <aheinecke@gnupg.com>
gpg> sign
pub rsa3072/1FDF723CF462B6B1
created: 2015-12-08 expires: 2025-12-05 usage: SC
trust: unknown validity: unknown
Primary key fingerprint: 94A5 C9A0 3C2F E5CA 3B09 5D8E 1FDF 723C F462 B6B1
Andre Heinecke <aheinecke@gnupg.com>
This key is due to expire on 2025-12-05.
Are you sure that you want to sign this key with your
key "Simon Josefsson <simon@josefsson.org>" (D73CF638C53C06BE)
Really sign? (y/N) y
gpg> quit
Save changes? (y/N) y
jas@kaka:~$
This is on my day-to-day machine, using the NitroKey Start with the offline key. No need to boot the old offline machine just to sign keys or extend expiry anymore! At FOSDEM 23 I managed to get at least one DD signature on my new key, and the Debian keyring maintainers accepted my Ed25519 key. Hopefully I can now finally let my 2014-era RSA3744 key expire in 2023-09-19 and not extend it any further. This should finish my transition to a simpler OpenPGP key setup, yay!
The nineteenth release of littler as a
CRAN package
landed this morning, following in the now seventeen year history (!!) as
a package started by Jeff in 2006, and joined
by me a few weeks later.
littler
is the first command-line interface for R as it predates
Rscript. It allows for piping as well for shebang
scripting via #!, uses command-line arguments more
consistently and still starts
faster. It also always loaded the methods package which
Rscript only began to do in recent years.
littler
lives on Linux and Unix, has its difficulties on macOS due to
yet-another-braindeadedness there (who ever thought case-insensitive
filesystems as a default were a good idea?) and simply does not exist on
Windows (yet the build system could be extended see RInside for
an existence proof, and volunteers are welcome!). See the FAQ
vignette on how to add it to your PATH. A few examples
are highlighted at the Github repo, as well
as in the examples
vignette.
This release is somewhat unique is not having any maintenance of the
small core but rather focussing exclusively on changes to the (many !!)
scripts collected in examples/scripts/.
We added new ones and extended several existing ones including
install2.r thanks to patches by @eitsupi. Of the new scripts, I already
use tttl.r quite a bit for tests on co-maintained packages
that use testthat (but as I have made known before my heart
belongs firmly to tinytest as nobody should need thirty-plus
recursive dependencies to run a few test predicates) as well as the
very new installRub.r to directly add r-universe Ubuntu binaries
onto an r2u systems as
demonstrated in a first
and second
tweet (and corresponding toot) this week.
The full change description follows.
Changes in littler
version 0.3.18 (2023-03-25)
Changes in examples scripts
roxy.r can now set an additional
--libpath
getRStudioDesktop.r and
getRStudioServer.r have updated default download
file
install2.r and installGithub.r can set
--type
r2u.r now has a --suffix
option
tt.r removes a redundant library
call
tttl.r has been added for
testthat::test_local()
installRub.r has been added to install r-universe
binaries on Ubuntu
install2.r has updated error capture messages
(Tatsuya Shima and Dirk in #104)
My CRANberries
service provides a comparison to the
previous release. Full details for the littler
release are provided as usual at the ChangeLog
page, and also on the package docs website.
The code is available via the GitHub repo, from
tarballs and now of course also from its CRAN
page and via install.packages("littler"). Binary
packages are available directly in Debian (currently in experimental
due to a release freeze) as well as (in a day or two) Ubuntu binaries at
CRAN thanks to the tireless Michael Rutter and of course via r2u following its next
refresh.
Comments and suggestions are welcome at the GitHub repo.
If you like this or other open-source work I do, you can now sponsor me at
GitHub.
A new release 0.2.7 of our RcppSMC package arrived at
CRAN earlier today. It contains
several extensions added by team member (and former GSoC student) Ilya Zarubin since the last
release. We were a little slow to release those but one of those CRAN
emails forced our hand for a release now. The updated uninitialized
variable messages in clang++-16 have found a fan in Brian
Ripley, and so he sent us a note. And as the issue was trivially
reproducible with clang++-15 here too I had it fixed in no
time. And both changes taken together form the incremental 0.2.7
release.
RcppSMC
provides Rcpp-based bindings to R for the Sequential Monte Carlo
Template Classes (SMCTC) by Adam Johansen described in his JSS article.
Sequential Monte Carlo is also referred to as Particle Filter
in some contexts. The package now also features the Google Summer of Code
work by Leah South in 2017, and by Ilya Zarubin in 2021.
The release is summarized below.
Changes in RcppSMC
version 0.2.7 (2023-03-22)
Extensive extensions for conditional SMC and resample, updated
hello_world example, added skeleton function for easier
package creation (Ilya in #67,#72)
The Framework is a 13.5" laptop body with swappable parts, which
makes it somewhat future-proof and certainly easily repairable,
scoring an "exceedingly rare" 10/10 score from ifixit.com.
There are two generations of the laptop's main board (both compatible
with the same body): the Intel 11th and 12th gen chipsets.
I have received my Framework, 12th generation "DIY", device in late
September 2022 and will update this page as I go along in the process
of ordering, burning-in, setting up and using the device over the
years.
Overall, the Framework is a good laptop. I like the keyboard, the
touch pad, the expansion cards. Clearly there's been some good work
done on industrial design, and it's the most repairable laptop I've
had in years. Time will tell, but it looks sturdy enough to survive me
many years as well.
This is also one of the most powerful devices I ever lay my hands
on. I have managed, remotely, more powerful servers, but this is the
fastest computer I have ever owned, and it fits in this tiny case. It
is an amazing machine.
On the downside, there's a bit of proprietary firmware required (WiFi,
Bluetooth, some graphics) and the Framework ships with a proprietary
BIOS, with currently no Coreboot support. Expect to need the
latest kernel, firmware, and hacking around a bunch of things to get
resolution and keybindings working right.
Like others, I have first found significant power management issues,
but many issues can actually be solved with some configuration. Some
of the expansion ports (HDMI, DP, MicroSD, and SSD) use power when
idle, so don't expect week-long suspend, or "full day" battery while
those are plugged in.
Finally, the expansion ports are nice, but there's only four of
them. If you plan to have a two-monitor setup, you're likely going to
need a dock.
Read on for the detailed review. For context, I'm moving from the
Purism Librem 13v4 because it
basically exploded on me. I
had, in the meantime, reverted back to an old ThinkPad X220, so I
sometimes compare the Framework with that venerable laptop as well.
This blog post has been maturing for months now. It started in
September 2022 and I declared it completed in March 2023. It's the
longest single article on this entire website, currently clocking at
about 13,000 words. It will take an average reader a full hour to go
through this thing, so I don't expect anyone to actually do
that. This introduction should be good enough for most people, read
the first section if you intend to actually buy a Framework. Jump
around the table of contents as you see fit for after you did buy the
laptop, as it might include some crucial hints on how to make it work
best for you, especially on (Debian) Linux.
Advice for buyers
Those are things I wish I would have known before buying:
consider buying 4 USB-C expansion cards, or at least a mix of 4
USB-A or USB-C cards, as they use less power than other cards and
you do want to fill those expansion slots otherwise they snag
around and feel insecure
you will likely need a dock or at least a USB hub if you want a
two-monitor setup, otherwise you'll run out of ports
you have to do some serious tuning to get proper (10h+ idle, 10
days suspend) power savings
in particular, beware that the HDMI, DisplayPort and
particularly the SSD and MicroSD cards take a significant amount
power, even when sleeping, up to 2-6W for the latter two
beware that the MicroSD card is what it says: Micro, normal SD
cards won't fit, and while there might be full sized one
eventually, it's currently only at the prototyping stage
Current status
I have the framework! It's setup with a fresh new Debian bookworm
installation. I've ran through a large number of tests and burn in.
I have decided to use the Framework as my daily driver, and had to buy
a USB-C dock to get my two monitors
connected, which was own adventure.
Update: Framework just (2023-03-23) just announced a whole bunch of
new stuff:
The recording is available in this video and it's not your
typical keynote. It starts ~25 minutes late, audio is crap, lightning
and camera are crap, clapping seems to be from whatever staff they
managed to get together in a room, decor is bizarre, colors are
shit. It's amazing.
Specifications
Those are the specifications of the 12th gen, in general terms. Your
build will of course vary according to your needs.
CPU: i5-1240P, i7-1260P, or i7-1280P (Up to 4.4-4.8 GHz, 4+8
cores), Iris Xe graphics
4 x USB-C user-selectable expansion ports, including
USB-C
USB-A
HDMI
DP
Ethernet
MicroSD
250-1000GB SSD
3.5mm combo headphone jack
Kill switches for microphone and camera
Battery: 55Wh
Camera: 1080p 60fps
Biometrics: Fingerprint Reader
Backlit keyboard
Power Adapter: 60W USB-C (or bring your own)
ships with a screwdriver/spludger
1 year warranty
base price: 1000$CAD, but doesn't give you much, typical builds
around 1500-2000$CAD
Actual build
This is the actual build I ordered. Amounts in CAD. (1CAD =
~0.75EUR/USD.)
Base configuration
CPU: Intel Core i5-1240P (AKA Alder Lake P 8 4.4GHz
P-threads, 8 3.2GHz E-threads, 16 total, 28-64W), 1079$
Memory: 16GB (1 x 16GB) DDR4-3200, 104$
Customization
Keyboard: US English, included
Expansion Cards
2 USB-C $24
3 USB-A $36
2 HDMI $50
1 DP $50
1 MicroSD $25
1 Storage 1TB $199
Sub-total: 384$
Accessories
Power Adapter - US/Canada $64.00
Total
Before tax: 1606$
After tax and duties: 1847$
Free shipping
Quick evaluation
This is basically the TL;DR: here, just focusing on broad pros/cons of
the laptop.
Pros
easily repairable (complete with QR codes pointing to repair
guides!), the 11th gen received a 10/10 score from
ifixit.com, which they call "exceedingly rare", the 12th gen
has a similar hardware design and would probably rate similarly
replaceable motherboard!!! can be reused as a NUC-like device, with a
3d-printed case, 12th gen board can be bought standalone and
retrofitted into an 11th gen case
not a passing fad: they made a first laptop with the 11th gen Intel
chipset in 2021, and a second motherboard with the 12th Intel
chipset in 2022
four modular USB-C ports which can fit HDMI, USB-C (pass-through,
can provide power on both sides), USB-A, DisplayPort, MicroSD,
external storage (250GB, 1TB), active modding community
nice power led indicating power level (charging, charged, etc) when
plugged
they used to have some difficulty keeping up with the orders: first
two batches shipped, third batch sold out, fourth batch should have
shipped in October 2021. they generally seem to keep up with
shipping. update (august 2022): they rolled out a second line of
laptops (12th gen), first batch shipped, second batch shipped
late, September 2022 batch was generally on time, see this
spreadsheet for a crowdsourced effort to track those
supply chain issues seem to be under control as of early 2023. I
got the Ethernet expansion card shipped within a week.
compared to my previous laptop (Purism Librem
13v4), it feels strangely
bulkier and heavier; it's actually lighter than the purism (1.3kg
vs 1.4kg) and thinner (15.85mm vs 18mm) but the design of the
Purism laptop (tapered edges) makes it feel thinner
no space for a 2.5" drive
rather bright LED around power button, but can be dimmed in the
BIOS (not low enough to my taste) I got used to it
fan quiet when idle, but can be noisy when running, for example if
you max a CPU for a while
battery described as "mediocre" by Ars Technica (above), confirmed
poor in my tests (see below)
no RJ-45 port, and attempts at designing ones are failing
because the modular plugs are too thin to fit (according to Linux
After Dark), so unlikely to have one in the future
Update: they cracked that nut and ship an 2.5 gbps Ethernet
expansion card with a realtek chipset, without any
firmware blob
a bit pricey for the performance, especially when compared to the
competition (e.g. Dell XPS, Apple M1)
12th gen Intel has glitchy graphics, seems like Intel hasn't fully
landed proper Linux support for that chipset yet
Initial hardware setup
A breeze.
Accessing the board
The internals are accessed through five TorX screws, but there's a nice
screwdriver/spudger that works well enough. The screws actually hold in
place so you can't even lose them.
The first setup is a bit counter-intuitive coming from the Librem
laptop, as I expected the back cover to lift and give me access to the
internals. But instead the screws is release the keyboard and touch
pad assembly, so you actually need to flip the laptop back upright and
lift the assembly off to get access to the internals. Kind of
scary.
I also actually unplugged a connector in lifting the assembly because
I lifted it towards the monitor, while you actually need to lift it
to the right. Thankfully, the connector didn't break, it just
snapped off and I could plug it back in, no harm done.
Once there, everything is well indicated, with QR codes all over the
place supposedly leading to online instructions.
Bad QR codes
Unfortunately, the QR codes I tested (in the expansion card slot, the
memory slot and CPU slots) did not actually work so I wonder how
useful those actually are.
After all, they need to point to something and that means a URL, a
running website that will answer those requests forever. I bet those
will break sooner than later and in fact, as far as I can tell, they
just don't work at all. I prefer the approach taken by the MNT reform
here which designed (with the 100 rabbits folks) an actual paper
handbook (PDF).
The first QR code that's immediately visible from the back of the
laptop, in an expansion cord slot, is a 404. It seems to be some
serial number URL, but I can't actually tell because, well, the page
is a 404.
I was expecting that bar code to lead me to an introduction page,
something like "how to setup your Framework laptop". Support actually
confirmed that it should point a quickstart guide. But in a
bizarre twist, they somehow sent me the URL with the plus (+) signs
escaped, like this:
(They have also "let the team know about this for feedback and help
resolve the problem with the link" which is a support code word for
"ha-ha! nope! not my problem right now!" Trust me, I know, my own
code word is "can you please make a ticket?")
Seating disks and memory
The "DIY" kit doesn't actually have that much of a setup. If you
bought RAM, it's shipped outside the laptop in a little plastic case,
so you just seat it in as usual.
Then you insert your NVMe drive, and, if that's your fancy, you also
install your own mPCI WiFi card. If you ordered one (which was my
case), it's pre-installed.
Closing the laptop is also kind of amazing, because the keyboard
assembly snaps into place with magnets. I have actually used the
laptop with the keyboard unscrewed as I was putting the drives in and
out, and it actually works fine (and will probably void your warranty,
so don't do that). (But you can.) (But don't, really.)
Hardware review
Keyboard and touch pad
The keyboard feels nice, for a laptop. I'm used to mechanical keyboard
and I'm rather violent with those poor things. Yet the key travel is
nice and it's clickety enough that I don't feel too disoriented.
At first, I felt the keyboard as being more laggy than my normal
workstation setup, but it turned out this was a graphics driver
issues. After enabling a composition manager, everything feels snappy.
The touch pad feels good. The double-finger scroll works well enough,
and I don't have to wonder too much where the middle button is, it
just works.
Taps don't work, out of the box: that needs to be enabled in Xorg,
with something like this:
But be aware that once you enable that tapping, you'll need to deal
with palm detection... So I have not actually enabled this in the end.
Power button
The power button is a little dangerous. It's quite easy to hit, as
it's right next to one expansion card where you are likely to plug in
a cable power. And because the expansion cards are kind of hard to
remove, you might squeeze the laptop (and the power key) when trying
to remove the expansion card next to the power button.
So obviously, don't do that. But that's not very helpful.
An alternative is to make the power button do something else. With
systemd-managed systems, it's actually quite easy. Add a
HandlePowerKey stanza to (say)
/etc/systemd/logind.conf.d/power-suspends.conf:
And the power button will suspend! Long-press to power off doesn't
actually work as the laptop immediately suspends...
Note that there's probably half a dozen other ways of doing this,
see this, this, or that.
Special keybindings
There is a series of "hidden" (as in: not labeled on the key)
keybindings related to the fn keybinding that I actually
find quite useful.
Key
Equivalent
Effect
Command
p
Pause
lock screen
xset s activate
b
Break
?
?
k
ScrLk
switch keyboard layout
N/A
It looks like those are defined in the microcontroller so it
would be possible to add some. For example, the SysRq key
is almost bound to fns in there.
Note that most other shortcuts like this are clearly documented
(volume, brightness, etc). One key that's less obvious is
F12 that only has the Framework logo on it. That actually
calls the keysym XF86AudioMedia which, interestingly, does
absolutely nothing here. By default, on Windows, it opens your
browser to the Framework website and, on Linux, your "default
media player".
The keyboard backlight can be cycled with fn-space. The
dimmer version is dim enough, and the keybinding is easy to find in
the dark.
A skinny elephant would be performed with altPrtScr (above F11) KEY, so for
example altfnF11b
should do a hard reset. This comment suggests you need to hold
the fnonly if "function lock" is on, but that's
actually the opposite of my experience.
Out of the box, some of the fn keys don't work. Mute,
volume up/down, brightness, monitor changes, and the airplane mode key
all do basically nothing. They don't send proper keysyms to Xorg at
all.
This is a known problem and it's related to the fact that the
laptop has light sensors to adjust the brightness
automatically. Somehow some of those keys (e.g. the brightness
controls) are supposed to show up as a different input device, but
don't seem to work correctly. It seems like the solution is for the
Framework team to write a driver specifically for this, but so far no
progress since July 2022.
In the meantime, the fancy functionality can be supposedly disabled with:
echo 'blacklist hid_sensor_hub' sudo tee /etc/modprobe.d/framework-als-blacklist.conf
Kill switches
The Framework has two "kill switches": one for the camera and the
other for the microphone. The camera one actually disconnects the USB
device when turned off, and the mic one seems to cut the circuit. It
doesn't show up as muted, it just stops feeding the sound.
Both kill switches are around the main camera, on top of the monitor,
and quite discreet. Then turn "red" when enabled (i.e. "red" means
"turned off").
Monitor
The monitor looks pretty good to my untrained eyes. I have yet to do
photography work on it, but some photos I looked at look sharp and the
colors are bright and lively. The blacks are dark and the screen is
bright.
I have yet to use it in full sunlight.
The dimmed light is very dim, which I like.
Screen backlight
I bind brightness keys to xbacklight in i3, but out of the box I get
this error:
sep 29 22:09:14 angela i3[5661]: No outputs have backlight property
It just requires this blob in /etc/X11/xorg.conf.d/backlight.conf:
This way I can control the actual backlight power with the brightness
keys, and they do significantly reduce power usage.
Multiple monitor support
I have been able to hook up my two old monitors to the HDMI and
DisplayPort expansion cards on the laptop. The lid closes without
suspending the machine, and everything works great.
I actually run out of ports, even with a 4-port USB-A hub, which gives
me a total of 7 ports:
power (USB-C)
monitor 1 (DisplayPort)
monitor 2 (HDMI)
USB-A hub, which adds:
keyboard (USB-A)
mouse (USB-A)
Yubikey
external sound card
Now the latter, I might be able to get rid of if I switch to a
combo-jack headset, which I do have (and still need to test).
But still, this is a problem. I'll probably need a powered USB-C dock
and better monitors, possibly with some Thunderbolt chaining, to
save yet more ports.
But that means more money into this setup, argh. And figuring out my
monitor situation is the kind of thing I'm not that big
of a fan of. And neither is shopping for USB-C (or is it Thunderbolt?)
hubs.
My normal autorandr setup doesn't work: I have tried saving a
profile and it doesn't get autodetected, so I also first need to do:
autorandr -l framework-external-dual-lg-acer
The magic:
autorandr -l horizontal
... also works well.
The worst problem with those monitors right now is that they have a
radically smaller resolution than the main screen on the laptop, which
means I need to reset the font scaling to normal every time I switch
back and forth between those monitors and the laptop, which means I
actually need to do this:
Expansion ports
I ordered a total of 10 expansion ports.
I did manage to initialize the 1TB drive as an encrypted storage,
mostly to keep photos as this is something that takes a massive amount
of space (500GB and counting) and that I (unfortunately) don't work on
very often (but still carry around).
The expansion ports are fancy and nice, but not actually that
convenient. They're a bit hard to take out: you really need to crimp
your fingernails on there and pull hard to take them out. There's a
little button next to them to release, I think, but at first it feels
a little scary to pull those pucks out of there. You get used to it
though, and it's one of those things you can do without looking
eventually.
There's only four expansion ports. Once you have two monitors, the
drive, and power plugged in, bam, you're out of ports; there's nowhere
to plug my Yubikey. So if this is going to be my daily driver, with a
dual monitor setup, I will need a dock, which means more crap firmware
and uncertainty, which isn't great. There are actually plans to make a
dual-USB card, but that is blocked on designing an actual
board for this.
I can't wait to see more expansion ports produced. There's a ethernet
expansion card which quickly went out of stock basically the day
it was announced, but was eventually restocked.
I would like to see a proper SD-card reader. There's a MicroSD card
reader, but that obviously doesn't work for normal SD cards, which
would be more broadly compatible anyways (because you can have a
MicroSD to SD card adapter, but I have never heard of the
reverse). Someone actually found a SD card reader that fits and
then someone else managed to cram it in a 3D printed case, which
is kind of amazing.
Still, I really like that idea that I can carry all those little
adapters in a pouch when I travel and can basically do anything I
want. It does mean I need to shuffle through them to find the right
one which is a little annoying. I have an elastic band to keep them
lined up so that all the ports show the same side, to make it easier
to find the right one. But that quickly gets undone and instead I have
a pouch full of expansion cards.
Another awesome thing with the expansion cards is that they don't just
work on the laptop: anything that takes USB-C can take those cards,
which means you can use it to connect an SD card to your phone, for
backups, for example. Heck, you could even connect an external display
to your phone that way, assuming that's supported by your phone of
course (and it probably isn't).
The expansion ports do take up some power, even when idle. See the
power management section below, and particularly the power usage
tests for details.
USB-C charging
One thing that is really a game changer for me is USB-C charging. It's
hard to overstate how convenient this is. I often have a USB-C cable
lying around to charge my phone, and I can just grab that thing and
pop it in my laptop. And while it will obviously not charge as fast as
the provided charger, it will stop draining the battery at least.
(As I wrote this, I had the laptop plugged in the Samsung charger that
came with a phone, and it was telling me it would take 6 hours to
charge the remaining 15%. With the provided charger, that flew down to
15 minutes. Similarly, I can power the laptop from the power grommet
on my desk, reducing clutter as I have that single wire out there
instead of the bulky power adapter.)
I also really like the idea that I can charge my laptop with a power
bank or, heck, with my phone, if push comes to shove. (And
vice-versa!)
This is awesome. And it works from any of the expansion ports, of
course. There's a little led next to the expansion ports as well,
which indicate the charge status:
red/amber: charging
white: charged
off: unplugged
I couldn't find documentation about this, but the forum
answered.
This is something of a recurring theme with the Framework. While it
has a good knowledge base and repair/setup guides (and the
forum is awesome) but it doesn't have a good "owner manual" that
shows you the different parts of the laptop and what they do. Again,
something the MNT reform did well.
Another thing that people are asking about is an external sleep
indicator: because the power LED is on the main keyboard assembly,
you don't actually see whether the device is active or not when the
lid is closed.
Finally, I wondered what happens when you plug in multiple power
sources and it turns out the charge controller is actually pretty
smart: it will pick the best power source and use it. The only
downside is it can't use multiple power sources, but that seems like
a bit much to ask.
Multimedia and other devices
Those things also work:
webcam: splendid, best webcam I've ever had (but my standards are
really low)
onboard mic: works well, good gain (maybe a bit much)
onboard speakers: sound okay, a little metal-ish, loud enough to be
annoying, see this thread for benchmarks, apparently pretty
good speakers
Combo jack mic tests
The Framework laptop ships with a combo jack on the left side, which
allows you to plug in a CTIA (source) headset. In human
terms, it's a device that has both a stereo output and a mono input,
typically a headset or ear buds with a microphone somewhere.
It works, which is better than the Purism (which only had audio
out), but is on par for the course for that kind of onboard
hardware. Because of electrical interference, such sound cards very
often get lots of noise from the board.
With a Jabra Evolve 40, the built-in USB sound card generates
basically zero noise on silence (invisible down to -60dB in Audacity)
while plugging it in directly generates a solid -30dB hiss. There is
a noise-reduction system in that sound card, but the difference is
still quite striking.
On a comparable setup (curie, a 2017 Intel NUC), there is
also a his with the Jabra headset, but it's quieter, more in the order
of -40/-50 dB, a noticeable difference. Interestingly, testing with my
Mee Audio Pro M6 earbuds leads to a little more hiss on curie, more on
the -35/-40 dB range, close to the Framework.
Also note that another sound card, the Antlion USB adapter that comes
with the ModMic 4, also gives me pretty close to silence on a quiet
recording, picking up less than -50dB of background noise. It's
actually probably picking up the fans in the office, which do make
audible noises.
In other words, the hiss of the sound card built in the Framework
laptop is so loud that it makes more noise than the quiet fans in the
office. Or, another way to put it is that two USB sound cards (the
Jabra and the Antlion) are able to pick up ambient noise in my office
but not the Framework laptop.
See also my audio page.
Performance tests
Compiling Linux 5.19.11
On a single core, compiling the Debian version of the Linux kernel
takes around 100 minutes:
I had to plug the normal power supply after a few minutes because
battery would actually run out using my desk's power grommet (34
watts).
During compilation, fans were spinning really hard, quite noisy, but
not painfully so.
The laptop was sucking 55 watts of power, steadily:
Time User Nice Sys Idle IO Run Ctxt/s IRQ/s Fork Exec Exit Watts
-------- ----- ----- ----- ----- ----- ---- ------ ------ ---- ---- ---- ------
Average 87.9 0.0 10.7 1.4 0.1 17.8 6583.6 5054.3 233.0 223.9 233.1 55.96
GeoMean 87.9 0.0 10.6 1.2 0.0 17.6 6427.8 5048.1 227.6 218.7 227.7 55.96
StdDev 1.4 0.0 1.2 0.6 0.2 3.0 1436.8 255.5 50.0 47.5 49.7 0.20
-------- ----- ----- ----- ----- ----- ---- ------ ------ ---- ---- ---- ------
Minimum 85.0 0.0 7.8 0.5 0.0 13.0 3594.0 4638.0 117.0 111.0 120.0 55.52
Maximum 90.8 0.0 12.9 3.5 0.8 38.0 10174.0 5901.0 374.0 362.0 375.0 56.41
-------- ----- ----- ----- ----- ----- ---- ------ ------ ---- ---- ---- ------
Summary:
CPU: 55.96 Watts on average with standard deviation 0.20
Note: power read from RAPL domains: package-0, uncore, package-0, core, psys.
These readings do not cover all the hardware in this device.
memtest86+
I ran Memtest86+ v6.00b3. It shows something like this:
Software setup
Once I had everything in the hardware setup, I figured, voil , I'm
done, I'm just going to boot this beautiful machine and I can get back
to work.
I don't understand why I am so na ve some times. It's mind boggling.
Obviously, it didn't happen that way at all, and I spent the best of
the three following days tinkering with the laptop.
Secure boot and EFI
First, I couldn't boot off of the NVMe drive I transferred from the
previous laptop (the Purism) and the
BIOS was not very helpful: it was just complaining about not finding
any boot device, without dropping me in the real BIOS.
At first, I thought it was a problem with my NVMe drive, because it's
not listed in the compatible SSD drives from upstream. But I
figured out how to enter BIOS (press F2 manically, of
course), which showed the NVMe drive was actually detected. It just
didn't boot, because it was an old (2010!!) Debian install without
EFI.
So from there, I disabled secure boot, and booted a grml image to
try to recover. And by "boot" I mean, I managed to get to the grml
boot loader which promptly failed to load its own root file system
somehow. I still have to investigate exactly what happened there, but
it failed some time after the initrd load with:
Unable to find medium containing a live file system
This, it turns out, was fixed in Debian lately, so a daily GRML
build will not have this problems. The upcoming 2022 release
(likely 2022.10 or 2022.11) will also get the fix.
I did manage to boot the development version of the Debian
installer which was a surprisingly good experience: it mounted the
encrypted drives and did everything pretty smoothly. It even offered
me to reinstall the boot loader, but that ultimately (and correctly, as
it turns out) failed because I didn't have a /boot/efi partition.
At this point, I realized there was no easy way out of this, and I
just proceeded to completely reinstall Debian. I had a spare NVMe
drive lying around (backups FTW!) so I just swapped that in, rebooted
in the Debian installer, and did a clean install. I wanted to switch
to bookworm anyways, so I guess that's done too.
Storage limitations
Another thing that happened during setup is that I tried to copy over
the internal 2.5" SSD drive from the Purism to the Framework 1TB
expansion card. There's no 2.5" slot in the new laptop, so that's
pretty much the only option for storage expansion.
I was tired and did something wrong. I ended up wiping the partition
table on the original 2.5" drive.
Oops.
It might be recoverable, but just restoring the partition table
didn't work either, so I'm not sure how I recover the data
there. Normally, everything on my laptops and workstations is designed
to be disposable, so that wasn't that big of a problem. I did manage
to recover most of the data thanks to git-annexreinit, but
that was a little hairy.
Bootstrapping Puppet
Once I had some networking, I had to install all the packages I
needed. The time I spent setting up my workstations with Puppet has
finally paid off. What I actually did was to restore two critical
directories:
/etc/ssh
/var/lib/puppet
So that I would keep the previous machine's identity. That way I could
contact the Puppet server and install whatever was missing. I used my
Puppet optimization
trick to do a batch
install and then I had a good base setup, although not exactly as it
was before. 1700 packages were installed manually on angela before
the reinstall, and not in Puppet.
I did not inspect each one individually, but I did go through /etc
and copied over more SSH keys, for backups and SMTP over SSH.
LVFS support
It looks like there's support for the (de-facto) standard LVFS
firmware update system. At least I was able to update the UEFI
firmware with a simple:
Those instructions come from the beta forum post. I performed the
BIOS update on 2023-01-16T16:00-0500.
Resolution tweaks
The Framework laptop resolution (2256px X 1504px) is big enough to
give you a pretty small font size, so welcome to the marvelous world
of "scaling".
The Debian wiki page has a few tricks for this.
Console
This will make the console and grub fonts more readable:
Xorg
Adding this to your .Xresources will make everything look much bigger:
! 1.5*96
Xft.dpi: 144
Apparently, some of this can also help:
! These might also be useful depending on your monitor and personal preference:
Xft.autohint: 0
Xft.lcdfilter: lcddefault
Xft.hintstyle: hintfull
Xft.hinting: 1
Xft.antialias: 1
Xft.rgba: rgb
It my experience it also makes things look a little fuzzier, which is
frustrating because you have this awesome monitor but everything looks
out of focus. Just bumping Xft.dpi by a 1.5 factor looks good to me.
The Debian Wiki has a page on HiDPI, but it's not as good as the
Arch Wiki, where the above blurb comes from. I am not using the
latter because I suspect it's causing some of the "fuzziness".
TODO: find the equivalent of this GNOME hack in i3? (gsettings set
org.gnome.mutter experimental-features
"['scale-monitor-framebuffer']"), taken from this Framework
guide
Issues
BIOS configuration
The Framework BIOS has some minor issues. One issue I personally
encountered is that I had disabled Quick boot and Quiet boot in
the BIOS to diagnose the above boot issues. This, in turn, triggers a
bug where the BIOS boot manager (F12) would just hang
completely. It would also fail to boot from an external USB drive.
The current fix (as of BIOS 3.03) is to re-enable both Quick
boot and Quiet boot. Presumably this is something that will get
fixed in a future BIOS update.
Note that the following keybindings are active in the BIOS POST
check:
Key
Meaning
F2
Enter BIOS setup menu
F12
Enter BIOS boot manager
Delete
Enter BIOS setup menu
WiFi compatibility issues
I couldn't make WiFi work at first. Obviously, the default Debian
installer doesn't ship with proprietary firmware (although that might
change soon) so the WiFi card didn't work out of the box. But even
after copying the firmware through a USB stick, I couldn't quite
manage to find the right combination of ip/iw/wpa-supplicant
(yes, after repeatedly copying a bunch more packages over to get
those bootstrapped). (Next time I should probably try something like
this post.)
Thankfully, I had a little USB-C dongle with a RJ-45 jack lying
around. That also required a firmware blob, but it was a single
package to copy over, and with that loaded, I had network.
Eventually, I did managed to make WiFi work; the problem was more on
the side of "I forgot how to configure a WPA network by hand from the
commandline" than anything else. NetworkManager worked fine and got
WiFi working correctly.
Note that this is with Debian bookworm, which has the 5.19 Linux
kernel, and with the firmware-nonfree (firmware-iwlwifi,
specifically) package.
Battery life
I was having between about 7 hours of battery on the Purism Librem
13v4, and that's after a year or two of battery life. Now, I still
have about 7 hours of battery life, which is nicer than my old
ThinkPad X220 (20 minutes!) but really, it's not that good for a new
generation laptop. The 12th generation Intel chipset probably improved
things compared to the previous one Framework laptop, but I don't have
a 11th gen Framework to compare with).
(Note that those are estimates from my status bar, not wall clock
measurements. They should still be comparable between the Purism and
Framework, that said.)
The battery life doesn't seem up to, say, Dell XPS 13, ThinkPad X1, and
of course not the Apple M1, where I would expect 10+ hours of battery
life out of the box.
That said, I do get those kind estimates when the machine is fully
charged and idle. In fact, when everything is quiet and nothing is
plugged in, I get dozens of hours of battery life estimated (I've
seen 25h!). So power usage fluctuates quite a bit depending on usage,
which I guess is expected.
Concretely, so far, light web browsing, reading emails and writing
notes in Emacs (e.g. this file) takes about 8W of power:
Expansion cards matter a lot in the battery life (see below for a
thorough discussion), my normal setup is 2xUSB-C and 1xUSB-A (yes,
with an empty slot, and yes, to save power).
Interestingly, playing a video in a (720p) window in a window takes up
more power (10.5W) than in full screen (9.5W) but I blame that on my
desktop setup (i3 + compton)... Not sure if mpv hits the
VA-API, maybe not in windowed mode. Similar results with 1080p,
interestingly, except the window struggles to keep up altogether. Full
screen playback takes a relatively comfortable 9.5W, which means a
solid 5h+ of playback, which is fine by me.
Fooling around the web, small edits, youtube-dl, and I'm at around 80%
battery after about an hour, with an estimated 5h left, which is a
little disappointing. I had a 7h remaining estimate before I started
goofing around Discourse, so I suspect the website is a pretty
big battery drain, actually. I see about 10-12 W, while I was probably at
half that (6-8W) just playing music with mpv in the background...
In other words, it looks like editing posts in Discourse with Firefox
takes a solid 4-6W of power. Amazing and gross.
(When writing about abusive power usage generates more power usage, is
that an heisenbug? Or schr dinbug?)
Power management
Compared to the Purism Librem 13v4, the ongoing power usage seems to
be slightly better. An anecdotal metric is that the Purism would take
800mA idle, while the more powerful Framework manages a little over
500mA as I'm typing this, fluctuating between 450 and 600mA. That is
without any active expansion card, except the storage. Those numbers
come from the output of tlp-stat -b and, unfortunately, the "ampere"
unit makes it quite hard to compare those, because voltage is not
necessarily the same between the two platforms.
TODO: i915 driver has a lot of parameters, including some about
power saving, see, again, the arch wiki, and particularly
enable_fbc=1
TL:DR; power management on the laptop is an issue, but there's various
tweaks you can make to improve it. Try:
powertop --auto-tune
apt install tlp && systemctl enable tlp
nvme.noacpi=1 mem_sleep_default=deep on the kernel command line
may help with standby power usage
keep only USB-C expansion cards plugged in, all others suck power
even when idle
consider upgrading the BIOS to latest beta (3.06 at the time of
writing), unverified power savings
latest Linux kernels (6.2) promise power savings as well
(unverified)
Update: also try to follow the official optimization guide. It
was made for Ubuntu but will probably also work for your distribution
of choice with a few tweaks. They recommend using tlpui but it's
not packaged in Debian. There is, however, a Flatpak
release. In my case, it resulted in the following diff to
tlp.conf: tlp.patch.
Background on CPU architecture
There were power problems in the 11th gen Framework laptop, according
to this report from Linux After Dark, so the issues with power
management on the Framework are not new.
The 12th generation Intel CPU (AKA "Alder Lake") is a big-little
architecture with "power-saving" and "performance" cores. There
used to be performance problems introduced by the scheduler in Linux
5.16 but those were eventually fixed in 5.18, which uses
Intel's hardware as an "intelligent, low-latency hardware-assisted
scheduler". According to Phoronix, the 5.19 release improved the
power saving, at the cost of some penalty cost. There were also patch
series to make the scheduler configurable, but it doesn't look
those have been merged as of 5.19. There was also a session about this
at the 2022 Linux Plumbers, but they stopped short of
talking more about the specific problems Linux is facing in Alder
lake:
Specifically, the kernel's energy-aware scheduling heuristics don't
work well on those CPUs. A number of features present there
complicate the energy picture; these include SMT, Intel's "turbo
boost" mode, and the CPU's internal power-management mechanisms. For
many workloads, running on an ostensibly more power-hungry Pcore can
be more efficient than using an Ecore. Time for discussion of the
problem was lacking, though, and the session came to a close.
All this to say that the 12gen Intel line shipped with this Framework
series should have better power management thanks to its
power-saving cores. And Linux has had the scheduler changes to make
use of this (but maybe is still having trouble). In any case, this
might not be the source of power management problems on my laptop,
quite the opposite.
Also note that the firmware updates for various chipsets are
supposed to improve things eventually.
On the other hand, The Verge simply declared the whole P-series
a mistake...
Attempts at improving power usage
I did try to follow some of the tips in this forum post. The
tricks powertop --auto-tune and tlp's
PCIE_ASPM_ON_BAT=powersupersave basically did nothing: I was stuck
at 10W power usage in powertop (600+mA in tlp-stat).
Apparently, I should be able to reach the C8 CPU power state (or
even C9, C10) in powertop, but I seem to be stock at
C7. (Although I'm not sure how to read that tab in powertop: in the
Core(HW) column there's only C3/C6/C7 states, and most cores are 85%
in C7 or maybe C6. But the next column over does show many CPUs in
C10 states...
As it turns out, the graphics card actually takes up a good chunk of
power unless proper power management is enabled (see below). After
tweaking this, I did manage to get down to around 7W power usage in
powertop.
Expansion cards actually do take up power, and so does the screen,
obviously. The fully-lit screen takes a solid 2-3W of power compared
to the fully dimmed screen. When removing all expansion cards and
making the laptop idle, I can spin it down to 4 watts power usage at
the moment, and an amazing 2 watts when the screen turned off.
Caveats
Abusive (10W+) power usage that I initially found could be a problem
with my desktop configuration: I have this silly status bar that
updates every second and probably causes redraws... The CPU certainly
doesn't seem to spin down below 1GHz. Also note that this is with an
actual desktop running with everything: it could very well be that
some things (I'm looking at you Signal Desktop) take up unreasonable
amount of power on their own (hello, 1W/electron, sheesh). Syncthing
and containerd (Docker!) also seem to take a good 500mW just sitting
there.
Beyond my desktop configuration, this could, of course, be a
Debian-specific problem; your favorite distribution might be better at
power management.
Idle power usage tests
Some expansion cards waste energy, even when unused. Here is a summary
of the findings from the powerstat page. I also include other
devices tested in this page for completeness:
Device
Minimum
Average
Max
Stdev
Note
Screen, 100%
2.4W
2.6W
2.8W
N/A
Screen, 1%
30mW
140mW
250mW
N/A
Backlight 1
290mW
?
?
?
fairly small, all things considered
Backlight 2
890mW
1.2W
3W?
460mW?
geometric progression
Backlight 3
1.69W
1.5W
1.8W?
390mW?
significant power use
Radios
100mW
250mW
N/A
N/A
USB-C
N/A
N/A
N/A
N/A
negligible power drain
USB-A
10mW
10mW
?
10mW
almost negligible
DisplayPort
300mW
390mW
600mW
N/A
not passive
HDMI
380mW
440mW
1W?
20mW
not passive
1TB SSD
1.65W
1.79W
2W
12mW
significant, probably higher when busy
MicroSD
1.6W
3W
6W
1.93W
highest power usage, possibly even higher when busy
Ethernet
1.69W
1.64W
1.76W
N/A
comparable to the SSD card
So it looks like all expansion cards but the USB-C ones are active,
i.e. they draw power with idle. The USB-A cards are the least concern,
sucking out 10mW, pretty much within the margin of error. But both the
DisplayPort and HDMI do take a few hundred miliwatts. It looks like
USB-A connectors have this fundamental flaw that they necessarily draw
some powers because they lack the power negotiation features of
USB-C. At least according to this post:
It seems the USB A must have power going to it all the time, that
the old USB 2 and 3 protocols, the USB C only provides power when
there is a connection. Old versus new.
Apparently, this is a problem specific to the USB-C to USB-A
adapter that ships with the Framework. Some people have actually
changed their orders to all USB-C because of this problem, but I'm
not sure the problem is as serious as claimed in the forums. I
couldn't reproduce the "one watt" power drains suggested elsewhere,
at least not repeatedly. (A previous version of this post did show
such a power drain, but it was in a less controlled test
environment than the series of more rigorous tests above.)
The worst offenders are the storage cards: the SSD drive takes at
least one watt of power and the MicroSD card seems to want to take all
the way up to 6 watts of power, both just sitting there doing
nothing. This confirms claims of 1.4W for the SSD (but not
5W) power usage found elsewhere. The former post has
instructions on how to disable the card in software. The MicroSD card
has been reported as using 2 watts, but I've seen it as high as 6
watts, which is pretty damning.
The Framework team has a beta update for the DisplayPort adapter
but currently only for Windows (LVFS technically possible, "under
investigation"). A USB-A firmware update is alsounder
investigation. It is therefore likely at least some of those power
management issues will eventually be fixed.
Note that the upcoming Ethernet card has a reported 2-8W power usage,
depending on traffic. I did my own power usage tests in
powerstat-wayland and they seem lower than 2W.
The upcoming 6.2 Linux kernel might also improve battery usage when
idle, see this Phoronix article for details, likely in early
2023.
Idle power usage tests under Wayland
Update: I redid those tests under Wayland, see powerstat-wayland
for details. The TL;DR: is that power consumption is either smaller or
similar.
Idle power usage tests, 3.06 beta BIOS
I redid the idle tests after the 3.06 beta BIOS update and ended
up with this results:
Device
Minimum
Average
Max
Stdev
Note
Baseline
1.96W
2.01W
2.11W
30mW
1 USB-C, screen off, backlight off, no radios
2 USB-C
1.95W
2.16W
3.69W
430mW
USB-C confirmed as mostly passive...
3 USB-C
1.95W
2.16W
3.69W
430mW
... although with extra stdev
1TB SSD
3.72W
3.85W
4.62W
200mW
unchanged from before upgrade
1 USB-A
1.97W
2.18W
4.02W
530mW
unchanged
2 USB-A
1.97W
2.00W
2.08W
30mW
unchanged
3 USB-A
1.94W
1.99W
2.03W
20mW
unchanged
MicroSD w/o card
3.54W
3.58W
3.71W
40mW
significant improvement! 2-3W power saving!
MicroSD w/ card
3.53W
3.72W
5.23W
370mW
new measurement! increased deviation
DisplayPort
2.28W
2.31W
2.37W
20mW
unchanged
1 HDMI
2.43W
2.69W
4.53W
460mW
unchanged
2 HDMI
2.53W
2.59W
2.67W
30mW
unchanged
External USB
3.85W
3.89W
3.94W
30mW
new result
Ethernet
3.60W
3.70W
4.91W
230mW
unchanged
Note that the table summary is different than the previous table: here
we show the absolute numbers while the previous table was doing a
confusing attempt at showing relative (to the baseline) numbers.
Conclusion: the 3.06 BIOS update did not significantly change idle
power usage stats except for the MicroSD card which has
significantly improved.
The new "external USB" test is also interesting: it shows how the
provided 1TB SSD card performs (admirably) compared to existing
devices. The other new result is the MicroSD card with a card which,
interestingly, uses less power than the 1TB SSD drive.
That's 8mAh per 10 minutes (and 2 seconds), or 48mA, or, with this
battery, about 127 hours or roughly 5 days of standby. Not bad!
In comparison, here is my really old x220, before:
sep 29 22:13:54 emma systemd-sleep[176315]: /sys/class/power_supply/BAT0/energy_now = 5070 [mWh]
... after:
sep 29 22:23:54 emma systemd-sleep[176486]: /sys/class/power_supply/BAT0/energy_now = 4980 [mWh]
... which is 90 mwH in 10 minutes, or a whopping 540mA, which was
possibly okay when this battery was new (62000 mAh, so about 100
hours, or about 5 days), but this battery is almost dead and has
only 5210 mAh when full, so only 10 hours standby.
And here is the Framework performing a similar test, before:
... which is 49mAh in a little over 10 minutes (and 4 seconds), or
292mA, much more than the Purism, but half of the X220. At this rate,
the battery would last on standby only 12 hours!! That is pretty
bad.
Note that this was done with the following expansion cards:
2 USB-C
1 1TB SSD drive
1 USB-A with a hub connected to it, with keyboard and LAN
Preliminary tests without the hub (over one minute) show that it
doesn't significantly affect this power consumption (300mA).
This guide also suggests booting with nvme.noacpi=1 but this
still gives me about 5mAh/min (or 300mA).
Adding mem_sleep_default=deep to the kernel command line does make a
difference. Before:
... which is 2mAh in 74 seconds, which is 97mA, brings us to a more
reasonable 36 hours, or a day and a half. It's still above the x220
power usage, and more than an order of magnitude more than the Purism
laptop. It's also far from the 0.4% promised by upstream, which
would be 14mA for the 3500mAh battery.
It should also be noted that this "deep" sleep mode is a little more
disruptive than regular sleep. As you can see by the timing, it took
more than 10 seconds for the laptop to resume, which feels a little
alarming as your banging the keyboard to bring it back to life.
You can confirm the current sleep mode with:
# cat /sys/power/mem_sleep
s2idle [deep]
In the above, deep is selected. You can change it on the fly with:
... better! 6 mAh in about 6 minutes, works out to 63.5mA, so more
than two days standby.
A longer test:
oct 01 09:22:56 angela systemd-sleep[62978]: /sys/class/power_supply/BAT1/charge_now = 3327 [mAh]
oct 01 12:47:35 angela systemd-sleep[63219]: /sys/class/power_supply/BAT1/charge_now = 3147 [mAh]
That's 180mAh in about 3.5h, 52mA! Now at 66h, or almost 3 days.
I wasn't sure why I was seeing such fluctuations in those tests, but
as it turns out, expansion card power tests show that they do
significantly affect power usage, especially the SSD drive, which can
take up to two full watts of power even when idle. I didn't control
for expansion cards in the above tests running them with whatever
card I had plugged in without paying attention so it's likely the
cause of the high power usage and fluctuations.
It might be possible to work around this problem by disabling USB
devices before suspend. TODO. See also this post.
In the meantime, I have been able to get much better suspend
performance by unplugging all modules. Then I get this result:
oct 04 11:15:38 angela systemd-sleep[257571]: /sys/class/power_supply/BAT1/charge_now = 3203 [mAh]
oct 04 15:09:32 angela systemd-sleep[257866]: /sys/class/power_supply/BAT1/charge_now = 3145 [mAh]
Which is 14.8mA! Almost exactly the number promised by Framework! With
a full battery, that means a 10 days suspend time. This is actually
pretty good, and far beyond what I was expecting when starting down
this journey.
So, once the expansion cards are unplugged, suspend power usage is
actually quite reasonable. More detailed standby tests are available
in the standby-tests page, with a summary below.
There is also some hope that the Chromebook edition
specifically designed with a specification of 14 days standby
time could bring some firmware improvements back down to the
normal line. Some of those issues were reported upstream in April
2022, but there doesn't seem to have been any progress there
since.
TODO: one final solution here is suspend-then-hibernate, which
Windows uses for this
TODO: consider implementing the S0ix sleep states , see also troubleshooting
TODO: consider https://github.com/intel/pm-graph
Standby expansion cards test results
This table is a summary of the more extensive standby-tests I have performed:
Device
Wattage
Amperage
Days
Note
baseline
0.25W
16mA
9
sleep=deep nvme.noacpi=1
s2idle
0.29W
18.9mA
~7
sleep=s2idle nvme.noacpi=1
normal nvme
0.31W
20mA
~7
sleep=s2idle without nvme.noacpi=1
1 USB-C
0.23W
15mA
~10
2 USB-C
0.23W
14.9mA
same as above
1 USB-A
0.75W
48.7mA
3
+500mW (!!) for the first USB-A card!
2 USB-A
1.11W
72mA
2
+360mW
3 USB-A
1.48W
96mA
<2
+370mW
1TB SSD
0.49W
32mA
<5
+260mW
MicroSD
0.52W
34mA
~4
+290mW
DisplayPort
0.85W
55mA
<3
+620mW (!!)
1 HDMI
0.58W
38mA
~4
+250mW
2 HDMI
0.65W
42mA
<4
+70mW
Conclusions:
USB-C cards take no extra power on suspend, possibly less
than empty slots, more testing required
USB-A cards take a lot more power on suspend
(300-500mW) than on regular idle (~10mW, almost negligible)
1TB SSD and MicroSD cards seem to take a reasonable
amount of power (260-290mW), compared to their runtime
equivalents (1-6W!)
DisplayPort takes a surprising lot of power (620mW), almost
double its average runtime usage (390mW)
HDMI cards take, surprisingly, less power (250mW) in
standby than the DP card (620mW)
and oddly, a second card adds less power usage (70mW?!) than the
first, maybe a circuit is used by both?
Standby expansion cards test results, 3.06 beta BIOS
Framework recently (2022-11-07) announced that they will publish
a firmware upgrade to address some of the USB-C issues, including
power management. This could positively affect the above result,
improving both standby and runtime power usage.
The update came out in December 2022 and I redid my analysis with
the following results:
Device
Wattage
Amperage
Days
Note
baseline
0.25W
16mA
9
no cards, same as before upgrade
1 USB-C
0.25W
16mA
9
same as before
2 USB-C
0.25W
16mA
9
same
1 USB-A
0.80W
62mA
3
+550mW!! worse than before
2 USB-A
1.12W
73mA
<2
+320mW, on top of the above, bad!
Ethernet
0.62W
40mA
3-4
new result, decent
1TB SSD
0.52W
34mA
4
a bit worse than before (+2mA)
MicroSD
0.51W
22mA
4
same
DisplayPort
0.52W
34mA
4+
upgrade improved by 300mW
1 HDMI
?
38mA
?
same
2 HDMI
?
45mA
?
a bit worse than before (+3mA)
Normal
1.08W
70mA
~2
Ethernet, 2 USB-C, USB-A
Full results in standby-tests-306. The big takeaway for me is that
the update did not improve power usage on the USB-A ports which is a
big problem for my use case. There is a notable improvement on the
DisplayPort power consumption which brings it more in line with the
HDMI connector, but it still doesn't properly turn off on suspend
either.
Even worse, the USB-A ports now sometimes fails to resume after
suspend, which is pretty annoying. This is a known problem
that will hopefully get fixed in the final release.
I looked at building this myself but failed to run it. I opened a
RFP in Debian so that we can ship this in Debian, and also documented
my work there.
Note that there is now a counter that tracks charge/discharge
cycles. It's visible in tlp-stat -b, which is a nice
improvement:
Ethernet expansion card
The Framework ethernet expansion card is a fancy little doodle:
"2.5Gbit/s and 10/100/1000Mbit/s Ethernet", the "clear housing lets
you peek at the RTL8156 controller that powers it". Which is another
way to say "we didn't completely finish prod on this one, so it kind
of looks like we 3D-printed this in the shop"....
The card is a little bulky, but I guess that's inevitable considering
the RJ-45 form factor when compared to the thin Framework laptop.
I have had a serious issue when trying it at first: the link LEDs
just wouldn't come up. I made a full bug report in the forum and
with upstream support, but eventually figured it out on my own. It's
(of course) a power saving issue: if you reboot the machine, the links
come up when the laptop is running the BIOS POST check and even when
the Linux kernel boots.
I first thought that the problem is likely related to the
powertop service which I run at boot time to tweak some power saving
settings.
It seems like this:
By default, USB power saving is active in the kernel, but not
force-enabled for incompatible drivers. That is, devices that
support suspension will suspend, drivers that do not, will not.
So the fix is actually to uninstall tlp or disable that setting by
adding this to /etc/tlp.conf:
USB_AUTOSUSPEND=0
... but that disables auto-suspend on all USB devices, which may
hurt other power usage performance. I have found that a a
combination of:
USB_AUTOSUSPEND=1
USB_DENYLIST="0bda:8156"
and this on the kernel commandline:
usbcore.quirks=0bda:8156:k
... actually does work correctly. I now have this in my
/etc/default/grub.d/framework-tweaks.cfg file:
# net.ifnames=0: normal interface names ffs (e.g. eth0, wlan0, not wlp166
s0)
# nvme.noacpi=1: reduce SSD disk power usage (not working)
# mem_sleep_default=deep: reduce power usage during sleep (not working)
# usbcore.quirk is a workaround for the ethernet card suspend bug: https:
//guides.frame.work/Guide/Fedora+37+Installation+on+the+Framework+Laptop/
108?lang=en
GRUB_CMDLINE_LINUX="net.ifnames=0 nvme.noacpi=1 mem_sleep_default=deep usbcore.quirks=0bda:8156:k"
# fix the resolution in grub for fonts to not be tiny
GRUB_GFXMODE=1024x768
Other than that, I haven't been able to max out the card because I
don't have other 2.5Gbit/s equipment at home, which is strangely
satisfying. But running against my Turris Omnia
router, I could pretty much max a gigabit fairly easily:
The card doesn't require any proprietary firmware blobs which is
surprising. Other than the power saving issues, it just works.
In my power tests (see powerstat-wayland), the Ethernet card seems
to use about 1.6W of power idle, without link, in the above "quirky"
configuration where the card is functional but without autosuspend.
Proprietary firmware blobs
The framework does need proprietary firmware to operate. Specifically:
the WiFi network card shipped with the DIY kit is a AX210 card that
requires a 5.19 kernel or later, and the firmware-iwlwifi non-free firmware package
the Bluetooth adapter also loads the firmware-iwlwifi
package (untested)
the graphics work out of the box without firmware, but certain
power management features come only with special proprietary
firmware, normally shipped in the firmware-misc-nonfree
but currently missing from the package
Note that, at the time of writing, the latest i915 firmware from
linux-firmware has a serious bug where loading all the
accessible firmware results in noticeable I estimate 200-500ms lag
between the keyboard (not the mouse!) and the display. Symptoms also
include tearing and shearing of windows, it's pretty nasty.
One workaround is to delete the two affected firmware files:
cd /lib/firmware && rm adlp_guc_70.1.1.bin adlp_guc_69.0.3.bin
update-initramfs -u
You will get the following warning during build, which is good as
it means the problematic firmware is disabled:
W: Possible missing firmware /lib/firmware/i915/adlp_guc_69.0.3.bin for module i915
W: Possible missing firmware /lib/firmware/i915/adlp_guc_70.1.1.bin for module i915
But then it also means that critical firmware isn't loaded, which
means, among other things, a higher battery drain. I was able to move
from 8.5-10W down to the 7W range after making the firmware work
properly. This is also after turning the backlight all the way down,
as that takes a solid 2-3W in full blast.
The proper fix is to use some compositing manager. I ended up using
compton with the following systemd unit:
compton is orphaned however, so you might be tempted to use
picom instead, but in my experience the latter uses much
more power (1-2W extra, similar experience). I also tried
compiz but it would just crash with:
anarcat@angela:~$ compiz --replace
compiz (core) - Warn: No XI2 extension
compiz (core) - Error: Another composite manager is already running on screen: 0
compiz (core) - Fatal: No manageable screens found on display :0
When running from the base session, I would get this instead:
Also note that the iwlwifi firmware also looks incomplete. Even with
the package installed, I get those errors in dmesg:
[ 19.534429] Intel(R) Wireless WiFi driver for Linux
[ 19.534691] iwlwifi 0000:a6:00.0: enabling device (0000 -> 0002)
[ 19.541867] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-72.ucode (-2)
[ 19.541881] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-72.ucode (-2)
[ 19.541882] iwlwifi 0000:a6:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-72.ucode failed with error -2
[ 19.541890] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-71.ucode (-2)
[ 19.541895] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-71.ucode (-2)
[ 19.541896] iwlwifi 0000:a6:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-71.ucode failed with error -2
[ 19.541903] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-70.ucode (-2)
[ 19.541907] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-70.ucode (-2)
[ 19.541908] iwlwifi 0000:a6:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-70.ucode failed with error -2
[ 19.541913] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-69.ucode (-2)
[ 19.541916] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-69.ucode (-2)
[ 19.541917] iwlwifi 0000:a6:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-69.ucode failed with error -2
[ 19.541922] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-68.ucode (-2)
[ 19.541926] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-68.ucode (-2)
[ 19.541927] iwlwifi 0000:a6:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-68.ucode failed with error -2
[ 19.541933] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-67.ucode (-2)
[ 19.541937] iwlwifi 0000:a6:00.0: firmware: failed to load iwlwifi-ty-a0-gf-a0-67.ucode (-2)
[ 19.541937] iwlwifi 0000:a6:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-67.ucode failed with error -2
[ 19.544244] iwlwifi 0000:a6:00.0: firmware: direct-loading firmware iwlwifi-ty-a0-gf-a0-66.ucode
[ 19.544257] iwlwifi 0000:a6:00.0: api flags index 2 larger than supported by driver
[ 19.544270] iwlwifi 0000:a6:00.0: TLV_FW_FSEQ_VERSION: FSEQ Version: 0.63.2.1
[ 19.544523] iwlwifi 0000:a6:00.0: firmware: failed to load iwl-debug-yoyo.bin (-2)
[ 19.544528] iwlwifi 0000:a6:00.0: firmware: failed to load iwl-debug-yoyo.bin (-2)
[ 19.544530] iwlwifi 0000:a6:00.0: loaded firmware version 66.55c64978.0 ty-a0-gf-a0-66.ucode op_mode iwlmvm
Some of those are available in the latest upstream firmware package
(iwlwifi-ty-a0-gf-a0-71.ucode, -68, and -67), but not all
(e.g. iwlwifi-ty-a0-gf-a0-72.ucode is missing) . It's unclear what
those do or don't, as the WiFi seems to work well without them.
I still copied them in from the latest linux-firmware package in the
hope they would help with power management, but I did not notice a
change after loading them.
There are also multiple knobs on the iwlwifi and iwlmvm
drivers. The latter has a power_schmeme setting which defaults to
2 (balanced), setting it to 3 (low power) could improve
battery usage as well, in theory. The iwlwifi driver also has
power_save (defaults to disabled) and power_level (1-5, defaults
to 1) settings. See also the output of modinfo iwlwifi and
modinfo iwlmvm for other driver options.
Graphics acceleration
After loading the latest upstream firmware and setting up a
compositing manager (compton, above), I tested the classic
glxgears.
Running in a window gives me odd results, as the gears basically grind
to a halt:
Running synchronized to the vertical refresh. The framerate should be
approximately the same as the monitor refresh rate.
137 frames in 5.1 seconds = 26.984 FPS
27 frames in 5.4 seconds = 5.022 FPS
Ouch. 5FPS!
But interestingly, once the window is in full screen, it does hit the
monitor refresh rate:
300 frames in 5.0 seconds = 60.000 FPS
I'm not really a gamer and I'm not normally using any of that fancy
graphics acceleration stuff (except maybe my browser does?).
I installed intel-gpu-tools for the intel_gpu_top
command to confirm the GPU was engaged when doing those simulations. A
nice find. Other useful diagnostic tools include glxgears and
glxinfo (in mesa-utils) and (vainfo in vainfo).
Following to this post, I also made sure to have those settings
in my about:config in Firefox, or, in user.js:
user_pref("media.ffmpeg.vaapi.enabled", true);
Note that the guide suggests many other settings to tweak, but those
might actually be overkill, see this comment and its parents. I
did try forcing hardware acceleration by setting gfx.webrender.all
to true, but everything became choppy and weird.
The guide also mentions installing the intel-media-driver package,
but I could not find that in Debian.
The Arch wiki has, as usual, an excellent reference on hardware
acceleration in Firefox.
Chromium / Signal desktop bugs
It looks like both Chromium and Signal Desktop misbehave with my
compositor setup (compton + i3). The fix is to add a persistent
flag to Chromium. In Arch, it's conveniently in
~/.config/chromium-flags.conf but that doesn't actually work in
Debian. I had to put the flag in
/etc/chromium.d/disable-compositing, like this:
It's possible another one of the hundreds of flags might fix this
issue better, but I don't really have time to go through this entire,
incomplete, and unofficial list (!?!).
Signal Desktop is a similar problem, and doesn't reuse those flags
(because of course it doesn't). Instead I had to rewrite the wrapper
script in /usr/local/bin/signal-desktop to use this instead:
exec /usr/bin/flatpak run --branch=stable --arch=x86_64 org.signal.Signal --disable-gpu-compositing "$@"
This was mostly done in this Puppet commit.
I haven't figured out the root of this problem. I did try using
picom and xcompmgr; they both suffer from the same issue. Another
Debian testing user on Wayland told me they haven't seen this problem,
so hopefully this can be fixed by switching to
wayland.
Graphics card hangs
I believe I might have this bug which results in a total
graphical hang for 15-30 seconds. It's fairly rare so it's not too
disruptive, but when it does happen, it's pretty alarming.
The comments on that bug report are encouraging though: it seems this
is a bug in either mesa or the Intel graphics driver, which means many
people have this problem so it's likely to be fixed. There's actually
a merge request on mesa already (2022-12-29).
It could also be that bug because the error message I get is
actually:
Jan 20 12:49:10 angela kernel: Asynchronous wait on fence 0000:00:02.0:sway[104431]:cb0ae timed out (hint:intel_atomic_commit_ready [i915])
Jan 20 12:49:15 angela kernel: i915 0000:00:02.0: [drm] GPU HANG: ecode 12:0:00000000
Jan 20 12:49:15 angela kernel: i915 0000:00:02.0: [drm] Resetting chip for stopped heartbeat on rcs0
Jan 20 12:49:15 angela kernel: i915 0000:00:02.0: [drm] GuC firmware i915/adlp_guc_70.1.1.bin version 70.1
Jan 20 12:49:15 angela kernel: i915 0000:00:02.0: [drm] HuC firmware i915/tgl_huc_7.9.3.bin version 7.9
Jan 20 12:49:15 angela kernel: i915 0000:00:02.0: [drm] HuC authenticated
Jan 20 12:49:15 angela kernel: i915 0000:00:02.0: [drm] GuC submission enabled
Jan 20 12:49:15 angela kernel: i915 0000:00:02.0: [drm] GuC SLPC enabled
It's a solid 30 seconds graphical hang. Maybe the keyboard and
everything else keeps working. The latter bug report is quite long,
with many comments, but this one from January 2023 seems to say
that Sway 1.8 fixed the problem. There's also an earlier patch to
add an extra kernel parameter that supposedly fixes that too. There's
all sorts of other workarounds in there, for example this:
from this comment... So that one is unsolved, as far as the
upstream drivers are concerned, but maybe could be fixed through Sway.
Weird USB hangs / graphical glitches
I have had weird connectivity glitches better described in this
post, but basically: my USB keyboard and mice (connected over a
USB hub) drop keys, lag a lot or hang, and I get visual glitches.
The fix was to tighten the screws around the CPU on the motherboard
(!), which is, thankfully, a rather simple repair.
USB docks are hell
Note that the monitors are hooked up to angela through a USB-C /
Thunderbolt dock from Cable Matters, with the lovely name of
201053-SIL. It has issues, see this blog
post for an in-depth discussion.
Shipping details
I ordered the Framework in August 2022 and received it about a month
later, which is sooner than expected because the August batch was
late.
People (including me) expected this to have an impact on the September
batch, but it seems Framework have been able to fix the delivery
problems and keep up with the demand.
As of early 2023, their website announces that laptops ship "within 5
days". I have myself ordered a few expansion cards in November 2022,
and they shipped on the same day, arriving 3-4 days later.
The supply pipeline
There are basically 6 steps in the Framework shipping pipeline, each
(except the last) accompanied with an email notification:
pre-order
preparing batch
preparing order
payment complete
shipping
(received)
This comes from the crowdsourced spreadsheet, which should be
updated when the status changes here.
I was part of the "third batch" of the 12th generation laptop, which
was supposed to ship in September. It ended up arriving on my door
step on September 27th, about 33 days after ordering.
It seems current orders are not processed in "batches", but in real
time, see this blog post for details on shipping.
Shipping trivia
I don't know about the others, but my laptop shipped through no less
than four different airplane flights. Here are the hops it took:
I can't quite figure out how to calculate exactly how much mileage
that is, but it's huge. The ride through Alaska is surprising enough
but the bounce back through Winnipeg is especially weird. I guess
the route happens that way because of Fedex shipping hubs.
There was a related oddity when I had my Purism laptop shipped: it
left from the west coast and seemed to enter on an endless, two week
long road trip across the continental US.
A new release 0.2.3 of pkgKitten
arrived on CRAN earlier, and
will be uploaded to Debian. pkgKitten
makes it simple to create new R packages via a simple function
invocation. A wrapper kitten.r exists in the littler
package to make it even easier.
This release improves the created Description: , and updated some of
the continuous integration.
Changes in version 0.2.3
(2023-03-11)
Small improvement to generated Description: field and
Title:
A new minor release 0.2.3 of our RcppRedis
package arrived on CRAN today.
RcppRedis
is one of several packages connecting R to the fabulous Redis in-memory datastructure store (and
much more). RcppRedis
does not pretend to be feature complete, but it may do some things
faster than the other interfaces, and also offers an optional coupling
with MessagePack binary
(de)serialization via RcppMsgPack. The
package has carried production loads on a trading floor for several
years.
This update is fairly mechanical. CRAN wants everybody off the C++11
train which is fair game given that it 2023 and most sane and lucky
people are facing sane and modern compilers so this makes sense. (And I
raise a toast to all those poor souls facing RHEL 7 / CentOS 7 with a
compiler from many moons ago: I hear it is a vibrant job market
out there so maybe time to make a switch ). As with a few of my other
packages, this release simply does away with the imposition of C++11 as
the package will compile just fine under C++14 or C++17 (as governed by
your version of R).
The detailed changes list follows.
Changes in version 0.2.3
(2023-03-08)
No longer set a C++ compilation standard as the default choices
by R are sufficient for the package
Switch include to Rcpp/Rcpp which signals use of all Rcpp
features including Modules
James Yang and I are
thrilled to announce the new CRAN package RcppFastAD which
arrived at CRAN last Monday as
version 0.0.1, and is as of today at version 0.0.2 with a first set of
small updates.
It is based on the FastAD header-only C++
library by James which provides a C++ implementation of both forward and
reverse mode of automatic differentiation in an easy-to-use header
library (which we wrapped here) that is both lightweight and performant.
With a little of bit of Rcpp glue, it
is also easy to use from R in simple C++ applications. Included in the
package are three example: a simple quadratic expression evaluating
x' S x for given x and S return the expression value with a
gradient, a linear regression example generalising this and using the
gradient to derive to arrive at the least-squares minimizing solution,
as well as the well-known Black-Scholes options pricer and its important
partial derivatives delta, rho, theta and vega derived via automatic
differentiation.
The NEWS file for these two initial releases follows.
Changes in version 0.0.2
(2023-03-05)
One C++ operation is protected from operating on a
nullptr
Additional tests have been added, tests now cover all three demo
/ example functions
Return values and code for the examples
linear_regression and quadratic_expression
have been adjusted
As I am sure everyone is aware, there is a growing interest in [SBOMs] as a
way of improving software security and resilience. In the last two years, the
US through the Exec Order, the EU through the proposed Cyber Resilience Act
(CRA) and this month the UK has issued a consultation paper looking at
software security and SBOMs appear very prominently in each
publication. []
Tim Retout wrote a blog post discussing AlmaLinux in the context of CentOS, RHEL and supply-chain security in general []:
Alma are generating and publishing Software Bill of Material (SBOM) files for
every package; these are becoming a requirement for all software sold to the
US federal government. What s more, they are sending these SBOMs to a third
party (CodeNotary) who store them in some sort of
Merkle tree system to make it
difficult for people to tamper with later. This should theoretically allow
end users of the distribution to verify the supply chain of the packages they
have installed?
Debian
Vagrant Cascadian noted that the Debian bookworm distribution has finally surpassed bullseye for reproducibility: 96.1% vs. 96.0%, despite having over 3500 more packages in the distribution.
Roland Clobus posted his latest update of the status of reproducible Debian ISO images noting that all major desktops build reproducibly with bullseye, bookworm and sid, with the caveat that when non-free firmware is activated, some non-reproducible files are generated .
23 reviews of Debian packages were added, 24 were updated and 20 were removed this month adding to our knowledge about identified issues. A new issue was added and identified by Chris Lamb [], and the timestamps_embedded_in_manpages_by_node_marked_man issue has been marked as resolved [].
Lastly, FC Stegerman reported two issues on Google s own issue tracker: one related to a non-deterministic Dependency Info Block [] and another about a virtual entry added by the signflinger tool causing unexpected differences between signed and unsigned APKs [].
diffoscopediffoscope is our in-depth and content-aware diff utility. Not only can it locate and diagnose reproducibility issues, it can provide human-readable diffs from many kinds of binary formats.
This month, Chris Lamb released versions 235 and 236; Mattia Rizzolo later released version 237.
Contributions include:
Chris Lamb:
Fix compatibility with PyPDF2 (re. issue #331) [][][].
Fix compatibility with ImageMagick version 7.1 [].
Require at least version 23.1.0 to run the Black source code tests [].
Update debian/tests/control after merging changes from others [].
Don t write test data during a test [].
Update copyright years [].
Merged a large number of changes from others.
Akihiro Suda edited the .gitlab-ci.yml configuration file to ensure that versioned tags are pushed to the container registry [].
Daniel Kahn Gillmor provided a way to migrate from PyPDF2 to pypdf (#1029741).
Efraim Flashner updated the tool metadata for isoinfo on GNU Guix [].
FC Stegerman added support for Android resources.arsc files [], improved a number of file-matching regular expressions [][] and added support for Android dexdump []; they also fixed a test failure (#1031433) caused by Debian s black package having been updated to a newer version.
updated the autopkgtest configuration to only install aapt and dexdump on architectures where they are available [], making sure that the latest diffoscope release is in a good fit for the upcoming Debian bookworm freeze.
reprotestReprotest version 0.7.23 was uploaded to both PyPI and Debian unstable, including the following changes:
Holger Levsen improved a lot of documentation [][][], tidied the documentation as well [][], and experimented with a new --random-locale flag [].
Vagrant Cascadian adjusted reprotest to no longer randomise the build locale and use a UTF-8 supported locale instead [ ] (re. #925879, #1004950), and to also support passing --vary=locales.locale=LOCALE to specify the locale to vary [].
Upstream patches
The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:
Testing framework
The Reproducible Builds project operates a comprehensive testing framework (available at tests.reproducible-builds.org) in order to check packages and other artifacts for reproducibility. In February, the following changes were made by Holger Levsen:
Add three new OSUOSL nodes [][][] and decommission the osuosl174 node [].
Change the order of listed Debian architectures to show the 64-bit ones first [].
Reduce the frequency that the Debian package sets and dd-list HTML pages update [].
Sort Tested suite consistently (and Debian unstable first) [].
Update the Jenkins shell monitor script to only query disk statistics every 230min [] and improve the documentation [][].
Other development work
disorderfs version 0.5.11-3 was uploaded by Holger Levsen, fixing a number of issues with the manual page [][][].
Bernhard M. Wiedemann published another monthly report about reproducibility within openSUSE.
If you are interested in contributing to the Reproducible Builds project, please visit the Contribute page on our website. You can get in touch with us via:
On Tue 13 July 2021 there was a
blog post
of new XMPP related software releases which have been uploaded to Debian 11 (bullseye).
Today, we will inform you about updates for the upcoming Debian release bookworm.
A lot of new releases have been provided by the upstream projects. There were lot of changes
to the XMPP clients like Dino, Gajim, Profanity, Poezio and others. Also the XMPP servers have been
enhanced.
Unfortunately, we can not provide a list of all the changes which have been done,
but will try to highlight some of the changes and new features.
BTW, feel free to join the Debian User Support on Jabber at
xmpp:debian@conference.debian.org?join.
You can find a list of 58 packages of the Debian XMPP team on the
XMPP QA Page.
Dino, modern XMPP client has been upgraded from
0.2.0 to 0.4.0. The new version supports encrypted calls and group calls and
reactions give you a way to respond to a message with an emoji.
You can find more information about Dino 0.3.0 and
Dino 0.4.0 in
the release notes of the upstream project. Dino is using GTK4 /
libadwaita which provides widgets for mobile-friendly UIs.
Changes has been done on the main view of Dino.
Gajim, a GTK+-based Jabber client has been upgraded from 1.3.1
to 1.7.1. Since 1.4 Gajim has got a new UI, which supports
spaces. 1.5.2 supports a content viewer for PEP nodes. 1.6.0 is
using libsoup3 and python 3.10. Audio preview looks a lot nicer
with a wave graph visualization and profile images (avatar) are
not limited to only JPG anymore. The plugins
gajim-appindicatorintegration, gajim-plugininstaller,
gajim-syntaxhighlight und gajim-urlimagepreview are obsolete,
these features has been moved to gajim.
There were a lot of releases in Gajim. You can find the full
story at https://gajim.org/post/
Profanity, the console based XMPP client has been upgraded
from 0.10.0 to 0.13.1. Profanity supports XEP-0377 Spam
Reporting, and XEP-0157 server contact information discovery.
It now marks a window with an attention flag, updated HTTP Upload
XEP-0363, and messages can be composed with an external editor. It also features easy quoting,
in-band account registration (XEP-0077), Print OMEMO verification
QR code, and many more.
Kaidan, a simple and user-friendly Jabber/XMPP client based on Qt has been
updated from 0.7.0 to 0.8.0. The new release supports
XEP-0085: Chat State Notifications and XEP-0313: Message
Archive Management.
Poezio, a console-based XMPP client as been updated from 0.13.1
to 0.14. Poezio is now under GPLv3+. The new release supports
request for voice and the /join command support using an XMPP
URI. More information at
https://lab.louiz.org/poezio/poezio/-/raw/v0.14/CHANGELOG.
[Swift][swift-im], back in Debian is the Swift XMPP client - a
cross-platform Client written in C++. In 2015 the client was
removed from testing and is back with version 5.0.
Server
prosody the lightweight extensible XMPP server has been
upgraded from 0.11.9 to 0.12.2. Mobile and connectivity
optimizations, a new module for HTTP file sharing, audio/video
calling support. See the release
announcement
for more info. You will also find a lot of new modules which have
been added to 0.12.0.
The version 0.12.3 is waiting migration from unstable to testing.
ejabberd, extensible realtime platform (XMPP server + MQTT
broker + SIP service) has been updated from Version 21.01 to 23.01.
The new version supports the latest version of MIX (XEP-0369).
There were also changes for SQL and MUC. See the release
information for 22.10
and 23.01 for more details.
Libs
libstrophe, xmpp C lib has been upgraded from 0.10.1 to
0.12.2. The lib has SASL EXTERNAL support (XEP-0178),
support for manual certificate verification and Stream Management
support (XEP-0198).
libomemo-c, new in Debian with version 0.5.0 - a fork of libsignal-protocol-c
Others
There were some changes of the Libervia, formerly known as Salut Toi (SaT) packages in Debian.
The most visible change is, that Salut Toi has been renamed to libervia:
salutatoi is now libervia-backend (0.9.0)
sat-xmpp-primitivus is now libervia-tui
sat-xmpp-core is now libervia-backend
sat-xmpp-jp is now libervia-cli
sat-pubsub is now libervia-pubsub (0.4.0)
gsasl has been updated from 1.10.0 to 2.2.0
libxeddsa 2.0.0 is new in Debian - toolkit around Curve25519 and Ed25519 key pairs
Happy chatting - keep in touch with your family and friends via Jabber / XMPP -
XMPP is an open standard of the Internet Engineering Task Force (IETF) for instant messaging.