This post is about some changes recently made to
Rust's
ErrorKind, which aims to categorise OS errors
in a portable way.
Audiences for this post
- The educated general reader interested in a case study involving
error handling, stability, API design, and/or Rust.
- Rust users who have tripped over these changes.
If this is you, you can cut to the chase and skip to
How to fix.
Background and context
Error handling principles
Handling different errors differently is often important (although,
sadly, often neglected). For example, if a program tries to read
its default configuration file, and gets a "file not found" error,
it can proceed with its default configuration, knowing that the user
hasn't provided a specific config.
If it gets some other error, it should probably complain and quit,
printing the message from the error (and the filename). Otherwise,
if the network fileserver is down (say), the program might
erroneously run with the default configuration and do something
entirely wrong.
Rust's portability aims
The Rust programming language tries to make it straightforward to
write portable code. Portable error handling is always a bit
tricky. One of Rust's facilities in this area is
std::io::ErrorKind
which is an enum which tries to categorise (and, sometimes,
enumerate) OS errors. The idea is that a program can check the
error kind, and handle the error accordingly.
That these
ErrorKinds are part of the Rust standard
library means that to get this right, you don't need to delve down
and get the actual underlying operating system error number, and
write separate code for each platform you want to support. You can
check whether the error is
ErrorKind::NotFound
(or whatever).
Because
ErrorKind is so important in many Rust APIs,
some code which isn't really doing an OS call can still have to
provide an
ErrorKind. For this purpose, Rust provides
a special category
ErrorKind::Other, which doesn't
correspond to any particular OS error.
Rust's stability aims and approach
Another thing Rust tries to do is keep existing code working.
More specifically, Rust tries to:
- Avoid making changes which would contradict the
previously-published documentation of Rust's language and
features.
- Tell you if you accidentally rely on properties which
are not part of the published documentation.
By and large, this has been very successful. It means
that if you write code now, and it compiles and runs cleanly, it is
quite likely that it will continue work properly in the future, even
as the language and ecosystem evolves.
This blog post is about a case where Rust failed to do (2), above,
and, sadly, it turned out that several people had accidentally
relied on something the Rust project definitely intended to change.
Furthermore, it was something which
needed to change.
And the new (corrected) way of using the API is not so obvious.
Rust enums, as relevant to
io::ErrorKind
(Very briefly:)
When you have a value which is an
io::ErrorKind, you
can compare it with specific values:
if error.kind() == ErrorKind::NotFound ...
But in Rust it's more usual to write something like this
(which you can read like a
switch statement):
match error.kind()
ErrorKind::NotFound => use_default_configuration(),
_ => panic!("could not read config file : ", &file, &error),
Here
_ means "anything else".
Rust insists that
match statements are
exhaustive, meaning that each one covers all the possibilities.
So if you left out the line with the
_, it wouldn't
compile.
Rust enums can also be marked
non_exhaustive, which is
a declaration by the API designer that they plan to add more kinds.
This has been done for
ErrorKind, so the
_
is mandatory, even if you write out all the possibilities that exist
right now: this ensures that if new
ErrorKinds appear,
they won't stop your code compiling.
Improving the error categorisation
The set of error categories stabilised in Rust 1.0 was too small.
It missed many important kinds of error. This makes writing
error-handling code awkward. In any case, we expect to add new
error categories occasionally. I set about trying to improve this
by
proposing
new ErrorKinds. This obviously needed considerable
community review, which is why it took about 9 months.
The trouble with
Other and tests
Rust has to assign an
ErrorKind to
every OS error,
even ones it doesn't really know about.
Until recently, it mapped all errors it didn't understand to
ErrorKind::Other - reusing the category for "not an OS
error at all".
Serious people who write serious code like to have serious tests.
In particular, testing error conditions is really important. For
example, you might want to test your program's handling of disk
full, to make sure it didn't crash, or corrupt files. You would set
up some contraption that would simulate a full disk. And then, in
your tests, you might check that the error was correct.
But until very recently (still now, in Stable Rust), there was
no
ErrorKind::StorageFull. You would
get
ErrorKind::Other. If you were diligent you would
dig out the OS error code (and check for
ENOSPC on
Unix, corresponding Windows errors, etc.). But that's tiresome.
The more obvious thing to do is to check that the kind
is
Other.
Obvious but wrong.
ErrorKind is
non_exhaustive,
implying that more error kinds will appears, and, naturally, these
would more finely categorise previously-
Other OS errors.
Unfortunately, the documentation note
Errors that are Other now may move to a different or a new
ErrorKind variant in the future.
was only added in May 2020.
So the wrongness of the "obvious" approach was, itself, not very
obvious.
And even with that docs note, there was no compiler warning or anything.
The unfortunate result is that there is a body of code out there
in the world which might break any time an error that was
previously
Other becomes properly categorised.
Furthermore, there was nothing stopping new people writing new
obvious-but-wrong code.
Chosen solution:
Uncategorized
The Rust developers wanted an engineered safeguard against the bug of
assuming that a particular error shows up as
Other.
They
chose the following solution:
There is now a new
ErrorKind::Uncategorized which is
now used for all OS errors for which there isn't a more specific
categorisation. The fallback translation of unknown errors was
changed from
Other to
Uncategorised.
This is de jure justified by the fact that this enum has always been
marked
non_exhaustive. But in practice because this
bug wasn't previously detected, there is such code in the wild.
That code now breaks (usually, in the form of failing test cases).
Usually when Rust starts to detect a particular programming error,
it is reported as a new warning, which doesn't break anything. But
that's not possible here, because this is a behavioural change.
The new
ErrorKind::Uncategorized is
marked
unstable. This makes it impossible to write
code on Stable Rust which insists that an error comes out
as
Uncategorized. So, one cannot now write code that
will break when new
ErrorKinds are added.
That's the intended effect.
The downside is that this does break old code, and, worse,
it is not as clear as it should be what the fixed code looks like.
Alternatives considered and rejected by the Rust developers
Not adding more
ErrorKinds
This was not tenable. The existing set is already too small, and
error categorisation is in any case expected to improve over time.
Just adding
ErrorKinds as had been done before
This would mean occasionally breaking test cases (or, possibly,
production code) when an error that was
previously
Other becomes categorised. The broken code
would have been "obvious", but de jure wrong, just as it is now,
So this option amounts to expecting this broken code to continue
to be written and continuing to break it occasionally.
Somehow using Rust's Edition system
The Rust language has a system to allow language evolution,
where code declares its Edition (2015, 2018, 2021).
Code from multiple editions can be combined, so that
the ecosystem can upgrade gradually.
It's not clear how this could be used for
ErrorKind, though.
Errors have to be passed between code with different editions.
If those different editions had different categorisations, the
resulting programs would have incoherent and broken error handling.
Also some of the schemes for making this change would mean that new
ErrorKinds could only be stabilised about once every 3
years, which is far too slow.
How to fix code broken by this change
Most main-line error handling code already has a fallback case for
unknown errors. Simply replacing any occurrence
of
Other with
_ is right.
How to fix thorough tests
The tricky problem is tests. Typically, a thorough test case wants
to check that the error is "precisely as expected" (as far as the
test can tell). Now that unknown errors come out as an
unstable
Uncategorized variant that's not so easy. If
the test is expecting an error that is currently not categorised,
you want to write code that says "if the error is any of the
recognised kinds, call it a test failure".
What does "any of the recognised kinds" mean here ? It doesn't
meany any of the kinds recognised
by the version of the Rust
stdlib that is actually in use. That set might get bigger.
When the test is compiled and run later, perhaps years later, the
error in this test case might indeed be categorised. What you
actually mean is "the error must not be any of the kinds
which
existed when the test was written".
IMO therefore the right solution for such a test case is to
cut and
paste the current list of stable
ErrorKinds into your
code. This will seem wrong at first glance, because the list in
your code and in Rust can get out of step. But when they do get out
of step
you want your version, not the stdlib's. So freezing the
list at a point in time is precisely right.
You probably only want to maintain one copy of this list, so put it
somewhere central in your codebase's test support machinery.
Periodically, you can update the list deliberately - and fix any
resulting test failures.
Unfortunately this approach is not suggested by the documentation.
In theory you could work all this out yourself from first
principles, given even the situation prior to May 2020, but it seems
unlikely that many people have done so. In particular, cutting and
pasting the list of recognised errors would seem very unnatural.
Conclusions
This was not an easy problem to solve well.
I think Rust has done a plausible job given the various constraints,
and the result is technically good.
It is a shame that this change to make the error handling stability
more correct caused the most trouble for the most careful people who
write the most thorough tests.
I also
think the docs could be improved.
edited shortly after posting, and again 2021-09-22 16:11 UTC, to fix HTML slips
comments
For some months now I ve been working on some patches to
Consfigurator to add support for Linux containers.
My goal is to make Consfigurator capable of both performing the initial setup
of a container and of entering the running container to apply configuration.
For the case of unprivileged LXCs running as non-root, my work-in-progress
branch can now do both of these things. As Consfigurator enters the container
directly using system calls, it should be decently fast at configuring
multiple containers on a host, and it will also be possible to have it do this
in parallel. The initial setup for the container uses Consfigurator s
existing support for building root filesystems, and it should be easy to
extend that to support arbitrary GNU/Linux distributions by teaching
Consfigurator how to invoke bootstrapping tools other than debootstrap(8).
Here s an example:
Another new release of 
So I decided to create a solution for this, always thinking about simplicity and in a way that makes it easy to get this missing functionality. I started by creating an input array pattern using 
This post was originally published in the 
The fifteenth release of 
Active noise cancellation (ANC) has been all the rage lately in the headphones
and in-ear monitors market. It seems after Apple got heavily praised for their
AirPods Pro, every somewhat serious electronics manufacturer released their
own design incorporating this technology.
The first headphones with ANC I remember trying on (in the early 2010s) were
the Bose QuietComfort 15. Although the concept did work (they indeed cancelled
some sounds), they weren't amazing and did a great job of convincing me ANC
was some weird fad for people who flew often.
As the years passed, chip size decreased, battery capacity improved and machine
learning blossomed truly a perfect storm for the wireless ANC headphones
market. I had mostly stayed a sceptic of this tech until recently a kind friend
offered to let me try a pair of Sony WH-1000X M3.
Having tested them thoroughly, I have to say I'm really tempted to buy them
from him, as they truly are fantastic headphones
I won't be keeping them though.
Whilst I really like what Sony has achieved here, I've grown to understand ANC
simply isn't for me. Some of the drawbacks of ANC somewhat bother me: the ear
pressure it creates is tolerable, but is an additional energy drain over long
periods of time and eventually gives me headaches. I've also found ANC
accentuates the motion sickness I suffer from, probably because it messes up
with some part of the inner ear balance system.
Most of all, I found that it didn't provide noticeable improvements over good
passive noise cancellation solutions, at least in terms of how high I have to
turn the volume up to hear music or podcasts clearly. The human brain works in
mysterious ways and it seems ANC cancelling a class of noises (low hums,
constant noises, etc.) makes other noises so much more noticeable. People
talking or bursty high pitched noises bothered me much more with ANC on than
without.
So for now, I'll keep using my trusty Senheiser HD 280 Pro
Last week has seen the release of the 
Starting with today, we provide binaries of Plasma 5.23 for Debian stable (bullseye), testing, and unstable, in each case for three architectures: amd64, i386, and aarch64.
To test the current beta, please add
In this blog post, I will cover a few related but also different topics around X2Go - the GNU/Linux based remote computing framework.
Introduction and Catch Up
For those, who haven't come across X2Go, so far... With X2Go [0] you can log into remote GNU/Linux machines graphically and launch headless desktop environments, seamless/published applications or access an already running desktop session (on a local Xserver or running as a headless X2Go desktop session) via X2Go's session shadowing / mirroring feature.
Graphical backend: NXv3
For several years, there was only one graphical backend available in X2Go, the NXv3 software. In NXv3, you have a headless or nested (it can do both) Xserver that has some remote magic built-in and is able to transfer the Xserver's graphical data to a remote client (NX proxy). Over the wire, the NX protocol allows for data compression (JPEG, PNG, etc.) and combines it with bitmap caching, so that the overall result is a fast and responsive desktop experience even on low latency and low bandwidth connections. This especially applies to X desktop environments that use many native X protocol operations for drawing windows and widget onto the screen. The more bitmaps involved (e.g. in applications with client-side rendering of window controls and such), the worse the quality of a session experience.
The current main maintainer of NVv3 (aka nx-libs [1]) is Ulrich Sibiller. Uli has my and the X2Go community's full appreciation, admiration and gratitude for all the work he does on nx-libs, constantly improving NXv3 without breaking compatibility with legacy use cases (yes, FreeNX is still alive, by the way).
NEW: Alternative Graphical Backend: X2Go Kdrive
Over the past 1.5 years, Oleksandr Shneyder (Alex), co-founder of X2Go, has been working on a re-implementation of an alternative, less X11-dependent graphical backend. The underlying Xserver technology is the kdrive part of the X.org server project. People on GNU/Linux might have used kdrive technology already: The Xephyr nested Xserver uses the kdrive implementation.
The idea of the X2Go Kdrive [2] implementation in X2Go is providing a headless Xserver on the X2Go Server side for running X11 based desktop sessions inside while using an X11-agnostic data protocol for sending the graphical desktop data to the client-side for rendering. Whereas, with NXv3 technology, you need a local Xserver on the client side, with X2Go Kdrive you only need a client app(lication) that can draw bitmaps into some sort of framebuffer, such as a client-side X11 Xserver, a client-side Wayland compositor or (hold your breath) an HTMLv5 canvas in a web browser.
X2Go Kdrive Client Implementations
During first half of this year, I tested and DEB-packaged Alex's X2Go HTMLv5 client code [3] and it has been available for testing in the X2Go nightly builds archive for a while now.
Of course, the native X2Go Client application has X2Go Kdrive support for a while, too, but it requires a Qt5 application in the background, the .png){kind=link}
_Manila_Skyline_-_panoramio.jpg){kind=link}