Testing Picolibc with the glibc tests Picolibc has a bunch of built-in tests, but more testing is always better, right? I decided to see how hard it would be to run some of the tests provided in the GNU C Library (glibc). Parallel meson build files Similar to how Picolibc uses meson build files to avoid modifying the newlib autotools infrastructure, I decided to take the glibc code and write meson build rules that would compile the tests against Picolibc header files and link against Picolibc libraries. I decided to select a single target for this project so I could focus on getting things building and not worry too much about making it portable. I wanted to pick something that had hardware floating point so that I would have rounding modes and exception support, so I picked the ARM Cortex M7 with hard float ABI:
It should be fairly easy to extend this to other targets, but for now,
that's what I've got working. There's a cross-cortex-m7.txt file
containing all of the cross compilation details which is used when
$ arm-none-eabi-gcc -mcpu=cortex-m7 -mfloat-abi=hard
All of the Picolibc-specific files live in a new picolibc directory so
they are isolated from the main glibc code.
Pre-loading a pile of hacks
Adapt Picolibc to support the Glibc test code required a bunch of
random hacks, from providing
_unlocked versions of the stdio macros
to stubbing out various unsupportable syscalls (like
chdir). Instead of modifying the Glibc code, I created a file called
hacks.h which is full of this stuff and used the gcc
parameter to read that into the compiler before starting compilation
on all of the files.
Supporting command line parameters
The glibc tests all support a range of command line parameters, some
of which turned out to be quite useful for this work. Picolibc had
limited semihosting support for accessing the command line, but that
required modifying applications to go fetch the command line using a
special semihosting function.
To make this easier, I added a new crt0 variant for picolibc called
(oddly) semihost. This extends the existing hosted variant by
adding a call to the semihosting library to fetch the current command
line and splitting that into words at each space. It doesn't handle
any quoting, but it's sufficient for the needs here.
Avoiding glibc headers
The glibc tests use some glibc-specific extensions to the standard
POSIX C library, so I needed to include those in the test
builds. Headers for those extensions are mixed in with the rest of the
POSIX standard headers, which conflict with the Picolibc versions. To
work around this, I stuck stub #include files in the picolibc directory
which directly include the appropriate headers for the glibc API
extensions. This includes things like
array_length.h. For other headers which weren't actually needed for
picolibc, I created empty files.
Adding more POSIX to Picolibc
At this point, code was compiling but failing to find various standard
POSIX functions which aren't available in Picolibc. That included some
syscalls which could be emulated using semihosting, like
getpagesize. It also included some generally
useful additions, like replacing
_r variants provide a target buffer size instead
of assuming that it was large enough as the Picolibc
Which tests are working?
So far, I've got some of the tests in malloc, math, misc and
There are a lot of tests in the malloc directory which cover glibc API
extensions or require POSIX syscalls not supported by semihosting. I
think I've added all of the tests which should be supported.
For the math tests, I'm testing the standard POSIX math APIs in both
float and double forms, except for the Bessel and Gamma
functions. Picolibc's versions of those are so bad that they violate
some pretty basic assumptions about error bounds built into the glibc
test code. Until Picolibc gets better versions of these functions,
we'll have to skip testing them this way.
In the misc directory, I've only added tests for ecvt, fcvt, gcvt,
dirname and hsearch. I don't think there are any other tests
there which should work.
Finally, for stdio-common, almost all of the tests expect a fully
functioning file system, which semihosting really can't support. As a
result, we're only able to run the printf, scanf and some of the
All in all, we're running 78 of the glibc test programs, which is a
small fraction of the total tests, but I think it's the bulk of the
tests which cover APIs that don't depend too much on the underlying
POSIX operating system.
Bugs found and fixed in Picolibc
This exercise has resulted in 17 fixes in Picolibc, which can be
- Math functions taking sNaN and not raising FE_INVALID and
returning qNaN. Almost any operation on an sNaN value is supposed
to signal an invalid operand and replace that with a qNaN so that
further processing doesn't raise another exception. This was
fairly easy to fix, just need to use
return x + x;instead of
- Math functions failing to set errno. I'd recently restructured the math library to get rid of the separate IEEE version of the functions which didn't set errno and missed a couple of cases that needed to use the errno-setting helper functions.
- Corner cases in string/float conversion, including the need to perform round-to-even for '%a' conversions in printf and supporting negative decimal values for fcvt. This particular exercise led to replacing the ecvtbuf and fcvtbuf APIs with glibc's ecvt_r and fcvt_r versions as those pass explicit buffer lengths, making overflow prevention far more reliable.
- A bunch of malloc entry points were not handling failure correctly; allocation values close to the maximum possible size caused numerous numeric wrapping errors with the usual consequences (allocations "succeed", but return a far smaller buffer than requested). Also, realloc was failing to check the return value from malloc before copying the old data, which really isn't a good idea.
- Tinystdio's POSIX support code was returning an error instead of EOF at end of file.
- Error bounds for the Picolibc math library aren't great; I had to generate Picolibc-specific ulps files. Most functions are between 0 and 3 ulps, but for some reason, the float version of erfc (ercf) has errors as large as 64 ulps. That needs investigation.