In 2020, Google introduced Core Web Vitals metrics to measure some aspects
of real-world user experience on the web. This blog has consistently achieved
good scores for two of these metrics: Largest Contentful Paint and
Interaction to Next Paint. However, optimizing the third metric, Cumulative
Layout Shift, which measures unexpected layout changes, has been more
challenging. Let s face it: optimizing for this metric is not really useful for
a site like this one. But getting a better score is always a good distraction.
To prevent the flash of invisible text when using web fonts, developers should
set the font-display property to swap in @font-face rules. This method
allows browsers to initially render text using a fallback font, then replace it
with the web font after loading. While this improves the LCP score, it causes
content reflow and layout shifts if the fallback and web fonts are not
metrically compatible. These shifts negatively affect the CLS score. CSS
provides properties to address this issue by overriding font metrics when using
fallback fonts: size-adjust,
ascent-override, descent-override,
and line-gap-override.
Two comprehensive articles explain each property and their computation methods
in detail: Creating Perfect Font Fallbacks in CSS and Improved font
fallbacks.
Interactive tuning tool
Instead of computing each property from font average metrics, I put together a
tool for interactively tuning fallback fonts.1
Instructions
Load your custom font.
Select a fallback font to tune.
Adjust the size-adjust property to match the width of your custom font with
the fallback font. With a proportional font, it is not possible to achieve a
perfect match.
Fine-tune the ascent-override property. Aim to align the final dot of the
last paragraph while monitoring the font s baseline. For more precise
adjustment, disable the option.
Modify the descent-override property. The goal is to make the two boxes
match. You may need to alternate between this and the previous property for
optimal results.
If necessary, adjust the line-gap-override property. This step is typically
not required.
The process needs to be repeated for each fallback font. Some platforms may not
include certain fonts. Notably, Android lacks most fonts found in other
operating systems. It replaces Georgia with Noto Serif, which is not
metrically-compatible.
Tool
This tool is not available from the Atom feed.
Results
For the body text of this blog, I get the following CSS definition:
@font-facefont-family:Merriweather;font-style:normal;font-weight:400;src:url("../fonts/merriweather.woff2")format("woff2");font-display:swap;@font-facefont-family:"Fallback for Merriweather";src:local("Noto Serif"),local("Droid Serif");size-adjust:98.3%;ascent-override:99%;descent-override:27%;@font-facefont-family:"Fallback for Merriweather";src:local("Georgia");size-adjust:106%;ascent-override:90.4%;descent-override:27.3%;font-family:Merriweather,"Fallback for Merriweather",serif;
After a month, the CLS metric improved to 0:
Recent Core Web Vitals scores for vincent.bernat.ch
About custom fonts
Using safe web fonts or a modern font stack is often simpler. However, I
prefer custom web fonts. Merriweather and Iosevka, which are used in
this blog, enhance the reading experience. An alternative approach could be to
use Georgia as a serif option. Unfortunately, most default monospace fonts are
ugly.
Furthermore, paragraphs that combine proportional and monospace fonts can create
visual disruption. This occurs due to mismatched vertical metrics or weights. To
address this issue, I adjust Iosevka s metrics and weight to align with
Merriweather s characteristics.
Similar tools already exist, like the Fallback Font Generator,
but they were missing a few features, such as the ability to load the
fallback font or to have decimals for the CSS properties. And no source
code.
Welcome to the May 2024 report from the Reproducible Builds project! In these reports, we try to outline what we have been up to over the past month and highlight news items in software supply-chain security more broadly. As ever, if you are interested in contributing to the project, please visit our Contribute page on our website.
Table of contents:
[ ] built by Homebrew will come with a cryptographically verifiable statement binding the bottle s content to the specific workflow and other build-time metadata that produced it. [ ] In effect, this injects greater transparency into the Homebrew build process, and diminishes the threat posed by a compromised or malicious insider by making it impossible to trick ordinary users into installing non-CI-built bottles.
The post also briefly touches on future work, including work on source provenance:
Homebrew s formulae already hash-pin their source artifacts, but we can go a step further and additionally assert that source artifacts are produced by the repository (or other signing identity) that s latent in their URL or otherwise embedded into the formula specification.
Distribution news
In Debian this month, Johannes Schauer Marin Rodrigues (aka josch) noticed that the Debian binary package bash version 5.2.15-2+b3 was uploaded to the archive twice. Once to bookworm and once to sid but with differing content. This is problem for reproducible builds in Debian due its assumption that the package name, version and architecture triplet is unique. However, josch highlighted that
This example with bash is especially problematic since bash is Essential:yes, so there will now be a large portion of .buildinfo files where it is not possible to figure out with which of the two differing bash packages the sources were compiled.
In response to this, Holger Levsen performed an analysis of all .buildinfo files and found that this needs almost 1,500 binNMUs to fix the fallout from this bug.
Elsewhere in Debian, Vagrant Cascadian posted about a Non-Maintainer Upload (NMU) sprint to take place during early June, and it was announced that there is now a #debian-snapshot IRC channel on OFTC to discuss the creation of a new source code archiving service to, perhaps, replace snapshot.debian.org. Lastly, 11 reviews of Debian packages were added, 15 were updated and 48 were removed this month adding to our extensive knowledge about identified issues. A number of issue types have been updated by Chris Lamb as well. [][]
Elsewhere in the world of distributions, deep within a larger announcement from Colin Percival about the release of version 14.1-BETA2, it was mentioned that the FreeBSD kernels are now built reproducibly.
In Fedora, however, the change proposal mentioned in our report for April 2024 was approved, so, per the ReproduciblePackageBuilds wiki page, the add-determinism tool is now running in new builds for Fedora 41 ( rawhide ). The add-determinism tool is a Rust program which, as its name suggests, adds determinism to files that are given as input by attempting to standardize metadata contained in binary or source files to ensure consistency and clamping to $SOURCE_DATE_EPOCH in all instances . This is essentially the Fedora version of Debian s strip-nondeterminism. However, strip-nondeterminism is written in Perl, and Fedora did not want to pull Perl in the buildroot for every package. The add-determinism tool eliminates many causes of non-determinism and work is ongoing to continue the scope of packages it can operate on.
Mailing list news
On our mailing list this month, regular contributor kpcyrd wrote to the list with an update on their source code indexing project, whatsrc.org. The whatsrc.org project, which was launched last month in response to the XZ Utils backdoor, now contains and indexes almost 250,000 unique source code archives. In their post, kpcyrd gives an example of its intended purpose, noting that it shown that whilst there seems to be consensus about [the] source code for zsh 5.9 in various Linux distributions, it does not align with the contents of the zsh Git repository .
Holger Levsen also posted to the list with a pre-announcement of sorts for the 2024 Reproducible Builds summit. In particular:
[Whilst] the dates and location are not fixed yet, however if you don help us with finding a suitable location soon, it is very likely that we ll meet again in Hamburg in the 2nd half of September 2024 [ ].
Miscellaneous news
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. This month strip-nondeterminism version 1.14.0-1 was uploaded to Debian unstable by Chris Lamb chiefly to incorporate a change from Alex Muntada to avoid a dependency on Sub::Override to perform monkey-patching and break circular dependencies related to debhelper []. Elsewhere in our tooling, Jelle van der Waa modified reprotest because the pipes module will be removed in Python version 3.13 [].
It was also noticed that a new blog post by Daniel Stenberg detailing How to verify a Curl release mentions the SOURCE_DATE_EPOCH environment variable. This is because:
The [curl] release tools document also contains another key component: the exact time stamp at which the release was done using integer second resolution. In order to generate a correct tarball clone, you need to also generate the new version using the old version s timestamp. Because the modification date of all files in the produced tarball will be set to this timestamp.
Furthermore, Fay Stegerman filed a bug against the Signal messenger app for Android to report that their reproducible builds cannot, in fact, be reproduced. However, Fay is quick to note that she has:
found zero evidence of any kind of compromise. Some differences are yet unexplained but everything I found seems to be benign. I am disappointed that Reproducible Builds have been broken for months but I have zero reason to doubt Signal s security in any way.
Lastly, it was observed that there was a concise and diagrammatic overview of supply chain threats on the SLSA website.
In this short [vision] paper we delve into the application of software engineering techniques, specifically variability management, to systematically identify and explicit points of variability that may give rise to reproducibility issues (e.g., language, libraries, compiler, virtual machine, OS, environment variables, etc.). The primary objectives are: i) gaining insights into the variability layers and their possible interactions, ii) capturing and documenting configurations for the sake of reproducibility, and iii) exploring diverse configurations to replicate, and hence validate and ensure the robustness of results. By adopting these methodologies, we aim to address the complexities associated with reproducibility and replicability in modern software systems and environments, facilitating a more comprehensive and nuanced perspective on these critical aspects.
The ability to verify research results and to experiment with methodologies are core tenets of science. As research results are increasingly the outcome of computational processes, software plays a central role. GNU Guix is a software deployment tool that supports reproducible software deployment, making it a foundation for computational research workflows. To achieve reproducibility, we must first ensure the source code of software packages Guix deploys remains available.
diffoscopediffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes such as uploading versions 266, 267, 268 and 269 to Debian, making the following changes:
New features:
Use xz --list to supplement output when comparing .xz archives; essential when metadata differs. (#1069329)
Include xz --verbose --verbose (ie. double) output. (#1069329)
Strip the first line from the xz --list output. []
Only include xz --list --verbose output if the xz has no other differences. []
Actually append the xz --listafter the container differences, as it simplifies a lot. []
Testing improvements:
Allow Debian testing to fail right now. []
Drop apktool from Build-Depends; we can still test APK functionality via autopkgtests. (#1071410)
Add a versioned dependency for at least version 5.4.5 for the xz tests as they fail under (at least) version 5.2.8. (#374)
Fix tests for 7zip 24.05. [][]
Fix all tests after additon of xz --list. [][]
Misc:
Update copyright years. []
In addition, James Addison fixed an issue where the HTML output showed only the first difference in a file, while the text output shows all differences [][][], Sergei Trofimovich amended the 7zip version test for older 7z versions that include the string [64] [][] and Vagrant Cascadian relaxed the versioned dependency to allow version 5.4.1 for the xz tests [] and proposed updates to guix for versions 267, 268 and pushed version 269 to Guix. Furthermore, Eli Schwartz updated the diffoscope.org website in order to explain how to install diffoscope on Gentoo [].
Website updates
There were a number of improvements made to our website this month, including Chris Lamb making the print CSS stylesheet nicer []. Fay Stegerman made a number of updates to the page about the SOURCE_DATE_EPOCH environment variable [][][] and Holger Levsen added some of their presentations to the Resources page. Furthermore, IOhannes zm lnig stipulated support for SOURCE_DATE_EPOCH in clang version 16.0.0+ [], Jan Zerebecki expanded the Formal definition page and fixed a number of typos on the Buy-in page [] and Simon Josefsson fixed the link to Trisquel GNU/Linux on the Projects page [].
Upstream patches
This month, we wrote a number of patches to fix specific reproducibility issues, including:
Reproducibility testing framework
The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In May, a number of changes were made by Holger Levsen:
Schedule the i386 architecture a bit more often. []
Adapt cleanup_nodes.sh to the new way of running our build services. []
Add 8 more workers for the i386 architecture. []
Update configuration now that the infom07 and infom08 nodes have been reinstalled as real i386 systems. []
Make diffoscope timeouts more visible on the #debian-reproducible-changes IRC channel. []
Mark the cbxi4a-armhf node as down. [][]
Only install the hdmi2usb-mode-switch package only on Debian bookworm and earlier [] and only install the haskell-platform package on Debian bullseye [].
Misc:
Install the ntpdate utility as we need it later. []
Document the progress on the i386 architecture nodes at Infomaniak. []
Drop an outdated and unnoticed notice. []
Add live_setup_schroot to the list of so-called zombie jobs. []
In addition, Mattia Rizzolo reinstalled the infom07 and infom08 nodes [] and Vagrant Cascadian marked the cbxi4a node as online [].
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:
Welcome to the February 2024 report from the Reproducible Builds project! In our reports, we try to outline what we have been up to over the past month as well as mentioning some of the important things happening in software supply-chain security.
Reproducible Builds at FOSDEM 2024
Core Reproducible Builds developer Holger Levsen presented at the main track at FOSDEM on Saturday 3rd February this year in Brussels, Belgium. However, that wasn t the only talk related to Reproducible Builds.
However, please see our comprehensive FOSDEM 2024 news post for the full details and links.
Three new reproducibility-related academic papers
A total of three separate scholarly papers related to Reproducible Builds have appeared this month:
Signing in Four Public Software Package Registries: Quantity, Quality, and Influencing Factors by Taylor R. Schorlemmer, Kelechi G. Kalu, Luke Chigges, Kyung Myung Ko, Eman Abdul-Muhd, Abu Ishgair, Saurabh Bagchi, Santiago Torres-Arias and James C. Davis (Purdue University, Indiana, USA) is concerned with the problem that:
Package maintainers can guarantee package authorship through software signing [but] it is unclear how common this practice is, and whether the resulting signatures are created properly. Prior work has provided raw data on signing practices, but measured single platforms, did not consider time, and did not provide insight on factors that may influence signing. We lack a comprehensive, multi-platform understanding of signing adoption and relevant factors. This study addresses this gap. (arXiv, full PDF)
[The] principle of reusability [ ] makes it harder to reproduce projects build environments, even though reproducibility of build environments is essential for collaboration, maintenance and component lifetime. In this work, we argue that functional package managers provide the tooling to make build environments reproducible in space and time, and we produce a preliminary evaluation to justify this claim.
This paper thus proposes an approach to automatically identify configuration options causing non-reproducibility of builds. It begins by building a set of builds in order to detect non-reproducible ones through binary comparison. We then develop automated techniques that combine statistical learning with symbolic reasoning to analyze over 20,000 configuration options. Our methods are designed to both detect options causing non-reproducibility, and remedy non-reproducible configurations, two tasks that are challenging and costly to perform manually. (HAL Portal, full PDF)
Distribution work
In Debian this month, 5 reviews of Debian packages were added, 22 were updated and 8 were removed this month adding to Debian s knowledge about identified issues. A number of issue types were updated as well. [ ][ ][ ][ ] In addition, Roland Clobus posted his 23rd update of the status of reproducible ISO images on our mailing list. In particular, Roland helpfully summarised that all major desktops build reproducibly with bullseye, bookworm, trixie and sid provided they are built for a second time within the same DAK run (i.e. [within] 6 hours) and that there will likely be further work at a MiniDebCamp in Hamburg. Furthermore, Roland also responded in-depth to a query about a previous report Fedora developer Zbigniew J drzejewski-Szmek announced a work-in-progress script called fedora-repro-build that attempts to reproduce an existing package within a koji build environment. Although the projects README file lists a number of fields will always or almost always vary and there is a non-zero list of other known issues, this is an excellent first step towards full Fedora reproducibility.
Jelle van der Waa introduced a new linter rule for Arch Linux packages in order to detect cache files leftover by the Sphinx documentation generator which are unreproducible by nature and should not be packaged. At the time of writing, 7 packages in the Arch repository are affected by this.
Elsewhere, Bernhard M. Wiedemann posted another monthly update for his work elsewhere in openSUSE.
diffoscopediffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes such as uploading versions 256, 257 and 258 to Debian and made the following additional changes:
Use a deterministic name instead of trusting gpg s use-embedded-filenames. Many thanks to Daniel Kahn Gillmor dkg@debian.org for reporting this issue and providing feedback. [][]
Don t error-out with a traceback if we encounter struct.unpack-related errors when parsing Python .pyc files. (#1064973). []
Don t try and compare rdb_expected_diff on non-GNU systems as %p formatting can vary, especially with respect to MacOS. []
Expand an older changelog entry with a CVE reference. []
Make test_zip black clean. []
In addition, James Addison contributed a patch to parse the headers from the diff(1) correctly [][] thanks! And lastly, Vagrant Cascadian pushed updates in GNU Guix for diffoscope to version 255, 256, and 258, and updated trydiffoscope to 67.0.6.
reprotestreprotest is our tool for building the same source code twice in different environments and then checking the binaries produced by each build for any differences. This month, Vagrant Cascadian made a number of changes, including:
Create a (working) proof of concept for enabling a specific number of CPUs. [][]
Consistently use 398 days for time variation rather than choosing randomly and update README.rst to match. [][]
Support a new --vary=build_path.path option. [][][][]
Website updates
There were made a number of improvements to our website this month, including:
Chris Lamb:
Improve the relative sizing of headers. []
Re-order and punch up the introduction and documentation on the SOURCE_DATE_EPOCH page. []
Update SOURCE_DATE_EPOCH documentation re. datetime.datetime.fromtimestamp. Thanks, James Addison. []
Reproducibility testing framework
The Reproducible Builds project operates a comprehensive testing framework (available at tests.reproducible-builds.org) in order to check packages and other artifacts for reproducibility. In February, a number of changes were made by Holger Levsen:
Grant Jan-Benedict Glaw shell access to the Jenkins node. []
Enable debugging for NetBSD reproducibility testing. []
Use /usr/bin/du --apparent-size in the Jenkins shell monitor. []
Revert reproducible nodes: mark osuosl2 as down . []
Thanks again to Codethink, for they have doubled the RAM on our arm64 nodes. []
Only set /proc/$pid/oom_score_adj to -1000 if it has not already been done. []
Add the opemwrt-target-tegra and jtx task to the list of zombie jobs. [][]
Vagrant Cascadian also made the following changes:
Overhaul the handling of OpenSSH configuration files after updating from Debian bookworm. [][][]
Add two new armhf architecture build nodes, virt32z and virt64z, and insert them into the Munin monitoring. [][] [][]
In addition, Alexander Couzens updated the OpenWrt configuration in order to replace the tegra target with mpc85xx [], Jan-Benedict Glaw updated the NetBSD build script to use a separate $TMPDIR to mitigate out of space issues on a tmpfs-backed /tmp [] and Zheng Junjie added a link to the GNU Guix tests [].
Lastly, node maintenance was performed by Holger Levsen [][][][][][] and Vagrant Cascadian [][][][].
Upstream patches
The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:
Core Reproducible Builds developer Holger Levsen presented at the main track at FOSDEM on Saturday 3rd February this year in Brussels, Belgium. Titled Reproducible Builds: The First Ten Years
In this talk Holger h01ger Levsen will give an overview about Reproducible Builds: How it started with a small BoF at DebConf13 (and before), then grew from being a Debian effort to something many projects work on together, until in 2021 it was mentioned in an Executive Order of the President of the United States. And of course, the talk will not end there, but rather outline where we are today and where we still need to be going, until Debian stable (and other distros!) will be 100% reproducible, verified by many.
h01ger has been involved in reproducible builds since 2014 and so far has set up automated reproducibility testing for Debian, Fedora, Arch Linux, FreeBSD, NetBSD and coreboot.
More information can be found on FOSDEM s own page for the talk, including a video recording and slides.
Separate from Holger s talk, however, there were a number of other talks about reproducible builds at FOSDEM this year:
For the fifth year in a row, I've asked Soci t de Transport de Montr al,
Montreal's transit agency, for the foot traffic data of Montreal's subway.
By clicking on a subway station, you'll be redirected to a graph of the
station's foot traffic.