Search Results: "Christian Kastner"

15 March 2026

Christian Kastner: Radxa Orion O6 - finally an arm64 board I'm happy with

The Radxa Orion O6 seems to be the arm64 device I've always wanted. Finally! Because it supports UEFI boot, Debian can be installed with the vanilla installation media. Neither custom images, nor additional firmware, nor any other tricks were required on my end. In fact, the entire process was indistinguishable from the amd64 installations I've been accustomed to for two decades: just plug in a USB flash drive, boot, and install. The system's specs are pretty neat. The CPU is ARMv9.2, though without SME, which would have been useful for debugging packages that use it in some way. It's also rich in interfaces, including a Gen4 PCIe x16 slot (with 8 lanes), which enables me to run tests utilizing GPUs. The BIOS version the board came with had a bug where WiFi was permanently disabled, but this has been fixed in the meantime. The only odd thing was the socket for the RTC battery: the CR1220 it requires seems to be rather niche: none of the hardware stores I visited carried it, so I had to order one online. I installed Debian trixie to a NVMe drive, and for 6 months now, it has been running flawlessly. I've been using this board as a development and debugging system for arm64 optimizations, in particular for ggml's feature-specific arm64 backends which are hard to debug on our porterboxes. However, the host will soon transition to a general CI worker for AI/ML related packages, similar to the ROCm CI we've built for testing packages utilizing AMD GPUs. I'll post more about this new CI soon. Purchase of this board was funded by the Debian Project, and was thus enabled through your donations.

16 December 2025

Christian Kastner: Simple-PPA, a minimalistic PPA implementation

Today, the Debusine developers launched Debusine repositories, a beta implementation of PPAs. In the announcement, Colin remarks that "[d]iscussions about this have been happening for long enough that people started referring to PPAs for Debian as 'bikesheds'"; a characterization that I'm sure most will agree with. So it is with great amusement that on this same day, I launch a second PPA implementation for Debian: Simple-PPA. Simple-PPA was never meant to compete with Debusine, though. In fact, it's entirely the opposite: from discussions at DebConf, I knew that it was only a matter of time until Debusine gained a PPA-like feature, but I needed a stop-gap solution earlier, and with some polish, what was once by Python script already doing APT processing for apt.ai.debian.net, recently became Simple-PPA. Consequently, Simple-PPA lacks (and will always lack) all of the features that Debusine offers: there is no auto-building, no CI, nor any other type of QA. It's the simplest possible type of APT repository: you just upload packages, they get imported into an archive, and the archive is exposed via a web server. Under the hood, reprepro does all the heavy lifting. However, this also means it's trivial to set up. The following is the entire configuration that simple-ppa.debian.net started with: 
# simple-ppa.conf
[CORE]
SignWith = 2906D748B7551BC8
ExportDir = /srv/www/simple-ppa
MailFrom: Simple-PPA <admin@simple-ppa.debian.net>
Codenames = sid forky trixie trixie-backports bookworm bookworm-backports
AlsoAllow = forky: unstable
            trixie: unstable
            bookworm: unstable
[simple-ppa-dev]
Label = Simple-PPA's self-hosted development repository
# ckk's key
Uploaders = allow * by key E76004C5CEF0C94C+
[ckk]
Label = Christian Kastner at Simple-PPA
Uploaders = allow * by key E76004C5CEF0C94C+
The CORE section just sets some defaults and sensible rules. Two PPAs are defined, simple-ppa-dev and ckk, which accept packages signed by the key with the ID E76004C5CEF0C94C. These PPAs use the global defaults, but individual PPAs can override Architectures, Suites, and Components, and of course allow an arbitrary number of users. Users upload to this archive using SFTP (e.g.: with dput-ng). Every 15 minutes, uploads get processed, with ACCEPTED or REJECTED mails sent to the Maintainer address. The APT archive of all PPAs is signed with a single global key. I myself intend to use Debusine repositories soon, as the autobuilding and the QA tasks Debusine offers are something I need. However, I do still see a niche use case for Simple-PPA: when you need an APT archive, but don't want to do a deep dive into reprepro (which is extremely powerful). If you'd like to give Simple-PPA a try, head over to simple-ppa.debian.net and follow the instructions for users.

31 July 2025

Russell Coker: Links July 2025

Louis Rossman made an informative YouTube video about right to repair and the US military [1]. This is really important as it helps promote free software and open standards. The ACM has an insightful article about hidden controls [2]. We need EU regulations about hidden controls in safety critical systems like cars. This Daily WTF article has some interesting security implications for Windows [3]. Earth.com has an interesting article about the rubber hand illusion and how it works on Octopus [4]. For a long time I have been opposed to eating Octopus because I think they are too intelligent. The Washington Post has an insightful article about the future of spies when everything is tracked by technology [5]. Micah Lee wrote an informative guide to using Signal groups for activism [6]. David Brin wrote an insightful blog post about the phases of the ongoing US civil war [7]. Christian Kastner wrote an interesting blog post about using Glibc hardware capabilities to use different builds of a shared library for a range of CPU features [8]. David Brin wrote an insightful and interesting blog post comparing President Carter with the criminals in the Republican party [9].
Related posts:
  1. Links May 2025 Christopher Biggs gave an informative Evrything Open lecture about voice...
  2. Links July 2024 Interesting Scientific American article about the way that language shapes...
  3. Links June 2025 Jonathan McDowell wrote part 2 of his blog series about...

12 July 2025

Christian Kastner: Easy dynamic dispatch using GLIBC Hardware Capabilities

TL;DR With GLIBC 2.33+, you can build a shared library multiple times targeting various optimization levels, and the dynamic linker/loader will pick the highest version supported by the current CPU. For example, with the layout below, on a Ryzen 9 5900X, x86-64-v3/libfoo0.so would be loaded: 
/usr/lib/glibc-hwcaps/x86-64-v4/libfoo0.so
/usr/lib/glibc-hwcaps/x86-64-v3/libfoo0.so
/usr/lib/glibc-hwcaps/x86-64-v2/libfoo0.so
/usr/lib/libfoo0.so
Longer Version GLIBC Hardware Capabilities or "hwcaps" are an easy, almost trivial way to add a simple form of dynamic dispatch to any amd64 or POWER build, provided that either the build target or the compiler's optimizations can make use of certain CPU extensions. Mo Zhou pointed me towards this when I was faced with the challenge of creating a performant Debian package for ggml, the tensor library behind llama.cpp and whisper.cpp.
The Challenge A performant yet universally loadable library needs to make use of some form of dynamic dispatch to leverage the most effective SIMD extensions available on any given CPU it may run on. Last January, when I first started with the packaging of ggml for Debian, ggml did have support for this through its GGML_CPU_ALL_VARIANTS=ON option, but this was limited to amd64. This meant that on all the other architectures that Debian supports, I would need to target some ancient baseline, thus effectively crippling the package there.
Dynamic Dispatch using hwcaps hwcaps were introduced in GLIBC 2.33 and replace the (now) Legacy Hardware Capabilities, which were removed in 2.37. The way hwcaps work is delightfully simple: the dynamic linker/loader will look for a shared library not just in the standard library paths, but also in subdirectories thereof of the form hwcaps/<level>, starting with the highest <level> that the current CPU supports. The levels are predefined. I'm using the amd64 levels below. For ggml, this meant that I simply could build the library in multiple passes, each time targeting a different <level>, and install the result in the corresponding subdirectory, which resulted in the following layout (reduced to libggml.so for brevity): 
/usr/lib/x86_64-linux-gnu/ggml/glibc-hwcaps/x86-64-v4/libggml.so
/usr/lib/x86_64-linux-gnu/ggml/glibc-hwcaps/x86-64-v3/libggml.so
/usr/lib/x86_64-linux-gnu/ggml/glibc-hwcaps/x86-64-v2/libggml.so
/usr/lib/x86_64-linux-gnu/ggml/libggml.so
In practice, this means that on a CPU supporting AVX512, the linker/loader would load x86-64-v4/libggml.so if it existed, and otherwise continue to look for the other levels, all the way down to the lowest one. On a CPU which supported only SSE4.2, the lookup process would be the same, ending with picking x86-64-v2/libggml.so. With QEMU, all of this was quickly verified. Note that the lowest-level library, targeting x86-64-v1, is not installed to a subdirectory, but to the path where the library would normally have been installed. This has the nice property that on systems not using GLIBC, and thus not having hwcaps available, package installation will still result in a loadable library, albeit the version with the worst performance. And a careful observer might have noticed that in the example above, the library is installed to a private ggml/ directory, so this mechanism also works when using RUNPATH or LD_LIBRARY_PATH. As mentioned above, Debian's ggml package will soon switch to GGML_CPU_ALL_VARIANTS=ON, but this was still quite the useful feature to discover.

18 March 2025

Christian Kastner: 15th Anniversary of My First Debian Upload

Time flies! 15 years ago, on 2010-03-18, my first upload to the Debian archive was accepted. Debian had replaced Windows as my primary OS in 2005, but it was only when I saw that package zd1211-firmware had been orphaned that I thought of becoming a contributor. I owned a Zyxel G-202 USB WiFi fob that needed said firmware, and as is so often is with open-source software, I was going to scratch my own itch. Bart Martens thankfully helped me adopt the package, and sponsored my upload. I then joined Javier Fern ndez-Sanguino Pe a as a cron maintainer and upstream, and also worked within the Debian Python Applications, Debian Python Modules, and Debian Science Teams, where Jakub Wilk and Yaroslav Halchenko were kind enough to mentor me and eventually support my application to become a Debian Maintainer. Life intervened, and I was mostly inactive in Debian for the next two years. Upon my return in 2014, I had Vincent Cheng to thank for sponsoring most of my newer work, and for eventually supporting my application to become a Debian Developer. It was around that time that I also attended my first DebConf, in Portland, which remains one of my fondest memories. I had never been to an open-source software conference before, and DebConf14 really knocked it out of the park in so many ways. After another break, I returned in 2019 to work mostly on Python and machine learning libraries. In 2020, I finally completed a process that I had first started in 2012 but had never managed to finish before: converting cron from source format 1.0 (one big diff) to source format 3.0 (quilt) (a series of patches). This was a process where I converted 25 years worth of organic growth into a minimal series of logically grouped changes (more here). This was my white whale. In early 2023, shortly after the launch of ChatGPT which triggered an unprecedented AI boom, I started contributing to the Debian ROCm Team, where over the following year, I bootstrapped our CI at ci.rocm.debian.net. Debian's current tooling lack a way to express dependencies on specific hardware other than CPU ISA, nor does it have the means to run autopkgtests using such hardware. To get autopkgtests to make use of AMD GPUs in QEMU VMs and in containers, I had to fork autopkgtest, debci, and a few other components, as well as create a fair share of new tooling for ourselves. This worked out pretty well, and the CI has grown to support 17 different AMD GPU architectures. I will share more on this in upcoming posts. I have mentioned a few contributors by name, but I have countless others to thank for collaborations over the years. It has been a wonderful experience, and I look forward to many years more.

6 February 2022

Christian Kastner: New release of sbuild-qemu Utilities

I just released a new version of the sbuild-qemu utilities as part of sbuild. Notable changes are support for new architectures, and a new sbuild-qemu-boot utility to start a VM and interact with its console directly through the terminal. As a reminder, these utilities leverage functionality in vmdb2, autopkgtest, and sbuild to use QEMU VM images for building and testing packages in strongly isolated environments and/or on foreign architectures and/or that might break the system. For example, the following would create an image for sid on the ppc64el architecture, and then use that image to build a package: 
# Creating an image still requires root, unfortunately
$ sudo sbuild-qemu-create --arch ppc64el -o simple.img unstable http://deb.debian.org/debian
$ sudo chown <user>:<group> simple.img

# Building a package doesn't
$ sbuild-qemu --arch ppc64el --image simple.img -d unstable [<other sbuild options>] FOO.dsc

# Neither does updating the image (runs apt-get update && dist-upgrade in the VM)
$ sbuild-qemu-update --arch ppc64el simple.img
Since the sbuild-qemu utilities build on top of the autopkgtest utilities, the created images can also be used for running autopkgtests: 
$ autopkgtest <test params> -- qemu --dpkg-architecture ppc64el simple.img
sbuild-qemu-boot This new release now ships with sbuild-qemu-boot which boots a VM directly to a console, where it can be used as a porterbox, or to evaluate changes that might break the system, etc. By default, the VM is started in read-only mode, so changes are not saved when the VM is shut down. This can be changed with the --read-write option. With the --shared-dir option, a directory can be shared with the host. It will be mounted at /shared within the VM. 
$ sbuild-qemu-boot --arch ppc64el --shared-dir /path/to/folder simple.img
Note that the console in the VM is 24x80 by default. This can be changed within the VM with stty to match your host, e.g.: 
$ stty rows M cols N
I'm still pondering whether to automate this in sbuild-qemu-create.
New Architectures Thanks to updates to vmdb2 and autopkgtest, the sbuild-qemu utilities now also support new architectures. Currently, amd64, i386, arm64, armhf, and ppc64el are supported, so most of the official architectures. However, there's an autopkgtest issue (#10031002) with the host-guest communication bridge on armhf and on multi-core arm64 (single-core seems to work fine) that currently makes images unusable for sbuild and autopkgtest. I haven't yet figured out what the problem seems to be, but for the time being, the images can still be used with sbuild-qemu-boot.

28 January 2021

Christian Kastner: qemu-sbuild-utils merged into sbuild

qemu-sbuild-utils have been merged into sbuild and are now shipped as package sbuild-qemu. The executables have been renamed from qemu-sbuild-* to sbuild-qemu-*, to be consisent with the other utilities provided by sbuild. I may or may not have botched the transitional dummy package, but as the original package never migrated to testing (this was deliberate) and popcon was low, I'm confident that people will manage. sbuild-qemu depends on the recently uploaded vdbm2, which added support for arm64, armhf, and ppc64el images. This is really exciting, as this means that sbuild-qemu and autopkgtest will soon be able to build for and test on most of the officially supported architectures, all from one host machine. MRs to enable these new features in autopkgtest have already been filed by Ryutaro Matsumoto. Support for the armel architecture is being discussed; support for the MIPS architectures is a more complicated issue, as they don't use GRUB. I'd like to thank Johannes Schauer for reaching out, initiating discussion, and collaborating on this merge!

16 January 2021

Christian Kastner: Keeping your Workstation Silent

I require moderately powerful boxes for my work. Moderately powerful boxes generate a non-negligible amount of heat, and therefore require appropriate cooling. Most cooling solutions involve a certain degree of noise, which I dislike. Water-cooling is supposed to be the most effective solution for noise reduction, but it always felt like too much of a hassle to try to me. Considering the various solutions I've tried in the past years, two observations I've made really stick out:
  1. Noctua's reputation for CPU cooling is well-earned.
  2. A "silent" (meaning sound-proofed) case, isn't. At least, not anymore.
I've tried numerous coolers in the past, some of monstrous proportions (always thinking that more mass must be better, and reputable brands are equally good), but I was never really satisfied; hence, I was doubtful that trying yet another cooler would make a difference. I'm glad I tried the Noctua NH-D15 anyway. With some tweaking to the fan profile in the BIOS, it's totally inaudible at normal to medium workloads, and just a very gentle hum at full load subtle enough to disappear in the background. For the past decade, I've also regularly purchased sound-proofed cases, but this habit appears anachronistic now. Years ago, sound-proofed cases helped contain the noise of a few HDDs. However, all of my boxes now contain NVMe drives (which, to me, are the biggest improvement to computing since CPUs going multi-core). On the other hand, some of my boxes now contain powerful GPUs used for GPGPU computing, and with the recent higher-end Nvidia and AMD cards all pulling in over 300W, there is a lot of heat to manage. The best way to quickly dump heat is with good airflow. Sound-proofing works against that. Its insulation restricts airflow, which ultimately causes even more noise, as the GPU's fans need to spin at very high RPMs. This is, of course, totally obvious in hindsight. I removed the front cover from a new GPU box, and the noise went down from almost-hair-dryer to acceptable-whoosh levels. I'm wondering whether a completely open box might not be even more effective.

4 November 2020

Christian Kastner: qemu-sbuild-utils 0.1: sbuild with QEMU

qemu-sbuild-utils, which were recently accepted into unstable, are a collection of scripts that wrap standard features of sbuild, autopkgtest-build-qemu and vmdb2 to provide a trivial interface for creating and using QEMU-based environments for package building:
  • qemu-sbuild-create creates VM images
  • qemu-sbuild-update is to VM images what sbuild-update is to chroots
  • qemu-sbuild wraps sbuild, adding all necessary (and some useful) options for autopkgtest mode.
Here's a simple two-line example for creating an image, and then using it to build a package: 
$ sudo qemu-sbuild-create -o simple.img unstable http://deb.debian.org/debian

$ qemu-sbuild --image simple.img -d unstable [sbuild-options] FOO.dsc
That's it. Both qemu-sbuild-create and qemu-sbuild automate certain things, but also accept a number of options. For example, qemu-sbuild-create can install additional packages either from one of the APT sources, or .deb packages from the local filesystem. qemu-sbuild will pass on every option it does not consume itself to sbuild, so it should mostly work as a drop-in replacement for it (see the Limitations section below for where it doesn't). The created images can also be used for running autopkgtest itself, of course.
Advantages Excellent isolation. One can go nuts in an environment, change or even break things, and the VM can always simply be reset, or rolled back to an earlier state. Snapshots are just terrific for so many reasons. With KVM acceleration and a fast local APT cache, builds are really fast. There's an overhead of a few seconds for booting the VM on my end, but that overhead is negligible in comparison to the overall build time. On the upside, with everything being memory-backed, even massive dependency installations are lightning fast. With the parameters of the target environment being configurable, it's possible to test builds in various settings (for example: single-core vs. multi-core, or with memory constraints). Technically, it should be possible to emulate, on one host, any other guest architecture (even if emulation might be slow because of missing hardware acceleration). This would present an attractive alternative to (possibly distant and/or slow) porter boxes. However, support for that in qemu-sbuild-utils is not quite there yet.
Limitations The utilities are currently only available on the amd64 architecture, for building packages in amd64 and i386 VMs. There are plans to support arm64 in the near future. qemu-sbuild-create needs root, for the debootstrap stage. I'm looking into ways around this (by extending vmdb2). In any case, image updating and package building do not need privileges. autopkgtest mode does not yet support interactivity, so one cannot drop into a shell with --build-failed-commands, for example. The easy workaround is to connect to the VM with SSH. For this, the image must contain the openssh-server package.
Alternatives I looked at qemubuilder, but had troubles getting it to work. In any case, the autopkgtest chroot mode of sbuild seemed far more powerful and useful to me. vectis looks incredibly promising, but I had already written qemu-sbuild-utils by the time I stumbled over it, and as my current setup works well for me for now and is simple enough to maintain, I decided to polish and publish it. I'm also looking into Docker- and other namespace-based isolation solutions (of which there are many), which I think are the way forward for the majority of packages (those that aren't too close to the kernel and/or hardware). Rather than relying on the kernel for isolation, KVM-based solutions like Amazon's Firecracker and QEMU's microvm machine type provide minimal VMs with almost no boot overhead. Firecracker, for example, claims less than 125ms from launch to /sbin/init. Investigating these is a medium-term project for me.
Why not schroot? I have a strong aversion towards chroot-based build environments. The concept is archaic. Far superior software- and/or hardware-based technologies for process isolation have emerged in the past two decades, and I think it is high time to leave chroot-based solutions behind.
Acknowledgments These utilities are just high-level abstractions. All the heavy lifting is done by sbuild, autopkgtest, and vmdb2.

21 October 2020

Christian Kastner: RStudio is a refreshingly intuitive IDE

I currently need to dabble with R for a smallish thing. I have previously dabbled with R only once, for an afternoon, and that was about a decade ago, so I had no prior experience to speak of regarding the language and its surrounding ecosystem. Somebody recommended that I try out RStudio, a popular IDE for R. I was happy to see that an open-source community edition exists, in the form of a .deb package no less, so I installed it and gave it a try. It's remarkable how intuitive this IDE is. My first guess at doing something has so far been correct every. single. time. I didn't have to open the help, or search the web, for any solutions, either -- they just seem to offer themselves up. And it's not just my inputs; it's the output, too. The RStudio window has multiple tiles, and each tile has multiple tabs. I found this quite confusing and intimidating on first impression, but once I started doing some work, I was surprised to see that whenever I did something that produced output in one or more of the tabs, it was (again) always in an intuitive manner. There's a fine line between informing with relevant context and distracting with irrelevant context, but RStudio seems to have placed itself on the right side of it. This, and many other features that pop up here and there, like the live-rendering of LaTeX equations, contributed to what has to be one of the most positive experiences with an IDE that I've had so far.

6 August 2020

Christian Kastner: My new favorite utility: autojump

Like any developer, I have amassed an impressive collection of directory trees both broad and deep. Navigating these trees became increasingly cumbersome, and setting CDPATH, using auto-completion, and working with the readline history search alleviated this only somewhat. Enter autojump, from the package of the same name. Whatever magic it uses is unbelievably effective. I estimate that in at least 95% of my cases, typing j <name-fragment> changes to the directory I was actually thinking of. Say I'm working on package scikit-learn. My clone of the Salsa repo is in ~/code/pkg-scikit-learn/scikit-learn. Changing to that directory is trivial, I only need to specify a name fragment: 
$ j sci
/home/christian/code/pkg-scikit-learn/scikit-learn
christian@workstation:~/code/pkg-scikit-learn/scikit-learn
But what if I want to work on scikit-learn upstream, to prepare a patch, for example? That repo has been cloned to ~/code/github/scikit-learn. No problem at all, just add another name fragment: 
$ j gi sci
/home/christian/code/github/scikit-learn
christian@workstation:~/code/github/scikit-learn
The magic, however, is most evident with directory trees I rarely enter. As in: I have a good idea of the directory name I wish to change to, but I don't really recall its exact name, nor where (in the tree) it is located. I used to rely on autocomplete to somehow get there which can involve hitting the [TAB] key far too many times, and falling back to find in the worst case, but now, autojump always seems gets me there on first try. I can't believe that this has been available in Debian for 10 years and I only discovered it now.

26 May 2020

Christian Kastner: Curved Monitor

It's been two weeks since I purchased my first curved monitor. Switching away from a flat panel proved to be a novel and unusual experience so much, in fact, that within the first five minutes, I already wanted to return it. Nevertheless, I gave it a try, and I'm glad I did, because not only did I eventually get over the initially perceived issues, I'm now extremely satisfied with it.
Shifted Perspective My sole motivation for the switch was that I had become irritated (to a probably irrational degree) by reading and writing text in whatever window tile was on the left side of my desktop. Even though my previous monitor wasn't a particularly large one with 24", the shift in perspective on the far side of that window always made me feel as if I were reading something to the side of me, rather than in front of me even if I turned to face it directly. It was time to try out a curved monitor.
Process Purchasing something like a monitor is always a pain; there's just so much choice. I would have preferred something with an IPS panel, 4K resolution, and either a 27" or 32" size, and would compromise for a VA panel and WQHD resolution. On geizhals.at, an Austrian price comparison site, ~50 monitors satisfied those criteria. Further limiting the list to reputable brands and reasonable prices still left me with more than two dozen options. Without going into the details why (I was just glad to be done with it), I eventually settled for an MSI Optix MAG271CQR, a 27" WQHD monitor with a VA panel. Once the new monitor arrived, I removed the old monitor from my VESA desk mount, installed the new one, booted, and gave it a try. Within the first five minutes of use, I made three key observations:
  1. My shifted perspective issue on the sides was solved (great!), and
  2. I had gained quite a bit of screen real estate (great!), but
  3. Because of the curvature, the bottom task bar now looked bent (Oh Noes).
Now, point (3) might not sound like that big of an issue, but when you're willing to change your monitor just because vim looks kind of weird to you when it's window is on the left side of the desktop, then a bent-looking task bar is a deal-breaker. I decided that I had to return it. However, that meant: removing it, re-boxing it, shipping it back, etc. Tedious work. As it was already mounted and connected, a friend encouraged me to give it a day or two anyway, just in case. That turned out to be great advice. I would never have expected this, but I got over the bent-looking task bar issue pretty fast. The pleasure of a corrected perspective on either side (everything just looks "right" now) more than makes up for the bent-looking tar bar at the bottom; I don't even notice it anymore. And the added screen real estate is a bonus I hadn't planned for. The MAG271CQR targets the gaming demographic, and thus comes loaded with various features. My new favorite is "Reader Mode", which has an effect quite similar to "Night Mode" on mobile devices (reduced brightness, blue light filter). My eyes barely tire anymore, even after a long day's use. It also has a Picture-in-Picture mode for a second input which I haven't tried yet, but should come in handy for SBCs and the like.

12 April 2020

Christian Kastner: hello, world

It slipped my mind that the recent March 17, 2020, marked the 10th anniversary of my first upload to the official Debian archive. How time flies! Although I was never much of a blogging person myself, I do enjoy reading other people's contributions, and on some occasions felt that the Planet Debian feed might have been a more appropriate medium for sharing a particular tidbit, instead of sharing it with a specific mailing list, or not sharing at all. So, to commemorate my anniversary, I venture into this new space. first upload: https://lists.debian.org/debian-devel-changes/2010/03/msg01807.html

10 May 2016

Reproducible builds folks: Reproducible builds: week 54 in Stretch cycle

What happened in the Reproducible Builds effort between May 1st and May 7th 2016: Media coverage There has been a surprising tweet last week: "Props to @FiloSottile for his nifty gvt golang tool. We're using it to get reproducible builds for a Zika & West Nile monitoring project." and to our surprise Kenn confirmed privately that he indeed meant "reproducible builds" as in "bit by bit identical builds". Wow. We're looking forward to learn more details about this; for now we just know that they are doing this for software quality reasons basically. Two of the four GSoC and Outreachy participants for Reproducible builds posted their introductions to Planet Debian: Toolchain fixes and other upstream developments dpkg 1.18.5 was uploaded fixing two bugs relevant to us: This upload made it necessary to rebase our dpkg on the version on sid again, which Niko Tyni and Lunar promptly did. Then a few days later 1.18.6 was released to fix a regression in the previous upload, and Niko promptly updated our patched version again. Following this Niko Tyni found #823428: "dpkg: many packages affected by dpkg-source: error: source package uses only weak checksums". Alexis Bienven e worked on tex related packages and SOURCE_DATE_EPOCH: Emmanuel Bourg uploaded jflex/1.4.3+dfsg-2, which removes timestamps from generated files. Packages fixed The following 285 packages have become reproducible due to changes in their build dependencies (mostly from GCC honouring SOURCE_DATE_EPOCH, see the previous week report): 0ad abiword abcm2ps acedb acpica-unix actiona alliance amarok amideco amsynth anjuta aolserver4-nsmysql aolserver4-nsopenssl aolserver4-nssqlite3 apbs aqsis aria2 ascd ascii2binary atheme-services audacity autodocksuite avis awardeco bacula ballerburg bb berusky berusky2 bindechexascii binkd boinc boost1.58 boost1.60 bwctl cairo-dock cd-hit cenon.app chipw ckermit clp clustalo cmatrix coinor-cbc commons-pool cppformat crashmail crrcsim csvimp cyphesis-cpp dact dar darcs darkradiant dcap dia distcc dolphin-emu drumkv1 dtach dune-localfunctions dvbsnoop dvbstreamer eclib ed2k-hash edfbrowser efax-gtk efax exonerate f-irc fakepop fbb filezilla fityk flasm flightgear fluxbox fmit fossil freedink-dfarc freehdl freemedforms-project freeplayer freeradius fxload gdb-arm-none-eabi geany-plugins geany geda-gaf gfm gif2png giflib gifticlib glaurung glusterfs gnokii gnubiff gnugk goaccess gocr goldencheetah gom gopchop gosmore gpsim gputils grcompiler grisbi gtkpod gvpe hardlink haskell-github hashrat hatari herculesstudio hpcc hypre i2util incron infiniband-diags infon ips iptotal ipv6calc iqtree jabber-muc jama jamnntpd janino jcharts joy2key jpilot jumpnbump jvim kanatest kbuild kchmviewer konclude krename kscope kvpnc latexdiff lcrack leocad libace-perl libcaca libcgicc libdap libdbi-drivers libewf libjlayer-java libkcompactdisc liblscp libmp3spi-java libpwiz librecad libspin-java libuninum libzypp lightdm-gtk-greeter lighttpd linpac lookup lz4 lzop maitreya meshlab mgetty mhwaveedit minbif minc-tools moc mrtrix mscompress msort mudlet multiwatch mysecureshell nifticlib nkf noblenote nqc numactl numad octave-optim omega-rpg open-cobol openmama openmprtl openrpt opensm openvpn openvswitch owx pads parsinsert pcb pd-hcs pd-hexloader pd-hid pd-libdir pear-channels pgn-extract phnxdeco php-amqp php-apcu-bc php-apcu php-solr pidgin-librvp plan plymouth pnscan pocketsphinx polygraph portaudio19 postbooks-updater postbooks powertop previsat progressivemauve puredata-import pycurl qjackctl qmidinet qsampler qsopt-ex qsynth qtractor quassel quelcom quickplot qxgedit ratpoison rlpr robojournal samplv1 sanlock saods9 schism scorched3d scummvm-tools sdlbasic sgrep simh sinfo sip-tester sludge sniffit sox spd speex stimfit swarm-cluster synfig synthv1 syslog-ng tart tessa theseus thunar-vcs-plugin ticcutils tickr tilp2 timbl timblserver tkgate transtermhp tstools tvoe ucarp ultracopier undbx uni2ascii uniutils universalindentgui util-vserver uudeview vfu virtualjaguar vmpk voms voxbo vpcs wipe x264 xcfa xfrisk xmorph xmount xyscan yacas yasm z88dk zeal zsync zynaddsubfx Last week the 1000th bug usertagged "reproducible" was fixed! This means roughly 2 bugs per day since 2015-01-01. Kudos and huge thanks to everyone involved! Please also note: FTBFS packages have not been counted here and there are still 600 open bugs with reproducible patches provided. Please help bringing that number down to 0! The following packages have become reproducible after being fixed: Some uploads have fixed some reproducibility issues, but not all of them: Uploads which fix reproducibility issues, but currently FTBFS: Patches submitted that have not made their way to the archive yet: Package reviews 54 reviews have been added, 6 have been updated and 44 have been removed in this week. 18 FTBFS bugs have been reported by Chris Lamb, James Cowgill and Niko Tyni. diffoscope development Thanks to Mattia, diffoscope 52~bpo8+1 is available in jessie-backports now. tests.reproducible-builds.org Misc. This week's edition was written by Reiner Herrmann, Holger Levsen and Mattia Rizzolo and reviewed by a bunch of Reproducible builds folks on IRC. Mattia also wrote a small ikiwiki macro for this blog to ease linking reproducible issues, packages in the package tracker and bugs in the Debian BTS.

30 November 2015

Lunar: Reproducible builds: week 31 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Reiner Herrmann submitted a patch against debhelper to make dh_installinit source files in a stable order. Chris Lamb found how to make cython output deterministic by ordering the keys used to traverse a dict. Reiner Herrmann proposed a patch for pyside-tools to remove the timestamps embedded by rcc in the generated Python code. Mattia Rizzolo rebased our custom version of debhelper on version 9.20151126. As no objections have been made so far, Mattia Rizzolo has filled #805872 asking -Wdate-time to be turned on by default in dpkg-buildflag. Guillem has since sent a final warning before proceeding as such in the next dpkg upload. Russ Allbery added support for SOURCE_DATE_EPOCH in podlators 4.00 which Niko Tyni intend to backport to Perl 5.22. Packages fixed The following packages have become reproducible due to changes in their build dependencies: fontforge, golang-github-tinylib-msgp, libpango-perl, libparanamer-java, libxaw, sqljet, stringtemplate4, uzbl, zope-mysqlda. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet: Lunar reported two issues making xz-utils unreproducible (#806328, #806331). reproducible.debian.net A seventh armhf build node has been added (resulting of two more armhf build jobs). Thanks to Vagrant Cascadian for putting this Raspberry Pi 2B to help. (h01ger) jenkins.debian.net has been made more robust against network and proxy failures. (h01ger) A new 100 GB partition has been set up on reproducible.debian.net to prevent disk space issues. Thanks to ProfitBricks for its continuous support to our continuous test system. (h01ger) New graphs showing usertagged bugs have been added on the dashboard to measure the progress without FTBFS issues. Please note that comparing the two graphs might be misleading as more than 1300 FTBFS bugs have been inventoried. (h01ger) Package reviews 78 reviews have been removed, 116 added and 49 updated this week. 25 new FTBFS have been filed by Chris West, Chris Lamb and Santiago Vila. New issues identified this week: timestamps_in_documentation_generated_with_libwibble, copyright_year_in_documentation_generated_by_sphinx, timestamps_in_documentation_generated_by_glib_genpod, random_order_of_tmpfiles_in_postinst, random_order_in_cython_output, timestamps_in_python_code_generated_by_pyside. Reiner Herrmann and Lunar improved the prebuilder script: the script can now be called through a symlink, run parallel builds, calls diffoscope by its new name and ensure to install its recommends, and save the text output aside the HTML one. Reiner also added a script to lookup the last update of notes for a given package. Misc. Santiago Villa has been recently working on making sure that Arch:all packages were properly buildable by running dpkg-buildpackage -A. This uncovered a question that is probably not currently addressed by the policy: on which architectures should architecture-independent be buildable?

1 September 2015

Bits from Debian: New Debian Developers and Maintainers (July and August 2015)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

12 July 2015

Lunar: Reproducible builds: week 11 in Stretch cycle

Debian is undertaking a huge effort to develop a reproducible builds system. I'd like to thank you for that. This could be Debian's most important project, with how badly computer security has been going.

PerniciousPunk in Reddit's Ask me anything! to Neil McGovern, DPL. What happened in the reproducible builds effort this week: Toolchain fixes More tools are getting patched to use the value of the SOURCE_DATE_EPOCH environment variable as the current time:

In the reproducible experimental toolchain which have been uploaded: Johannes Schauer followed up on making sbuild build path deterministic with several ideas. Packages fixed The following 311 packages became reproducible due to changes in their build dependencies : 4ti2, alot, angband, appstream-glib, argvalidate, armada-backlight, ascii, ask, astroquery, atheist, aubio, autorevision, awesome-extra, bibtool, boot-info-script, bpython, brian, btrfs-tools, bugs-everywhere, capnproto, cbm, ccfits, cddlib, cflow, cfourcc, cgit, chaussette, checkbox-ng, cinnamon-settings-daemon, clfswm, clipper, compton, cppcheck, crmsh, cupt, cutechess, d-itg, dahdi-tools, dapl, darnwdl, dbusada, debian-security-support, debomatic, dime, dipy, dnsruby, doctrine, drmips, dsc-statistics, dune-common, dune-istl, dune-localfunctions, easytag, ent, epr-api, esajpip, eyed3, fastjet, fatresize, fflas-ffpack, flann, flex, flint, fltk1.3, fonts-dustin, fonts-play, fonts-uralic, freecontact, freedoom, gap-guava, gap-scscp, genometools, geogebra, git-reintegrate, git-remote-bzr, git-remote-hg, gitmagic, givaro, gnash, gocr, gorm.app, gprbuild, grapefruit, greed, gtkspellmm, gummiboot, gyp, heat-cfntools, herold, htp, httpfs2, i3status, imagetooth, imapcopy, imaprowl, irker, jansson, jmapviewer, jsdoc-toolkit, jwm, katarakt, khronos-opencl-man, khronos-opengl-man4, lastpass-cli, lava-coordinator, lava-tool, lavapdu, letterize, lhapdf, libam7xxx, libburn, libccrtp, libclaw, libcommoncpp2, libdaemon, libdbusmenu-qt, libdc0, libevhtp, libexosip2, libfreenect, libgwenhywfar, libhmsbeagle, libitpp, libldm, libmodbus, libmtp, libmwaw, libnfo, libpam-abl, libphysfs, libplayer, libqb, libsecret, libserial, libsidplayfp, libtime-y2038-perl, libxr, lift, linbox, linthesia, livestreamer, lizardfs, lmdb, log4c, logbook, lrslib, lvtk, m-tx, mailman-api, matroxset, miniupnpd, mknbi, monkeysign, mpi4py, mpmath, mpqc, mpris-remote, musicbrainzngs, network-manager, nifticlib, obfsproxy, ogre-1.9, opal, openchange, opensc, packaging-tutorial, padevchooser, pajeng, paprefs, pavumeter, pcl, pdmenu, pepper, perroquet, pgrouting, pixz, pngcheck, po4a, powerline, probabel, profitbricks-client, prosody, pstreams, pyacidobasic, pyepr, pymilter, pytest, python-amqp, python-apt, python-carrot, python-django, python-ethtool, python-mock, python-odf, python-pathtools, python-pskc, python-psutil, python-pypump, python-repoze.tm2, python-repoze.what, qdjango, qpid-proton, qsapecng, radare2, reclass, repsnapper, resource-agents, rgain, rttool, ruby-aggregate, ruby-albino, ruby-archive-tar-minitar, ruby-bcat, ruby-blankslate, ruby-coffee-script, ruby-colored, ruby-dbd-mysql, ruby-dbd-odbc, ruby-dbd-pg, ruby-dbd-sqlite3, ruby-dbi, ruby-dirty-memoize, ruby-encryptor, ruby-erubis, ruby-fast-xs, ruby-fusefs, ruby-gd, ruby-git, ruby-globalhotkeys, ruby-god, ruby-hike, ruby-hmac, ruby-integration, ruby-jnunemaker-matchy, ruby-memoize, ruby-merb-core, ruby-merb-haml, ruby-merb-helpers, ruby-metaid, ruby-mina, ruby-net-irc, ruby-net-netrc, ruby-odbc, ruby-ole, ruby-packet, ruby-parseconfig, ruby-platform, ruby-plist, ruby-popen4, ruby-rchardet, ruby-romkan, ruby-ronn, ruby-rubyforge, ruby-rubytorrent, ruby-samuel, ruby-shoulda-matchers, ruby-sourcify, ruby-test-spec, ruby-validatable, ruby-wirble, ruby-xml-simple, ruby-zoom, rumor, rurple-ng, ryu, sam2p, scikit-learn, serd, shellex, shorewall-doc, shunit2, simbody, simplejson, smcroute, soqt, sord, spacezero, spamassassin-heatu, spamprobe, sphinxcontrib-youtube, splitpatch, sratom, stompserver, syncevolution, tgt, ticgit, tinyproxy, tor, tox, transmissionrpc, tweeper, udpcast, units-filter, viennacl, visp, vite, vmfs-tools, waffle, waitress, wavtool-pl, webkit2pdf, wfmath, wit, wreport, x11proto-input, xbae, xdg-utils, xdotool, xsystem35, yapsy, yaz. Please note that some packages in the above list are falsely reproducible. In the experimental toolchain, debhelper exported TZ=UTC and this made packages capturing the current date (without the time) reproducible in the current test environment. The following packages became reproducible after getting fixed: Ben Hutchings upstreamed several patches to fix Linux reproducibility issues which were quickly merged. Some uploads fixed some reproducibility issues but not all of them: Uploads that should fix packages not in main: Patches submitted which have not made their way to the archive yet: reproducible.debian.net A new package set has been added for lua maintainers. (h01ger) tracker.debian.org now only shows reproducibility issues for unstable. Holger and Mattia worked on several bugfixes and enhancements: finished initial test setup for NetBSD, rewriting more shell scripts in Python, saving UDD requests, and more debbindiff development Reiner Herrmann fixed text comparison of files with different encoding. Documentation update Juan Picca added to the commands needed for a local test chroot installation of the locales-all package. Package reviews 286 obsolete reviews have been removed, 278 added and 243 updated this week. 43 new bugs for packages failing to build from sources have been filled by Chris West (Faux), Mattia Rizzolo, and h01ger. The following new issues have been added: timestamps_in_manpages_generated_by_ronn, timestamps_in_documentation_generated_by_org_mode, and timestamps_in_pdf_generated_by_matplotlib. Misc. Reiner Herrmann has submitted patches for OpenWrt. Chris Lamb cleaned up some code and removed cruft in the misc.git repository. Mattia Rizzolo updated the prebuilder script to match what is currently done on reproducible.debian.net.