Debian Contributions: 2025-04 Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

DebConf 25 Preparations, by Stefano Rivera and Santiago Ruano Rinc n DebConf 25 preparations continue. In April, the bursary team reviewed and ranked bursary applications. Santiago Ruano Rinc n examined the current state of the conference s finances, to see if we could allocate any more money to bursaries. Stefano Rivera supported the bursary team s work with infrastructure and advice and added some metrics to assist Santiago s budget review. Santiago was also involved in different parts of the organization, including Content team matters, as reviewing the first of proposals, preparing public information about the new Academic Track; or coordinating different aspects of the Day trip activities and the Conference Dinner.

PyPA tools updates, by Stefano Rivera Around the beginning of the freeze (in retrospect, definitely too late) Stefano looked at updating setuptools in the archive to 78.1.0. This brings support for more comprehensive license expressions (PEP-639), that people are expected to adopt soon upstream. While the reverse-autopkgtests all passed, it all came with some unexpected complications, and turned into a mini-transition. The new setuptools broke shebangs for scripts (pypa/setuptools#4952). It also required a bump of wheel to 0.46 and wheel 0.46 now has a dependency outside the standard library (it de-vendored packaging). This meant it was no longer suitable to distribute a standalone wheel.whl file to seed into new virtualenvs, as virtualenv does by default. The good news here is that setuptools doesn t need wheel any more, it included its own implementation of the bdist_wheel command, in 70.1. But the world hadn t adapted to take advantage of this, yet. Stefano scrambled to get all of these issues resolved upstream and in Debian:

• pip: Don t check for wheel when invoked with --no-use-pep517 (pypa/pip#13330), automatically do --no-use-pep517 builds without wheel (pypa/pip#13358, rejected).
• virtualenv: Don t include wheel (pypa/virtualenv#2868) except on Python 3.8 (pypa/virtualenv#2876) as pip dropped Python 3.8 support in the same release that included #13330.
• python3.13: Update bundled setuptools in test.wheeldata (python/cpython#132415).
• python-cffi: No need to install wheel any more (python-cffi/cffi#165).

We re now at the point where python3-wheel-whl is no longer needed in Debian unstable, and it should migrate to trixie.

Removing libcrypt-dev from build-essential, by Helmut Grohne The crypt function was originally part of glibc, but it got separated to libxcrypt. As a result, libc6-dev now depends on libcrypt-dev. This poses a cycle during architecture cross bootstrap. As the number of packages actually using crypt is relatively small, Helmut proposed removing the dependency. He analyzed an archive rebuild kindly performed by Santiago Vila (not affiliated with Freexian) and estimated the necessary changes. It looks like we may complete this with modifications to less than 300 source packages in the forky cycle. Half of the bugs have been filed at this time. They are tracked with libcrypt-* usertags.

Miscellaneous contributions

• Carles uploaded a new version of simplemonitor.
• Carles improved the documentation of salsa-ci-team/pipeline regarding piuparts arguments.
• Carles closed an FTBFS on gcc-15 on qnetload.
• Carles worked on Catalan translations using po-debconf-manager: reviewed 57 translations and created their merge requests in salsa, created 59 bug reports for packages that didn t merge in more than 30 days. Followed-up merge requests and comments in bug reports. Managed some translations manually for packages that are not in Salsa.
• Lucas did some work on the DebConf Content and Bursary teams.
• Lucas fixed multiple CVEs and bugs involving the upgrade from bookworm to trixie in ruby3.3.
• Lucas fixed a CVE in valkey in unstable.
• Stefano updated beautifulsoup4, python-authlib, python-html2text, python-packaging, python-pip, python-soupsieve, and unidecode.
• Stefano packaged python-dependency-groups, a new vendored library in python-pip.
• During an afternoon Bug Squashing Party in Montevideo, Santiago uploaded a couple of packages fixing RC bugs #1057226 and #1102487. The latter was a sponsored upload.
• Thorsten uploaded new upstream versions of brlaser, ptouch-driver and sane-airscan to get the latest upstream bug fixes into Trixie.
• Rapha l filed an upstream bug on zim for a graphical glitch that he has been experiencing.
• Colin Watson upgraded openssh to 10.0p1 (also known as 10.0p2), and debugged various follow-up bugs. This included adding riscv64 support to vmdb2 in passing, and enabling native wtmpdb support so that wtmpdb last now reports the correct tty for SSH connections.
• Colin fixed dput-ng s override option, which had never previously worked.
• Colin fixed a security bug in debmirror.
• Colin did his usual routine work on the Python team: 21 packages upgraded to new upstream versions, 8 CVEs fixed, and about 25 release-critical bugs fixed.
• Helmut filed patches for 21 cross build failures.
• Helmut uploaded a new version of debvm featuring a new tool debefivm-create to generate EFI-bootable disk images compatible with other tools such as libvirt or VirtualBox. Much of the work was prototyped in earlier months. This generalizes mmdebstrap-autopkgtest-build-qemu.
• Helmut continued reporting undeclared file conflicts and suggested package removals from unstable.
• Helmut proposed build profiles for libftdi1 and gnupg2. To deal with recently added dependencies in the architecture cross bootstrap package set.
• Helmut managed the /usr-move transition. He worked on ensuring that systemd would comply with Debian s policy. Dumat continues to locate problems here and there yielding discussion occasionally. He sent a patch for an upgrade problem in zutils.
• Anupa worked with the Debian publicity team to publish Micronews and Bits posts.
• Anupa worked with the DebConf 25 content team to review talk and event proposals for DebConf 25.

Last week, we (Helmut, Jochen, Holger, Gioele and josch) met in W rzburg for a Debian crossbuilding & bootstrap sprint. We would like to thank Angest pselt e. V. for generously providing us with their hacker space which we were able to use exclusively during the four-day-sprint. We d further like to thank Debian for their sponsorship of accommodation of Helmut and Jochen. The most important topics that we worked on together were:

• publicity and funding for bootstrappable and cross-buildable Debian, driven by Gioele, including the creation of a list of usecases and slogans [everyone]
• proof-of-concept for substituting coreutils with alternative implementations such as busybox, toybox or uutils [Helmut, Jochen, josch]
• writing a patch for documenting the Multi-Arch field in Debian policy #749826 [Helmut, Holger, Jochen, josch]
• turning build profile spec text into a patch for Debian policy #757760 [Helmut, Jochen, josch]

Our TODO items for after the sprint are:

• josch needs to fix bootstrap.debian.net
• josch exports the package lists computed by bootstrap.debian.net in a machine readable format for Holger
• writing a mail to d-devel about making coreutils non-essential

In addition to what was already listed above, people worked on the following tasks specifically:

• Holger now wants a crossbootstrap pkg set for reproducible builds.
• Holger worked on some reproducible builds issues, uploaded ~10 sequoia related packages and did a devscripts upload.
• Jochen worked on creating initrds
• Jochen helped Holger with sequoia/rust packaging
• Jochen worked on sbuild
• Jochen discussed cross bootstrapping with Helmut and josch
• Jochen fixed bugs in devscripts (debrebuild/debootstrap, build-rdeps, proxy.py)
• Jochen worked on reproduce.d.n
• Jochen worked on src:kokkos resulting in #1101487
• Gioele gathered information and material for possible funding for bootstrapping-related projects.
• Gioele ported src:libreplaygain from cdbs to dh.
• Helmut dug into lingering debvm issues some. Jochen tracked down the ARM32 autopkgtest regression to #1079443 which is now worked around.
• Helmut collected feedback on linux-libc-dev being a:all.
• Helmut collected feedback on dropping libcrypt-dev from build-essential and initiated work with Santiago Vila
• Helmut collected feedback on how sbuild would want to interface with a better build containment
• josch reviewed and merged the following MRs:
  • devscripts#493 debootsnap
  • sbuild#148
  • sbuild#157
• josch worked on making the Debian Linux kernel packaging use hooks installed in /usr/share/kernel/*.d and gathered feedback from the other sprint participants in how to best move this forward, culminating in the opening of #1101733 against src:linux.

Thank you all for attending this sprint, for making it so productive and for the amazing atmosphere and enlightening discussions!

Debian Contributions: 2025-02 Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

Debian.Social administration, by Stefano Rivera Over the last year, the Debian.social services outgrew the infrastructure that was supporting them. The matrix bridge in particular was hosted on a cloud instance backed by a large expensive storage volume. Debian.CH rented a new large physical server to host all these instances, earlier this year. Stefano set up Incus on the new physical machine and migrated all the old debian.social LXC Containers, libvirt VMs, and cloud instances into Incus-managed LXC containers. Stefano set up Prometheus monitoring and alerts for the new infrastructure and a Grafana dashboard. The current stack of debian.social services seem to comfortably fit on the new machine, with good room to grow.

DebConf 25, by Santiago Ruano Rinc n and Stefano Rivera DebConf 25 preparations continue. The team is currently finalizing a budget. Stefano helped to review the current budget proposals and suggest approaches for balancing it. Stefano installed a Zammad instance to organize queries from attendees, for the registration and visa teams. Santiago continued discussions with possible caterers so we can have options for the different diet requirements and that could fit into the DebConf budget. Also, in collaboration with Anupa, Santiago pushed the first draft changes to document the venue information in the DebConf 25 website and how to get to Brest.

Time-based test failure in requests, by Colin Watson Colin fixed a fun bug in the Python requests package. Santiago Vila has been running tests of what happens when Debian packages are built on a system in which time has been artificially set to somewhere around the end of the support period for the next Debian release, in order to make it easier to do things like issuing security updates for the lifetime of that release. In this case, the failure indicated an expired test certificate, and since the repository already helpfully included scripts to regenerate those certificates, it seemed natural to try regenerating them just before running tests. However, this failed for more obscure reasons and Colin spent some time investigating. This turned out to be because the test CA was missing the CA constraint and so recent versions of OpenSSL reject it; Colin sent a pull request to fix this.

Priority list for outdated packages, by Santiago Ruano Rinc n Santiago started a discussion on debian-devel about packages that have a history of security issues and that are outdated regarding new upstream releases. The goal of the mentioned effort is to have a prioritized list of packages needing some work, from a security point of view. Moreover, the aim of publicly sharing the list of packages with the Debian Developers community is to make it easier to look at the packages maintained by teams, or even other maintainers where help could be welcome. Santiago is planning to take into account the feedback provided in debian-devel and to propose a tooling that could help to regularly bring collective awareness of these packages.

Miscellaneous contributions

• Carles worked on English to Catalan po-debconf translations: reviewed translations, created merge requests and followed up with developers for more than 30 packages using po-debconf-manager.
• Carles helped users, fixed bugs and implemented downloading updated templates on po-debconf-manager.
• Carles packaged a new upstream version of python-pyaarlo.
• Carles improved reproducibility of qnetload (now reported as reproducible) and simplemonitor (followed up with upstream and pending update of Debian package).
• Carles collaborated with debian-history package: fixed FTBFS from master branch, enabled salsa-ci and investigated reproducibility.
• Emilio improved support for automatically marking CVEs as NOT-FOR-US in the security-tracker, closing #1073012.
• Emilio updated xorg-server and xwayland in unstable, fixing the last round of security vulnerabilities.
• Stefano prepared a few PyPy and cPython uploads, and started the python3.13-only transition.
• Helmut Grohne sent patches for 24 cross build failures.
• Helmut fixed two problems in the Debian /usr-merge analysis tool. In one instance, it would overmatch Debian bugs to issues and in another it would fail to recognize Pre-Depends as a conflict mechanism.
• Helmut attempted making rebootstrap work for gcc-15 with limited success as very many packages FTBFS with gcc-15 due to using function declarations without arguments.
• Helmut provided a change to the security-tracker that would pre-compute /data/json during database updates rather than on demand resulting in a reduced response time.
• Colin uploaded OpenSSH security updates for testing/unstable, bookworm, bullseye, buster, and stretch.
• Colin fixed upstream monitoring for 26 Python packages, and upgraded 54 packages (mostly Python-related, but also PuTTY) to new upstream versions.
• Colin updated python-django in bookworm-backports to 4.2.18 (issuing BSA-121), and added new backports of python-django-dynamic-fixture and python-django-pgtrigger, all of which are dependencies of debusine.
• Thorsten Alteholz finally managed to upload hplip to fix two release critical and some normal bugs. The next step in March would be to upload the latest version of hplip.
• Faidon updated crun in unstable & trixie, resolving a long-standing request of enabling criu support and thus enabling podman with checkpoint/restore functionality (With gratitude to Salvatore Bonaccorso and Reinhard Tartler for the cooperation and collaboration).
• Faidon uploaded a number of packages (librdkafka, libmaxminddb, python-maxminddb, lowdown, tox, tox-uv, pyproject-api, xiccd and gdnsd) bringing them up to date with new upstream releases, resolving various bugs.
• Lucas Kanashiro uploaded some ruby packages involved in the Rails 7 transition with new upstream releases.
• Lucas triaged a ruby3.1 bug (#1092595)) and prepared a fix for the next stable release update.
• Lucas set up the needed wiki pages and updated the Debian Project status in the Outreachy portal, in order to send out a call for projects and mentors for the next round of Outreachy.
• Anupa joined Santiago to prepare a list of companies to contact via LinkedIn for DebConf 25 sponsorship.
• Anupa printed Debian stickers and sponsorship brochures, flyers for DebConf 25 to be distributed at FOSS ASIA summit 2025.
• Anupa participated in the Debian publicity team meeting and discussed the upcoming events and tasks.
• Rapha l packaged zim 0.76.1 and integrated an upstream patch for another regression that he reported.
• Rapha l worked with the Debian System Administrators for tracker.debian.org to better cope with gmail s requirement for mails to be authenticated.

Welcome to the December 2024 report from the Reproducible Builds project! Our monthly reports outline what we ve been up to over the past month and highlight items of news from elsewhere in the world of software supply-chain security when relevant. As ever, however, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. Table of contents:

1. reproduce.debian.net
2. debian-repro-status
3. On our mailing list
4. Enhancing the Security of Software Supply Chains
5. diffoscope
6. Supply-chain attack in the Solana ecosystem
7. Website updates
8. Debian changes
9. Other development news
10. Upstream patches
11. Reproducibility testing framework

---

reproduce.debian.net Last month saw the introduction of reproduce.debian.net. Announced at the recent Debian MiniDebConf in Toulouse, reproduce.debian.net is an instance of rebuilderd operated by the Reproducible Builds project. rebuilderd is our server designed monitor the official package repositories of Linux distributions and attempts to reproduce the observed results there. This month, however, we are pleased to announce that not only does the service now produce graphs, the reproduce.debian.net homepage itself has become a start page of sorts, and the amd64.reproduce.debian.net and i386.reproduce.debian.net pages have emerged. The first of these rebuilds the amd64 architecture, naturally, but it also is building Debian packages that are marked with the no architecture label, all. The second builder is, however, only rebuilding the i386 architecture. Both of these services were also switched to reproduce the Debian trixie distribution instead of unstable, which started with 43% of the archive rebuild with 79.3% reproduced successfully. This is very much a work in progress, and we ll start reproducing Debian unstable soon. Our i386 hosts are very kindly sponsored by Infomaniak whilst the amd64 node is sponsored by OSUOSL thank you! Indeed, we are looking for more workers for more Debian architectures; please contact us if you are able to help.

debian-repro-status Reproducible builds developer kpcyrd has published a client program for reproduce.debian.net (see above) that queries the status of the locally installed packages and rates the system with a percentage score. This tool works analogously to arch-repro-status for the Arch Linux Reproducible Builds setup. The tool was packaged for Debian and is currently available in Debian trixie: it can be installed with apt install debian-repro-status.

On our mailing list On our mailing list this month:

• Bernhard M. Wiedemann wrote a detailed post on his long journey towards a bit-reproducible Emacs package. In his interesting message, Bernhard goes into depth about the tools that they used and the lower-level technical details of, for instance, compatibility with the version for glibc within openSUSE.
• Shivanand Kunijadar posed a question pertaining to the reproducibility issues with encrypted images. Shivanand explains that they must use a random IV for encryption with AES CBC. The resulting artifact is not reproducible due to the random IV used. The message resulted in a handful of replies, hopefully helpful!
• User Danilo posted an in interesting question related to their attempts in trying to achieve reproducible builds for Threema Desktop 2.0. The question resulted in a number of replies attempting to find the right combination of compiler and linker flags (for example).
• Longstanding contributor David A. Wheeler wrote to our list announcing the release of the Census III of Free and Open Source Software: Application Libraries report written by Frank Nagle, Kate Powell, Richie Zitomer and David himself. As David writes in his message, the report attempts to answer the question what is the most popular Free and Open Source Software (FOSS)? .
• Lastly, kpcyrd followed-up to a post from September 2024 which mentioned their desire for someone to implement a hashset of allowed module hashes that is generated during the kernel build and then embedded in the kernel image , thus enabling a deterministic and reproducible build. However, they are now reporting that somebody implemented the hash-based allow list feature and submitted it to the Linux kernel mailing list . Like kpcyrd, we hope it gets merged.

Enhancing the Security of Software Supply Chains: Methods and Practices Mehdi Keshani of the Delft University of Technology in the Netherlands has published their thesis on Enhancing the Security of Software Supply Chains: Methods and Practices . Their introductory summary first begins with an outline of software supply chains and the importance of the Maven ecosystem before outlining the issues that it faces that threaten its security and effectiveness . To address these:

First, we propose an automated approach for library reproducibility to enhance library security during the deployment phase. We then develop a scalable call graph generation technique to support various use cases, such as method-level vulnerability analysis and change impact analysis, which help mitigate security challenges within the ecosystem. Utilizing the generated call graphs, we explore the impact of libraries on their users. Finally, through empirical research and mining techniques, we investigate the current state of the Maven ecosystem, identify harmful practices, and propose recommendations to address them.

A PDF of Mehdi s entire thesis is available to download.

diffoscope diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made the following changes, including preparing and uploading versions 283 and 284 to Debian:

• Update copyright years. [ ]
• Update tests to support file 5.46. [ ][ ]
• Simplify tests_quines.py::test_ differences,differences_deb to simply use assert_diff and not mangle the test fixture. [ ]

Supply-chain attack in the Solana ecosystem A significant supply-chain attack impacted Solana, an ecosystem for decentralised applications running on a blockchain. Hackers targeted the @solana/web3.js JavaScript library and embedded malicious code that extracted private keys and drained funds from cryptocurrency wallets. According to some reports, about $160,000 worth of assets were stolen, not including SOL tokens and other crypto assets.

Website updates Similar to last month, there was a large number of changes made to our website this month, including:

• Chris Lamb:
  • Make the landing page hero look nicer when the vertical height component of the viewport is restricted, not just the horizontal width.
  • Rename the Buy-in page to Why Reproducible Builds? [ ]
  • Removing the top black border. [ ][ ]
• Holger Levsen:
  • Fixed a number of issues on the 2024 Summit page, including fixing the path to a sponsor logo [ ] but also added the event documentation from Aspiration [ ].
  • Check and cleanup a presentation formerly linked from the About page on the Debian wiki. [ ]
  • Link to reproduce.debian.net on the Involved Projects page. [ ]
  • Fix a number of links on the Talks & Resources page. [ ][ ][ ][ ]
• hulkoba:
  • Remove the sidebar-type layout and move to a static navigation element. [ ][ ][ ][ ]
  • Create and merge a new Success stories page, which highlights the success stories of Reproducible Builds, showcasing real-world examples of projects shipping with verifiable, reproducible builds. These stories aim to enhance the technical resilience of the initiative by encouraging community involvement and inspiring new contributions. . [ ]
  • Further changes to the homepage. [ ]
  • Remove the translation icon from the navigation bar. [ ]
  • Remove unused CSS styles pertaining to the sidebar. [ ]
  • Add sponsors to the global footer. [ ]
  • Add extra space on large screens on the Who page. [ ]
  • Hide the side navigation on small screens on the Documentation pages. [ ]

Debian changes There were a significant number of reproducibility-related changes within Debian this month, including:

• Santiago Vila uploaded version 0.11+nmu4 of the dh-buildinfo package. In this release, the dh_buildinfo becomes a no-op ie. it no longer does anything beyond warning the developer that the dh-buildinfo package is now obsolete. In his upload, Santiago wrote that We still want packages to drop their [dependency] on dh-buildinfo, but now they will immediately benefit from this change after a simple rebuild.
• Holger Levsen filed Debian bug #1091550 requesting a rebuild of a number of packages that were built with a very old version of dpkg.
• Fay Stegerman contributed to an extensive thread on the debian-devel development mailing list on the topic of Supporting alternative zlib implementations . In particular, Fay wrote about her results experimenting whether zlib-ng produces identical results or not.
• kpcyrd uploaded a new rust-rebuilderd-worker, rust-derp, rust-in-toto and debian-repro-status to Debian, which passed successfully through the so-called NEW queue.
• Gioele Barabucci filed a number of bugs against the debrebuild component/script of the devscripts package, including:
  • #1089087: Address a spurious extra subdirectory in the build path.
  • #1089201: Extra zero bytes added to .dynstr when rebuilding CMake projects.
  • #1089088: Some binNMUs have a 1-second offset in some timestamps.
• Gioele Barabucci also filed a bug against the dh-r package to report that the Recommends and Suggests fields are missing from rebuilt R packages. At the time of writing, this bug has no patch and needs some help to make over 350 binary packages reproducible.
• Lastly, 8 reviews of Debian packages were added, 11 were updated and 11 were removed this month adding to our knowledge about identified issues.

Other development news In other ecosystem and distribution news:

• Jan-Benedict Glaw published the 6th NetBSD Reproducibility Report and reported on our mailing list as well.
• Developer unmush wrote a long post on the GNU Guix blog on the topic of Adding a fully-bootstrapped Mono to the distribution.
• The Glasgow Haskell Compiler (GHC) has released a new version of their compiler. This release introduces a new experimental flag, -fobject-determinism, which enables deterministic object code generation .
• The IzzyOnDroid Android APK repository published an extensive Review of 2024 and Outlook for 2025 which includes statistics and future plans related to reproducible builds (including having passed the 30% mark this month).
• The historic Arch Linux reproducibility tests that were hosted at tests.reproducible-builds.org/archlinux now redirect to reproducible.archlinux.org instead. In fact, everything Arch-related has now been removed from the jenkins.debian.net.git repository, as those continuous integration tests have been disabled for some time.
• reprotest version 0.7.29 was uploaded to Debian unstable by Vagrant Cascadian. It included contributions already covered in previous months as well as new ones from Rebecca N. Palmer, such as:
  • Stop using pkg_resources. [ ][ ]
  • The as_file attribute is not a method. [ ]
  • Use a non-constant object to test memory address capture. [ ]
• rebuilderd was updated as follows by kpcyrd:
  • Migrate diesel dependency from 1.x to 2.x. [ ]
  • Migrate clap dependency from 2 to 4. [ ]
  • Refactor reqwest code, and the replace openssl dependency with the memory-safe rustls. [ ][ ]

• Lastly, in openSUSE, Bernhard M. Wiedemann published another report for the distribution. There, Bernhard reports about the success of building R-B-OS , a partial fork of openSUSE with only 100% bit-reproducible packages. This effort was sponsored by the NLNet NGI0 initiative.

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

• Bernhard M. Wiedemann: cargo-packaging/rusty_v8, cockpit, collectd, deepin-daemon, deepin-file-manager, esbuild, grpc, hyperkitty, icedtea-web, java-atk-wrapper, kdenetwork-filesharing, kicad, kompare, librespeed-cli, lincity-ng, mraa, ollama, opa-fmgui, opencryptoki, opencryptoki, openmpi4:gnu-hpc, openwsman, patterns-microos, portmidi, presage, procps, sad, scons/nst, sendmail, static-initrd, suse-hpc, swtpm, tiny, vtk, xdg-desktop-portal and yast.
• Chris Lamb:
  • #1089011 filed against pyorbital.
  • #1089095 filed against python-pbcore.
• Gioele Barabucci:
  • #1089088 filed against sbuild.
• James Addison:
  • #1090395 filed against binutils.
• Johannes Schauer Marin Rodrigues:
  • #1089092 filed against hurd.
• Moritz Schlarb:
  • #1090078 filed against firehol.
• Roland Clobus:
  • #1090981 filed against dictionaries-common.

Reproducibility testing framework The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In November, a number of changes were made by Holger Levsen, including:

• reproduce.debian.net-related:
  • Add a new i386.reproduce.debian.net rebuilder. [ ][ ][ ][ ][ ][ ]
  • Make a number of updates to the documentation. [ ][ ][ ][ ][ ]
  • Run i386.reproduce.debian.net run on a public port to allow external workers. [ ]
  • Add a link to the /api/v0/pkgs/list endpoint. [ ]
  • Add support for a statistics page. [ ][ ][ ][ ][ ][ ]
  • Limit build logs to 20 MiB and diffoscope output to 10 MiB. [ ]
  • Improve the frontpage. [ ][ ]
  • Explain that we re testing arch:any and arch:all on the amd64 architecture, but only arch:any on i386. [ ]
• Misc:
  • Remove code for testing Arch Linux, which has moved to reproduce.archlinux.org. [ ][ ]
  • Don t install dstat on Jenkins nodes anymore as its been removed from Debian trixie. [ ]
  • Prepare the infom08-i386 node to become another rebuilder. [ ]
  • Add debug date output for benchmarking the reproducible_pool_buildinfos.sh script. [ ]
  • Install installation-birthday everywhere. [ ]
  • Temporarily disable automatic updates of pool links on buildinfos.debian.net. [ ]
  • Install Recommends by default on Jenkins nodes. [ ]
  • Rename rebuilder_stats.py to rebuilderd_stats.py. [ ]
  • r.d.n/stats: minor formatting changes. [ ]
  • Install files under /etc/cron.d/ with the correct permissions. [ ]

and Jochen Sprickerhof made the following changes:

• Always prefer official .buildinfo on buildinfos.debian.net files. [ ][ ][ ]
• Add a rebuilder_stats.py scripts. [ ]

Lastly, Gioele Barabucci also classified packages affected by 1-second offset issue filed as Debian bug #1089088 [ ][ ][ ][ ], Chris Hofstaedtler updated the URL for Grml s dpkg.selections file [ ], Roland Clobus updated the Jenkins log parser to parse warnings from diffoscope [ ] and Mattia Rizzolo banned a number of bots and crawlers from the service [ ][ ].
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Mastodon: @reproducible_builds@fosstodon.org
• Mailing list: rb-general@lists.reproducible-builds.org
• Twitter: @ReproBuilds

Most of my Debian contributions this month were sponsored by Freexian, as well as one direct donation via Liberapay (thanks!). OpenSSH I issued a bookworm update with a number of fixes that had accumulated over the last year, especially fixing GSS-API key exchange which was quite broken in bookworm. base-passwd A few months ago, the adduser maintainer started a discussion with me (as the base-passwd maintainer) and the shadow maintainer about bringing all three source packages under one team, since they often need to cooperate on things like user and group names. I agreed, but hadn t got round to doing anything about it until recently. I ve now officially moved it under team maintenance. debconf Gioele Barabucci has been working on eliminating duplicated code between debconf and cdebconf, ultimately with the goal of migrating to cdebconf (which I m not sure I m convinced of as a goal, but if we can make improvements to both packages as part of working towards it then there s no harm in that). I finally got round to reviewing and merging confmodule changes in each of debconf and cdebconf. This caused an installer regression due to a weirdness in cdebconf-udeb s packaging, which I fixed - sorry about that! I ve also been dealing with a few patch submissions that had been in my queue for a long time, but more on that next month if all goes well. CI issues I noticed and fixed a problem with Restrictions: needs-sudo in autopkgtest. I fixed broken aptly images in the Salsa CI pipeline. Python team Last month, I mentioned some progress on sorting out the multipart vs. python-multipart name conflict in Debian (#1085728), and said that I thought we d be able to finish it soon. I was right! We got it all done this month:

• gavodachs
• matrix-synapse (uploaded by Andrej Shadura)
• Packaged multipart
• Converted python-wadllib to use the packaged version of multipart
• Made trac (build-)depend on python3-multipart, fixing build failures in trac-customfieldadmin and trac-wysiwyg

The Python 3.13 transition continues, and last month we were able to add it to the supported Python versions in testing. (The next step will be to make it the default.) I fixed lots of problems in aid of this, including:

• audioread
• celery
• cloud-sptheme
• djangorestframework
• dominate
• fenics-basix (investigated and suggested a fix, although it hasn t yet been uploaded)
• ipykernel
• ipython
• jupyter-server (contributed upstream)
• mdp (contributed upstream)
• pastescript (contributed upstream)
• pypandoc (contributed upstream)
• python-aiosmtpd
• python-cheroot
• python-hjson
• python-miio
• python-pyramid
• python-trustme (uploaded by Robie Basak)
• rich (investigated and tested; uploaded by Sandro Tosi)
• supervisor
• tomopy

Sphinx 8.0 removed some old intersphinx_mapping syntax which turned out to still be in use by many packages in Debian. The fixes for this were individually trivial, but there were a lot of them:

• alot
• dot2tex (contributed upstream)
• flask-paginate (contributed upstream)
• ipython
• klepto
• nose
• pathos
• pox
• ppft
• psycopg3
• pybindgen (contributed upstream)
• pyina
• pyrr (contributed upstream)
• pytest-mpi (contributed upstream)
• python-aiohttp-security (contributed upstream)
• python-aiohttp-session (contributed upstream)
• python-anyqt (contributed upstream)
• python-argon2
• python-cai (contributed upstream)
• python-django-analytical (contributed upstream)
• python-iptables (contributed upstream)
• python-nacl
• python-pygtrie
• python-proto-plus (contributed upstream)
• python-requests-toolbelt
• python-tinycss
• python-whoosh
• symmetrize (contributed upstream)
• vcr.py
• wtforms-components
• wtforms-test
• yoyo
• zope.deferredimport

I found that twisted 24.11.0 broke tests in buildbot and wokkel, and fixed those. I packaged python-flatdict, needed for a new upstream version of python-semantic-release. I tracked down a test failure in vdirsyncer (which I ve been using for some years, but had never previously needed to modify) and contributed a fix upstream. I fixed some packages to tolerate future versions of dh-python that will drop their dependency on python3-setuptools:

• pyfribidi
• pylibmc
• python-mkdocs

I fixed django-cte to remove a build-dependency on the obsolete python3-nose package. I added Django 5.1 support to django-polymorphic. (There are a number of other packages that still need work here.) I fixed various other build/test failures:

• black
• datalad-next: #1080969 and #1088038 (contributed upstream)
• dipy (uploaded by Andreas Tille)
• pyfribidi
• pylibmc, fixing a build failure in cachelib
• pympress
• pytest-forked (contributed upstream)
• python-mongomock: c03f91b51048 and c485fb8fef23
• python-nox

I upgraded these packages to new upstream versions:

• aioftp
• alot
• astroid
• buildbot
• cloudpickle (fixing a Python 3.13 failure)
• django-countries
• django-sass-processor
• djoser (fixing CVE-2024-21543)
• ipython
• jsonpickle
• lazr.delegates
• loguru (fixing a Python 3.13 failure)
• netmiko
• pydantic
• pydantic-core
• pydantic-settings
• pydoctor
• pygresql
• pylint (fixing Python 3.13 failures #1089758 and #1091029)
• pypandoc (fixing a Python 3.12 warning)
• python-aiohttp (fixing CVE-2024-52303 and CVE-2024-52304
• python-aiohttp-security
• python-argcomplete
• python-asyncssh
• python-click
• python-cytoolz
• python-jira (fixing a Python 3.13 failure)
• python-limits
• python-line-profiler
• python-mkdocs
• python-model-bakery
• python-pgspecial
• python-pyramid (fixing CVE-2023-40587)
• python-pythonjsonlogger
• python-semantic-release
• python-utils
• python-venusian
• pyupgrade
• pyzmq
• quart
• six
• sqlparse
• twisted
• vcr.py
• vulture
• yoyo
• zope.configuration
• zope.testrunner

I updated the team s library style guide to remove material related to Python 2 and early versions of Python 3, which is no longer relevant to any current Python packaging work. Other Python upstream work I happened to notice a Twisted upstream issue requesting the removal of the deprecated twisted.internet.defer.returnValue, realized it was still used in many places in Debian, and went on a PR-filing spree informed by codesearch to try to reduce the future impact of such a change on Debian:

• foolscap
• gnome-keysign
• magic-wormhole
• matrix-synapse
• python-autobahn
• python-testtools
• python-treq
• tahoe-lafs: Inconclusive investigation
• txtorcon

Other small fixes Santiago Vila has been building the archive with make --shuffle (also see its author s explanation). I fixed associated bugs in cccc (contributed upstream), groff, and spectemu. I backported an upstream patch to putty to fix undefined behaviour that affected use of the small keypad . I removed groff s Recommends: libpaper1 (#1091375, #1091376), since it isn t currently all that useful and was getting in the way of a transition to libpaper2. I filed an upstream bug suggesting better integration in this area.

TL;DR: procmail is a security liability and has been abandoned upstream for the last two decades. If you are still using it, you should probably drop everything and at least remove its SUID flag. There are plenty of alternatives to chose from, and conversion is a one-time, acceptable trade-off.

Procmail is unmaintained procmail is unmaintained. The "Final release", according to Wikipedia, dates back to September 10, 2001 (3.22). That release was shipped in Debian since then, all the way back from Debian 3.0 "woody", twenty years ago. Debian also ships 25 uploads on top of this, with 3.22-21 shipping the "3.23pre" release that has been rumored since at least the November 2001, according to debian/changelog at least:

procmail (3.22-1) unstable; urgency=low
  * New upstream release, which uses the  standard' format for Maildir
    filenames and retries on name collision. It also contains some
    bug fixes from the 3.23pre snapshot dated 2001-09-13.
  * Removed  sendmail' from the Recommends field, since we already
    have  exim' (the default Debian MTA) and  mail-transport-agent'.
  * Removed suidmanager support. Conflicts: suidmanager (<< 0.50).
  * Added support for DEB_BUILD_OPTIONS in the source package.
  * README.Maildir: Do not use locking on the example recipe,
    since it's wrong to do so in this case.
 -- Santiago Vila <sanvila@debian.org>  Wed, 21 Nov 2001 09:40:20 +0100

All Debian suites from buster onwards ship the 3.22-26 release, although the maintainer just pushed a 3.22-27 release to fix a seven year old null pointer dereference, after this article was drafted. Procmail is also shipped in all major distributions: Fedora and its derivatives, Debian derivatives, Gentoo, Arch, FreeBSD, OpenBSD. We all seem to be ignoring this problem. The upstream website (http://procmail.org/) has been down since about 2015, according to Debian bug #805864, with no change since. In effect, every distribution is currently maintaining its fork of this dead program. Note that, after filing a bug to keep Debian from shipping procmail in a stable release again, I was told that the Debian maintainer is apparently in contact with the upstream. And, surprise! they still plan to release that fabled 3.23 release, which has been now in "pre-release" for all those twenty years. In fact, it turns out that 3.23 is considered released already, and that the procmail author actually pushed a 3.24 release, codenamed "Two decades of fixes". That amounts to 25 commits since 3.23pre some of which address serious security issues, but none of which address fundamental issues with the code base.

Procmail is insecure By default, procmail is installed SUID root:mail in Debian. There's no debconf or pre-seed setting that can change this. There has been two bug reports against the Debian to make this configurable (298058, 264011), but both were closed to say that, basically, you should use dpkg-statoverride to change the permissions on the binary. So if anything, you should immediately run this command on any host that you have procmail installed on:

dpkg-statoverride --update --add root root 0755 /usr/bin/procmail

Note that this might break email delivery. It might also not work at all, thanks to usrmerge. Not sure. Yes, everything is on fire. This is fine. In my opinion, even assuming we keep procmail in Debian, that default should be reversed. It should be up to people installing procmail to assign it those dangerous permissions, after careful consideration of the risk involved. The last maintainer of procmail explicitly advised us (in that null pointer dereference bug) and other projects (e.g. OpenBSD, in [2]) to stop shipping it, back in 2014. Quote:

Executive summary: delete the procmail port; the code is not safe and should not be used as a basis for any further work.

I just read some of the code again this morning, after the original author claimed that procmail was active again. It's still littered with bizarre macros like:

#define bit_set(name,which,value) \
  (value?(name[bit_index(which)] =bit_mask(which)):\
  (name[bit_index(which)]&=~bit_mask(which)))

... from regexp.c, line 66 (yes, that's a custom regex engine). Or this one:

#define jj  (aleps.au.sopc)

It uses insecure functions like strcpy extensively. malloc() is thrown around gotos like it's 1984 all over again. (To be fair, it has been feeling like 1984 a lot lately, but that's another matter entirely.) That null pointer deref bug? It's fixed upstream now, in this commit merged a few hours ago, which I presume might be in response to my request to remove procmail from Debian. So while that's nice, this is the just tip of the iceberg. I speculate that one could easily find an exploitable crash in procmail if only by running it through a fuzzer. But I don't need to speculate: procmail had, for years, serious security issues that could possibly lead to root privilege escalation, remotely exploitable if procmail is (as it's designed to do) exposed to the network. Maybe I'm overreacting. Maybe the procmail author will go through the code base and do a proper rewrite. But I don't think that's what is in the cards right now. What I expect will happen next is that people will start fuzzing procmail, throw an uncountable number of bug reports at it which will get fixed in a trickle while never fixing the underlying, serious design flaws behind procmail.

Procmail has better alternatives The reason this is so frustrating is that there are plenty of modern alternatives to procmail which do not suffer from those problems. Alternatives to procmail(1) itself are typically part of mail servers. For example, Dovecot has its own LDA which implements the standard Sieve language (RFC 5228). (Interestingly, Sieve was published as RFC 3028 in 2001, before procmail was formally abandoned.) Courier also has "maildrop" which has its own filtering mechanism, and there is fdm (2007) which is a fetchmail and procmail replacement. Update: there's also mailprocessing, which is not an LDA, but processing an existing folder. It was, however, specifically designed to replace complex Procmail rules. But procmail, of course, doesn't just ship procmail; that would just be too easy. It ships mailstat(1) which we could probably ignore because it only parses procmail log files. But more importantly, it also ships:

• lockfile(1) - conditional semaphore-file creator
• formail(1) - mail (re)formatter

lockfile(1) already has a somewhat acceptable replacement in the form of flock(1), part of util-linux (which is Essential, so installed on any normal Debian system). It might not be a direct drop-in replacement, but it should be close enough. formail(1) is similar: the courier maildrop package ships reformail(1) which is, presumably, a rewrite of formail. It's unclear if it's a drop-in replacement, but it should probably possible to port uses of formail to it easily.

Update: the maildrop package ships a SUID root binary (two, even). So if you want only reformail(1), you might want to disable that with:

dpkg-statoverride --update --add root root 0755 /usr/bin/lockmail.maildrop 
dpkg-statoverride --update --add root root 0755 /usr/bin/maildrop

It would be perhaps better to have reformail(1) as a separate package, see bug 1006903 for that discussion.

The real challenge is, of course, migrating those old .procmailrc recipes to Sieve (basically). I added a few examples in the appendix below. You might notice the Sieve examples are easier to read, which is a nice added bonus.

Conclusion There is really, absolutely, no reason to keep procmail in Debian, nor should it be used anywhere at this point. It's a great part of our computing history. May it be kept forever in our museums and historical archives, but not in Debian, and certainly not in actual release. It's just a bomb waiting to go off. It is irresponsible for distributions to keep shipping obsolete and insecure software like this for unsuspecting users. Note that I am grateful to the author, I really am: I used procmail for decades and it served me well. But now, it's time to move, not bring it back from the dead.

Appendix

Previous work It's really weird to have to write this blog post. Back in 2016, I rebuilt my mail setup at home and, to my horror, discovered that procmail had been abandoned for 15 years at that point, thanks to that LWN article from 2010. I would have thought that I was the only weirdo still running procmail after all those years and felt kind of embarrassed to only "now" switch to the more modern (and, honestly, awesome) Sieve language. But no. Since then, Debian shipped three major releases (stretch, buster, and bullseye), all with the same vulnerable procmail release. Then, in early 2022, I found that, at work, we actually had procmail installed everywhere, possibly because userdir-ldap was using it for lockfile until 2019. I sent a patch to fix that and scrambled to remove get rid of procmail everywhere. That took about a day. But many other sites are now in that situation, possibly not imagining they have this glaring security hole in their infrastructure.

Procmail to Sieve recipes I'll collect a few Sieve equivalents to procmail recipes here. If you have any additions, do contact me. All Sieve examples below assume you drop the file in ~/.dovecot.sieve.

deliver mail to "plus" extension folder Say you want to deliver user+foo@example.com to the folder foo. You might write something like this in procmail:

MAILDIR=$HOME/Maildir/
DEFAULT=$MAILDIR
LOGFILE=$HOME/.procmail.log
VERBOSE=off
EXTENSION=$1            # Need to rename it - ?? does not like $1 nor 1
:0
* EXTENSION ?? [a-zA-Z0-9]+
        .$EXTENSION/

That, in sieve language, would be:

require ["variables", "envelope", "fileinto", "subaddress"];
########################################################################
# wildcard +extension
# https://doc.dovecot.org/configuration_manual/sieve/examples/#plus-addressed-mail-filtering
if envelope :matches :detail "to" "*"  
  # Save name in $ name  in all lowercase
  set :lower "name" "$ 1 ";
  fileinto "$ name ";
  stop;
 

Subject into folder This would file all mails with a Subject: line having FreshPorts in it into the freshports folder, and mails from alternc.org mailing lists into the alternc folder:

:0
## mailing list freshports
* ^Subject.*FreshPorts.*
.freshports/
:0
## mailing list alternc
* ^List-Post.*mailto:.*@alternc.org.*
.alternc/

Equivalent Sieve:

if header :contains "subject" "FreshPorts"  
    fileinto "freshports";
  elsif header :contains "List-Id" "alternc.org"  
    fileinto "alternc";
 

Mail sent to root to a reports folder This double rule:

:0
* ^Subject: Cron
* ^From: .*root@
.rapports/

Would look something like this in Sieve:

if header :comparator "i;octet" :contains "Subject" "Cron"  
  if header :regex :comparator "i;octet"  "From" ".*root@"  
        fileinto "rapports";
   
 

Note that this is what the automated converted does (below). It's not very readable, but it works.

Bulk email I didn't have an equivalent of this in procmail, but that's something I did in Sieve:

if header :contains "Precedence" "bulk"  
    fileinto "bulk";
 

Any mailing list This is another rule I didn't have in procmail but I found handy and easy to do in Sieve:

if exists "List-Id"  
    fileinto "lists";
 

This or that I wouldn't remember how to do this in procmail either, but that's an easy one in Sieve:

if anyof (header :contains "from" "example.com",
           header :contains ["to", "cc"] "anarcat@example.com")  
    fileinto "example";
 

You can even pile up a bunch of options together to have one big rule with multiple patterns:

if anyof (exists "X-Cron-Env",
          header :contains ["subject"] ["security run output",
                                        "monthly run output",
                                        "daily run output",
                                        "weekly run output",
                                        "Debian Package Updates",
                                        "Debian package update",
                                        "daily mail stats",
                                        "Anacron job",
                                        "nagios",
                                        "changes report",
                                        "run output",
                                        "[Systraq]",
                                        "Undelivered mail",
                                        "Postfix SMTP server: errors from",
                                        "backupninja",
                                        "DenyHosts report",
                                        "Debian security status",
                                        "apt-listchanges"
                                        ],
           header :contains "Auto-Submitted" "auto-generated",
           envelope :contains "from" ["nagios@",
                                      "logcheck@",
                                      "root@"])
     
    fileinto "rapports";
 

Automated script There is a procmail2sieve.pl script floating around, and mentioned in the dovecot documentation. It didn't work very well for me: I could use it for small things, but I mostly wrote the sieve file from scratch.

Progressive migration Enrico Zini has progressively migrated his procmail setup to Sieve using a clever way: he hooked procmail inside sieve so that he could deliver to the Dovecot LDA and progressively migrate rules one by one, without having a "flag day". See this explanatory blog post for the details, which also shows how to configure Dovecot as an LMTP server with Postfix.

Other examples The Dovecot sieve examples are numerous and also quite useful. At the time of writing, they include virus scanning and spam filtering, vacation auto-replies, includes, archival, and flags.

Harmful considered harmful I am aware that the "considered harmful" title has a long and controversial history, being considered harmful in itself (by some people who are obviously not afraid of contradictions). I have nevertheless deliberately chosen that title, partly to make sure this article gets maximum visibility, but more specifically because I do not have doubts at this moment that procmail is, clearly, a bad idea at this moment in history.

Developing story I must also add that, incredibly, this story has changed while writing it. This article is derived from this bug I filed in Debian to, quite frankly, kick procmail out of Debian. But filing the bug had the interesting effect of pushing the upstream into action: as mentioned above, they have apparently made a new release and merged a bunch of patches in a new git repository. This doesn't change much of the above, at this moment. If anything significant comes out of this effort, I will try to update this article to reflect the situation. I am actually happy to retract the claims in this article if it turns out that procmail is a stellar example of defensive programming and survives fuzzing attacks. But at this moment, I'm pretty confident that will not happen, at least not in scope of the next Debian release cycle.

Here's what happened in the Reproducible Builds effort between Sunday April 23 and Saturday April 29 2017: Past and upcoming events On April 26th Chris Lamb gave a talk at foss-north 2017 in Gothenburg, Sweden on Reproducible Builds. Between May 5th-7th the Reproducible Builds Hackathon 2017 will take place in Hamburg, Germany. Then on May 26th Bernhard M. Wiedemann will give a talk titled reproducible builds in openSUSE (2017) at the openSUSE Conference 2017 in N rnberg, Germany. Media coverage Already on April 19th Sylvain Beucler wrote a yet another follow-up post Practical basics of reproducible builds 3, after part 1 and part 2 of his series. Toolchain development and fixes Michael Woerister of the Rust project has implemented file maps that affect all path-related compiler information, including "error messages, metadata, debuginfo, and the file!() macro alike". Ximin Luo with support from some other Rust developers and contributors helped steer the final result into something that was compatible with reproducible builds. Many thanks to all involved, especially for the patience of discussing this over several months. Ximin wrote a first-attempt patch to fix R build-path issues. It made 460/477 R packages reproducible, but also caused 3 of these to FTBFS. See randomness_in_r_rdb_rds_databases for details. Bugs filed and patches sent upstream Chris Lamb:

• #861133 filed against tf.
• #861443 filed against ora2pg.

Bernhard M. Wiedemann filed a number of patches upstream:

• intltool
• xine-ui
• tboot
• javapackages
• mxml, which got merged already.
• calibre, also merged.
• wammu, merged as well.

Reviews of unreproducible packages 102 package reviews have been added, 64 have been updated and 24 have been removed in this week, adding to our knowledge about identified issues. 3 issue types have been updated:

• Added captures_build_path_in_beam_files, recategorise some erlang packages with captures_build_path into this issue instead.
• Removed timestamps_in_beam_files.
• Holger started reviewing blacklisted_on_jenkins and blacklisted_on_jenkins_armhf_only and found quite some packages which don't need to be blacklisted anymore.

Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Aaron M. Ucko (1)
• Adrian Bunk (1)
• Chris Lamb (4)
• Santiago Vila (2)

diffoscope development diffoscope 82 was uploaded to experimental by Chris Lamb. It included contributions from:

• Chris Lamb:
  • Add support for Ogg Vorbis files.
• Vagrant Cascadian:
  • Add support for .dtb (device tree blob) files. (Closes: #861109).

Changes from previous weeks that were also released with 82:

• Ximin Luo
  • Add support for R .rds and .rdb object files.
• Chris Lamb
  • Add support for comparing Pcap files.
  • Add support for .docx and .odt files via docx2txt & odt2txt.
  • Add support for PGP files via pgpdump.
  • Various documentation and test improvements.
  • Various bug fixes and code quality improvements.
• Sylvain Beucler
  • Display differences in zip platform-specific timestamps.

Misc. This week's edition was written by Ximin Luo, Chris Lamb and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

What happened in the Reproducible Builds effort between Sunday November 27 and Saturday December 3 2016: Reproducible work in other projects

• Ducible is a new tool to make Windows builds reproducible.
• Manish Goregaokar wrote about Reflections on Rusting Trust.

Media coverage, etc.

• There was a Reproducible Builds hackathon in Boston with contributions from Dafydd, Valerie, Clint, Harlen, Anders, Robbie and Ben. (See the "Bugs filed" section below for the results).
• Distrowatch mentioned Webconverger's reproducible status.

Bugs filed Chris Lamb:

• #846588 filed against minicoredumper.
• #846647 filed against tinyeartrainer.
• #846842 filed against nethogs.

Clint Adams:

• #846892 filed against pkg-mozilla-archive-keyring.

Dafydd Harries:

• #846893 filed against flac.

Daniel Shahaf:

• #846832 filed against patat.

Reiner Herrmann:

• #845991 filed against pathogen.

Valerie R Young:

• #846878 filed against taggrepper.
• #846890 filed against ipsvd.
• #846891 filed against integrit.

Reviews of unreproducible packages 15 package reviews have been added, 4 have been updated and 26 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been added:

• nondeterminstic_ordering_in_casacore_tables
• nondeterminstic_output_from_uglifyjs

Weekly QA work During our reproducibility testing, some FTBFS bugs have been detected and reported by:

• Chris Lamb (5)
• Lucas Nussbaum (8)
• Santiago Vila (1)

diffoscope development

• diffoscope 63 was uploaded to unstable by Ximin Luo:
  • Greatly improve speed for large archives by fixing O(n^2) complexity for archive member lookup.
  • add +/- buttons to toggle visibility of parts of the diff
  • Output coloured diff using colordiff(1) via --text-color= never,auto,always

Is is available now in Debian, Archlinux and on PyPI. strip-nondeterminism development

• At the Reproducible Builds Boston hackathon Anders Kaseorg filed #846895 treat .par files as Zip archives, including a patch which was merged into master.

reprotest development

• reprotest 0.4 was uploaded to unstable by Ximin Luo:
  • disorderfs variation: don't query the testbed, put that in the script instead
  • Add a build_path_same variation to run builds from the same path
  • Fix auto-presets in the case of a file in the current directory
  • Fix d/control so reprotesting reprotest in sbuild works (6 reproductions)
  • Add util-linux to Recommends since we use it to vary some things

tests.reproducible-builds.org

• Holger made a couple of changes:
  • Group all "done" and all "open" usertagged bugs together in the bugs graphs and move the "done bugs" from the bottom of these gaps.
  • Update list of packages installed on .debian.org machines.
  • Made the maintenance jobs run every 2h instead of 3h.
  • Various bug fixes and minor improvements.
• After thorough review by Mattia, some patches by Valerie were merged in preparation of the switch from sqlite to Postgresql, most notably a conversion to the sqlalchemy expression language.
• Holger gave a talk at Profitbricks about how Debian is using 168 cores, 503 GB RAM and 5 TB storage to make jenkins.debian.net and tests.reproducible-builds.org run. Many thanks to Profitbricks for supporting jenkins.debian.net since August 2012!
• Holger created a Jenkins job to build reprotest from git master branch.
• Finally, the Jenkins Naginator plugin was installed to retry git cloning in case of Alioth/network failures, this will benefit all jobs using Git on jenkins.debian.net.

Misc. This week's edition was written by Chris Lamb, Valerie Young, Vagrant Cascadian, Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

What happened in the Reproducible Builds effort between Sunday October 16 and Saturday October 22 2016: Media coverage

• Chris Lamb presented at Software Freedom Kosovo on reproducible builds on Saturday 22nd October.

Upcoming events

• J rgen Rose will be giving a talk on Enforcing reproducible builds with Eclipse Package Drone at EclipseCon Europe 2016 in Ludwigsburg, Germany on October 27th.

buildinfo.debian.net In order to build packages reproducibly, you not only need identical sources but also some external definition of the environment used for a particular build. This definition includes the inputs and the outputs and, in the Debian case, are available in a $package_$architecture_$version.buildinfo file. We anticipate the next dpkg upload to sid will create .buildinfo files by default. Whilst it's clear that we also need to teach dak to deal with them (#763822) its not actually clear how to handle .buildinfo files after dak has processed them and how to make them available to the world. To this end, Chris Lamb has started development on a proof-of-concept .buildinfo server to see what issues arise. Source Reproducible work in other projects

• Ximin Luo submitted a patch to GCC as a prerequisite for future patches to make debugging symbols reproducible.

Packages reviewed and fixed, and bugs filed

• #841274 filed against node-once by Chris Lamb.
• #841342 filed against zshdb by Chris Lamb.
• #841427 filed against unifdef by Chris Lamb.
• #841440 filed against rdp-alignment by Chris Lamb.
• #841497 filed against cf-python by Chris Lamb.
• #841694 filed against dvdauthor by Reiner Herrmann.
• #841698 filed against node-lodash by Chris Lamb.
• #841701 filed against libtext-charwidth-perl by Reiner Herrmann.
• #841702 filed against libapt-pkg-perl by Reiner Herrmann.
• #841703 filed against libio-pty-perl by Reiner Herrmann.
• #841707 filed against eximdoc4 by Chris Lamb.

Reviews of unreproducible packages 99 package reviews have been added, 3 have been updated and 6 have been removed in this week, adding to our knowledge about identified issues. 6 issue types have been added:

• bin_sh_is_bash
• captures_build_arch
• captures_build_path_via_assert
• docbook_to_man_one_byte_delta_on_i386
• graphviz_nondeterminstic_output
• randomness_in_documentation_generated_by_scilab

Weekly QA work During of reproducibility testing, some FTBFS bugs have been detected and reported by:

• Chris Lamb (23)
• Daniel Reichelt (2)
• Lucas Nussbaum (1)
• Santiago Vila (18)

diffoscope development

• Mattia Rizzolo:
  • tests/ppu: skip some PPU tests if ppudump is < 3.0.0
  • ppu: ignore decoding errors from ppudump while filtering the output
  • ppu: don't do run a full ppudump while only looking for PPU file version
  • debian: bump debhelper compat level to 10, no changes needed.
• Michel Messerschmidt:
  • Add rudimentary support for OpenDocumentFormat files

tests.reproducible-builds.org

• h01ger increased the diskspace for reproducible content on Jenkins. Thanks to ProfitBricks.
• Valerie Young supplied a patch to make Python SQL interface more SQLite/PostgresSQL agnostic.
• lynxis worked hard to make LEDE and OpenWrt builds happen on two hosts.

Misc. Our poll to find a good time for an IRC meeting is still running until Tuesday, October 25st; please reply as soon as possible. We need a logo! Some ideas and requirements for a Reproducible Builds logo have been documented in the wiki. Contributions very welcome, even if simply by forwarding this information. This week's edition was written by Chris Lamb & Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

OpenStack Newton is released, and uploaded to Sid OpenStack Newton was released on the Thursday 6th of October. I was able to upload nearly all of it before the week-end, though there was a bit of hick-ups still, as I forgot to upload python-fixtures 3.0.0 to unstable, and only realized it thanks to some bug reports. As this is a build time dependency, it didn t disrupt Sid users too much, but 38 packages wouldn t build without it. Thanks to Santiago Vila for pointing at the issue here. As of writing, a lot of the Newton packages didn t migrate to Testing yet. It s been migrating in a very messy way. I d love to improve this process, but I m not sure how, if not filling RC bugs against 250 packages (which would be painful to do), so they would migrate at once. Suggestions welcome. Bye bye Jenkins For a few years, I was using Jenkins, together with a post-receive hook to build Debian Stable backports of OpenStack packages. Though nearly a year and a half ago, we had that project to build the packages within the OpenStack infrastructure, and use the CI/CD like OpenStack upstream was doing. This is done, and Jenkins is gone, as of OpenStack Newton. Current status As of August, almost all of the packages Git repositories were uploaded to OpenStack Gerrit, and the build now happens in OpenStack infrastructure. We ve been able to build all packages a release OpenStack Newton Debian packages using this system. This non-official jessie backports repository has also been validated using Tempest. Goodies from Gerrit and upstream CI/CD It is very nice to have it built this way, so we will be able to maintain a full CI/CD in upstream infrastructure using Newton for the life of Stretch, which means we will have the tools to test security patches virtually forever. Another thing is that now, anyone can propose packaging patches without the need for an Alioth account, by sending a patch for review through Gerrit. It is our hope that this will increase the likeliness of external contribution, for example from 3rd party plugins vendors (ie: networking driver vendors, for example), or upstream contributors themselves. They are already used to Gerrit, and they all expected the packaging to work this way. They are all very much welcome. The upstream infra: nodepool, zuul and friends
The OpenStack infrastructure has been described already in planet.debian.org, by Ian Wienand. So I wont describe it again, he did a better job than I ever would. How it works All source packages are stored in Gerrit with the deb- prefix. This is in order to avoid conflict with upstream code, and to easily locate packaging repositories. For example, you ll find Nova packaging under https://git.openstack.org/cgit/openstack/deb-nova. Two Debian repositories are stored in the infrastructure AFS (Andrew File System, which means a copy of that repository exist on each cloud were we have compute resources): one for the actual deb-* builds, under jessie-newton , and one for the automatic backports, maintained in the deb-auto-backports gerrit repository. We re using a git tag based workflow. Every Gerrit repository contains all of the upstream branch, plus a debian/newton branch, which contains the same content as a tag of upstream, plus the debian folder. The orig tarball is generated using git archive , then used by sbuild to produce binaries. To package a new upstream release, one simply needs to git merge -X theirs FOO (where FOO is the tag you want to merge), then edit debian/changelog so that the Debian package version matches the tag, then do git commit -a amend , and simply git review . At this point, the OpenStack CI will build the package. If it builds correctly, then a core reviewer can approve the merge commit , the patch is merged, then the package is built and the binary package published on the OpenStack Debian package repository. Maintaining backports automatically The automatic backports is maintained through a Gerrit repository called deb-auto-backports containing a packages-list file that simply lists source packages we need to backport. On each new CR (change request) in Gerrit, thanks to some madison-lite and dpkg compare-version magic, the packages-list is used to compare what s in the Debian archive and what we have in the jessie-newton-backports repository. If the version is lower in our repository, or if the package doesn t exist, then a build is triggered. There is the possibility to backport from any Debian release (using the -d flag in the packages-list file), and even we can use jessie-backports to just rebuild the package. I also had to write a hack to just download from jessie-backports without rebuilding, because rebuilding the webkit2gtk package (needed by sphinx) was taking too resources (though we ll try to never use it, and rebuild packages when possible). The nice thing with this system, is that we don t need to care much about maintaining packages up-to-date: the script does that for us. Upstream Debian repository are NOT for production The produced package repositories are there because we have interconnected build dependencies, needed to run unit test at build time. It is the only reason why such Debian repository exist. They are not for production use. If you wish to deploy OpenStack, we very much recommend using packages from distributions (like Debian or Ubuntu). Indeed, the infrastructure Debian repositories are updated multiple times daily. As a result, it is very likely that you will experience failures to download (hash or file size mismatch and such). Also, the functional tests aren t yet wired in the CI/CD in OpenStack infra, and therefore, we cannot guarantee yet that the packages are usable. Improving the build infrastructure There s a bunch of things which we could do to improve the build process. Let me give a list of things we want to do.

• Get sbuild pre-setup in the Jessie VM images, so we can win 3 minutes per build. This means writing a diskimage-builder element for sbuild.
• Have the infrastructure use a state-of-the-art Debian ftp-sync mirror, instead of the current reprepro mirroring which produces an unsigned reprository, which we can t use for sbuild-createchroot. This will improve things a lot, as currently, there s lots of build failures because of httpredir.debian.org mirror inconsistencies (and these are very frustrating loss of time).
• For each packaging change, there s 3 build: the check job, the gate job, and the POST job. This is a loss of time and resources, as we need to build a package once only. It will be hopefully possible to fix this when the OpenStack infra team will deploy Zuul 3.

Generalizing to Debian During Debconf 16, I had very interesting talks with the DSA (Debian System Administrator) about deploying such a CI/CD for the whole of the Debian archive, interfacing Gerrit with something like dgit and a build CI. I was told that I should provide a proof of concept first, which I very much agreed with. Such a PoC is there now, within OpenStack infra. I very much welcome any Debian contributor to try it, through a packaging patch. If you wish to do so, you should read how to contribute to OpenStack here: https://wiki.openstack.org/wiki/How_To_Contribute#If_you.27re_a_developer and then simply send your patch with git review . This system, however, currently only fits the git tag based packaging workflow. We d have to do a little bit more work to make it possible to use pristine-tar (basically, allow to push in the upstream and pristine-tar branches without any CI job connected to the push). Dear DSA team, as we now nice PoC that is working well, on which the OpenStack PKG team is maintaining 100s of packages, shall we try to generalize and provide such infrastructure for every packaging team and DDs?

Here is what happened in the Reproducible Builds effort between Sunday September 18 and Saturday September 24 2016: Outreachy We intend to participate in Outreachy Round 13 and look forward for new enthusiastic applications to contribute to reproducible builds. We're offering four different areas to work on:

• Improve test and debugging tools.
• Improving reproducibility of Debian packages.
• Improving Debian infrastructure.
• Help collaboration across distributions.

Reproducible Builds World summit #2 We are planning e a similar event to our Athens 2015 summit and expect to reveal more information soon. If you haven't been contacted yet but would like to attend, please contact holger. Toolchain development and fixes Mattia uploaded dpkg/1.18.10.0~reproducible1 to our experimental repository. and covered the details for the upload in a mailing list post. The most important change is the incorporation of improvements made by Guillem Jover (dpkg maintainer) to the .buildinfo generator. This is also in the hope that it will speed up the merge in the upstream. One of the other relevant changes from before is that .buildinfo files generated from binary-only builds will no longer include the hash of the .dsc file in Checksums-Sha256 as documented in the specification. Even if it was considered important to include a checksum of the source package in .buildinfo, storing it that way breaks other assumptions (eg. that Checksums-Sha256 contains only files part of that are part of a single upload, wheras the .dsc might not be part of that upload), thus we look forward for another solution to store the source checksum in .buildinfo. Bugs filed

• #838713 filed against python-xlib by Chris Lamb.
• #838754 filed against golang-google-grpc by Chris Lamb.
• #838188 filed against ocaml by Johannes Schauer.
• #838785 filed against funnelweb by Reiner Herrmann.

Reviews of unreproducible packages 250 package reviews have been added, 4 have been updated and 4 have been removed in this week, adding to our knowledge about identified issues. 4 issue types have been added:

• captures_users_gecos issue
• timestamps_in_org_mode_html_output toolchain issue.
• varnish_vmodtool_random_file_id
• gpg_keyring_magic_bytes_differ

3 issue types have been updated:

• timestamps_in_org_mode_html_output more generally.
• timestamps_in_documentation_generated_by_texi2html
• randomness_in_ocaml_preprocessed_files

Weekly QA work FTBFS bugs have been reported by:

• Chris Lamb (11)
• Santiago Vila (2)

Documentation updates h01ger created a new Jenkins job so that every commit pushed to the master branch for the website will update reproducible-builds.org. diffoscope development

• Mattia Rizzolo:
  • Skip rlib tests if the "nm" tool is missing
• Ximin Luo:
  • Give better advice about what envvar to set to make the console work
  • tests/basic-command-line: check exit code and use a more complex example
  • Add a script to check sizes of dependencies
  • Don't use unicode quotes to avoid breakage under LC_ALL=C

strip-nondeterminism development

• Chris Lamb:
  • .perltidyrc: Add from lintian.
  • .perltidyrc: We use tabs, not spaces.
  • Run perltidy

reprotest development

• Ximin Luo uploaded reprotest 0.3 and 0.3.1 to unstable with these changes:
  • Make some variations more reliable, so tests don't fail
  • Add a safety device to guard against typos
  • Address lintian warnings
  • Remove any existing artifact, in case the build script doesn't overwrite it
  • Fix the logic of some tests, and don't vary fileordering on Debian buildds
  • Use the magic of VIRTUALENV_DOWNLOAD=no, seen in tox's own autopkgtest tests
  • Flush so subprocess output is guaranteed to appear later
  • Don't error if the build command generates stderr
  • Default tests to run on "null" only since it takes effort to set up the others
  • hey dawg i herd u liek tests so i put some tests in ur tests so u can test while u test
  • Make no_clear_on_error optional; we don't want to pass it in everywhere e.g. tests
  • Output a nice big obvious summary at the end when successful
  • Don't repeat documentation in two different places, move it all to --help
  • More reliable build artifact pattern matching, and update docs
  • Use a list comprehension for slightly more idiomatic python than map/lambda
  • Make multi-component artifact patterns like '.deb .changes' work correctly
  • Add a --no-clean-on-error option so you can analyse what went wrong
  • More help text for virtual_server_args
  • Support shell patterns as the build_artifact
  • Use sys.exit inside main(), not check() as it's more idiomatic
  • Pass kind='build' to check_exec so it doesn't time out after 100s
  • adt_testbed: add stdout/stderr to the "auxverb failed" error message, if available
  • If no virtual_server is given then use "null"

tests.reproducible-builds.org

• The full rebuild of all packages in unstable (for all tested archs) with the new build path variation has been completed. This has had the result that we are down to ~75% reproducible packages in unstable now. In comparison, for testing (where we don't vary the build path) we are still at ~90%. IRC notifications for unstable have been enabled again. (Holger)
• Make the notes job robust about bad data (see #833695 and #833738). (Holger)
• Setup profitbricks-build7 running stretch as testing reproducible builds of F-Droid need to use a newer version of vagrant in order to support running vagrant VMs with kvm on kvm. (Holger)
• The misbehaving 'opi2a' armhf node has been replaced with a Jetson-TK1 board kindly donated by NVidia. This machine is using an NVIDIA tegra-k1 (cortex-a15) quad-core board. (vagrant and Holger)

Misc. This week's edition was written by Chris Lamb, Holger Levsen and Mattia Rizzolo and reviewed by a bunch of Reproducible Builds folks on IRC.

What happened in the Reproducible Builds effort between Sunday September 11 and Saturday September 17 2016: Toolchain developments Ximin Luo started a new series of tools called (for now) debrepatch, to make it easier to automate checks that our old patches to Debian packages still apply to newer versions of those packages, and still make these reproducible. Ximin Luo updated one of our few remaining patches for dpkg in #787980 to make it cleaner and more minimal. The following tools were fixed to produce reproducible output:

• naturaldocs/1.51-2 by Petter Reinholdtsen, original patch by Chris Lamb.

Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible - in our current test setup - after being fixed:

• elog/3.1.2-1-1 by Roger Kalt, original patch by Reiner Herrmann.
• eyed3/0.6.18-3 by Petter Reinholdtsen, original patch by Chris Lamb.
• frog/0.13.5-1 by Maarten van Gompel, original patch by Chris Lamb.
• gtranslator/2.91.7-3 by Andreas Henriksson, original patch by Reiner Herrmann.
• sozi/12.05-1.1 by Daniel Kahn Gillmor, original patch by Chris Lamb.

The following updated packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

• evince/3.21.92-1 by Michael Biebl.
• gnome-control-center/1:3.21.92-2 by Rapha l Hertzog.
• libipathverbs/1.3-2 by Ana Beatriz Guerrero Lopez.
• pagekite/0.5.8e-2 by Petter Reinholdtsen.

The following 3 packages were not changed, but have become reproducible due to changes in their build-dependencies: jaxrs-api python-lua zope-mysqlda. Some uploads have addressed some reproducibility issues, but not all of them:

• eurephia/1.1.0-6 by Alberto Gonzalez Iniesta, original patch by Chris Lamb.
• fdroidserver/0.7.0-1 by Hans-Christoph Steiner, original patch by Chris Lamb.
• mini-buildd/1.0.18 by Stephan S rken.
• nbc/1.2.1.r4+dfsg-3 by Petter Reinholdtsen, original patch by Chris Lamb.
• ncurses/6.0+20160910-1 by Sven Joachim, #818067 by Niels Thykier.
• python-kinterbasdb/3.3.0-4 by Santiago Vila, original patch by Chris Lamb.
• snapper/0.3.3-1 Hideki Yamane, original patch by Sascha Steinbiss.

Patches submitted that have not made their way to the archive yet:

• #838188 filed against ocaml by Johannes Schauer.

Reviews of unreproducible packages 462 package reviews have been added, 524 have been updated and 166 have been removed in this week, adding to our knowledge about identified issues. 25 issue types have been updated:

• Added a new annotation for issues called "fix-deterministic" to help us update package reviews more easily. This indicates whether we expect that an issue would always happen on Jenkins; i.e. if there is a successful build, then we know the issue is fixed for that package and can update our notes.
• Added random_order_in_sisu_javax_inject_named and too_much_input_for_diff.
• Removed timestamps_in_manpages_generated_by_ronn.
• Updated timestamps_in_allegro_dat_files. Additionally, 21 issues were marked with "fix-deterministic".

Weekly QA work FTBFS bugs have been reported by:

• Chris Lamb (10)
• Filip Pytloun (1)
• Santiago Vila (1)

diffoscope development A new version of diffoscope 60 was uploaded to unstable by Mattia Rizzolo. It included contributions from:

• Mattia Rizzolo:
  • Various packaging and testing improvements.
• HW42:
  • minor wording fixes
• Reiner Herrmann:
  • minor wording fixes

It also included from changes previous weeks; see either the changes or commits linked above, or previous blog posts 72 71 70. strip-nondeterminism development New versions of strip-nondeterminism 0.027-1 and 0.028-1 were uploaded to unstable by Chris Lamb. It included contributions from:

• Chris Lamb:
  • Testing improvements, including better handling of timezones.

disorderfs development A new version of disorderfs 0.5.1 was uploaded to unstable by Chris Lamb. It included contributions from:

• Andrew Ayer and Chris Lamb:
  • Support relative paths for ROOTDIR; it no longer needs to be an absolute path.
• Chris Lamb:
  • Print the behaviour (shuffle/reverse/sort) on startup to stdout.

It also included from changes previous weeks; see either the changes or commits linked above, or previous blog posts 70. Misc. This week's edition was written by Ximin Luo and reviewed by a bunch of Reproducible Builds folks on IRC.

What happened in the Reproducible Builds effort between Sunday August 21 and Saturday August 27 2016: GSoC and Outreachy updates

• Satyam Zode's final report for GSOC 2016

Packages reviewed and fixed, and bugs filed

• #834976 filed against auto-apt-proxy by Chris Lamb.
• #835637 filed against myghty by Chris Lamb.
• #835061 filed against varnish by Chris Lamb.
• #835633 filed against pleiades by Chris Lamb.
• #835625 filed against nikwi by Chris Lamb.
• #835263 filed against binutils-m68hc1x by Chris Lamb.
• #835448 filed against eekboek by Chris Lamb.
• #835143 filed against ttf-tiresias by Chris Lamb.
• #835637 filed against myghty by Chris Lamb.
• #835495 filed against broccoli by Chris Lamb.
• #835129 filed against dateutils by Chris Lamb.
• #835051 filed against sheepdog by Chris Lamb.
• #835145 filed against udpcast by Chris Lamb.
• #834983 filed against eyed3 by Chris Lamb.
• #835617 filed against congress by Chris Lamb.
• #835376 filed against lilyterm by Chris Lamb.
• #835130 filed against ircd-ircu by Chris Lamb.
• #835262 filed against radare2 by Chris Lamb.
• #835193 filed against phpdox by Chris Lamb.
• #835265 filed against argyll by Chris Lamb.
• #835259 filed against quvi by Chris Lamb.
• #835371 filed against dispcalgui by Chris Lamb.
• #835147 filed against javatools by Chris Lamb.
• #834988 filed against twitter-bootstrap3 by Chris Lamb.
• #835463 filed against fdroidserver by Chris Lamb.
• #835646 filed against dh-lua by Chris Lamb.
• #835447 filed against libmodule-build-withxspp-perl by Chris Lamb.
• #835418 filed against libfm by Chris Lamb.
• #835143 filed against ttf-tiresias by Chris Lamb.
• #834993 filed against oss4 by Reiner Herrmann.

Reviews of unreproducible packages 10 package reviews have been added and 6 have been updated this week, adding to our knowledge about identified issues. A large number of issue types have been updated:

• Add captures_build_path issue and some packages affected by it
• Add golang_compiler_captures_build_path_in_binary and move obfs4proxy to it
• Move 7 golang packages from captures_build_path to golang_compiler_captures_build_path_in_binary
• Add new:
  • random_order_in_lua_version_substvar
  • href_links_mangled_by_node_marked
  • perl_extutils_xspp_captures_build_path
  • timestamp_added_by_java_util_properties
• Fixes for issues created:
  • perl_extutils_xspp_captures_build_path
  • random_order_in_lua_version_substvar
  • random_order_in_javahelper_substvars
• Patch for zope_random_field_order_in_dzproduct uploaded.
• Rename:
  • use_epydoc randomness_in_documentation_generated_by_epydoc so it at least matches the others
  • random_order_in_javahelper_depends issue to random_order_in_javahelper_substvars
• Fix the link for the golang issue, previous link is for random_build_path_by_golang_compiler a different issue
• Add a tip regarding how to call ./configure for rust
• Add offending source line for gcc-defaults.

Weekly QA work 29 FTBFS bugs have been reported by:

• Chris Lamb (27)
• Daniel Stender (1)
• Santiago Vila (1)

diffoscope development

• Chris Lamb:
  • Add tests for skip_unless_tool_exists helper.
  • comparators/elf.py: Specify string format arguments as logging function parameters, not using interpolation.
  • presenters/html:
    • Use html.escape over xml.sax.saxutils.escape.
    • Don't use unsafe cgi.escape method as its quote kwarg -- which enables escaping of quotation marks -- is False by default.
  • Tidy imports in Debian comparator and tests.
  • Skip Haskell tests if GHC version does not match. (Closes: #835055)
  • Use pytest.xfail over assert False.
  • Use the debian_fallback.X as the fallback for debian.DotBuildinfoFile (not debian.X).
  • Rename skip_unless_tool_exists -> skip_unless_tools_exist and fix logic.
  • Avoid ugly DRY violations in diffoscope.comparators.__init__ by dynamically importing classes via a single list.
• Satyam Zode:
  • Improve diffoscope behaviour for .changes
• Mattia Rizzolo:
  • Be even more verbose about failing tests
  • d/control: alternate build-dependency on default-jdk-headless default-jdk, to ease backporting to older debian releases
  • add default-jdk to the alternate packages for javap for Debian; default-jdk-headless is not available in older Debian releases
  • in the tests only, normalize xxd's output so that we can compare jessie's xxd with stretch's
  • tests:
    • skip test_squashfs.py.test_superblock if squashfs is too old
    • rewrite tool_older_than() into skip_unless_tool_is_older_than()
    • factor out a tools_missing() function
• J r my Bobbio:
  • Properly skip test requiring python-debian when unavailable
• Ximin Luo:
  • Add a --no-max flag to disable all limits and have max_report_size also honour 0 to mean "no limit"
  • Actually only scan whole file when filename ends in ".rom"
  • Show the timestamp when logging so I know which steps take longer

Holger also created another test job for diffoscope on jenkins.debian.net, so that now also all commits to branches other than master are being tested. strip-nondeterminism development strip-nondeterminism 0.023-1 was uploaded by Chris Lamb:

 * Support Android .apk files with the JAR normalizer.
 * handlers/png.pm: Drop unused Archive::Zip import
 * Remove hyphen from non-determinism and non-deterministic.
 * javaproperties.pm: Match more styles of .properties and loosen filename matching.
 * Improve tests:
   - Make fixture runner generic to all normalizer types.
   - Replace (single) pearregistry test with a fixture.
   - Set a canonical time for fixture tests.
   - Add gzip testcase fixture.
   - Replace t/javadoc.t with fixture
   - Replace t/ar.t with a fixture.
   - t/javaproperties: move pom.properties and version.properties tests to fixtures
   - t/fixtures.t: move to using subtests
   - t/fixtures.t: Explicitly test that we can find a normalizer
   - t/fixtures.t: Don't run normalizer if we didn't find one.

strip-nondeterminism 0.023-2 uploaded by Mattia Rizzolo to allow stderr in autopkgtest. disorderfs development

• Chris Lamb:
  • Add --sort-dirents=yes no option for forcing deterministic.
  • Testsuite:
    • Add tests for sorting and reversing directory entries.
    • shuffle: Test that --shuffle-dirents works as directed.
    • common: Factor out utility to get variations from mount target.
    • common: Factor out "Fail" utility.

tests.reproducible-builds.org Debian:

• Since we introduced build path variations for unstable and experimental last week, our IRC channel has been flooded with notifcations about packages becoming unreproducible - and you might have noticed some of your packages having become unreproducible recently too. To make our IRC more bearable again, notifications for status changes on i386 and armhf have been disabled, so that now we only get notifications for status changes in unstable. (h01ger)
• Link to jenkins documentation in every page (h01ger)
• The "pre build" check, whether a node is up, now also detects if a node has a read-only filesystem, which sometimes happens on some broken armhf nodes. (h01ger)
• To further improve monitoring of those armhf nodes Work to make them send mails (through an ISP which is blocking outgoing mails) has been started and should be finished next week. (h01ger)
• As one of the armhf nodes (opi2a) is acting strange, a workaround has been added to make it's deployment work despite that. (h01ger)
• Collapse whitespace to avoid ugly "trailing underlines" in hyperlinks for diffoscope results and pkg sets (Chris Lamb)
• Give details HTML elements "cursor: pointer" CSS property to highlight they are clickable. (Chris Lamb)
• The db connection timeout has been raised to a minute when using SQLAlchemy too. (h01ger).

Somewhat related to reproducible builds there has been a first Debian jenkins team maintainance meeting on the #debian-qa IRC channel, to discuss current issues with the setup and to start the work of migrating jenkins.debian.net to jenkins.debian.org. The next meeting will take place on September 28th 2016 at 19 UTC. Misc. This week's edition was written by Chris Lamb and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

What happened in the Reproducible Builds effort between Sunday August 14 and Saturday August 20 2016: Fasten your seatbelts Important note: we enabled build path variation for unstable now, so your package(s) might become unreproducible, while previously it was said to be reproducible given a specific build path it probably still is reproducible but read on for the details below in the tests.reproducible-builds.org section! As said many times: this is still research and we are working to make it reality. Media coverage Daniel Stender blogged about python packaging and explained some caveats regarding reproducible builds. Toolchain developments Thomas Schmitt uploaded xorriso which now obeys SOURCE_DATE_EPOCH. As stated in its man pages:

ENVIRONMENT
[...]
SOURCE_DATE_EPOCH  belongs to the specs of reproducible-builds.org.  It
is supposed to be either undefined or to contain a decimal number which
tells the seconds since january 1st 1970. If it contains a number, then
it is used as time value to set the  default  of  --modification-date=,
--gpt_disk_guid,  and  --set_all_file_dates.  Startup files and program
options can override the effect of SOURCE_DATE_EPOCH.

Packages reviewed and fixed, and bugs filed The following packages have become reproducible after being fixed:

• cadencii/3.3.9+svn20110818.r1732-5 by Ying-Chun Liu, original patch by Chris Lamb.
• cfengine3/3.7.4-3 by Antonio Radici.
• findbugs/3.1.0~preview2-1 by Emmanuel Bourg.
• gcpegg/5.1-14 by Bdale Garbee, original patch by Chris Lamb.
• gnunet-gtk/0.10.1-4 by Bertrand Marc, original patch by Chris Lamb.
• konwert/1.8-12 by Yann Dirson, original patch by Chris Lamb.
• link-grammar/5.3.8-2 by Jeremy Bicha, original patch by Chris Lamb.
• metastudent-data/2.0.1-1 by Sascha Steinbiss.
• pixmap/2.6pl4-20 by Paul Slootman, original patch by Chris Lamb Maria Valentina Marin a.k.a. Akira.
• python-afl/0.5.4-2 by Daniel Stender.
• tf5/5.0beta8-6 by Russ Allbery, original patch by Chris Lamb.
• triplane/1.0.8-2 by Markus Koschanyover, original patch by Chris Lamb.
• vips/8.3.3-1 by Laszlo Boszormenyi, original patch by Chris Lamb.
• xzgv/0.9.1-4 by Theodore Y. Ts'o, original patch by Chris Lamb.

The following updated packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

• vulkan/1.0.21.0+dfsg1-1 by Timo Aaltonen.

The following 2 packages were not changed, but have become reproducible due to changes in their build-dependencies: tagsoup tclx8.4. Some uploads have addressed some reproducibility issues, but not all of them:

• cappuccino/0.5.1-4 by Breno Leitao, original patch by Chris Lamb.
• darkice/1.3-0.1 by Nicolas Bouleng.
• flask-restful/0.3.5-1 by Ond ej Nov , original patch by Chris Lamb.
• kodi/16.1+dfsg1-2 by Balint Reczey, original patch by Lukas Rechberger.
• libaws/3.3.2-3 by Nicolas Bouleng.
• liblog4ada/1.3-1 by Nicolas Bouleng.
• libxmlezout/1.06.1-8 by Nicolas Bouleng.
• ruby-rmagick/2.15.4+dfsg-2 by Antonio Terceir, original patch from Chris Lamb.
• taggrepper/0.05-2 by Kumar Appaiah, original patch and original patch from Maria Valentina Marin.
• xotcl/1.6.8-2 by Stefan Sobernig, original patch by Chris Lamb.

Patches submitted that have not made their way to the archive yet:

• #834755 against apt-cacher-ng by Chris Lamb.
• #834976 against auto-apt-proxy by Chris Lamb.
• #834549 against botan1.10 by Chris Lamb.
• #834861 against cookiecutter by Chris Lamb.
• #834779 against dicom3tools by Chris Lamb.
• #834859 against echoping by Chris Lamb.
• #834983 against eyed3 by Chris Lamb.
• #834735 against filepp by Chris Lamb.
• #834957 against flashrom by Chris Lamb.
• #834773 against gkrellm by Chris Lamb.
• #834292 against gr-radar by Chris Lamb.
• #834956 against ircd-irc2 by Chris Lamb.
• #834897 against jpy by Chris Lamb.
• #834452 against libdc0 by Chris Lamb.
• #834324 against libnss-ldap by Reiner Herrmann.
• #834945 against libquvi by Chris Lamb.
• #834534 against librep by Chris Lamb.
• #834537 against mozvoikko by Chris Lamb.
• #834857 against nagios-nrpe by Chris Lamb.
• #834316 against openocd by Reiner Herrmann.
• #834993 against oss4 by Reiner Herrmann.
• #834302 against pd-flite by Reiner Herrmann.
• #834754 against phpab by Chris Lamb.
• #834279 against phpldapadmin by Chris Lamb.
• #834277 against pybit by Chris Lamb.
• #834553 against pyicu by Chris Lamb.
• #834541 against ruby-pg by Chris Lamb.
• #835051 against sheepdog by Chris Lamb.
• #834896 against ttf-tiresias by Chris Lamb.
• #834780 against tuxpaint-config by Chris Lamb.
• #834988 against twitter-bootstrap3 by Chris Lamb.
• #834776 against uhub by Chris Lamb.
• #835061 against varnish by Chris Lamb.

Bug tracker house keeping:

• Chris Lamb pinged 164 bugs he filed more than 90 days ago which have a patch and had no maintainer reaction.

Reviews of unreproducible packages 55 package reviews have been added, 161 have been updated and 136 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been updated:

• Added user_in_documentation_generated_by_gsdoc
• Added locale_differences_in_pom

Weekly QA work FTBFS bugs have been reported by:

• Chris Lamb (16)
• Santiago Vila (2)

diffoscope development Chris Lamb, Holger Levsen and Mattia Rizzolo worked on diffoscope this week. Improvements were made to SquashFS and JSON comparison, the https://try.diffoscope.org/ web service, documentation, packaging, and general code quality. diffoscope 57, 58, and 59 were uploaded to unstable by Chris Lamb. Versions 57 and 58 were both broken, so Holger set up a job on jenkins.debian.net to test diffoscope on each git commit. He also wrote a CONTRIBUTING document to help prevent this from happening in future. From these efforts, we were also able to learn that diffoscope is now reproducible even when built across multiple architectures:

< h01ger>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope.html shows these packages were built on amd64:
< h01ger>    bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger>    366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger>   and on i386:
< h01ger>    bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger>    366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger>   and on armhf:
< h01ger>    bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger>    366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb

And those also match the binaries uploaded by Chris in his diffoscope 59 binary upload to ftp.debian.org, yay! Eating our own dogfood and enjoying it! tests.reproducible-builds.org Debian related:

• show percentage of results in the last 24/48h (h01ger)
• switch python database backend to SQLAlchemy (Valerie)
• vary build path varitation for unstable and experimental for all architectures. (h01ger)

The last change probably will have an impact you will see: your package might become unreproducible in unstable and this will be shown on tracker.debian.org, while it will still be reproducible in testing. We've done this, because we think reproducible builds are possible with arbitrary build paths. But: we don't think those are a realistic goal for stretch, where we still recommend to use .buildinfo to record the build patch and then do rebuilds using that path. We are doing this, because besides doing theoretical groundwork we also have a practical goal: enable users to independently verify builds. And if they only can do this with a fixed path, so be it. For now To be clear: for Stretch we recommend that reproducible builds are done in the same build path as the "original" build. Finally, and just for our future references, when we enabled build path variation on Saturday, August 20th 2016, the numbers for unstable were:

suite	all	reproducible	unreproducible	ftbfs	depwait	not for this arch	blacklisted
unstable/amd64	24693	21794 (88.2%)	1753 (7.1%)	972 (3.9%)	65 (0.2%)	95 (0.3%)	10 (0.0%)
unstable/i386	24693	21182 (85.7%)	2349 (9.5%)	972 (3.9%)	76 (0.3%)	103 (0.4%)	10 (0.0%)
unstable/armhf	24693	20889 (84.6%)	2050 (8.3%)	1126 (4.5%)	199 (0.8%)	296 (1.1%)	129 (0.5%)

Misc. Ximin Luo updated our git setup scripts to make it easier for people to write proper descriptions for our repositories. This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

What happened in the Reproducible Builds effort between June 26th and July 2nd 2016: Read on to find out why we're lagging some weeks behind ! GSoC and Outreachy updates

• Ceridwen described using autopkgtest code to communicate with containers and how to test the container handling.
• reprotest 0.1 has been accepted into Debian unstable, and any user reports, bug reports, feature requests, etc. would be appreciated. This is still an alpha release, and nothing is set in stone.

Toolchain fixes

• Matthias Klose uploaded doxygen/1.8.11-3 to Debian unstable (closing #792201) with the upstream patch improving SOURCE_DATE_EPOCH support by using UTC as timezone when parsing the value. This was the last patch we were carrying in our repository, thus this upload obsoletes the version in our experimental repository.
• cmake/3.5.2-2 was uploaded by Felix Geyer, which sorts file lists obtained with file(GLOB).
• Dmitry Shachnev uploaded sphinx/1.4.4-2, which fixes a timezone related issue when SOURCE_DATE_EPOCH is set.

With the doxygen upload we are now down to only 2 modified packages in our repository: dpkg and rdfind. Weekly reports delay and the future of statistics To catch up with our backlog of weekly reports we have decided to skip some of the statistics for this week. We might publish them in a future report, or we might switch to a format where we summarize them more (and which we can create (even) more automatically), we'll see. We are doing these weekly statistics because we believe it's appropriate and useful to credit people's work and make it more visible. What do you think? We would love to hear your thoughts on this matter! Do you read these statistics? Somewhat? Actually, thanks to the power of notmuch, Holger came up with what you can see below, so what's missing for this week are the uploads fixing irreprodubilities. Which we really would like to show for the reasons stated above and because we really really need these uploads to happen But then we also like to confirm the bugs are really gone, which (atm) requires manual checking, and to look for the words "reproducible" and "deterministic" (and spelling variations) in debian/changelogs of all uploads, to spot reproducible work not tracked via the BTS. And we still need to catch up on the backlog of weekly reports. Bugs submitted with reproducible usertags It seems DebCamp in Cape Town was hugely successful and made some people get a lot of work done: 61 bugs have been filed with reproducible builds usertags and 60 of them had patches:

• #829365 against pdl by Reiner Herrmann: please make the build reproducible.
• #829362 against ruby-ronn by Chris Lamb: please make the output reproducible.
• #829357 against ctop by Chris Lamb: please make the build reproducible.
• #829325 against txt2tags by Chris Lamb: please make the output reproducible.
• #829323 against pdl by Reiner Herrmann: please generated sorted output.
• #829322 against libpdl-netcdf-perl by Reiner Herrmann: please make the build reproducible.
• #829320 against libpdl-io-hdf5-perl by Reiner Herrmann: please make the build reproducible.
• #829297 against check-all-the-things by Chris Lamb: please make the build reproducible.
• #829296 against perl by Chris Lamb: please make the output of ParseXS.pm reproducible.
• #829295 against libextutils-parsexs-perl by Chris Lamb: please make the output reproducible.
• #829270 against tomsfastmath by Reiner Herrmann: please make the build reproducible.
• #829263 against libmemcached-libmemcached-perl by Chris Lamb: please make the build reproducible.
• #829262 against slashem by Reiner Herrmann: please make the build reproducible.
• #829261 against passwordsafe by Reiner Herrmann: please make the build reproducible.
• #829249 against ncftp by Reiner Herrmann: please make the build reproducible.
• #829133 against icon by Reiner Herrmann: please make the build reproducible.
• #829129 against ayttm by Reiner Herrmann: please make the build reproducible.
• #829011 against link-grammar by Jeremy Bicha and Chris Lamb: please make the build reproducible.
• #829000 against fracplanet by Reiner Herrmann: please make the build reproducible.
• #828994 against syncthing by Dhole: please make the build reproducible.
• #828993 against openttd by Reiner Herrmann: please make the build reproducible.
• #828989 against ntp by Reiner Herrmann: please make the build reproducible.
• #828977 against faust by Reiner Herrmann: please make the build reproducible.
• #828971 against clasp by Reiner Herrmann: please make the build reproducible.
• #828969 against dogecoin by Reiner Herrmann: please make the build reproducible.
• #828909 against qtruby by Reiner Herrmann: please make the build reproducible.
• #828906 against mailfilter by Reiner Herrmann: please make the build reproducible.
• #828898 against lordsawar by Reiner Herrmann: please make the build reproducible.
• #828891 against bbdb by Reiner Herrmann: please make the build reproducible.
• #828890 against libsdl2-gfx by Reiner Herrmann: please make the build reproducible.
• #828888 against aspell-en by Reiner Herrmann: please make the build reproducible.
• #828876 against ario by Reiner Herrmann: please make the build reproducible.
• #828867 against zephyr by Reiner Herrmann: please make the build reproducible.
• #828856 against lrzsz by Reiner Herrmann: please make the build reproducible.
• #828855 against wmweather by Reiner Herrmann: please make the build reproducible.
• #828852 against moblin-icon-theme by Reiner Herrmann: please make the build reproducible.
• #828810 against fakeroot by Juan Picca: [PATCH] Make the build reproducible.
• #828793 against minicom by Reiner Herrmann: please make the build reproducible.
• #828791 against fio by Reiner Herrmann: please make the build reproducible.
• #828790 against debiandoc-sgml-doc by Dhole: please make the build reproducible.
• #828788 against pyparted by Reiner Herrmann: please make the build reproducible.
• #828786 against tcptraceroute by Reiner Herrmann: please make the build reproducible.
• #828780 against iptraf by Reiner Herrmann: please make the build reproducible.
• #828778 against module-assistant by Reiner Herrmann: please make the build reproducible.
• #828766 against liblucy-perl by Reiner Herrmann: please make the build reproducible.
• #828762 against slang2 by Reiner Herrmann: please make the build reproducible.
• #828756 against python-reportlab by Reiner Herrmann: please make the build reproducible.
• #828752 against mod-dnssd by Reiner Herrmann: please make the build reproducible.
• #828749 against telepathy-salut by Reiner Herrmann: please make the build reproducible.
• #828748 against libphonenumber by Reiner Herrmann: please make the build reproducible.
• #828745 against directfb by Reiner Herrmann: please make the build reproducible.
• #828683 against mc by Reiner Herrmann and Yury V. Zaytsev: please make the build reproducible.
• #828681 against keyutils by Reiner Herrmann: please make the build reproducible.
• #828680 against debiandoc-sgml-doc-pt-br by Dhole: please make the build reproducible.
• #828639 against libmarpa-r2-perl by Reiner Herrmann: please make the build reproducible.
• #828636 against libembperl-perl by Axel Beckert, Reiner Herrmann and gregor herrmann: please make the build reproducible.
• #828635 against libnet-tclink-perl by Reiner Herrmann: please make the build reproducible.
• #828628 against media-player-info by Reiner Herrmann: please make the build reproducible.
• #828627 against uim by Alexis Bienven e: please make the build reproducible (locale).
• #828226 against tiger by Alexis Bienven e: please make the build reproducible (environment, locale).
• #828222 against pygoocanvas by Chris Lamb: please make the build reproducible.
• #828216 against pyinfra by Daniel Stender: assertion error with timestamps.

Package reviews 437 new reviews have been added (though most of them were just linking the bug, "only" 56 new issues in packages were found), an unknown number has been been updated and 60 have been removed in this week, adding to our knowledge about identified issues. 4 new issue types have been found:

• random_order_of_pdf_ids_generated_by_latex
• unsorted_pdl_output
• timestamps_in_output_generated_by_txt2tags
• random_order_in_documentation_generated_by_naturaldocs

Weekly QA work 98 FTBFS bugs have been reported by Chris Lamb and Santiago Vila. diffoscope development

• Reiner Herrmann clarified the help text for the input arguments.
• #829115 against diffoscope by Axel Beckert and Mattia Rizzolo:: /comparators/ps.py: TypeError: cannot use a string pattern on a bytes-like object.

strip-nondeterminism development

• Chris Lamb made sure that .zhfst files are treated as ZIP files.

tests.reproducible-builds.org

• Mattia Rizzolo uploaded pbuilder/0.225.1~bpo8+1 to jessie-backports and it has been installed on all build nodes. As a consequence all armhf and i386 builds will be done with eatmydata; this will hopefully cut down the build time by a noticable factor.

Misc. This week's edition was written by Mattia Rizzolo, Reiner Herrmann, Ceridwen and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

What happened in the Reproducible Builds effort between June 19th and June 25th 2016. Media coverage

• Holger Levsen gave a talk at openSUSE Conference 2016 explaining the general idea and status of Reproducible Builds. This talk is available as video recording.
• This was followed by Bernhard Wiedemannn, detailing his work on Reproducible Builds for openSUSE which is also available as video recording:
  • openSUSE uses SOURCE_DATE_EPOCH now too
  • How to create bit-for-bit identical RPMs
  • How strip-nondeterminism is Python and thus unsuitable for the openSUSE base system
• Mozilla awarded $77k to work on reproducible builds for Tails. The goal is to enable anyone (given sufficient technical skills and hardware resources) to rebuild from source a given Tails release, in order to independently verify that it matches the ISO image that was published. A substantial part of this work will be done in Debian: for example, to make the side-effects of some packages' post-installation scripts deterministic. On the longer term, this work should benefit other projects that want to make their own builds reproducible (e.g. operating system images for the cloud and embedded systems, operating system installation media, other Live systems).

GSoC and Outreachy updates

• Valerie Young published a report detailing what she did the last week regarding improving tests.reproducible-builds.org/debian.
• Ceridwen announced reprotest's arrival in the NEW queue and discussed autopkgtest's layout.

Toolchain fixes

• Anton Gladky uploaded an NMU of python-setuptools, which sorts entries in native_libs.txt files (#825857).

Other upstream fixes Emil Velikov searched on IRC for hints on how to guarantee unique values during build to invalidate shader caches in Mesa, when also no VCS information is available. A possible solution is a timestamp, which is unique enough for local builds, but can still be reproducible by allowing it to be overwritten with SOURCE_DATE_EPOCH. Packages fixed The following 9 packages have become reproducible due to changes in their build dependencies: cclib librun-parts-perl llvm-toolchain-snapshot python-crypto python-openid r-bioc-shortread r-bioc-variantannotation ruby-hdfeos5 sqlparse The following packages have become reproducible after being fixed:

• allegro4.4/2:4.4.2-9 by Tobias Hansen, original patch by Reiner Herrmann.
• atomicparsley/0.9.6-1 by Jonas Smedegaard.
• dwarfutils/20160613-1 by Fabian Wolff, original patch by Reiner Herrmann.
• dwm/6.1-3 by Hugo Lefeuvre, original patch by Reiner Herrmann.
• funtools/1.4.6+git150811-3 by Ole Streicher, original patch by Alexis Bienven e.
• golang-github-appc-spec/0.8.4+dfsg-1 by Dmitry Smirnov.
• golang-github-appc-docker2aci/0.11.1+dfsg-1 Dmitry Smirnov.
• hdf-eos5/5.1.15.dfsg.1-7 by Alastair McKinstry.
• hmmer2/2.3.2-11 by Sascha Steinbiss, original patch by Chris Lamb.
• jpy/0.8-2 by Alastair McKinstry.
• lazarus/1.6+dfsg-4 by Paul Gevers.
• pgbackrest/1.02-2 by Adrian Vondendriesch.
• siege/4.0.2-1 by Josue Abarca.
• starlink-pal/0.5.0-4 by Ole Streicher, original patch by Chris Lamb.
• stylish-haskell/0.5.17.0-2 by Sean Whitton.
• xprobe/0.3-3 by Sophie Brun, original patch by Reiner Herrmann.

Some uploads have fixed some reproducibility issues, but not all of them:

• hdfeos4/2.19v1.00+dfsg.1-5 by Alastair McKinstry.
• icu4j/57.1-2 by Tony Mancill, original patch by Chris Lamb.
• pari/2.7.6-1 by Bill Allombert,

Patches submitted that have not made their way to the archive yet:

• #827684 against cgoban by Chris Lamb: set SHELL to static value.
• #827731 against tin by Alexis Bienven e: drop patch which overwrites __DATE__/__TIME__ macros, since gcc can handle it now
• #827863 against swedish by Alexis Bienven e: use C locale for sorting.
• #827987 against glances by Chris Lamb: Use SOURCE_DATE_EPOCH for embedded timestamp.
• #827994 against cmtk by Chris Lamb: use C locale for sorting.
• #828008 against aghermann by Chris Lamb: honour SOURCE_DATE_EPOCH for timestamps embedded into manpages.
• #828012 against bind9 by Chris Lamb: honour SOURCE_DATE_EPOCH for embedded timestamp.
• #828017 against frog by Chris Lamb: don't include pyc/pyo files in the package.
• #828021 against extra-cmake-modules by Scarlett Clark: normalize permission and file order in tarballs.
• #828060 against libffado by Chris Lamb: exclude file with test output from package.
• #828066 against gsmlib by Chris Lamb: honour SOURCE_DATE_EPOCH for timestamps embedded into manpages.
• #828067 against grib-api by Chris Lamb: exclude pyc files from package.
• #828122 against libxmlbird by Chris Lamb: sort list of globbed files.
• #828123 against magnum by Chris Lamb: use static value for embedded hostname.
• #828131 against pyjwt by Chris Lamb: exclude coverage data from package.
• #828145 against mkdocs by Chris Lamb: honour SOURCE_DATE_EPOCH for embedded timestamp.
• #828164 against zeal by Chris Lamb: use UTC for embedded timestamp.
• #828168 against x42-plugins by Daniel Shahaf: use printf instead of non-portable echo.

Package reviews 139 reviews have been added, 20 have been updated and 21 have been removed in this week. New issues found:

• timestamps_in_pdf_generated_by_reportlab
• r_base_appends_built_header_to_description_files
• timestamps_in_documentation_generated_by_mkdocs

53 FTBFS bugs have been reported by Chris Lamb, Santiago Vila and Mateusz ukasik. diffoscope development

• Satyam Zode added argument completion.
• Chris Lamb made the confusing "No differences found inside, yet data differs" message less confusing.

Quote of the week "My builds are so reproducible, they fail exactly every second time." Johannes Ziemke (@discordianfish) Misc. This week's edition was written by Chris Lamb (lamby), Reiner Herrmann and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

What happened in the Reproducible Builds effort between June 12th and June 18th 2016: Media coverage

• Sune Vuorela blogged about making Qt documentation reproducible while spending good times in the Alps.

GSoC and Outreachy updates Weekly reports by our participants:

• Scarlett Clark
• Satyam Zode

Toolchain fixes

• texlive-bin/2016.20160513.41080-3 has been uploaded to unstable, featuring support for FORCE_SOURCE_DATE. See the last post for details on it.
• doxygen/1.4.4-1 has been uploaded to unstable, fixing #822197 (upstream bug), which caused some generated html file to contain unreproducible memory addresses of Python objects used at build time.
• debhelper/9.20160618 has been uploaded to unstable, fixing #824490, which instructs ant to not save the username of the build user in the generated files. Original patch by Emmanuel Bourg.
• HW42 reported a long-known (although only internally) bug in our dpkg-buildinfo. this particular bug doesn't affect our current infrastructure, but it's a blocker for having .buildinfo support merged upstream.
• epydoc/3.0.1+dfsg-13 and 3.0.1+dfsg-14 have been uploaded by Kenneth J. Pronovici which fixes nondeterministic ordering issues in generated documentation and removes memory addresses. Original patches (#825968 and #827416) by Sascha Steinbiss.

With this upload of texlive-bin we decided to stop keeping our patched fork of as most of the patches for SOURCE_DATE_EPOCH support had been integrated upstream already, and the last one (making FORCE_SOURCE_DATE default to 1) had been refused. So, we are now going to let the archive be rebuilt against unstable's texlive-bin and see how many packages will become unreproducible with this change; once enough data will be collected we will ponder whether FORCE_SOURCE_DATE should be exported by helper tools (such as debhelper) or manually exported by every package that needs it. (For those wondering: we still recommend to follow SOURCE_DATE_EPOCH always and don't recommend other projects to implement FORCE_SOURCE_DATE ) With the drop of texlive-bin we now have only three modified packages in our experimental repository. Reproducible work in other projects

• Ed Maste sent a patch to have ELF Tool Chain's elfcopy support SOURCE_DATE_EPOCH.
• Ed Maste changed FreeBSD's ar(1) to have a reproducible output by default when invoked with -s.
• Ed Maste merged changes from NetBSD's makefs to FreeBSD's to add a command line option for setting the timestamp.
• Ed Maste reported on FreeBSD package reproducibility details from the investigation for his BSDCan talk, including diffoscope results for the non-reproducible packages.
• Holger Levsen spent some time thinking and documenting how to store package notes and issues in a multi-distribution friendly manner, so that we can share these issues and notes between reproducible efforts across different projects. Thoughts we had about this topic during the Athens meeting last December are included in the updated README.

Packages fixed The following 12 packages have become reproducible due to changes in their build dependencies: django-floppyforms flask-restful hy jets3t kombu llvm-toolchain-3.8 moap python-bottle python-debtcollector python-django-debug-toolbar python-osprofiler stevedore The following packages have become reproducible after being fixed:

• adios/1.9.0-10 by Alastair McKinstry.
• airstrike/0.99+1.0pre6a-8 by Markus Koschany, original patch by Reiner Herrmann.
• apt-dater/1.0.3-1 by Patrick Matth i, original patch by Chris Lamb.
• bitlbee/3.4.2-1 by Jelmer Vernoo .
• dar/2.5.5-1 by Laszlo Boszormenyi.
• fastjet/3.0.6+dfsg-2 by Mattia Rizzolo, original patch by Maria Valentina Marin.
• libretro-beetle-pce-fast/0.9.38.7+git20160609-1 by S rgio Benjamim, original patch by Reiner Herrmann.
• libretro-beetle-psx/0.9.38.6+git20151019-2 by S rgio Benjamim, original patch by Reiner Herrmann.
• mx/1.99.4-1 by Ying-Chun Liu.
• python-coverage/4.1+dfsg.1-1 by Ben Finney.
• shiro/1.2.5-1 by Tony Mancill, original patch by Chris Lamb.
• splix/2.0.0+svn315-5 by Didier Raboud.
• u-boot/2016.07~rc1+dfsg1-3 by Vagrant Cascadian, debugging and patch by HW42, patches submitted upstream.

Some uploads have fixed some reproducibility issues, but not all of them:

• gcc-mingw-w64/18 by Stephen Kitt.
• gnuplot/5.0.3+dfsg3-6 by Anton Gladky, original patch by Alexis Bienven e.

Uploads with reproducibility fixes that currently fail to build:

• ruby2.3/2.3.1-3 by Christian Hofstaedtler, avoids unreproducible rbconfig.rb files by always using bash for building.

Patches submitted that have not made their way to the archive yet:

• #827109 against asciijump by Reiner Herrmann: sort source files for deterministic linking order.
• #827112 against boswars by Reiner Herrmann: sort source files for deterministic linking order.
• #827114 against overgod by Reiner Herrmann: use C locale for sorting source files.
• #827115 against netpbm by Alexis Bienven e: honour SOURCE_DATE_EPOCH while generating output.
• #827124 against funguloids by Reiner Herrmann: use C locale for sorting files.
• #827145 against scummvm by Reiner Herrmann: don't embed extra fields in zip archive and build with proper host architecture.
• #827150 against netpanzer by Reiner Herrmann: sort source files for deterministic linking order.
• #827172 against reaver by Alexis Bienven e: sort object files for deterministic linking order.
• #827187 against latex2html by Alexis Bienven e: iterate deterministic over Perl hashes; honour SOURCE_DATE_EPOCH for output; strip username from output; sort index keys.
• #827313 against cherrypy3 by Sascha Steinbiss: prevent memory addresses in output.
• #827361 against matplotlib by Alexis Bienven e: honour SOURCE_DATE_EPOCH in output and sort keys while iterating over dict.
• #827382 against dwarfutils by Reiner Herrmann: fix array size, which caused memory from outside a table to be embedded into output.
• #827384 against skytools3 by Sascha Steinbiss: use stable sorting order and remove timestamps from documentation.
• #827419 against ldaptor by Sascha Steinbiss: sort list of input files and prevent home directory from leaking into documentation.
• #827546 against git-buildpackage by Sascha Steinbiss: replace timestamps in documentation with changelog date; prevent temporary paths in documentation.
• #827572 against xprobe by Reiner Herrmann: sort list of object files in static library archives.

Package reviews 36 reviews have been added, 12 have been updated and 31 have been removed in this week. 17 FTBFS bugs have been reported by Chris Lamb, Santiago Vila and Dominic Hargreaves. diffoscope development Satyam worked on argument completion (#826711) for diffoscope. strip-nondeterminism development Mattia Rizzolo uploaded strip-nondeterminism 0.019-1~bpo8+1 to jessie-backports. reprotest development Ceridwen filed an Intent To Package (ITP) bug for reprotest as #827293. tests.reproducible-builds.org

• Mattia Rizzolo uploaded pbuilder 0.225 to unstable, providing built-in support for eatmydata. We're planning to use it in armhf and i386 builders where we don't build in tmpfs, to increase the build speed some more.
• Valery Young reworked the appearance of the package page, hopefully making them more intuitive and usable. In the process she changed the script generating them to use a real templating system, thus improving maintenance for the future.
• Holger adjusted the scheduler to reschedule packages in state 'depwait' after two days instead of three.
• Mattia added the bug title next to the bug numbers in the notes.

Misc. This week's edition was written by Mattia Rizzolo, Reiner Herrmann, Ed Maste and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

What happened in the Reproducible Builds effort between June 5th and June 11th 2016: Media coverage Ed Maste gave a talk at BSDCan 2016 on reproducible builds (slides, video). GSoC and Outreachy updates Weekly reports by our participants:

• Scarlett Clark worked on making some packages reproducible, focusing on KDE backend and utility programs.
• Ceridwen published an initial design for the interface for reprotest, including a discussion on different types of build variations and the difficulties of specifying certain types of variations.
• Valerie Young improved documentation for building our tests website, began migrating Debian-specific pages into a new namespace, and planned future work around its navigation.

Documentation update - Ximin Luo proposed a modification to our SOURCE_DATE_EPOCH spec explaining FORCE_SOURCE_DATE. Some upstream build tools (e.g. TeX, see below) have expressed a desire to control which cases of embedded timestamps should obey SOURCE_DATE_EPOCH. They were not convinced by our arguments on why this is a bad idea, so we agreed on an environment variable FORCE_SOURCE_DATE for them to implement their desired behaviour - named generically, so that at least we can set it centrally. For more details, see the text just linked. However, we strongly urge most build tools not to use this, and instead obey SOURCE_DATE_EPOCH unconditionally in all cases. Toolchain fixes

• TeX Live 2016 released with SOURCE_DATE_EPOCH support for all engines except LuaTeX and original TeX.
• Continued discussion (alternative archive) with TeX upstream, about SOURCE_DATE_EPOCH corner cases, eventually resulting in the FORCE_SOURCE_DATE proposal from above.
• gcc-5/5.4.0-4 by Matthias Klose now avoids storing -fdebug-prefix-map in DW_AT_producer, thanks to original patch by Daniel Kahn Gillmor.
• sphinx/1.4.3-1 by Dmitry Shachnev now drops Debian-specific patches relating to SOURCE_DATE_EPOCH applied upstream, original patch by Alexis Bienven e.
• asciidoctor/1.5.4-2 by C dric Boutillier now supports SOURCE_DATE_EPOCH, thanks to original patch by Alexis Bienven e.
• dh-python/1.5.4-2 by Piotr O arowski now behaves better in some cases, thanks to original patch by Chris Lamb.

Packages fixed The following 16 packages have become reproducible due to changes in their build-dependencies: apertium-dan-nor apertium-swe-nor asterisk-prompt-fr-armelle blktrace canl-c code-saturne coinor-symphony dsc-statistics frobby libphp-jpgraph paje.app proxycheck pybit spip tircd xbs The following 5 packages are new in Debian and appear to be reproducible so far: golang-github-bowery-prompt golang-github-pkg-errors golang-gopkg-dancannon-gorethink.v2 libtask-kensho-perl sspace The following packages had older versions which were reproducible, and their latest versions are now reproducible again after being fixed:

• excellent-bifurcation/0.0.20071015-8 by Vincent Cheng, original patch by Reiner Herrmann.
• gnurobbo/0.68+dfsg-2 by Stephen Kitt, original patch by Reiner Herrmann.

The following packages have become reproducible after being fixed:

• gdbm/1.8.3-14 by Matthias Klose, original patch by J r my Bobbio.
• miceamaze/4.2.1-3 by Sarah COUDERT, original patch by Reiner Herrmann.
• netcdf/1:4.4.1~rc2-1~exp3 by Bas Couwenberg.
• osmo-bts/0.4.0-2 by Ruben Undheim.
• pd-hcs/0.1-3 by IOhannes m zm lnig.
• pd-hid/0.7-2 by IOhannes m zm lnig.
• python-certbot/0.8.0-1 by Harlan Lieberman-Berg, original patch by Chris Lamb.
• python-csb/1.2.3+dfsg-3 by Sascha Steinbiss.
• python-osprofiler/1.3.0-2 by Thomas Goirand.
• usb-modeswitch-data/20160112-3 by Didier Raboud, original patch by intrigeri.

Some uploads have fixed some reproducibility issues, but not all of them:

• bzr/2.7.0-7 by Jelmer Vernoo .
• clanlib/1.0~svn3827-5 by Stephen Kitt, original patch by Chris Lamb.
• dwarfutils/20160507+git20160523.9086738-1 by Fabian Wolff.
• fakeroot/1.20.2-2 by Clint Adams, original patch.
• fastqtl/2.184+dfsg-2 by Dylan A ssi, original patch by Chris Lamb.
• gnuplot/5.0.3+dfsg3-3 by Anton Gladky.
• lazarus/1.6+dfsg-3 by Paul Gevers.
• python-pygit2/0.24.0-3 by Ond ej Nov .

Patches submitted that have not made their way to the archive yet:

• #806331 against xz-utils by Ximin Luo: make the selected POSIX shell stable accross build environments
• #806494 against gnupg by intrigeri: Make man pages not embed a build-time dependent timestamp
• #806945 against bash by Reiner Herrmann and Ximin Luo: Use the system man2html, and set PGRP_PIPE unconditionally.
• #825857 against python-setuptools by Anton Gladky: sort libs in native_libs.txt
• #826408 against brainparty by Reiner Herrmann: Sort object files for deterministic linking order
• #826416 against blockout2 by Reiner Herrmann: Sort the list of source files
• #826418 against xgalaga++ by Reiner Herrmann: Sort source files to get a deterministic linking order
• #826423 against kraptor by Reiner Herrmann: Sort source files for deterministic linking order
• #826431 against traceroute by Reiner Herrmann: Sort lists of libraries/source/object files
• #826544 against doc-debian by intrigeri: make the created files stable regardless of the locale
• #826676 against python-openstackclient by Chris Lamb: make the build reproducible
• #826677 against cadencii by Chris Lamb: make the build reproducible
• #826760 against dctrl-tools by Reiner Herrmann: Sort object files for deterministic linking order
• #826951 against slicot by Alexis Bienven e: please make the build reproducible (fileordering)
• #826982 against hoichess by Reiner Herrmann: Sort object files for deterministic linking order

Package reviews 68 reviews have been added, 19 have been updated and 28 have been removed in this week. New and updated issues:

• cryptographic_signature
• timestamps_in_maven_version_files
• ftbfs_build-indep_not_build_on_some_archs
• timestamps_added_by_xbean_spring
• timestamps_in_maven_metadata_local_xml_files
• timestamps_in_documentation_generated_by_asciidoctor

26 FTBFS bugs have been reported by Chris Lamb, 1 by Santiago Vila and 1 by Sascha Steinbiss. diffoscope development

• Mattia Rizzolo uploaded diffoscope/54 to jessie-backports.

strip-nondeterminism development

• Mattia uploaded strip-nondeterminism/0.018-1 to jessie-backports, to support a debhelper backport.
• Andrew Ayer uploaded strip-nondeterminism/0.018-2 fixing #826700, a packaging improvement for Multi-Arch to ease cross-build situations.
• 2 days later Andrew released strip-nondeterminism/0.019; now strip-nondeterminism is able to:
  • recursively normalize JAR files embedded within JAR files (#823917)
  • clamp the timestamp, the same way tar >=1.28-2.2 can (for now available only for gzip archives)

disorderfs development

• Andrew Ayer released disorderfs/0.4.3, fixing a issue with umask handling (#826891)

tests.reproducible-builds.org

• Valerie Young namespaced the Debian-specific pages to /debian/ namespace, with redirects to for the previous URLs.
• Holger Levsen improved the reliability of build jobs: the availability of both build nodes (for a given build) is now being tested when a build job is started, to better cope when one of the 25 build nodes go down for some reason.
• Ximin Luo improved the index of identified issues to include the total popcon scores of each issue, which is now also used for sorting that page.

Misc. Steven Chamberlain submitted a patch to FreeBSD's makefs to allow reproducible builds of the kfreebsd installer. Ed Maste committed a patch to FreeBSD's binutils to enable determinstic archives by default in GNU ar. Helmut Grohne experimented with cross+native reproductions of dash with some success, using rebootstrap. This week's edition was written by Ximin Luo, Chris Lamb, Holger Levsen, Mattia Rizzolo and reviewed by a bunch of Reproducible builds folks on IRC.

What happened in the Reproducible Builds effort between May 22nd and May 28th 2016: Media coverage

• Holger Levsen was invited to RIPE72 to talk about Reproducible Builds and a hope for a more secure future (Video, 25min and slides) in the cooperative workgroup.
• Scarlett Clark blogged about her first week of Outreachy work trying to make packages build reproducible, describing her efforts with kapptemplate, choqok and kdevplatform.

Documentation update

• The wiki page TimestampsProposal has been extended to cover more usage examples and to list more softwares supporting SOURCE_DATE_EPOCH. (Axel Beckert, Dhole and Ximin Luo)
• h01ger started a reference card for tools and information about reproducible builds but hasn't progressed much yet. Help with it is much welcome, this is also a good opportunity to learn about this project The idea is simply to have one coherent place with pointers to all the stuff we have and provide, without repeating nor replacing other documentation.

Toolchain fixes

• Alexis Bienven e submitted a patch (#824050) against emacs24 for SOURCE_DATE_EPOCH support in autoloads files, but upstream already disabled timestamps by default some time before.
• proj/4.9.2-3 uploaded by Bas Couwenberg (original patch by Alexis Bienven ) properly initializes memory with zero to prevent the nad2bin tool from leaking random memory content into output artefacts.
• Reiner Herrmann submitted a patch (#825569, upstream) against Ruby to sort object files in generated Makefiles, which are used to compile C sources that are part of Ruby projects.

Packages fixed The following 18 packages have become reproducible due to changes in their build dependencies: canl-c configshell dbus-java dune-common frobby frown installation-guide jexcelapi libjsyntaxpane-java malaga octave-ocs paje.app pd-boids pfstools r-cran-rniftilib scscp-imcce snort vim-addon-manager The following packages have become reproducible after being fixed:

• apngasm/2.7-2 by Manuel A. Fernandez Montecelo, original patch by Reiner Herrmann.
• apngdis/2.5-2 by Manuel A. Fernandez Montecelo, original patch by Reiner Herrmann.
• apngopt/1.2-2 by Manuel A. Fernandez Montecelo, original patch by Reiner Herrmann.
• asio/1:1.10.6-4 by Markus Wanner.
• bowtie/1.1.2-4 by Sascha Steinbiss.
• bowtie2/2.2.9-2 by Sascha Steinbiss.
• breeze/4:5.6.4-1 by Maximiliano Curia, original patch by Eduard Sanou.
• cain/1.10+dfsg-2 by Sascha Steinbiss.
• cdist/4.0.0-2 by Dmitry Bogatov, original patch by Chris Lamb.
• crossroads/2.81-1 by Reiner Herrmann.
• elasticsearch/1.7.5-1 by Emmanuel Bourg.
• gdal/2.1.0+dfsg-3 by Bas Couwenberg, original patch by Alexis Bienven e.
• grass/7.0.4-2 by Bas Couwenberg, original patch by Alexis Bienven e.
• jaligner/1.0+dfsg-3 by Michael R. Crusoe.
• khmer/2.0+dfsg-7 by Michael R. Crusoe.
• libhdf4/4.2.11-2 by Bas Couwenberg.
• libjspeex-java/0.9.7-4 by Emmanuel Bourg.
• libmialm/1.0.9-1 by Gert Wollny.
• libtheora/1.1.1+dfsg.1-10 by Petter Reinholdtsen.
• odil/0.6.0-2 by Julien Lamy.
• pacemaker/1.1.15~rc3-1 by Ferenc W gner.
• pvm/3.4.6-1 by James Clarke.
• rna-star/2.5.2a+dfsg-2 by Sascha Steinbiss.
• smalt/0.7.6-6 by Sascha Steinbiss.
• soapdenovo2/240+dfsg-3 by Sascha Steinbiss.
• svtplay-dl/1.1-1 by Olof Johansson.
• t-coffee/11.00.8cbe486-3 by Sascha Steinbiss.
• toppred/1.10-3 by Sascha Steinbiss.
• transdecoder/3.0.0+dfsg-2 by Michael R. Crusoe.
• vpim/0.695-1.4 by Herbert Parentes Fortes Neto.
• vtk-dicom/0.7.7-2 by Gert Wollny.
• xplc/0.3.13-6 by Reiner Herrmann.

Some uploads have fixed some reproducibility issues, but not all of them:

• codeblocks/16.01+dfsg-1 by Vincent Cheng, original patch by Fabian Wolff.
• gmt/5.2.1+dfsg-6 by Bas Couwenberg, original patch by Alexis Bienven .

Patches submitted that have not made their way to the archive yet:

• #803547 against bbswitch (reopened) by Reiner Herrmann: sort members of tar archive
• #806945 against bash (follow-up) by Reiner Herrmann: use system man2html instead of embedded copy
• #825122 against kapptemplate by Scarlett Clark: set owner/group of members in tarball to root
• #825138 against console-setup by Reiner Herrmann: fix umask issue; sort entries in shell script; sort fontsets/charmaps locale-independently
• #825285 against kodi by Lukas Rechberger: replace build timestamps with version numbers
• #825322 against choqok by Scarlett Clark: force UTF-8 locale so kconfig_compiler behaves correctly
• #825544 against wavemon by Reiner Herrmann: sort list of object files
• #825545 against dwm by Reiner Herrmann: sort list of header files
• #825547 against tennix by Reiner Herrmann: sort list of data files being archived
• #825584 against ffmpeg2theora by Reiner Herrmann: sort list of source files
• #825588 against kball by Reiner Herrmann: sort list of source files
• #825634 against miceamaze by Reiner Herrmann: sort list of object files
• #825643 against dash by Reiner Herrmann: fix sorting of struct members in generated source file
• #825655 against libselinux by Reiner Herrmann: sort list of source files
• #825656 against libsepol by Reiner Herrmann: sort list of source files
• #825674 against libsemanage by Reiner Herrmann: sort list of source files

Package reviews 123 reviews have been added, 57 have been updated and 135 have been removed in this week.

• 5 new issues have been identified:
  • timestamps_in_pdf_generated_by_imagemagick
  • ruby_mkmf_makefile_unsorted_objects
  • timestamps_from_timestamp_macro
  • scons_doesnt_pass_environment_to_build_tools
  • timestamps_from_cpp_macros_in_d

21 FTBFS bugs have been reported by Chris Lamb and Santiago Vila. strip-nondeterminism development

• strip-nondeterminsim development: treat *.htb as Zip files (by Sascha Steinbiss).
• strip-nondeterminism 0.017-1 uploaded by h01ger.

tests.reproducible-builds.org

• The kde pkg set was extended, though the change ain't visible yet, as there are currently non-installable packages in it (and so the set can't be computed). (h01ger)

Misc.

• Mattia improved misc.git/reports (=the tools to help writing the weekly statistics for this blog) some more.

This week's edition was written by Reiner Herrmann and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

What happened in the Reproducible Builds effort between May 15th and May 21st 2016: Media coverage Blog posts from our GSoC and Outreachy contributors:

• Val's first post describing her plans for improving tests.reproducible-builds.org.
• ceridwen's first post describing her plans for reprotest, the universal reproducibility testing tool.
• Scarlett's and Satyam's first posts were linked in previous week's report already.

Documentation update Ximin Luo clarified instructions on how to set SOURCE_DATE_EPOCH. Toolchain fixes

• Joao Eriberto Mota Filho uploaded txt2man/1.5.6-4, which honours SOURCE_DATE_EPOCH to generate reproducible manpages (original patch by Reiner Herrmann).
• Dmitry Shachnev uploaded sphinx/1.4.1-1 to experimental with improved support for SOURCE_DATE_EPOCH (original patch by Alexis Bienven e).
• Emmanuel Bourg submitted a patch against debhelper to use a fixed username while building ant packages.

Other upstream fixes

• Doxygen merged a patch by Ximin Luo, which uses UTC as timezone for embedded timestamps.
• CMake applied a patch by Reiner Herrmann in their next branch, which sorts file lists obtained with file(GLOB).
• GNU tar 1.29 with support for --clamp-mtime has been released upstream, closing #816072, which was the blocker for #759886 "dpkg-dev: please make mtimes of packaged files deterministic" which we now hope will be closed soon.

Packages fixed The following 18 packages have become reproducible due to changes in their build dependencies: abiword angband apt-listbugs asn1c bacula-doc bittornado cdbackup fenix gap-autpgrp gerbv jboss-logging-tools invokebinder modplugtools objenesis pmw r-cran-rniftilib x-loader zsnes The following packages have become reproducible after being fixed:

• ball/1.4.3~beta1-1 by Andreas Tille.
• chrony/2.3-1 by Vincent Blut, original patch by Alexis Bienven e.
• corosync/2.3.5-7 by Ferenc W gner.
• gap-ctbllib/1r2p2.dfsg.0-3 by Bill Allombert, original patch by Alexis Bienven e.
• kdiff3/0.9.98-3 by Eike Sauer.
• netcdf-cxx/4.3.0+ds-3 by Bas Couwenberg.
• netcdf-fortran/4.4.4+ds-2 by Bas Couwenberg.
• primesieve/5.6.0+ds-2 by Jerome Benoit.
• ray/2.3.1-4 by Sascha Steinbiss.
• rocksdb/4.5.1-1 by Laszlo Boszormenyi, original patch by Chris Lamb.
• shapelib/1.3.0-8 by Bas Couwenberg.
• swift/2.7.0-3 by Ond ej Nov .
• t-coffee/11.00.8cbe486-3 by Sascha Steinbiss.
• webkit2pdf/0.3-2 by Ricardo Mones, original patch by Reiner Herrmann.
• wise/2.4.1-18 by Sascha Steinbiss.

Some uploads have fixed some reproducibility issues, but not all of them:

• bzr/2.7.0-6 by Jelmer Vernoo .
• libsdl2/2.0.4+dfsg2-1 by Manuel A. Fernandez Montecelo.
• pvm/3.4.5-13 by James Clarke.
• refpolicy/2:2.20140421-11 by Laurent Bigonville.
• subvertpy/0.9.3-4 by Jelmer Vernoo .

Patches submitted that have not made their way to the archive yet:

• #824413 against binutils by Chris Lamb: filter build user and date from test log case-insensitively
• #824452 against python-certbot by Chris Lamb: prevent PID from being embedded into documentation (forwarded upstream)
• #824453 against gtk-gnutella by Chris Lamb: use SOURCE_DATE_EPOCH for deterministic timestamp (merged upstream)
• #824454 against python-latexcodec by Chris Lamb: fix for parsing the changelog date
• #824472 against torch3 by Alexis Bienven e: sort object files while linking
• #824501 against cclive by Alexis Bienven e: use SOURCE_DATE_EPOCH as embedded build date
• #824567 against tkdesk by Alexis Bienven e: sort order of files which are parsed by mkindex script
• #824592 against twitter-bootstrap by Alexis Bienven e: use shell-independent printing
• #824639 against openblas by Alexis Bienven e: sort object files while linking
• #824653 against elkcode by Alexis Bienven e: sort list of files locale-independently
• #824668 against gmt by Alexis Bienven e: use SOURCE_DATE_EPOCH for embedded timestamp (similar patch by Bas Couwenberg already applied and forwarded upstream)
• #824808 against gdal by Alexis Bienven e: sort object files while linking
• #824951 against libtomcrypt by Reiner Herrmann: use SOURCE_DATE_EPOCH for timestamp embedded into metadata

Reproducibility-related bugs filed:

• #824420 against python-phply by ceridwen: parsetab.py file is not included when building with DEB_BUILD_OPTIONS="nocheck"
• #824572 against dpkg-dev by XImin Luo: request to export SOURCE_DATE_EPOCH in /usr/share/dpkg/*.mk.

Package reviews 51 reviews have been added, 19 have been updated and 15 have been removed in this week.

• Two new issues have been found:
  • random_order_in_static_library_by_icmake
  • timestamps_in_manpages_generated_by_latex2man

22 FTBFS bugs have been reported by Chris Lamb, Santiago Vila, Niko Tyni and Daniel Schepler. tests.reproducible-builds.org

• Icons have been added to the package test history pages (h01ger).
• Test performance and variation pages have been splitted out of the dashboard view (h01ger).
• A new pkg set has been added: packages maintained by debian-med-packaging@l.a.d.o (h01ger).
• The preliminary results for testing have improved further:
  • testing/amd64 has 20944 / 90.2% reproducible packages now,
  • testing/i386 has reached 20013 / 86.3% reproducible packages,
  • testing/armhf has come close with 19951 / 86.0%.

Misc.

• During the discussion on debian-devel about PIE, an archive rebuild was suggested by B lint R czey, and Holger Levsen suggested to coordinate this with a required archive rebuild for reproducible builds.
• Ximin Luo improved misc.git/reports (=the tools to help writing the weekly statistics for this blog) quite a bit, h01ger contributed a little too.

This week's edition was written by Reiner Herrmann and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.