Gunnar Wolf: On Dmitry Bogatov and empowering privacy-protecting tools

--fixdebugpath
build flag feature
by default, which should result in reproducible builds (for C code) even with
varying paths. So far we haven't had many results due to disturbances in our
build network in the last days, but it seems this would mean roughly between
5-15% additional unreproducible packages - compared to what we see now. We'll
keep you updated on the numbers (and problems with compilers and common
frameworks) as we find them.
lynxis continued work to test LEDE
and OpenWrt on two different
hosts, to include date variation in the tests.
Mattia and Holger worked on the (mass) deployment scripts, so that the - for space
reasons - only jenkins.debian.net GIT
clone resides in ~jenkins-adm/
and not anymore in Holger's homedir, so
that soon Mattia (and possibly others!) will be able to fully maintain this setup,
while Holger is doing siesta.
Miscellaneous
Chris, dkg, h01ger and Ximin attended a Core Infrastricture
Initiative summit meeting in New York
City, to discuss and promote this Reproducible Builds project. The CII was set
up in the wake of the Heartbleed SSL vulnerability to support software projects
that are critical to the functioning of the internet.
This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of
Reproducible Builds folks on IRC.
SOURCE_DATE_EPOCH
support in autoloads files, but upstream already disabled timestamps by default some time before.SOURCE_DATE_EPOCH
variable, and the Javadoc task use en
as the default locale if none was specified and SOURCE_DATE_EPOCH
is set.SOURCE_DATE_EPOCH
.hostname
command with the fixed string "localhost".dh_installinit
source files in a stable order.
Chris Lamb found how to make cython output deterministic by ordering the keys used to traverse a dict
.
Reiner Herrmann proposed a patch for pyside-tools to remove the timestamps embedded by rcc
in the generated Python code.
Mattia Rizzolo rebased our custom version of debhelper on version 9.20151126.
As no objections have been made so far, Mattia Rizzolo has filled #805872 asking -Wdate-time
to be turned on by default in dpkg-buildflag. Guillem has since sent a final warning before proceeding as such in the next dpkg upload.
Russ Allbery added support for SOURCE_DATE_EPOCH in podlators 4.00 which Niko Tyni intend to backport to Perl 5.22.
Packages fixed
The following packages have become reproducible due to changes in their
build dependencies:
fontforge,
golang-github-tinylib-msgp,
libpango-perl,
libparanamer-java,
libxaw,
sqljet,
stringtemplate4,
uzbl,
zope-mysqlda.
The following packages became reproducible after getting fixed:
help2man
instead of embedded copy.help2man
instead of embedded copy.LC_ALL
instead of LANG
to ensure how dd
output looks like.LC_ALL=C
when enumerating files to link.debian/changelog
entry.SOURCE_DATE_EPOCH
in VersionInfo.cmake
.armhf
build node has been added (resulting of two more armhf
build jobs). Thanks to Vagrant Cascadian for putting this Raspberry Pi 2B to help. (h01ger)
jenkins.debian.net has been made more robust against network and proxy failures. (h01ger)
A new 100 GB partition has been set up on reproducible.debian.net to prevent disk space issues. Thanks to ProfitBricks for its continuous support to our continuous test system. (h01ger)
New graphs showing usertagged bugs have been added on the dashboard to measure the progress without FTBFS issues. Please note that comparing the two graphs might be misleading as more than 1300 FTBFS bugs have been inventoried. (h01ger)
Package reviews
78 reviews have been removed, 116 added and 49 updated this week.
25 new FTBFS have been filed by Chris West, Chris Lamb and Santiago Vila.
New issues identified this week: timestamps_in_documentation_generated_with_libwibble, copyright_year_in_documentation_generated_by_sphinx, timestamps_in_documentation_generated_by_glib_genpod, random_order_of_tmpfiles_in_postinst, random_order_in_cython_output, timestamps_in_python_code_generated_by_pyside.
Reiner Herrmann and Lunar improved the prebuilder script: the script can now be called through a symlink, run parallel builds, calls diffoscope by its new name and ensure to install its recommends, and save the text output aside the HTML one.
Reiner also added a script to lookup the last update of notes for a given package.
Misc.
Santiago Villa has been recently working on making sure that Arch:all
packages were properly buildable by running dpkg-buildpackage -A
. This uncovered a question that is probably not currently addressed by the policy: on which architectures should architecture-independent be buildable?
xorriso
new flags for -alter_date
to avoid update ctimes. Report by Daniel Kahn Gillmor.pod2man
now that we have SOURCE_DATE_EPOCH specified. There is really a whole class of issues that are currently undiscovered waiting for tests running on a different date. This is likely to should happen soon.
Chris Lamb uploaded a new version of debhelper in the reproducible repository, cherry-picking a fix for interactions between ddebs and udebs.
Packages fixed
The following packages became reproducible due to changes in their
build dependencies:
aspic,
django-guardian,
erlang-sqlite3,
etcd,
libnative-platform-java,
mingw-ocaml,
nose2,
oar,
obexftp,
py3cairo,
python-dugong,
python-secretstorage,
python-setuptools,
qct,
qdox,
recutils,
s3ql,
wine.
The following packages became reproducible after getting fixed:
__TIME__
and __DATE__
macros.debian/changelog
entry.armhf
and amd64
nodes in now finished. The remaining reproducibility tests running on the Jenkins host has been removed. armhf
results and graphs are now visible in dashboard. We can now test the whole archive in 2-3 weeks using the current 12 amd64
jobs and 3 months using the current 6 armhf
builders. We will be looking at improving the armhf
sitation, maybe using more native systems or via arm64
. (h01ger)
The Jenkins UI is now more responsive since all jobs building packages have been moved to remote hosts. (h01ger)
A new job has been added to collect information about build nodes to be included in the variation table. (h01ger)
The currently scheduled page has been split for amd64 and armhf. They now give an overview (refreshed every minute, thanks to Chris Lamb) of the packages currently being tested. (h01ger)
Several cleanup and bugfixes have been made, especially in the remote building and maintenance scripts. They should now be more robust against network problems. The automatic scheduler is now also run closer to when schroots and pbuilders are updated. (h01ger, mapreri)
Package reviews
16 reviews have
been removed, 54 added and 55 updated this week.
Santiago Vila renamed lc_messages_randomness
with the more descriptive different_pot_creation_date_in_gettext_mo_files.
New issues added this week: timestamps_in_reports_generated_by_abi_compliance_checker, umask_and_timestamp_variation_in_tgz_generated_by_abi_compliance_checker, and timestamps_added_by_blast2.
23 new FTBFS bugs have been filled by Chris Lamb, and Niko Tyni.
Misc.
Red Hat developper Mike McLean had a talk at Flock 2015 about reproducible builds in Koji. Slides and video recording are available. Koji is the build infrastructure used by Fedora, Red Hat and other distributions. It already keeps track of the environment used for a given build, so the required changes for handling the environment are smaller than the ones in Debian. Fedora is still missing a team effort to fix non-determinism in the package builds, but it is great to see Fedora moving forward.