What happened in the
Reproducible
Builds effort between Sunday August 14 and Saturday August 20 2016:
Fasten your seatbelts
Important note: we enabled build path variation for
unstable now, so your package(s) might become unreproducible, while previously it was said to be reproducible given a specific build path it probably still is reproducible but read on for the details below in the tests.reproducible-builds.org section! As said many times: this is still research and we are working to make it reality.
Media coverage
Daniel Stender blogged about
python
packaging and
explained some caveats regarding reproducible builds.
Toolchain developments
Thomas Schmitt uploaded
xorriso which
now
obeys SOURCE_DATE_EPOCH. As
stated in its man pages:
ENVIRONMENT
[...]
SOURCE_DATE_EPOCH belongs to the specs of reproducible-builds.org. It
is supposed to be either undefined or to contain a decimal number which
tells the seconds since january 1st 1970. If it contains a number, then
it is used as time value to set the default of --modification-date=,
--gpt_disk_guid, and --set_all_file_dates. Startup files and program
options can override the effect of SOURCE_DATE_EPOCH.
Packages reviewed and fixed, and bugs filed
The following packages have become reproducible after being fixed:
- cadencii/3.3.9+svn20110818.r1732-5 by Ying-Chun Liu, original patch by Chris Lamb.
- cfengine3/3.7.4-3 by Antonio Radici.
- findbugs/3.1.0~preview2-1 by Emmanuel Bourg.
- gcpegg/5.1-14 by Bdale Garbee, original patch by Chris Lamb.
- gnunet-gtk/0.10.1-4 by Bertrand Marc, original patch by Chris Lamb.
- konwert/1.8-12 by Yann Dirson, original patch by Chris Lamb.
- link-grammar/5.3.8-2 by Jeremy Bicha, original patch by Chris Lamb.
- metastudent-data/2.0.1-1 by Sascha Steinbiss.
- pixmap/2.6pl4-20 by Paul Slootman, original patch by
Chris Lamb Maria Valentina Marin a.k.a. Akira.
- python-afl/0.5.4-2 by Daniel Stender.
- tf5/5.0beta8-6 by Russ Allbery, original patch by Chris Lamb.
- triplane/1.0.8-2 by Markus Koschanyover, original patch by Chris Lamb.
- vips/8.3.3-1 by Laszlo Boszormenyi, original patch by Chris Lamb.
- xzgv/0.9.1-4 by Theodore Y. Ts'o, original patch by Chris Lamb.
The following updated packages appear to be reproducible now, for reasons we
were not able to figure out. (Relevant changelogs did not mention reproducible
builds.)
- vulkan/1.0.21.0+dfsg1-1 by Timo Aaltonen.
The following 2 packages were not changed, but have become reproducible due
to changes in their build-dependencies:
tagsoup tclx8.4.
Some uploads have addressed some reproducibility issues, but not all of them:
Patches submitted that have not made their way to the archive yet:
Bug tracker house keeping:
- Chris Lamb pinged 164 bugs he filed more than 90 days ago which have a patch and had no maintainer reaction.
Reviews of unreproducible packages
55 package reviews have been added, 161 have been updated and 136 have been
removed in this week, adding to our knowledge about
identified
issues.
2 issue types have been updated:
Weekly QA work
FTBFS bugs have been reported by:
- Chris Lamb (16)
- Santiago Vila (2)
diffoscope development
Chris Lamb, Holger Levsen and Mattia Rizzolo worked on diffoscope this week.
Improvements were made to SquashFS and JSON comparison, the
https://try.diffoscope.org/ web service, documentation,
packaging, and general code quality.
diffoscope 57, 58, and 59 were uploaded to unstable by Chris Lamb. Versions 57
and 58 were both broken, so Holger set up a job on jenkins.debian.net to test
diffoscope on each git commit. He also wrote a CONTRIBUTING document to help
prevent this from happening in future.
From these efforts, we were also able to learn that diffoscope is now reproducible
even when built across multiple architectures:
< h01ger> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope.html shows these packages were built on amd64:
< h01ger> bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger> 366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger> and on i386:
< h01ger> bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger> 366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger> and on armhf:
< h01ger> bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger> 366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
And those
also match the binaries uploaded by Chris in his diffoscope 59 binary upload to ftp.debian.org, yay! Eating our own dogfood and enjoying it!
tests.reproducible-builds.org
Debian related:
- show percentage of results in the last 24/48h (h01ger)
- switch python database backend to SQLAlchemy (Valerie)
- vary build path varitation for unstable and experimental for all architectures. (h01ger)
The last change probably will have an impact you will see: your package might become unreproducible in
unstable and this will be shown on
tracker.debian.org, while it will still be reproducible in
testing.
We've done this, because we think reproducible builds are possible with arbitrary build paths. But: we don't think those are a realistic goal for
stretch, where we still recommend to use .buildinfo to record the build patch and then do rebuilds using that path.
We are doing this, because besides doing theoretical groundwork we also have a practical goal: enable users to independently verify builds. And if they only can do this with a fixed path, so be it. For now
To be clear: for Stretch we recommend that reproducible builds are done in the same build path as the "original" build.
Finally, and just for our future references, when we enabled build path variation on Saturday, August 20th 2016, the numbers for
unstable were:
| suite |
all |
reproducible |
unreproducible |
ftbfs |
depwait |
not for this arch |
blacklisted |
| unstable/amd64 |
24693 |
21794 (88.2%) |
1753 (7.1%) |
972 (3.9%) |
65 (0.2%) |
95 (0.3%) |
10 (0.0%) |
| unstable/i386 |
24693 |
21182 (85.7%) |
2349 (9.5%) |
972 (3.9%) |
76 (0.3%) |
103 (0.4%) |
10 (0.0%) |
| unstable/armhf |
24693 |
20889 (84.6%) |
2050 (8.3%) |
1126 (4.5%) |
199 (0.8%) |
296 (1.1%) |
129 (0.5%) |
Misc.
Ximin Luo updated our git setup scripts to make it easier for people to write
proper descriptions for our repositories.
This week's edition was written by Ximin Luo and Holger Levsen and reviewed by
a bunch of Reproducible Builds folks on IRC.
Do you know what s happening in the wires of your network? There is a major
FLOSS player in the field of real time intrusion detection (IDS), inline
intrusion prevention (IPS) and network security monitoring (NSM).
I m talking about Suricata, a mature, fast and robust network threat detection
engine. Suricata is a community driven project, supported by the
Open InfoSec Foundation (OISF).
For those who doesn t know how Suricata works, it usually runs by loading a set
of pre-defined rules for matching different network protocols and flow
behaviours. In this regards, Suricata has been always ruleset-compatible with
the other famous IDS: snort.
The last major release of Suricata is 4.0.0, and I m uploading the package for
Debian stretch-backports as I write this line. This means the updated package
should be available for general usage after the usual buildds processing ends
inside the Debian archive.
You might be wondering, How to start using Suricata 4.0 with Debian Stretch?
First, I would recommend reading the docs. Please checkout:
The idea is simply to have one coherent place with pointers to all the stuff we have and provide, without repeating nor replacing other documentation.
