Here's what happened in the Reproducible Builds effort between Sunday July 16 and Saturday July 22 2017: Toolchain development Bernhard M. Wiedemann wrote a tool to automatically run through different sources of non-determinism, and report which of these caused irreproducibility. Dan Kegel's patches to fpm were merged. Bugs filed Patches submitted upstream:

• Bernhard M. Wiedemann:
  • Sort file lists:
    • eric5 merged
    • libsass-python merged
    • tcl merged
    • python3.6 in progress
    • drbd
    • blobwars
  • Instead of fixing ordering issues in a custom .pak archive (same as blobwars above), we allow to install individual data files to avoid the issue:
    • edgar avoid sort
  • Omit the build date entirely:
    • fence-agents
  • SOURCE_DATE_EPOCH support:
    • criu, merged
    • dapl, merged
    • shorewall, merged
    • youtube-dl in progress
    • automake
    • crosstool-ng
    • docker
    • drbd
    • drbd-utils
    • getdp
    • infinipath-psm
    • opa-fm
    • opa-psm2
    • texinfo
  • geany/glfw unknown

Patches filed in Debian:

• Adrian Bunk:
  • #868599 filed against ocaml-curses.
  • #868609 filed against le.
  • #868612 filed against mixxx.
  • #868855 filed against softhsm2.
  • #868858 filed against gwc.
  • #869086 filed against dsniff.
• Chris Lamb:
  • #868790 filed against castle-game-engine, forwarded upstream.
  • #868843 filed against xorg-server, forwarded upstream.
  • #869516 filed against libcdio.
• Drew Parsons:
  • #868505 filed against sdpa.
• Lucas Nussbaum:
  • #868904 filed against gwc.
  • #868927 filed against python-pybedtools.
• Sascha Steinbiss:
  • #868772 filed against ragel.

Reviews of unreproducible packages 73 package reviews have been added, 44 have been updated and 50 have been removed in this week, adding to our knowledge about identified issues. No issue types were updated. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (106)
• Daniel Stender (1)
• Drew Parsons (1)
• F lix Sipma (1)
• Lucas Nussbaum (25)

diffoscope development

• Juliana Rodrigues:
  • Add new XML comparator. (Closes: #866120)
• Guangyuan Yang:
  • Fix 2 cases in test_device on FreeBSD
• Chris Lamb:
  • comparators.xml: Fix EPUB "missing file" tests; they ship a META-INF/container.xml file.
  • comparators.sqlite: Simplify file detection in Sqlite3Database.RE_FILE_TYPE
  • Style and attribution fixes to XML comparator and comparators.directory
• Ximin Luo:
  • main, logging: restore old logger settings to avoid pytest vomiting in certain situations
  • comparators/directory: Fix #868534 by expecting less strict test output

reprotest development

• Ximin Luo:
  • Use autopkgtest upstream paths, makes things easier to import
  • Add script for importing autopkgtest code, and import autopkgtest 4.4

Ximin also restarted the discussion with autopkgtest-devel about code reuse for reprotest. Santiago Torres began a series of patches to make reprotest more distro-agnostic, with the aim of making it usable on Arch Linux. Ximin reviewed these patches. Misc. This week's edition was written by Ximin Luo, Bernhard M. Wiedemann and Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Since debhelper/10.3, there has been a number of performance related changes. The vast majority primarily improves bulk performance or only have visible effects at larger input sizes. Most visible cases are:

• dh + dh_* now scales a lot better for large number of binary packages. Even more so with parallel builds.
• Most dh_* tools are now a lot faster when creating many directories or installing files.
• dh_prep and dh_clean now bulk removals.
• dh_install can now bulk some installations. For a concrete corner-case, libssl-doc went from approximately 11 seconds to less than a second. This optimization is implicitly disabled with exclude (among other).
• dh_installman now scales a lot better with many manpages. Even more so with parallel builds.
• dh_installman has restored its performance under fakeroot (regression since 10.2.2)

For debhelper, this mostly involved:

• avoiding fork+exec of commands for things doable natively in perl. Especially, when each fork+exec only process one file or dir.
• bulking as many files/dirs into the call as possible, where fork+exec is still used.
• caching / memorizing slow calls (e.g. in parts of pkgfile inside Dh_Lib)
• adding an internal API for dh to do bulk check for pkgfiles. This is useful for dh when checking if it should optimize out a helper.
• and, of course, doing things in parallel where trivially possible.

How to take advantage of these improvements in tools that use Dh_Lib:

• If you use install_ file,prog,lib,dir , then it will come out of the box. These functions are available in Debian/stable. On a related note, if you use doit to call install (or mkdir ), then please consider migrating to these functions instead.
• If you need to reset owner+mode (chown 0:0 FILE + chmod MODE FILE), consider using reset_perm_and_owner. This is also available in Debian/stable.
  • CAVEAT: It is not recursive and YMMV if you do not need the chown call (due to fakeroot).
• If you have a lot of items to be processed by a external tool, consider using xargs(). Since 10.5.1, it is now possible to insert the items anywhere in the command rather than just in the end.
• If you need to remove files, consider using the new rm_files function. It removes files and silently ignores if a file does not exist. It is also available since 10.5.1.
• If you need to create symlinks, please consider using make_symlink (available in Debian/stable) or make_symlink_raw_target (since 10.5.1). The former creates policy compliant symlinks (e.g. fixup absolute symlinks that should have been relative). The latter is closer to a ln -s call.
• If you need to rename a file, please consider using rename_path (since 10.5). It behaves mostly like mv -f but requires dest to be a (non-existing) file.
• Have a look at whether on_pkgs_in_parallel() / on_items_in_parallel() would be suitable for enabling parallelization in your tool.
  • The emphasis for these functions is on making parallelization easy to add with minimal code changes. It pre-distributes the items which can lead to unbalanced workloads, where some processes are idle while a few keeps working.

Credits: I would like to thank the following for reporting performance issues, regressions or/and providing patches. The list is in no particular order:

• Helmut Grohne
• Kurt Roeckx
• Gianfranco Costamagna
• Iain Lane
• Sven Joachim
• Adrian Bunk
• Michael Stapelberg

Should I have missed your contribution, please do not hesitate to let me know.
Filed under: Debhelper, Debian

Since debhelper/10.3, there has been a number of performance related changes. The vast majority primarily improves bulk performance or only have visible effects at larger input sizes. Most visible cases are:

• dh + dh_* now scales a lot better for large number of binary packages. Even more so with parallel builds.
• Most dh_* tools are now a lot faster when creating many directories or installing files.
• dh_prep and dh_clean now bulk removals.
• dh_install can now bulk some installations. For a concrete corner-case, libssl-doc went from approximately 11 seconds to less than a second. This optimization is implicitly disabled with exclude (among other).
• dh_installman now scales a lot better with many manpages. Even more so with parallel builds.
• dh_installman has restored its performance under fakeroot (regression since 10.2.2)

For debhelper, this mostly involved:

• avoiding fork+exec of commands for things doable natively in perl. Especially, when each fork+exec only process one file or dir.
• bulking as many files/dirs into the call as possible, where fork+exec is still used.
• caching / memorizing slow calls (e.g. in parts of pkgfile inside Dh_Lib)
• adding an internal API for dh to do bulk check for pkgfiles. This is useful for dh when checking if it should optimize out a helper.
• and, of course, doing things in parallel where trivially possible.

How to take advantage of these improvements in tools that use Dh_Lib:

• If you use install_ file,prog,lib,dir , then it will come out of the box. These functions are available in Debian/stable. On a related note, if you use doit to call install (or mkdir ), then please consider migrating to these functions instead.
• If you need to reset owner+mode (chown 0:0 FILE + chmod MODE FILE), consider using reset_perm_and_owner. This is also available in Debian/stable.
  • CAVEAT: It is not recursive and YMMV if you do not need the chown call (due to fakeroot).
• If you have a lot of items to be processed by a external tool, consider using xargs(). Since 10.5.1, it is now possible to insert the items anywhere in the command rather than just in the end.
• If you need to remove files, consider using the new rm_files function. It removes files and silently ignores if a file does not exist. It is also available since 10.5.1.
• If you need to create symlinks, please consider using make_symlink (available in Debian/stable) or make_symlink_raw_target (since 10.5.1). The former creates policy compliant symlinks (e.g. fixup absolute symlinks that should have been relative). The latter is closer to a ln -s call.
• If you need to rename a file, please consider using rename_path (since 10.5). It behaves mostly like mv -f but requires dest to be a (non-existing) file.
• Have a look at whether on_pkgs_in_parallel() / on_items_in_parallel() would be suitable for enabling parallelization in your tool.
  • The emphasis for these functions is on making parallelization easy to add with minimal code changes. It pre-distributes the items which can lead to unbalanced workloads, where some processes are idle while a few keeps working.

Credits: I would like to thank the following for reporting performance issues, regressions or/and providing patches. The list is in no particular order:

• Helmut Grohne
• Kurt Roeckx
• Gianfranco Costamagna
• Iain Lane
• Sven Joachim
• Adrian Bunk
• Michael Stapelberg

Should I have missed your contribution, please do not hesitate to let me know.
Filed under: Debhelper, Debian

Here's what happened in the Reproducible Builds effort between Sunday July 9 and Saturday July 15 2017: Packages reviewed and fixed, and bugs filed

• Adrian Bunk:
  • #867771 filed against python-trollius.
  • #867773 filed against relatorio.
  • #867781 filed against gcin.
  • #867890 filed against apache-directory-jdbm.
  • #867900 filed against yapsy.
  • #867906 filed against wcsaxes.
• Chris Lamb:
  • #867753 filed against grunt (forwarded upstream).
  • #867848 filed against gconf (forwarded upstream).
  • #868133 filed against grep.
  • #868321 filed against node-marked-man (forwarded-upstream).
• Bernhard M. Wiedemann:
  • chromium
  • chromium
  • kubernetes
  • qtscriptgenerator
  • crash merged
  • ghc-rpm-macros merged
  • samba
  • htmldoc
  • efl
  • webkitgtk3 merged
  • fence-agents
  • fio merged

Reviews of unreproducible packages 13 package reviews have been added, 12 have been updated and 19 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been added:

• build_path_captured_by_valac
• timestamps_in_javascript_generated_by_node_grunt_banner

3 issue types have been updated:

• Add fix for timestamps_embedded_in_manpages_by_node_marked_man toolchain issue.
• Add fix for timestamps_in_javascript_generated_by_node_grunt_banner toolchain issue.
• timestamps_in_javascript_generated_by_node_grunt_banner is actually in src:grunt

Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (47)

diffoscope development Version 84 was uploaded to unstable by Mattia Rizzolo. It included contributions already reported from the previous weeks, as well as new ones:

• Ximin Luo:
  • Attempt to fix fsimage test on Jenkins
  • Use a tempdir rather than ./cache for guestfs cache
  • Add more debugging output to test_fsimage

After the release, development continued in git with contributions from:

• Mattia Rizzolo:
  • RequiredToolNotFound.get_package(): just call the new get_package_provider()
  • Add a get_package_provider() function, returning the package name that best matches the system
  • Move from deprecated platform.linux_distribution() to the external package distro

strip-nondeterminism development Versions 0.036-1, 0.037-1 and 0.038-1 were uploaded to unstable by Chris Lamb. They included contributions from:

• Niels Thykier:
  • Add missing use statements in handler modules
  • dh_strip_nondeterminism: Assumes tmpdir() exists
  • File::StripNondeterminism:
    • Apply perltidy
    • Lazy load most handlers
    • Lazy load remaining handlers
• Chris Lamb:
  • Add missing File::Temp imports in JAR and PNG handlers. This appears to have been exposed by lazily-loading handlers in #867982.

reprotest development Development continued in git with contributions from:

• Ximin Luo:
  • presets: use newer flag --no-sign for dpkg-buildpackage
  • Document --diffoscope-args= --exclude-directory-metadata and use it in presets
• Mattia Rizzolo:
  • Bump debhelper compat level to 10.
  • Bump Standards-Version to 4.0.0.

buildinfo.debian.net development

• Chris Lamb:
  • Avoid a race condition between check-and-creation of Buildinfo instances.

tests.reproducible-builds.org

• Mattia Rizzolo:
  • Make database backups quicker to restore by avoiding --column-inserts's pg_dump option.
  • Fixup the deployment scripts after the stretch migration.
  • Fixup Apache redirects that were broken after introducing the buster suite
  • Fixup diffoscope jobs that were not always installing the highest possible version of diffoscope
• Holger Levsen:
  • Add a node health check for a too big jenkins.log.

Misc. This week's edition was written by Bernhard M. Wiedemann, Chris Lamb, Mattia Rizzolo, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday July 2 and Saturday July 8 2017: Reproducible work in other projects Ed Maste pointed to a thread on the LLVM developer mailing list about container iteration being the main source of non-determinism in LLVM, together with discussion on how to solve this. Ignoring build path issues, container iteration order was also the main issue with rustc, which was fixed by using a fixed-order hash map for certain compiler structures. (It was unclear from the thread whether LLVM's builds are truly path-independent or rather that they haven't done comparisons between builds run under different paths.) Bugs filed

• Adrian Bunk:
  • #867499 filed against tiptop.
  • #867773 filed against relatorio.
  • #867781 filed against gcin.
• Chris Lamb:
  • #866945 filed against tinymux.
  • #867753 filed against grunt.
  • #867848 filed against gconf.

Patches submitted upstream:

• Bernhard M. Wiedemann:
  • perl-Class-MethodMaker sort hashtable
  • perl-Mouse sort file list
  • thunderbird sort symbol list
  • SOURCE_DATE_EPOCH support:
    • graphviz, merged
    • gnupg, merged in improved variant by upstream
    • janus-gateway, merged
    • libkcapi merged
  • txt2tags fixup on earlier patch, merged

Reviews of unreproducible packages 52 package reviews have been added, 62 have been updated and 20 have been removed in this week, adding to our knowledge about identified issues. No issue types were updated or added this week. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (143)
• Andreas Beckmann (1)
• Dmitry Shachnev (1)
• Lucas Nussbaum (3)
• Niko Tyni (3)
• Scott Kitterman (1)
• Sean Whitton (1)

diffoscope development Development continued in git with contributions from:

• Ximin Luo:
  • Add a PartialString class to help with lazily-loaded output formats such as html-dir.
  • html and html-dir output:
    • add a size-hint to the diff headers and lazy-load buttons
    • add new limit flags and deprecate old ones
  • html-dir output
    • split index pages up if they get too big
    • put css/icon data in separate files to avoid duplication
  • main: warn if loading a diff but also giving diff-calculation flags
  • Test fixes for Python 3.6 and CI environments without imagemagick (#865625).
  • Fix a performance regression (#865660) involving the Wagner-Fischer algorithm for calculating levenshtein distance.

With these changes, we are able to generate a dynamically loaded HTML diff for GCC-6 that can be displayed in a normal web browser. For more details see this mailing list post. Misc. This week's edition was written by Ximin Luo, Bernhard M. Wiedemann and Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday June 25 and Saturday July 1 2017: Upcoming and past events Our next IRC meeting is scheduled for July 6th at 17:00 UTC (agenda). Topics to be discussed include an update on our next Summit, a potential NMU campaign, a press release for buster, branding, etc. Toolchain development and fixes

• James McCoy reviewed and merged Ximin Luo's script debpatch into the devscripts Git repository. This is useful for rebasing our patches onto new versions of Debian packages.

Packages fixed and bugs filed

• Adrian Bunk:
  • #866713 filed against debhelper.
• Chris Lamb:
  • #865994 filed against xabacus.
  • #866164 filed against qmidinet.
  • #866169 filed against singularity-container.
  • #866330 filed against cd-hit.

Ximin Luo uploaded dash, sensible-utils and xz-utils to the deferred uploads queue with a delay of 14 days. (We have had patches for these core packages for over a year now and the original maintainers seem inactive so Debian conventions allow for this.) Patches submitted upstream:

• openmpi
• gtk-doc fixed by sorting directory listings
• samba
• TeX
• nedit
• criu
• tvheadend sort
• tvheadend date
• cfengine

Reviews of unreproducible packages 4 package reviews have been added, 4 have been updated and 35 have been removed in this week, adding to our knowledge about identified issues. One issue types has been updated:

• Add upstream URL for random_order_of_pdf_ids_generated_by_latex

One issue type has been added:

• timestamps_in_manpages_added_by_golang_go_flags

Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (68)
• Daniel Schepler (1)
• Michael Hudson-Doyle (1)
• Scott Kitterman (6)

diffoscope development

• Daniel Shahaf:
  • Fix markup in the man page synopsis. Thanks to Niels Thykier for the report. (Closes: #866577)
• Mattia Rizzolo:
  • Bump backport version check in debian/rules
• Ximin Luo:
  • Fix a progressbar failure
  • Put the 400MB "fsimage" cache in a more obvious place
  • Fix CI tests under Python 3.6
  • Add a --exclude-directory-metadata option. (Closes: #866241)
  • Raise warning for getfacl. Remove a redundant try-clause
  • Fix recursive indentation of headers
  • Use a loop rather than recursion
  • In html-dir mode, put css/icon in separate files to avoid duplication
  • diffcontrol UI tweaks
  • Split index pages up if they get too big

tests.reproducible-builds.org

• Vagrant Cascadian working on testing Debian:
  • Upgraded the 27 armhf build machines to stretch.
  • Fix mtu check to only display status when eth0 is present.
• Helmut Grohne worked on testing Debian:
  • Limit diffoscope memory usage to 10GB virtual per process. It currently tends to use 50GB virtual, 36GB resident which is bad for everything else sharing the machine. (This is #865660)
• Alexander Couzens working on testing LEDE:
  • Add multiple architectures / targets.
  • Limit LEDE and OpenWrt jobs to only allow one build at the same time using the jenkins build blocker plugin.
  • Move git log -1 > .html to node document environment().
  • Update TODOs.
• Mattia Rizzolo working on testing Debian:
  • Updated the maintenance script for postgresql-9.6.
  • Restored the postgresql database from backup after Holger accidently purged it.
• Holger Levsen working on:
  • Merging all the above commits.
  • Added a check for (known) Jenkins zombie jobs and report them. (This is an long known problem with jenkins; deleted jobs sometimes come back )
  • Upgraded the remaining amd64 nodes to stretch.
  • Accidentially purged postgres-9.4 from jenkins, so we could test our backups
  • Updated our stretch upgrade TODOs.

Misc. This week's edition was written by Chris Lamb, Ximin Luo, Holger Levsen, Bernhard Wiedemann, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday June 18 and Saturday June 24 2017: Upcoming and Past events Our next IRC meeting is scheduled for the 6th of July at 17:00 UTC with this agenda currently:

1. Introductions
2. Reproducible Builds Summit update
3. NMU campaign for buster
4. Press release: Debian is doing Reproducible Builds for Buster
5. Reproducible Builds Branding & Logo
6. should we become an SPI member
7. Next meeting
8. Any other business

On June 19th, Chris Lamb presented at LinuxCon China 2017 on Reproducible Builds. On June 23rd, Vagrant Cascadian held a Reproducible Builds question and answer session at Open Source Bridge. Reproducible work in other projects LEDE: firmware-utils and mtd-utils/mkfs.jffs2 now honor SOURCE_DATE_EPOCH. Toolchain development and fixes There was discussion on #782654 about packaging bazel for Debian. Dan Kegel wrote a patch to use ar determinitiscally for Homebrew, a package manager for MacOS. Dan Kegel worked on using SOURCE_DATE_EPOCH and other reproduciblity fixes in fpm, a multi plattform package builder. The Fedora Haskell team disabled parallel builds to achieve reproducible builds. Bernhard M. Wiedemann submitted many patches upstream:

• Sorting file lists:
  • python setuptools
  • cpython tarfile/zipfile
  • lyx
  • dpdk
  • perl Alien::SDL
  • gtk-doc, no fix yet
• Use date from SOURCE_DATE_EPOCH:
  • uwsgi
  • recode
  • perl Glib

Packages fixed and bugs filed Patches submitted upstream:

• Bernhard M. Wiedemann
  • samba, sorting file lists
  • open-mpi, merged, sorting file lists
  • virtualbox, incomplete fix, use gzip -n
  • xen/etherboot, xen/mini-os, use gzip -n
  • nautilus-dropbox, drop date
  • samba, use date from SOURCE_DATE_EPOCH.

Other patches filed in Debian:

• Adrian Bunk:
  • #865577 filed against cloud-init.
  • #865813 filed against jumpnbump.
• Chris Lamb:
  • #865554 filed against janus.
  • #865623 filed against cracklib2.
  • #865751 filed against nfstrace.
  • #865753 filed against tigervnc.

Reviews of unreproducible packages 573 package reviews have been added, 154 have been updated and 9 have been removed in this week, adding to our knowledge about identified issues. 1 issue type has been updated:

• random_order_of_pdf_ids_generated_by_latex, added the name of the function that possibly causes this issue.

Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (98)

diffoscope development Version 83 was uploaded to unstable by Chris Lamb. It also moved the previous changes from experimental (to where they were uploaded) to unstable. It included contributions from previous weeks. You can read about these changes in our previous weeks' posts, or view the changelog directly (raw form). We plan to maintain a backport of this and future versions in stretch-backports. Ximin Luo also worked on better html-dir output for very very large diffs such as those for GCC. So far, this includes unreleased work on a PartialString data structure which will form a core part of a new and more intelligent recursive display algorithm. strip-nondeterminism development Versions 0.035-1 was uploaded to unstable from experimental by Chris Lamb. It included contributions from:

• Bernhard M. Wiedemann
  • Add CPIO handler and test case.
• Chris Lamb
  • Packaging improvements.

Later in the week Mattia Rizzolo uploaded 0.035-2 with some improvements to the autopkgtest and to the general packaging. We currently don't plan to maintain a backport in stretch-backports like we did for jessie-backports. Please speak up if you think otherwise. reproducible-website development

• Chris Lamb:
  • Add OpenEmbedded to projects page after a discussion at LinuxCon China.
  • Update some metadata for existing talks.
  • Add 13 missing talks.

tests.reproducible-builds.org

• Alexander 'lynxis' Couzens
  • LEDE: do a quick sha256sum before calling diffoscope. The LEDE build consists of 1000 packages, using diffoscope to detect whether two packages are identical takes 3 seconds in average, while calling sha256sum on those small packages takes less than a second, so this reduces the runtime from 3h to 2h (roughly). For Debian package builds this is neglectable, as each build takes several minutes anyway, thus adding 3 seconds to each build doesn't matter much.
  • LEDE/OpenWrt: move toolchain.html creation to remote node, as this is were the toolchain is build.
  • LEDE: remove debugging output for images.
  • LEDE: fixup HTML creation for toolchain, build path, downloaded software and GIT commit used.
• Mattia 'mapreri' Rizzolo:
  • Debian: introduce Buster.
  • Debian: explain how to migrate from squid3 (in jessie) to squid (in stretch).
• Holger 'h01ger' Levsen:
  • Debian:
    • Add jenkins jobs to create schroots and configure pbuilder for Buster.
    • Add Buster to README/about jenkins.d.n.
    • Teach jessie and ubuntu 16.04 systems how to debootstrap Buster.
    • Only update indexes and pkg_sets every 30min as the jobs almost run for 15 min now that we test four suites (compared to three before).
    • Create HTML dashboard, live status and dd-list pages less often.
    • (Almost) stop scheduling old packages in stretch, new versions will still be scheduled and tested as usual.
    • Increase scheduling limits, especially for untested, new and depwait packages.
    • Replace Stretch with Buster in the repository comparison page.
    • Only keep build_service logs for a day, not three.
    • Add check for hanging mounts to node health checks.
    • Add check for haveged to node health checks.
    • Disable ntp.service on hosts running in the future, needed on stretch.
    • Install amd64 kernels on all i386 systems. There is a performance issue with i386 kernels, for which a bug should be filed. Installing the amd64 kernel is a sufficient workaround, but it breaks our 32/64 bit kernel variation on i386.
  • LEDE, OpenWrt: Fix up links and split TODO list.
  • Upgrade i386 systems (used for Debian) and pb3+4-amd64 (used for coreboot, LEDE, OpenWrt, NetBSD, Fedora and Arch Linux tests) to Stretch
  • jenkins: use java 8 as required by jenkins >= 2.60.1

Misc. This week's edition was written by Ximin Luo, Holger Levsen, Bernhard M. Wiedemann, Mattia Rizzolo, Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday June 11 and Saturday June 17 2017: Upcoming events

• On June 19th, Chris Lamb presented at LinuxCon China 2017 on Reproducible Builds.
• h01ger created a poll for a date for the next Reproducible Builds summit, please vote if you are interested in attending.
• Our next IRC meeting will be on the 6th of July at 17:00 UTC with the following agenda.

Upstream patches and bugs filed

• Bernhard M. Wiedemann:
  • gnuradio + volk
  • pymol
  • distorm
  • qtscriptgenerator
  • cpython
  • x3270 x3270
  • sphinx
  • obs-service-tar_scm
  • osc
  • matplotlib merged
  • pyparted merged
  • bjoern merged

Reviews of unreproducible packages 1 package review has been added, 19 have been updated and 2 have been removed in this week, adding to our knowledge about identified issues. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (1)
• Edmund Grimley Evans (1)

diffoscope development

• Chris Lamb:
  • Document the 'h' variable in our raw_feeder
  • Split diffoscope.difference into feeders
  • Tidy diffoscope/difference.py
• Ximin Luo:
  • Add a PartialString class
  • More ydiff/linediff from diffoscope. difference => diff to group unified_diff related things together
  • difference: has_children -> has_visible_children, and take into account comments
  • Add various traverse_ methods to Difference
  • Move side-by-side and linediff algorithms to difference.py
  • Refactor html-dir presenter to be a class instance, avoiding global state
  • html-dir: show/hide diff comments, which can be very large

tests.reproducible-builds.org As you might have noticed, Debian stretch was released last week. Since then, Mattia and Holger renamed our testing suite to stretch and added a buster suite so that we keep our historic results for stretch visible and can continue our development work as usual. In this sense, happy hacking on buster; may it become the best Debian release ever and hopefully the first reproducible one!

• Vagrant Cascadian:
  • Proposed reducing the number of concurrent builds each armhf machine runs, to reduce out-of-memory errors, crashes and false positives with FTBFS on armhf, reducing the number of armhf build workers from 70 to 51.
• Valerie Young: Add highlighting in navigation for the new nodes health pages.
• Mattia Rizzolo:
  • Do not dump database ACL in the backups.
  • Deduplicate SSLCertificateFile directive into the common-directives-ssl macro
  • Apache: t.r-b.o: redirect /testing/ to /stretch/
  • db: s/testing/stretch/g
  • Start adding code to test buster...
• Holger Levsen:
  • Update README.infrastructure to explain who has root access where.
  • reproducible_nodes_info.sh: correctly recognize zero builds per day.
  • Add build nodes health overview page, then split it in three: health overview, daily munin graphs and weekly munin graphs.
  • reproducible_worker.sh: improve handling of systemctl timeouts.
  • reproducible_build_service: sleep less and thus restart failed workers sooner.
  • Replace ftp.(de uk us).debian.org with deb.debian.org everywhere.
  • Performance page: also show local problems with _build_service.sh (which are autofixed after a maximum of 133.7 minutes).
  • Rename nodes_info job to html_nodes_info.
  • Add new node health check jobs, split off from maintenance jobs, run every 15 minutes.
    • Add two new checks: 1. for correct future (2019 is incorrect atm, and we sometimes got that). 2.) for writeable /tmp (sometimes happens on borked armhf nodes).
  • Add jobs for testing buster.
  • s/testing/stretch/g in all the code.
  • Finish the code to deal with buster.
  • Teach jessie and Ubuntu 16.04 how to debootstrap buster.

Axel Beckert is currently in the process of setting up eight LeMaker HiKey960 boards. These boards were sponsored by Hewlett Packard Enterprise and will be hosted by the SOSETH students association at ETH Zurich. Thanks to everyone involved here and also thanks to Martin Michlmayr and Steve Geary who initiated getting these boards to us. Misc. This week's edition was written by Chris Lamb, Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday June 4 and Saturday June 10 2017: Past and upcoming events On June 10th, Chris Lamb presented at the Hong Kong Open Source Conference 2017 on reproducible builds. Patches and bugs filed

• Chris Lamb: #864082 filed against fontconfig, forwarded upstream

Reviews of unreproducible packages 7 package reviews have been added, 10 have been updated and 14 have been removed in this week, adding to our knowledge about identified issues. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (4)
• Chris Lamb (1)
• Christoph Biedl (1)
• Niko Tyni (1)

Two FTBFS issues of LEDE (exposed in our setup) were found and were fixed:

• build: ensure that flock is available for make download (Felix Fietkau)
• include/toplevel: set env GIT_ASKPASS=/bin/true (Alexander 'lynxis' Couzens)

diffoscope development

• Chris Lamb: Some code style improvements

tests.reproducible-builds.org: Alexander 'lynxis' Couzens made some changes for testing LEDE and OpenWrt:

• Build tar before downloading everything: On system without tar --sort=name we need to compile tar before downloading everything
• Set CONFIG_AUTOREMOVE to reduce required space
• Create a workaround for signing keys: LEDE signs the release with a signing key, but generates the signing key if it's not present. To have a reproducible release we need to take care of signing keys.
• openwrt_get_banner(): use staging_dir instead of build_dir because the former is persistent among the two builds.
• Don't build all packages to improve development speed for now.
• Only build one board instead of all boards. Reducing the build time improves developing speed. Once the image is reproducible we will enable more boards.
• Disable node_cleanup_tmpdirs

Hans-Christoph Steiner, for testing F-Droid:

• Do full git reset/clean like Jenkins does
• hard code WORKSPACE dir names, as WORKSPACE cannot be generated from $0 as it's a temporary name.

Daniel Shahaf, for testing Debian:

• Remote scheduler:
  • English fix to error message.
  • Allow multiple architectures in one invocation.
  • Refactor: Break out a helper function. Rename variable to disambiguate with scheduling_args.message.
• Include timestamps in logs
• Set timestamps to second resolution (was millisecond by default).

Holger 'h01ger' Levsen, for testing Debian:

• Improvements to the breakages page:
  • List broken packages and diffoscope problems first, and t.r-b.o problems last.
  • Reword, drop 'caused by'.
• Add niceness to our list of variations, running with niceness of 11 for the first build and niceness of 10 for the second one. Thanks to Vagrant for the idea.
• Automatic scheduler:
  • Reschedule after 12h packages that failed with error 404
  • Run scheduler every 3h instead of every 6h
• Add basic README about the infrastructure and merge Vagrants notes about his console host.

Misc. This week's edition was written by Ximin Luo, Chris Lamb and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday May 28 and Saturday June 3 2017: Past an upcoming events

• On June 9th, Chris Lamb will present at the Hong Kong Open Source Conference 2017 on reproducible builds.
• We restarted our IRC meetings, now with a monthly schedule where we meet on the first Thursday of the month at a varying time. Our next meeting will be on the first 6th of July at 17:00 UTC. At the June meeting we had the following agenda, and if you are interested there is an automated summary and full logs too.
  1. Introductions
  2. Feedback for the reproducible.json spec format - is that suitable for Guix and F-Droid as well? LEDE/OpenWrt? Coreboot?
  3. writing parser + user interface for reproducible.json
  4. tests.r-b.o/Debian once Stretch has been released
  5. Reproducible Builds Summit 2017
  6. Next meeting
  7. Any other business

Documentation updates

• Holger added a link to the blog post with a report about the recent hackathon to our website.

Toolchain development and fixes

• Chris Lamb wrote a proof-of-concept implementation for #863622 ("apt: warn when installing packages that are not reproducible"). Try it, it's fun!
• Russ worked on making the debian-policy package build reproducibly which is a good opportunity to shamelessly plug that #844431 ("debian-policy: Packages should be reproducible") is still a work in progress.

Patches and bugs filed

• Bernhard M. Wiedemann:
  • pycryptopp
  • rpm
  • rpm
  • bam
  • boost/jam
  • premake
  • ruby/rdoc
  • ruby/rdoc
  • sdcc (merged)

4 package reviews have been added, 6 have been updated and 25 have been removed in this week, adding to our knowledge about identified issues. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (2)
• Chris Lamb (1)

diffoscope development

• Chris Lamb:
  • Add comparator for Fontconfig cache files.
• anthraxx:
  • tools: extend external Arch Linux tools
• Ximin Luo:
  • Add a --exclude-command CLI for filtering out long-running commands like "readelf --debug-dump=info"
  • Use unicode chars for the +/- controls instead of hacky punctuation
  • Don't show +/- controls for differences with no children
  • Fix create_limited_print_func
  • Add a size() method to Difference and check that self._visuals is empty in get_reverse()
  • Add a reader for the JSON format
  • Allow the "source" param to overridden compare() methods to be given as a positional argument

tests.reproducible-builds.org Mattia Rizzolo:

• packages-tests: go back to the shorter way of specifying JUnit jobs.
• update_jdn: call jenkins-jobs instead of jenkins-job-builder, as that's how it is named in 1.6.1 version.
• Upload jenkins-job-builder 1.6.1-1~bpo8+1 to jessie-backports.

Daniel Kahn Gillmor:

• Contribute a few typo fixes.

Vagrant Cascadian:

• Update README to reflect switch to PostgreSQL (some time ago).
• Add three new boards for armhf tests:
  • ff64a-armhf-rb.debian.net: Firefly-rk3399, Rockchip six-core (Cortex-A72 x2 + Cortex-A53 x4), 2GB RAM, USB-sata (future plans for native sata.)
  • jtx1a-armhf-rb.debian.net: Jetson-tx1, quad-core (big.LITTLE Cortex-A53/A57), ~3.5GB RAM, native SATA ~500GB disk
  • odc2a-armhf-rb.debian.net: Odroic-C2, quad-core (Cortex-A53), ~2GB ram, 128GB eMMC
• Ressurect rpi2c-armhf-rb.debian.net (Raspberry PI 2B, broadcom bcm2836 quad-core (cortex-A7), 1GB RAM) from the dead.

Holger Levsen:

• Configure the rc.local service on all build nodes to only start when the network is actually up, as configuring half of them to run in the future requires networking to determine the real current date. See Running Services After the Network is up if you want to learn more about networking with systemd. Somehow this doesnt work yet on the Ubuntu 16.05 arm64 nodes; help welcome, either on #debian-qa or #debian-reproducible.
• Add rpi2c back to the armhf network.
• Also add ff64a, odc2a & jtx1a to the armhf network.
  • Add pbuilder/schroot-setup jobs for the new nodes.
  • Add 10 new armhf builder jobs.
  • Disable all build jobs on odc2a again as haveged segfaults on 4.12~rc2, which is needed for this board & setup.
• Reproducible_cleanup_nodes.sh: adapt for new build service
• Make performance.html show if build jobs are down due to remote node problems and make code to count enabled jobs more robust.
• reproducible_build_service: Make it cope with disabled workers.
• jenkins-master-wrapper: Fail loudly if remote node is not accessable.
• Increase maximum scheduling queue sizes, to adjust for faster scheduler.
• New script, init_node, to initialize new build nodes (derived from refactored existing script).
• Update INSTALL documentation to reflect that we're testing arm64 now too.

Misc. This week's edition was written by Chris Lamb, Bernhard M. Wiedemann and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday May 21 and Saturday May 27 2017: Past and upcoming events Bernhard M. Wiedemann gave a short talk on reproducible builds in openSUSE at the openSUSE Conference 2017. Slides and video recordings are available on that page. Chris Lamb will present at the Hong Kong Open Source Conference 2017 on reproducible builds on June 9th. Our next IRC meeting has been scheduled for Thursday June 1 at 16:00 UTC with this agenda. Academia Justin Cappos continued his work on the reproducible builds paper, with text and suggestions from Ximin Luo integrated. Toolchain developments #863470: "ftp.debian.org: security sync must not exclude .buildinfo" - while this bug isn't fixed, you need to make sure not to build jessie updates with stretch's dpkg, or else the upload will be rejected. Ximin Luo built GCC twice and ran diffoscope on them. Unfortunately the results were 1.7 GB in size and it can't be displayed in a web browser. 99/171 of the .debs are reproducible, though. He's now working on diffoscope (see below) to make it generate output more intelligently for such large size diffs. Here is a summary diff where the recursion depth cut-off was set low, so the size is reasonable and one can still see the outlines of where to look next. debuerreotype was newly added to Debian unstable. It is a reproducible, snapshot-based Debian rootfs builder. Patches and bugs filed

• Adrian Bunk:
  • #863202 filed against adonthell-data.
  • #863208 filed against mash.
  • #863414 filed against coyim.
• Chris Lamb:
  • #863117 filed against scilab.
• James Clarke:
  • #863493 filed against symfony.
• Justin Cappos:
  • Follow up about #822197 against sphinx.
• Bernhard M. Wiedemann:
  • FBReader
  • hwinfo
  • intel-cmt-cat
  • libtorrent
  • mupdf
  • pesign
  • python-spidermonkey
  • selinux
  • silc
  • TPM2.0-TSS
  • dosemu merged
  • ebiso merged
  • honggfuzz merged
  • john merged
  • libtomcrypt merged
  • opalvoip merged
  • PyQt4/5 merged
  • spacenav merged
  • wavemon merged

Reviews of unreproducible packages 29 package reviews have been added, 49 have been updated and 23 have been removed in this week, adding to our knowledge about identified issues. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (10)
• Chris Lamb (2)
• James Clarke (1)

diffoscope development Development continued in git, with commits from:

• Ximin Luo:
  • Refactor container-related logic to make the code clearer.
  • Various improvements to the progress bar, making it behave more accurately and make it compatible with --debug logging output.
  • Fix --exclude control.tar.gz.
  • When enforcing max-container-depth, show which internal files differ, without showing their details.
  • Add --max-container-depth CLI option.

strip-nondeterminism development Version 0.034-1 was uploaded to unstable by Chris Lamb. It included previous weeks' contributions from:

• Chris Lamb
  • Only print log messages by default if the file was actually modified. (Closes: #863033)
• Bernhard M. Wiedemann
  • zip: make sure we have permissions on extracted file
  • Add function prototypes.

tests.reproducible-builds.org:

• Alexander Couzens
  • Use Alexander's LEDE git repo to test his mksquashfs patches.
• Daniel Shahaf
  • Refactored reproducible_remote_scheduler.py to add support for multiple suites in one invocation.
• Holger Levsen
  • Prevent the two fdroid jobs from running together by using the Build Blocker Plugin.
  • A niceness variation was also added (see #863440) to the Debian tests, but this change was reverted for now, as it was breaking stuff and needs to be readded properly.
  • Some adjustments to the Debian scheduler, still due to the improve performance through the new build services.
• Mattia Rizzolo
  • Update dsa-check-running-kernel from dsa-nagios (to support kernel 4.x as present in stretch) on all jenkins nodes.

Misc. This week's edition was written by Ximin Luo, Bernhard M. Wiedemann, Chris Lamb and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday May 7 and Saturday May 13 2017: Report from Reproducible Builds Hamburg Hackathon We were 16 participants from 12 projects: 7 Debian, 2 repeatr.io, 1 ArchLinux, 1 coreboot + LEDE, 1 F-Droid, 1 ElectroBSD + privoxy, 1 GNU R, 1 in-toto.io, 1 Meson and 1 openSUSE. Three people came from the USA, 3 from the UK, 2 Finland, 1 Austria, 1 Denmark and 6 from Germany, plus we several guests from our gracious hosts at the CCCHH hackerspace as well as a guest from Australia We had four presentations:

• "Reproducible Builds everywhere" by h01ger
• https://in-toto.io by Justin Cappos
• http://repeatr.io by Eric Myhre
• http://mesonbuild.com by Jussi Pakkanen

Some of the things we worked on:

• h01ger did orga stuff for this very hackathon, discussed tests.r-b.o with various non-Debian contributors, filed some bugs and restarted the policy discussion in #844431. He also did some polishing work on tests.r-b.o which shall be covered in next issue of our weekly blog.
• Justin Cappos involved many of us in interesting discussions and started to write an academic paper about Reproducible Builds of which he shared an early beta on our mailinglist.
• Chris Lamb (lamby) filed a number of patches for individual packages, worked on diffoscope, merged many changes to strip-nondeterminism and also filed #862073 against dak to upload buildinfo files to external services.
• Maria Glukhova (siamezzze) fixed a bug with plots on tests.reproducible-builds.org and worked on diffoscope test coverage.
• Lynxis worked on a new squashfs upstream release improving support for reproducible squashfs filesystems and also had some time to hack on coreboot and show others how to install coreboot on real hardware.
• Michael Poehn worked on integrating F-Droid builds into tests.reproducible-builds.org, on the F-Droid verification utility and also ran some app reproducibility tests.
• Bernhard worked on various unreproducible issues upstream and submitted fixes for curl, bzr, ant.
• Erin Myhre worked on bootstrapping cleanroom builds of compiler components in Repeatr sandboxes.
• Calvin Behling merged improvements to reppl for a cleaner storage format and better error handling and did design work for next version of repeatr pipeline execution. Calvin also lead the reproducibility testing of restaurant mood lighting.
• Eric and Calvin also claim to have had all sorts of useful exchanges about the state of other projects, and learned a lot about where to look for more info about debian bootstrap and archive mirroring from steven and lamby
• Phil Hands came by to say hi and worked on testing d-i on jenkins.debian.net.
• Chris West (Faux) worked on extending misc.git:has-only.py, and started looking at Britney.

We had a Debian focussed meeting where we discussed a number of topics:

• IRC meetings: yes, we want to try again to have them, monthly, a poll for a good date is being held.
• Debian tests post Stretch: we'll add tests for stable/Stretch.
• .buildinfo files, how forward: we need sourceful uploads for any arch:all packages. dak should send .buildinfo files to buildinfo.debian.net.
• (pre?) Stretch release press release: we should do that, esp. as our achievements are largely unrelated to Stretch.
• Reproducible Builds Summit 3: yes, we want that.
• what to do (in notes.git) with resolved issues: keep the issues.
• strip-nondeterminism quo vadis: Justin reminded us that strip-nondeterminism is a workaround we want to get rid off.

And then we also had a lot of fun in the hackerspace, enjoying some of their gimmicks, such as being able to open physical doors with ssh or controlling light and music with an webbrowser without authentication (besides being in the right network). (This wasn't the hackathon per-se, but some of us appreciated these sights and so we thought you would too.) Many thanks to:

• Debian for sponsoring food and accomodation!
• Dock Europe for providing us with really nice accomodation in the house!
• CCC Hamburg for letting us use their hackerspace for >3 days non-stop!

News and media coverage openSUSE has had a security breach in their infrastructure, including their build services. As of this writing, the scope and impact are still unclear, however the incident illustrates that no one should rely on being able to secure their infrastructure at all times. Reproducible Builds help mitigate this by allowing independent verification of build results, by parties that are unaffected by the compromise. (Whilst this can happen to anyone. Kudos to openSUSE for being open about it. Now let's continue working on Reproducible Builds everywhere!) On May 13th Chris Lamb gave a talk on Reproducible Builds at OSCAL 2017 in Tirana, Albania. Toolchain bug reports and fixes

• Chris Lamb:
  • Fixed a long-standing toolchain issue (#842635, #858389) in docbook-to-man by adopting the package.
  • #862003 filed against debhelper.
  • #862073 filed against ftp.debian.org, to upload buildinfo files to buildinfo.debian.net.
• Steven Chamberlain:
  • #862059 filed against sbuild, for signing buildinfo files.
• Ximin Luo:
  • #862112 filed against r-base.
  • #862113 filed against gcc-6.
  • #862116 filed against dpkg.

Packages' bug reports

• Chris Lamb:
  • #862088 filed against compass-h5bp-plugin.
  • #862140 filed against ofxstatement-plugins.
  • #862179 filed against acct.
  • #862183 filed against libjgroups-java.
  • #862195 filed against sendip.
  • #862451 filed against wammu, forwarded upstream.
  • #862484 filed against seqan2.
  • #862553 filed against vim-command-t.
  • #862588 filed against tkhtml1.
  • #862592 filed against taskcoach.
• Chris West:
  • #862252 filed against dns-root-data.

Reviews of unreproducible packages 11 package reviews have been added, 2562 have been updated and 278 have been removed in this week, adding to our knowledge about identified issues. Most of the updates were to move ~1800 packages affected by the generic catch-all captures_build_path (out of ~2600 total) to the more specific gcc_captures_build_path, fixed by our proposed patches to GCC. 5 issue types have been updated:

• Updated docbook_to_man_one_byte_delta
• Added ocaml_captures_build_path
• Added gcj_captures_build_path
• Added gcc_captures_build_path
• Re-added docbook_to_man_one_byte_delta

Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (1)
• Chris Lamb (2)
• Chris West (1)

diffoscope development diffoscope development continued on the experimental branch:

• Maria Glukhova:
  • Code refactoring and more tests.
• Chris Lamb:
  • Add safeguards against unpacking recursive or deeply-nested archives. (Closes: #780761)

strip-nondeterminism development

• strip-nondeterminism 0.033-1 and -2 were uploaded to unstable by Chris Lamb. It included contributions from:
• Bernhard M. Wiedemann:
  • Add cpio handler.
  • Code quality improvements.
• Chris Lamb:
  • Add documentation and increase verbosity, in support of the long-term aim of removing the need for this tool.

reprotest development

• reprotest 0.6.1 and 0.6.2 were uploaded to unstable by Ximin Luo. It included contributions from:
• Ximin Luo:
  • Add a documentation section on "Known bugs".
  • Move developer documentation away from the man page.
  • Mention release instructions in the previous changelog.
  • Preserve directory structure when copying artifacts. Otherwise hash output on a successful reproduction sometimes fails, because find(1) can't find the artifacts using the original artifact_pattern.
• Chris Lamb
  • Add proper release instructions and a keyring.

trydiffoscope development

• Chris Lamb:
  • Uses the diffoscope from Debian experimental if possible.

Misc. This week's edition was written by Ximin Luo, Holger Levsen and Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday April 23 and Saturday April 29 2017: Past and upcoming events On April 26th Chris Lamb gave a talk at foss-north 2017 in Gothenburg, Sweden on Reproducible Builds. Between May 5th-7th the Reproducible Builds Hackathon 2017 will take place in Hamburg, Germany. Then on May 26th Bernhard M. Wiedemann will give a talk titled reproducible builds in openSUSE (2017) at the openSUSE Conference 2017 in N rnberg, Germany. Media coverage Already on April 19th Sylvain Beucler wrote a yet another follow-up post Practical basics of reproducible builds 3, after part 1 and part 2 of his series. Toolchain development and fixes Michael Woerister of the Rust project has implemented file maps that affect all path-related compiler information, including "error messages, metadata, debuginfo, and the file!() macro alike". Ximin Luo with support from some other Rust developers and contributors helped steer the final result into something that was compatible with reproducible builds. Many thanks to all involved, especially for the patience of discussing this over several months. Ximin wrote a first-attempt patch to fix R build-path issues. It made 460/477 R packages reproducible, but also caused 3 of these to FTBFS. See randomness_in_r_rdb_rds_databases for details. Bugs filed and patches sent upstream Chris Lamb:

• #861133 filed against tf.
• #861443 filed against ora2pg.

Bernhard M. Wiedemann filed a number of patches upstream:

• intltool
• xine-ui
• tboot
• javapackages
• mxml, which got merged already.
• calibre, also merged.
• wammu, merged as well.

Reviews of unreproducible packages 102 package reviews have been added, 64 have been updated and 24 have been removed in this week, adding to our knowledge about identified issues. 3 issue types have been updated:

• Added captures_build_path_in_beam_files, recategorise some erlang packages with captures_build_path into this issue instead.
• Removed timestamps_in_beam_files.
• Holger started reviewing blacklisted_on_jenkins and blacklisted_on_jenkins_armhf_only and found quite some packages which don't need to be blacklisted anymore.

Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Aaron M. Ucko (1)
• Adrian Bunk (1)
• Chris Lamb (4)
• Santiago Vila (2)

diffoscope development diffoscope 82 was uploaded to experimental by Chris Lamb. It included contributions from:

• Chris Lamb:
  • Add support for Ogg Vorbis files.
• Vagrant Cascadian:
  • Add support for .dtb (device tree blob) files. (Closes: #861109).

Changes from previous weeks that were also released with 82:

• Ximin Luo
  • Add support for R .rds and .rdb object files.
• Chris Lamb
  • Add support for comparing Pcap files.
  • Add support for .docx and .odt files via docx2txt & odt2txt.
  • Add support for PGP files via pgpdump.
  • Various documentation and test improvements.
  • Various bug fixes and code quality improvements.
• Sylvain Beucler
  • Display differences in zip platform-specific timestamps.

Misc. This week's edition was written by Ximin Luo, Chris Lamb and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday April 2 and Saturday April 8 2017: Media coverage

• Daniel Peebles reported that tar still has problem with deterministic output and long filenames.
• Dustin Kirkland wrote a blog post referencing reproducible builds as a followup to a recent thread on Hacker News.

Toolchain development and fixes

• Ximin Luo uploaded custom version of GCC6 to our repository to test BUILD_PATH_PREFIX_MAP patches.

Reviews of unreproducible packages 27 package reviews have been added, 14 have been updated and 17 have been removed in this week, adding to our knowledge about identified issues. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Aaron M. Ucko (1)
• Adrian Bunk (1)
• Chris Lamb (2)

tests.reproducible-builds.org

• Chris Lamb added support for a .bz2 version of reproducible.json, available at https://reproducible.debian.net/reproducible.json.bz2
• FreeGeek donated a network switch to make room for some additional boards on the armhf build network.

Misc. This week's edition was written by Chris Lamb, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

The following contributors got their Debian Developer accounts in the last two months:

• Karen M Sandler (karen)
• Sebastien Badia (sbadia)
• Christos Trochalakis (ctrochalakis)
• Adrian Bunk (bunk)
• Michael Lustfield (mtecknology)
• James Clarke (jrtc27)
• Sean Whitton (spwhitton)
• Jerome Georges Benoit (calculus)
• Daniel Lange (dlange)
• Christoph Biedl (cbiedl)
• Gustavo Panizzo (gefa)
• Gert Wollny (gewo)
• Benjamin Barenblat (bbaren)
• Giovani Augusto Ferreira (giovani)
• Mechtilde Stehmann (mechtilde)
• Christopher Stuart Hoskin (mans0954)

The following contributors were added as Debian Maintainers in the last two months:

• Dmitry Bogatov
• Dominik George
• Gordon Ball
• Sruthi Chandran
• Michael Shuler
• Filip Pytloun
• Mario Anthony Limonciello
• Julien Puydt
• Nicholas D Steeves
• Raoul Snyman

Congratulations!

What happened in the Reproducible Builds effort between Sunday December 18 and Saturday December 24 2016: Media coverage 100% Of The 289 Coreboot Images Are Now Built Reproducibly by Phoronix, with more details in German by Pro-Linux.de. We have further reports on our Reproducible Builds World summit #2 in Berlin from Rok Garbas of NixOS as well as Clemens Lang of MacPorts Debian infrastructure work Dak now archives buildinfo files thanks to a patch from Chris Lamb. We also have mostly finalised a design of how they will be distributed by the Debian FTP mirror network which we will start implementing soon. This is great for the future of Debianb but unfortunately this also means that we won't have .buildinfo files for Stretch as Debian will not rebuild its source packages and because these binary packages currently in the archive were mostly built with dpkg > 1.18.11. reprepro/5.0.0-1 has added support for dealing with .buildinfo files that are included in .changes files. (Closes: #843402) Reproducible work in other projects The Chromium project is now working on making their build process (mostly) deterministic. Their motivation is to save both "[money] (less hardware is required) and developer time (reduced latency by having less work to do on the TS and CI)". Unreproducible bugs filed

• Chris Lamb:
  • #848721 filed against apt.
  • #848866 filed against libcorelinux.
  • #849314 filed against node-gulp.
  • #849333 filed against faker.
• Dhole:
  • #848633 filed against sugar-toolkit-gtk3.

Reviews of unreproducible packages 39 package reviews have been added, 75 have been updated and 44 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been updated:

• Updated captures_build_path
• Added nondeterministic_ordering_in_desktop_files_by_python_sugar3

Weekly QA work During our reproducibility testing, some FTBFS bugs have been detected and reported by:

• Adrian Bunk (1)
• Chris Lamb (7)
• Lucas Nussbaum (4)

diffoscope development diffoscope 66 was uploaded to unstable by Chris Lamb. It included contributions from:

• Emanuel Bronshtein:
  • Use ssh-keygen for comparing OpenSSH public keys
  • Use js-beautify as JavaScript code beautifier for .js files (with tests).
  • Many CSS & HTML improvements.
  • Change all HTTP URLs to HTTPS where applicable.
• anthraxx:
  • Enable the use of ssh-keygen on Arch Linux.
• Maria Glukhova:
  • Add detection of order-only difference in plain text format. (Closes: #848049)
  • Change icc-recognizing regexp to reflect changes in file type description. (Closes: #848814)
• Chris Lamb:
  • Update tests for compatibility with enjarify >= 1.0.3. (Closes: #849142)
  • When skipping tests because the version of an external is too low, print the detected version.
  • Avoid unpacking packages twice when comparing .changes. (Closes: #843531)
  • Add a simple profiling framework (enabled via --profile).
  • Various code quality and reliability improvements.
  • Document how to sign PyPI uploads.

strip-nondeterminism development strip-nondeterminism 0.029-1 was uploaded to unstable by Chris Lamb. It included no new content from this week, but rather included contributions from previous weeks. reproducible-website development The website is now also accessible via the https://www.reproducible-builds.org URL.

• Clemens Lang:
  • Add the definition of "reproducible", as drafted at the reproducible builds world summit in Berlin. Thanks to all participants in the sessions that worked these out!
• Valerie R Young:
  • Force ordering of titles.
  • Various formatting improvements.
• Holger Levsen:
  • Various usability and formatting improvements.
  • https://www.reproducible-builds.org
• Chris Lamb:
  • Various usability, style and wording improvements.
  • Add Debconf15, Skroutz.gz and MiniDebconfCambridge15 talks to resouces page.

tests.reproducible-builds.org

• We changed the data storage backend from a single sqlite3 database file (651 MB) to a PostgreSQL database. With this change we'll be able to scale a lot more and add testing of the arm64 architecture.
  • Valerie Young wrote most of the code, Mattia Rizzolo reviewed and helped improve the code and Holger deployed it and found some minor bugs which have been fixed.
• We are now testing the arm64 architecture for all packages on all suites, arranged by Holger. Many thanks to codethink for providing us with access to eight 8-core arm64 machines with 64GB memory, which allows us to rebuild Debian very fast!

Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC and the mailing lists.

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Android

• Chris Lamb was so kind to send in a patch for apktool to make the build reproducible (#845475). Although this was not enough to fix the issue it set me on the right path to eventually resolve bug number 845475.

Debian Games

• I packaged a couple of new upstream releases for extremetuxracer, fifechan, fife, unknown-horizons, freeciv, atanks and armagetronad. Most notably fifechan was accepted by the FTP team which allowed me to package new versions of fife and unknown-horizons which are both back in testing again. I expect that upstream will make their final release sometime in December. Atanks has been orphaned a while ago and since upstream is still active and I kinda like the game I decided to adopt it. I also uploaded a backport of Freeciv 2.5.6 to jessie-backports.
• In November we received a bunch of RC bug reports again because, hey, it is almost time for the Freeze, let s break some packages. Thus I spent some time fixing freeorion (#843132), pokerth (#843078), simutrans (#828545), freeciv (#844198) and warzone2100 (#844870).
• I also updated the debian-games blend, we are at version 1.6 now, and made some smaller adjustments. The most important change was adding a new binary package, games-all, that installs..well, all! I know this will make at least one person on this planet happy. Actually I was kind of forced into adding it because blends-dev automatically creates it as a requirement for choosing blends with the Debian Installer. But don t be afraid games-all only recommends games-finest, the rest is suggested.
• Last but not least I worked on performous and could close a wishlist bug report (#425898). The submitter asked to suggest some free song packages for this karaoke game.

Debian Java

• I sponsored uncommons-watchmaker for Kai-Chung and also reviewed libnative-platform-java and granted upload rights to him.
• I packaged new upstream releases of lombok-patcher, electric, undertow, sweethome3d and sweethome3d-furniture-editor.
• I spent quite some time on reviewing (especially the copyright review took most of the time) and improving the packaging for tycho (#816604) which is a precondition for packaging the latest upstream release of Eclipse, a popular Java IDE. Luca Vercelli has been working on it for the last couple of months and he did most of the initial packaging. Unfortunately I was only able to upload the package last week which means that the chances for updating Eclipse for Stretch are slim.
• Due to time constraints I could not finish the Netbeans update in time which I had started back in October. This is on my priority list for December now.
• Several security issues were reported against Tomcat 6,7,8 . I helped with reviewing some of the patches that Emmanuel prepared for Jessie and worked on fixing the same bugs in Wheezy.

Debian LTS This was my ninth month as a paid contributor and I have been paid to work 11 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following:

• From 14. November until 21. November I was in charge of our LTS frontdesk. I triaged bugs in teeworlds, libdbd-mysql-perl, bash, libxml2, tiff, firefox-esr, drupal7, moin, libgc, w3m and sniffit.
• DLA-715-1. Issued a security update for drupal7 fixing 2 CVE.
• DLA-717-1. Issued a security update for moin fixing 2 CVE.
• DLA-728-1. Issued a security update for tomcat6 fixing 8 CVE. (Debian bug #845385 was assigned a CVE later).
• DLA-729-1. Issued a security update for tomcat7 fixing 8 CVE. (Debian bug #845385 was assigned a CVE later).
• Especially the patches and the subsequent testing for CVE-2016-0762 and CVE-2016-6816 required most of the time.

Non-maintainer uploads

• I uploaded an NMU for angband to fix #837394. The patch was kindly prepared by Adrian Bunk.

It is already this time of the year again. See you next year for another report.

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Android, Java, Games and LTS topics, this might be interesting for you. Debian Android

• I sponsored and reviewed android-platform-external-libunwind and android-platform-frameworks-base for Kai-Chung. Two more packages, android-framework-23 and android-platform-tools-swt, are currently waiting in the NEW queue and hopefully they become available soon. Kai-Chung also became Debian Maintainer this month and I granted him upload permissions for android-platform-tools-base for a start. Congratulations.
• I packaged a new upstream release of apktool (2.2.1).

Debian Games

• I fixed RC bugs in lordsawar (#839323) and doomsday (#839338).
• I packaged new upstream releases of atanks, lordsawar, blockattack and peg-e.
• I completed the Bullet transition (#839243). Bullet 2.85 has also been released this month but it is now too late for Stretch because the transition freeze is already on the 5th of November. I expect more point releases a la 2.85.x during the coming weeks and I intend to provide an updated package in experimental soon.
• I did some cleanups, package upgrades and bug fixes for box2d and redeclipse (apparently redeclipse-server requires the -data package to be present now).
• I uploaded Redeclipse 1.5.6 to jessie-backports in the hope that more players will be able to connect to the multiplayer servers. Unfortunately network compatibility breaks rather frequently.
• I applied a patch from Gianfranco Costamagna to address an Multiarch installation issue (#841824) in FreeOrion.

Debian Java

• This month I tended to upgrade more Java packages including new upstream releases of libjide-oss-java, activemq, easymock, libapache-mod-jk, svnkit, sweethome3d-furniture-editor, sweethome3d-textures-editor, sweethome3d-furniture, sweethome3d, sweethome3d-furniture-nonfree, maven-enforcer, bsaf, libjcommon-java, libjfreechart-java, libcsv-java, jansi, jansi-native, jboss-xnio and undertow.
• The update of maven-enforcer caused a regression in its reverse-dependencies (#841197) and required a patch and upload of another revision. The Jfreechart library upgrade also required a compatibility patch for statcvs.
• I updated stegosuite and fixed an incorrect Section field in debian/control.
• I updated the Debian packaging of libjgraph-java and libjemmy2-java.
• I decided to work on getting Eclipse into shape again. The current version is outdated and affected by several bugs at the moment and I really think it should not be released with Stretch. The new version though requires a major package update because the build system is Maven based now. Luca Vercelli already worked on sat4j and Tycho, two preconditions for packaging Eclipse. I reviewed sat4j and uploaded an NMU (#815911) later. I m still working on Tycho (#816604) and I hope to finish it soon.
• Netbeans 8.2 was released. I have already completed the work on libnb-platform18-java (updated package is available in Git) but I haven t started yet with netbeans itself. I intend to continue the fun in November.

Debian LTS This was my eight month as a paid contributor and I have been paid to work 13 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following:

• From 10. October until 17. October I was in charge of our LTS frontdesk. I triaged bugs in libgd2, graphicsmagick, libxrender, mupdf, libxfixes, guile-2.0, glance, inspircd, libxi, libxv, libxst, spip, libxml2, libarchive and jasper.
• DLA-648-1. Issued a security update for c-ares fixing 1 CVE.
• DLA-664-1. Issued a security update for libxrender fixing 2 CVE.
• DLA-666-1. Issued a security update for guile-2.0 fixing 2 CVE.
• DLA-667-1. Issued a security update for libxv fixing 1 CVE.
• DLA-668-1. Issued a security update for libass fixing 2 CVE. I triaged CVE-2016-7970 and marked the version in Wheezy as not affected.
• DLA-673-1. Issued a security update for kdepimlibs fixing 1 CVE.

Non-maintainer uploads

• I fixed various RC bugs in gnudoq and xsok which are not maintained by the Games Team. The following games are available in Stretch again: gnudoq (#817296, #817484), xsok (#817738) and I also worked on four more bug fixes to improve the game s desktop integration and internationalization support.
• I fixed another RC bug in trackballs (#831119) but while I was working on the update I discovered that the game frequently segfaults which makes it kind of unplayable (#839788). I haven t found a solution yet but I suspect the switch to guile-2.0 and related patches introduced this behavior.

QA

• I uploaded a new revision of criticalmass and applied a patch from Adrian Bunk to fix #811816, a FTBFS.
• I triaged an RC bug for raptor2 (#824735) and the issue could be closed after the bug reporter confirmed that raptor2 built fine again.

What happened in the Reproducible Builds effort between Sunday September 25 and Saturday October 1 2016: Statistics For the first time, we reached 91% reproducible packages in our testing framework on testing/amd64 using a determistic build path. (This is what we recommend to make packages in Stretch reproducible.) For unstable/amd64, where we additionally test for reproducibility across different build paths we are at almost 76% again. IRC meetings We have a poll to set a time for a new regular IRC meeting. If you would like to attend, please input your available times and we will try to accommodate for you. There was a trial IRC meeting on Friday, 2016-09-31 1800 UTC. Unfortunately, we did not activate meetbot. Despite this participants consider the meeting a success as several topics where discussed (eg changes to IRC notifications of tests.r-b.o) and the meeting stayed within one our length. Upcoming events Reproduce and Verify Filesystems - Vincent Batts, Red Hat - Berlin (Germany), 5th October, 14:30 - 15:20 @ LinuxCon + ContainerCon Europe 2016. From Reproducible Debian builds to Reproducible OpenWrt, LEDE & coreboot - Holger "h01ger" Levsen and Alexander "lynxis" Couzens - Berlin (Germany), 13th October, 11:00 - 11:25 @ OpenWrt Summit 2016. Introduction to Reproducible Builds - Vagrant Cascadian will be presenting at the SeaGL.org Conference In Seattle (USA), November 11th-12th, 2016. Previous events GHC Determinism - Bartosz Nitka, Facebook - Nara (Japan), 24th September, ICPF 2016. Toolchain development and fixes Michael Meskes uploaded bsdmainutils/9.0.11 to unstable with a fix for #830259 based on Reiner Herrmann's patch. This fixed locale_dependent_symbol_order_by_lorder issue in the affected packages (freebsd-libs, mmh). devscripts/2.16.8 was uploaded to unstable. It includes a debrepro script by Antonio Terceiro which is similar in purpose to reprotest but more lightweight; specific to Debian packages and without support for virtual servers or configurable variations. Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible in our testing framework after being fixed:

• ara/1.0.32 by Chris Lamb, original patch by Chris Lamb.
• fracplanet/0.4.0-5 by Chris Lamb, original patch by Reiner Herrman.
• gnarwl/3.6.dfsg-8 by Bernhard Schmidt, original patch by Chris Lamb.
• kgb-bot/1.34-1 by gregor herrmann, original patch by gregor herrmann.
• survex/1.2.29-1 by Olly Betts.
• zpaq/1.10-3 by Chris Lamb, original patch by Reiner Herrman.
• fig2dev/1:3.2.6-3 by Roland Rosenfeld.
• luxio/11-1 by Daniel Silverstone.
• monkeysign/2.1.1 by Antoine Beaupr , original patch by Daniel Kahn Gillmor.
• openarena-085-data/0.8.5split-9 by Simon McVittie.
• openarena-088-data/0.8.8-7 by Simon McVittie.
• openarena-data/0.8.5split-9 by Simon McVittie.
• rc/1.7.4-1 by Reiner Herrmann, original patch by Chris Lamb.

The following updated packages appear to be reproducible now for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

• gkrellm/2.3.8-1 by Sandro Tosi
• glassfish/1:2.1.1-b31g+dfsg1-4 by Emmanuel Bourg

Some uploads have addressed some reproducibility issues, but not all of them:

• apache2/2.4.23-5 by Rapha l Hertzog
• freeradius/3.0.11+dfsg-1 by Michael Stapelberg
• libnss-ldap/265-4 by Chris Lamb
• lift/2.5.0-1 by Nicolas Delvaux
• linux/4.8~rc8-1~exp1 by Ben Hutchings
• nose2/0.6.5-2 by Barry Warsaw
• postgresql-9.6/9.6.0-1 Christoph Berg
• strace/4.13-0.1 by Nicolas Braud-Santoni
• yersinia/0.7.3-3 by No l K the

Patches submitted that have not made their way to the archive yet:

• #838888 filed against dh-haskell by Chris Lamb.
• #838971 filed against slang2 by Chris Lamb.
• #839587 filed against sympa by Chris Lamb.
• #839181 filed against transmission-remote-gtk by Chris Lamb.
• #838829 filed against vala by Sebastian Reichel.
• #838970 filed against webkit2pdf by Chris Lamb.
• #831569 filed against websockets by Chris Lamb.
• #839347 filed against xml-core by Lucas Nussbaum.
• #839526 filed against xml-core by Adrian Bunk.

Reviews of unreproducible packages 77 package reviews have been added, 178 have been updated and 80 have been removed in this week, adding to our knowledge about identified issues. 6 issue types have been updated:

• Added clilibs_line_order, records_build_flags and hevea_captures_build_path.
• Removed locale_dependent_symbol_order_by_lorder, fixed in bsdmainutils/9.0.11.
• Updated diffoscope_runs_forever, captures_build_path.

Weekly QA work As part of reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (3)
• Chris Lamb (12)
• Lucas Nussbaum (3)
• Sebastian Reichel (1)

diffoscope development A new version of diffoscope 61 was uploaded to unstable by Chris Lamb. It included contributions from:

• Ximin Luo:
  • Improve the CLI --help text and add an --output-empty option.
• Chris Lamb:
  • Add a progress bar and show it if stdout is a TTY. You can read more about it here. It can also be read by higher-level programs via the --status-fd CLI option.
• Maria Glukhova:
  • Behaviour improvements in the case of OS-level errors.
• Mattia Rizzolo:
  • Testing and packaging improvements.

Post-release there were further contributions from:

• Chris Lamb:
  • Code architecture improvements.
• Maria Glukhova:
  • Testing improvements.

reprotest development A new version of reprotest 0.3.2 was uploaded to unstable by Ximin Luo. It included contributions from:

• Ximin Luo:
  • Add a --diffoscope-arg CLI option to pass extra args to diffoscope.

Post-release there were further contributions from:

• Chris Lamb:
  • Code quality improvements.

tests.reproducible-builds.org

• Hans-Christoph Steiner continued work on setting up reproducible tests for F-Droid.
• Holger cleaned up the script creating the page showing breakages, so that it now also cleans up some of the breakage it finds.
• IRC notifications about diffoscope crashes and artifacts available for investigations have been dropped; instead the breakages page has a permanent pointer. (h01ger)
• IRC notifications from the automatic package scheduler and status changes for packages have been moved -- as a temporary trial -- to #debian-reproducible-changes on irc.oftc.net (Mattia).

Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

I recently dug through my history of involvement with TeX (Live), and found out that in January there are a lot of anniversaries I should celebrate: 14 years ago I started building binaries for TeX Live, 11 years ago I proposed the packaging TeX Live for Debian, 10 years ago the TeX Live packages entered Debian. There are other things to celebrate next year (2017), namely the 10 year anniversary of the (not so new anymore) infrastructure in short tlmgr of TeX Live packaging, but this will come later. In this blog post I want to concentrate on my involvement in TeX Live and Debian. Those of you not interested in boring and melancholic look-back onto history can safely skip reading this one. For those a bit interested in the history of TeX in Debian, please read on. Debian releases and TeX systems The TeX system of choice has been for long years teTeX, curated by Thomas Esser. Digging through the Debian Archive and combining it with changelog entries as well as personal experiences since I joined Debian, here is a time line of TeX in Debian, all to my best knowledge.

Date	Version	Name	teTeX/TeX Live	Maintainers
1993-96	<1	?	?	Christoph Martin
6/1996	1.1	Buzz	?
12/1996	1.2	Rec	?
6/1997	1.3	Bo	teTeX 0.4
7/1998	2.0	Ham	teTeX 0.9
3/1999	2.1	Slink	teTeX 0.9.9N
8/2000	2.2	Potato	teTeX 1.0
7/2002	3.0	Woody	teTeX 1.0
6/2005	3.1	Sarge	teTeX 2.0	Atsuhito Kohda
4/2007	4.0	Etch	teTeX 3.0, TeX Live 2005	Frank K ster, NP
2/2009	5.0	Lenny	TeX Live 2007	NP
2/2011	6.0	Squeeze	TeX Live 2009
5/2013	7.0	Whezzy	TeX Live 2012
4/2015	8.0	Jessie	TeX Live 2014
???	???	Stretch	TeX Live 2015

The history of TeX in Debian is thus split more or less in 10 years teTeX, and 10 years TeX Live. While I cannot check back to the origins, my guesses are that already in the very first releases (te)TeX was included. The first release I can confirm (via the Debian archive) shipping teTeX is the release Bo (June 1997). Maintainership during the first 10 years showed some fluctuation: The first years/releases (till about 2002) were dominated by Christoph Martin with Adrian Bunk and few others, who did most packaging work on teTeX version 1. After this Atsuhito Kohda with help from Hilmar Preusse and some people brought teTeX up to version 2, and from 2004 to 2007 Frank K ster, again with help of Hilmar Preusse and some other, took over most of the work on teTeX. Other names appearing throughout the changelog are (incomplete list) Julian Gilbey, Ralf Stubner, LaMont Jones, and C.M Connelly (and many more bug-reporters and fixers). Looking at the above table I have to mention the incredible amount of work that both Atsuhito Kohda and Frank K ster have put into the teTeX packages, and many of their contributions have been carried over into the TeX Live packages. While there haven t been many releases during their maintainership, their work has inspired and supported the packaging of TeX Live to a huge extend. Start of TeX Live I got involved in TeX Live back in 2002 when I started building binaries for the alpha-linux architecture. I can t remember when I first had the idea to package TeX Live for Debian, but here is a time line from my first email to the Debian Developers mailing list concerning TeX Live, to the first accepted upload:

Date	Subject/Link	Comment
2005-01-11	binaries for different architectures in debian packages	The first question concerning packaging TeX Live, about including pre-built binaries
2005-01-25	Debian-TeXlive Proposal II	A better proposal, but still including pre-built binaries
2005-05-17	Proposal for a tex-base package	Proposal for tex-base, later tex-common, as basis for both teTeX and TeX live packages
2015-06-10	Bug#312897: ITP: texlive	ITP bug for TeX Live
2005-09-17	Re: Take over of texinfo/info packages	Taking over texinfo which was somehow orphaned started here
2005-11-28	Re: texlive-basic_2005-1_i386.changes REJECTED	My answer to the rejection by ftp-master of the first upload. This email sparked a long discussion about packaging and helped improve the naming of packages (but not really the packaging itself).
2006-01-12	Upload of TeX Live 2005-1 to Debian	The first successful upload
2006-01-22	Accepted texlive-base 2005-1 (source all)	TeX Live packages accepted into Debian/experimental

One can see from the first emails that at that time I didn t have any idea about Debian packaging and proposed to ship the binaries built within the TeX Live system on Debian. What followed was first a long discussion about whether there is any need for just another TeX system. The then maintainer Frank K ster took a clear stance in favor of including TeX Live, and after several rounds of proposals, tests, rejections and improvements, the first successful upload of TeX Live packages to Debian/experimental happened on 12 January 2006, so exactly 10 years ago. Packaging Right from the beginning I used a meta-packaging approach. That is, instead of working directly with the source packages, I wrote (Perl) scripts that generated the source packages from a set of directives. There were several reasons why I choose to introduce this extra layer:

• The original format of the TeX Live packaging information (tpm) were xml files that were parsed with an XML parser (libxml). I guess (from what I have seen over the years) I was the only one ever properly parsing these .tpm files for packaging.
• TeX Live packages were often reshuffled, and Debian package name changed, which would have caused a certain level of pain for the creation of original tar files and packaging.
• Flexibility in creating additional packages and arbitrary dependencies

Till now I am not 100% sure whether it was the best idea, but the scripts remain in place till now, only adapted to the new packaging paradigm in TeX Live (without xml) and adding new functionality. This allows me to just kick off one script that does do all the work, including building .orig.tar.gz, source packages, binary packages. For those interested to follow the frantic activity during the first years, there is a file CHANGES.packaging which for the years from 2005 to 2011 documents very extensively the changes I made in these years. I don t want to count the hours the went into all this Development over the years TeX Live 2005 was just another TeX system but not the preferred one in Debian Etch and beyond. But then in May 2006, Thomas Esser announced the end of development of teTeX, which cleared the path for TeX Live as main TeX system in Debian (and the world!). The next release of Debian, Lenny (1/2009), already carried only TeX Live. Unfortunately it was only TeX Live 2007 and not 2008, mostly due to me having been involved in rewriting the upstream infrastructure based on Debian package files instead of the notorious xml files. This took quite a lot of attention and time from Debian away to upstream development, but this will be discussed in a different post. Similarly, the release of TeX Live included in Debian Squeeze (released 2/2011) was only TeX Live 2009 (instead of 2010), but since then (Wheezy and Jessie) the releases of TeX Live in Debian were always the latest released ones. Current status Since about 2013 I am trying to keep a regular schedule of new TeX Live packages every month. These helps me to keep up with the changes in upstream packaging and reduces the load of packaging a new release of TeX Live. It also bring to users of unstable and testing a very up-to-date TeX system, where packages at most lack 1 month of behind the TeX Live net updates. Future As most of the readers here know, besides caring for TeX (Live) and related packages in Debian, I am also responsible for the TeX Live Manager (tlmgr) and most of upstream s infrastructure including network distribution. Thus, my (spare, outside work) time needs to be distributed between all these projects (and some others) which leaves less and less time for Debian packaging. Fortunately the packaging is in a state that making regular updates once a month is less of a burden, since most steps are automatized. What is still a bit of a struggle is adapting the binary package (src:texlive-bin) to new releases. But also this has become simpler due to less invasive changes over the years. All in all, I don t have many plans for TeX Live in Debian besides keeping the current system running as it is. Search for and advise to future maintainers and collaborators I would be more than happy if new collaborators appear, with fresh ideas and some spare time. Unfortunately, my experience over these 10 years with people showing up and proposing changes (anyone remembers the guy proposing a complete rewrite in ML or so?) is that nobody really wants to invest time and energy, but search for quick solutions. This is not something that will work with a package like TeX Live, sized of several gigabyte (the biggest in the Debian archive), and complicated inner workings. I advise everyone being interested in helping to package TeX Live for Debian, to first install normal TeX Live from TUG, get used to what actions happen during updates (format rebuilds, hyphenation patters, map file updates). One does not need to have a perfect understanding of what exactly happens down there in the guts (I didn t have in the beginning, either), but if you want to help packaging and never heard about what format dumps or map files are, then this might be a slight obstacle. Conclusion TeX Live is the only TeX system in wide use across lots of architectures and operating systems, and the only comparable system, MikTeX, is Windows specific (also there are traces of ports to Unix). Backed by all the big user groups of TeX, TeX Live will remain the prime choice for the foreseeable future, and thus also TeX Live in Debian.