As we noted in an earlier article, passwords are a liability and we'd prefer to get rid of them, but the current reality is that we do use a plethora of passwords in our daily lives. This problem is especially acute for technology professionals, particularly system administrators, who have to manage a lot of different machines. But it also affects regular users who still use a large number of passwords, from their online bank to their favorite social-networking site. Despite the remarkable memory capacity of the human brain, humans are actually terrible at recalling even short sets of arbitrary characters with the precision needed for passwords. Therefore humans reuse passwords, make them trivial or guessable, write them down on little paper notes and stick them on their screens, or just reset them by email every time. Our memory is undeniably failing us and we need help, which is where password managers come in. Password managers allow users to store an arbitrary number of passwords and just remember a single password to unlock them all. But there is a large variety of password managers out there, so which one should we be using? At my previous job, an inventory was done of about 40 different free-software password managers in different stages of development and of varying quality. So, obviously, this article will not be exhaustive, but instead focus on a smaller set of some well-known options that may be interesting to readers.

KeePass: the popular alternative The most commonly used password-manager design pattern is to store passwords in a file that is encrypted and password-protected. The most popular free-software password manager of this kind is probably KeePass. An important feature of KeePass is the ability to auto-type passwords in forms, most notably in web browsers. This feature makes KeePass really easy to use, especially considering it also supports global key bindings to access passwords. KeePass databases are designed for simultaneous access by multiple users, for example, using a shared network drive. KeePass has a graphical interface written in C#, so it uses the Mono framework on Linux. A separate project, called KeePassX is a clean-room implementation written in C++ using the Qt framework. Both support the AES and Twofish encryption algorithms, although KeePass recently added support for the ChaCha20 cipher. AES key derivation is used to generate the actual encryption key for the database, but the latest release of KeePass also added using Argon2, which was the winner of the July 2015 password-hashing competition. Both programs are more or less equivalent, although the original KeePass seem to have more features in general. The KeePassX project has recently been forked into another project now called KeePassXC that implements a set of new features that are present in KeePass but missing from KeePassX like:

• auto-type on Linux, Mac OS, and Windows
• database merging which allows multi-user support
• using the web site's favicon in the interface

So far, the maintainers of KeePassXC seem to be open to re-merging the project "if the original maintainer of KeePassX in the future will be more active and will accept our merge and changes". I can confirm that, at the time of writing, the original KeePassX project now has 79 pending pull requests and only one pull request was merged since the last release, which was 2.0.3 in September 2016. While KeePass and derivatives allow multiple users to access the same database through the merging process, they do not support multi-party access to a single database. This may be a limiting factor for larger organizations, where you may need, for example, a different password set for different technical support team levels. The solution in this case is to use separate databases for each team, with each team using a different shared secret.

Pass: the standard password manager? I am currently using password-store, or pass, as a password manager. It aims to be "the standard Unix password manager". Pass is a GnuPG-based password manager that features a surprising number of features given its small size:

• copy-paste support
• Git integration
• multi-user/group support
• pluggable extensions (in the upcoming 1.7 release)

The command-line interface is simple to use and intuitive. The following, will, for example, create a pass repository, a 20 character password for your LWN account and copy it to the clipboard:

    $ pass init
    $ pass generate -c lwn 20

The main issue with pass is that it doesn't encrypt the name of those entries: if someone were to compromise my machine, they could easily see which sites I have access to simply by listing the passwords stored in ~/.password-store. This is a deliberate design decision by the upstream project, as stated by a mailing list participant, Allan Odgaard:

Using a single file per item has the advantage of shell completion, using version control, browse, move and rename the items in a file browser, edit them in a regular editor (that does GPG, or manually run GPG first), etc.

Odgaard goes on to point out that there are alternatives that do encrypt the entire database (including the site names) if users really need that feature. Furthermore, there is a tomb plugin for pass that encrypts the password store in a LUKS container (called a "tomb"), although it requires explicitly opening and closing the container, which makes it only marginally better than using full disk encryption system-wide. One could also argue that password file names do not hold secret information, only the site name and username, perhaps, and that doesn't require secrecy. I do believe those should be kept secret, however, as they could be used to discover (or prove) which sites you have access to and then used to perform other attacks. One could draw a parallel with the SSH known_hosts file, which used to be plain text but is now hashed so that hosts are more difficult to discover. Also, sharing a database for multi-user support will require some sort of file-sharing mechanism. Given the integrated Git support, this will likely involve setting up a private Git repository for your team, something which may not be accessible to the average Linux user. Nothing keeps you, however, from sharing the ~/.password-store directory through another file sharing mechanism like (say) Syncthing or Dropbox. You can use multiple distinct databases easily using the PASSWORD_STORE_DIR environment variable. For example, you could have a shell alias to use a different repository for your work passwords with:

    alias work-pass="PASSWORD_STORE_DIR=~/work-passwords pass"

Group support comes from a clever use of the GnuPG multiple-recipient encryption support. You simply have to specify multiple OpenPGP identities when initializing the repository, which also works in subdirectories:

    $ pass init -p Ateam me@example.com joelle@example.com
    mkdir: created directory '/home/me/.password-store/Ateam'
    Password store initialized for me@example.com, joelle@example.com
    [master 0e3dbe7] Set GPG id to me@example.com, joelle@example.com.
     1 file changed, 2 insertions(+)
     create mode 100644 Ateam/.gpg-id

The above will configure pass to encrypt the passwords in the Ateam directory for me@example.com and joelle@example.com. Pass depends on GnuPG to do the right thing when encrypting files and how those identities are treated is entirely delegated to GnuPG's default configuration. This could lead to problems if arbitrary keys can be injected into your key ring, which could confuse GnuPG. I would therefore recommend using full key fingerprints instead of user identifiers. Regarding the actual encryption algorithms used, in my tests, GnuPG 1.4.18 and 2.1.18 seemed to default to 256-bit AES for encryption, but that has not always been the case. The chosen encryption algorithm actually depends on the recipient's key preferences, which may vary wildly: older keys and versions may use anything from 128-bit AES to CAST5 or Triple DES. To figure out which algorithm GnuPG chose, you may want to try this pipeline:

    $ echo test   gpg -e -r you@example.com   gpg -d -v
    [...]
    gpg: encrypted with 2048-bit RSA key, ID XXXXXXX, created XXXXX
      "You Person You <you@example.com>"
    gpg: AES256 encrypted data
    gpg: original file name=''
    test

As you can see, pass is primarily a command-line application, which may make it less accessible to regular users. The community has produced different graphical interfaces that are either using pass directly or operate on the storage with their own GnuPG integration. I personally use pass in combination with Rofi to get quick access to my passwords, but less savvy users may want to try the QtPass interface, which should be more user-friendly. QtPass doesn't actually depend on pass and can use GnuPG directly to interact with the pass database; it is available for Linux, BSD, OS X, and Windows.

Browser password managers Most users are probably already using a password manager through their web browser's "remember password" functionality. For example, Chromium will ask if you want it to remember passwords and encrypt them with your operating system's facilities. For Windows, this encrypts the passwords with your login password and, for GNOME, it will store the passwords in the gnome-keyring storage. If you synchronize your Chromium settings with your Google account, Chromium will store those passwords on Google's servers, encrypted with a key that is stored in the Google Account itself. So your passwords are then only as safe as your Google account. Note that this was covered here in 2010, although back then Chromium didn't synchronize with the Google cloud or encrypt with the system-level key rings. That facility was only added in 2013. In Firefox, there's an optional, profile-specific master password that unlocks all passwords. In this case, the issue is that browsers are generally always open, so the vault is always unlocked. And this is for users that actually do pick a master password; users are often completely unaware that they should set one. The unlocking mechanism is a typical convenience-security trade-off: either users need to constantly input their master passwords to login or they don't, and the passwords are available in the clear. In this case, Chromium's approach of actually asking users to unlock their vault seems preferable, even though the developers actually refused to implement the feature for years. Overall, I would recommend against using a browser-based password manager. Even if it is not used for critical sites, you will end up with hundreds of such passwords that are vulnerable while the browser is running (in the case of Firefox) or at the whim of Google (in the case of Chromium). Furthermore, the "auto-fill" feature that is often coupled with browser-based password managers is often vulnerable to serious attacks, which is mentioned below. Finally, because browser-based managers generally lack a proper password generator, users may fail to use properly generated passwords, so they can then be easily broken. A password generator has been requested for Firefox, according to this feature request opened in 2007, and there is a password generator in Chrome, but it is disabled by default and hidden in the mysterious chrome://flags URL.

Other notable password managers Another alternative password manager, briefly mentioned in the previous article, is the minimalistic Assword password manager that, despite its questionable name, is also interesting. Its main advantage over pass is that it uses a single encrypted JSON file for storage, and therefore doesn't leak the name of the entries by default. In addition to copy/paste, Assword also supports automatically entering passphrases in fields using the xdo library. Like pass, it uses GnuPG to encrypt passphrases. According to Assword maintainer Daniel Kahn Gillmor in email, the main issue with Assword is "interaction between generated passwords and insane password policies". He gave the example of the Time-Warner Cable registration form that requires, among other things, "letters and numbers, between 8 and 16 characters and not repeat the same characters 3 times in a row". Another well-known password manager is the commercial LastPass service which released a free-software command-line client called lastpass-cli about three years ago. Unfortunately, the server software of the lastpass.com service is still proprietary. And given that LastPass has had at least two serious security breaches since that release, one could legitimately question whether this is a viable solution for storing important secrets. In general, web-based password managers expose a whole new attack surface that is not present in regular password managers. A 2014 study by University of California researchers showed that, out of five password managers studied, every one of them was vulnerable to at least one of the vulnerabilities studied. LastPass was, in particular, vulnerable to a cross-site request forgery (CSRF) attack that allowed an attacker to bypass account authentication and access the encrypted database.

Problems with password managers When you share a password database within a team, how do you remove access to a member of the team? While you can, for example, re-encrypt a pass database with new keys (thereby removing or adding certain accesses) or change the password on a KeePass database, a hostile party could have made a backup of the database before the revocation. Indeed, in the case of pass, older entries are still in the Git history. So access revocation is a problematic issue found with all shared password managers, as it may actually mean going through every password and changing them online. This fundamental problem with shared secrets can be better addressed with a tool like Vault or SFLvault. Those tools aim to provide teams with easy ways to store dynamic tokens like API keys or service passwords and share them not only with other humans, but also make them accessible to machines. The general idea of those projects is to store secrets in a central server and send them directly to relevant services without human intervention. This way, passwords are not actually shared anymore, which is similar in spirit to the approach taken by centralized authentication systems like Kerberos. If you are looking at password management for teams, those projects may be worth a look. Furthermore, some password managers that support auto-typing were found to be vulnerable to HTML injection attacks: if some third-party ad or content is able to successfully hijack the parent DOM content, it masquerades as a form that could fool auto-typing software as demonstrated by this paper that was submitted at USENIX 2014. Fortunately, KeePass was not vulnerable according to the security researchers, but LastPass was, again, vulnerable.

Future of password managers? All of the solutions discussed here assume you have a trusted computer you regularly have access to, which is a usage pattern that seems to be disappearing with a majority of the population. You could consider your phone to be that trusted device, yet a phone can be lost or stolen more easily than a traditional workstation or even a laptop. And while KeePass has Android and iOS ports, those do not resolve the question of how to share the password storage among those devices or how to back them up. Password managers are fundamentally file-based, and the "file" concept seems to be quickly disappearing, faster than we technologists sometimes like to admit. Looking at some relatives' use of computers, I notice it is less about "files" than images, videos, recipes, and various abstract objects that are stored in the "cloud". They do not use local storage so much anymore. In that environment, password managers lose their primary advantage, which is a local, somewhat offline file storage that is not directly accessible to attackers. Therefore certain password managers are specifically designed for the cloud, like LastPass or web browser profile synchronization features, without necessarily addressing the inherent issues with cloud storage and opening up huge privacy and security issues that we absolutely need to address. This is where the "password hasher" design comes in. Also known as "stateless" or "deterministic" password managers, password hashers are emerging as a convenient solution that could possibly replace traditional password managers as users switch from generic computing platforms to cloud-based infrastructure. We will cover password hashers and the major security challenges they pose in a future article.

Note: this article first appeared in the Linux Weekly News.

As we noted in an earlier article, passwords are a liability and we'd prefer to get rid of them, but the current reality is that we do use a plethora of passwords in our daily lives. This problem is especially acute for technology professionals, particularly system administrators, who have to manage a lot of different machines. But it also affects regular users who still use a large number of passwords, from their online bank to their favorite social-networking site. Despite the remarkable memory capacity of the human brain, humans are actually terrible at recalling even short sets of arbitrary characters with the precision needed for passwords. Therefore humans reuse passwords, make them trivial or guessable, write them down on little paper notes and stick them on their screens, or just reset them by email every time. Our memory is undeniably failing us and we need help, which is where password managers come in. Password managers allow users to store an arbitrary number of passwords and just remember a single password to unlock them all. But there is a large variety of password managers out there, so which one should we be using? At my previous job, an inventory was done of about 40 different free-software password managers in different stages of development and of varying quality. So, obviously, this article will not be exhaustive, but instead focus on a smaller set of some well-known options that may be interesting to readers.

KeePass: the popular alternative The most commonly used password-manager design pattern is to store passwords in a file that is encrypted and password-protected. The most popular free-software password manager of this kind is probably KeePass. An important feature of KeePass is the ability to auto-type passwords in forms, most notably in web browsers. This feature makes KeePass really easy to use, especially considering it also supports global key bindings to access passwords. KeePass databases are designed for simultaneous access by multiple users, for example, using a shared network drive. KeePass has a graphical interface written in C#, so it uses the Mono framework on Linux. A separate project, called KeePassX is a clean-room implementation written in C++ using the Qt framework. Both support the AES and Twofish encryption algorithms, although KeePass recently added support for the ChaCha20 cipher. AES key derivation is used to generate the actual encryption key for the database, but the latest release of KeePass also added using Argon2, which was the winner of the July 2015 password-hashing competition. Both programs are more or less equivalent, although the original KeePass seem to have more features in general. The KeePassX project has recently been forked into another project now called KeePassXC that implements a set of new features that are present in KeePass but missing from KeePassX like:

• auto-type on Linux, Mac OS, and Windows
• database merging which allows multi-user support
• using the web site's favicon in the interface

So far, the maintainers of KeePassXC seem to be open to re-merging the project "if the original maintainer of KeePassX in the future will be more active and will accept our merge and changes". I can confirm that, at the time of writing, the original KeePassX project now has 79 pending pull requests and only one pull request was merged since the last release, which was 2.0.3 in September 2016. While KeePass and derivatives allow multiple users to access the same database through the merging process, they do not support multi-party access to a single database. This may be a limiting factor for larger organizations, where you may need, for example, a different password set for different technical support team levels. The solution in this case is to use separate databases for each team, with each team using a different shared secret.

Pass: the standard password manager? I am currently using password-store, or pass, as a password manager. It aims to be "the standard Unix password manager". Pass is a GnuPG-based password manager that features a surprising number of features given its small size:

• copy-paste support
• Git integration
• multi-user/group support
• pluggable extensions (in the upcoming 1.7 release)

The command-line interface is simple to use and intuitive. The following, will, for example, create a pass repository, a 20 character password for your LWN account and copy it to the clipboard:

    $ pass init
    $ pass generate -c lwn 20

The main issue with pass is that it doesn't encrypt the name of those entries: if someone were to compromise my machine, they could easily see which sites I have access to simply by listing the passwords stored in ~/.password-store. This is a deliberate design decision by the upstream project, as stated by a mailing list participant, Allan Odgaard:

Using a single file per item has the advantage of shell completion, using version control, browse, move and rename the items in a file browser, edit them in a regular editor (that does GPG, or manually run GPG first), etc.

Odgaard goes on to point out that there are alternatives that do encrypt the entire database (including the site names) if users really need that feature. Furthermore, there is a tomb plugin for pass that encrypts the password store in a LUKS container (called a "tomb"), although it requires explicitly opening and closing the container, which makes it only marginally better than using full disk encryption system-wide. One could also argue that password file names do not hold secret information, only the site name and username, perhaps, and that doesn't require secrecy. I do believe those should be kept secret, however, as they could be used to discover (or prove) which sites you have access to and then used to perform other attacks. One could draw a parallel with the SSH known_hosts file, which used to be plain text but is now hashed so that hosts are more difficult to discover. Also, sharing a database for multi-user support will require some sort of file-sharing mechanism. Given the integrated Git support, this will likely involve setting up a private Git repository for your team, something which may not be accessible to the average Linux user. Nothing keeps you, however, from sharing the ~/.password-store directory through another file sharing mechanism like (say) Syncthing or Dropbox). You can use multiple distinct databases easily using the PASSWORD_STORE_DIR environment variable. For example, you could have a shell alias to use a different repository for your work passwords with:

    alias work-pass="PASSWORD_STORE_DIR=~/work-passwords pass"

Group support comes from a clever use of the GnuPG multiple-recipient encryption support. You simply have to specify multiple OpenPGP identities when initializing the repository, which also works in subdirectories:

    $ pass init -p Ateam me@example.com joelle@example.com
    mkdir: created directory '/home/me/.password-store/Ateam'
    Password store initialized for me@example.com, joelle@example.com
    [master 0e3dbe7] Set GPG id to me@example.com, joelle@example.com.
     1 file changed, 2 insertions(+)
     create mode 100644 Ateam/.gpg-id

The above will configure pass to encrypt the passwords in the Ateam directory for me@example.com and joelle@example.com. Pass depends on GnuPG to do the right thing when encrypting files and how those identities are treated is entirely delegated to GnuPG's default configuration. This could lead to problems if arbitrary keys can be injected into your key ring, which could confuse GnuPG. I would therefore recommend using full key fingerprints instead of user identifiers. Regarding the actual encryption algorithms used, in my tests, GnuPG 1.4.18 and 2.1.18 seemed to default to 256-bit AES for encryption, but that has not always been the case. The chosen encryption algorithm actually depends on the recipient's key preferences, which may vary wildly: older keys and versions may use anything from 128-bit AES to CAST5 or Triple DES. To figure out which algorithm GnuPG chose, you may want to try this pipeline:

    $ echo test   gpg -e -r you@example.com   gpg -d -v
    [...]
    gpg: encrypted with 2048-bit RSA key, ID XXXXXXX, created XXXXX
      "You Person You <you@example.com>"
    gpg: AES256 encrypted data
    gpg: original file name=''
    test

As you can see, pass is primarily a command-line application, which may make it less accessible to regular users. The community has produced different graphical interfaces that are either using pass directly or operate on the storage with their own GnuPG integration. I personally use pass in combination with Rofi to get quick access to my passwords, but less savvy users may want to try the QtPass interface, which should be more user-friendly. QtPass doesn't actually depend on pass and can use GnuPG directly to interact with the pass database; it is available for Linux, BSD, OS X, and Windows.

Browser password managers Most users are probably already using a password manager through their web browser's "remember password" functionality. For example, Chromium will ask if you want it to remember passwords and encrypt them with your operating system's facilities. For Windows, this encrypts the passwords with your login password and, for GNOME, it will store the passwords in the gnome-keyring storage. If you synchronize your Chromium settings with your Google account, Chromium will store those passwords on Google's servers, encrypted with a key that is stored in the Google Account itself. So your passwords are then only as safe as your Google account. Note that this was covered here in 2010, although back then Chromium didn't synchronize with the Google cloud or encrypt with the system-level key rings. That facility was only added in 2013. In Firefox, there's an optional, profile-specific master password that unlocks all passwords. In this case, the issue is that browsers are generally always open, so the vault is always unlocked. And this is for users that actually do pick a master password; users are often completely unaware that they should set one. The unlocking mechanism is a typical convenience-security trade-off: either users need to constantly input their master passwords to login or they don't, and the passwords are available in the clear. In this case, Chromium's approach of actually asking users to unlock their vault seems preferable, even though the developers actually refused to implement the feature for years. Overall, I would recommend against using a browser-based password manager. Even if it is not used for critical sites, you will end up with hundreds of such passwords that are vulnerable while the browser is running (in the case of Firefox) or at the whim of Google (in the case of Chromium). Furthermore, the "auto-fill" feature that is often coupled with browser-based password managers is often vulnerable to serious attacks, which is mentioned below. Finally, because browser-based managers generally lack a proper password generator, users may fail to use properly generated passwords, so they can then be easily broken. A password generator has been requested for Firefox, according to this feature request opened in 2007, and there is a password generator in Chrome, but it is disabled by default and hidden in the mysterious chrome://flags URL.

Other notable password managers Another alternative password manager, briefly mentioned in the previous article, is the minimalistic Assword password manager that, despite its questionable name, is also interesting. Its main advantage over pass is that it uses a single encrypted JSON file for storage, and therefore doesn't leak the name of the entries by default. In addition to copy/paste, Assword also supports automatically entering passphrases in fields using the xdo library. Like pass, it uses GnuPG to encrypt passphrases. According to Assword maintainer Daniel Kahn Gillmor in email, the main issue with Assword is "interaction between generated passwords and insane password policies". He gave the example of the Time-Warner Cable registration form that requires, among other things, "letters and numbers, between 8 and 16 characters and not repeat the same characters 3 times in a row". Another well-known password manager is the commercial LastPass service which released a free-software command-line client called lastpass-cli about three years ago. Unfortunately, the server software of the lastpass.com service is still proprietary. And given that LastPass has had at least two serious security breaches since that release, one could legitimately question whether this is a viable solution for storing important secrets. In general, web-based password managers expose a whole new attack surface that is not present in regular password managers. A 2014 study by University of California researchers showed that, out of five password managers studied, every one of them was vulnerable to at least one of the vulnerabilities studied. LastPass was, in particular, vulnerable to a cross-site request forgery (CSRF) attack that allowed an attacker to bypass account authentication and access the encrypted database.

Problems with password managers When you share a password database within a team, how do you remove access to a member of the team? While you can, for example, re-encrypt a pass database with new keys (thereby removing or adding certain accesses) or change the password on a KeePass database, a hostile party could have made a backup of the database before the revocation. Indeed, in the case of pass, older entries are still in the Git history. So access revocation is a problematic issue found with all shared password managers, as it may actually mean going through every password and changing them online. This fundamental problem with shared secrets can be better addressed with a tool like Vault or SFLvault. Those tools aim to provide teams with easy ways to store dynamic tokens like API keys or service passwords and share them not only with other humans, but also make them accessible to machines. The general idea of those projects is to store secrets in a central server and send them directly to relevant services without human intervention. This way, passwords are not actually shared anymore, which is similar in spirit to the approach taken by centralized authentication systems like Kerberos). If you are looking at password management for teams, those projects may be worth a look. Furthermore, some password managers that support auto-typing were found to be vulnerable to HTML injection attacks: if some third-party ad or content is able to successfully hijack the parent DOM content, it masquerades as a form that could fool auto-typing software as demonstrated by this paper that was submitted at USENIX 2014. Fortunately, KeePass was not vulnerable according to the security researchers, but LastPass was, again, vulnerable.

Future of password managers? All of the solutions discussed here assume you have a trusted computer you regularly have access to, which is a usage pattern that seems to be disappearing with a majority of the population. You could consider your phone to be that trusted device, yet a phone can be lost or stolen more easily than a traditional workstation or even a laptop. And while KeePass has Android and iOS ports, those do not resolve the question of how to share the password storage among those devices or how to back them up. Password managers are fundamentally file-based, and the "file" concept seems to be quickly disappearing, faster than we technologists sometimes like to admit. Looking at some relatives' use of computers, I notice it is less about "files" than images, videos, recipes, and various abstract objects that are stored in the "cloud". They do not use local storage so much anymore. In that environment, password managers lose their primary advantage, which is a local, somewhat offline file storage that is not directly accessible to attackers. Therefore certain password managers are specifically designed for the cloud, like LastPass or web browser profile synchronization features, without necessarily addressing the inherent issues with cloud storage and opening up huge privacy and security issues that we absolutely need to address. This is where the "password hasher" design comes in. Also known as "stateless" or "deterministic" password managers, password hashers are emerging as a convenient solution that could possibly replace traditional password managers as users switch from generic computing platforms to cloud-based infrastructure. We will cover password hashers and the major security challenges they pose in a future article.

Note: this article first appeared in the Linux Weekly News.

Better late than never, we'd like to welcome our three Outreachy interns for this round, lasting from the 6th of December 2016 to the 6th of March 2017. Elizabeth Ferdman is working in the Clean Room for PGP and X.509 (PKI) Key Management. Maria Glukhova is working in Reproducible builds for Debian and free software. Urvika Gola is working in improving voice, video and chat communication with free software. From the official website: Outreachy helps people from groups underrepresented in free and open source software get involved. We provide a supportive community for beginning to contribute any time throughout the year and offer focused internship opportunities twice a year with a number of free software organizations. The Outreachy program is possible in Debian thanks to the effort of Debian developers and contributors that dedicate part of their free time to mentor students and outreach tasks, and the help of the Software Freedom Conservancy, who provides administrative support for Outreachy, as well as the continued support of Debian's donors, who provide funding for the internships. Debian will also participate in the next round for Outreachy, during the summer of 2017. More details will follow in the next weeks. Join us and help extend Debian! You can follow the work of the Outreachy interns reading their blogs (they are syndicated in Planet Debian), and chat with us in the #debian-outreach IRC channel and mailing list. Congratulations, Elizabeth, Maria and Urvika!

What happened in the Reproducible Builds effort between Sunday January 15 and Saturday January 21 2017: Media Coverage

• Valerie Young presented Reproducible Builds for a Better Future (video) at linux.conf.au 2017.
• Chris Lamb presented Reproducible Builds: Two years in the trenches (video) at linux.conf.au 2017.

Upcoming Events

• The Reproducible Build Zoo will be presented by Vagrant Cascadian at the Embedded Linux Conference in Portland, Oregon, February 22nd.
• Dennis Gilmore and Holger Levsen will present on "Reproducible Builds and Fedora" at Devconf.cz on February 27th.
• Introduction to Reproducible Builds will be presented by Vagrant Cascadian at Scale15x in Pasadena, California, March 5th.
• Verifying Software Freedom with Reproducible Builds will be presented by Vagrant Cascadian at Libreplanet2017 in Boston, March 25th-26th.

Toolchain development and fixes Ximin Luo continued work on data formats, code, and test cases for SOURCE_PREFIX_MAP. He also continued to talk with the rustc team on the topic. Chris Lamb submitted a patch to implement SOURCE_DATE_EPOCH for wordwarvi, a game which gave extra points to people who built it from source within one hour. This fixes Debian #786593. Launchpad bug 1657704 was filed for them to start accepting buildinfo files. Bugs filed

• Chris Lamb:
  • #851809 filed against mono.
• Daniel Shahaf:
  • #851764 filed against hhsuite.

Reviews of unreproducible packages 10 package reviews have been added, 149 have been updated and 153 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been updated:

• ftbfs_due_to_jenkins_semaphore_setup
• clilibs_line_order

Weekly QA work During our reproducibility testing, the following FTBFS bugs have been detected and reported by:

• Chris Lamb (3)
• Ond ej Kobli ek (1)

diffoscope development diffoscope 69 was uploaded to unstable by Chris Lamb. It included contributions from:

• Maria Glukhova:
  • Bug fixes: #850055, #850501.
  • Improve comparison of APK files: #850502.
  • Add comparison of image metadata: #849395.
• Chris Lamb:
  • Many bug fixes: #849411, #850485, #850807, #850850, #851588.
  • Add comparison of ICO files: #850730.
  • Optimisations: disable profiling unless needed, and pre-compile regexes.
  • Many code quality improvements.
• Mattia Rizzolo:
  • Deduplicate code for recognising file types based on RE_FILE_TYPE and RE_FILE_EXTENSION.
  • Improve code quality in tests.

Further development continued in Git, and will be released as version 70 next week:

• Chris Lamb:
  • Add tests for --html-dir output and improve code quality elsewhere in tests.
  • Add markdown and reStructuredText output, as well as tests for these.
  • Improve software architecture of presenters.
  • Fix error-checking in the Haskell comparator.
• James Clarke:
  • Haskell comparator: properly extract version from interface files.
• Mattia Rizzolo:
  • Improve some documentation.
• Brett Smith:
  • Improve documentation including --help output.

reproducible-builds.org website development

• Brett Smith:
  • berlin2016: List Conservancy consistently as a participant.
• Chris Lamb:
  • Add Valerie's talk to resources page.
• Daniel Shahaf:
  • Improved the "How to chair a meeting" section.

tests.reproducible-builds.org

• Holger added arm64 to https://tests.reproducible-builds.org/debian/index_variations.html
• Mattia improved our process for building the performance page so that stats for new architectures are computed correctly without manual intervention.
• Holger enhanced the build node maintenance scripts to correctly detect if /dev/shm is mounted incorrectly (due to #851427) and deployed an /etc/rc.local startup script to all systems which works around it. As a result, jenkins_semaphore_setup_issue should be obsolete.
• Mattia improved the diskspace monitoring visible at our munin page for the 44 nodes we're currently running.
• Holger added 6GB more RAM to jenkins.debian.net, for a total of 64GB RAM, to better cope with the new jobs due to arm64. As usual, thanks to profitbricks.com for the hardware resources enabling this work.

Misc. This week's edition was written by Ximin Luo, Vagrant Cascadian, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Human languages, part 1 I do enjoy writing blog posts, but sometimes time is lacking, other times inspiration. As I was eating dinner today, I was lost in thoughts and my eyes stopped on the documentation from a certain medication for coughing (the thing almost noone reads, like EULAs). I was surprised quite surprised with te text that I said well, some blog posts on languages might be interesting . Background: My mother tongue is Romanian. While growing up and learning foreign languages, I considered only the utilitarian aspect of languages, but as I get older (not old, older! ), I find human languages more and more interesting. Back to the subject: this being Switzerland, most everything is written in German, French and Italian (in this order of frequency). German is still a foreign language to me (let's say I get by, but not nicely), French is the fancy high class cousin, and Italian Well Italian is a special case. When I first saw Italian on TV (a newscast while travelling in Italy) I was shocked at how much one can understand without learning Italian in any way (much, much more than German after years of trying). So Italian language is quite close, and usually one understands a third to half; this is both spoken (less in common language, more in official language) and in writing. In this particular case, the instructions of use say (sorry for typos, manually copying):

In casi rarissimi possono manifestarsi reazioni di ipersensibilit grave con tumefazione del viso, difficolt respiratoria (dispnea) e diminuzione della pressione arteriosa.

What was surprising here was not the list of side effects (hah), but that this short phrase is 98% identical to the translation in Romanian; not only words, but also phrase structure. I don't think I've ever seen this before:

n cazuri rare se pot manifesta reac ii de hipersensibilitate grav cu tumefac ie a fe ei, dificultate respiratorie (dispnee) i diminuare a presiunii arteriale.

Not all the words are identical, but even the one that is obviously different (it. 'del viso', ro. 'a fe ei') is easily translatable as 'visiune' in Romanian means 'to see', so the link is clear. This phrase structure is also quite a natural way to say the things in Romanian. I was then curious to see the French version, which is:

Dans de tr s rares cas, [the medicine] peut d clencher de violentes r actions d'hypersensibilit s'accompagnant d'un gonflement du visage, de d tresse respiratorie e d'une chute de tension.

French is usually quite different from Romanian that one has to learn it (for quite a while, especially for grammar) in order to be proficient in it, but here you can make also word-by-word translation (transposition?) that doesn't lose the meaning:

n cazuri tare rari, [the medicime] poate declan a violente reac ii de hypersensibilitate acompaniate de o umflare a fe ei, de ???? respiratorie i de ???? a tensiunii.

Basically here we have two non-equivalent words, and a bit more wierd phrase structure it sounds more like coloquial speech than written language but for French is also surprisingly close. You'd invert some of the adjective-noun pairs (fr. 'violentes r actions' is understandable in Romanian as 'violente reac ii', but it sounds very poetical and you'd usually write it as 'reac ii violente'). The next phrase is no longer that similar, but the one after is again obviously identical:

Se osserva effetti collaterali qui non descritti, dovrebbe informare il suo medico, il suo farmacista o il suo drogerie.

which is:

[Dac /If] se observ effecte colaterale care nu sunt descrise, trebuie informat medicul vostru, farmacistul vostru sau [xxxx - no real equivalent].

And the French is also identical, modulo again 'droguiste':

Si vouz remarquez de effets secondaires qui ne sont pas mentionn s dans cette notice, veuillez en informer votre m decin, votre pharmacien ou votre droguiste.

which is:

Dac remarca i efecte secundare care nu sunt men ionate n aceast not , informa i medicul vostru, farmacistul vostru sau al vostru [xxxx].

This is even closer; 'votre' is more similar to '[al] vostru' than 'suo', and the phrase structure is much more natural - this is exactly how you'd write it in 'native' Romanian, whereas the Italian is not (I had to add the 'if' to make it parseable). The 'vous' in 'vouz remarquez' is 'voi' in Romanian, but doesn't need to be added as it would be redundant; but it doesn't confuse the phrase. The 'veuillez en informer' doesn't have a 1:1 translation (it would be written as 'v rug m s informa i'), but is still understandable; a false friend translation would be 'vede i s informa i'/see to inform/voir informer. Why is this all surprising? Because Romanian has a significant amount of words of Slavic origin (~11% in overall vocabulary, 15% in most commonly used 2500 words) and some from other nearby countries (Turkey, Greek, some Hungarian and German). At a stretch, it's even possible to write simple but complete sentences entirely with words from Slavic origin, as I learned from this interesting youtube video. Also, our accent is almost always confused with Russian, not with Italian or French. So to get to a summary: normally you see sentence elements that are similar or identical, but not entire sentences, and definitely not phrases. What made the three languages here keep, in this particular case, not only similar but almost identical words and also almost identical phrase structure? Is it the subject (medicine)? Maybe. Is it a random fluke? If so, I don't remember seeing it before. Do I just see similarities where there are none? Possibly , but at least I thought it worth mentioning; it was quite surprising to me. Did my brain get confused by too many languages and I misinterpreted words that don't really exist (e.g. I was sure that 'vizajul' is a Romanian word, but upon checking, it isn't )? Also possible. In any case, for me it was a good subject for a blog post. Now let's not go near Spanish and definitely not near Portuguese

What happened in the Reproducible Builds effort between Sunday January 8 and Saturday January 14 2017: Upcoming Events

• The Reproducible Build Zoo will be presented by Vagrant Cascadian at the Embedded Linux Conference in Portland, Oregon, February 22nd
• Dennis Gilmore and Holger Levsen will present about "Reproducible Builds and Fedora" at Devconf.cz on February, 27th.
• Introduction to Reproducible Builds will be presented by Vagrant Cascadian at Scale15x in Pasadena, California, March 5th

Reproducible work in other projects Reproducible Builds have been mentioned in the FSF high-priority project list. The F-Droid Verification Server has been launched. It rebuilds apps from source that were built by f-droid.org and checks that the results match. Bernhard M. Wiedemann did some more work on reproducibility for openSUSE. Bootstrappable.org (unfortunately no HTTPS yet) was launched after the initial work was started at our recent summit in Berlin. This is another topic related to reproducible builds and both will be needed in order to perform "Diverse Double Compilation" in practice in the future. Toolchain development and fixes Ximin Luo researched data formats for SOURCE_PREFIX_MAP and explored different options for encoding a map data structure in a single environment variable. He also continued to talk with the rustc team on the topic. Daniel Shahaf filed #851225 ('udd: patches: index by DEP-3 "Forwarded" status') to make it easier to track our patches. Chris Lamb forwarded #849972 upstream to yard, a Ruby documentation generator. Upstream has fixed the issue as of release 0.9.6. Alexander Couzens (lynxis) has made mksquashfs reproducible and is looking for testers. It compiles on BSD systems such as FreeBSD, OpenBSD and NetBSD. Bugs filed Chris Lamb:

• #850683 filed against gerbv.

Lucas Nussbaum:

• #850973 filed against shogun.

Nicola Corna:

• #851190 filed against gerbv.

Reviews of unreproducible packages 13 package reviews have been added and 13 have been removed in this week, adding to our knowledge about identified issues. 1 issue type has been added:

• help2man_puts_traceback_in_generated_man_page

Weekly QA work During our reproducibility testing, the following FTBFS bugs have been detected and reported by:

• Chris Lamb (3)
• Lucas Nussbaum (11)
• Nicola Corna (1)

diffoscope development Many bugs were opened in diffoscope during the past few weeks, which probably is a good sign as it shows that diffoscope is much more widely used than a year ago. We have been working hard to squash many of them in time for Debian stable, though we will see how that goes in the end

• Mattia Rizzolo:
  • Code quality and style improvements.
• Maria Glukhova:
  • Add image metadata comparison.
  • Zipinfo included in APK files comparison. (Closes: #850502)
  • Remove archive name from apktool.yml and rename it. (Closes: #850501)
• Chris Lamb:
  • Correctly escape value of href="" elements (re. #849411)
  • Support comparing .ico files using img2txt (Closes: #850730) and fixes and extra tests in subsequent commits.
  • comparators.utils.file: Include magic file type when we know the file format but can't find file-specific details. (Closes: #850850)
  • And other code quality and style improvements.

reproducible-website development

• Daniel Shahaf:
  • Add how-to-chair-a-meeting.md.
• Holger Levsen:
  • Add links to reports by several attendees of berlin2016

tests.reproducible-builds.org

• Ximin Luo and Holger Levsen worked on stricter tests to check that /dev/shm and /run/shm are both mounted with the correct permissions. Some of our build machines currently still fail this test, and the problem is probably the root cause of the FTBFS of some packages (which fails with issues regarding sem_open). The proper fix is still being discussed in #851427.
• Valerie Young worked on creating and linking autogenerated schema documentation for our database used to store the results.
• Holger Levsen added a graph with diffoscope crashes and timeouts.
• Holger also further improved the daily mail notifications about problems.

Misc. This week's edition was written by Ximin Luo, Chris Lamb and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

2 more weeks of my awesome Outreachy journey have passed, so it is time to make an update on my progress. I continued my work on improving diffoscope by fixing bugs and completing wishlist items. These include:

Improving APK support I worked on #850501 and #850502 to improve the way diffoscope handles APK files. Thanks to Emanuel Bronshtein for providing clear description on how to reproduce these bugs and ideas on how to fix them. And special thanks to Chris Lamb for insisting on providing tests for these changes! That part actually proved to be little more tricky, and I managed to mess up with these tests (extra thanks to Chris for cleaning up the mess I created). Hope that also means I learned something from my mistakes. Also, I was pleased to see F-droid Verification Server as a sign of F-droid progress on reproducible builds effort - I hope these changes to diffoscope will help them!

Adding support for image metadata That came from #849395 - a request was made to compare image metadata along with image content. Diffoscope has support for three types of images: JPEG, MS Windows Icon (*.ico) and PNG. Among these, PNG already had good image metadata support thanks to sng tool, so I worked on .jpeg and .ico files support. I initially tried to use exiftool for extracting metadata, but then I discovered it does not handle .ico files, so I decided to use a bigger force - ImageMagick s identify - for this task. I was glad to see it had that handy -format option I could use to select only the necessary fields (I found their -verbose, well, too verbose for the task) and presenting them in the defined form, negating the need of filtering its output. What was particulary interesting and important for me in terms of learning: while working on this feature, I discovered that, at the moment, diffoscope could not handle .ico files at all - img2txt tool, that was used for retrieving image content, did not support that type of images. But instead of recognizing this as a bug and resolving it, I started to think of possible workaround, allowing for retrieving image metadata even after retrieving image content failed. Definetely not very good thinking. Thanks Mattia Rizzolo for actually recognizing this as a bug and filing it, and Chris Lamb for fixing it!

Other work

Order-like differences, part 2 In the previous post, I mentioned Lunar s suggestion to use hashing for finding order-like difference in wide variety of input data. I implemented that idea, but after discussion with my mentor, we decided it is probably not worth it - this change would alter quite a lot of things in core modules of diffoscope, and the gain would be not really significant. Still, implementing that was an important experience for me, as I had to hack on deepest and, arguably, most difficult modules of diffoscope and gained some insight on how they work.

Comparing with several tools (work in progress) Although my initial motivation for this idea was flawed (the workaround I mentioned earlier for .ico files), it still might be useful to have a mechanism that would allow to run several commands for finding difference, and then give the output of those that succeed, failing if and only if they all have failed. One possible case when it might happen is when we use commands coming from different tools, and one of them is not installed. It would be nice if we still used the other and not the uninformative binary diff (that is a default fallback option for when something goes wrong with more clever comparison). I am still in process of polishing this change, though, and still in doubt if it is needed at all.

Side note - Outreachy and my university progress In my Outreachy application, I promised that if I am selected into this round, I will do everything I can to unload the required time period from my university time commitements. I did that by moving most of my courses to the first half of the academic year. Now, the main thing that is left for me to do is my Master s thesis. I consulted my scientific advisors from both universities that I am formally attending (SFEDU and LUT - I am in double degree program), and as a result, they agreed to change my Master s thesis topic to match my Outreachy work. Now, that should have sounded like an excellent news - merging these activities together actually mean I can allocate much more time to my work on reproducible builds, even beyond the actual internship time period. That was intended to remove a burden from my shoulders. Still, I feel a bit uneasy. The drawback of this decision lies in fact I have no idea on how to write scientific report based on pure practical work. I know other students from my universities have done such things before, but choosing my own topic means my scientific advisors can t help me much - this is just out of their area of expertise. Well, wish me luck - I m up to the challenge!

What happened in the Reproducible Builds effort between Sunday January 1 and Saturday January 7 2017: GSoC and Outreachy updates

• Maria Glukhova blogged about Getting to know diffoscope better.

Toolchain development

• #849999 was filed: "dpkg-dev should not set SOURCE_DATE_EPOCH to the empty string"

Packages reviewed and fixed, and bugs filed Chris Lamb:

• #849968 filed against pciutils.
• #849972 filed against yard.
• #850556 filed against sed.

Dhole:

• #849926 filed against nxt-firmware.

Reviews of unreproducible packages 13 package reviews have been added, 4 have been updated and 6 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been added/updated:

• Add new randomness_in_documentation_generated_by_yardoc toolchain issue.
• Add fix for randomness_in_documentation_generated_by_yardoc toolchain issue.

Upstreaming of reproducibility fixes Merged:

• calibre
• jna
• rocksndiamonds
• avfs
• jackaudio1
• jackaudio2
• gri

Opened:

• rockdodger

Weekly QA work During our reproducibility testing, the following FTBFS bugs have been detected and reported by:

• Chris Lamb (4)

diffoscope development diffoscope 67 was uploaded to unstable by Chris Lamb. It included contributions from :

[ Chris Lamb ]
* Optimisations:
  - Avoid multiple iterations over archive by unpacking once for an ~8X
    runtime optimisation.
  - Avoid unnecessary splitting and interpolating for a ~20X optimisation
    when writing --text output.
  - Avoid expensive diff regex parsing until we need it, speeding up diff
    parsing by 2X.
  - Alias expensive Config() in diff parsing lookup for a 10% optimisation.
* Progress bar:
  - Show filenames, ELF sections, etc. in progress bar.
  - Emit JSON on the the status file descriptor output instead of a custom
    format.
* Logging:
  - Use more-Pythonic logging functions and output based on __name__, etc.
  - Use Debian-style "I:", "D:" log level format modifier.
  - Only print milliseconds in output, not microseconds.
  - Print version in debug output so that saved debug outputs can standalone
    as bug reports.
* Profiling:
  - Also report the total number of method calls, not just the total time.
  - Report on the total wall clock taken to execute diffoscope, including
    cleanup.
* Tidying:
  - Rename "NonExisting" -> "Missing".
  - Entirely rework diffoscope.comparators module, splitting as many separate
    concerns into a different utility package, tidying imports, etc.
  - Split diffoscope.difference into diffoscope.diff, etc.
  - Update file references in debian/copyright post module reorganisation.
  - Many other cleanups, etc.
* Misc:
  - Clarify comment regarding why we call python3(1) directly. Thanks to J r my
    Bobbio <lunar@debian.org>.
  - Raise a clearer error if trying to use --html-dir on a file.
  - Fix --output-empty when files are identical and no outputs specified.
[ Reiner Herrmann ]
* Extend .apk recognition regex to also match zip archives (Closes: #849638)
[ Mattia Rizzolo ]
* Follow the rename of the Debian package "python-jsbeautifier" to
  "jsbeautifier".
[ siamezzze ]
* Fixed no newline being classified as order-like difference.

reprotest development reprotest 0.5 was uploaded to unstable by Chris Lamb. It included contributions from:

[ Ximin Luo ]
* Stop advertising variations that we're not actually varying.
  That is: domain_host, shell, user_group.
* Fix auto-presets in the case of a file in the current directory.
* Allow disabling build-path variations. (Closes: #833284)
* Add a faketime variation, with NO_FAKE_STAT=1 to avoid messing with
  various buildsystems. This is on by default; if it causes your builds
  to mess up please do file a bug report.
* Add a --store-dir option to save artifacts.

Other contributions (not yet uploaded):

• Chris Lamb:
  • setup.py: Specify a Python3 env shebang
  • Drop reprotest/ runner,tools from DEP-5 copyright wildcard matching.

reproducible-builds.org website development

• website.git notifications now go to#reproducible-builds. Thanks todanielsh and the KGB bot maintainers.
• Daniel Shahaf:
  • events: add IRC meeting
  • Move mailing list from 'who' to 'resources'.
• Holger Levsen:
  • link to meetbot.debian.net IRC meetings archives
• Chris Lamb:
  • tools: Underscore that disorderfs is deliberately adding such randomness.

tests.reproducible-builds.org

• Debian arm64 architecture was fully tested in all three suites in just 15 days. Thanks again to Codethink.co.uk for their support!
• Log diffoscope profiling info. (lamby)
• Run pg_dump with -O --column-inserts to make easier to import our main database dump into a non-PostgreSQL database. (mapreri)
• Debian armhf network: CPU frequency scaling was enabled for three Firefly boards, enabling the CPUs to run at full speed. (vagrant)
• Arch Linux and Fedora tests have been disabled (h01ger)
• Improve mail notifications about daily problems. (h01ger)

Misc. This week's edition was written by Chris Lamb, Holger Levsen and Vagrant Cascadian, reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

I apologize to all potential readers of this blog for not writing a comprehensive Introduction post with details of the project I am taking part in during my internship, as well as some story about how I ended up there. Let me just say that I was a Debian user for years when I discovered it is taking part in Outreachy as one of organisations. Their Reproducible Builds effort has a noble goal and a bunch of great people behind it - I had no chances not to get excited by it. Looking for a place where my skills could be of any use, I discovered diffoscope - the tool for in-depth comparassion of files, archives etc. My mentor, Mattia Rizzolo, supported my decision to work on it, so now I am concentrating my efforts on improving diffoscope. As my first steps, I am doing small (but hopefully still somewhat important) job of fixing existing bugs. It helps me to better understand how diffoscope works, as well as introduces me to the workflow of opensource development. During December, I have done several small contributions, mostly fixing bugs.

Test data and jessie-backports First of them could be somewhat called cleaning up after my own mistake, although that mistake wasn t trivial. During the application period, I have fixed a bug with diffoscope failing while comparing symlinks to directory. That was a small change, but I included some tests for that case anyway. And that actually caused problems. With these tests, I included test data: two folders with symlinks. All was good in unstable version of Debian, but in jessie-backports, that commit caused build to fail. After some digging, I discovered the problem was caused by build process including copying that data. That was done using shutils Python module, and older version of that module, included in jessie, could not handle copying symlinks to directory properly. Thanks to my mentor for giving me a hint on how to resolve this: using temporary folders and creating these symlinks at runtime. That way, we ensured tests run without problems during build process on jessie. What have I learned: A great deal, actually. I spent too much time on that one, but I learned how to build packages, what happens during dpkg-buildpackage run and what debhelper tools are for. I also learned a bit about what chroot is and how to use it for testing.

ICC profile files and file type recognizing regexp Another one was also about failing tests and, therefore, failing build. Failing tests were all due to ICC files were not recognized by diffoscope. Turned out libmagic got an update which changed the description of ICC profile files. Diffoscope was relying on regexp applied to file type description to recognize the file, so I changed regexp to reflect the changes in libmagic. What have I learned: How diffoscope recognizes file types. Got me thinking: maybe there is a better way? That regexp-based approach is doomed to cause problems with every file type description change. I have this question still lingering in my mind - maybe I will come up with an idea later.

Order-like difference in text files Next, I decided to do something a bit bigger and fullfilled a feature request. That request was for detecting order-like difference in text files (when files has the same lines, but in different order). I did it by collecting added and removed lines in diff output in lists, sorting and then comparing them. Sadly, I forgot about one particular case - when one of the files is missing the newline at the end of file. I was kindly reminded of that quite soon in comments on the bug-tracker (thanks danielsh!) and have already fixed that. I also recieved feedback on how better implement it deeper in the diffoscope - not using the results of diff, but rather comparing sum of hashes of the lines directly in the difference module. I am yet to try that. What have I learned: That a call to diff is actually the slowest part of the diffoscope run when done on two big text files. Could it help somehow in speeding it up? I don t know yet. I also learned to comment on bugs in Debian bugtracker and was surprised by how much feedback I got. Thanks to my mentor for pushing me to do that - I definetely need to overcome my fear of communications to be more effective!

Random FTBFS There was also a very nasty bug that caused diffoscope to fail to be built from source randomly, failing with non-informative Fatal Python error: deallocated None. It already seemed strange when it was first reported; It got only more strange when suddenly that bug ceased to be reproducible. We hoped that would mean that bug was caused by some external tool, and was fixed there. Turns out it was not that easy. I tested this on two separate computers and on virtual machine; I used different versions of diffoscope. Well. Seems like that bug is still somehow tied to diffoscope version and not some external tool version - I still can do git checkout 64 and be able to reproduce the bug (still randomly, though). Although I spent quite a lot of time on that one, the only result was the information about connection between bug apperances and diffoscope version. I still wasn t able to get to the root of the problem - hopefully, someone else will be able to, given the information I found. What have I learned: git-bisect! Thanks to my friend for pointing me to it, that tool came handy in that situation. Also, got some experience in catching nasty bugs like that (pity that no experience in squashing them). I had some extra time commitements in December, one of them (Reproducible Builds Summit II) connected to my internship and one (my exam session in university) not. In January, I should be able to allocate more time to that work - I hope it will help me achieve more significant results. Many thanks to Mattia Rizzolo, Chris Lamb, Holger Levsen and all other folks of Reproducible Builds project - I cannot stress enough how important your support is to me. Wish you all a great 2017!

What happened in the Reproducible Builds effort between Sunday December 18 and Saturday December 24 2016: Media coverage 100% Of The 289 Coreboot Images Are Now Built Reproducibly by Phoronix, with more details in German by Pro-Linux.de. We have further reports on our Reproducible Builds World summit #2 in Berlin from Rok Garbas of NixOS as well as Clemens Lang of MacPorts Debian infrastructure work Dak now archives buildinfo files thanks to a patch from Chris Lamb. We also have mostly finalised a design of how they will be distributed by the Debian FTP mirror network which we will start implementing soon. This is great for the future of Debianb but unfortunately this also means that we won't have .buildinfo files for Stretch as Debian will not rebuild its source packages and because these binary packages currently in the archive were mostly built with dpkg > 1.18.11. reprepro/5.0.0-1 has added support for dealing with .buildinfo files that are included in .changes files. (Closes: #843402) Reproducible work in other projects The Chromium project is now working on making their build process (mostly) deterministic. Their motivation is to save both "[money] (less hardware is required) and developer time (reduced latency by having less work to do on the TS and CI)". Unreproducible bugs filed

• Chris Lamb:
  • #848721 filed against apt.
  • #848866 filed against libcorelinux.
  • #849314 filed against node-gulp.
  • #849333 filed against faker.
• Dhole:
  • #848633 filed against sugar-toolkit-gtk3.

Reviews of unreproducible packages 39 package reviews have been added, 75 have been updated and 44 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been updated:

• Updated captures_build_path
• Added nondeterministic_ordering_in_desktop_files_by_python_sugar3

Weekly QA work During our reproducibility testing, some FTBFS bugs have been detected and reported by:

• Adrian Bunk (1)
• Chris Lamb (7)
• Lucas Nussbaum (4)

diffoscope development diffoscope 66 was uploaded to unstable by Chris Lamb. It included contributions from:

• Emanuel Bronshtein:
  • Use ssh-keygen for comparing OpenSSH public keys
  • Use js-beautify as JavaScript code beautifier for .js files (with tests).
  • Many CSS & HTML improvements.
  • Change all HTTP URLs to HTTPS where applicable.
• anthraxx:
  • Enable the use of ssh-keygen on Arch Linux.
• Maria Glukhova:
  • Add detection of order-only difference in plain text format. (Closes: #848049)
  • Change icc-recognizing regexp to reflect changes in file type description. (Closes: #848814)
• Chris Lamb:
  • Update tests for compatibility with enjarify >= 1.0.3. (Closes: #849142)
  • When skipping tests because the version of an external is too low, print the detected version.
  • Avoid unpacking packages twice when comparing .changes. (Closes: #843531)
  • Add a simple profiling framework (enabled via --profile).
  • Various code quality and reliability improvements.
  • Document how to sign PyPI uploads.

strip-nondeterminism development strip-nondeterminism 0.029-1 was uploaded to unstable by Chris Lamb. It included no new content from this week, but rather included contributions from previous weeks. reproducible-website development The website is now also accessible via the https://www.reproducible-builds.org URL.

• Clemens Lang:
  • Add the definition of "reproducible", as drafted at the reproducible builds world summit in Berlin. Thanks to all participants in the sessions that worked these out!
• Valerie R Young:
  • Force ordering of titles.
  • Various formatting improvements.
• Holger Levsen:
  • Various usability and formatting improvements.
  • https://www.reproducible-builds.org
• Chris Lamb:
  • Various usability, style and wording improvements.
  • Add Debconf15, Skroutz.gz and MiniDebconfCambridge15 talks to resouces page.

tests.reproducible-builds.org

• We changed the data storage backend from a single sqlite3 database file (651 MB) to a PostgreSQL database. With this change we'll be able to scale a lot more and add testing of the arm64 architecture.
  • Valerie Young wrote most of the code, Mattia Rizzolo reviewed and helped improve the code and Holger deployed it and found some minor bugs which have been fixed.
• We are now testing the arm64 architecture for all packages on all suites, arranged by Holger. Many thanks to codethink for providing us with access to eight 8-core arm64 machines with 64GB memory, which allows us to rebuild Debian very fast!

Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC and the mailing lists.

Had some printer fun this week. It was fun in the sense that failure modes are interesting, not that there was much joy in the process. My current document printer is an HP that I bought back in early 2008; soon 9 years old, that is. When I got the printer I was quite happy: it supports Postscript, it supports memory extension (which allowed me to go from the built-in 64MB to a whopping 320MB), it is networked and has automatic duplex. Not good for much more than document printing, but that it did well. I didn't print a lot on it (averaged it was well below the recommended monthly limit), which might explain the total trouble-free operation, but I did change the toner cartridges a couple of times. The current cartridges were running low for a while, but I didn't need to change them yet. As I printed a user manual at the beginning of the week (~300+ pages in total), I ran out of the black half-way through. Bought a new cartridge, installed it, and the first strange thing was that it still showed Black empty - please replace . I powered the printer off and turned it on again (the miracle cure for all IT-related things), and things seemed OK, so I restarted printing. However, this time, the printer was going through 20-30 pages, and then was getting stuck in "Printing document" with green led blinking. Waited for 20 minutes, nothing. So cancel the job (from the printer), restart printing, all fine. The next day I wanted to print a single page, and didn't manage to. Checked that the PDF is normal, checked an older PDF which I printed successfully before, nothing worked. Changed drivers, unseated & re-seated the extra memory, changed operating systems, nothing. Not even the built-in printer diagnostic pages were printing. The internet was all over with "HP formatter issues"; apparently some HP printers had "green" (i.e. low-quality) soldering, and were failing after a while. But people were complaining about 1-2-4 years, not 9 that my printer worked, and it was very suspicious that all troubles started after my cartridge replacement. Or, more likely, due to the recent sudden increase in printing. Given that formatter board fixes (bake in the oven for N minutes at a specific temperature to reflow the soldering) are temporary and that you can't find replacement parts for this printer, I started looking for a new printer. To my surprise (and dismay at the waste that capitalism produces), a new printer from a higher class was cheaper than replacing all 4 cartridges in my printer. So I had a 90% full black cartridge that I couldn't reuse, but I'd get a new printer for not much more. Interestingly, in 9 years, the development was:

• In the series of printers that I had (home office use), one can't get a Ethernet-only networked duplex printer; the M252 series has only an 'n' variant (Ethernet only, no duplex), or 'dw' variant (Ethernet, WiFi, Duplex); if one wants duplex but no WiFi, it's available only in the next series, the M452.
• The CPU speeds increased 2-3 and memory capacity by 2-4 ; however, memory or font expansion is no longer possible.
• The M252 series still uses Fast Ethernet (which is enough and consumes less power), whereas the M452 series has Gigabit.
• It seems the cartridges come in two different capacities, but basically colour laser printers still employ the same 4-colour cartridge set (compare to photo printers at 9+).
• I did just a brief examination of the market, but for home use, it seems the recommendation is still HP for no-troubles use or other brands for cheaper costs. Of course it varies a lot in reviews, but this is what I understood from forums; maybe I'm biased.
• There was no increase in real resolution; the native grid is still 600dpi (photo inkjet printers are also stuck at 360/720 native for a while), but the ImageRet software processing seems to have advanced (from what the white-papers say).
• Print speed however has visibly increased; still the same 2-3 increase, but this is wall-clock speed increase, whereas the CPU/memory is less relevant.

I was however happy that one can still get OS-independent (Postscript), networked printers that are small enough for home use and don't (necessarily) come with WiFi. However, one thing still bothered me: did I have such problems because the printer died of overwork at old age, or was it related to the cartridge change? So I start searching again, and I find a post on a forum (oh Google, why did you remove "forum search" and replaced it with "language level"?) that details a hidden procedure to format the internal storage of the printer, exactly for my printer model, exactly for my symptoms. Huh, I will lose page count, but this is worth a try So I do press the required keys, I see the printer booting and saying "erasing ", then asking for language, which makes me happy because it seems the forum post was correct in one regard. I confirm English, the printer reboots once more, and then when it comes up it warns me: "Yellow cartridge is a non-HP original, please confirm". I get confused, and re-seat all cartridges, to no avail. Yellow is non-HP. Sigh, maybe that cartridge had something that confused the printer? When I visit its web page however, all cartridges except the newly installed black one are marked as Non-HP; this only means that I can't see their remaining toner level, but otherwise the printer is restored back to life. I take the opportunity to also perform a firmware upgrade (only five years newer firmware, but still quite old), but this doesn't solve the Non-HP message. The printer works now, and I'm left wondering: was this all a DRM-related failure, something like new cartridge chip which had some new code that confused the printer so bad it needed reformatting, at which point the old cartridge code is no longer supported (for whatever reason)? Was it just a fluke, unrelated to DRM? Was the problem that I powered off the printer soon after replacing the cartridge, while it was still doing something (e.g. preparing to do a calibration after the change)? And another, more practical question: I have 3 cartridges to replace still; they were at 10% before this entire saga, and I'm not able to see their level anymore, but they'll get down to empty soon. The black cartridge in the printer is already at 77%, which is surprising as I didn't print that much. So should I replace the cartridges on what is a possibly fully functional, but also possibly a dying printer? Or buy a new one for slightly more, throwing out possibly good hardware? Even though I understand the business reason behind it, I hate the whole concept of "the printer is free, you pay for the ink". Though in my case "free" didn't mean bad, as a lifetime of 9 years is good enough for a printer.

What happened in the Reproducible Builds effort between Sunday December 4 and Saturday December 10 2016: Toolchain development and fixes Anders Kaseorg opened a pull request to asciidoc upstream, to make it generate reproducible documentation. (#782294) Bugs filed Chris Lamb:

• #847768 filed against hoichess.

Clint Adams:

• #847008 filed against tircd.
• #847021 filed against oolite.
• #847024 filed against torrus.
• #847039 filed against jqueryui.

Dafydd Harries:

• #847033 filed against newlib.
• #847035 filed against bulletml.

Robbie Harwood:

• #846971 filed against maildrop.
• #847009 filed against amanda.

Valerie R Young:

• #846975 filed against libranlip.
• #847016 filed against magicfilter.
• #847020 filed against mailfront.
• #847022 filed against mimefilter.
• #847023 filed against netqmail.
• #847026 filed against psgml.
• #847028 filed against qmail-run.
• #847029 filed against qmail-tools.
• #847030 filed against skalibs.
• #847031 filed against socklog.
• #847032 filed against tinydyndns.
• #847034 filed against twoftpd.
• #847036 filed against ucspi-tcp.
• #847037 filed against ucspi-unix.

Reviews of unreproducible packages 47 package reviews have been added, 84 have been updated and 3 have been removed in this week, adding to our knowledge about identified issues. 1 new issue type has been added: lessc_captures_build_path Weekly QA work During our reproducibility testing, some FTBFS bugs have been detected and reported by:

• Chris Lamb (8)

diffoscope development Chris Lamb fixed a division-by-zero in the progress bar, split out trydiffoscope into a separate package, and made some performance enhancements. Maria Glukhova fixed build issues with Python 3.4 strip-nondeterminism development Anders Kaseorg added support for .par files, by allowing them to be treated as Zip archives; and Chris Lamb improved some documentation. reprotest development Ximin Luo added the ability to vary the build time using faketime, as well as other code quality improvements and cleanups. He also discovered a little-known fact about faketime - that it also modifies filesystem timestamps by default. He submitted a PR to libfaketime upstream to improve the documentation on this, which was quickly accepted, and also disabled this feature in reprotest's own usage of faketime. buildinfo.debian.net development There was further work on buildinfo.debian.net code. Chris Lamb added support for buildinfo format 0.2 and made rejection notices clearer; and Emanuel Bronshtein fixed some links to use HTTPS. Misc. This week's edition was written by Ximin Luo and reviewed by a bunch of Reproducible Builds folks on IRC and via email.

On CVE-2016-4484, a (security)? bug in the cryptsetup initramfs integration On November 4, I was made aware of a security vulnerability in the integration of cryptsetup into initramfs. The vulnerability was discovered by security researchers Hector Marco and Ismael Ripoll of CyberSecurity UPV Research Group and got CVE-2016-4484 assigned. In this post I'll try to reflect a bit on

• the nature of the vulnerability
• the way it was published and discussed by technology news press

What CVE-2016-4484 is all about Basically, the vulnerability is about two separate but related issues:

1. Initramfs rescue shell considered harmful The main topic that Hector Marco and Ismael Ripoll address in their publication is that Debian exits into a rescue shell in case of failure during initramfs, and that this can be triggered by entering a wrong password ~93 times in a row. Indeed the Debian initramfs implementation as provided by initramfs-tools exits into a rescue shell (usually a busybox shell) after a defined amount of failed attempts to make the root filesystem available. The loop in question is in local_device_setup() at the local initramfs script In general, this behaviour is considered as a feature: if the root device hasn't shown up after 30 rounds, the rescue shell is spawned to provide the local user/admin a way to debug and fix things herself. Hector Marco and Ismael Ripoll argue that in special environments, e.g. on public computers with password protected BIOS/UEFI and bootloader, this opens an attack vector and needs to be regarded as a security vulnerability:

It is common to assume that once the attacker has physical access to the computer, the game is over. The attackers can do whatever they want. And although this was true 30 years ago, today it is not. There are many "levels" of physical access. [...] In order to protect the computer in these scenarios: the BIOS/UEFI has one or two passwords to protect the booting or the configuration menu; the GRUB also has the possibility to use multiple passwords to protect unauthorized operations. And in the case of an encrypted system, the initrd shall block the maximum number of password trials and prevent the access to the computer in that case.

While Hector and Ismael have a valid point in that the rescue shell might open an additional attack vector in special setups, this is not true for the vast majority of Debian systems out there: in most cases a local attacker can alter the boot order, replace or add boot devices, modify boot options in the (GNU GRUB) bootloader menu or modify/replace arbitrary hardware parts. The required scenario to make the initramfs rescue shell an additional attack vector is indeed very special: locked down hardware, password protected BIOS and bootloader but still local keyboard (or serial console) access are required at least. Hector and Ismael argue that the default should be changed for enhanced security:

[...] But then Linux is used in more hostile environments, this helpful (but naive) recovery services shall not be the default option.

For the reasons explained about, I tend to disagree to Hectors and Ismaels opinion here. And after discussing this topic with several people I find my opinion reconfirmed: the Debian Security Team disputes the security impact of the issue and others agree. But leaving the disputable opinion on a sane default aside, I don't think that the cryptsetup package is the right place to change the default, if at all. If you want added security by a locked down initramfs (i.e. no rescue shell spawned), then at least the bootloader (GNU GRUB) needs to be locked down by default as well. To make it clear: if one wants to lock down the boot process, bootloader and initramfs should be locked down together. And the right place to do this would be the configurable behaviour of grub-mkconfig. Here, one can set a password for GRUB and the boot parameter 'panic=1' which disables the spawning of a rescue shell in initramfs. But as mentioned, I don't agree that this would be sane defaults. The vast majority of Debian systems out there don't have any security added by locked down bootloader and initramfs and the benefit of a rescue shell for debugging purposes clearly outrivals the minor security impact in my opinion. For the few setups which require the added security of a locked down bootloader and initramfs, we already have the relevant options documented in the Securing Debian Manual:

• 4.3 Change the BIOS (again)
• 4.4 Set a LILO or GRUB password
• 4.5 Disable root prompt on the initramfs
• 4.6 Remove root prompt on the kernel

After discussing the topic with initramfs-tools maintainers today, Guilhem and me (the cryptsetup maintainers) finally decided to not change any defaults and just add a 'sleep 60' after the maximum allowed attempts were reached. 2. tries=n option ignored, local brute-force slightly cheaper Apart from the issue of a rescue shell being spawned, Hector and Ismael also discovered a programming bug in the cryptsetup initramfs integration. This bug in the cryptroot initramfs local-top script allowed endless retries of passphrase input, ignoring the tries=n option of crypttab (and the default of 3). As a result, theoretically unlimited attempts to unlock encrypted disks were possible when processed during initramfs stage. The attack vector here was that local brute-force attacks are a bit cheaper. Instead of having to reboot after max tries were reached, one could go on trying passwords. Even though efficient brute-force attacks are mitigated by the PBKDF2 implementation in cryptsetup, this clearly is a real bug. The reason for the bug was twofold:

• First, the condition in setup_mapping() responsible for making the function fail when the maximum amount of allowed attempts is reached, was never met:

  setup_mapping()
   
    [...]
    # Try to get a satisfactory password $crypttries times
    count=0                              
  
    while [ $crypttries -le 0 ]   [ $count -lt $crypttries ]; do
        export CRYPTTAB_TRIED="$count"
        count=$(( $count + 1 ))
        [...]
    done
    if [ $crypttries -gt 0 ] && [ $count -gt $crypttries ]; then
        message "cryptsetup: maximum number of tries exceeded for $crypttarget"
        return 1
    fi
    [...]
   
  

  As one can see, the while loop stops when $count -lt $crypttries. Thus the second condition $count -gt $crypttries is never met. This can easily be fixed by decreasing $count by one in case of a successful unlock attempt along with changing the second condition to $count -ge $crypttries:

  setup_mapping()
   
    [...]
    while [ $crypttries -le 0 ]   [ $count -lt $crypttries ]; do
        [...]
        # decrease $count by 1, apparently last try was successful.
        count=$(( $count - 1 ))
        [...]
    done
    if [ $crypttries -gt 0 ] && [ $count -ge $crypttries ]; then
        [...]
    fi
    [...]
   
  

  Christian Lamparter already spotted this bug back in October 2011 and provided a (incomplete) patch, but back then I even managed to merge the patch in an improper way, making it even more useless: The patch by Christian forgot to decrease $count by one in case of a successful unlock attempt, resulting in warnings about maximum tries exceeded even for successful attemps in some circumstances. But instead of adding the decrease myself and keeping the (almost correct) condition $count -eq $crypttries for detection of exceeded maximum tries, I changed back the condition to the wrong original $count -gt $crypttries that again was never met. Apparently I didn't test the fix properly back then. I definitely should do better in future!
• Second, back in December 2013, I added a cryptroot initramfs local-block script as suggested by Goswin von Brederlow in order to fix bug #678692. The purpose of the cryptroot initramfs local-block script is to invoke the cryptroot initramfs local-top script again and again in a loop. This is required to support complex block device stacks. In fact, the numberless options of stacked block devices are one of the biggest and most inglorious reasons that the cryptsetup initramfs integration scripts became so complex over the years. After all we need to support setups like rootfs on top of LVM with two separate encrypted PVs or rootfs on top of LVM on top of dm-crypt on top of MD raid. The problem with the local-block script is that exiting the setup_mapping() function merely triggers a new invocation of the very same function. The guys who discovered the bug suggested a simple and good solution to this bug: When maximum attempts are detected (by second condition from above), the script sleeps for 60 seconds. This mitigates the brute-force attack options for local attackers - even rebooting after max attempts should be faster.

About disclosure, wording and clickbaiting I'm happy that Hector and Ismael brought up the topic and made their argument about the security impacts of an initramfs rescue shell, even though I have to admit that I was rather astonished about the fact that they got a CVE assigned. Nevertheless I'm very happy that they informed the Security Teams of Debian and Ubuntu prior to publishing their findings, which put me in the loop in turn. Also Hector and Ismael were open and responsive when it came to discussing their proposed fixes. But unfortunately the way they advertised their finding was not very helpful. They announced a speech about this topic at the DeepSec 2016 in Vienna with the headline Abusing LUKS to Hack the System. Honestly, this headline is missleading - if not wrong - in several ways:

• First, the whole issue is not about LUKS, neither is it about cryptsetup itself. It's about Debians integration of cryptsetup into the initramfs, which is a compeletely different story.
• Second, the term hack the system suggests that an exploit to break into the system is revealed. This is not true. The device encryption is not endangered at all.
• Third - as shown above - very special prerequisites need to be met in order to make the mere existance of a LUKS encrypted device the relevant fact to be able to spawn a rescue shell during initramfs.

Unfortunately, the way this issue was published lead to even worse articles in the tech news press. Topics like Major security hole found in Cryptsetup script for LUKS disk encryption or Linux Flaw allows Root Shell During Boot-Up for LUKS Disk-Encrypted Systems suggest that a major security vulnerabilty was revealed and that it compromised the protection that cryptsetup respective LUKS offer. If these articles/news did anything at all, then it was causing damage to the cryptsetup project, which is not affected by the whole issue at all. After the cat was out of the bag, Marco and Ismael aggreed that the way the news picked up the issue was suboptimal, but I cannot fight the feeling that the over-exaggeration was partly intended and that clickbaiting is taking place here. That's a bit sad.

Links

• Hector Marco & Ismael Ripoll: CVE-2016-4484: Cryptsetup Initrd root Shell
• Public announcement by Hector Marco of CVE-2014-4484 on oss-security
• Discussion about CVE-2016-4484 on dm-crypt@saout.de

What happened in the Reproducible Builds effort between Sunday November 6 and Saturday November 12 2016: Media coverage Matthew Garrett blogged about Tor, TPMs and service integrity attestation and how reproducible builds are the base for systems integrity. The Linux Foundation announced renewed funding for us as part of the Core Infrastructure Initiative. Thank you! Outreachy updates Maria Glukhova has been accepted into the Outreachy winter internship and will work with us the Debian reproducible builds team. To quote her words

siamezzze: I've been accepted to #outreachy winter internship - going to
work with Debian reproducible builds team. So excited about that! <3
Debian

Toolchain development and fixes dpkg:

• Thanks to a series of dpkg uploads by Guillem Jover, all our toolchain changes are now finally available in sid!
• This means your packages should now be reproducible without having to use our custom APT repository.
• Ximin Luo opened #843925 to remind the fact that dpkg-buildpackage should sign buildinfo files.
• We hope to have detailed post about the new dpkg and the new .buildinfo files for debian-devel-announce soon!

debrebuild:

• srebuild / debrebuild work was resumed by Johannes Schauer and others in #774415.

Bugs filed Chris Lamb:

• #844102 filed against lava-dispatcher
• #844103 filed against lava-server
• #844111 filed against python-defaults
• #843698 filed against tunnelx
• #843967 filed against asyncpg
• #843865 filed against keystone
• #844101 filed against suil

Daniel Shahaf:

• #844232 filed against daisy-player
• #844236 filed against libhtml-lint-perl
• #844228 filed against ebook-speaker

Niko Tyni:

• #843432 filed against libwww-curl-perl

Reiner Herrman:

• #844095 filed against fdm
• #844096 filed against gringo
• #843469 filed against dpkg

Reviews of unreproducible packages 136 package reviews have been added, 5 have been updated and 7 have been removed in this week, adding to our knowledge about identified issues. 3 issue types have been updated:

• Added: random_order_in_dh_pythonX_substvars, valac_permutes_get_type_calls
• Updated: timestamps_in_static_libraries

Weekly QA work During of reproducibility testing, some FTBFS bugs have been detected and reported by:

• Chris Lamb (29)
• Niko Tyni (1)

diffoscope development A new version of diffoscope 62~bpo8+1 was uploaded to jessie-backports by Mattia Rizzolo. Meanwhile in git, Ximin Luo greatly improved speed by fixing a O(n²) lookup which was causing diffs of large packages such as GCC and glibc to take many more hours than was necessary. When this commit is released, we should hopefully see full diffs for such packages again. Currently we have 197 source packages which - when built - diffoscope fails to analyse. buildinfo.debian.net development

• Submissions with duplicate Installed-Build-Depends entries are rejected now that a bug in dpkg causing them has been fixed. Thanks to Guillem Jover.
• Add a new page for every (source, version) combination, for example diffoscope 62.
• DigitalOcean have generously offered to sponsor the hardware buildinfo.debian.net is running on.

tests.reproducible-builds.org Debian:

• For privacy reasons, the new dpkg-genbuildinfo includes Build-Path only if it is under /build. HW42 updated our jobs so this is the case for our builds too, so you can see the build path in the .buildinfo files.
• HW42 also updated our jobs to vary the basename of the source extraction directory. This detects packages that incorrectly assume a $pkg-$version directory naming scheme (which is what dpkg-source -x gives but is not mandated by Debian nor always-true) or that they're being built from a SCM.
• The new dpkg-genbuildinfo also records a sanitised Environment. This is different in our builds, so HW42, Reiner and Holger updated our jobs to hide these differences from diffoscope output.
• Package-set improvements:
  • Holger refactored the create_meta_pkg_sets job so that it's easier to add new package sets.
  • This job is now also using dose-extra from jessie-backports so that it can deal with versioned provides.
  • Added 4 new package sets: debian-edu, debian-edu_build-depends, maint_pkg-grass-devel, maint_debian-accessibility, maint_pkg-openstack.
  • Switched to using the new URL for tails manifests to generate the tails package set.
  • Renamed maint_lua to maint_debian-lua
• Valerie Young contributed four patches for our long-planned transition from SQLite to PostgreSQL.
• In anticipation of the freeze, already-tested packages from unstable and testing on amd64 are now scheduled with equal priority.

reproducible-builds.org website F-Droid was finally added to our list of partner projects. (This was an oversight and they had already been working with us for some time.) Misc. This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

I had mentioned previously that I had been forced to purchase a new laptop. I decided that I didn't want another Thinkpad. The Lenovo ones no longer have the high quality they had in the IBM days and while support is still pretty good by todays dismal standards it's not worth the premium price. (If I'm buying it with my own money that is.) I had heard good thing about Dells' Linux support so I looked into their offerings and ended up buying a Precision 7510. Mind you this model came with Windows 7 installed but I didn't mind. As I wanted to install Debian according to my own specs anyway, I was ok with just knowing that the hardware would be compatible. So I prepared a Jessie USB installation stick (This model doesn't have a CD/DVD drive.) and shrunk down the Windows installation (but not deleted it altogether for reasons to be explained below.) At this point it is traditional to give a long, tortured account of how Heaven and Earth had to be moved to get Linux installed. But that is a thing of the past. The combination of good hardware and the excellent work of the debian-installer team, made the setup a breeze with only a couple of minor bumps in the road. One is that the kernel on the Jessie cd was not quite up to snuff. Downloading 4.6.0 from backports did the trick. Post-install, to get the most out of my nifty new 4K display, I needed the latest, alas non-free, nvidia-drivers. And for stable wifi (I always install over ethernet for this reason) I had to install the firmware-iwlwifi package. Everything else even my printer either "just worked" or needed only minor fiddling around. Having used this machine for a while, the biggest problem I have is with the keyboard. It is nowhere near as tactile and comfortable to use as the old IBM Thinkpads. Even Lenovo Thinkpad keyboards are better. I'm a hunt-and-peck type myself but it is annoying. I think a real touch typist would hate it. The cursor and home, end, page up, page down etc. keys are in the wrong place and home and end are actually function keys. There is a pointer and a trackpad and two sets of mouse buttons which seems like a waste of space. In fact much space is wasted everywhere, space which could be used to improve the keyboard. Other than that I like it. The battery life is not the best but fairly good. It's a bit heavier than I was used to but I've gotten used to it. Although I didn't go with the SSD option, it is not that noisy; again you can get used to it. All in all, I think it is worth it for the price.
I installed Debian but I only really use it as a base to run VMWare Workstation. I occasionally have to support software across multiple platforms but I don't want the hassle or expense of multiple computers so I have Windows (the original installation upgraded to Windows 10) and Mac OS X running in VMs. Plus I have another VM running Kubuntu LTS for my day to day computing, another Debian install running sid for packaging, and Minix. Backups are as simple as making a snapshot of the VM. If something accidently gets screwed up, I can easily revert it back to a known good state. Ideally, I would like to replace VMWare with a free solution such as qemu or virtualbox etc. but as far as I know VMWare is far ahead in emulation capabilities (OpenGL support for example.) which is vital for efficiently using the proprietary OS's. Things were going swimmingly until a few days ago which brings me to part two of this post. I booted into the Windows 10 VM only to be greeted by a message from the Windows boot manager that "A component of the operating system has expired." I tried going back to a snapshot from September (when this definitely was working) but I still got the same thing. A bit of googling revealed this has happened to others and the advice seemed to be to reset the computers date and reinstall Windows 10. It took several tries but I finally got that done, completed the task I needed to do and shut it down. At the end of the day I shut the whole laptop down and thought no more of it. The next day I boot up and...where is grub? It seems that during the Windows reinstall, it had overwritten grub with the Windows boot loader. And while grub is nice enough to add an entry for Windows when detected, Windows does not extend the same courtesy to Linux. Ok time to bring out my trusty USB stick again and reinstall grub. Oops I've wiped it off to store other things. No matter, download another image and do it again. Reboot and...back in Windows. Fiddle around in the EFI settings until I can get it to boot from USB. Now i'm in the shell provided by debian-installer so I can mount and chroot my Linux partition and reinstall grub. Except no I can't because it is Luks encrypted. Ok apt-get install cryptsetup, open it with my passphrase and now I can mount the partion, chroot into and reinstall grub. Except no I can't because it is a logical volume group. Back to apt-get, install lvm2, vgscan (because of course I've forgotten the name of the group,) vgchange and now I can mount, chroot, etc. etc. Except no I can't.

# mount /dev/mapper/vg00-root /mnt
# chroot /mnt
# grub-install /dev/sda
error: cannot find a device for /boot/grub (is /dev mounted?).

sigh

# mount /dev/sda5 /boot
special device /dev/sda5 does not exist.

Well, /dev is mounted but it does indeed not contain a device called sda5.

# /etc/init.d/udev start
udev requires a mounted procfs.  not started.

Very well then.

# mount -t proc none /proc
# /etc/init.d/udev start

Nope. proc needs sysfs.

# mount -t sysfs none /sys
# /etc/init.d/udev start

Still no. You get a warning about how it is a bad idea to run udev from an interactive shell and there is still not /dev/sda5. Time to start googling again. It turns out what I should have done is open another shell from the installer environment and do...

# mount --bind /dev/ /mnt/dev

Now I can mount /boot/grub and reinstall grub and it should all work right? I should be so lucky. Ok back to square one. I now did what I should have done in the first place and searched the Debian wiki. Sure enough there is a page which deals exactly with my predicament. Finally I get everything installed correctly and triumphantly reboot into Linux. Of course now Windows doesn't work again...

This year, Outreachy featured internships from organizations such as Debian, Fedora, GNOME, the Linux Kernel, Mozilla, Python, and Wikimedia, just to name a few. Each organization features mentored projects and in order to apply, applicants must contact the mentor, introduce themselves on the appropriate channels and make a small contribution to the project. After that, applicants might be required to fulfill additional tasks to demonstrate their abilities. Successful applicants will make quality contributions, communicate effectively with mentors, ask questions, fulfill tasks, help out their peers via mailing lists, and/or blog about their experience. One of the projects I applied to was the Clean Room for PGP and X.509 (PKI) Key Management. The project aims to create a Live Disc that enables users to create and manage their PGP keys easily and securely, using a text-based UI. I ve been a Debian user for about a year, but before applying to the project I didn t know much about GnuPG or public key encryption. Since then, I ve made some contributions and attended my first keysigning event in San Francisco featuring a lecture by Neal Walfield (more on that below). For my initial contribution, Daniel Pocock, the mentor for this project, asked that I write a script that lists the USB flash devices connected to the system and specifies which device the system booted from. Here s the bash script that I wrote, and that was enough to submit an application for Debian. My next task was to write a dns hook script for the dehydrated project, a shell client for signing certificates with Let s Encrypt (for free!). The script completes a dns challenge sent by the ACME-server by provisioning a TXT record for a given domain in order to prove ownership of the domain. I chose to write it in python and used the dnspython API. I posted my solution on github and there are many more here. At the lecture, Neal talked about good practices for key creation and management. Here are a few of those points:

• Don t store your master key locally
• Store your master key offline on a smartcard such as GnuK or NitroKey and store backups on a USB stick.
• Neal mentioned that the OpenPGP card is not open hardware and according to this recent post neither is the Yubikey
• To manage the key, use a dedicated offline computer such as a relatively cheap x40 or x60 Thinkpad (my two cents: use a Thinkpad like X200 or T400 flashed with Libreboot, which solves the proprietary firmware problem) and remove the wireless network card.
• Use Tails which wipes memory on shutdown.
• Use subkeys an encryption subkey is automatically created with gpg2 --gen-key. Create an additional signing subkey.
• Generate a secure passphrase for your master key using 5-12 words. Example: pipe after harm horse split seize radar bulb
  • Funny cartoon illustrating this concept
• Refresh your keys regularly for new preferences and revokation certificates. An alternative to gpg2 --refresh-keys is parcimonie, which uses tor and refreshes keys one at a time.
• Don t back up .gnupg/random_seed
• More OpenPGP Best Practices

See the slides for Neal s full presentation.

This year, Outreachy featured internships from organizations such as Debian, Fedora, GNOME, the Linux Kernel, Mozilla, Python, and Wikimedia, just to name a few. Each organization features mentored projects and in order to apply, applicants must contact the mentor, introduce themselves on the appropriate channels and make a small contribution to the project. After that, applicants might be required to fulfill additional tasks to demonstrate their abilities. Successful applicants will make quality contributions, communicate effectively with mentors, ask questions, fulfill tasks, help out their peers via mailing lists, and/or blog about their experience. One of the projects I applied to was the Clean Room for PGP and X.509 (PKI) Key Management. The project aims to create a Live Disc that enables users to create and manage their PGP keys easily and securely, using a text-based UI. I ve been a Debian user for about a year, but before applying to the project I didn t know much about GnuPG or public key encryption. Since then, I ve made some contributions and attended my first keysigning event in San Francisco featuring a lecture by Neal Walfield (more on that below). For my initial contribution, Daniel Pocock, the mentor for this project, asked that I write a script that lists the USB flash devices connected to the system and specifies which device the system booted from. Here s the bash script that I wrote, and that was enough to submit an application for Debian. My next task was to write a dns hook script for the dehydrated project, a shell client for signing certificates with Let s Encrypt (for free!). The script completes a dns challenge sent by the ACME-server by provisioning a TXT record for a given domain in order to prove ownership of the domain. I chose to write it in python and used the dnspython API. I posted my solution on github and there are many more here. At the lecture, Neal talked about good practices for key creation and management. Here are a few of those points:

• Don t store your master key locally
• Store your master key offline on a smartcard such as GnuK or NitroKey and store backups on a USB stick.
• Neal mentioned that the OpenPGP card is not open hardware and according to this recent post neither is the Yubikey
• To manage the key, use a dedicated offline computer such as a relatively cheap x40 or x60 Thinkpad (my two cents: use a Thinkpad like X200 or T400 flashed with Libreboot, which solves the proprietary firmware problem) and remove the wireless network card.
• Use Tails which wipes memory on shutdown.
• Use subkeys an encryption subkey is automatically created with gpg2 --gen-key. Create an additional signing subkey.
• Generate a secure passphrase for your master key using 5-12 words. Example: pipe after harm horse split seize radar bulb
  • Funny cartoon illustrating this concept
• Refresh your keys regularly for new preferences and revokation certificates. An alternative to gpg2 --refresh-keys is parcimonie, which uses tor and refreshes keys one at a time.
• Don t back up .gnupg/random_seed
• More OpenPGP Best Practices

See the slides for Neal s full presentation.

What happened in the Reproducible Builds effort between Sunday September 25 and Saturday October 1 2016: Statistics For the first time, we reached 91% reproducible packages in our testing framework on testing/amd64 using a determistic build path. (This is what we recommend to make packages in Stretch reproducible.) For unstable/amd64, where we additionally test for reproducibility across different build paths we are at almost 76% again. IRC meetings We have a poll to set a time for a new regular IRC meeting. If you would like to attend, please input your available times and we will try to accommodate for you. There was a trial IRC meeting on Friday, 2016-09-31 1800 UTC. Unfortunately, we did not activate meetbot. Despite this participants consider the meeting a success as several topics where discussed (eg changes to IRC notifications of tests.r-b.o) and the meeting stayed within one our length. Upcoming events Reproduce and Verify Filesystems - Vincent Batts, Red Hat - Berlin (Germany), 5th October, 14:30 - 15:20 @ LinuxCon + ContainerCon Europe 2016. From Reproducible Debian builds to Reproducible OpenWrt, LEDE & coreboot - Holger "h01ger" Levsen and Alexander "lynxis" Couzens - Berlin (Germany), 13th October, 11:00 - 11:25 @ OpenWrt Summit 2016. Introduction to Reproducible Builds - Vagrant Cascadian will be presenting at the SeaGL.org Conference In Seattle (USA), November 11th-12th, 2016. Previous events GHC Determinism - Bartosz Nitka, Facebook - Nara (Japan), 24th September, ICPF 2016. Toolchain development and fixes Michael Meskes uploaded bsdmainutils/9.0.11 to unstable with a fix for #830259 based on Reiner Herrmann's patch. This fixed locale_dependent_symbol_order_by_lorder issue in the affected packages (freebsd-libs, mmh). devscripts/2.16.8 was uploaded to unstable. It includes a debrepro script by Antonio Terceiro which is similar in purpose to reprotest but more lightweight; specific to Debian packages and without support for virtual servers or configurable variations. Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible in our testing framework after being fixed:

• ara/1.0.32 by Chris Lamb, original patch by Chris Lamb.
• fracplanet/0.4.0-5 by Chris Lamb, original patch by Reiner Herrman.
• gnarwl/3.6.dfsg-8 by Bernhard Schmidt, original patch by Chris Lamb.
• kgb-bot/1.34-1 by gregor herrmann, original patch by gregor herrmann.
• survex/1.2.29-1 by Olly Betts.
• zpaq/1.10-3 by Chris Lamb, original patch by Reiner Herrman.
• fig2dev/1:3.2.6-3 by Roland Rosenfeld.
• luxio/11-1 by Daniel Silverstone.
• monkeysign/2.1.1 by Antoine Beaupr , original patch by Daniel Kahn Gillmor.
• openarena-085-data/0.8.5split-9 by Simon McVittie.
• openarena-088-data/0.8.8-7 by Simon McVittie.
• openarena-data/0.8.5split-9 by Simon McVittie.
• rc/1.7.4-1 by Reiner Herrmann, original patch by Chris Lamb.

The following updated packages appear to be reproducible now for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

• gkrellm/2.3.8-1 by Sandro Tosi
• glassfish/1:2.1.1-b31g+dfsg1-4 by Emmanuel Bourg

Some uploads have addressed some reproducibility issues, but not all of them:

• apache2/2.4.23-5 by Rapha l Hertzog
• freeradius/3.0.11+dfsg-1 by Michael Stapelberg
• libnss-ldap/265-4 by Chris Lamb
• lift/2.5.0-1 by Nicolas Delvaux
• linux/4.8~rc8-1~exp1 by Ben Hutchings
• nose2/0.6.5-2 by Barry Warsaw
• postgresql-9.6/9.6.0-1 Christoph Berg
• strace/4.13-0.1 by Nicolas Braud-Santoni
• yersinia/0.7.3-3 by No l K the

Patches submitted that have not made their way to the archive yet:

• #838888 filed against dh-haskell by Chris Lamb.
• #838971 filed against slang2 by Chris Lamb.
• #839587 filed against sympa by Chris Lamb.
• #839181 filed against transmission-remote-gtk by Chris Lamb.
• #838829 filed against vala by Sebastian Reichel.
• #838970 filed against webkit2pdf by Chris Lamb.
• #831569 filed against websockets by Chris Lamb.
• #839347 filed against xml-core by Lucas Nussbaum.
• #839526 filed against xml-core by Adrian Bunk.

Reviews of unreproducible packages 77 package reviews have been added, 178 have been updated and 80 have been removed in this week, adding to our knowledge about identified issues. 6 issue types have been updated:

• Added clilibs_line_order, records_build_flags and hevea_captures_build_path.
• Removed locale_dependent_symbol_order_by_lorder, fixed in bsdmainutils/9.0.11.
• Updated diffoscope_runs_forever, captures_build_path.

Weekly QA work As part of reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (3)
• Chris Lamb (12)
• Lucas Nussbaum (3)
• Sebastian Reichel (1)

diffoscope development A new version of diffoscope 61 was uploaded to unstable by Chris Lamb. It included contributions from:

• Ximin Luo:
  • Improve the CLI --help text and add an --output-empty option.
• Chris Lamb:
  • Add a progress bar and show it if stdout is a TTY. You can read more about it here. It can also be read by higher-level programs via the --status-fd CLI option.
• Maria Glukhova:
  • Behaviour improvements in the case of OS-level errors.
• Mattia Rizzolo:
  • Testing and packaging improvements.

Post-release there were further contributions from:

• Chris Lamb:
  • Code architecture improvements.
• Maria Glukhova:
  • Testing improvements.

reprotest development A new version of reprotest 0.3.2 was uploaded to unstable by Ximin Luo. It included contributions from:

• Ximin Luo:
  • Add a --diffoscope-arg CLI option to pass extra args to diffoscope.

Post-release there were further contributions from:

• Chris Lamb:
  • Code quality improvements.

tests.reproducible-builds.org

• Hans-Christoph Steiner continued work on setting up reproducible tests for F-Droid.
• Holger cleaned up the script creating the page showing breakages, so that it now also cleans up some of the breakage it finds.
• IRC notifications about diffoscope crashes and artifacts available for investigations have been dropped; instead the breakages page has a permanent pointer. (h01ger)
• IRC notifications from the automatic package scheduler and status changes for packages have been moved -- as a temporary trial -- to #debian-reproducible-changes on irc.oftc.net (Mattia).

Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

We have been making a lot of minor changes to lintian.d.o and the underlying report framework. Most of them were hardly noticeable to the naked. In fact, I probably would not have spotted any of them, if I had not been involved in writing them. Nonetheless, I felt like sharing them, so here goes. User visible changes:

• The generated reports are now in HTML5 rather than XHTML. [commit:9652c2e]
• Add alt texts to graphs for [commit:83434ec]
• Wrap front page index in a <nav> and reword link text so they can stand alone [commit:13993c8]
• Fix index of maintainer names, so characters outside [A-Z] always go at the end plus add a link directly to that section. [commit:70a24ed, commit:94eb258]
• Put the banner links in a <nav> [commit:baa77fc]
• Always create an index of packages on the individual maintainer pages [commit:124f9a8]
• Wrap the above-mentioned indices in a <nav> tag [commit:2ecea9b]
• Make the links in these indices self-contained [commit:bfc5292, commit:1043d4d]
• Fix mojibake in maintainer names [commit:e907388]

In case you were wondering, the section title is partly a pun as half of these changes were intended to assist visually impaired users. They were triggered by me running into Sam Hartmann at DebConf16, where I asked him about how easy Debian s websites were for blind people. Allegedly, we are generally doing quite good in his opinion (with one exception, for which Sam filed Bug#830213), which was a positive surprise for me. On a related note: Thanks Luke Faraone and Asheesh Laroia for getting helping me started on these changes. Reporting framework / Internal changes:

• Harness supports a no generate reports to make report generation optional. [commit:21fab0b]
• Reduced memory requirements for reports generation by ~15% [commit:534d169]
• Refactored reporting frontends to share the same bootstrap code as lintian/lintian-info ( dplint ) [commit:7d9a7d1, commit:97cf104, plus a bunch of refactoring leading up to those commits]
• Add sub-resource integrity fields in <link> tags [commit:27a367d]
• Add a Content-Security-Policy (via a <meta http-equiv>-tag) [commit:b022a39, commit:30bdc90]
• Support mirrors using Packages.xz files and without SHA1 fields [commit:d9cf314, commit:cb05f1d]
• Shuffle the lintian backlog to randomize the workload [commit:9b8873f]
• Stop lintian when the dead-line/run-time limit is exceeded [commit:7032be5, commit:7b3190d, commit:62532f8]

With the last change + the no generate reports option, we were able to schedule lintian more frequently. Originally, lintian only ran once a day. With the no generate reports , we added a second run and with the last changes, we bumped it to 4 times a day. Unsurprisingly, it means that we are now reprocessing the archive a lot faster than previously. All of the above is basically the all the note-worthy changes on the Lintian reporting framework since the Partial rewrite of lintian s reporting setup (~1 years ago).
Filed under: Debian, Lintian

What happened in the Reproducible Builds effort between Sunday August 14 and Saturday August 20 2016: Fasten your seatbelts Important note: we enabled build path variation for unstable now, so your package(s) might become unreproducible, while previously it was said to be reproducible given a specific build path it probably still is reproducible but read on for the details below in the tests.reproducible-builds.org section! As said many times: this is still research and we are working to make it reality. Media coverage Daniel Stender blogged about python packaging and explained some caveats regarding reproducible builds. Toolchain developments Thomas Schmitt uploaded xorriso which now obeys SOURCE_DATE_EPOCH. As stated in its man pages:

ENVIRONMENT
[...]
SOURCE_DATE_EPOCH  belongs to the specs of reproducible-builds.org.  It
is supposed to be either undefined or to contain a decimal number which
tells the seconds since january 1st 1970. If it contains a number, then
it is used as time value to set the  default  of  --modification-date=,
--gpt_disk_guid,  and  --set_all_file_dates.  Startup files and program
options can override the effect of SOURCE_DATE_EPOCH.

Packages reviewed and fixed, and bugs filed The following packages have become reproducible after being fixed:

• cadencii/3.3.9+svn20110818.r1732-5 by Ying-Chun Liu, original patch by Chris Lamb.
• cfengine3/3.7.4-3 by Antonio Radici.
• findbugs/3.1.0~preview2-1 by Emmanuel Bourg.
• gcpegg/5.1-14 by Bdale Garbee, original patch by Chris Lamb.
• gnunet-gtk/0.10.1-4 by Bertrand Marc, original patch by Chris Lamb.
• konwert/1.8-12 by Yann Dirson, original patch by Chris Lamb.
• link-grammar/5.3.8-2 by Jeremy Bicha, original patch by Chris Lamb.
• metastudent-data/2.0.1-1 by Sascha Steinbiss.
• pixmap/2.6pl4-20 by Paul Slootman, original patch by Chris Lamb Maria Valentina Marin a.k.a. Akira.
• python-afl/0.5.4-2 by Daniel Stender.
• tf5/5.0beta8-6 by Russ Allbery, original patch by Chris Lamb.
• triplane/1.0.8-2 by Markus Koschanyover, original patch by Chris Lamb.
• vips/8.3.3-1 by Laszlo Boszormenyi, original patch by Chris Lamb.
• xzgv/0.9.1-4 by Theodore Y. Ts'o, original patch by Chris Lamb.

The following updated packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

• vulkan/1.0.21.0+dfsg1-1 by Timo Aaltonen.

The following 2 packages were not changed, but have become reproducible due to changes in their build-dependencies: tagsoup tclx8.4. Some uploads have addressed some reproducibility issues, but not all of them:

• cappuccino/0.5.1-4 by Breno Leitao, original patch by Chris Lamb.
• darkice/1.3-0.1 by Nicolas Bouleng.
• flask-restful/0.3.5-1 by Ond ej Nov , original patch by Chris Lamb.
• kodi/16.1+dfsg1-2 by Balint Reczey, original patch by Lukas Rechberger.
• libaws/3.3.2-3 by Nicolas Bouleng.
• liblog4ada/1.3-1 by Nicolas Bouleng.
• libxmlezout/1.06.1-8 by Nicolas Bouleng.
• ruby-rmagick/2.15.4+dfsg-2 by Antonio Terceir, original patch from Chris Lamb.
• taggrepper/0.05-2 by Kumar Appaiah, original patch and original patch from Maria Valentina Marin.
• xotcl/1.6.8-2 by Stefan Sobernig, original patch by Chris Lamb.

Patches submitted that have not made their way to the archive yet:

• #834755 against apt-cacher-ng by Chris Lamb.
• #834976 against auto-apt-proxy by Chris Lamb.
• #834549 against botan1.10 by Chris Lamb.
• #834861 against cookiecutter by Chris Lamb.
• #834779 against dicom3tools by Chris Lamb.
• #834859 against echoping by Chris Lamb.
• #834983 against eyed3 by Chris Lamb.
• #834735 against filepp by Chris Lamb.
• #834957 against flashrom by Chris Lamb.
• #834773 against gkrellm by Chris Lamb.
• #834292 against gr-radar by Chris Lamb.
• #834956 against ircd-irc2 by Chris Lamb.
• #834897 against jpy by Chris Lamb.
• #834452 against libdc0 by Chris Lamb.
• #834324 against libnss-ldap by Reiner Herrmann.
• #834945 against libquvi by Chris Lamb.
• #834534 against librep by Chris Lamb.
• #834537 against mozvoikko by Chris Lamb.
• #834857 against nagios-nrpe by Chris Lamb.
• #834316 against openocd by Reiner Herrmann.
• #834993 against oss4 by Reiner Herrmann.
• #834302 against pd-flite by Reiner Herrmann.
• #834754 against phpab by Chris Lamb.
• #834279 against phpldapadmin by Chris Lamb.
• #834277 against pybit by Chris Lamb.
• #834553 against pyicu by Chris Lamb.
• #834541 against ruby-pg by Chris Lamb.
• #835051 against sheepdog by Chris Lamb.
• #834896 against ttf-tiresias by Chris Lamb.
• #834780 against tuxpaint-config by Chris Lamb.
• #834988 against twitter-bootstrap3 by Chris Lamb.
• #834776 against uhub by Chris Lamb.
• #835061 against varnish by Chris Lamb.

Bug tracker house keeping:

• Chris Lamb pinged 164 bugs he filed more than 90 days ago which have a patch and had no maintainer reaction.

Reviews of unreproducible packages 55 package reviews have been added, 161 have been updated and 136 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been updated:

• Added user_in_documentation_generated_by_gsdoc
• Added locale_differences_in_pom

Weekly QA work FTBFS bugs have been reported by:

• Chris Lamb (16)
• Santiago Vila (2)

diffoscope development Chris Lamb, Holger Levsen and Mattia Rizzolo worked on diffoscope this week. Improvements were made to SquashFS and JSON comparison, the https://try.diffoscope.org/ web service, documentation, packaging, and general code quality. diffoscope 57, 58, and 59 were uploaded to unstable by Chris Lamb. Versions 57 and 58 were both broken, so Holger set up a job on jenkins.debian.net to test diffoscope on each git commit. He also wrote a CONTRIBUTING document to help prevent this from happening in future. From these efforts, we were also able to learn that diffoscope is now reproducible even when built across multiple architectures:

< h01ger>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope.html shows these packages were built on amd64:
< h01ger>    bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger>    366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger>   and on i386:
< h01ger>    bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger>    366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger>   and on armhf:
< h01ger>    bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger>    366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb

And those also match the binaries uploaded by Chris in his diffoscope 59 binary upload to ftp.debian.org, yay! Eating our own dogfood and enjoying it! tests.reproducible-builds.org Debian related:

• show percentage of results in the last 24/48h (h01ger)
• switch python database backend to SQLAlchemy (Valerie)
• vary build path varitation for unstable and experimental for all architectures. (h01ger)

The last change probably will have an impact you will see: your package might become unreproducible in unstable and this will be shown on tracker.debian.org, while it will still be reproducible in testing. We've done this, because we think reproducible builds are possible with arbitrary build paths. But: we don't think those are a realistic goal for stretch, where we still recommend to use .buildinfo to record the build patch and then do rebuilds using that path. We are doing this, because besides doing theoretical groundwork we also have a practical goal: enable users to independently verify builds. And if they only can do this with a fixed path, so be it. For now To be clear: for Stretch we recommend that reproducible builds are done in the same build path as the "original" build. Finally, and just for our future references, when we enabled build path variation on Saturday, August 20th 2016, the numbers for unstable were:

suite	all	reproducible	unreproducible	ftbfs	depwait	not for this arch	blacklisted
unstable/amd64	24693	21794 (88.2%)	1753 (7.1%)	972 (3.9%)	65 (0.2%)	95 (0.3%)	10 (0.0%)
unstable/i386	24693	21182 (85.7%)	2349 (9.5%)	972 (3.9%)	76 (0.3%)	103 (0.4%)	10 (0.0%)
unstable/armhf	24693	20889 (84.6%)	2050 (8.3%)	1126 (4.5%)	199 (0.8%)	296 (1.1%)	129 (0.5%)

Misc. Ximin Luo updated our git setup scripts to make it easier for people to write proper descriptions for our repositories. This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.