The question is whether an organization should have a designated leader who is on a sustained, public campaign advocating about an unrelated issue that many consider controversial. It really doesn t matter what your view about the controversial issue is; a leader who refuses to stop talking loudly about unrelated issues eventually creates an untenable distraction from the radical activism you re actively trying to advance. The message of universal software freedom is a radical cause; it s basically impossible for one individual to effectively push forward two unrelated controversial agendas at once. In short, the radical message of software freedom became overshadowed by RMS radical views about sexual morality.There is an open letter calling for the removal of the entire Board of the Free Software Foundation in response. I haven t signed the letter because the Free Software Foundation Board s vote to reinstate Stallman was not unanimous, so the call to remove all of them does not make sense to me. I agree with the open letter s call to remove Stallman from other positions of leadership. I hope that this whole situation can be resolved quickly.
Welcome to the February 2020 report from the Reproducible Builds project. One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the reproducible builds effort is to provide the ability to demonstrate these binaries originated from a particular, trusted, source release: if identical results are generated from a given source in all circumstances, reproducible builds provides the means for multiple third-parties to reach a consensus on whether a build was compromised via distributed checksum validation or some other scheme. In this month s report, we cover:
If you are interested in contributing to the project, please visit our Contribute page on our website.
All computation that occurs inside a DetTrace container is a pure function of the initial filesystem state of the container. Reproducible containers can be used for a variety of purposes, including replication for fault-tolerance, reproducible software builds and reproducible data analytics. We use DetTrace to achieve, in an automatic fashion, reproducibility for 12,130 Debian package builds, containing over 800 million lines of code, as well as bioinformatics and machine learning workflows.There was also considerable discussion on our mailing list regarding this research and a presentation based on the paper will occur at the ASPLOS 2020 conference between March 16th 20th in Lausanne, Switzerland. The many virtues of Reproducible Builds were touted as benefits for software compliance in a talk at FOSDEM 2020, debating whether the Careful Inventory of Licensing Bill of Materials Have Impact of FOSS License Compliance which pitted Jeff McAffer and Carol Smith against Bradley Kuhn and Max Sills. (~47 minutes in). Nobuyoshi Nakada updated the canonical implementation of the Ruby programming language a change such that filesystem globs (ie. calls to list the contents of filesystem directories) will henceforth be sorted in ascending order. Without this change, the underlying nondeterministic ordering of the filesystem is exposed to the language which often results in an unreproducible build. Vagrant Cascadian reported on our mailing list regarding a quick reproducible test for the GNU Guix distribution, which resulted in 81.9% of packages registering as reproducible in his installation:
$ guix challenge --verbose --diff=diffoscope ...
2,463 store items were analyzed:
- 2,016 (81.9%) were identical
- 37 (1.5%) differed
- 410 (16.6%) were inconclusive
Jeremiah Orians announced on our mailing list the release of a number of tools related to cross-compilation such as M2-Planet
and mescc-tools-seed
. This project attemps a full bootstrap of a cross-platform compiler for the C programming language (written in C itself) from hex, the ultimate goal being able to demonstrate fully-bootstrapped compiler from hex to the GCC GNU Compiler Collection. This has many implications in and around Ken Thompson s Trusting Trust attack outlined in Thompson s 1983 Turing Award Lecture.
Twitter user @TheYoctoJester posted an executive summary of reproducible builds in the Yocto Project:
Finally, Reddit user tofflos
posted to the /r/Java subreddit asking about how to achieve reproducible builds with Maven and Chris Lamb noticed that the Linux kernel documentation about reproducible builds of it is available on the kernel.org homepages in an attractive HTML format.
debian-installer
package to allow all arguments and options from sources.list
files (such as [check-valid-until=no]
, etc.) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure. (#13)
Thorsten Glaser followed-up to a bug filed against the dpkg-source
component that was originally filed in late 2015 that claims that the build tool does not respect permissions when unpacking tarballs if the umask is set to 0002
.
Matthew Garrett posted to the debian-devel
mailing list on the topic of Producing verifiable initramfs images as part of a wider conversation on being able to trust the entire software stack on our computers.
59 reviews of Debian packages were added, 30 were updated and 42 were removed this month adding to our knowledge about identified issues. Many issue types were noticed and categorised by Chris Lamb, including:
openstack_pkg_tools_python_shebang_and_dependencies
build_path_in_code_generated_by_dgbus_codegen
random_argument_handling_in_javatools_jh_build
fortran_captures_build_path
python-rpm-macros
(do not save time-based .pyc
files for tests)solfege
(filesystem ordering issue sent upstream via email; package is orphaned upstream)DVDStyler
(zip timestamps, submitted upstream)python-pikepdf
(recreate unreproducible .pyc
files)137
to Debian:
sng
image utility appears to return with an exit code of 1 if there are even minor errors in the file. (#950806)classes2.dex
, classes3.dex
from .apk
files extracted by apktool
. (#88)str.format
if we are just returning the string. [ ]Command.VALID_RETURNCODES
. [ ]Vcs-Git
to specify the debian
packaging branch. [ ]
reprotest is our end-user tool to build same source code twice in widely differing environments and then checks the binaries produced by each build for any differences. This month, versions 0.7.13
and 0.7.14
were uploaded to Debian unstable by Holger Levsen after Vagrant Cascadian added support for GNU Guix [ ].
SOURCE_DATE_EPOCH
documentation [ ] and normalised various terms to unreproducible [ ].
Chris Lamb added a Meson.build example [ ] and improved the documentation for the CMake [ ] to the SOURCE_DATE_EPOCH
documentation, replaced anyone can with anyone may as, well, not everyone has the resources, skills, time or funding to actually do what it refers to [ ] and improved the pre-processing for our report generation [ ][ ][ ][ ] etc.
In addition, Holger Levsen updated our news page to improve the list of reports [ ], added an explicit mention of the weekly news time span [ ] and reverted sorting of news entries to have latest on top [ ] and Mattia Rizzolo added Codethink as a non-fiscal sponsor [ ] and lastly Tianon Gravi added a Docker Images link underneath the Debian project on our Projects page [ ].
rdiff-backup
fixed, build date in Python manpageclick-man
fixed, toolchain, build date in Python manpagerpm
report filesystem-order related nondeterminismpython-enaml
report nondeterministic .enamlc
compiler outputk9s
datepython-xcffib
fix a test with a race conditionsnakemake
.designate
.pynwb
.python-oslo.reports
.mate-desktop
(forwarded upstream).javatools
.xavs2
.snapd-glib
(forwarded upstream).openstack-pkg-tools
.python-django
(Always build the documentation in English)python_example
(sort glob
/ readdir
)autogen
.autoconf
.m4
.intltool
.binutils-dev
.yodl-doc
.gdb-source
.automake-1.15
.libtool
.builtin-pho
database of .buildinfo
files. This has resulted in so that we now know that Debian bullseye contains 4,557 source packages for the amd64
architecture without corresponding .buildinfo
files and 25,668 source packages with .buildinfo
files. [ ][ ][ ][ ][ ]buildinfo@
to the root
user. [ ]devscripts
from buster-backports [ ].i386
architecture, to allow the latter to catch up a bit. [ ]arm64
architecture builders [ ]. The usual build node maintenance was performed by Holger Levsen, Mattia Rizzolo [ ][ ] and Vagrant Cascadian.
#reproducible-builds
on irc.oftc.net
.
rb-general@lists.reproducible-builds.org
This month s report was written by Bernhard M. Wiedemann, Chris Lamb and Holger Levsen. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.
CFLAGS
, CXXFLAGS
, ... etc as used when it was compiled. Which often entails the -g
flag for debugging which can seriously inflate the size of the generated object code. And once stored in $ RHOME /etc/Makeconf
we cannot on the fly override these values.
But there is always a way. Sometimes even two.
The first is local and can be used via the (personal) ~/.R/Makevars
file (about which I will have to say more in another post). But something I have been using quite a bite lately uses the flags for the shared library linker. Given that we can have different code flavours and compilation choices---between C, Fortran and the different C++ standards---one can end up with a few lines. I currently use this which uses -Wl,
to pass an the -S
(or --strip-debug
) option to the linker (and also reiterates the desire for a shared library, presumably superfluous):
SHLIB_CXXLDFLAGS = -Wl,-S -shared
SHLIB_CXX11LDFLAGS = -Wl,-S -shared
SHLIB_CXX14LDFLAGS = -Wl,-S -shared
SHLIB_FCLDFLAGS = -Wl,-S -shared
SHLIB_LDFLAGS = -Wl,-S -shared
Let's consider an example: my most recently uploaded package RProtoBuf. Built under a standard 64-bit Linux setup (Ubuntu 17.04, g++ 6.3) and not using the above, we end up with library containing 12 megabytes (!!) of object code:
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ ls -lh src/RProtoBuf.so
-rwxr-xr-x 1 edd edd 12M Aug 14 20:22 src/RProtoBuf.so
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$
However, if we use the flags shown above in .R/Makevars
, we end up with much less:
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ ls -lh src/RProtoBuf.so
-rwxr-xr-x 1 edd edd 626K Aug 14 20:29 src/RProtoBuf.so
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$
So we reduced the size from 12mb to 0.6mb, an 18-fold decrease. And the file
tool still shows the file as 'not stripped' as it still contains the symbols. Only debugging information was removed.
What reduction in size can one expect, generally speaking? I have seen substantial reductions for C++ code, particularly when using tenmplated code. More old-fashioned C code will be less affected. It seems a little difficult to tell---but this method is my new build default as I continually find rather substantial reductions in size (as I tend to work mostly with C++-based packages).
The second option only occured to me this evening, and complements the first which is after all only applicable locally via the ~/.R/Makevars
file. What if we wanted it affect each installation of a package? The following addition to its src/Makevars
should do:
strippedLib: $(SHLIB)
if test -e "/usr/bin/strip"; then /usr/bin/strip --strip-debug $(SHLIB); fi
.phony: strippedLib
We declare a new Makefile
target strippedLib
. But making it dependent on $(SHLIB)
, we ensure the standard target of this Makefile
is built. And by making the target .phony
we ensure it will always be executed. And it simply tests for the strip
tool, and invokes it on the library after it has been built. Needless to say we get the same reduction is size. And this scheme may even pass muster with CRAN, but I have not yet tried.
Lastly, and acknowledgement. Everything in this post has benefited from discussion with my former colleague Dan Dillon who went as far as setting up tooling in his r-stripper repository. What we have here may be simpler, but it would not have happened with what Dan had put together earlier.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.
buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.
You grant us and our legal successors the right to store and display your Content and make incidental copies as necessary to render the Website and provide the Service.Section D.5 then goes on to say:
[...] You grant each User of GitHub a nonexclusive, worldwide license to access your Content through the GitHub Service, and to use, display and perform your Content, and to reproduce your Content solely on GitHub as permitted through GitHub's functionality
The new TOS is potentially very bad for copylefted Free Software. It potentially neuters it entirely, so GPL licensed software hosted on Github has an implicit BSD-like licenseHess has since removed all his content (mostly mirrors) from GitHub. Others disagree. In a well-reasoned blog post, Debian developer Jonathan McDowell explained the rationale behind the changes:
My reading of the GitHub changes is that they are driven by a desire to ensure that GitHub are legally covered for the things they need to do with your code in order to run their service.This seems like a fair point to make: GitHub needs to protect its own rights to operate the service. McDowell then goes on to do a detailed rebuttal of the arguments made by Glaser, arguing specifically that section D.5 "does not grant [...] additional rights to reproduce outside of GitHub". However, specific problems arise when we consider that GitHub is a private corporation that users have no control over. The "Services" defined in the ToS explicitly "refers to the applications, software, products, and services provided by GitHub". The term "Services" is therefore not limited to the current set of services. This loophole may actually give GitHub the right to bypass certain provisions of licenses used on GitHub. As Hess detailed in a later blog post:
If Github tomorrow starts providing say, an App Store service, that necessarily involves distribution of software to others, and they put my software in it, would that be allowed by this or not? If that hypothetical Github App Store doesn't sell apps, but licenses access to them for money, would that be allowed under this license that they want to my software?However, when asked on IRC, Bradley M. Kuhn of the Software Freedom Conservancy explained that "ultimately, failure to comply with a copyleft license is a copyright infringement" and that the ToS do outline a process to deal with such infringement. Some lawyers have also publicly expressed their disagreement with Glaser's assessment, with Richard Fontana from Red Hat saying that the analysis is "basically wrong". It all comes down to the intent of the ToS, as Kuhn (who is not a lawyer) explained:
any license can be abused or misused for an intent other than its original intent. It's why it matters to get every little detail right, and I hope Github will do that.He went even further and said that "we should assume the ambiguity in their ToS as it stands is favorable to Free Software". The ToS are in effect since February 28th; users "can accept them by clicking the broadcast announcement on your dashboard or by continuing to use GitHub". The immediacy of the change is one of the reasons why certain people are rushing to remove content from GitHub: there are concerns that continuing to use the service may be interpreted as consent to bypass those licenses. Hess even hosted a separate copy of the ToS [PDF] for people to be able to read the document without implicitly consenting. It is, however, unclear how a user should remove their content from the GitHub servers without actually agreeing to the new ToS.
[...] unless there is a Contributor License Agreement to the contrary, whenever you make a contribution to a repository containing notice of a license, you license your contribution under the same terms, and agree that you have the right to license your contribution under those terms.I was concerned this would establish the controversial practice of forcing CLAs on every GitHub user. I managed to find a post from a lawyer, Kyle E. Mitchell, who commented on the draft and, specifically, on the CLA. He outlined issues with wording and definition problems in that section of the draft. In particular, he noted that "contributor license agreement is not a legal term of art, but an industry term" and "is a bit fuzzy". This was clarified in the final draft, in section D.6, by removing the use of the CLA term and by explicitly mentioning the widely accepted norm for licenses: "inbound=outbound". So it seems that section D.6 is not really a problem: contributors do not need to necessarily delegate copyright ownership (as some CLAs require) when they make a contribution, unless otherwise noted by a repository-specific CLA. An interesting concern he raised, however, was with how GitHub conducted the drafting process. A blog post announced the change on February 7th with a link to a form to provide feedback until the 21st, with a publishing deadline of February 28th. This gave little time for lawyers and developers to review the document and comment on it. Users then had to basically accept whatever came out of the process as-is. Unlike every software project hosted on GitHub, the ToS document is not part of a Git repository people can propose changes to or even collaboratively discuss. While Mitchell acknowledges that "GitHub are within their rights to update their terms, within very broad limits, more or less however they like, whenever they like", he sets higher standards for GitHub than for other corporations, considering the community it serves and the spirit it represents. He described the process as:
[...] consistent with the value of CYA, which is real, but not with the output-improving virtues of open process, which is also real, and a great deal more pleasant.Mitchell also explained that, because of its position, GitHub can have a major impact on the free-software world.
And as the current forum of preference for a great many developers, the knock-on effects of their decisions throw big weight. While GitHub have the wheel and they ve certainly earned it for now they can do real damage.In particular, there have been some concerns that the ToS change may be an attempt to further the already diminishing adoption of the GPL for free-software projects; on GitHub, the GPL has been surpassed by the MIT license. But Kuhn believes that attitudes at GitHub have begun changing:
GitHub historically had an anti-copyleft culture, which was created in large part by their former and now ousted CEO, Preston-Warner. However, recently, I've seen people at GitHub truly reach out to me and others in the copyleft community to learn more and open their minds. I thus have a hard time believing that there was some anti-copyleft conspiracy in this ToS change.
While we are confident that these Terms serve the best needs of the community, we take our users' feedback very seriously and we are looking closely at ways to address their concerns.Regardless, free-software enthusiasts have other concerns than the new ToS if they wish to use GitHub. First and foremost, most of the software running GitHub is proprietary, including the JavaScript served to your web browser. GitHub also created a centralized service out of a decentralized tool (Git). It has become the largest code hosting service in the world after only a few years and may well have become a single point of failure for free software collaboration in a way we have never seen before. Outages and policy changes at GitHub can have a major impact on not only the free-software world, but also the larger computing world that relies on its services for daily operation. There are now free-software alternatives to GitHub. GitLab.com, for example, does not seem to have similar licensing issues in its ToS and GitLab itself is free software, although based on the controversial open core business model. The GitLab hosting service still needs to get better than its grade of "C" in the GNU Ethical Repository Criteria Evaluations (and it is being worked on); other services like GitHub and SourceForge score an "F". In the end, all this controversy might have been avoided if GitHub was generally more open about the ToS development process and gave more time for feedback and reviews by the community. Terms of service are notorious for being confusing and something of a legal gray area, especially for end users who generally click through without reading them. We should probably applaud the efforts made by GitHub to make its own ToS document more readable and hope that, with time, it will address the community's concerns.
Note: this article first appeared in the Linux Weekly News.
Great pleasure is to be found not only in keeping up an old and established friendship but also in beginning and building up a new one. Reading this in a beautifully svelte hardback, I tackled a randomly-chosen letter per day rather than attempting to read it cover-to-cover. Breaking with a life-long tradition, I even decided to highlight sections in pen so I could return to them at ease. I hope it's not too hackneyed to claim I gained a lot from "building up" a relationship with this book. Alas, it is one of those books that is too easy to recommend given that it might make one appear wise and learned, but if you find yourself in a slump, either in life or in your reading habits, it certainly has my approval.
wget
froze while pulling Weblate s hook, there was nothing to interrupt it, so the hook stopped working since d montools thought it s already running and wouldn t re-trigger it. Killing wget
helped, but I decided I need to do something with it to prevent the situation from happening in the future.
I ve been using systemd at work for the last year, so I am now confident I m happier with systemd than with d montools, so I decided to switch the server to systemd. Not surprisingly, I prepared unit files in about 5 minutes without having to look into the manuals again, while with d montools I had to check things every time I needed to change something. The tricky thing was the switch itself. It is a virtual server, presumably running in Xen, and I don t have access to the console, so if I init
and sysvinit s init
, I realised I can install systemd as /sbin/init
and ask already running System V init to re-exec. However, systemd s init
can t talk to System V init, so before installing systemd I made a backup on it. It s also important to stop all running services (except probably ssh) to make sure systemd doesn t start second instances of each. And then: /tmp/init u
and we re running systemd! A couple of additional checks, and it s safe to reboot.
Only when I did all that I realised that in the case of systemd not working I d probably not be able to undo my changes if my connection interrupted. So, even though at the end it worked, probably it s not a good idea to perform such manipulations when you don t have an alternative way to connect to the server :)
$ sudo apt-get build-dep telepathy-qt
cd
into that folder$ mkdir ~/telepathy-qt-stuff
$ cd ~/telepathy-qt-stuff
$ apt-get source telepathy-qt
$ apt-get source -b telepathy-qt
$ ls *.deb
$ dpkg -i libtelepathy-qt4-2_0.9.6.1-?_amd64.deb libtelepathy-qt4-dev_0.9.6.1-?_amd64.deb libtelepathy-qt4-farstream2_0.9.6.1-?_amd64.deb
$ dpkg -l grep telepathy-qt
This should return something like :/etc/apt/sources.list
file. Be sure to run sudo apt-get update
after any changes to the source file
$ git clone https://github.com/resiprocate/resiprocate
$ cd resiprocate
$ apt-get install libpq-dev dh-autoreconf
$ apt-get build-dep resiprocate
$ apt-get install -t jessie-backports libradcli-dev
$ ./build/debian.sh
$ sudo make
sudo make check
Steps to install reSIProcate
First you have to configure the telepathy-qt library properly to be able to install reSIProcate. It s important to notice that you shouldn t install telepathy-qt from apt-get because in this way it wont have the telepathy-qt4-service shared library.
$ mkdir ~/telepathy-qt-stuff $ cd ~/telepathy-qt-stuff $ git clone https://github.com/dpocock/telepathy-qt-debian $ cd telepathy-qt-debian $ git checkout jessie-build-all-shared $ cd ..
Then you should download the tar http://http.debian.net/debian/pool/main/t/telepathy-qt/telepathy-qt_0.9.6.1.orig.tar.gz in the telepathy-qt-stuff folder and continue:
$ tar xzf telepathy-qt_0.9.6.1.orig.tar.gz $ cd telepathy-qt_0.9.6.1 $ [ -d debian ] && echo "warning: debian/ already exists!" $ cp -r ../telepathy-qt-debian/debian . $ dpkg-buildpackage -rfakeroot -i.* -j13 -us -uc $ cd .. $ ls *.deb
Now you should see a list of libtelepathy-qt* and telepathy-qt* .deb packages. You just have to install a few more packages:
$ dpkg -i libtelepathy-qt4-2_0.9.6.1-2_amd64.deb libtelepathy-qt4-dev_0.9.6.1-2_amd64.deb libtelepathy-qt4-farstream2_0.9.6.1-2_amd64.deb
After that you have the necessary packages to install reSIProcate.
$ dpkg -l grep telepathy-qt
Should return you something like this:
ii |
libtelepathy-qt4-2:amd64 |
0.9.6.1-2 |
amd64 |
Telepathy framework Qt 4 library |
ii |
libtelepathy-qt4-dev |
0.9.6.1-2 |
amd64 |
Qt 4 Telepathy library (headers and static library) |
ii |
libtelepathy-qt4-farstream2:amd64 |
0.9.6.1-2 |
amd64 |
Telepathy/Farsight integration Qt 4 library |
After installing telepathy-qt properly you would be able to configure reSIProcate.
Make sure you have added backports to your /etc/apt/sources.list file
$ git clone https://github.com/resiprocate/resiprocate $ cd resiprocate $ apt-get install libpq-dev dh-autoreconf $ apt-get build-dep resiprocate $ apt-get install -t jessie-backports libradcli-dev $ ./build/debian.sh $ make
Next.