Chris Lamb: Free software activities in May 2020
Here is my monthly update covering what I have been doing in the free software world during May 2020 (previous month):
-
Opened a pull request against the kitty shell to set a default socket timeout when retrieving remote items via the
icatcommand-line tool. (#659) - Opened a pull request to make the documentation for the Wand Python/ImageMagick graphics library to build in reproducible manner. [...]
- Fixed an issue in my tickle-me-email library that implements Gettings Things Done (GTD)-like behaviours in IMAP inboxes to prevent a traceback when adding text attachments that were not valid UTF-8. ...]
- The Open Source Initiative held their twice-annual multi-day 'face-to-face' board meeting this time held virtually and participated in the accompanying conversations on strategy, tactical and governance issues, as well as the usual discussions regarding licensing and policy (minutes pending). I also attended the regular monthly meeting for Software in the Public Interest (minutes).
-
For my django-slack library (which provides a convenient library between projects using the Django and the Slack chat platform) I reviewed and merged a pull request from Cleiton de Lima to fix an issue due to the
force_textmethod was deprecated favour offorce_str. [...]
- Attended more of OpenUK's Teabreak Tuesday and Future Leaders' Training events as well did more work as part of being on the judging panel of the OpenUK Awards. Nominations are open through 15th June in five different categories.
- Opened a pull request for the Redis key-value database to correct a "conflicts" spelling error. [...]
In Lintian, the static analysis tool for Debian packages:
-
New features:
- Check for packages that use
$ misc:Pre-Depends)in theDependsfield. (#961290) - Check for packages installing icon cache files directly under
/usr/share/icons/hicoloras they will invariably clash with other packages. (#959855) - Check for
Homepagefields indebian/controlthat point to known directory listing pages. (#960366) - Update
data/fields/perl-provides. [...]
- Check for packages that use
-
Bug fixes:
- Mark that
dh-sequence-gnomeprovides the services ofdh_gnomeanddh_gnome_clean. (#958500) - Accommodate commented-out lines when checking for
no-dh-sequencer. (#960485)
- Mark that
-
Reporting/output:
- Clarify error message when the file to check does not exist. (#960970)
- Mark the
debian-rules-uses-as-needed-linker-flagtag as beingpedantic&experimentaluntil at least the release of Debian bullseye. (#959696)
- Code improvements:
Reproducible builds
One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes.
The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom.
Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.
- Opened a pull request to make the documentation for the Wand Python/ImageMagick graphics library to build in reproducible manner. [...]
-
In Debian:
- Submitted 16 patches to fix specific reproducibility issues in ataqv, bowtie2, briquolo, cryptominisat, elinks, mistral, nulib2, process-cpp, properties-cpp, python-watcherclient, tree-puzzle, tree-puzzle, tree-puzzle, vows, wand & wolfssl.
- Kept isdebianreproducibleyet.com up to date. [...]
- Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
-
In disorderfs, our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues, I replaced the term "dirents" in place of "directory entries" in human-readable output/log messages [...] and used the astyle source code formatter with the default settings to the main
disorderfs.cppfile [...]. - Drafted, published and publicised our monthly report.
Elsewhere in our tooling, I made the following changes to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, including preparing and uploading versions 142, 143, 144, 145 and 146 to Debian:
-
Comparison improvements:
- Improve fuzzy matching of JSON files as
filenow supports recognising JSON data. (#106) - Refactor
.changesand.buildinfohandling to show all details (including the GnuPG header and footer components) even when referenced files are not present. (#122) - Use our
BuildinfoFilecomparator (etc.) regardless of whether the associated files (such as theorig.tar.gzand the.deb) are present. [...] - Include GnuPG signature data when comparing
.buildinfo,.changes, etc. [...] - Add support for printing Android APK signatures via
apksigner(1). (#121) - Identify "iOS App Zip archive data" as
.zipfiles. (#116) - Add support for Apple Xcode
.mobilepovisionfiles. (#113)
- Improve fuzzy matching of JSON files as
- Bug fixes:
-
Output improvements:
- Never emit the same
id="foo"anchor reference twice in the HTML output, otherwise identically-named parts will not be able to linked to via a#fooanchor. (#120) - Never emit an empty "id" anchor either; it is not possible to link to
#. [...] - Don't pretty-print the output when using the
--jsonpresenter; it will usually be too complicated to be readable by the human anyway. [...] - Use the SHA256 over MD5 hash when generating page names for the HTML directory-style presenter. (#124)
- Never emit the same
-
Reporting improvements:
- Clarify the message when we truncate the number of lines to standard error [...] and reduce the number of maximum lines printed to 25 as usually the error is obvious by then [...].
- Print the amount of free space that we have available in our temporary directory as a debugging message. [...]
- Clarify
Command [ ] failed with exit codemessages to remove duplicateexited with exitbut also to note thatdiffoscopeis interpreting this as an error. [...] - Don't leak the full path of the temporary directory in
Command [ ] exited with 1messages. (#126) - Clarify the warning message when we cannot import the
debianPython module. [...] - Don't repeat
stderr fromif both commands emit the same output. [...] - Clarify that an external command emits for both files, otherwise it can look like we are repeating itself when, in reality, it is being run twice. [...]
- Testsuite improvements:
-
Dockerfileimprovements:- Add a
.dockerignorefile to whitelist files we actually need in our container. (#105) - Use
ARGinstead ofENVwhen setting up theDEBIAN_FRONTENDenvironment variable at runtime. (#103) - Run as a non-root user in container. (#102)
- Install/remove the
build-essentialduring build so we can install the recommended packages from Git. [...]
- Add a
-
Codebase improvements:
- Bump the officially required version of Python from 3.5 to 3.6. (#117)
- Drop the (default)
shell=Falsekeyword argument tosubprocess.Popenso that the potentially-unsafeshell=Trueis more obvious. [...] - Perform string normalisation in Black [...] and include the Black output in the assertion failure too [...].
- Inline
MissingFile's special handling ofdeb822to prevent leaking through abstract layers. [...][...] - Allow a bare
try/exceptblock when cleaning up temporary files with respect to theflake8quality assurance tool. [...] - Rename
in_dsc_pathtodsc_in_same_dirto clarify the use of this variable. [...] - Abstract out the duplicated parts of the
debian_fallbackclass [...] and add descriptions for the file types. [...] - Various commenting and internal documentation improvements. [...][...]
- Rename the
Opensslcommand class toOpenSSLPKCS7to accommodate other command names with this prefix. [...]
-
Misc:
- Rename the
--debuggercommand-line argument to--pdb. [...] - Normalise filesystem
stat(2)"birth times" (ie.st_birthtime) in the same way we do with thestat(1)command'sAccess:andChange:times to fix a nondeterministic build failure in GNU Guix. (#74) - Ignore case when ordering our file format descriptions. [...]
- Drop, add and tidy various module imports. [...][...][...][...]
- Rename the
I also performed a huge overhaul of diffoscope's website:
- Add a completely new design. [...][...]
- Add a separate, canonical page for every new release. [...][...][...]
- Generate a 'latest release' section and display that with the corresponding date on the homepage. [...]
- Add an RSS feed of our releases [...][...][...][...][...][...] and add to Planet Debian [...].
- Dynamically generate our contributor list [...] and supported file formats [...] from the main Git repository.
- Use Jekyll's
absolute_urlandrelative_urlwhere possible [...][...] and move a number of configuration variables to_config.yml[...][...].
Lastly, I made a large number of changes to the main Reproducible Builds website and documentation:
- Rename the "Who" page to "Projects". [...]
- Ensure that Jekyll enters the
_docssubdirectory to find the_docs/index.mdfile after an internal move. (#27) - Wrap
ltmain.shetc. in preformatted quotes. [...] - Wrap the
SOURCE_DATE_EPOCHPython examples onto more lines to prevent visual overflow on the page. [...] - Correct a "preferred" spelling error. [...]
Debian LTS
-
Investigated and triaged
freerdp,keystone,nginx,tcpreplay&thunderbird, as well as tended to the general upkeep of thedla-needed.txtandela-needed.txtfiles, adding various notes, references, attributions and citations. - Frontdesk duties including responding to user/developer questions, reviewing others' packages, participating in mailing list discussions as well as attending our second regular IRC contributor meeting.
-
Issued DLA 2201-1 to prevent a Denial of Service (DoS) vulnerability the
ntpnetwork time protocol server/client.ntpallowed an "off-path" attacker to block unauthenticated synchronisation via a server mode packet with a spoofed source IP address because transmissions were rescheduled even if a packet lacked a valid "origin timestamp". -
Issued DLA 2203-1 for the SQLite database to prevent a denial of service attack. In the event of a semantic error in an aggregate query, SQLite did not return early from the
resetAccumulator()function which would lead to a crash via a segmentation fault. - Issued DLA 2204-1 for the Mailman mailing list manager to prevent an arbitrary content injection attack.
- Issued DLA 2211-1 in order to prevent an XML external entity vulnerability in log4net, a logging API for the ECMA Common Language Infrastructure (CLI), sometimes referred to as "Mono". This type of attack occurs when XML input containing a reference to an internet-faced entity is processed by a weakly configured XML parser.
- Prepared and issued ELA-229-1 and DLA 2217-1 for the Apache Tomcat Java server to prevent a remote code execution exploit.
Debian
I filed the following bug reports in Debian this month:
-
apksigner: Uses en-dashes (U+2013) in manpage over two hyphens. (#960778) -
devscripts:dd-list -nouresults in "unknown option: [ ]". (#960891) -
node-redis: autopkgtest regressions against Redis 6.x. (#960105)
Lastly, I made the following uploads to Debian:
-
python-django (
3.0.6-1) New upstream bugfix release. -
redis:
6.0.0-1New upstream "General Availability" release.6.0.0-2Mark an autopkgtest as beingflakyfor now.6.0.1-1,6.0.1-2&6.0.1-2~bpo10+1New upstream "General Availability" releases.6.0.3-1&6.0.3-1~bpo10+1New upstream "General Availability" releases.
-
memcached (
1.6.6-1) New upstream release, fixing a build failure ons390xand similar architectures. (#960459) -
bfs (
1.7-1) New upstream release & move todebhelper-compatlevel 13.
4.7.7-1), also uploading it to buster-backports.
The next video (following
Something I ve found myself doing as the pandemic rolls on is picking
out and (re-)reading through sections of the
A new version of
Previously: 
We are happy to announce that registration for
Some people believe that automatic contact tracing apps will
help contain the Coronavirus epidemic. They won t.
Sorry to bring the bad news, but IT and mobile phones and artificial
intelligence will not solve every problem.
In my opinion, those that promise to solve these things with
artificial intelligence / mobile phones / apps / your-favorite-buzzword
are at least overly optimistic and blinder Aktionismus (*),
if not naive, detachted from reality,
or fraudsters that just want to get some funding.
(*) there does not seem to be an English word for this doing something
just for the sake of doing something, without thinking about whether it makes sense to do so
Here are the reasons why it will not work:
TL;DR; For those (admins) of you who run GNU/Linux on staff computers: How do you organize your graphical remote support in your company? Get in touch, share your expertise and experiences.
Researching on FLOSS based Linux Desktops
When bringing GNU/Linux desktops to a generic folk of productive office users on a large scale, graphical remote support is a key feature when organizing helpdesk support teams' workflows.
In a research project that I am currently involved in, we investigate the different available remote support technologies (VNC screen mirroring, ScreenCasts, etc.) and the available frameworks that allow one to provide a remote support infrastructure 100% on-premise.
In this research project we intend to find FLOSS solutions for everything required for providing a large scale GNU/Linux desktop to end users, but we likely will have to recommend non-free solutions, if a FLOSS approach is not available for certain demands. Depending on the resulting costs, bringing forth a new software solution instead of dumping big money in subscription contracts for non-free software is seen as a possible alternative.
As a member of the X2Go upstream team and maintainer of several remote desktop related tools and frameworks in Debian, I'd consider myself as sort of in-the-topic. The available (as FLOSS) underlying technologies for plumbing a remote support framework are pretty much clear (x11vnc, recent pipewire-related approaches in Wayland compositors, browser-based screencasting). However, I still lack a good spontaneous answer to the question: "How to efficiently software-side organize a helpdesk scenario for 10.000+ users regarding graphical remote support?".
Framework for Remote Desktop in Webbrowsers
In fact, in the context of my X2Go activities, I am currently planning to put together a Django-based framework for running X2Go sessions in a web browser. The framework that we will come up with (two developers have already been hired for an initial sprint in July 2020) will be designed to be highly pluggable and it will probably be easy to add remote support / screen sharing features further on.
And still, I walk around with the question in mind: Do I miss anything? Is there anything already out there that provides a remote support solution as 100% FLOSS, that has enterprise grade, that up-scales well, that has a modern UI design, etc. Something that I simply haven't come across, yet?
Looking forward to Your Feedback
Please get in touch (OFTC/Freenode IRC, Telegram, Email), if you can fill the gap and feel like sharing your ideas and experiences.
light+love