Welcome to the October 2021 report from the Reproducible Builds project!
This month Samanta Navarro posted to the oss-security security mailing on a novel category of exploit in the .tar archive format, where a single .tar file contains different contents depending on the tar utility being used. Naturally, this has consequences for reproducible builds as Samanta goes onto reply:

Arch Linux uses libarchive (bsdtar) in its build environment. The default tar program installed is GNU tar. It is possible to create a source distribution which leads to different files seen by the build environment than compared to a careful reviewer and other Linux distributions.

I have submitted bug reports and patches to some projects but eventually I had to conclude that the problem itself cannot be fixed by these implementations alone. The best choice for these tools would be to only allow archives which are fully compatible to standards but this in turn would render a lot of archives broken.

Reproducibility, repeatability and traceability of builds, drawing heavily on best-practices championed by the Reproducible Builds project.

I got a PR in my repository a few days ago leading back to a team trying to make it easier for packages to be reproducible from source.

Events and presentations

• PackagingCon is a conference for developers of package management software, their communities and other stakeholders. This virtual event, which will take place on the 9th and 10th November 2021, has a mission is to bring different ecosystems together . The schedule for the event is now available to view online.
• The Linux Foundation s OpenSSF group announced that the next OpenSSF quarterly town hall will take place on 15th November 2021. Registration is now open.

• Last month, Wolfgang Mauerer gave a presentation at the MiniDebConf 2021 Regensburg about the Civil Infrastructure Platform that covered many subjects including Reproducible Builds. PDF slides of the talk are available, as is a video recording.
• In addition, Trevor Rosen from SolarWinds presented at the Linux Foundation s Supply Chain Security Con last month on incorporating in-toto into their build system. in-toto a framework to secure the integrity of software supply chains. Trevor also discusses building everything twice to validate the first build la reproducible builds. (PDF slides)
• Lastly, Mattia Rizzolo posted an update on the next Reproducible Builds in-person event to our mailing list: currently we are thinking ahead to 2022 .

Community news On our mailing list this month:

• Jeremiah announced the release of version 1.4 of stage0-posix, part of a broader effort to provide an ultra-minimal bootstrap seed to increase trust in our software stack.
• Chris Lamb mentioned that Azure are offering free compute power for open source projects which might be useful for one of the many rebuilder projects .
• kpcyrd announced the release of rebuilderd v0.15.0, but also linked to a Twitter thread that contains intro on how rebuilderd works and a walk-through on how to write custom integrations.
• Fredrik Str mberg offered an update on the Sigsum project and some specific milestones within transparency logging efforts: after a year of design iterations we have not only designed a transparency log but also decided to turn it into a project of its own .

There were quite a few changes to the Reproducible Builds website and documentation this month as well, including Feng Chai updating some links on our publications page [ ] and marco updated our project metadata around the Bitcoin Core building guide [ ].
Lastly, we ran another productive meeting on IRC during October. A full set of notes from the meeting is available to view.

Distribution work Qubes was heavily featured in the latest edition of Linux Weekly News, and a significant section was dedicated to discussing reproducibility. For example, it was mentioned that the Qubes project has been working on incorporating reproducible builds into its continuous integration (CI) infrastructure . But the LWN article goes on to describe that:

The current goal is to be able to build the Qubes OS Debian templates solely from packages that can be built reproducibly. Templates in Qubes OS are VM images that can be used to start an application qube quickly based on the template. The qube will have read-only access to the root filesystem of the template, so that the same root filesystem can be shared with multiple application qubes. There are official templates for several variants of both Fedora and Debian, as well as community maintained templates for several other distributions.

You can view the whole article on LWN, and Fr d ric also published a lengthy summary about their work on reproducible builds in Qubes as well for those wishing to learn more.
In Debian this month, 133 reviews of Debian packages were added, 81 were updated and 24 were removed this month, adding to Debian s ever-growing knowledge about identified issues. A number of issues were categorised and added by Chris Lamb and Vagrant Cascadian too [ ][ ][ ]. In addition, work on alternative snapshot service has made progress by Fr d ric Pierret and Holger Levsen this month, including moving from the existing host (snapshot.notset.fr) to snapshot.reproducible-builds.org (more info) thanks to OSUOSL for the machine and hosting and Debian for the disks.
Finally, Bernhard M. Wiedemann posted his monthly reproducible builds status report.

diffoscope diffoscope is our in-depth and content-aware diff utility. Not only can it locate and diagnose reproducibility issues, it can provide human-readable diffs from many kinds of binary formats. This month, Chris Lamb made the following changes, including preparing and uploading versions 186, 187, 188 and 189 to Debian

• New features:
  • Add support for Python Sphinx inventory files (usually named objects.inv on-disk). [ ]
  • Add support for comparing .pyc files. Thanks to Sergei Trofimovich for the inspiration. [ ]
  • Try some alternative suffixes (e.g. .py) to support distributions that strip or retain them. [ ][ ]
• Bug fixes:
  • Fix Python decompilation tests under Python 3.10+ [ ] and for Python 3.7 [ ].
  • Don t raise a traceback if we cannot unmarshal Python bytecode. This is in order to support Python 3.7 failing to load .pyc files generated with newer versions of Python. [ ]
  • Skip Python bytecode testing where we do not have an expected diff. [ ]
• Codebase improvements:
  • Use our file_version_is_lt utility instead of accepting both versions of uImage expected diff. [ ]
  • Split out a custom call to assert_diff for a .startswith equivalent. [ ]
  • Use skipif instead of manual conditionals in some tests. [ ]

In addition, Jelle van der Waa added external tool references for Arch Linux for ocamlobjinfo, openssl and ffmpeg [ ][ ][ ] and added Arch Linux as a Continuous Integration (CI) test target. [ ] and Vagrant Cascadian updated the testsuite to skip Python bytecode comparisons when file(1) is older than 5.39. [ ] as well as added external tool references for the Guix distribution for dumppdf and ppudump. [ ][ ]. Vagrant Cascadian also updated the diffoscope package in GNU Guix [ ][ ]. Lastly, Guangyuan Yang updated the FreeBSD package name on the website [ ], Mattia Rizzolo made a change to override a new Lintian warning due to the new test files [ ], Roland Clobus added support to detect and log if the GNU_BUILD_ID field in an ELF binary been modified [ ], Sandro J ckel updated a number of helpful links on the website [ ] and Sergei Trofimovich made the uImage test output support file() version 5.41 [ ].

reprotest reprotest is the Reproducible Build s project end-user tool to build same source code twice in widely differing environments, checking the binaries produced by the builds for any differences. This month, reprotest version 0.7.18 was uploaded to Debian unstable by Holger Levsen, which also included a change by Holger to clarify that Python 3.9 is used nowadays [ ], but it also included two changes by Vasyl Gello to implement realistic CPU architecture shuffling [ ] and to log the selected variations when the verbosity is configured at a sufficiently high level [ ]. Finally, Vagrant Cascadian updated reprotest to version 0.7.18 in GNU Guix.

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix unreproducible packages. We try to send all of our patches upstream where appropriate. We authored a large number of such patches this month, including:

• Bernhard M. Wiedemann:
  • gtk4 (date-related issue)
  • ipxe (version upgrade)
  • libsoup2 (build failure in the future)
  • pari (parallelism-related issue)
• Chris Lamb:
  • #901307 filed against sphinx-gallery (re-opened with extensive updates).
  • #995809 filed against libinput.
  • #995865 filed against python-pipx.
  • #996200 filed against node-inquirer.
  • #996674 filed against libminidns-java.
  • #996834 filed against pytools.
  • #996881 filed against pikepdf.
  • #996948 filed against sphinx (forwarded upstream)
  • #996999 filed against fenics-basix.
  • #997000 filed against snakemake.
  • #997689 filed against smplayer.
  • #997949 filed against python-duniterpy.
  • #998104 filed against afnix.
  • #998059 filed against sphinx (forwarded upstream).
• Vagrant Cascadian:
  • #977684 filed against mahimahi (filed upstream).
  • #985187 filed against ffmpeg (forwarded upstream)
  • #995646 filed against abntex.
  • #995647 filed against cfi.
  • #995648 filed against cffi.
  • #995650 filed against chktex.
  • #995651 filed against fdutils.
  • #995652 filed against gnu-standards.
  • #995654 filed against malaga.
  • #995741 filed against latex-mk.
  • #995745 filed against kannel.
  • #995747 filed against xnee.
  • #995886, #995896, #995953 & #995954 filed against cxref.
  • #995960 filed against xnee.
  • #996184 filed against binutils-or1k-elf.
  • #996194 & #996572 filed against gcc-arm-none-eabi.
  • #996599 filed against xdmf.
  • #996679 filed against flightgear.
  • #997036 & #997037 filed against kvirc.

Testing framework The Reproducible Builds project runs a testing framework at tests.reproducible-builds.org, to check packages and other artifacts for reproducibility. This month, the following changes were made:

• Holger Levsen:
  • Debian-related changes:
    • Incorporate a fix from bremner into builtin-pho related to binary-NMUs. [ ]
    • Keep bullseye environments around longer, in an attempt to fix a Jenkins issue. [ ]
    • Improve the documentation of buildinfos.debian.net. [ ]
    • Improve documentation for the builtin-pho setup. [ ][ ]
  • OpenWrt-related changes:
    • Also use -j1 for better debugging. [ ]
    • Document that that Python 3.x is now used. [ ]
    • Enable further debugging for the toolchain build. [ ]
  • New snapshot.reproducible-builds.org service:
    • Actually add new node. [ ][ ]
    • Install xfsprogs on snapshot.reproducible-builds.org. [ ]
    • Create account for fpierret on new node. [ ]
    • Run node_health_check job on new node too. [ ]
• Mattia Rizzolo:
  • Debian-related changes:
    • Handle schroot errors when invoking diffoscope instead of masking them. [ ][ ]
    • Declare and define some variables separately to avoid masking the subshell return code. [ ]
    • Fix variable name. [ ]
    • Improve log reporting. [ ]
    • Execute apt-get update with the -q argument to get more decent logs. [ ]
    • Set the Debian HTTP mirror and proxy for snapshot.reproducible-builds.org. [ ]
    • Install the libarchive-tools package (instead of bsdtar) when updating Jenkins nodes. [ ]
  • Be stricter about errors when starting the node agent [ ] and don t overwrite NODE_NAME so that we can expect Jenkins to properly set for us [ ].
  • Explicitly warn if the NODE_NAME is not a fully-qualified domain name (FQDN). [ ]
  • Document whether a node runs in the future. [ ]
  • Disable postgresql_autodoc as it not available in bullseye. [ ]
  • Don t be so eager when deleting schroot internals, call to schroot -e to terminate the schroots instead. [ ]
  • Only consider schroot underlays for deletion that are over a month old. [ ][ ]
  • Only try to unmount /proc if it s actually mounted. [ ]
  • Move the db_backup task to its own Jenkins job. [ ]

Lastly, Vasyl Gello added usage information to the reproducible_build.sh script [ ].

Contributing If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Twitter: @ReproBuilds
• Mailing list: rb-general@lists.reproducible-builds.org

1. Some kind of 45 leaning prong to dry bottles and flasks on
2. A tea tray and coasters
3. a replacement prop arm/foot for my computer keyboard (something like this but for the Lenovo Ultranav)
4. some attempted representation of Borges Library of Babel, a la @jwz The Library of Babel, again
5. an exploration of the geometry of Susanna Clarke s Piranesi
6. further iterations of my castle
7. Small tins to keep loose-leaf tea in
8. Who am I kidding, bound to be a map from DOOM. E1M1 perhaps, or something more regular (MAP07? E2M8?) See also this amazing print of Quake 3: Arena's "Camping Grounds"
9. replacement bits for kids toy sets, e.g. a bolt with long shank from Early Learning Centre Build It Deluxe Set, without all 4 of which you can't build much of anything
10. A stand for a decorative Christmas bauble (kid's hand print on it) A roll of cellotape works pretty well in the mean time
11. DIY bits-and-bobs sorter/storage (nuts and bolts etc)
12. A space ship from Elite/Frontier. Probably a Cobra mk3 or maybe a Viper mk2. In a glow-in-the-dark PLA which i d overpaint with gunmetal except for the fuselage
13. A watch stand/holder/storage thing Except it would look nicer in wood (And I m more inclined to get rid of all but one watch instead)
14. Little tabbed 7 dividers for vinyl records, with A-Z cut into the tabs 12 ones might be a stretch (something like this)
15. A low-profile custom trackpoint cover like the ones by SaotoTech (e.g.)
16. A vinyl record. (Not sure that any 3D printer I would have access to would have the necessary resolution. I haven't done any research yet.)
17. A free-standing inclined vinyl record display stand (e.g.)
18. A "Settlers of Catan" set. I've got the travel edition which is great but it would be nicer to have a larger-sized set. There are some things I really like about the travel set that the full-size set lacks; so designing and printing a larger set myself could incorporate them. Also I don't feel inclined to buy the full-size set for 50 or so to end up with essentially the same game I already bought. No doubt I'd spent at least that much in PLA.
19. Little kids trinkets. Pacman ghosts, that sort-of thing. Whatever my daughters come up with next.
20. Lego storage/sorters.
21. Some kind of lenticular picture. Perhaps a gift or Christmas card combined in one.
22. A bracket to install a Gotek drive in my Amiga 500 (e.g.). I've managed without but the fit isn't great.
23. An attempt at using the 3D printer for 2D drawing. I would never get the same kind of quality results you can get from a proper plotter, but still Take a look at some proper plotter art!
24. Garden decorations. I like the idea of porous geometric shapes that you can plant mosses or ferns into, but also things which might be taken over and "used" by nature in ways I hadn't thought of.
25. Floor plans / 3D plans of my house (including variations if I remodelled)

from contextlib import contextmanager
@contextmanager
def bad():
    print(&aposin the context manager&apos)
    try:
        print("yielding value")
        yield &aposvalue&apos
    finally:
        return print(&aposcleaning up&apos)
@contextmanager
def good():
    print(&aposin the context manager&apos)
    try:
        print("yielding value")
        yield &aposvalue&apos
    finally:
        print(&aposcleaning up&apos)
with bad() as v:
    print(&aposgot v = %s&apos % v)
    raise Exception(&aposexception not raised!&apos)  # SILENCED!
print("this code is reached")
with good() as v:
    print(&aposgot v = %s&apos % v)
    raise Exception(&aposexpection normally raised&apos)
print("NOT REACHED (expected)")

1. define a bad context manager (the things you use with with statements) with contextlib.contextmanager) which:
   1. prints a debug statement
   2. return a value
   3. then returns and prints a debug statement
2. define a good context manager in much the same way, except it doesn't return, it just prints statement
3. use the bad context manager to show how it bypasses an exception
4. use the good context manager to show how it correctly raises the exception

in the context manager
yielding value
got v = value
cleaning up
this code is reached
in the context manager
yielding value
got v = value
cleaning up
Traceback (most recent call last):
  File "/home/anarcat/wikis/anarc.at/wtf.py", line 31, in <module>
    raise Exception('expection normally raised')
Exception: expection normally raised

If the suite was exited due to an exception, and the return value from the exit() method was false, the exception is reraised. If the return value was true, the exception is suppressed, and execution continues with the statement following the with statement.

• [DLA 2788-1] strongswan security update for one CVE
• [DLA 2789-1] squashfs-tools security update for one CVE
• [DLA 2792-1] faad2 security update for seven CVEs
• [DLA 2796-1] jbig2dec security update for two CVEs
• [DLA 2800-1] cups security update for one CVE
• [#998042] for jbig2dec in Buster

• ELA-494-1 for curl
• ELA-497-1 for squashfs-tools
• ELA-498-1 for openssl
• ELA-501-1 for faad2
• ELA-504-1 for jbig2dec
• ELA-508-1 for cups

• c2050
• fxlinuxprint
• c2esp
• lprint
• usb-modeswitch
• cups-bjnp
• dymo-cups-drivers
• pnm2ppa
• sane-airscan
• ippsample

• libapogee3
• libasi
• planetary-system-stacker
• astap
• astap-cli

• sshesame
• c2go

• npd6
• arpalert
• entropybroker

Focus This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

• freetype: restore donations
• OFTC website: cleanups, document account verification
• feed2exec: cleanups
• iotop-py: IRC channel
• whohas: fix URLs (1 2)
• Debian screenshots: deleted qutim (Windows)
• Debian wiki pages: BisectDebian, CrossCompiling, DebianHistory, Derivatives/Census/ToriOS, DeveloperNews, Hardware/Wanted, IRC, Manual-Howto, Mobile, PortsDocs/New, PrivacyIssues, screen, StaticLinking, Teams/DebianOnMobile
• FOSSjobs wiki pages: Resources

Issues

• Crashes in smart-notifier, gnome-shell-extension-shortcuts, bookworm, gdb
• AppArmor issues in avahi
• Conffile removal in gnome-session-common, dkms
• Features in Launchpad, ScanCode, plocate
• New versions of valgrind
• Warnings in smart-notifier
• Usability issues in GreaseMonkey
• Missing systemd integration for ddccontrol
• Syntax error in zephyr, python-editor
• Missing deps in libevolution, intel-gpu-tools
• Leakage in gnome-shell-extension-appindicator
• Broken links in libp11-kit-dev, nm.debian.org
• Out of sync description in libp11-kit-dev
• Incorrect links in cwrap.org
• Unknown errors in sm/freetype

Review

• Spam: reported 2 Debian bug reports and 73 Debian mailing list posts
• Patches: merged whohas/iotop-py/libusbgx patches
• Debian wiki: RecentChanges for the month
• Debian BTS usertags: changes for the month
• Debian screenshots:
  • approved 9menu awesome circuit-macros fvwm gpicview lm-sensors lxqt mousepad openbox qutim task-lxqt-desktop xfce4-settings
  • rejected webhttrack (screenshot of screenshots website), libc6 (random Android app), tesseract-ocr-all (random website), bpytop (hard to read, proprietary background image), p7zip-full (help output)

Administration

• Debian BTS: unarchive/reopen/triage bugs for reintroduced packages
• Debian IRC: disable anti-spam channel modes for some channels, rescue #debian-printing from retired chanop
• Debian wiki: unblock IP addresses, approve accounts

Communication

• Sent Debian Misc Developer News #56
• Respond to queries from Debian users and contributors on the mailing lists and IRC

Sponsors The valgrind bug was sponsored by my employer. All other work was done on a volunteer basis.

Uniform representation The Motoko programming language, designed by Andreas Rossberg and implemented by a pretty cool team at DFINITY is a high-level language with strict semantics and a strong, structural, equi-recursive type system that compiles down to WebAssembly. Because the type system supports polymorphism, it represents all values uniformly. Simplified for the purpose of this blog post, they are always pointers into a heap object where the first word of the heap object, the heap tag, contains information about the value:

 
  tag    
 

The tag is something like array, int64, blob, variant, record, , and it has two purposes:

• The garbage collector uses it to understand what kind of object it is looking at, so that it can move it around and follow pointers therein. Variable size objects such as arrays include the object size in a subsequent word of the heap object.
• Some types have values that may have different shapes on the heap. For example, the ropes used in our text representation can either be a plain blob, or a concatenation node of two blobs. For these types, the tag of the heap object is inspected.

The optional type, naively The optional type (?t) is another example of such a type: Its values can either be null, or ?v for some value v of type t, and the primitive operations on this type are the two introduction forms, an analysis function, and a projection for non-null values:

null : () -> ?t
some : t -> ?t
is_some : ?t -> bool
project : ?t -> t     // must only be used if is_some holds

It is natural to use the heap tag to distinguish the two kind of values:

• The null value is a simple one-word heap object with just a tag that says that this is null:

   
    null  
   

• The other values are represented by a two-word object: The tag some, indicating that it is a ?v, and then the payload, which is the pointer that represents the value v:

   
    some   payload  
   

With this, the four operations can be implemented as follows:

def null():
  ptr <- alloc(1)
  ptr[0] = NULL_TAG
  return ptr
def some(v):
  ptr <- alloc(2)
  ptr[0] = SOME_TAG
  ptr[1] = v
  return ptr
def is_some(p):
  return p[0] == SOME_TAG
def project(p):
  return p[1]

The problem with this implementation is that null() and some(v) both allocate memory. This is bad if they are used very often, and very liberally, and this is the case in Motoko: For example the iterators used for the for (x in e) construct have type

type Iter<T> =   next : () -> ?T  

and would unavoidably allocate a few words on each iteration. Can we avoid this?

Static values It is quite simple to avoid this for for null: Just statically create a single null value and use it every time:

static_null = [NULL_TAG]
def null():
  return static_null

This way, at least null() doesn t allocate. But we gain more: Now every null value is represented by the same pointer, and since the pointer points to static memory, it does not change even with a moving garbage collector, so we can speed up is_some:

def is_some(p):
  return p != static_null

This is not a very surprising change so far, and most compilers out there can and will do at least the static allocation of such singleton constructors. For example, in Haskell, there is only a single empty list ([]) and a single Nothing value in your program, as you can see in my videos exploring the Haskell heap. But can we get rid of the allocation in some(v) too?

Unwrapped optional values If we don t want to allocate in some(v), what can we do? How about simply

def some(v):
  return v

That does not allocate! But it is also broken. At type ??Int, the values null, ?null and ??null are distinct values, but here this breaks. Or, more formally, the following laws should hold for our four primitive operations:

1. is_some(null()) = false
2. v. is_some(some(v)) = true
3. p. project(some(p)) = p

But with the new definition of some, we d get is_some(some(null())) = false. Not good! But note that we only have a problem if we are wrapping a value that is null or some(v). So maybe take the shortcut only then, and write the following:

def some(v):
  if v == static_null   v[0] == SOME_TAG:
    ptr <- alloc(2)
    ptr[0] = SOME_TAG
    ptr[1] = v
    return ptr
  else:
    return v

The definition of is_some can stay as it is: It is still the case that all null values are represented by static_null. But the some values are now no longer all of the same shape, so we have to change project():

def project(p):
  if p[0] == SOME_TAG:
    return p[1]
  else:
    return p

Now things fall into place: A value ?v can, in many cases, be represented the same way as v, and no allocation is needed. Only when v is null or ?null or ??null or ???null etc. we need to use the some heap object, and thus have to allocate. In fact, it doesn t cost much to avoid allocation even for ?null:

static_some_null = [SOME_TAG, static_null]
def some(v):
  if v == static_null:
    return static_some_null
  else if v[0] == SOME_TAG:
    ptr <- alloc(2)
    ptr[0] = SOME_TAG
    ptr[1] = v
    return ptr
  else:
    return v

So unless one nests the ? type two levels deep, there is no allocation whatsoever, and the only cost is a bit more branching in some and project. That wasn t hard, but quite rewarding, as one can now use idioms like the iterator shown above with greater ease.

Examples The following tables shows the representation of various values before and after. Here [ ] is a pointed-to dynamically allocated heap object, a statically allocated heap object, N = NULL_TAG and S = SOME_TAG.

type	value	before	after
Null	null	N	N
?Int	null	N	N
?Int	?23	[S,23]	23
??Int	null	N	N
??Int	?null	[S, N ]	S, N
??Int	??23	[S,[S,23]]	23
???Int	null	N	N
???Int	?null	[S, N ]	S, N
???Int	??null	[S,[S, N ]]	[S, S, N ]
???Int	???23	[S,[S,[S,23]]]	23

Concluding remarks

• If you know what parametric polymorphism is, and wonder how this encoding can work in a language that has that, note that this representation is on the low-level of the untyped run-time value representation: We don t need to know the type of v in some(v), merely its heap representation.
• The uniform representation in Motoko is a bit more involved: The pointers are tagged (by subtracting 1) and some scalar values are represented directly in place (shifted left by 1 bit). But this is luckily orthogonal to what I showed here.
• Could Haskell do this for its Maybe type? Not so easily:
  • The Maybe type is not built-in, but rather a standard library-defined algebraic data type. But the compiler could feasible detect that this is option-like?
  • Haskell is lazy, so at runtime, the type Maybe could be Nothing, or Just v, or, and this is crucial, a yet to be evaluated expression, also called a thunk. And one definitely needs to distinguish between a thunk t :: Maybe a that may evaluate to Nothing, and a value Just t :: Maybe a that definitely is Just, but contains a value, which may be a thunk.
  Something like this may work for a strict Maybe type or unlifted sums like (# (# #) a #), but may clash with other ticks in GHC, such as pointer tagging.
• As I said above, I don t expect this to be novel, and I am happy to add references to prior art here.
• Given that a heap object with tag SOME_TAG now always encodes a tower ? null for n>0, one could try to optimize that even more by just storing the n:

   
    some    n   
   

  But that seems unadvisable: It is only a win if you have deep towers, which is probably rare. Worse, now the project function would have to return such a heap object with n decremented, so now projection might have to allocate, which goes against the cost model expected by the programmer.
• If you rather want to see code than blog posts, feel free to check out Motoko PR #2115.
• Does this kind of stuff excite you? Motoko is open source, so your contributions may be welcome!

Changes in version 0.0.15 (2021-10-26)

• Corrected scope of 'extern "C"' declaration to the actually C-callable function (noticed by clang++, change requested by CRAN)

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

What is Neo-Colonialism? The term "neo-colonialism" was coined by Kwame Nkrumah, first president of Ghana. In Neo-Colonialism, the Last Stage of Imperialism (1965), he wrote:

In place of colonialism, as the main instrument of imperialism, we have today neo-colonialism ... [which] like colonialism, is an attempt to export the social conflicts of the capitalist countries. ... The result of neo-colonialism is that foreign capital is used for the exploitation rather than for the development of the less developed parts of the world. Investment, under neo-colonialism, increases, rather than decreases, the gap between the rich and the poor countries of the world.

So basically, if colonialism is Europeans bringing genocide, war, and its religion to the Africa, Asia, and the Americas, neo-colonialism is the Americans (note the "n") bringing capitalism to the world. Before we see how this applies to the Internet, we must therefore make a detour into US history. This matters, because anyone would be hard-pressed to decouple neo-colonialism from the empire under which it evolves, and here we can only name the United States of America.

US Declaration of Independence Let's start with the United States declaration of independence (1776). Many Americans may roll their eyes at this, possibly because that declaration is not actually part of the US constitution and therefore may have questionable legal standing. Still, it was obviously a driving philosophical force in the founding of the nation. As its author, Thomas Jefferson, stated:

it was intended to be an expression of the American mind, and to give to that expression the proper tone and spirit called for by the occasion

In that aging document, we find the following pearl:

We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness.

As a founding document, the Declaration still has an impact in the sense that the above quote has been called an:

"immortal declaration", and "perhaps [the] single phrase" of the American Revolutionary period with the greatest "continuing importance." (Wikipedia)

Let's read that "immortal declaration" again: "all men are created equal". "Men", in that context, is limited to a certain number of people, namely "property-owning or tax-paying white males, or about 6% of the population". Back when this was written, women didn't have the right to vote, and slavery was legal. Jefferson himself owned hundreds of slaves. The declaration was aimed at the King and was a list of grievances. A concern of the colonists was that the King:

has excited domestic insurrections amongst us, and has endeavoured to bring on the inhabitants of our frontiers, the merciless Indian Savages whose known rule of warfare, is an undistinguished destruction of all ages, sexes and conditions.

This is a clear mark of the frontier myth which paved the way for the US to exterminate and colonize the territory some now call the United States of America. The declaration of independence is obviously a colonial document, having being written by colonists. None of this is particularly surprising, historically, but I figured it serves as a good reminder of where the Internet is coming from, since it was born in the US.

A Declaration of the Independence of Cyberspace Two hundred and twenty years later, in 1996, John Perry Barlow wrote a declaration of independence of cyberspace. At this point, (almost) everyone has a right to vote (including women), slavery was abolished (although some argue it still exists in the form of the prison system); the US has made tremendous progress. Surely this text will have aged better than the previous declaration it is obviously derived from. Let's see how it reads today and how it maps to how the Internet is actually built now.

Borders of Independence One of the key ideas that Barlow brings up is that "cyberspace does not lie within your borders". In that sense, cyberspace is the final frontier: having failed to colonize the moon, Americans turn inwards, deeper into technology, but still in the frontier ideology. And indeed, Barlow is one of the co-founder of the Electronic Frontier Foundation (the beloved EFF), founded six years prior. But there are other problems with this idea. As Wikipedia quotes:

The declaration has been criticized for internal inconsistencies.[9] The declaration's assertion that 'cyberspace' is a place removed from the physical world has also been challenged by people who point to the fact that the Internet is always linked to its underlying geography.[10]

And indeed, the Internet is definitely a physical object. First controlled and severely restricted by "telcos" like AT&T, it was somewhat "liberated" from that monopoly in 1982 when an anti-trust lawsuit broke up the monopoly, a key historical event that, one could argue, made the Internet possible. (From there on, "backbone" providers could start competing and emerge, and eventually coalesce into new monopolies: Google has a monopoly on search and advertisement, Facebook on communications for a few generations, Amazon on storage and computing, Microsoft on hardware, etc. Even AT&T is now pretty much as consolidated as it was before.) The point is: all those companies have gigantic data centers and intercontinental cables. And those are definitely prioritizing the western world, the heart of the empire. Take for example Google's latest 3,900 mile undersea cable: it does not connect Argentina to South Africa or New Zealand, it connects the US to UK and Spain. Hardly a revolutionary prospect.

Private Internet But back to the Declaration:

Do not think that you can build it, as though it were a public construction project. You cannot. It is an act of nature and it grows itself through our collective actions.

In Barlow's mind, the "public" is bad, and private is good, natural. Or, in other words, a "public construction project" is unnatural. And indeed, the modern "nature" of development is private: most of the Internet is now privately owned and operated. I must admit that, as an anarchist, I loved that sentence when I read it. I was rooting for "us", the underdogs, the revolutionaries. And, in a way, I still do: I am on the board of Koumbit and work for a non-profit that has pivoted towards censorship and surveillance evasion. Yet I cannot help but think that, as a whole, we have failed to establish that independence and put too much trust in private companies. It is obvious in retrospect, but it was not, 30 years ago. Now, the infrastructure of the Internet has zero accountability to traditional political entities supposedly representing the people, or even its users. The situation is actually worse than when the US was founded (e.g. "6% of the population can vote"), because the owners of the tech giants are only a handful of people who can override any decision. There's only one Amazon CEO, he's called Jeff Bezos, and he has total control. (Update: Bezos actually ceded the CEO role to Andy Jassy, AWS and Amazon music founder, while remaining executive chairman. I would argue that, as the founder and the richest man on earth, he still has strong control over Amazon.)

Social Contract Here's another claim of the Declaration:

We are forming our own Social Contract.

I remember the early days, back when "netiquette" was a word, it did feel we had some sort of a contract. Not written in standards of course -- or barely (see RFC1855) -- but as a tacit agreement. How wrong we were. One just needs to look at Facebook to see how problematic that idea is on a global network. Facebook is the quintessential "hacker" ideology put in practice. Mark Zuckerberg explicitly refused to be "arbiter of truth" which implicitly means he will let lies take over its platforms. He also sees Facebook as place where everyone is equal, something that echoes the Declaration:

We are creating a world that all may enter without privilege or prejudice accorded by race, economic power, military force, or station of birth.

(We note, in passing, the omission of gender in that list, also mirroring the infamous "All men are created equal" claim of the US declaration.) As the Wall Street Journal's (WSJ) Facebook files later shown, both of those "contracts" have serious limitations inside Facebook. There are VIPs who systematically bypass moderation systems including fascists and rapists. Drug cartels and human traffickers thrive on the platform. Even when Zuckerberg himself tried to tame the platform -- to get people vaccinated or to make it healthier -- he failed: "vaxxer" conspiracies multiplied and Facebook got angrier. This is because the "social contract" behind Facebook and those large companies is a lie: their concern is profit and that means advertising, "engagement" with the platform, which causes increased anxiety and depression in teens, for example. Facebook's response to this is that they are working really hard on moderation. But the truth is that even that system is severely skewed. The WSJ showed that Facebook has translators for only 50 languages. It's a surprisingly hard to count human languages but estimates range the number of distinct languages between 2500 and 7000. So while 50 languages seems big at first, it's actually a tiny fraction of the human population using Facebook. Taking the first 50 of the Wikipedia list of languages by native speakers we omit languages like Dutch (52), Greek (74), and Hungarian (78), and that's just a few random nations picks from Europe. As an example, Facebook has trouble moderating even a major language like Arabic. It censored content from legitimate Arab news sources when they mentioned the word al-Aqsa because Facebook associates it with the al-Aqsa Martyrs' Brigades when they were talking about the Al-Aqsa Mosque... This bias against Arabs also shows how Facebook reproduces the American colonizer politics. The WSJ also pointed out that Facebook spends only 13% of its moderation efforts outside of the US, even if that represents 90% of its users. Facebook spends three more times moderating on "brand safety", which shows its priority is not the safety of its users, but of the advertisers.

Military Internet Sergey Brin and Larry Page are the Lewis and Clark of our generation. Just like the latter were sent by Jefferson (the same) to declare sovereignty over the entire US west coast, Google declared sovereignty over all human knowledge, with its mission statement "to organize the world's information and make it universally accessible and useful". (It should be noted that Page somewhat questioned that mission but only because it was not ambitious enough, Google having "outgrown" it.) The Lewis and Clark expedition, just like Google, had a scientific pretext, because that is what you do to colonize a world, presumably. Yet both men were military and had to receive scientific training before they left. The Corps of Discovery was made up of a few dozen enlisted men and a dozen civilians, including York an African American slave owned by Clark and sold after the expedition, with his final fate lost in history. And just like Lewis and Clark, Google has a strong military component. For example, Google Earth was not originally built at Google but is the acquisition of a company called Keyhole which had ties with the CIA. Those ties were brought inside Google during the acquisition. Google's increasing investment inside the military-industrial complex eventually led Google to workers organizing a revolt although it is currently unclear to me how much Google is involved in the military apparatus. Other companies, obviously, do not have such reserve, with Microsoft, Amazon, and plenty of others happily bidding on military contracts all the time.

Spreading the Internet I am obviously not the first to identify colonial structures in the Internet. In an article titled The Internet as an Extension of Colonialism, Heather McDonald correctly identifies fundamental problems with the "development" of new "markets" of Internet "consumers", primarily arguing that it creates a digital divide which creates a "lack of agency and individual freedom":

Many African people have gained access to these technologies but not the freedom to develop content such as web pages or social media platforms in their own way. Digital natives have much more power and therefore use this to create their own space with their own norms, shaping their online world according to their own outlook.

But the digital divide is certainly not the worst problem we have to deal with on the Internet today. Going back to the Declaration, we originally believed we were creating an entirely new world:

This governance will arise according to the conditions of our world, not yours. Our world is different.

How I dearly wished that was true. Unfortunately, the Internet is not that different from the offline world. Or, to be more accurate, the values we have embedded in the Internet, particularly of free speech absolutism, sexism, corporatism, and exploitation, are now exploding outside of the Internet, into the "real" world. The Internet was built with free software which, fundamentally, was based on quasi-volunteer labour of an elite force of white men with obviously too much time on their hands (and also: no children). The mythical writing of GCC and Emacs by Richard Stallman is a good example of this, but the entirety of the Internet now seems to be running on random bits and pieces built by hit-and-run programmers working on their copious free time. Whenever any of those fails, it can compromise or bring down entire systems. (Heck, I wrote this article on my day off...) This model of what is fundamentally "cheap labour" is spreading out from the Internet. Delivery workers are being exploited to the bone by apps like Uber -- although it should be noted that workers organise and fight back. Amazon workers are similarly exploited beyond belief, forbidden to take breaks until they pee in bottles, with ambulances nearby to carry out the bodies. During peak of the pandemic, workers were being dangerously exposed to the virus in warehouses. All this while Amazon is basically taking over the entire economy. The Declaration culminates with this prophecy:

We will spread ourselves across the Planet so that no one can arrest our thoughts.

This prediction, which first felt revolutionary, is now chilling.

Colonial Internet The Internet is, if not neo-colonial, plain colonial. The US colonies had cotton fields and slaves, we have disposable cell phones and Foxconn workers. Canada has its cultural genocide, Facebook has his own genocides in Ethiopia, Myanmar, and mob violence in India. Apple is at least implicitly accepting the Uyghur genocide. And just like the slaves of the colony, those atrocities are what makes the empire run. The Declaration actually ends like this, a quote which I have in my fortune cookies file:

We will create a civilization of the Mind in Cyberspace. May it be more humane and fair than the world your governments have made before.

That is still inspiring to me. But if we want to make "cyberspace" more humane, we need to decolonize it. Work on cyberpeace instead of cyberwar. Establish clear code of conduct, discuss ethics, and question your own privileges, biases, and culture. For me the first step in decolonizing my own mind is writing this article. Breaking up tech monopolies might be an important step, but it won't be enough: we have to do a culture shift as well, and that's the hard part.

Appendix: an apology to Barlow I kind of feel bad going through Barlow's declaration like this, point by point. It is somewhat unfair, especially since Barlow passed away a few years ago and cannot mount a response (even humbly assuming that he might read this). But then again, he himself recognized he was a bit too "optimistic" in 2009, saying: "we all get older and smarter":

I'm an optimist. In order to be libertarian, you have to be an optimist. You have to have a benign view of human nature, to believe that human beings left to their own devices are basically good. But I'm not so sure about human institutions, and I think the real point of argument here is whether or not large corporations are human institutions or some other entity we need to be thinking about curtailing. Most libertarians are worried about government but not worried about business. I think we need to be worrying about business in exactly the same way we are worrying about government.

And, in a sense, it was a little naive to expect Barlow to not be a colonist. Barlow is, among many things, a cattle rancher who grew up on a colonial ranch in Wyoming. The ranch was founded in 1907 by his great uncle, 17 years after the state joined the Union, and only a generation or two after the Powder River War (1866-1868) and Black Hills War (1876-1877) during which the US took over lands occupied by Lakota, Cheyenne, Arapaho, and other native American nations, in some of the last major First Nations Wars.

Appendix: further reading There is another article that almost has the same title as this one: Facebook and the New Colonialism. (Interestingly, the <title> tag on the article is actually "Facebook the Colonial Empire" which I also find appropriate.) The article is worth reading in full, but I loved this quote so much that I couldn't resist reproducing it here:

Representations of colonialism have long been present in digital landscapes. ( Even Super Mario Brothers, the video game designer Steven Fox told me last year. You run through the landscape, stomp on everything, and raise your flag at the end. ) But web-based colonialism is not an abstraction. The online forces that shape a new kind of imperialism go beyond Facebook.

It goes on:

Consider, for example, digitization projects that focus primarily on English-language literature. If the web is meant to be humanity s new Library of Alexandria, a living repository for all of humanity s knowledge, this is a problem. So is the fact that the vast majority of Wikipedia pages are about a relatively tiny square of the planet. For instance, 14 percent of the world s population lives in Africa, but less than 3 percent of the world s geotagged Wikipedia articles originate there, according to a 2014 Oxford Internet Institute report.

And they introduce another definition of Neo-colonialism, while warning about abusing the word like I am sort of doing here:

I m loath to toss around words like colonialism but it s hard to ignore the family resemblances and recognizable DNA, to wit, said Deepika Bahri, an English professor at Emory University who focuses on postcolonial studies. In an email, Bahri summed up those similarities in list form:

1. ride in like the savior
2. bandy about words like equality, democracy, basic rights
3. mask the long-term profit motive (see 2 above)
4. justify the logic of partial dissemination as better than nothing
5. partner with local elites and vested interests
6. accuse the critics of ingratitude

In the end, she told me, if it isn t a duck, it shouldn t quack like a duck.

Another good read is the classic Code and other laws of cyberspace (1999, free PDF) which is also critical of Barlow's Declaration. In "Code is law", Lawrence Lessig argues that:

computer code (or "West Coast Code", referring to Silicon Valley) regulates conduct in much the same way that legal code (or "East Coast Code", referring to Washington, D.C.) does (Wikipedia)

And now it feels like the west coast has won over the east coast, or maybe it recolonized it. In any case, Internet now christens emperors.

Dropdown for GitHub workflows input parameters Sometimes when we look at CI/CD tools embedded within git-based software repository manager like GitHub, GitLab or Bitbucket, we ran into a lack of some features. This time me and my DevOps/SRE team were facing a pain of not being able to have the option to create drop-downs within GitHub workflows using input parameters. Although this functionality is already available on other platforms such as Bitbucket, the specific client we were working on stored the code inside GitHub. At first I thought that someone has already solved this problem somehow, but doing an extensive search on the internet I found several angry GitHub users opening requests within the Support Community and even in the stack overflow. So I decided to create a solution for this, always thinking about simplicity and in a way that makes it easy to get this missing functionality. I started by creating an input array pattern using commas and using a tag (the selector) e.g brackets as the default value marker. Here is an example of what an input string would look like:

name: gh-action-dropdown-list-input
on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment'
        required: true
        default: 'dev,staging,[uat],prod'

Now the final question that would turn out to be the most complicated to deal with. How can I change the GitHub Actions interface to replace the input pattern we created earlier to a dropdown? The simplest answer I thought was to create a chrome and firefox extension that would do all this logic behind the scenes and replace the HTML input element with the selected tag containing the array values and leaving the tag value (selector) always as the default. All code was developed in pure JavaScript, open-source licensed under Apache 2.0 and available at https://github.com/arthurbdiniz/gh-action-dropdown-list-input.

Install extension

• Chrome
• Firefox

Once installed, the extension is ready to use and the final result we see is the Actions interface with drop-downs. :)

Configuring selectors Go to the top right corner of the browser you are using and click on the extension logo. A screen will popup with tag options. Choose the right tags for you and save it.

This action might require reloading the GitHub workflow tab.

---

Have fun using drop-downs inside GitHub. If you liked this project please share this post and if possible star within the repository. Also feel free to connect with me on LinkedIn: https://www.linkedin.com/in/arthurbdiniz

References

• https://github.community/t/can-workflow-dispatch-input-be-option-list/127338
• https://stackoverflow.com/questions/69296314/dropdown-for-github-workflows-input-parameters

• Adopt and introduce a new Openstack component into our cloud: Manila this was the right choice if we were interested in a general NFS as a service offering for our Cloud VPS users.
• Put the data on Cinder volumes and serve NFS from a couple of virtual machines created by hand this was the right choice if we wanted something that required low effort to engineer and adopt.

• #1946002 generic driver: service instance ends up with 2 ports in the admin network
• #1945463 generic driver: service instance name template is ignored
• #1944980 generic driver: support additional SSH options for service instance
• #1944696 [DOC] document additional generic driver configuration options

• https://review.opendev.org/c/openstack/manila/+/810702
• https://review.opendev.org/c/openstack/manila/+/810719
• https://review.opendev.org/c/openstack/manila/+/811736

• Remove RRA_SET_LIBDIR macro, previously used by PAM modules to guess the install path.
• Fix RRA_SET_LDFLAGS to always use the multilib directory if it exists, required for OpenSSL 3.0 support.
• Perl tests and test support libraries now require Perl 5.10 or later.
• Update RRA_LIB_PYTHON to use sysconfig by preference.
• Fix SUN_LEN on macOS X Big Sur.
• Add support for testing pam_end argments to the PAM testing framework.
• Fix error handling in portable k_haspag replacement if getgroups fails.
• Avoid deprecation warnings in Autoconf macros from Autoconf 2.71.
• Work around fallout for Perl::Critic tests from the renaming of Perl::Critic::Freenode to Perl::Critic::Community, and suppress warnings that I haven't found useful.
• Tweak the perltidy configuration.
• Add a new t/docs/changes.t Perl module test for the format of Changes files.

Changes in version 0.0.14 (2021-10-17)

• Updated continuous integration to run on Linux only.
• Edited checkNonAscii.cpp for readability.
• More robust title display in intradayMarketMonitor.R.
• New C++-based function to read and convert encoding via the R-supplied iconv library, noted a potential variability.
• New function wday returning day of the week as integer.
• The signature to as.data.table was standardized.
• A new function rollMinMax was added illustrating use of the NumVec class from tidyCpp.
• The configure script can record the last commit date and sha1 to automate timestamping builds, but not activated in this release.
• checkCRANStatus() now works correctly for single-package lookups (Jordan Mark Barbone in #4).

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

source /path/to/collab-qa-tools/activate.sh

cqa-scanlogs   grep -v OK > todo

split --lines=30 --numeric-suffixes todo todo

cqa-fetchbugs --TODO=todo01

cqa-annotate --TODO=todo01

#!/bin/sh
set -eu
for todo in $@; do
  # force downloading bugs
  awk ' print(".bugs." $1) ' "$ todo "   xargs rm -f
  cqa-fetchbugs --TODO="$ todo "
  cqa-annotate \
    --template=template.txt.jinja2 \
    --TODO="$ todo "
done

From:   fullname   <  email  >
To: submit@bugs.debian.org
Subject:   package  : FTBFS with ruby3.0:   summary  
Source:   package  
Version:   version   split:'+rebuild'   first  
Severity: serious
Justification: FTBFS
Tags: bookworm sid ftbfs
User: debian-ruby@lists.debian.org
Usertags: ruby3.0
Hi,
We are about to enable building against ruby3.0 on unstable. During a test
rebuild,   package   was found to fail to build in that situation.
To reproduce this locally, you need to install ruby-all-dev from experimental
on an unstable system or build chroot.
Relevant part (hopefully):
 % for line in extract % >   line  
 % endfor % 
The full build log is available at
https://people.debian.org/~kanashiro/ruby3.0/round2/builds/3/  package  /  filename   replace:".log",".build.txt"  

######## ruby-cocaine_0.5.8-1.1+rebuild1633376733_amd64.log ########
--------- Error:
     Failure/Error: undef_method :exitstatus
     FrozenError:
       can't modify frozen object: pid 2351759 exit 0
     # ./spec/support/unsetting_exitstatus.rb:4:in  undef_method'
     # ./spec/support/unsetting_exitstatus.rb:4:in  singleton class'
     # ./spec/support/unsetting_exitstatus.rb:3:in  assuming_no_processes_have_been_run'
     # ./spec/cocaine/errors_spec.rb:55:in  block (2 levels) in <top (required)>'
Deprecation Warnings:
Using  should  from rspec-expectations' old  :should  syntax without explicitly enabling the syntax is deprecated. Use the new  :expect  syntax or explicitly enable  :should  with  config.expect_with(:rspec)    c  c.syntax = :should   instead. Called from /<<PKGBUILDDIR>>/spec/cocaine/command_line/runners/backticks_runner_spec.rb:19:in  block (2 levels) in <top (required)>'.
If you need more of the backtrace for any of these deprecations to
identify where to make the necessary changes, you can configure
 config.raise_errors_for_deprecations! , and it will turn the
deprecation warnings into errors, giving you the full backtrace.
1 deprecation warning total
Finished in 6.87 seconds (files took 2.68 seconds to load)
67 examples, 1 failure
Failed examples:
rspec ./spec/cocaine/errors_spec.rb:54 # When an error happens does not blow up if running the command errored before execution
/usr/bin/ruby3.0 -I/usr/share/rubygems-integration/all/gems/rspec-support-3.9.3/lib:/usr/share/rubygems-integration/all/gems/rspec-core-3.9.2/lib /usr/share/rubygems-integration/all/gems/rspec-core-3.9.2/exe/rspec --pattern ./spec/\*\*/\*_spec.rb --format documentation failed
ERROR: Test "ruby3.0" failed:
----------------
ERROR: Test "ruby3.0" failed:      Failure/Error: undef_method :exitstatus
----------------
package: ruby-cocaine
lines: 30
------------------------------------------------------------------------
s: skip
i: ignore this package permanently
r: report new bug
f: view full log
------------------------------------------------------------------------
Action [s i r f]:

• s - skip this package and do nothing. You can run cqa-annotate again later and come back to it.
• i - ignore this package completely. New runs of cqa-annotate won't ask about it again. This is useful if the package only fails in your rebuilds due to another package, and would just work when that other package gets fixes. In the Ruby transition this happens when A depends on B, while B builds a C extension and failed to build against the new Ruby. So once B is fixed, A should just work (in principle). But even if A would even have problems of its own, we can't really know before B is fixed so we can retry A.
• r - report a bug. cqa-annotate will expand the template with the data from the current log, and feed it to mutt. This is currently a limitation: you have to use mutt to report bugs. After you report the bug, cqa-annotate will ask if it should edit the TODO file. In my opinion it's best to not do this, and annotate the package with a bug number when you have one (see below).
• f - view the full log. This is useful when the extract displayed doesn't have enough info, or you want to inspect something that happened earlier (or later) during the build.

----------------
ERROR: Test "ruby3.0" failed:      Failure/Error: undef_method :exitstatus
----------------
package: ruby-cocaine
lines: 30
------------------------------------------------------------------------
s: skip
i: ignore this package permanently
1: 996206 serious ruby-cocaine: FTBFS with ruby3.0: ERROR: Test "ruby3.0" failed:      Failure/Error: undef_method :exitstatus  
r: report new bug
f: view full log
------------------------------------------------------------------------
Action [s i 1 r f]:

Focus This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

• gadget-tool: build cleanup
• bootstrappable.org: https (1 2)
• iotop-py: fix crash, handle Linux kernel API changes (1 2)
• jq: document security issue
• esprima-python: cleanup typos, links, license mistakes
• feed2exec: save page resources in wayback machine plugin
• purple-discord: add missing linking, fix compile warnings
• check-all-the-things: remove duplicate TODO, add pigz-test check
• devscripts: sync BTS tags
• Debian puppet: block BTS spammer
• Debian package uploads: pytest-rerunfailures, purple-discord (1 2), harmony, quesoglc
• Debian wiki pages: AutomatedUpgrade, Debhelper, DebianInstaller/Build, DebianInstaller/Loader/SuspectedVirus, Derivatives/Census (Pengwin, Quirinux), DeveloperNews, Hardware/Database, InstallingDebianOn/Turris/Omnia, Printing, StaticLinking, Statistics, SuitesAndReposExtension, Teams/ReleaseTeam/ReleaseCheckList
• FOSSjobs wiki pages: Resources

Issues

• Crashes in lintian-brush, gnome-shell-extension-shortcuts
• Broken symlinks in speech-dispatcher, ucx
• Dependency bloat in librecaptcha
• Test hangs in pyemd
• Features in apt-listchanges, evolution (1 2 3), baobab, pipewire, reportbug (1 2), systemd-coredump (1 2), find-dbgsym-packages (1 2), gdb
• Syntax highlighting issues in gtksourceview (1 2)
• Outdated links in duktape
• License change needed in cloudflared
• Rebuild needed for ricochet-im
• New versions of pidgin
• Usability issues in GNOME Keyring certificate viewer (also mistakenly filed against Firefox)
• Underlinking in ucx
• False positives in lintian
• Security issues in systemd-coredump (sent privately, rejected)

Review

• Spam: reported 1 Sourceware Bugzilla comment, 1 KDE Bugzilla comment, 580 Debian bug reports and 178 Debian mailing list posts
• Debian packages: sponsored iotop-c backports
• Debian wiki: RecentChanges for the month
• Debian BTS usertags: changes for the month
• Debian screenshots:
  • approved ark audacious bpfmon chameleon-cursor-theme chromium color-picker convertall convert-pgn cpufrequtils cpuid cpuidtool cpulimit cpustat dclock desktop-base docker.io elementary-xfce-icon-theme engrampa ephoto eterm exadrums exifprobe exiftags file-roller firefox fspanel imgp ipython3 linphone-desktop linux-cpupower live-image-xfce-desktop lxqt-archiver mate-desktop mwm network-manager nm-tray pamix pulsemixer python3-pip supertuxkart transmission-remote-gtk xarchiver xfce4 xfce4-datetime-plugin xscreensaver
  • rejected fonts-recommended (just debian/control), obfsproxy (private data), gpart/p7zip/p7zip-full/unzip/unrar (usage), cruft (doesn't show use of tool), weboob-qt/acetoneiso/klettres (screenshot of screenshots website), default-jre-headless/minetest-mod-basic-materials (illustration), gamine/weboob-qt/libstdc++6/med-imaging (photograph), drumstick-tools (wrong package), totem (newspaper article)

Administration

• Debian BTS: reopened bugs closed by a spammer
• Debian wiki: unblock IP addresses, approve accounts

Communication

• Respond to queries from Debian users and contributors on the mailing lists and IRC

Sponsors The purple-discord/harmony/pyemd/librecaptcha/esprima-python work was sponsored by my employer. All other work was done on a volunteer basis.

Someone is Wrong on the Internet If you look up how to only forward ports with ssh, you may come across solutions like this:

ssh -nNT -L 8000:example.com:80 user@bastion.example.com

Or perhaps this, if you also wanted to send ssh to the background:

ssh -NT -L 3306:db.example.com:3306 example.com &

Both of these use at least one option that is entirely redundant, and the second can cause ssh to fail to connect if you happen to be using password authentication. However, they seem to still persist in various articles about ssh port forwarding. I myself was using the first variation until just recently, and I figured I would write this up to inform others who might be still using these solutions. The correct option for this situation is not -nNT but simply -N, as in:

ssh -N -L 8000:example.com:80 user@bastion.example.com

If you want to also send ssh to the background, then you ll want to add -f instead of using your shell s built-in & feature, because you can then input passwords into ssh if necessary¹ Honestly, that s the point of this article, so you can stop here if you want. If you re looking for a detailed explaination of what each of these options actually does, or if you have no idea what I m talking about, read on!

What is SSH Port Forwarding? ssh is a powerful tool for remote access to servers, allowing you to execute commands on a remote machine. It can also forward ports through a secure tunnel with the -L and -R options. Basically, you can forward a connection to a local port to a remote server like so:

ssh -L 8080:other.example.com:80 ssh.example.com

In this example, you connect to ssh.example.com and then ssh forwards any traffic on your local machine port 8080² to other.example.com port 80 via ssh.example.com. This is a really powerful feature, allowing you to jump³ inside your firewall with just an ssh server exposed to the world. It can work in reverse as well with the -R option, allowing connections on a remote host in to a server running on your local machine. For example, say you were running a website on your local machine on port 8080 but wanted it accessible on example.com port 80⁴. You could use something like:

ssh -R 8080:example.com:80 example.com

The trouble with ssh port forwarding is that, absent any additional options, you also open a shell on the remote machine. If you re planning to both work on a remote machine and use it to forward some connection, this is fine, but if you just need to forward a port quickly and don t care about a shell at that moment, it can be annoying, especially since if the shell closes ssh will close the forwarding port as well. This is where the -N option comes in.

SSH just forwarding ports In the ssh manual page⁵, -N is explained like so:

Do not execute a remote command. This is useful for just forwarding ports.

This is all we need. It instructs ssh to run no commands on the remote server, just forward the ports specified in the -L or -R options. But people seem to think that there are a bunch of other necessary options, so what do those do?

SSH and stdin -n controls how ssh interacts with standard input, specifically telling it not to:

Redirects stdin from /dev/null (actually, prevents reading from stdin). This must be used when ssh is run in the background. A common trick is to use this to run X11 programs on a remote machine. For example, ssh -n shadows.cs.hut.fi emacs & will start an emacs on shadows.cs.hut.fi, and the X11 connection will be automatically for warded over an encrypted channel. The ssh program will be put in the background. (This does not work if ssh needs to ask for a password or passphrase; see also the -f option.)

SSH passwords and backgrounding -f sends ssh to background, freeing up the terminal in which you ran ssh to do other things.

Requests ssh to go to background just before command execution. This is useful if ssh is going to ask for passwords or passphrases, but the user wants it in the background. This implies -n. The recommended way to start X11 programs at a remote site is with something like ssh -f host xterm.

As indicated in the description of -n, this does the same thing as using the shell s & feature with -n, but allows you to put in any necessary passwords first.

SSH and pseudo-terminals -T is a little more complicated than the others and has a very short explanation:

Disable pseudo-terminal allocation.

It has a counterpart in -t, which is explained a little better:

Force pseudo-terminal allocation. This can be used to execute arbitrary screen-based programs on a remote machine, which can be very useful, e.g. when implementing menu services. Multiple -t options force tty allocation, even if ssh has no local tty.

As the description of -t indicates, ssh is allocating a pseudo-terminal on the remote machine, not the local one. However, I have confirmed⁶ that -N doesn t allocate a pseudo-terminal either, since it doesn t run any commands. Thus this option is entirely unnecessary.

What s a pseudo-terminal? This is a bit complicated, but basically it s an interface used in UNIX-like systems, like Linux or BSD, that pretends to be a terminal (thus pseudo-terminal). Programs like your shell, or any text-based menu system made in libraries like ncurses, expect to be connected to one (when used interactively at least). Basically it fakes as if the input it is given (over the network, in the case of ssh) was typed on a physical terminal device and do things like raise an interrupt (SIGINT) if Ctrl+C is pressed.

Why? I don t know why these incorrect uses of ssh got passed around as correct, but I suspect it s a form of cargo cult, where we use example commands others provide and don t question what they do. One stack overflow answer I read that provided these options seemed to think -T was disabling the local pseudo-terminal, which might go some way towards explaining why they thought it was necessary. I guess the moral of this story is to question everything and actually read the manual, instead of just googling it.

1. Not that you SHOULD be using ssh with password authentication anyway, but people do.
2. Only on your loopback address by default, so that you re not allowing random people on your network to use your tunnel.
3. In fact, ssh even supports Jump Hosts, allowing you to automatically forward an ssh connection through another machine.
4. I can t say I recommend a setup like this for anything serious, as you d need to ssh as root to forward ports less than 1024. SSH forwarding is not for permanent solutions, just short-lived connections to machines that would be otherwise inaccessible.
5. Specifically, my source is the ssh(1) manual page in OpenSSH 8.4, shipped as 1:8.4p1-5 in Debian bullseye.
6. I just forwarded ports with -N and then logged in to that same machine and looked at psuedo-terminal allocations via ps ux. No terminal is associated with ssh connections using just the -N option.

qemu-img convert Kali-Linux-2021.3-vmware-amd64.vmdk ../kali

• [1] https://www.kali.org/get-kali/
• [2] https://sourceforge.net/projects/metasploitable/files/latest/download