It was 2013, and I was on a break from work between Christmas and New Year of
2013. I had been working at Linaro for well over a year, on the LAVA
project. I was living and breathing automated testing infrastructure,
mostly for testing low-level components such as kernels and bootloaders, on
real hardware.
At this point I was also a Debian contributor for quite some years, and had
become an official project members two years prior. Most of my involvement was
in the Ruby team, where we were already consistently running upstream test
suites during package builds.
During that break, I put these two contexts together, and came to the
conclusion that Debian needed a dedicated service that would test the contents
of the Debian archive. I was aware of the existance of
autopkgtest, and started working on a very simple service that
would later become Debian CI.
In January 2014, debci was initially announced on that month's Misc
Developer News, and later uploaded to Debian. It's been
continuously developed for the last 10 years, evolved from a single shell
script running tests in a loop into a distributed system with 47
geographically-distributed machines as of writing this piece, became part of
the official Debian release process gating migrations to testing, had 5 Summer
of Code and Outrechy interns working on it, and processed beyond 40 million
test runs.
In there years, Debian CI has received contributions from a lot of people, but
I would like to give special credits to the following:
Ian Jackson - created autopkgtest.
Martin Pitt - was the maintainer of autopkgtest when Debian CI launched and
helped a lot for some time.
Paul Gevers - decided that he wanted Debian CI test runs to control testing
migration. While at it, became a member of the Debian Release Team and the
other half of the permanent Debian CI team together with me.
Lucas Kanashiro - Google Summer of Code intern, 2014.
Brandon Fairchild - Google Summer of Code intern, 2014.
Candy Tsai - Outreachy intern, 2019.
Pavit Kaur - Google Summer of Code intern, 2021
Abiola Ajadi - Outreachy intern, December 2021-2022.
Em 2023 o tradicional Debian Day est
sendo celebrado de forma especial, afinal no dia 16 de agostoo Debian completou
30 anos!
Para comemorar este marco especial na vida do Debian, a
comunidade Debian Brasil organizou uma semana
de palestras online de 14 a 18 de agosto. O evento foi chamado de
Debian 30 anos.
Foram realizadas 2 palestras por noite, das 19h s 22h, transmitidas pelo canal
Debian Brasil no YouTube
totalizando 10 palestras. As grava es j est o dispon veis tamb m no canal
Debian Brasil no Peertube.
Nas 10 atividades tivemos as participa es de 9 DDs, 1 DM, 3 contribuidores(as).
A audi ncia ao vivo variou bastante, e o pico foi na palestra sobre preseed com
o Eriberto Mota quando tivemos 47 pessoas assistindo.
Obrigado a todos(as) participantes pela contribui o que voc s deram para o
sucesso do nosso evento.
Antonio Terceiro
Aquila Macedo
Charles Melara
Daniel Lenharo de Souza
David Polverari
Eriberto Mota
Giovani Ferreira
Jefferson Maier
Lucas Kanashiro
Paulo Henrique de Lima Santana
Sergio Durigan Junior
Thais Araujo
Thiago Andrade
Veja abaixo as fotos de cada atividade:
Nova gera o: uma entrevista com iniciantes no projeto Debian
Instala o personalizada e automatizada do Debian com preseed
Manipulando patches com git-buildpackage
debian.social: Socializando Debian do jeito Debian
Proxy reverso com WireGuard
Celebra o dos 30 anos do Debian!
Instalando o Debian em disco criptografado com LUKS
O que a equipe de localiza o j conquistou nesses 30 anos
Debian - Projeto e Comunidade!
Design Gr fico e Software livre, o que fazer e por onde come ar
In 2023 the traditional Debian Day is
being celebrated in a special way, after all on August 16th Debian turned 30
years old!
To celebrate this special milestone in the Debian's life, the
Debian Brasil community organized a week with
talks online from August 14th to 18th. The event was named
Debian 30 years.
Two talks were held per night, from 7:00 pm to 10:00 pm, streamed on the
Debian Brasil channel on YouTube
totaling 10 talks. The recordings are also available on the
Debian Brazil channel on Peertube.
We had the participation of 9 DDs, 1 DM, 3 contributors in 10 activities.
The live audience varied a lot, and the peak was on the preseed talk with
Eriberto Mota when we had 47 people watching.
Thank you to all participants for the contribution you made to the success of
our event.
Antonio Terceiro
Aquila Macedo
Charles Melara
Daniel Lenharo de Souza
David Polverari
Eriberto Mota
Giovani Ferreira
Jefferson Maier
Lucas Kanashiro
Paulo Henrique de Lima Santana
Sergio Durigan Junior
Thais Araujo
Thiago Andrade
Veja abaixo as fotos de cada atividade:
Nova gera o: uma entrevista com iniciantes no projeto Debian
Instala o personalizada e automatizada do Debian com preseed
Manipulando patches com git-buildpackage
debian.social: Socializando Debian do jeito Debian
Proxy reverso com WireGuard
Celebra o dos 30 anos do Debian!
Instalando o Debian em disco criptografado com LUKS
O que a equipe de localiza o j conquistou nesses 30 anos
Debian - Projeto e Comunidade!
Design Gr fico e Software livre, o que fazer e por onde come ar
Core Python Packages, by Stefano Rivera
Just before the freeze, pip added support for PEP-668.
This is a scheme devised by Debian with other distributions and the Python
Packaging Authority, to allow distributors to mark Python installations as
being managed by a distribution package manager. When this EXTERNALLY-MANAGED
flag is present, installers like pip will refuse to install packages outside
a virtual environment. This protects users from breaking unrelated software
on their systems, when installing packages with pip or similar tools. Stefano
quickly got this version of pip into the archive, marked Debian s Python
interpreters as EXTERNALLY-MANAGED, and worked with the upstream to add a
mechanism to allow users to override the restriction. Debian bookworm will
likely be the first distro release to implement this change.
The transition from Python 3.10 to 3.11 was one of the last to complete before
the bookworm freeze (as 3.11 only released at the end of October 2022). Stefano
helped port some Python packages to 3.11, in January, and also kicked off the
final transition to remove Python 3.10 support.
Stefano did a big round of bug triage in the cPython interpreter (and related)
packages, applying some provided patches, and fixing some long-standing minor
bugs in the packaging.
To allow Debian packages to more accurately reflect upstream-specified
dependencies that only apply under specific Python interpreter versions, in the
future, Stefano
added more metadata
to the python3 binary package.
Python s unittest runner would successfully exit with 0 passed tests, if it
couldn t find any tests. This means that configuration / layout changes can
cause test failures to go unnoticed, because the tests aren t being run any
more in Debian packages. Stefano
proposed a change to Python
3.12 to change this behavior and treat 0 tests as a kind of failure.
debvm, by Helmut Grohne
With support from Johannes Schauer Marin Rodrigues, and Jochen Sprickerhof,
Helmut Grohne wrote debvm, a tool for
quickly creating and running Debian virtual machine images for various
architectures and Debian and Ubuntu releases. This is meant for development
and testing purposes and has already identified a number of bugs in e.g.
fakechroot (#1029490), Linux
(#1029270), and runit
(#1028181).
Rails 6 and Redmine 5 available in bullseye-backports, by Utkarsh Gupta
Bullseye users can now upgrade to the latest 6.1 branch of Rails, v6.1.7, and
the latest Redmine version, v5.0.4. The Ruby team received numerous requests
to backport the latest version of Rails and Redmine, especially since there was
no redmine shipped in the bullseye release itself. So this is big news for all
users as we ve not only successfully backported both the packages, but also
fixed all the CVEs and RC bugs in the process!
This work was sponsored by Entrouvert.
Patches metadata in the Package Tracker, by Rapha l Hertzog
Building on the great Ultimate Debian Database work of Lucas Nussbaum and on
his suggestion, Rapha l enhanced the
Debian Package Tracker to display action items
when the patches metadata indicate that some patches were not forwarded
upstream, or when the metadata were invalid. One can now also browse the
patches metadata from the Links panel on the right.
Fixed kernel bug that broke debian-installer on computers with Mediatek wifi devices, by Helmut Grohne
As part of our regular work on Kali Linux for OffSec,
they funded Helmut s work to fix the MT7921e driver. When being loaded without
firmware available, it would not register itself, but upon module release it
would unregister itself causing a kernel oops.
This was commonly observed in Kali Linux when reloading the module to add
firmware. Helmut Grohne identified the cause and
sent a patch, a
different variant of which is now
heading into Linux
and available from Kali Linux.
Printing in Debian, by Thorsten Alteholz
There are about 40 packages in Debian that take care of sending output to
printers, scan documents, or even send documents to fax machines. In the light
of the upcoming/already ongoing freeze, these packages had to be updated to
the latest version and bugs had to be fixed. Basically this applies to large
packages like cups, cups-filters, hplip but also the smaller ones that
shouldn t be neglected. All in all Thorsten uploaded 13 packages with new
upstream versions or improved packaging and could resolve 14 bugs. Further
triaging led to 35 bugs that could be closed, either because they were already
fixed and not closed in an earlier upload or they could not be reproduced with
current software versions.
There is also work to do to prepare for the future. Historically, printing on
Linux required finding a PPD file for your printer and finding some software
that is able to render your documents with this PPD. These days, driverless
printing is becoming more common and the use of PPD files has decreased.
In the upcoming version 3.0 of cups, PPD files are no longer supported and so
called printer applications need to be used. In order not to lose the ability
to print documents, this big transition needs to be carefully planned. This
started in the beginning of 2023 and will hopefully be finished with the
release of Debian Trixie. More information can be found in
this Debian Printing Wiki article.
In preparation for this transition Thorsten created three new packages.
Yade update, by Anton Gladky
Last month, Anton updated the yade package to the newest 2023.02a version,
which includes new features.
Yade is a software package for discrete element method (DEM) simulations, which
are widely used in scientific and engineering fields for the simulation of
granular systems. Yade is an open-source project that is being used worldwide
for different tasks, such as geomechanics, civil engineering, mining, and
materials science.
The Yade package in Debian supports different precision levels for its
simulations. This means that researchers and engineers can select the needed
precision level without recompiling the package, saving time and effort.
Miscellaneous contributions
Helmut Grohne continues to improve cross building (mostly Qt) and
architecture bootstrap (mostly loong64 and musl).
With the end of the year approaching fast, I thought putting my year in
retrospective via music would be a fun thing to do.
Albums
In 2022, I added 51 new albums to my collection nearly one a week! I listed
them below in the order in which I acquired them.
I purchased most of these albums when I could and borrowed the rest at
libraries. If you want to browse though, I added links to the album covers
pointing either to websites where you can buy them or to Discogs when digital
copies weren't available1.
Browsing through the albums, I can see my tastes really shifted a lot in the
last few years. I used to listen to a lot of Hip-Hop, but the recent trends in
this genre2 really turn me off. In fact, it seems I didn't add a single
Hip-Hop album to my collection this year...
Metal also continues to dominate the list. Many thanks to Angry Metal
Guy for being the best metal reviewing website out there.
Concerts
2022 was also a big change for me, as I started going to much more concerts than
I previously did. metalfinder has been working great and I'm really happy
with it.
Here are the concerts I went to in 2022:
April 26th: Arch Enemy, Unto Others, Behemoth, Napalm Death
September 16th: NOFX
November 4th: Aeternam
November 7th: Jordi Savall & Hesp rion XXI
November 13th: VOLA
November 19th: Vulgaires Machins, Anti-Flag
November 26th: Soen
December 12h: Oddisee
December 16th: Jail, B ton Arm , Reckless Upstarts, Union Thugs, La Gachette
I'm looking forward continuing to go to a lot of concerts in 2023!
Some of the albums especially the O ! ones are pretty
underground. For most of those, I actually have physical copies I bought
and ripped.
Mostly mumble rap, beats than are less and less sample-based,
extreme commercialisation and lyrics that are less and less political and
engaged.
Desde de setembro de 2015, o
time de publicidade
do Projeto Debian passou a publicar a cada dois meses
listas com os nomes dos(as) novos(as)
Desenvolvedores(as) Debian (DD - do
ingl s Debian Developer) e
Mantenedores(as) Debian
(DM - do ingl s Debian Maintainer).
Estamos aproveitando estas listas para publicar abaixo os nomes dos(as)
brasileiros(as) que se tornaram Desenvolvedores(as) e Mantenedores(as) Debian a
partir de julho de 2015.
Desenvolvedores(as) Debian / Debian Developers / DDs:
Marcos Talau
Esta lista ser atualizada quando o time de publicidade do Debian publicar
novas listas com DMs e DDs e tiver brasileiros.
Para ver a lista completa de Mantenedores(as) e Desenvolvedores(as) Debian,
inclusive outros(as) brasileiros(as) antes de julho de 2015 acesse:
https://nm.debian.org/public/people
Desde de setembro de 2015, o
time de publicidade
do Projeto Debian passou a publicar a cada dois meses
listas com os nomes dos(as) novos(as)
Desenvolvedores(as) Debian (DD - do
ingl s Debian Developer) e
Mantenedores(as) Debian
(DM - do ingl s Debian Maintainer).
Estamos aproveitando estas listas para publicar abaixo os nomes dos(as)
brasileiros(as) que se tornaram Desenvolvedores(as) e Mantenedores(as) Debian a
partir de julho de 2015.
Desenvolvedores(as) Debian / Debian Developers / DDs:
Marcos Talau
Esta lista ser atualizada quando o time de publicidade do Debian publicar
novas listas com DMs e DDs e tiver brasileiros.
Para ver a lista completa de Mantenedores(as) e Desenvolvedores(as) Debian,
inclusive outros(as) brasileiros(as) antes de julho de 2015 acesse:
https://nm.debian.org/public/people
The Ruby team is working now on transitioning to ruby 3.0. Even though most
packages will work just fine, there is substantial amount of packages that
require some work to adapt. We have been doing test rebuilds for a while during
transitions, but usually triaged the problems manually.
This time I decided to try
collab-qa-tools, a set of
scripts Lucas Nussbaum uses when he does archive-wide rebuilds. I'm really glad
that I did, because those tols save a lot of time when processing a large
number of build failures. In this post, I will go through how to triage a set
of build logs using collab-qa-tools.
I have
somesomeimprovements
to the code. Given my last merge request is very new and was not merged yet,
a few of the things I mention here may apply only to
my own ruby3.0 branch.
collab-qa-tools also contains a few tools do perform the builds in the
cloud, but since we already had the builds done, I will not be mentioning that
part and will write exclusively about the triaging tools.
Installing collab-qa-tools
The first step is to clone the git repository. Make sure you have the
dependencies from debian/control installed (a few Ruby libraries).
One of the patches I sent, and was already accepted, is the ability to run it
without the need to install:
source /path/to/collab-qa-tools/activate.sh
This will add the tools to your $PATH.
Preparation
The first think you need to do is getting all your build logs in a directory.
The tools assume .log file extension, and they can be named
$ PACKAGE _*.log or just $ PACKAGE .log.
Creating a TODO file
cqa-scanlogs grep -v OK > todo
todo will contain one line for each log with a summary of the failure, if
it's able to find one. collab-qa-tools has a large set of regular expressions
for finding errors in the build logs
It's a good idea to split the TODO file in multiple ones. This can easily be
done with split(1), and can be used to delimit triaging sessions, and/or to
split the triaging between multiple people. For example this will create
todo into todo00, todo01, ..., each containing 30 lines:
split --lines=30 --numeric-suffixes todo todo
Triaging
You can now do the triaging. Let's say we split the TODO files, and will start
with todo01.
The first step is calling cqa-fetchbugs (it does what it says on the tin):
cqa-fetchbugs --TODO=todo01
Then, cqa-annotate will guide you through the logs and allow you to report
bugs:
cqa-annotate --TODO=todo01
I wrote myself a process.sh wrapper script for cqa-fetchbugs and
cqa-annotate that looks like this:
#!/bin/shset-eufor todo in$@;do# force downloading bugsawk' print(".bugs." $1) '"$ todo" xargs rm-f
cqa-fetchbugs --TODO="$ todo"
cqa-annotate \--template=template.txt.jinja2 \--TODO="$ todo"done
The --template option is a recent contribution of mine. This is a template
for the bug reports you will be sending. It uses
Liquid templates,
which is very similar to Jinja2 for Python. You will notice that I am even
pretending it is Jinja2 to trick vim into doing syntax highlighting
for me. The template I'm using looks like this:
From: fullname<email>
To: submit@bugs.debian.org
Subject: package: FTBFS with ruby3.0: summary
Source: package
Version: versionsplit:'+rebuild'first
Severity: serious
Justification: FTBFS
Tags: bookworm sid ftbfs
User: debian-ruby@lists.debian.org
Usertags: ruby3.0
Hi,
We are about to enable building against ruby3.0 on unstable. During a test
rebuild, package was found to fail to build in that situation.
To reproduce this locally, you need to install ruby-all-dev from experimental
on an unstable system or build chroot.
Relevant part (hopefully):
%forlineinextract% > line %endfor%
The full build log is available at
https://people.debian.org/~kanashiro/ruby3.0/round2/builds/3/package/filenamereplace:".log",".build.txt"
The cqa-annotate loop
cqa-annotate will parse each log file, display an extract of what it found as
possibly being the relevant part, and wait for your input:
######## ruby-cocaine_0.5.8-1.1+rebuild1633376733_amd64.log ########
--------- Error:
Failure/Error: undef_method :exitstatus
FrozenError:
can't modify frozen object: pid 2351759 exit 0
# ./spec/support/unsetting_exitstatus.rb:4:in undef_method'
# ./spec/support/unsetting_exitstatus.rb:4:in singleton class'
# ./spec/support/unsetting_exitstatus.rb:3:in assuming_no_processes_have_been_run'
# ./spec/cocaine/errors_spec.rb:55:in block (2 levels) in <top (required)>'
Deprecation Warnings:
Using should from rspec-expectations' old :should syntax without explicitly enabling the syntax is deprecated. Use the new :expect syntax or explicitly enable :should with config.expect_with(:rspec) c c.syntax = :should instead. Called from /<<PKGBUILDDIR>>/spec/cocaine/command_line/runners/backticks_runner_spec.rb:19:in block (2 levels) in <top (required)>'.
If you need more of the backtrace for any of these deprecations to
identify where to make the necessary changes, you can configure
config.raise_errors_for_deprecations! , and it will turn the
deprecation warnings into errors, giving you the full backtrace.
1 deprecation warning total
Finished in 6.87 seconds (files took 2.68 seconds to load)
67 examples, 1 failure
Failed examples:
rspec ./spec/cocaine/errors_spec.rb:54 # When an error happens does not blow up if running the command errored before execution
/usr/bin/ruby3.0 -I/usr/share/rubygems-integration/all/gems/rspec-support-3.9.3/lib:/usr/share/rubygems-integration/all/gems/rspec-core-3.9.2/lib /usr/share/rubygems-integration/all/gems/rspec-core-3.9.2/exe/rspec --pattern ./spec/\*\*/\*_spec.rb --format documentation failed
ERROR: Test "ruby3.0" failed:
----------------
ERROR: Test "ruby3.0" failed: Failure/Error: undef_method :exitstatus
----------------
package: ruby-cocaine
lines: 30
------------------------------------------------------------------------
s: skip
i: ignore this package permanently
r: report new bug
f: view full log
------------------------------------------------------------------------
Action [s i r f]:
You can then choose one of the options:
s - skip this package and do nothing. You can run cqa-annotate again
later and come back to it.
i - ignore this package completely. New runs of cqa-annotate won't ask
about it again.
This is useful if the package only fails in your rebuilds due to another
package, and would just work when that other package gets fixes. In the Ruby
transition this happens when A depends on B, while B builds a C extension and
failed to build against the new Ruby. So once B is fixed, A should
just work (in principle). But even if A would even have problems of its own,
we can't really know before B is fixed so we can retry A.
r - report a bug. cqa-annotate will expand the template with the data
from the current log, and feed it to mutt. This is currently a limitation:
you have to use mutt to report bugs.
After you report the bug, cqa-annotate will ask if it should edit the TODO
file. In my opinion it's best to not do this, and annotate the package with a
bug number when you have one (see below).
f - view the full log. This is useful when the extract displayed doesn't
have enough info, or you want to inspect something that happened earlier (or
later) during the build.
When there are existing bugs in the package, cqa-annotate will list them
among the options. If you choose a bug number, the TODO file will be annotated
with that bug number and new runs of cqa-annotate will not ask about that
package anymore. For example after I reported a bug for ruby-cocaine for the
issue listed above, I aborted with a ctrl-c, and when I run my process.sh
script again I then get this prompt:
----------------
ERROR: Test "ruby3.0" failed: Failure/Error: undef_method :exitstatus
----------------
package: ruby-cocaine
lines: 30
------------------------------------------------------------------------
s: skip
i: ignore this package permanently
1: 996206 serious ruby-cocaine: FTBFS with ruby3.0: ERROR: Test "ruby3.0" failed: Failure/Error: undef_method :exitstatus
r: report new bug
f: view full log
------------------------------------------------------------------------
Action [s i 1 r f]:
Chosing 1 will annotate the TODO file with the bug number, and I'm done with
this package. Only a few other hundreds to go.
"Again, we see there is nothing you can possess which I cannot take away."
The cinema was a rare and expensive treat in my youth, so I first came across Raiders of the Lost Ark by recording it from television onto a poor quality VHS. I only mention this as it meant I watched a slightly different film to the one intended, as my copy somehow missed off the first 10 minutes. For those not as intimately familiar with the film as me, this is just in time to see a Belloq demand Dr. Jones hand over the Peruvian head (see above), just in time to learn that Indy loathes snakes, and just in time to see the inadvertent reproduction of two Europeans squabbling over the spoils of a foreign land.
What this truncation did to my interpretation of the film (released thirty years ago today on June 19th 1981) is interesting to explore. Without Jones' physical and moral traits being demonstrated on-screen (as well as missing the weighing the gold head and the rollercoaster boulder scene), it actually made the idea of 'Indiana Jones' even more of a mythical archetype. The film wisely withholds Jones' backstory, but my directors cut deprived him of even more, and counterintuitively imbued him with even more of a legendary hue as the elision made his qualities an assumption beyond question. Indiana Jones, if you can excuse the clich , needed no introduction at all.
Good artists copy, great artists steal. And oh boy, does Raiders steal. I've watched this film about twenty times over the past two decades and it's now firmly entered into my personal canon. But watching it on its thirtieth anniversary was different not least because I could situate it in a broader cinematic context. For example, I now see the Gestapo officer in Major Strasser from Casablanca (1942), in fact just as I can with many of Raiders' other orientalist tendencies: not only in its breezy depictions of backwards sand people, but also of North Africa as an entrep t and playground for a certain kind of Western gangster. The opening as well, set in an equally reductionist pseudo-Peru, now feels like Werner Herzog's Aguirre, the Wrath of God (1972) but without, of course, any self-conscious colonial critique.
The imagery of the ark appears to be borrowed from James Tissot's The Ark Passes Over the Jordan, part of the fin de siecle fascination with the occult and (ironically enough given the background of Raiders' director), a French Catholic revival.
I can now also appreciate some of the finer edges that make this film just so much damn fun to watch. For instance, the comic book conceit that Jones and Belloq are a 'shadowy reflection' of one other and that they need 'only a nudge' to make one like the other. As is the idea that Belloq seems to be actually enjoying being evil. I also spotted Jones rejecting the martini on the plane. This feels less like a comment on corrupting effect of alcohol (he drinks rather heavily elsewhere in the film), but rather a subtle distancing from James Bond. This feels especially important given that the action-packed cold open is, let us be honest for a second, ripped straight from the 007 franchise.
John William's soundtracks are always worth mentioning. The corny Raiders March does almost nothing for me, but the highly-underrated 'Ark theme' certainly does. I delight in its allusions to Gregorian chant, the diabolus in musica and the Hungarian minor scale, fusing the Christian doctrine of the Holy Trinity (the stacked thirds, get it?), the ars antiqua of the Middle Ages with an 'exotic' twist that the Russian Five associated with central European Judaism.
The best use of the ark leitmotif is, of course, when it is opened. Here, Indy and Marion are saved by not opening their eyes whilst the 'High Priest' Belloq and the rest of the Nazis are all melted away. I'm no Biblical scholar, but I'm almost certain they were alluding to Leviticus 16:2 here:
The Lord said to Moses: Tell your brother Aaron that he is not to come whenever he chooses into the Most Holy Place behind the curtain in front of the atonement cover on the ark, or else he will die, for I will appear in the cloud above the mercy seat.
But would it be too much of a stretch to also see the myth of Orpheus and Eurydices too? Orpheus's wife would only be saved from the underworld if he did not turn around until he came to his own house. But he turned round to look at his wife, and she instantly slipped back into the depths:
For he who overcome should turn back his gaze
Towards the Tartarean cave,
Whatever excellence he takes with him
He loses when he looks on those below.
Perhaps not, given that Marion and the ark are not lost in quite the same way. But whilst touching on gender, it was interesting to update my view of archaeologist Ren Belloq. To countermand his slight queer coding (a trope of Disney villains such as Scar, Jafar, Cruella, etc.), there is a rather clumsy subplot involving Belloq repeatedly (and half-heartedly) failing to seduce Marion. This disavows any idea that Belloq isn't firmly heterosexual, essential for the film's mainstream audience, but it is especially important in Raiders because, if we recall the relationship between Belloq and Jones: 'it would take only a nudge to make you like me'. (This would definitely put a new slant on 'Top men'.)
However, my favourite moment is where the Nazis place the ark in a crate in order to transport it to the deserted island. On route, the swastikas on the side of the crate spontaneously burn away, and a disturbing noise is heard in the background. This short scene has always fascinated me, partly because it's the first time in the film that the power of the ark is demonstrated first-hand but also because gives the object an other-worldly nature that, to the best of my knowledge, has no parallel in the rest of cinema.
Still, I had always assumed that the Aak disfigured the swastikas because of their association with the Nazis, interpreting the act as God's condemnation of the Third Reich. But now I catch myself wondering whether the ark would have disfigured any iconography as a matter of principle or whether their treatment was specific to the swastika. We later get a partial answer to this question, as the 'US Army' inscriptions in the Citizen Kane warehouse remain untouched.
Far from being an insignificant concern, the filmmakers appear to have wandered into a highly-contested theological debate. As in, if the burning of the swastika is God's moral judgement of the Nazi regime, then God is clearly both willing and able to intervene in human affairs. So why did he not, to put it mildly, prevent Auschwitz? From this perspective, Spielberg appears to be limbering up for some of the academic critiques surrounding Holocaust representations that will follow Schindler's List (1993).
Given my nostalgic and somewhat ironic attachment to Raiders, it will always be difficult for me to objectively appraise the film. Even so, it feels like it is underpinned by an earnest attempt to entertain the viewer, largely absent in the affected cynicism of contemporary cinema. And when considered in the totality of Hollywood's output, its tonal and technical flaws are not actually that bad or at least Marion's muddled characterisation and its breezy chauvinism (for example) clearly have far worse examples.
Perhaps the most remarkable thing about the film in 2021 is that it hasn't changed that much at all. It spawned one good sequel (The Last Crusade), one bad one (The Temple of Doom), and one hardly worth mentioning at all, yet these adventures haven't affected the original Raiders in any meaningful way. In fact, if anything has affected the original text it is, once again, George Lucas himself, as knowing the impending backlash around the Star Wars prequels adds an inadvertent paratext to all his earlier works.
Yet in a 1978 discussion prior to the creation of Raiders, you can get a keen sense of how Lucas' childlike enthusiasm will always result in something either extremely good or something extremely bad somehow no middle ground is quite possible. Yes, it's easy to rubbish his initial ideas 'We'll call him Indiana Smith! but hasn't Lucas actually captured the essence of a heroic 'Americana' here, and that the final result is simply a difference of degree, not kind?
"Again, we see there is nothing you can possess which I cannot take away."
The cinema was a rare and expensive treat in my youth, so I first came across Raiders of the Lost Ark by recording it from television onto a poor quality VHS. I only mention this as it meant I watched a slightly different film to the one intended, as my copy somehow missed off the first 10 minutes. For those not as intimately familiar with the film as me, this is just in time to see a Belloq demand Dr. Jones hand over the Peruvian head (see above), just in time to learn that Indy loathes snakes, and just in time to see the inadvertent reproduction of two Europeans squabbling over the spoils of a foreign land.
What this truncation did to my interpretation of the film (released thirty years ago today on June 19th 1981) is interesting to explore. Without Jones' physical and moral traits being demonstrated on-screen (as well as missing the weighing the gold head and the rollercoaster boulder scene), it actually made the idea of 'Indiana Jones' even more of a mythical archetype. The film wisely withholds Jones' backstory, but my directors cut deprived him of even more, and counterintuitively imbued him with even more of a legendary hue as the elision made his qualities an assumption beyond question. Indiana Jones, if you can excuse the clich , needed no introduction at all.
Good artists copy, great artists steal. And oh boy, does Raiders steal. I've watched this film about twenty times over the past two decades and it's now firmly entered into my personal canon. But watching it on its thirtieth anniversary was different not least because I could situate it in a broader cinematic context. For example, I now see the Gestapo officer in Major Strasser from Casablanca (1942), in fact just as I can with many of Raiders' other orientalist tendencies: not only in its breezy depictions of backwards sand people, but also of North Africa as an entrep t and playground for a certain kind of Western gangster. The opening as well, set in an equally reductionist pseudo-Peru, now feels like Werner Herzog's Aguirre, the Wrath of God (1972) but without, of course, any self-conscious colonial critique.
The imagery of the ark appears to be borrowed from James Tissot's The Ark Passes Over the Jordan, part of the fin de siecle fascination with the occult and (ironically enough given the background of Raiders' director), a French Catholic revival.
I can now also appreciate some of the finer edges that make this film just so much damn fun to watch. For instance, the comic book conceit that Jones and Belloq are a 'shadowy reflection' of one other and that they need 'only a nudge' to make one like the other. As is the idea that Belloq seems to be actually enjoying being evil. I also spotted Jones rejecting the martini on the plane. This feels less like a comment on corrupting effect of alcohol (he drinks rather heavily elsewhere in the film), but rather a subtle distancing from James Bond. This feels especially important given that the action-packed cold open is, let us be honest for a second, ripped straight from the 007 franchise.
John William's soundtracks are always worth mentioning. The corny Raiders March does almost nothing for me, but the highly-underrated 'Ark theme' certainly does. I delight in its allusions to Gregorian chant, the diabolus in musica and the Hungarian minor scale, fusing the Christian doctrine of the Holy Trinity (the stacked thirds, get it?), the ars antiqua of the Middle Ages with an 'exotic' twist that the Russian Five associated with central European Judaism.
The best use of the ark leitmotif is, of course, when it is opened. Here, Indy and Marion are saved by not opening their eyes whilst the 'High Priest' Belloq and the rest of the Nazis are all melted away. I'm no Biblical scholar, but I'm almost certain they were alluding to Leviticus 16:2 here:
The Lord said to Moses: Tell your brother Aaron that he is not to come whenever he chooses into the Most Holy Place behind the curtain in front of the atonement cover on the ark, or else he will die, for I will appear in the cloud above the mercy seat.
But would it be too much of a stretch to also see the myth of Orpheus and Eurydices too? Orpheus's wife would only be saved from the underworld if he did not turn around until he came to his own house. But he turned round to look at his wife, and she instantly slipped back into the depths:
For he who overcome should turn back his gaze
Towards the Tartarean cave,
Whatever excellence he takes with him
He loses when he looks on those below.
Perhaps not, given that Marion and the ark are not lost in quite the same way. But whilst touching on gender, it was interesting to update my view of archaeologist Ren Belloq. To countermand his slight queer coding (a trope of Disney villains such as Scar, Jafar, Cruella, etc.), there is a rather clumsy subplot involving Belloq repeatedly (and half-heartedly) failing to seduce Marion. This disavows any idea that Belloq isn't firmly heterosexual, essential for the film's mainstream audience, but it is especially important in Raiders because, if we recall the relationship between Belloq and Jones: 'it would take only a nudge to make you like me'. (This would definitely put a new slant on 'Top men'.)
However, my favourite moment is where the Nazis place the ark in a crate in order to transport it to the deserted island. On route, the swastikas on the side of the crate spontaneously burn away, and a disturbing noise is heard in the background. This short scene has always fascinated me, partly because it's the first time in the film that the power of the ark is demonstrated first-hand but also because gives the object an other-worldly nature that, to the best of my knowledge, has no parallel in the rest of cinema.
Still, I had always assumed that the Aak disfigured the swastikas because of their association with the Nazis, interpreting the act as God's condemnation of the Third Reich. But now I catch myself wondering whether the ark would have disfigured any iconography as a matter of principle or whether their treatment was specific to the swastika. We later get a partial answer to this question, as the 'US Army' inscriptions in the Citizen Kane warehouse remain untouched.
Far from being an insignificant concern, the filmmakers appear to have wandered into a highly-contested theological debate. As in, if the burning of the swastika is God's moral judgement of the Nazi regime, then God is clearly both willing and able to intervene in human affairs. So why did he not, to put it mildly, prevent Auschwitz? From this perspective, Spielberg appears to be limbering up for some of the academic critiques surrounding Holocaust representations that will follow Schindler's List (1993).
Given my nostalgic and somewhat ironic attachment to Raiders, it will always be difficult for me to objectively appraise the film. Even so, it feels like it is underpinned by an earnest attempt to entertain the viewer, largely absent in the affected cynicism of contemporary cinema. And when considered in the totality of Hollywood's output, its tonal and technical flaws are not actually that bad or at least Marion's muddled characterisation and its breezy chauvinism (for example) clearly have far worse examples.
Perhaps the most remarkable thing about the film in 2021 is that it hasn't changed that much at all. It spawned one good sequel (The Last Crusade), one bad one (The Temple of Doom), and one hardly worth mentioning at all, yet these adventures haven't affected the original Raiders in any meaningful way. In fact, if anything has affected the original text it is, once again, George Lucas himself, as knowing the impending backlash around the Star Wars prequels adds an inadvertent paratext to all his earlier works.
Yet in a 1978 discussion prior to the creation of Raiders, you can get a keen sense of how Lucas' childlike enthusiasm will always result in something either extremely good or something extremely bad somehow no middle ground is quite possible. Yes, it's easy to rubbish his initial ideas 'We'll call him Indiana Smith! but hasn't Lucas actually captured the essence of a heroic 'Americana' here, and that the final result is simply a difference of degree, not kind?
Today marks the 90th day of me joining Canonical to work on Ubuntu full-time! So since
it s been a while already, this blog post is long due. :)
The News
I joined Canonical, this February, to work on Ubuntu full-time! \o/
Those who know, they know that this is really very exciting for me because Canonical has
been a dream company for me, for real (more about this below!). And hey, this is my first
job, ever, so all the more reason to be psyched about, isn t it? ^_^
P.S. Keep reading and we ll meet my squad really sooon!
The Story
Being an undergrad student (batch 2017-2021), I ve been slightly worried during my last
two semesters, naturally, thinking about how s it all gonna pan out and what will I be doing,
et al, because I ve been seeing all my friends and batchmates getting placed in companies
or going for masters or at least having some sort of plans for their future and I, on the
other hand, was hopelessly clueless. :D
Well, to be fair, I did Google Summer of Code twice,
in 2019 and
2020, became a
Debian Developer in 2019, been a part of
GCI and Outreachy, contributed
to over dozens of open-source projects, et al, et al. So I wasn t all completely hopeless
but for sure was completely clueless , heh.
And for full disclosure, I was only slightly panicking because firstly, I did get placed
in several companies and secondly, I didn t really need a job immediately since I was already
getting paid to work on Debian stuff by Freexian, which was good enough. :)
(and honestly, Freexian has my whole heart! - more on that later sometime.)
But that s not the point. I was still confused and worried and my mom & dad, more so than
anyone. Ugh. We were all figuring out and she asked me places that I was interested to work
in. And whilst I wasn t clear about things I wanted to do (and still am!) but I was (very)
clear about this and so I told her about Canonical and also did tell her that it s a bit too
ambitious for me to think about it now so I ll probably apply after some experience or something.
and as they say, the world works in mysterious ways and well, it did for me! So back during
the Ruby sprints (Feb 20), Kanashiro, the guy ( ), mentioned that his team was hiring and
has a vacant position but I won t be eligible since I was still in my junior year. It was
since then I ve been actively praying for Cronus, the god of time, to wave his magic wand and
align it in such a way that the next opening should be somewhere near my graduation. And guess
what? IT HAPPENED!
9 months later, in November 20, Kanashiro told me his team is hiring yet again and that I
could apply this time! Without much (since there was some ) delay, I applied and started
asking all sorts of questions to Kanashiro. No words are enough for him, he literally helped
me throughout the process; from referring me to answering all sorts of doubts I had!
And roughly after 2 months of interviewing, et al, my ambitious dream did come true and I
finalyyyy signed my contract! \o/
(the interview process and what went on during those 10 weeks is a story for later ;))
The Server Team! \o
This position, which I didn t mention earlier, was for the Server Team which is a team of
15 people, working to make Ubuntu server the best! And as I
tweeted sometime back, the team
is absolutely lovely, super kind, and consists of the best of teammates one could possibly
ask for!
Here s a quick sneak peek into our weekly team meeting. Thanks to
Rafael for taking such a lovely picture. And yes,
the cat Luna is a part of our squad!
And oh, did I mention that we re completely remote and distributed?
FUN FACT: Our team covers all the TZs, that is, at any point of time (during weekdays),
you ll find someone or the other from the team around! \o/
Anyway, our squad, managed by Rick is divided into two halves: Squeaky Wheels and
Table Flip. Cool names, right?
Squeaky Wheels does the distro side of stuff and consists of Christian, Andreas, Rafael, Robie,
Bryce, Sergio, Kanashiro, Athos, and now myself as well! And OTOH, Table Flip consists of Dan,
Chad, Paride, Lucas, James, and Grant.
Even though I interact w/ Squeaky Wheels more (basically daily), each of my teammates is
absolutely lovely and equally awesome!
Whilst I ll talk more about things here in the upcoming months, this is it for now! If there s
anything, in particular, you d like to know more about, let me know!
And lastly, here s us vibing our way through, making Ubuntu server better, cause that s how
we roll!
Until next time. :wq for today.
Welcome to the October 2020 report from the Reproducible Builds project.
In our monthly reports, we outline the major things that we have been up to over the past month. As a brief reminder, the motivation behind the Reproducible Builds effort is to ensure flaws have not been introduced in the binaries we install on our systems. If you are interested in contributing to the project, please visit our main website.
The previous year has seen great progress in Arch Linux to get reproducible builds in the hands of the users and developers. In this talk we will explore the current tooling that allows users to reproduce packages, the rebuilder software that has been written to check packages and the current issues in this space.
GNU Mes rebuild is definitely an application of [Diverse Double-Compiling]. [..] This is an awesome application of DDC, and I believe it s the first publicly acknowledged use of DDC on a binary
Debian-related work
In August, Lucas Nussbaum performed an archive-wide rebuild of packages to test enabling the reproducible=+fixfilepath Debian build flag by default. Enabling this fixfilepath feature will likely fix reproducibility issues in an estimated 500-700 packages. However, this month Vagrant Cascadian posted to the debian-devel mailing list:
It would be great to see the reproducible=+fixfilepath feature enabled by default in dpkg-buildflags, and we would like to proceed forward with this soon unless we hear any major concerns or other outstanding issues. [ ] We would like to move forward with this change soon, so please raise any concerns or issues not covered already.
Debian Developer Stuart Prescott has been improving python-debian, a Python library that is used to parse Debian-specific files such as changelogs, .dscs, etc. In particular, Stuart is working on adding support for .buildinfo files used for recording reproducibility-related build metadata:
This can mostly be a very thin layer around the existing Deb822 types, using the existing Changes code for the file listings, the existing PkgRelations code for the package listing and gpg_* functions for signature handling.
Bernhard M. Wiedemann also reported three issues against bison, ibus and postgresql12.
Tools
diffoscope is our in-depth and content-aware diff utility. Not only could you locate and diagnose reproducibility issues, it provides human-readable diffs of all kinds too. This month, Chris Lamb uploaded version 161 to Debian (later backported by Mattia Rizzolo), as well as made the following changes:
Move test_ocaml to the assert_diff helper. []
Update tests to support OCaml version 4.11.1. Thanks to Sebastian Ramacher for the report. (#972518)
Bump minimum version of the Black source code formatter to 20.8b1. (#972518)
In addition, Jean-Romain Garnier temporarily updated the dependency on radare2 to ensure our test pipelines continue to work [], and for the GNU Guix distribution Vagrant Cascadian diffoscope to version 161 [].
In related development, trydiffoscope is the web-based version of diffoscope. This month, Chris Lamb made the following changes:
Mark a --help-only test as being a superficial test. (#971506)
Add a real, albeit flaky, test that interacts with the try.diffoscope.org service. []
Bump debhelper compatibility level to 13 [] and bump Standards-Version to 4.5.0 [].
Lastly, disorderfs version 0.5.10-2 was uploaded to Debian unstable by Holger Levsen, which enabled security hardening via DEB_BUILD_MAINT_OPTIONS [] and dropped debian/disorderfs.lintian-overrides [].
Add a citation link to the academic article regarding dettrace [], and added yet another supply-chain security attack publication [].
Reformatted the Jekyll s Liquid templating language and CSS formatting to be consistent [] as well as expand a number of tab characters [].
Used relative_url to fix missing translation icon on various pages. []
Published two announcement blog posts regarding the restarting of our IRC meetings. [][]
Added an explicit note regarding the lack of an in-person summit in 2020 to our events page. []
Testing framework
The Reproducible Builds project operates a Jenkins-based testing framework that powers tests.reproducible-builds.org. This month, Holger Levsen made the following changes:
Make a number of updates to reflect that our sponsor Profitbricks has renamed itself to IONOS. [][][][]
Run a F-Droid maintenance routine twice a month to utilise its cleanup features. []
Fix the target name in OpenWrt builds to ath79 from ath97. []
Add a missing Postfix configuration for a node. []
Temporarily disable Arch Linux builds until a core node is back. []
Make a number of changes to our thanks page. [][][]
Build node maintenance was performed by both Holger Levsen [][] and Vagrant Cascadian [][][], Vagrant Cascadian also updated the page listing the variations made when testing to reflect changes for in build paths [] and Hans-Christoph Steiner made a number of changes for F-Droid, the free software app repository for Android devices, including:
Do not fail reproducibility jobs when their cleanup tasks fail. []
Skip libvirt-related sudo command if we are not actually running libvirt. []
Use direct URLs in order to eliminate a useless HTTP redirect. []
If you are interested in contributing to the Reproducible Builds project, please visit the Contribute page on our website. However, you can also get in touch with us via:
Welcome to the September 2020 report from the Reproducible Builds project. In our monthly reports, we attempt to summarise the things that we have been up to over the past month, but if you are interested in contributing to the project, please visit our main website.
This month, the Reproducible Builds project was pleased to announce a donation from Amateur Radio Digital Communications (ARDC) in support of its goals. ARDC s contribution will propel the Reproducible Builds project s efforts in ensuring the future health, security and sustainability of our increasingly digital society. Amateur Radio Digital Communications (ARDC) is a non-profit which was formed to further research and experimentation with digital communications using radio, with a goal of advancing the state of the art of amateur radio and to educate radio operators in these techniques. You can view the full announcement as well as more information about ARDC on their website.
In August s report, we announced that Jennifer Helsby (redshiftzero) launched a new reproduciblewheels.com website to address the lack of reproducibility of Python wheels . This month, Kushal Das posted a brief follow-up to provide an update on reproducible sources as well.
The Threema privacy and security-oriented messaging application announced that within the next months , their apps will become fully open source, supporting reproducible builds :
This is to say that anyone will be able to independently review Threema s security and verify that the published source code corresponds to the downloaded app.
The previous year has seen great progress in Arch Linux to get reproducible builds in the hands of the users and developers. In this talk we will explore the current tooling that allows users to reproduce packages, the rebuilder software that has been written to check packages and the current issues in this space.
During the Reproducible Builds summit in Marrakesh, GNU Guix, NixOS and Debian were able to produce a bit-for-bit identical binary when building GNU Mes, despite using three different major versions of GCC. Since the summit, additional work resulted in a bit-for-bit identical Mes binary using tcc and this month, a fuller update was posted by the individuals involved.
diffoscopediffoscope is our in-depth and content-aware diff utility that can not only locate and diagnose reproducibility issues, it provides human-readable diffs of all kinds too.
In September, Chris Lamb made the following changes to diffoscope, including preparing and uploading versions 159 and 160 to Debian:
New features:
Show ordering differences only in strings(1) output by applying the ordering check to all differences across the codebase. []
Bug fixes:
Mark some PGP tests that they require pgpdump, and check that the associated binary is actually installed before attempting to run it. (#969753)
Don t raise exceptions when cleaning up after guestfs cleanup failure. []
Ensure we check FALLBACK_FILE_EXTENSION_SUFFIX, otherwise we run pgpdump against all files that are recognised by file(1) as data. []
Codebase improvements:
Add some documentation for the EXTERNAL_TOOLS dictionary. []
Abstract out a variable we use a couple of times. []
Improve the documentation on how to signup to Salsa. []
Add some more links to academic papers. []
Also include the general news in our RSS feed [] and drop including weekly reports from the RSS feed (they are never shown now that we have over 10 items) [].
Update ordering and location of various news and links to tarballs, etc. [][][]
In addition, Holger Levsen re-added the documentation link to the top-level navigation [] and documented that the jekyll-polyglot package is required []. Lastly, diffoscope.org and reproducible-builds.org were transferred to Software Freedom Conservancy. Many thanks to Brett Smith from Conservancy, J r my Bobbio (lunar) and Holger Levsen for their help with transferring and to Mattia Rizzolo for initiating this.
Upstream patches
The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of these patches, including:
Testing framework
The Reproducible Builds project operates a Jenkins-based testing framework to power tests.reproducible-builds.org. This month, Holger Levsen made the following changes:
Highlight important bad conditions in colour. [][]
Add support for detecting more problems, including Jenkins shutdown issues [], failure to upgrade Arch Linux packages [], kernels with wrong permissions [], etc.
Misc:
Delete old schroot sessions after 2 days, not 3. []
In addition, stefan0xC fixed a query for unknown results in the handling of Arch Linux packages [] and Mattia Rizzolo updated the template that notifies maintainers by email of their newly-unreproducible packages to ensure that it did not get caught in junk/spam folders []. Finally, build node maintenance was performed by Holger Levsen [][][][], Mattia Rizzolo [][] and Vagrant Cascadian [][][].
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:
Because of the lock-down in France and thanks to Lucas, I have been able to make some progress rebuilding Debian with clang instead of gcc.
TLDR
Instead of patching clang itself, I used a different approach this time: patching Debian tools or implementing some workaround to mitigate an issue.
The percentage of packages failing drop from 4.5% to 3.6% (1400 packages to 1110 - on a total of 31014).
I focused on two classes of issues:
Symbol differences
Historically, symbol management for C++ in Debian has been a pain. Russ Allbery wrote a blog post in 2012 explaining the situation. AFAIK, it hasn't changed much.
Once more, I took the dirty approach: if there new or missing symbols, don't fail the build.
The rational is the following: Packages in the Debian archive are supposed to build without any issue. If there is new or missing symbols, it is probably clang generating a different library but this library is very likely working as expected (and usable by a program compiled with g++ or clang). It is purely a different approach taken by the compiler developer.
In order to mitigate this issue, before the build starts, I am modifying dpkg-gensymbols to transform the error into a warning.
So, the typical Debian error some new symbols appeared in the symbols file or some symbols or patterns disappeared in the symbols file will NOT fail the build.
Unsurprisingly, all but one package (libktorrent) build.
Even if I am pessimistic, I reported a bug on dpkg-dev to evaluate if we could improve dpkg-gensymbol not to fail on these cases.
For maintainers & upstream
Maintainer of Debian/Ubuntu packages? I am providing a list of failing packages per maintainer: https://clang.debian.net/maintainers.php
For upstream, it is also easy to test with clang. Usually, apt install clang && CC=clang CXX=clang++ <build step> is good enough.
Conclusion
With these two changes, I have been able to fix about 290 packages. I think I will be able to get that down a bit more but we will soon reach a plateau as many warnings/issues will have to fix in the C/C++ code itself.
After quite some time without publishing anything here, I decided to share the
latest events. It is a hard time for most of us but with all this time at home,
one can also do great things.
I would like to start with the wonderful idea the Debian Brasil community had!
Why not create an online Debian related conference to keep people s minds busy
and also share knowledge? After brainstorming, we came up with our online
conference called #FiqueEmCasaUseDebian
(in English it would be #StayHomeUseDebian). It started on May 3rd and will
last until May 30th (yes, one month)! Every weekday, we have one online talk at
night and on every Saturday, a Debian packaging workshop. The feedback so far
has been awesome and the Brazilian Debian community is reaching out to more
people than usual at our regular conferences (as you might have imagined,
Brazil is huge and it is hard to bring people at the same place). At the end of
the month, we will have the first MiniDebConf
online
and I hope it will be as successful as our experience here in Brazil.
Another thing that deserves a highlight is the fact that I became an Ubuntu
Core Developer this month; yay! After 9 months of working almost daily on the
Ubuntu Server, I was able to get my upload rights to the Ubuntu archive. I was
tired of asking for sponsorship, and probably my peers were tired of me too.
I could spend more time here complaining about the Brazilian government but I
think it isn t worth it. Let s try to do something useful instead!
As part of the LLVM release cycle, I am continuing rebuilding the Debian archive with clang instead of gcc to evaluate potential regressions.
Processed results are available on the website: https://clang.debian.net/status.php - Now includes some fancy graphs to show the evolution
Raw logs are published on github: https://github.com/opencollab/clang.debian.net/tree/master/logs
Since my last blog post on the subject (August 2017), Clang is more and more present in the tech ecosystem. It is now the compiler used to build Firefox and Chrome upstream binaries on all the supported architectures/operating systems. More architectures are supported, it has a new linker (lld), a new hybrid IR (MLIR), a lot of checkers in clang-tidy, cross-language linking with Rust, etc.
Results
Now, about Debian results, we rebuilt using 8.0.1, 9.0.1 and 10.0rc2. Results are pretty similar to what we had with previous versions: between 4 to 5% of packages are failing when gcc is replaced by clang.
Even if most of the software are still using gcc as compiler, we can see that clang has a positive effect on code quality. With many different kinds of errors and warnings found clang over the years, we noticed a steady decline of the number of errors. For example, the number of incorrect C/C++ main declarations has been decreasing years after years:
Errors found
The biggest offender is still the qmake changes which doesn't allow the used workaround (replacing /usr/bin/gcc by /usr/bin/clang) - about 250 errors. Most of these packages would probably compile fine with clang. More on the Qt bug tracker. The workaround proposed in the bug isn't applicable for us as we use the dropped-in replacement of the
The second error is still some differences in symbol generation. Unlike gcc, it seems that clang doesn't generate some symbols (or adds some). As a bunch of Debian packages are checking the list of symbols in the library (for ABI management), the build fails on purpose. For example, with libcec, the symbol _ZN10P8PLATFORM14CConditionImplD1Ev@Base 3.1.0 isn't generated anymore. I am not expecting this to be a big deal: the generated libraries probably works most of the time. More on C++ symbol management in Debian.
Current status
As previously said in a blog post, I don't think there is a strong intensive to go away from gcc for most of the Linux distributions. The big reason for BSD was the license (even if the move to the Apache 2 license wasn't received positively by some of them).
While the LLVM/clang ecosystem clearly won the tooling battle, as a C/C++ compiler, gcc is still an excellent compiler which supports more architecture and more languages.
In term of new warnings and checks, as the clang community moved the efforts in clang-tidy (which requires more complex tooling), out of the box, gcc provides a better experience (as example, see the Firefox meta bug to build with -Werror with the default warnings using gcc 9, gcc 10 and clang trunk for example).
Next steps
I see some potential next steps to decrease the number of failure:
Workaround the Qt/Qmake issue
Fix the objective-c header include issues (echo "#include <objc/objc.h>" > foo.m && clang -c foo.m is currently failing)
Identify why clang generates more/less symbols that gcc in the library and try to fix that
Rebuild the archive with clang-7 - Seems that I have some data problem
As part of the LLVM release cycle, I am continuing rebuilding the Debian archive with clang instead of gcc to evaluate potential regressions.
Processed results are available on the website: https://clang.debian.net/status.php - Now includes some fancy graphs to show the evolution
Raw logs are published on github: https://github.com/opencollab/clang.debian.net/tree/master/logs
Since my last blog post on the subject (August 2017), Clang is more and more present in the tech ecosystem. It is now the compiler used to build Firefox and Chrome upstream binaries on all the supported architectures/operating systems. More architectures are supported, it has a new linker (lld), a new hybrid IR (MLIR), a lot of checkers in clang-tidy, cross-language linking with Rust, etc.
Results
Now, about Debian results, we rebuilt using 8.0.1, 9.0.1 and 10.0rc2. Results are pretty similar to what we had with previous versions: between 4 to 5% of packages are failing when gcc is replaced by clang.
Even if most of the software are still using gcc as compiler, we can see that clang has a positive effect on code quality. With many different kinds of errors and warnings found clang over the years, we noticed a steady decline of the number of errors. For example, the number of incorrect C/C++ main declarations has been decreasing years after years:
Errors found
The biggest offender is still the qmake changes which doesn't allow the used workaround (replacing /usr/bin/gcc by /usr/bin/clang) - about 250 errors. Most of these packages would probably compile fine with clang. More on the Qt bug tracker. The workaround proposed in the bug isn't applicable for us as we use the dropped-in replacement of the compiler.
The second error is still some differences in symbol generation. Unlike gcc, it seems that clang doesn't generate some symbols (or adds some). As a bunch of Debian packages are checking the list of symbols in the library (for ABI management), the build fails on purpose. For example, with libcec, the symbol _ZN10P8PLATFORM14CConditionImplD1Ev@Base 3.1.0 isn't generated anymore. I am not expecting this to be a big deal: the generated libraries probably works most of the time. More on C++ symbol management in Debian.
I reported this bug upstream a while back: https://bugs.llvm.org/show_bug.cgi?id=30441
Current status
As previously said in a blog post, I don't think there is a strong intensive to go away from gcc for most of the Linux distributions. The big reason for BSD was the license (even if the move to the Apache 2 license wasn't received positively by some of them).
While the LLVM/clang ecosystem clearly won the tooling battle, as a C/C++ compiler, gcc is still an excellent compiler which supports more architecture and more languages.
In term of new warnings and checks, as the clang community moved the efforts in clang-tidy (which requires more complex tooling), out of the box, gcc provides a better experience (as example, see the Firefox meta bug to build with -Werror with the default warnings using gcc 9, gcc 10 and clang trunk for example).
Next steps
I see some potential next steps to decrease the number of failure:
Workaround the Qt/Qmake issue
Fix the objective-c header include issues (echo "#include <objc/objc.h>" > foo.m && clang -c foo.m is currently failing)
Identify why clang generates more/less symbols that gcc in the library and try to fix that
Rebuild the archive with clang-7 - Seems that I have some data problem
Like each month, here comes a report about the work of paid contributors to Debian LTS.
Individual reports
In October, about 197 work hours have been dispatched among 13 paid contributors. Their reports are available:
Antoine Beaupr did 21h (out of 16h allocated + 8.75h remaining, thus keeping 3.75h for November).
Ben Hutchings did 20 hours (out of 15h allocated + 9 extra hours, thus keeping 4 extra hours for November).
Lucas Kanashiro did 2 hours (out of 5h allocated, thus keeping 3 hours for November).
Markus Koschany did 19 hours (out of 20.75h allocated, thus keeping 1.75 extra hours for November).
Ola Lundqvist did 7.5h (out of 7h allocated + 0.5 extra hours).
Rapha l Hertzog did 13.5 hours (out of 12h allocated + 1.5 extra hours).
Roberto C. Sanchez did 11 hours (out of 20.75 hours allocated + 14.75 hours remaining, thus keeping 24.50 extra hours for November, he will give back remaining hours at the end of the month).
Evolution of the situation
The number of sponsored hours increased slightly to 183 hours per month. With the increasing number of security issues to deal with, and with the number of open issues not really going down, I decided to bump the funding target to what amounts to 1.5 full-time position.
The security tracker currently lists 50 packages with a known CVE and the dla-needed.txt file 36 (we re a bit behind in CVE triaging apparently).
Thanks to our sponsors
New sponsors are in bold.
Here's what happened in the Reproducible Builds effort between Sunday October 29 and Saturday November 4 2017:
Past events
From October 31st November 2nd we held the 3rd Reproducible Builds summit in Berlin, Germany. A full, in-depth report will be posted in the next week or so.
Upcoming events
On November 8th Jonathan Bustillos Osornio (jathan) will present at CubaConf Havana.
On November 17th Chris Lamb will present at Open Compliance Summit, Yokohama, Japan on how reproducible builds ensures the long-term sustainability of technology infrastructure.
Reviews of unreproducible packages
7 package reviews have been added, 43 have been updated and 47 have been removed in this week,
adding to our knowledge about identified issues.
Weekly QA work
During our reproducibility testing, FTBFS bugs have been detected and reported by:
diffoscope development
Version 88 was uploaded to unstable by Mattia Rizzolo.
It included contributions
(already covered by posts of the previous weeks) from:
Mattia Rizzolo
tests/comparators/dtb: compatibility with version 1.4.5. (Closes: #880279)
Chris Lamb
comparators:
binwalk: improve names in output of "internal" members. #877525
Omit misleading "any of" prefix when only complaining about one module in ImportError messages.
Don't crash on malformed "md5sums" files. (Closes: #877473)
tests/comparators:
ps: ps2ascii > 9.21 now varies on timezone, so skip this test for now.
dtby: only parse the version number, not any "-dirty" suffix.
debian/watch: Use HTTPS URI.
Ximin Luo
comparators:
utils/file: Diff container metadata centrally. This fixes a last remaining bug in fuzzy-matching across containers. (Closes: #797759)
Fix all the affected comparators after the above change.
Holger Levsen
Bump Standards-Version to 4.1.1, no changes needed.
strip-nondeterminism development
Version 0.040-1 was uploaded to unstable by Mattia Rizzolo.
It included contributions
already covered by posts of the previous weeks, as well as new ones from:
Version 0.5.2-2 was uploaded to unstable by Holger Levsen.
It included contributions
already covered by posts of the previous weeks, as well as new ones from:
archlinux: try to re-enable one schroot creation job
lynxis
lede: replace TMPDIR -> RESULTSDIR
lede: openwrt_get_banner(): use locals instead of globals
lede: add newline to $CONFIG
lede: show git log -1 in jenkins log
Holger Levsen:
lede: add very simple landing page
Juliana Oliveira Rodrigues
archlinux: adds pacman-git dependencies
kpcyrd
archlinux: disable signature verification when running in the future
archlinux: use pacman-git until the next release
archlinux: make pacman fail less early
archlinux: use sudo to prepare chroot
archlinux: remove -rf for regular file
archlinux: avoid possible TOCTOU issue
archlinux: Try to fix tar extraction
archlinux: fix sha1sums parsing
Misc.
This week's edition was written by Bernhard M. Wiedemann, Chris Lamb, Mattia Rizzolo & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.
In this post I describe the work that I ve done until the end of October
in the context of the Debian LTS team. This month I was allocated 5h and spent
just 2h of them because I have written my master s qualification text (I am
almost on my deadline to finish it). During November I intend to finish these
3h pending, so I did not request more hours.
I basically worked with CVE-2017-0903 which is an issue related to YAML
deserialization of gem specifications that could allow one execute remote code.
Two packages in wheezy could be affected by this security vulnerability,
rubygems and ruby1.9.1. The issue affects just RubyGems source code,
but before Ruby version 1.9.1 it was maintained in a separated package, after
that it was incorporated by ruby interpreter source package.
After carefully read the
upstream blogpost
and reviewed the
commit that intruduced this vulnerability,
I was able to figure out whether the mentioned packages were affected or not.
The modification was not present in both of them, and after some tests I did
confirm that those versions of rubygems were not affected. The two packages
were marked as not affected by CVE-2017-0903 in wheezy.
Well, this was the summary of my activities in the Debian LTS team in
October. See you next month :)
Em 2023 o tradicional 
With the end of the year approaching fast, I thought putting my year in
retrospective via music would be a fun thing to do.
Albums
In 2022, I added 51 new albums to my collection nearly one a week! I listed
them below in the order in which I acquired them.
I purchased most of these albums when I could and borrowed the rest at
libraries. If you want to browse though, I added links to the album covers
pointing either to websites where you can buy them or to Discogs when digital
copies weren't available
"Again, we see there is nothing you can possess which I cannot take away."
The imagery of the ark appears to be borrowed from James Tissot's The Ark Passes Over the Jordan, part of the fin de siecle fascination with the occult and (ironically enough given the background of Raiders' director), a French Catholic revival.
And oh, did I mention that we re completely remote and distributed?
This month, the Reproducible Builds project restarted our IRC meetings, managing to convene twice: the first time on October 12th (
Like