• installDeps.r installs all dependencies of package (directory or tarball)
• installRSPM.r relies on RSPM to install binary packages
• installBSPM.r relies on BSPM to install binary packages (esp. on Linux)
• cranIncomimg.r checks the incoming queue for one or more packages
• urlUpdates.r checks and/or updates stale URLs leading to redirects

Changes in littler version 0.3.12 (2020-10-04)

• Changes in examples
  • Updates to scripts tt.r, cos.r, cow.r, c4r.r, com.r
  • New script installDeps.r to install dependencies
  • Several updates tp script check.r
  • New script installBSPM.r and installRSPM.r for binary package installation (Dirk and I aki in #81)
  • New script cranIncoming.r to check in Incoming
  • New script urlUpdate.r validates URLs as R does
• Changes in package
  • Travis CI now uses BSPM
  • A package documentation website was added
  • Vignettes now use minidown resulting in much reduced filesizes: from over 800kb to under 50kb (Dirk in #83)

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

This is to say that anyone will be able to independently review Threema s security and verify that the published source code corresponds to the downloaded app.

Events Sadly, due to the unprecedented events in 2020, there will be no in-person Reproducible Builds event this year. However, the Reproducible Builds project intends to resume meeting regularly on IRC, starting on Monday, October 12th at 18:00 UTC (full announcement). The cadence of these meetings will probably be every two weeks, although this will be discussed and decided on at the first meeting. (An editable agenda is available.) On 18th September, Bernhard M. Wiedemann gave a presentation in German titled Wie reproducible builds Software sicherer machen ( How reproducible builds make software more secure ) at the Internet Security Digital Days 2020 conference. (View video.) On Saturday 10th October, Morten Linderud will give a talk at Arch Conf Online 2020 on The State of Reproducible Builds in the Arch Linux distribution:

The previous year has seen great progress in Arch Linux to get reproducible builds in the hands of the users and developers. In this talk we will explore the current tooling that allows users to reproduce packages, the rebuilder software that has been written to check packages and the current issues in this space.

During the Reproducible Builds summit in Marrakesh, GNU Guix, NixOS and Debian were able to produce a bit-for-bit identical binary when building GNU Mes, despite using three different major versions of GCC. Since the summit, additional work resulted in a bit-for-bit identical Mes binary using tcc and this month, a fuller update was posted by the individuals involved.

Development work In openSUSE, Bernhard M. Wiedemann published his monthly Reproducible Builds status update.

Debian Chris Lamb uploaded a number of Debian packages to address reproducibility issues that he had previously provided patches for, including cfingerd (#831021), grap (#870573), splint (#924003) & schroot (#902804) Last month, an issue was identified where a large number of Debian .buildinfo build certificates had been tainted on the official Debian build servers, as these environments had files underneath the /usr/local/sbin directory to prevent the execution of system services during package builds. However, this month, Aurelien Jarno and Wouter Verhelst fixed this issue in varying ways, resulting in a special policy-rcd-declarative-deny-all package. Building on Chris Lamb s previous work on reproducible builds for Debian .ISO images, Roland Clobus announced his work in progress on making the Debian Live images reproducible. [ ] Lucas Nussbaum performed an archive-wide rebuild of packages to test enabling the reproducible=+fixfilepath Debian build flag by default. Enabling the fixfilepath feature will likely fix reproducibility issues in an estimated 500-700 packages. The test revealed only 33 packages (out of 30,000 in the archive) that fail to build with fixfilepath. Many of those will be fixed when the default LLVM/Clang version is upgraded. 79 reviews of Debian packages were added, 23 were updated and 17 were removed this month adding to our knowledge about identified issues. Chris Lamb added and categorised a number of new issue types, including packages that captures their build path via quicktest.h and absolute build directories in documentation generated by Doxygen , etc. Lastly, Lukas Puehringer s uploaded a new version of the in-toto to Debian which was sponsored by Holger Levsen. [ ]

diffoscope diffoscope is our in-depth and content-aware diff utility that can not only locate and diagnose reproducibility issues, it provides human-readable diffs of all kinds too. In September, Chris Lamb made the following changes to diffoscope, including preparing and uploading versions 159 and 160 to Debian:

• New features:
  • Show ordering differences only in strings(1) output by applying the ordering check to all differences across the codebase. [ ]
• Bug fixes:
  • Mark some PGP tests that they require pgpdump, and check that the associated binary is actually installed before attempting to run it. (#969753)
  • Don t raise exceptions when cleaning up after guestfs cleanup failure. [ ]
  • Ensure we check FALLBACK_FILE_EXTENSION_SUFFIX, otherwise we run pgpdump against all files that are recognised by file(1) as data. [ ]
• Codebase improvements:
  • Add some documentation for the EXTERNAL_TOOLS dictionary. [ ]
  • Abstract out a variable we use a couple of times. [ ]
• diffoscope.org website improvements:
  • Make the (long) demonstration GIF less prominent on the page. [ ]

In addition, Paul Spooren added support for automatically deploying Docker images. [ ]

Website and documentation This month, a number of updates to the main Reproducible Builds website and related documentation. Chris Lamb made the following changes:

• Update a few titles and the ordering of some top-level navigation elements. [ ]
• Drafted, published and publicised August s monthly report.
• Improve the documentation on how to signup to Salsa. [ ]
• Add some more links to academic papers. [ ]
• Also include the general news in our RSS feed [ ] and drop including weekly reports from the RSS feed (they are never shown now that we have over 10 items) [ ].
• Update ordering and location of various news and links to tarballs, etc. [ ][ ][ ]
• Kept isdebianreproducibleyet.com up to date. [ ]
• Worked with Amateur Radio Digital Communications in order to announce their generous sponsorship of the Reproducible Builds project.

In addition, Holger Levsen re-added the documentation link to the top-level navigation [ ] and documented that the jekyll-polyglot package is required [ ]. Lastly, diffoscope.org and reproducible-builds.org were transferred to Software Freedom Conservancy. Many thanks to Brett Smith from Conservancy, J r my Bobbio (lunar) and Holger Levsen for their help with transferring and to Mattia Rizzolo for initiating this.

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of these patches, including:

• Bernhard M. Wiedemann:
  • cfn-python-lint (build failure)
  • clutter (avoid a random ID in HTML from xsltproc)
  • kubernetes (1-bit order in manual page)
  • libint (merged, filesystem order)
  • libmysofa (disable -fprofile-arcs and code coverage)
  • libnet (merged, date)
  • libqb (date / copyright)
  • libsemigroups (CPU detection)
  • nauty (CPU type detection)
• Chris Lamb:
  • #970024 filed against cif2cell.
  • #970278 filed against smartlist.
  • #970383 filed against evince.
  • #970498 filed against libiio.
  • #970851 filed against check-pgbackrest.
  • #970908 filed against smartdns.
  • #971420 filed against jhbuild.
• kpcyrd:
  • git2-rs (sort return ordering of readdir(3))
• Vagrant Cascadian:
  • #970874 filed against nix.
  • #970888 & #970890 filed against fai.
  • #970893 filed against debdelta.
  • #971400 & #971402 filed against vboot-utils.
  • #971408 filed against simde.

Bernhard M. Wiedemann also reported issues in git2-rs, pyftpdlib, python-nbclient, python-pyzmq & python-sidpy.

Testing framework The Reproducible Builds project operates a Jenkins-based testing framework to power tests.reproducible-builds.org. This month, Holger Levsen made the following changes:

• Debian:
  • Shorten the subject of nodes have gone offline notification emails. [ ]
  • Also track bugs that have been usertagged with usrmerge. [ ]
  • Drop abort-related codepaths as that functionality has been removed from Jenkins. [ ]
  • Update the frequency we update base images and status pages. [ ][ ][ ][ ]
• Status summary view page:
  • Add support for monitoring systemctl status [ ] and the number of diffoscope processes [ ].
  • Show the total number of nodes [ ] and colourise critical disk space situations [ ].
  • Improve the visuals with respect to vertical space. [ ][ ]
• Debian rebuilder prototype:
  • Resume building random packages again [ ] and update the frequency that packages are rebuilt. [ ][ ]
  • Use --no-respect-build-path parameter until sbuild 0.81 is available. [ ]
  • Treat the inability to locate some packages as a debrebuild problem, and not as a issue with the rebuilder itself. [ ]
• Arch Linux:
  • Update various components to be compatible with Arch Linux s move to the xz compression format. [ ][ ][ ]
  • Allow scheduling of old packages to catch up on the backlog. [ ][ ][ ]
  • Improve formatting on the summary page. [ ][ ]
  • Update HTML pages once every hour, not every 30 minutes. [ ]
  • Use the Ubuntu (!) GPG keyserver to validate packages. [ ]
• System health checks:
  • Highlight important bad conditions in colour. [ ][ ]
  • Add support for detecting more problems, including Jenkins shutdown issues [ ], failure to upgrade Arch Linux packages [ ], kernels with wrong permissions [ ], etc.
• Misc:
  • Delete old schroot sessions after 2 days, not 3. [ ]
  • Use sudo to cleanup diffoscope schroot sessions. [ ]

In addition, stefan0xC fixed a query for unknown results in the handling of Arch Linux packages [ ] and Mattia Rizzolo updated the template that notifies maintainers by email of their newly-unreproducible packages to ensure that it did not get caught in junk/spam folders [ ]. Finally, build node maintenance was performed by Holger Levsen [ ][ ][ ][ ], Mattia Rizzolo [ ][ ] and Vagrant Cascadian [ ][ ][ ].
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Twitter: @ReproBuilds
• Mastodon: @reproducible_builds@fosstodon.org
• Mailing list: rb-general@lists.reproducible-builds.org

Curiosity I guess this would be common to most of us. While I grew up, right from the childhood itself, the sky was always an intriguing view. The Stars, the Moon, the Eclipses; were all fascinating. As a child, in my region, religion and culture; the mythology also built up stories around it. Lunar Eclipses have a story of its own. During Solar Eclipses, parents still insist that we do not go out. And to be done with the food eating before/after the eclipse. Then there s the Hindu Astrology part, which claims its own theories and drags in mythology along. For example, you ll still find the Hindu Astrology making recommendations to follow certain practices with the planets, to get auspicious personal results. As far as I know, other religions too have similar beliefs about the planets. As a child, we are told the Moon to be addressed as an Uncle ( ). There s also a rhyme around it, that many of us must have heard. And if you look at our god, Lord Mahadev, he s got a crescent on his head

Lord Mahadev

Reality Fast-forward to today, as I grew, so did some of my understanding. It is fascinating how mankind has achieved so much understanding of our surrounding. You could go through the documentaries on Mars Exploration, for example; to see how the rovers are providing invaluable data. As a mere individual, there s a limit to what one can achieve. But the questions flow in free.

• Is there life beyond us
• What s out there in the sky
• Why is all this the way it is

Hobby The very first step, for me, for every such curiosity, has been to do the ground work, with the resources I have. To study on the subject. I have done this all my life. For example, I started into the Software domain as: A curiosity => A Hobby => A profession Same was the case with some of the other hobbies, equally difficult as Astronomy, that I developed a liking for. Just did the ground work, studied on those topics and then applied the knowledge to further improve it and build up some experience. And star gazing came in no different. As a complete noob, had to start with the A B C on the subject of Astronomy. Familiarize myself with the usual terms. As so on PS: Do keep in mind that not all hobbies have a successful end. For example, I always craved to be good with graphic designing, image processing and the likes, where I ve always failed. Never was able to keep myself motivated enough. Similar was my experience when trying to learn playing a musical instrument. Just didn t work out for me, then. There s also a phase in it, where you fail and then learn from the failures and proceed further, and then eventually succeed. But we all like to talk about the successes. :-)

Astronomy So far, my impression has been that this topic/domain will not suit most of the people. While the initial attraction may be strong, given the complexity and perseverance that Astronomy requires, most people would lose interest in it very soon. Then there s the realization factor. If one goes with an expectation to get quick results, they may get disappointed. It isn t like a point and shoot device that d give you results on the spot. There s also the expectation side of things. If you are a person more accustomed to taking pretty selfies, which always come right because the phone manufacturer does heavy processing on the images to ensure that you get to see the pretty fake self, for the most of the times; then star gazing with telescopes could be a frustrating experience altogether. What you get to see in the images on the internet will be very different than what you d be able to see with your eyes and your basic telescope. There s also the cost aspect. The more powerful (and expensive) your telescope, the better your view. And all things aside, it still may get you lose interest, after you ve done all the ground work and spent a good chunk of money on it. Simply because the object you are gazing at is more a still image, which can quickly get boring for many. On the other hand, if none of the things obstruct, then the domain of Astronomy can be quite fascinating. It is a continuous learning domain (reminds me of CI in our software field these days). It is just the beginning for us here, and we hope to have a lasting experience in it.

The Internet I have been indebted to the internet right from the beginning. The internet is what helped me be able to achieve all I wanted. It is one field with no boundaries. If there is a will, there is a way; and often times, the internet is the way.

• I learnt computers over the internet.
• Learnt more about gardening and plants over the internet
• Learnt more about fish care-taking over the internet

And many many more things. Some of the communities over the internet are a great way to participation. They bridge the age gap, the regional gap and many more. For my Astronomy need, I was glad to see so many active communities, with great participants, on the internet.

Telescope While there are multiple options to start star gazing, I chose to start with a telescope. But as someone completely new to this domain, there was a long way to go. And to add to that, the real life: work + family I spent a good 12+ months reading up on the different types of telescopes, what they are, their differences, their costs, their practical availability etc. The good thing is that the market has offerings for everything. From a very basic binocular to a fully automatic Maksutov-Cassegrain scope. It all would depend on your budget.

Automatic vs Manual To make it easy for the users, the market has multiple options in the offering. One could opt-in for a cheap, basic and manually operated telescope; which would require the user to do a lot of ground study. On the other hand, users also have the option of automatic telescopes which do the hard work of locating and tracking the planetary objects. Either option aside, the end result of how much you ll be able to observe the sky, still depends on many many more factors: Enthusiasm over time, Light Pollution, Clear Skies, Timing etc. PS: The planetary objects move at a steady pace. Objects you lock into your view now will be gone out of the FOV in just a matter of minutes.

My Telescope After spending so much of the time reading up on types of telescopes, my conclusion was that a scope with high aperture and focal length was the way to go forward. This made me shorten the list to Dobsonians. But the Dobsonians aren t a very cheap telescope, whether manual or automatic. My final decision made me acquire a 6" Dobsonian Telescope. It is a Newtonian Reflecting Telescope with a 1200mm focal length and 150mm diameter. Another thing about this subject is that most of the stuff you do in Astronomy; right from the telescope selection, to installation, to star gazing; most of it is DIY, so your mileage may vary with the end result and experience. For me, installation wasn t very difficult. I was able to assemble the base Dobsonian mount and the scope in around 2 hours. But the installation manual, I had been provided with, was very brief. I ended up with one module in the mount wrongly fit, which I was able to fix later, with the help of online forums.

Dobsonian Mount

In this image you can see that the side facing out, where the handle will go, is wrong. If fit this way, the handle will not withstand any weight at all.

Correct Panel Side

The right fix of the handle base board. In this image, the handle is on the other side that I m holding. Because the initial fit put in some damage to the engineered wood, I fixed it up by sealing with some adhesive. With that, this is what my final telescope looks like.

Final Telescope

Clear Skies While the telescope was ready, the skies were not. For almost next 10 days, we had no clear skies at all. All I could do was wait. Wait so much that I had forgotten to check on the skies. Luckily, my wife noticed clear skies this week for a single day. Clear enough that we could try out our telescope for the very first time.

Me posing for a shot

Telescope As I said earlier, in my opinion, it takes a lot of patience and perseverance on this subject. And most of the things here are DIY. To start with, we targeted the Moon. Because it is easy. I pointed the scope to the moon, then looked into the finder scope to center it, and then looked through the eyepiece. And blank. Nothing out there. Turns out, the finder scope and the viewer s angle weren t aligned. This is common and the first DIY step, when you plan to use your telescope for viewing. Since our first attempt was unplanned and just random because we luckily spotted that the skies were clear, we weren t prepared for this. Lucky enough, mapping the difference in the alignment, in the head, is not very difficult. After a couple of minutes, I could make out the point in the finder scope, where the object if projected, would show proper in the viewer. With that done, it was just mesmerizing to see the Moon, in a bit more detail, than what I ve seen all these years of my life.

Moon

Moon

Moon

Moon

The images are not exactly what we saw with our eyes. The view was much more vivid than these pictures. But as a first timer, I really wanted to capture this first moment of a closer view of the Moon. In the whole process; that of ground work studying about telescopes, installation of the telescope, astronomy basics and many other things; the most difficult part in this entire journey, was to point my phone to the viewing eyepiece, to get a shot of the object. This requirement just introduced me to astrophotography. And then, Dobsonians aren t the best model for astrophotography, to what I ve learnt so far. Hopefully, I ll find my ways to do some DIY astrophotography with the tools I have. Or extend my arsenal over time. But overall, we ve been very pleased with the subject of Astronomy. It is a different feel altogether and we re glad to have forayed into it.

Debian

This was my 21st month of contributing to Debian. I became a DM in late March last year and a DD last Christmas! \o/ I ve been busy with my undergraduation stuff but I still squeezed out some time for the regular Debian work. Here are the following things I did in Debian this month:

Uploads and bug fixes:

• rails (2:6.0.3.3+dfsg-1) - Fixing CVE-2020-15169/bug #970040.
• ruby-openid-connect (1.1.8-1) - New upstream version, v1.1.8.
• ruby-mini-magick (4.10.1-1) - Fixing FTBFS, bug #966936.
• ruby-open-graph-reader (0.7.0+dfsg-1) - New upstream version, v0.7.0.
• ruby-string-direction (1.2.2-1) - New upstream version, v1.2.2.
• ruby-jquery-rails (4.3.5-1) - New upstream version, v4.3.5.
• ruby-leaflet-rails (1.6.0+dfsg-1) - New upstream version, v1.6.0.
• ruby-jquery-rails (4.3.5-2) - Fixing FTBFS, bug #956604.
• ruby-rubocop-packaging (0.5.0-1) - Add support for auto-correct.

Other $things:

• Attended the Debian Ruby team meeting. Logs here.
• Mentoring for newcomers.
• FTP Trainee reviewing.
• Moderation of -project mailing list.
• Sponsored trace-cmd for Sudip, ruby-asset-sync for Nilesh, and mariadb-mysql-kbs for William.

---

RuboCop::Packaging - Helping the Debian Ruby team! \o/ This Google Summer of Code, I worked on writing a linter that could flag offenses for lines of code that are very troublesome for Debian maintainers while trying to package and maintain Ruby libraries and applications! Whilst the GSoC period is over, I ve been working on improving that tool and have extended that linter to now auto-correct these offenses by itself! \o/
You can now just use the -A flag and you re done! Boom! The ultimate game-changer! Here s a quick demo for this feature: A few quick updates on RuboCop::Packaging:

• Has 4 cops, solving 4 different issues.
• 3 of them support auto-correction. Just use the -A flag.
• 5 releases so far, latest being v0.5.0.
• GitHub Repository: https://github.com/utkarsh2102/rubocop-packaging/
• Release notes: https://github.com/utkarsh2102/rubocop-packaging/releases/
• Documentation: https://docs.rubocop.org/rubocop-packaging/
• Style guide: https://packaging.rubystyle.guide/
• Being used by over 55 projects! \o/

I ve also spent a considerable amount of time in raising awareness about this and in more general sense, about downstream maintenance.
As a result, I raised a bunch of PRs which got really good response. I got all of the 20 PRs merged upstream, fixing these issues.

---

Debian (E)LTS

Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success. And Debian Extended LTS (ELTS) is its sister project, extending support to the Jessie release (+2 years after LTS support). This was my twelfth month as a Debian LTS and third month as a Debian ELTS paid contributor.
I was assigned 19.75 hours for LTS and 15.00 hours for ELTS and worked on the following things:
(for LTS, I over-worked for 11 hours last month on the survey so only had 8.75 hours this month!)

LTS CVE Fixes and Announcements:

• Issued DLA 2362-1, fixing CVE-2020-11984, for uwsgi.
  For Debian 9 Stretch, these problems have been fixed in version 2.0.14+20161117-3+deb9u3.
• Issued DLA 2363-1, fixing CVE-2020-17446, for asyncpg.
  For Debian 9 Stretch, these problems have been fixed in version 0.8.4-1+deb9u1.
• Started working on ruby-kaminari, ruby-rack-cors, and ruby-json-jwt.

ELTS CVE Fixes and Announcements:

• Issued ELA 274-1, fixing CVE-2020-11984, for uwsgi.
  For Debian 8 Jessie, these problems have been fixed in version 2.0.7-1+deb8u3.
• Issued ELA 275-1, fixing CVE-2020-14363, for libx11.
  For Debian 8 Jessie, these problems have been fixed in version 2:1.6.2-3+deb8u4.
• Issued ELA 278-1, fixing CVE-2020-8184, for ruby-rack.
  For Debian 8 Jessie, these problems have been fixed in version 1.5.2-3+deb8u4.
• Also worked on updating the version of clamAV from v0.101.5 to v0.102.4.
  This was a bit tricky package to work on since it involved an ABI/API change and was more or less a transition. Super thanks to Emilio for his invaluable help and him taking over the package, finishing, and uploading it in the end.

Other (E)LTS Work:

• Front-desk duty from 31-08 to 06-09 and from 28-09 onward for both LTS and ELTS.
• Triaged apache2, cryptsetup, nasm, node-bl, plinth, qemu, rsync, ruby-doorkeeper, and uwsgi.
• Marked CVE-2020-15094/symfony as not-affected for Stretch.
• Marked CVE-2020- 9490,11993 /apache2 as ignored for Stretch.
• Marked CVE-2020-8244/node-bl as no-dsa for Stretch.
• Marked CVE-2020-24978/nasm as no-dsa for Stretch.
• Marked CVE-2020-25073/plinth as no-dsa for Stretch.
• Marked CVE-2020-15094/symfony as not-affected for Jessie.
• Marked CVE-2020-14382/cryptsetup as not-affected for Jessie.
• Marked CVE-2020-14387/rsync as not-affected for Jessie.
• Auto EOL ed ark, collabtive, linux, nasm, node-bl, and thunderbird for Jessie.
• Use mktemp instead of tempfile in bin/auto-add-end-of-life.sh.
• Attended the fifth LTS meeting. Logs here.
• General discussion on LTS private and public mailing list.

---

Until next time.
:wq for today.

Tips

• if you have some text to present, outline keywords so that you can present your subject without reading every word
• ideally, don't read from your slides - they are there to help people follow, not for people to read
• even better: make your slides pretty with only a few words, or don't make slides at all

Further advice:

• 7 tips by Jeffrey Veen
• 10 tips by Neil Patel
• The Art of Presenting by Matt Westgate (video)
• Presenting You by Emma Jane Hogbin (video)

I'm currently using Pandoc with PDF input (with a trip through LaTeX) for most slides, because PDFs are more reliable and portable than web pages. I've also used Libreoffice, Pinpoint, and S5 (through RST) in the past. I miss Pinpoint, too bad that it died. Some of my presentations are available in my GitLab.com account:

• Presentations while I worked at Koumbit
• Short presentation about PRISM
• Security training
• Ethics in computing, based on this blog post
• Presentation about the Maple Spring, at OHM2013
• First presentation at Tor

See also my list of talks and presentations which I can't seem to keep up to date.

Tools

Beamer (LaTeX)

• LaTeX class
• Do not use directly unless you are a LaTeX expert or masochist, see Pandoc below
• see also powerdot
• Home page

Darkslide

• HTML, Javascript
• presenter notes, table of contents, Markdown, RST, Textile, themes, code samples, auto-reload
• Home page, demo

Impress.js

• Javascript
• Zooms in and out, 3D support
• Source code, demo
• Hekyll uses Jekyll as a backend

Impressive

• simply displays PDFs or images
• page transitions, overview screen, highlighting
• Home page

Libreoffice Impress

• Powerpoint clone
• Makes my life miserable
• PDF export, presenter notes, outline view, etc
• Home page, screenshots

Magicpoint

• ancestor of everyone else (1997!)
• text input format, image support, talk timer, slide guides, HTML/Postscript export, draw on slides, X11 output
• no release since 2008
• Home page

mdp and lookatme (commandline)

• Commandline-only, markdown
• Home page
• lookatme is similar

Pandoc

• Allows converting from basically whatever into slides, including Beamer, DZSlides, reveal.js, slideous, slidy, Powerpoint
• PDF, HTML, Powerpoint export, presentation notes, full screen background images
• nice plain text or markdown input format
• Home page, documentation

PDF Presenter

• PDF presentation tool, shows presentation notes
• basically "Keynote for Linux"
• Home page, pdf-presenter-console in Debian

Pinpoint

• Native GNOME app
• Full screen slides, PDF export, live change, presenter notes, pango markup, video, image backgrounds
• Home page
• Abandoned since at least 2019

Reveal.js

• HTML, Javascript
• PDF export, Markdown, LaTeX support, syntax-highlighting, nested slides, speaker notes
• Source code, demo

S5

• HTML, CSS
• incremental, bookmarks, keyboard controls
• can be transformed from ReStructuredText (RST) with rst2s5 with python-docutils
• Home page, demo

sent

• X11 only
• plain text, black on white, image support, and that's it
• from the suckless.org elitists
• Home page

Sozi

• Entire presentation is one poster, zooming and jumping around
• SVG + Javascript
• Home page, demo

Other options Another option I have seriously considered is just generate a series of images with good resolution, hopefully matching the resolution (or at least aspect ratio) of the output device. Then you flip through a series of images one by one. In that case, any of those image viewers (not an exhaustive list) would work:

• Geeqie
• GNOME's eog
• pho
• feh
• fim
• sxiv

Update: it turns out I already wrote a somewhat similar thing when I did a recent presentation. If you're into rants, you might enjoy the README file accompanying the Kubecon rant presentation. TL;DR: "makes me want to scream" and "yet another unsolved problem space, sigh" (refering to "display images full-screen" specifically).

• It refuses to boot my Debian image with Secure Boot enabled.
• It produces ACPI errors on every key press on the internal keyboard.

ACPI BIOS Error (bug): Could not resolve symbol [\_SB.PCI0.GPP3], AE_NOT_FOUND (20200528/psargs-330)
ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_FOUND) (20200528/psparse-529)

• Updated my tickle-me-email library (which implements Getting Things Done (GTD)-like behaviours in IMAP inboxes) to support adding 'TODO' entries as read or unread based on a runtime configuration parameter. [...]

• As part of my role of being the assistant Secretary of the Open Source Initiative and a board director of Software in the Public Interest, I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet as well as the usual internal discussions, etc. I participated in the OSI's inaugural State of the Source conference and began the 'onboarding' of a new project to SPI.

• Uploaded a number of Debian packages to close reproducibility issues that I had previously provided patches for, including cfingerd (#831021), grap (#870573), splint (#924003) & schroot (#902804)
• Categorised a huge number of packages and issues in the Reproducible Builds "notes" repository and began work on an auto-classifier script.
• I also submitted 8 patches to fix specific reproducibility issues within Debian for check-pgbackrest, cif2cell, evince, haskell-haskell-gi-base, libiio, jhbuild, smartdns & smartlist.

• Updated the main Reproducible Builds website and documentation:
  • Worked with Amateur Radio Digital Communications in order to announce their generous sponsorship of the Reproducible Builds project.
  • Drafted, published and publicised August's monthly report.
  • Update a few titles and the ordering of some top-level navigation elements. [...]
  • Improve the documentation on how to signup to Salsa. [...]
  • Add some more links to academic papers. [...]
  • Also include the general news in our RSS feed [...] and drop including weekly reports from the RSS feed (they are never shown now that we have over 10 items) [...].
  • Update ordering and location of various news and links to tarballs, etc. [...][...][...]
• Kept isdebianreproducibleyet.com up to date. [...]
• Continued collaborative work on a scientific/academic article, hopefully to be published within a few months.

• New features:
  • Show "ordering differences" only in strings(1) output by applying the ordering check to all differences across the codebase. [...]
• Bug fixes:
  • Mark some PGP tests that they require pgpdump, and check that the associated binary is actually installed before attempting to run it. (#969753)
  • Don't raise exceptions when cleaning up after guestfs cleanup failure. [...]
  • Ensure we check FALLBACK_FILE_EXTENSION_SUFFIX, otherwise we run pgpdump against all files that are recognised by file(1) as data. [...]
• Codebase improvements:
  • Add some documentation for the EXTERNAL_TOOLS dictionary. [...]
  • Abstract out a variable we use a couple of times. [...]
• diffoscope.org website improvements:
  • Make the (long) demonstration GIF less prominent. [...]

• Bug fixes:
  • Don't emit odd-mark-in-description for large numbers such as 300,000. (#969528)
  • Update the expected Vcs- Browser,Git location of modules and applications maintained by recently-merged Python module/app teams. (#970743)
  • Relax checks around looking for the dh(1) sequencer by not looking for the preceding target:. (#970920)
  • Don't try and open debian/patches/series if it does not exist. [...]
  • Update all $LINTIAN_VERSION assignments in scripts and not just the ones we specify; we had added and removed some during development. [...]
• Tag updates:
  • Clarify which Vcs-* the vcs-field-not-canonical tag is being emitted for and update its long description to remove misleading messages. (#970201)
  • Clarify logic in the long description for odd-mark-in-description. [...]
  • Correct a typo in the long description for the odd-mark-in-description tag. [...]
• Developer documentation updates:
  • Add prominent and up-to-date information on how to run the testsuite. (#923696)
  • Drop recommendation to update debian/changelog manually. [...]
  • Apply wrap-and-sort -sa to the debian subdirectory. [...]
  • Merge data/README into CONTRIBUTING.md for greater visibility [...] and move CONTRIBUTING.md to use #-style Markdown headers [...].

• Investigated and triaged golang-1.7, inspircd, kleopatra, libdbi-perl, open-build-service, openssl, openssl1.0, python-django., tinymce, yaws & zeromq3, amongst others.
• Frontdesk duties, responding to user/developer questions, reviewing others' packages, attending our monthly IRC meeting, participating in mailing list discussions, etc.
• Issued DLA 2368-1 for grunt to fix an arbitrary code execution vulnerability due to the unsafe loading of YAML documents.
• Issued DLA 2370-1 and ELA-281-1 for the python-pip Python package installer to fix a directory traversal attack where arbitrary local files (eg. /root/.ssh/authorized_keys) could be overridden.
• Issued DLA 2372-1 and ELA-282-1 to prevent a denial of service attack in libproxy, a library to make applications HTTP proxy-aware.
• Issued DLA 2374-1 for the gnome-shell component of the GNOME desktop. In certain configurations, when logging out of an account, the password box from the login dialog could reappear with the password visible in plaintext.
• Issued DLA 2380-1 for ruby-gon, a library to send/convert data to Javascript from Ruby applications, to prevent cross-site scripting (XSS) vulnerabilities.

• python-django (2.2.16-1 & 3.1.1-1) New upstream security releases.
• memcached:
  • 1.6.7+dfsg-1 New upstream release.
  • 1.6.7+dfsg-2 & 1.6.7+dfsg-3 Run tests in single CPU mode. (#968603)
• redis (6.0.7-1 & 6.0.8-1) New upstream releases & miscellaneous packaging updates.
• docbook-to-man (2.0.0-45) Ensure shunit2 autopkgtest dependency is cross-architecture friendly. (#969604)

• bluez-source: Contains bluez-source-tmp directory. (#970130)
• bookworm: Manual page contains debugging/warning/error information from running binary. (#970277)
• jhbuild: Missing runtime dependency on python3-distutils. (#971418)
• wxwidgets3.0: Links in documentation points to within the original build path, not the installed path. (#970431)

• Work on lomiri-app-launch (Debian packaging, upstream work, upload to Debian)
• Fork lomiri-url-dispatcher from url-dispatcher (upstream work)
• Upload lomiri-url-dispatcher to Debian
• Fork out suru-icon-theme and make it its own upstream project
• Package and upload suru-icon-theme to Debian
• First glance at lomiri-ui-toolkit (currently FTBFS, needs to be revisited)
• Update of Mir (1.7.0 -> 1.8.0) in Debian
• Fix net-cpp FTBFS in Debian
• Fix FTBFS in gsettings-qt.
• Fix FTBFS in mir (support of binary-only and arch-indep-only builds)
• Coordinate with Marius Gripsgard and Robert Tari on shift over from Ubuntu Indicator to Ayatana Indicators
• Upload ayatana-indicator-* (and libraries) to Debian (new upstream releases)
• Package and upload to Debian: qmenumodel (still in Debian's NEW queue)
• Package and upload to Debian: ayatana-indicator-sound
• Symbol-Updates (various packages) for non-standard architectures
• Fix FTBFS of qtpim-opensource-src in Debian since Qt5.14 had landed in unstable
• Fix FTBFS on non-standard architectures of qtsystems, qtpim and qtfeedback
• Fix wlcs in Debian (for non-standard architectures), more Symbol-Updates (esp. for the mir DEB package)
• Symbol-Updates (mir, fix upstream tinkering with debian/libmiral3.symbols)
• Fix FTBFS in lomiri-url-dispatcher against Debian unstable, file merge request upstream
• Upstream release of qtmir 0.6.1 (via merge request)
• Improve check_whitespace.py script as used in lomiri-api to ignore debian/ subfolder
• Upstream release of lomiri-api 0.1.1 and upload to Debian unstable.

• UBports: Packaging of Lomiri Operating Environment for Debian (part 02)
• UBports: Packaging of Unity8 Desktop Environment for Debian (part 01)
• DDPO UBports Team: https://qa.debian.org/developer.php?login=team%2Bubports%40tracker.debia...
• DDPO Ayatana Packagers: https://qa.debian.org/developer.php?login=pkg-ayatana-devel%40lists.alio...

$ time bgpq4 -h whois.radb.net AS-HURRICANE   wc -l
909869
1.96s user 0.15s system 2% cpu 1:17.64 total
$ time bgpq4 -h rr.ntt.net AS-HURRICANE   wc -l
927865
1.86s user 0.08s system 12% cpu 14.098 total

$ git clone https://github.com/vincentbernat/irrd-legacy.git -b blade/master
$ cd irrd-legacy
$ docker build . -t irrd-snapshot:latest
[ ]
Successfully built 58c3e83a1d18
Successfully tagged irrd-snapshot:latest
$ docker container run --rm --detach --publish=43:43 irrd-snapshot
4879cfe7413075a0c217089dcac91ed356424c6b88808d8fcb01dc00eafcc8c7
$ time bgpq4 -h localhost AS-HURRICANE   wc -l
904137
1.72s user 0.11s system 96% cpu 1.881 total

1. building IRRd,¹
2. retrieving various IRR databases, and
3. assembling the final container with the result of the two previous stages.

$ whois -BrG 2a0a:e805:400::/40
[ ]
inet6num:       2a0a:e805:400::/40
netname:        FR-BLADE-CUSTOMERS-DE
country:        DE
geoloc:         50.1109 8.6821
admin-c:        BN2763-RIPE
tech-c:         BN2763-RIPE
status:         ASSIGNED
mnt-by:         fr-blade-1-mnt
remarks:        synced with cmdb
created:        2020-05-19T08:04:58Z
last-modified:  2020-05-19T08:04:58Z
source:         RIPE
route6:         2a0a:e805:400::/40
descr:          Blade IPv6 - AMS1
origin:         AS64476
mnt-by:         fr-blade-1-mnt
remarks:        synced with cmdb
created:        2019-10-01T08:19:34Z
last-modified:  2020-05-19T08:05:00Z
source:         RIPE

$ bgpq4 -6 -S RIPE -b AS64476
NN = [
    2a0a:e805::/40,
    2a0a:e805:100::/40,
    2a0a:e805:300::/40,
    2a0a:e805:400::/40,
    2a0a:e805:500::/40
];

Code The module takes a list of RPSL objects to synchronize and returns the body of an email update if a change is needed:

- name: prepare RIPE objects
  irr_sync:
    irr: RIPE
    mntner: fr-blade-1-mnt
    source: whois-ripe.txt
  register: irr

Prerequisites The source file should be a set of objects to sync using the RPSL language. This would be the same content you would send manually by email. All objects should be managed by the same maintainer, which is also provided as a parameter. Signing and sending the result is not the responsibility of this module. You need two additional tasks for this purpose:

- name: sign RIPE objects
  shell:
    cmd: gpg --batch --user noc@example.com --clearsign
    stdin: "  irr.objects  "
  register: signed
  check_mode: false
  changed_when: false
- name: update RIPE objects by email
  mail:
    subject: "NEW: update for RIPE"
    from: noc@example.com
    to: "auto-dbm@ripe.net"
    cc: noc@example.com
    host: smtp.example.com
    port: 25
    charset: us-ascii
    body: "  signed.stdout  "

You also need to authorize the PGP keys used to sign the updates by creating a key-cert object and adding it as a valid authentication method for the corresponding mntner object:

key-cert:  PGPKEY-A791AAAB
certif:    -----BEGIN PGP PUBLIC KEY BLOCK-----
certif:    
certif:    mQGNBF8TLY8BDADEwP3a6/vRhEERBIaPUAFnr23zKCNt5YhWRZyt50mKq1RmQBBY
[ ]
certif:    -----END PGP PUBLIC KEY BLOCK-----
mnt-by:    fr-blade-1-mnt
source:    RIPE
mntner:    fr-blade-1-mnt
[ ]
auth:      PGPKEY-A791AAAB
mnt-by:    fr-blade-1-mnt
source:    RIPE

Module definition Starting from the skeleton described in the previous article, we define the module:

module_args = dict(
    irr=dict(type='str', required=True),
    mntner=dict(type='str', required=True),
    source=dict(type='path', required=True),
)
result = dict(
    changed=False,
)
module = AnsibleModule(
    argument_spec=module_args,
    supports_check_mode=True
)

Getting existing objects To grab existing objects, we use the whois command to retrieve all the objects from the provided maintainer.

# Per-IRR variations:
# - whois server
whois =  
    'ARIN': 'rr.arin.net',
    'RIPE': 'whois.ripe.net',
    'APNIC': 'whois.apnic.net'
 
# - whois options
options =  
    'ARIN': ['-r'],
    'RIPE': ['-BrG'],
    'APNIC': ['-BrG']
 
# - objects excluded from synchronization
excluded = ["domain"]
if irr == "ARIN":
    # ARIN does not return these objects
    excluded.extend([
        "key-cert",
        "mntner",
    ])
# Grab existing objects
args = ["-h", whois[irr],
        "-s", irr,
        *options[irr],
        "-i", "mnt-by",
        module.params['mntner']]
proc = subprocess.run("whois", *args, capture_output=True)
if proc.returncode != 0:
    raise AnsibleError(
        f"unable to query whois:  args ")
output = proc.stdout.decode('ascii')
got = extract(output, excluded)

The first part of the code setup some IRR-specific constants: the server to query, the options to provide to the whois command and the objects to exclude from synchronization. The second part invokes the whois command, requesting all objects whose mnt-by field is the provided maintainer. Here is an example of output:

$ whois -h whois.ripe.net -s RIPE -BrG -i mnt-by fr-blade-1-mnt
[ ]
inet6num:       2a0a:e805:300::/40
netname:        FR-BLADE-CUSTOMERS-FR
country:        FR
geoloc:         48.8566 2.3522
admin-c:        BN2763-RIPE
tech-c:         BN2763-RIPE
status:         ASSIGNED
mnt-by:         fr-blade-1-mnt
remarks:        synced with cmdb
created:        2020-05-19T08:04:59Z
last-modified:  2020-05-19T08:04:59Z
source:         RIPE
[ ]
route6:         2a0a:e805:300::/40
descr:          Blade IPv6 - PA1
origin:         AS64476
mnt-by:         fr-blade-1-mnt
remarks:        synced with cmdb
created:        2019-10-01T08:19:34Z
last-modified:  2020-05-19T08:05:00Z
source:         RIPE
[ ]

The result is passed to the extract() function. It parses and normalizes the results into a dictionary mapping object names to objects. We store the result in the got variable.

def extract(raw, excluded):
    """Extract objects."""
    # First step, remove comments and unwanted lines
    objects = "\n".join([obj
                         for obj in raw.split("\n")
                         if not obj.startswith((
                                 "#",
                                 "%",
                         ))])
    # Second step, split objects
    objects = [RPSLObject(obj.strip())
               for obj in re.split(r"\n\n+", objects)
               if obj.strip()
               and not obj.startswith(
                   tuple(f" x :" for x in excluded))]
    # Last step, put objects in a dict
    objects =  repr(obj): obj
               for obj in objects 
    return objects

RPSLObject() is a class enabling normalization and comparison of objects. Look at the module code for more details.

>>> output="""
... inet6num:       2a0a:e805:300::/40
... [ ]
... """
>>> pprint( k: str(v) for k,v in extract(output, excluded=[]) )
 '<Object:inet6num:2a0a:e805:300::/40>':
   'inet6num:       2a0a:e805:300::/40\n'
   'netname:        FR-BLADE-CUSTOMERS-FR\n'
   'country:        FR\n'
   'geoloc:         48.8566 2.3522\n'
   'admin-c:        BN2763-RIPE\n'
   'tech-c:         BN2763-RIPE\n'
   'status:         ASSIGNED\n'
   'mnt-by:         fr-blade-1-mnt\n'
   'remarks:        synced with cmdb\n'
   'source:         RIPE',
 '<Object:route6:2a0a:e805:300::/40>':
   'route6:         2a0a:e805:300::/40\n'
   'descr:          Blade IPv6 - PA1\n'
   'origin:         AS64476\n'
   'mnt-by:         fr-blade-1-mnt\n'
   'remarks:        synced with cmdb\n'
   'source:         RIPE' 

Comparing with wanted objects Let s build the wanted dictionary using the same structure, thanks to the extract() function we can use verbatim:

with open(module.params['source']) as f:
    source = f.read()
wanted = extract(source, excluded)

The next step is to compare got and wanted to build the diff object:

if got != wanted:
    result['changed'] = True
    if module._diff:
        result['diff'] = [
            dict(before_header=k,
                 after_header=k,
                 before=str(got.get(k, "")),
                 after=str(wanted.get(k, "")))
            for k in set((*wanted.keys(), *got.keys()))
            if k not in wanted or k not in got or wanted[k] != got[k]]

Returning updates The module does not have a side effect. If there is a difference, we return the updates to send by email. We choose to include all wanted objects in the updates (contained in the source variable) and let the IRR ignore unmodified objects. We also append the objects to be deleted by adding a delete: attribute to each them them.

# We send all source objects and deleted objects.
deleted_mark = f" 'delete:':16 deleted by CMDB"
deleted = "\n\n".join([f" got[k].raw \n deleted_mark "
                       for k in got
                       if k not in wanted])
result['objects'] = f" source \n\n deleted "
module.exit_json(**result)

---

The complete code is available on GitHub. The module supports both --diff and --check flags. It does not return anything if no change is detected. It can work with APNIC, RIPE and ARIN. It is not perfect: it may not detect some changes,³ it is not able to modify objects not owned by the provided maintainer⁴ and some attributes cannot be modified, requiring to manually delete and recreate the updated object.⁵ However, this module should automate 95% of your needs.

---

1. Other IRRs exist without being attached to a RIR. The most notable one is RADb.
2. ARIN is phasing out this method in favor of IRR-online. RIPE has an API available, but email updates are still supported and not planned to be deprecated. APNIC plans to expose an API.
3. For ARIN, we cannot query key-cert and mntner objects and therefore we cannot detect changes in them. It is also not possible to detect changes to the auth mechanisms of a mntner object.
4. APNIC do not assign top-level objects to the maintainer associated with the owner.
5. Changing the status of an inetnum object requires deleting and recreating the object.

About Amateur Radio Digital Communications (ARDC) Amateur Radio Digital Communications (ARDC) is a non-profit that was formed to further research and experimentation with digital communications using radio, with a goal of advancing the state of the art of amateur radio and to educate radio operators in these techniques. It does this by managing the allocation of network resources, encouraging research and experimentation with networking protocols and equipment, publishing technical articles and number of other activities to promote the public good of amateur radio and other related fields. ARDC has recently begun to contribute funding to organisations, groups, individuals and projects towards these and related goals, and their grant to the Reproducible Builds project is part of this new initiative. Amateur radio is an entirely volunteer activity performed by knowledgeable hobbyists who have proven their ability by passing the appropriate government examinations. No remuneration is permitted. Ham radio, as it is also known, has proven its value in advancements of the state of the communications arts, as well as in public service during disasters and in times of emergency. For more information about ARDC, please see their website at ampr.org.

About the Reproducible Builds project One of the original promises of open source software was that peer review would result in greater end-user security and stability of our digital ecosystem. However, although it is theoretically possible to inspect and build the original source code in order to avoid maliciously-inserted flaws, almost all software today is distributed in prepackaged form. This disconnect allows third-parties to compromise systems by injecting code into seemingly secure software during the build process, as well as by manipulating copies distributed from app stores and other package repositories. In order to address this, Reproducible builds are a set of software development practices, ideas and tools that create an independently-verifiable path from the original source code, all the way to what is actually running on our machines. Reproducible builds can reveal the injection of backdoors introduced by the hacking of developers own computers, build servers and package repositories, but can also expose where volunteers or companies have been coerced into making changes via blackmail, government order, and so on. A world without reproducible builds is a world where our digital infrastructure cannot be trusted and where online communities are slower to grow, collaborate less and are increasingly fragile. Without reproducible builds, we leave space for greater encroachments on our liberties both by individuals as well as powerful, unaccountable actors such as governments, large corporations and autocratic regimes. The Reproducible Builds project began as a project within the Debian community, but is now working with many crucial and well-known free software projects such as Coreboot, openSUSE, OpenWrt, Tails, GNU Guix, Arch Linux, Tor, and many others. It is now an entirely Linux distribution independent effort and serves as the central clearing house for all issues related to securing build systems and software supply chains of all kinds. For more about the Reproducible Builds project, please see their website at reproducible-builds.org.
If you are interested in ensuring the ongoing security of the software that underpins our civilisation, and wish to sponsor the Reproducible Builds project, please reach out to the project by emailing contact@reproducible-builds.org.

int foo[5];
...
foo[8] = bar;

/* 1-element array */
struct foo  
    ...
    char contents[1];
 ;
/* GNU C extension: 0-element array */
struct foo  
    ...
    char contents[0];
 ;
/* C standard: flexible array */
struct foo  
    ...
    char contents[];
 ;
instance = kmalloc(sizeof(struct foo) + content_size);

int foo[5];
...
for (i = 0; i < some_argument; i++)  
    ...
    foo[i] = bar;
    ...
 

/* safe if strlen(foo) + 1 < sizeof(string) */
wrote  = sprintf(string, "Foo: %s\n", foo);
/* overflows if strlen(foo) + strlen(bar) > sizeof(string) */
wrote += sprintf(string + wrote, "Bar: %s\n", bar);
/* writing way beyond the end of "string" now ... */
wrote += sprintf(string + wrote, "Baz: %s\n", baz);

/* safe, assuming available <= sizeof(string), and for this example
 * assume strlen(foo) < sizeof(string) */
wrote  = snprintf(string, available, "Foo: %s\n", foo);
/* if (strlen(bar) > available - wrote), this is still safe since the
 * write into "string" will be truncated, but now "wrote" has been
 * incremented by how much snprintf() *would* have written, so "wrote"
 * is now larger than "available". */
wrote += snprintf(string + wrote, available - wrote, "Bar: %s\n", bar);
/* string + wrote is beyond the end of string, and availabe - wrote wraps
 * around to a giant positive value, making the write effectively 
 * unbounded. */
wrote += snprintf(string + wrote, available - wrote, "Baz: %s\n", baz);

2020, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 License.

TL;DR: IKR! HOW DO I FIX THIS!? Make sure you have SUBSCRIBE_FORM_SECRET set in your mailman configuration:

SECRET=$(tr -dc 'A-Za-z0-9' < /dev/urandom   head -c 30)'
echo "SUBSCRIBE_FORM_SECRET = '$SECRET'" >> /etc/mailman/mm.cfg

This will add a magic token to all forms in the Mailman web forms that will force the attacker to at least get a token before asking for registration. There are, of course, other ways of performing the attack then, but it's more expensive than a single request for the attacker and keeps most of the junk out.

Other solutions I originally deployed a different fix, using referrer checks and an IP block list:

RewriteMap hosts-deny  txt:/etc/apache2/blocklist.txt
RewriteCond $ hosts-deny:% REMOTE_ADDR NOT-FOUND  !=NOT-FOUND [OR]
RewriteCond $ hosts-deny:% REMOTE_HOST NOT-FOUND  !=NOT-FOUND [OR]
RewriteCond % HTTP_REFERER  !^https://lists.torproject.org/$ [NC]
RewriteRule ^/cgi-bin/mailman/subscribe/ - [F]
# see also https://www.w3.org/TR/referrer-policy/#referrer-policy-origin
Header always set Referrer-Policy "origin"

I kept those restrictions in place because it keeps the spammers from even hitting the Mailman CGI, which is useful to preserve our server resources. But if "they" escalate with smarter crawlers, the block list will still be useful. You can use this query to extract the top 10 IP addresses used for subscription attempts:

awk '  print $NF  ' /var/log/mailman/subscribe   sort   uniq -c   sort -n   tail -10    awk '  print $2 " " $1  '

Note that this might include email-based registration, but in our logs those are extremely rare: only two in three weeks, out of over 73,000 requests. I also use this to keep an eye on the logs:

tail -f  /var/log/mailman/subscribe /var/log/apache2/lists.torproject.org-access.log   grep -v 'GET /pipermail/'

The server-side mitigations might also be useful if you happen to run an extremely old version of Mailman, that is pre-2.1.18, but it's now over 6 years old and part of every supported Debian release out there (all the way back to Debian 8 jessie).

Why does that attack work? Because Mailman 2 doesn't have CSRF tokens in its forms by default, anyone can send a POST request to /mailman/subscribe/LISTNAME to have Mailman send an email to the user. In the old "Internet is for nice people" universe, that wasn't a problem: all it does is ask the victim if they want to subscribe to LISTNAME. Innocuous, right? But in the brave, new, post-Eternal-September, "Internet is for stupid" universe, some assholes think it's a good idea to make a form that collects hundreds of mailing list URLs and spam them through an iframe. To see what that looks like, you can look at the rendered source code behind samedyfreeday.co.uk (not linking to avoid promoting it). That site does what is basically a distributed cross-site scripting attack against Mailman servers. Obviously, CSRF protection should be enabled by default in Mailman, but there you go. Hopefully this will help some folks... (The latest Mailman 3 release doesn't suffer from such idiotic defaults and ships with proper CSRF protection out of the box.)

• stmmac IPQ806x FIFO sizes. I sent out an RFC patch for these, but didn t get any replies. I probably just need to submit this.
• NAND. This is missing support for the QCOM ADM DMA engine. I ve sent out the patch I found to enable this, and have had some feedback, so I m hopeful it will get in at some point.
• LCD. AFAICT LCD is an ST7735 device, which has kernel support, but I haven t spent serious effort getting the SPI configuration to work.
• Touchscreen. Again, this seems to be a zt2046q or similar, which has a kernel driver, but the basic attempts I ve tried don t get any response.
• Proper SFP functionality. The IPQ806x has a PCS module, but the stmmac driver doesn t have an easy way to plumb this in. I have ideas about how to get it working properly (and it can be hacked up with a fixed link config) but it s not been a high priority.
• Device tree additions. Some of the later bits I ve enabled aren t yet in the mainline RB3011 DTB. I ll submit a patch for that at some point.

- name: create a device in NetBox
  netbox_device:
    netbox_url: http://netbox.local
    netbox_token: s3cret
    data:
      name: to3-p14.sfo1.example.com
      device_type: QFX5110-48S
      device_role: Compute Switch
      site: SFO1

Code The module has the following signature and it syncs NetBox with the content of the provided YAML file:

netbox_sync:
  source: netbox.yaml
  api: https://netbox.example.com
  token: s3cret

The synchronized objects are:

• sites,
• manufacturers,
• device types,
• device roles,
• devices, and
• IP addresses.

In our environment, the YAML file is generated from our configuration management database and contains a set of devices and a list of IP addresses:

devices:
  ad2-p6.sfo1.example.com:
     datacenter: sfo1
     manufacturer: Cisco
     model: Catalyst 2960G-48TC-L
     role: net_tor_oob_switch
  to1-p6.sfo1.example.com:
     datacenter: sfo1
     manufacturer: Juniper
     model: QFX5110-48S
     role: net_tor_gpu_switch
# [ ]
ips:
  - device: ad2-p6.example.com
    ip: 172.31.115.18/21
    interface: oob
  - device: to1-p6.example.com
    ip: 172.31.115.33/21
    interface: oob
  - device: to1-p6.example.com
    ip: 172.31.254.33/32
    interface: lo0.0
# [ ]

The network team is not the sole tenant in NetBox. While adding new objects or modifying existing ones should be relatively safe, deleting unwanted objects can be risky. The module only deletes objects it did create or modify. To identify them, it marks them with a specific tag, cmdb. Most objects in NetBox accept tags.

Module definition Starting from the skeleton described in the previous article, we define the module:

module_args = dict(
    source=dict(type='path', required=True),
    api=dict(type='str', required=True),
    token=dict(type='str', required=True, no_log=True),
    max_workers=dict(type='int', required=False, default=10)
)
result = dict(
    changed=False
)
module = AnsibleModule(
    argument_spec=module_args,
    supports_check_mode=True
)

It contains an additional optional arguments defining the number of workers to talk to NetBox and query the existing objects in parallel to speedup the execution.

Abstracting synchronization We need to synchronize different object types, but once we have a list of objects we want in NetBox, the grunt work is always the same:

• check if the objects already exist,
• retrieve them and put them in a form suitable for comparison,
• retrieve the extra objects we don t want anymore,
• compare the two sets, and
• add missing objects, update existing ones, delete extra ones.

We code these behaviours into a Synchronizer abstract class. For each kind of object, a concrete class is built with the appropriate class attributes to tune its behaviour and a wanted() method to provide the objects we want. I am not explaining the abstract class code here. Have a look at the source if you want.

Synchronizing tags and tenants As a starter, here is how we define the class synchronizing the tags:

class SyncTags(Synchronizer):
    app = "extras"
    table = "tags"
    key = "name"
    def wanted(self):
        return  "cmdb": dict(
            slug="cmdb",
            color="8bc34a",
            description="synced by network CMDB") 

The app and table attributes defines the NetBox objects we want to manipulate. The key attribute is used to determine how to lookup for existing objects. In this example, we want to lookup tags using their names. The wanted() method is expected to return a dictionary mapping object keys to the list of wanted attributes. Here, the keys are tag names and we create only one tag, cmdb, with the provided slug, color and description. This is the tag we will use to mark the objects we create or modify. If the tag does not exist, it is created. If it exists, the provided attributes are updated. Other attributes are left untouched. We also want to create a specific tenant for objects accepting such an attribute (devices and IP addresses):

class SyncTenants(Synchronizer):
    app = "tenancy"
    table = "tenants"
    key = "name"
    def wanted(self):
        return  "Network": dict(slug="network",
                                description="Network team") 

Synchronizing sites We also need to synchronize the list of sites. This time, the wanted() method uses the information provided in the YAML file: it walks the devices and builds a set of datacenter names.

class SyncSites(Synchronizer):
    app = "dcim"
    table = "sites"
    key = "name"
    only_on_create = ("status", "slug")
    def wanted(self):
        result = set(details["datacenter"]
                     for details in self.source['devices'].values()
                     if "datacenter" in details)
        return  k: dict(slug=k,
                        status="planned")
                for k in result 

Thanks to the use of the only_on_create attribute, the specified attributes are not updated if they are different. The goal of this synchronizer is mostly to collect the references to the different sites for other objects.

>>> pprint(SyncSites(**sync_args).wanted())
 'sfo1':  'slug': 'sfo1', 'status': 'planned' ,
 'chi1':  'slug': 'chi1', 'status': 'planned' ,
 'nyc1':  'slug': 'nyc1', 'status': 'planned' 

Synchronizing manufacturers, device types and device roles The synchronization of manufacturers is pretty similar, except we do not use the only_on_create attribute:

class SyncManufacturers(Synchronizer):
    app = "dcim"
    table = "manufacturers"
    key = "name"
    def wanted(self):
        result = set(details["manufacturer"]
                     for details in self.source['devices'].values()
                     if "manufacturer" in details)
        return  k:  "slug": slugify(k) 
                for k in result 

Regarding the device types, we use the foreign attribute linking a NetBox attribute to the synchronizer handling it.

class SyncDeviceTypes(Synchronizer):
    app = "dcim"
    table = "device_types"
    key = "model"
    foreign =  "manufacturer": SyncManufacturers 
    def wanted(self):
        result = set((details["manufacturer"], details["model"])
                     for details in self.source['devices'].values()
                     if "model" in details)
        return  k[1]: dict(manufacturer=k[0],
                           slug=slugify(k[1]))
                for k in result 

The wanted() method refers to the manufacturer using its key attribute. In this case, this is the manufacturer name.

>>> pprint(SyncManufacturers(**sync_args).wanted())
 'Cisco':  'slug': 'cisco' ,
 'Dell':  'slug': 'dell' ,
 'Juniper':  'slug': 'juniper' 
>>> pprint(SyncDeviceTypes(**sync_args).wanted())
 'ASR 9001':  'manufacturer': 'Cisco', 'slug': 'asr-9001' ,
 'Catalyst 2960G-48TC-L':  'manufacturer': 'Cisco',
                           'slug': 'catalyst-2960g-48tc-l' ,
 'MX10003':  'manufacturer': 'Juniper', 'slug': 'mx10003' ,
 'QFX10002-36Q':  'manufacturer': 'Juniper', 'slug': 'qfx10002-36q' ,
 'QFX10002-72Q':  'manufacturer': 'Juniper', 'slug': 'qfx10002-72q' ,
 'QFX5110-32Q':  'manufacturer': 'Juniper', 'slug': 'qfx5110-32q' ,
 'QFX5110-48S':  'manufacturer': 'Juniper', 'slug': 'qfx5110-48s' ,
 'QFX5200-32C':  'manufacturer': 'Juniper', 'slug': 'qfx5200-32c' ,
 'S4048-ON':  'manufacturer': 'Dell', 'slug': 's4048-on' ,
 'S6010-ON':  'manufacturer': 'Dell', 'slug': 's6010-on' 

The device roles are defined like this:

class SyncDeviceRoles(Synchronizer):
    app = "dcim"
    table = "device_roles"
    key = "name"
    def wanted(self):
        result = set(details["role"]
                     for details in self.source['devices'].values()
                     if "role" in details)
        return  k: dict(slug=slugify(k),
                        color="8bc34a")
                for k in result 

Synchronizing devices A device is mostly a name with references to a role, a model, a datacenter and a tenant. These references are declared as foreign keys using the synchronizers defined previously.

class SyncDevices(Synchronizer):
    app = "dcim"
    table = "devices"
    key = "name"
    foreign =  "device_role": SyncDeviceRoles,
               "device_type": SyncDeviceTypes,
               "site": SyncSites,
               "tenant": SyncTenants 
    remove_unused = 10
    def wanted(self):
        return  name: dict(device_role=details["role"],
                           device_type=details["model"],
                           site=details["datacenter"],
                           tenant="Network")
                for name, details in self.source['devices'].items()
                if  "datacenter", "model", "role"  <= set(details.keys()) 

The remove_unused attribute is a safety implemented to fail if we have to delete more than 10 devices: this may be the indication there is a bug somewhere, unless one of your datacenter suddenly caught fire.

>>> pprint(SyncDevices(**sync_args).wanted())
 'ad2-p6.sfo1.example.com':  'device_role': 'net_tor_oob_switch',
                             'device_type': 'Catalyst 2960G-48TC-L',
                             'site': 'sfo1',
                             'tenant': 'Network' ,
 'to1-p6.sfo1.example.com':  'device_role': 'net_tor_gpu_switch',
                             'device_type': 'QFX5110-48S',
                             'site': 'sfo1',
                             'tenant': 'Network' ,
[ ]

Synchronizing IP addresses The last step is to synchronize IP addresses. We do not attach them to a device.² Instead, we specify the device names in the description of the IP address:

class SyncIPs(Synchronizer):
    app = "ipam"
    table = "ip-addresses"
    key = "address"
    foreign =  "tenant": SyncTenants 
    remove_unused = 1000
    def wanted(self):
        wanted =  
        for details in self.source['ips']:
            if details['ip'] in wanted:
                wanted[details['ip']]['description'] = \
                    f" details['device']  (and others)"
            else:
                wanted[details['ip']] = dict(
                    tenant="Network",
                    status="active",
                    dns_name="",        # information is present in DNS
                    description=f" details['device'] :  details['interface'] ",
                    role=None,
                    vrf=None)
        return wanted

There is a slight difficulty: NetBox allows duplicate IP addresses, so a simple lookup is not enough. In case of multiple matches, we choose the best by preferring those tagged with cmdb, then those already attached to an interface:

def get(self, key):
    """Grab IP address from NetBox."""
    # There may be duplicate. We need to grab the "best".
    results = super(Synchronizer, self).get(key)
    if len(results) == 0:
        return None
    if len(results) == 1:
        return results[0]
    scores = [0]*len(results)
    for idx, result in enumerate(results):
        if "cmdb" in result.tags:
            scores[idx] += 10
        if result.interface is not None:
            scores[idx] += 5
    return sorted(zip(scores, results),
                  reverse=True, key=lambda k: k[0])[0][1]

Getting the current and wanted states Each synchronizer is initialized with a reference to the Ansible module, a reference to a pynetbox s API object, the data contained in the provided YAML file and two empty dictionaries for the current and expected states:

source = yaml.safe_load(open(module.params['source']))
netbox = pynetbox.api(module.params['api'],
                      token=module.params['token'])
sync_args = dict(
    module=module,
    netbox=netbox,
    source=source,
    before= ,
    after= 
)
synchronizers = [synchronizer(**sync_args) for synchronizer in [
    SyncTags,
    SyncTenants,
    SyncSites,
    SyncManufacturers,
    SyncDeviceTypes,
    SyncDeviceRoles,
    SyncDevices,
    SyncIPs
]]

Each synchronizer has a prepare() method whose goal is to compute the current and wanted states. It returns True in case of a difference:

# Check what needs to be synchronized
try:
    for synchronizer in synchronizers:
        result['changed']  = synchronizer.prepare()
except AnsibleError as e:
    result['msg'] = e.message
    module.fail_json(**result)

Applying changes Back to the skeleton described in the previous article, the last step is to apply the changes if there is a difference between these states. Each synchronizer registers the current and wanted states in sync_args["before"][table] and sync_args["after"][table] where table is the name of the table for a given NetBox object type. The diff object is a bit elaborate as it is built table by table. This enables Ansible to display the name of each table before the diff representation:

# Compute the diff
if module._diff and result['changed']:
    result['diff'] = [
        dict(
            before_header=table,
            after_header=table,
            before=yaml.safe_dump(sync_args["before"][table]),
            after=yaml.safe_dump(sync_args["after"][table]))
        for table in sync_args["after"]
        if sync_args["before"][table] != sync_args["after"][table]
    ]
# Stop here if check mode is enabled or if no change
if module.check_mode or not result['changed']:
    module.exit_json(**result)

Each synchronizer also exposes a synchronize() method to apply changes and a cleanup() method to delete unwanted objects. Order is important due to the relation between the objects.

# Synchronize
for synchronizer in synchronizers:
    synchronizer.synchronize()
for synchronizer in synchronizers[::-1]:
    synchronizer.cleanup()
module.exit_json(**result)

---

The complete code is available on GitHub. Compared to using netbox.netbox collection, the logic is written in Python instead of trying to glue Ansible tasks together. I believe this is both more flexible and easier to read, notably when trying to delete outdated objects. While I did not test it, it should also be faster. An alternative would have been to reuse code from the netbox.netbox collection, as it contains similar primitives. Unfortunately, I didn t think of it until now.

---

1. In my opinion, a good option for a source of truth is to use YAML files in a Git repository. You get versioning for free and people can get started with a text editor.
2. This limitation is mostly due to laziness: we do not really care about this information. Our main motivation for putting IP addresses in NetBox is to keep track of the used IP addresses. However, if an IP address is already attached to an interface, we leave this association untouched.