For DebConf20, we are recommending that speakers pre-record the presentation part of their talks, and will have live Q&A. We had a smaller online MiniDebConf a couple of months ago, where for instance I had connectivity issues during my talk, so even though it feels too artificial, I guess pre-recording can decrease by a lot the likelihood of a given talk going bad. Paul Gevers and I submitted a short 20 min talk giving an update on autopkgtest, ci.debian.net and friends. We will provide the latest updates on autopkgtest, autodep8, debci, ci.debian.net, and its integration with the Debian testing migration software, britney. We agreed on a split of the content, each one recorded their part, and I offered to join them together. The logical chaining of the topics is such that we can't just concatenate the recordings, so we need to interlace our parts. So I set out to do a full video editing work. I have done this before, although in a simpler way, for one of the MiniDebconfs we held in Curitiba. In that case, it was just cutting the noise at the beginning and the end of the recording, and adding beginning and finish screens with sponsors logos etc. The first issue I noticed was that both our recordings had a decent amount of audio noise. To extract the audio track from the videos, I resorted to How can I extract audio from video with ffmpeg? on Stack Overflow:

ffmpeg -i input-video.avi -vn -acodec copy output-audio.aac

I then edited the audio with Audacity. I passed a noise reduction filter a couple of times, then a compressor filter to amplify my recording on mine, as Paul's already had a good volume. And those are my more advanced audio editing skills, which I acquired doing my own podcast. I now realized I could have just muted the audio tracks from the original clip and align the noise-free audio with it, but I ended up creating new video files with the clean audio. Another member of the Stack Overflow family came to the rescue, in How to merge audio and video file in ffmpeg. To replace the audio stream, we can do something like this:

ffmpeg -i video.mp4 -i audio.wav -c:v copy -c:a aac -map 0:v:0 -map 1:a:0 output.mp4

Paul's recording had a 4:3 aspect ratio, while the requested format is 16:9. This late in the game, there was zero chance I would request him to redo the recording. So I decided to add those black bars on the side to make it the right aspect when showing full screen. And yet again the quickest answer I could find came from the Stack Overflow empire: ffmpeg: pillarbox 4:3 to 16:9:

ffmpeg -i "input43.mkv" -vf "scale=640x480,setsar=1,pad=854:480:107:0" [etc..]

The final editing was done with pitivi, which is what I have used before. I'm a very basic user, but I could do what I needed. It was basically splitting the clips at the right places, inserting the slides as images and aligning them with the video, and making most our video appear small in the corner when presenting the slides. P.S.: all the command lines presented here are examples, basically copied from the linked Q&As, and have to be adapted to your actual input and output formats.

I'm starting a new blog, and instead of going into the technical details on how it's made, or on how I migrated my content from the previous ones, I want focus on why I did it. It's been a while since I have written a blog post. I wanted to get back into it, but also wanted to finally self-host my blog/homepage because I have been let down before. And sadly, I was not let down by a for-profit, privacy invading corporation, but by a free software organization. The sad story of wiki.softwarelivre.org My first blog that was hosted in a blog engine written by me, which was hosted in a TWiki, and later Foswiki instance previously available at wiki.softwarelivre.org, hosted by ASL.org. I was the one who introduced the tool to the organization in the first place. I had come from a previous, very fruitful experience on the use of wikis for creation of educational material while in university, which ultimately led me to become a core TWiki, and then Foswiki developer. In 2004, I had just moved to Porto Alegre, got involved in ASL.org, and there was a demand for a tool like that. 2 years later, I left Porto Alegre, and some time after that the daily operations of ASL.org when it became clear that it was not really prepared for remote participation. I was still maintaing the wiki software and the OS for quite some years after, until I wasn't anymore. In 2016, the server that hosted it went haywire, and there were no backups. A lot of people and free software groups lost their content forever. My blog was the least important content in there. To mention just a few examples, here are some groups that lost their content in there:

• The Brazilian Foswiki user group hosted a bunch of getting started documentation in Portuguese, organized meetings, and coordinated through there.
• GNOME Brazil hosted its homepage there until the moment the server broke.
• The Inkscape Brazil user group had an amazing space there where they shared tutorials, a gallery of user-contributed drawings, and a lot more.

Some of this can still be reached via the Internet Archive Wayback Machine, but that is only useful for recovering content, not for it to be used by the public. The announced tragedy of softwarelivre.org My next blog after that was hosted at softwarelivre.org, a Noosfero instance also hosted by ASL.org. When it was introduced in 2010, this Noosfero instance became responsible for the main domain softwarelivre.org name. This was a bold move by ASL.org, and was a demonstration of trust in a local free software project, led by a local free software cooperative (Colivre). I was a lead developer in the Noosfero project for a long time, and I was also involved in maintaining that server as well. However, for several years there is little to no investment in maintaining that service. I already expect that it will probably blow up at some point as the wiki did, or that it may be just shut down on purpose. On the responsibility of organizations Today, a large part of wast mot people consider "the internet" is controlled by a handful of corporations. Most popular services on the internet might look like they are gratis (free as in beer), but running those services is definitely not without costs. So when you use services provided by for-profit companies and are not paying for them with money, you are paying with your privacy and attention. Society needs independent organizations to provide alternatives. The market can solve a part of the problem by providing ethical services and charging for them. This is legitimate, and as long as there is transparency about how peoples' data and communications are handled, there is nothing wrong with it. But that only solves part of the problem, as there will always be people who can't afford to pay, and people and communities who can afford to pay, but would rather rely on a nonprofit. That's where community-based services, provided by nonprofits, are also important. We should have more of them, not less. So it makes me worry to realize ASL.org left the community in the dark. Losing the wiki wasn't even the first event of its kind, as the listas.softwarelivre.org mailing list server, with years and years of community communications archived in it, broke with no backups in 2012. I do not intend to blame the ASL.org leadership personally, they are all well meaning and good people. But as an organization, it failed to recognize the importance of this role of service provider. I can even include myself in it: I was member of the ASL.org board some 15 years ago; I was involved in the deployment of both the wiki and Noosfero, the former as a volunteer and the later professionally. Yet, I did nothing to plan the maintenance of the infrastructure going forward. When well meaning organizations fail, people who are not willing to have their data and communications be exploited for profit are left to their own devices. I can afford a virtual private server, and have the technical knowledge to import my old content into a static website generator, so I did it. But what about all the people who can't, or don't? Of course, these organizations have to solve the challenge of being sustainable, and being able to pay professionals to maintain the services that the community relies on. We should be thankful to these organizations, and their leadership needs to recognize the importance of those services, and actively plan for them to be kept alive.

I'm starting a new blog, and instead of going into the technical details on how it's made, or on how I migrated my content from the previous ones, I want to focus on why I did it. It's been a while since I have written a blog post. I wanted to get back into it, but also wanted to finally self-host my blog/homepage because I have been let down before. And sadly, I was not let down by a for-profit, privacy invading corporation, but by a free software organization. The sad story of wiki.softwarelivre.org My first blog that was hosted in a blog engine written by me, which was hosted in a TWiki, and later Foswiki instance previously available at wiki.softwarelivre.org, hosted by ASL.org. I was the one who introduced the tool to the organization in the first place. I had come from a previous, very fruitful experience on the use of wikis for creation of educational material while in university, which ultimately led me to become a core TWiki, and then Foswiki developer. In 2004, I had just moved to Porto Alegre, got involved in ASL.org, and there was a demand for a tool like that. 2 years later, I left Porto Alegre, and some time after that the daily operations of ASL.org when it became clear that it was not really prepared for remote participation. I was still maintaing the wiki software and the OS for quite some years after, until I wasn't anymore. In 2016, the server that hosted it went haywire, and there were no backups. A lot of people and free software groups lost their content forever. My blog was the least important content in there. To mention just a few examples, here are some groups that lost their content in there:

• The Brazilian Foswiki user group hosted a bunch of getting started documentation in Portuguese, organized meetings, and coordinated through there.
• GNOME Brazil hosted its homepage there until the moment the server broke.
• The Inkscape Brazil user group had an amazing space there where they shared tutorials, a gallery of user-contributed drawings, and a lot more.

Some of this can still be reached via the Internet Archive Wayback Machine, but that is only useful for recovering content, not for it to be used by the public. The announced tragedy of softwarelivre.org My next blog after that was hosted at softwarelivre.org, a Noosfero instance also hosted by ASL.org. When it was introduced in 2010, this Noosfero instance became responsible for the main domain softwarelivre.org name. This was a bold move by ASL.org, and was a demonstration of trust in a local free software project, led by a local free software cooperative (Colivre). I was a lead developer in the Noosfero project for a long time, and I was also involved in maintaining that server as well. However, for several years there is little to no investment in maintaining that service. I already expect that it will probably blow up at some point as the wiki did, or that it may be just shut down on purpose. On the responsibility of organizations Today, a large part of wast mot people consider "the internet" is controlled by a handful of corporations. Most popular services on the internet might look like they are gratis (free as in beer), but running those services is definitely not without costs. So when you use services provided by for-profit companies and are not paying for them with money, you are paying with your privacy and attention. Society needs independent organizations to provide alternatives. The market can solve a part of the problem by providing ethical services and charging for them. This is legitimate, and as long as there is transparency about how peoples' data and communications are handled, there is nothing wrong with it. But that only solves part of the problem, as there will always be people who can't afford to pay, and people and communities who can afford to pay, but would rather rely on a nonprofit. That's where community-based services, provided by nonprofits, are also important. We should have more of them, not less. So it makes me worry to realize ASL.org left the community in the dark. Losing the wiki wasn't even the first event of its kind, as the listas.softwarelivre.org mailing list server, with years and years of community communications archived in it, broke with no backups in 2012. I do not intend to blame the ASL.org leadership personally, they are all well meaning and good people. But as an organization, it failed to recognize the importance of this role of service provider. I can even include myself in it: I was member of the ASL.org board some 15 years ago; I was involved in the deployment of both the wiki and Noosfero, the former as a volunteer and the later professionally. Yet, I did nothing to plan the maintenance of the infrastructure going forward. When well meaning organizations fail, people who are not willing to have their data and communications be exploited for profit are left to their own devices. I can afford a virtual private server, and have the technical knowledge to import my old content into a static website generator, so I did it. But what about all the people who can't, or don't? Of course, these organizations have to solve the challenge of being sustainable, and being able to pay professionals to maintain the services that the community relies on. We should be thankful to these organizations, and their leadership needs to recognize the importance of those services, and actively plan for them to be kept alive.

Hello, In early May, I got selected as a Google Summer of Code student for Debian to work on a project which is to write a linter (an extension to RuboCop).
This tool is mostly to help the Debian Ruby team. And that is the best part, I love working in/for/with the Ruby team!
(I ve been an active part of the team for 19 months now :)) More details about the project can be found here, on the wiki.
And also, I have got the best mentors I could ve possibly asked for: Antonio Terceiro and David Rodr guez So, the program began on 1st June and I ve been working since then. I log my daily updates at gsocwithutkarsh2102.tk.
The blog for the first part of phase 1 can be found here and that of second part of phase 1 can be found here. Whilst the daily updates are available at the above site^, I ll breakdown the important parts here:

• After the, what I d like to call, successful Phase 1, whilst using this extension on GitLab s omnibus-gitlab repository, I discovered a bug (as reported via issue #5), which basically threw the following error:

  An error occurred while Packaging/GemspecGit cop was inspecting /home/utkarsh/github/omnibus-gitlab/files/gitlab-cookbooks/gitlab/recipes/bootstrap_disable.rb.
  To see the complete backtrace run rubocop -d.

  which was not good.
• This bug was a false-negative and the fix turned out to be simple, which was raised via PR #6.
  Since the extension was now working fine against omnibus-gitlab (which is quite huge!), I rolled out a v0.1.1 release!
• Next, I implemented the 2nd cop, fixing require and require_relative calls which map from spec(s)/test(s) to the lib directory.
  This was raised via a WIP PR #4.
• We had our meeting where we discussed that it would rather make sense to split these cops into two parts and also, thanks to David s elaborative comment on the situation.
  With this, we decided to split the cops into two.
• Then I worked on:
  • Splitting the cops via commit @6765fec8.
  • Dropping Ruby2.4 support via commit @3681e9a3.
  • Diversifying tests via commit @335a14e2.
• At this point, we hit yet another obstacle. Correctly determining the root directory of a project at runtime. This is tricky. Not impossible (of course) but tricky.
  So my homework was to find such a thing that does that by default.
• For the next 4 days, I tried to find something that could do this bit. But unfortunately, I couldn t.
  So I wrote one myself. I wrote get_root, which solves this problem.
  The only thing that one needs to take care while using get_root is that it has to be a git repository. That s a trade-off.
• In the next meeting, we discussed that this is a bit of an overkill. get_root should ve been written as a helper function and not as another library, which David and Antonio pointed out correctly. But it was already too late :P
  I had already made a v 0.1.0 release.
  (So in case you re a Rubyist and want to find the root directory of a git repository, consider using this :P)
  Besides, Antonio also pointed out that it should take another arguement as to from point from where the file is being inspected. Hm, unsure how to do that..
• Meanwhile, I exported get_root as a helper function, David solved all the problem altogether at once via rubocop s PR #8314.
  This introduced a new (public) method #project_root which superseeded get_root and one could now get the root directory via RuboCop::ConfigLoader.project_root. Ain t he amazing!? \o/
• This also means, I reverted those changes altogether and tweaked my WIP PR to inculcate these changes via commit @b06a8f86.
  However, the specs fail. But that doesn t mean that the changes aren t correct :P
  They are pretty much right and working fine. To make that sure, I locally installed this library and used on other projects to make sure that it indeed is working alright, as it should! \o/
• And here I am on the 15th day :)

Well, the best part yet?
rubocop-packaging is being used by batalert, arbre, rspec-stubbed_env, rspec-pending_for, ISO8601, get_root ( ), gir_ffi, linter, and cucumber-rails. Whilst it has been a lot of fun so far, my plate has started to almost overflow. It seems that I ve got a lot of things to work on (and already things that are due!).
From my major project, college *stuff to my GSoC project, Debian (E)LTS, and a lot *more. Thanks to Antonio for helping me out with *other things (which maps back to his sayings in Paris \o/).

---

Until next time.
:wq for today.

Here s my (ninth) monthly update about the activities I ve done in the F/L/OSS world.

Debian

This was my 16th month of contributing to Debian. I became a DM in late March last year and a DD last Christmas! \o/ This month was a little intense. I did a lot of different kinds of things in Debian this month. Whilst most of my time went on doing security stuff, I also sponsored a bunch of packages. Here are the following things I did this month:

Uploads and bug fixes:

• rails (2:5.2.4.3+dfsg-1) - fix a bunch of CVEs in Sid and Bullseye.
• ruby-json (2.1.0+dfsg-2+deb10u1) - backport CVE-2020-10663 fix to Buster.
• ruby-json (2.0.1+dfsg-3+deb9u1) - backport CVE-2020-10663 fix to Stretch.
• ruby-kaminari (1.0.1-6) - add patch to fix CVE-2020-11082.
• ruby2.3 (2.3.3-1+deb9u8) - backport CVE-2020-10663 fix to Stretch.
• python-libusb1 (1.8-1.1) - NMU for a source-only upload.
• pry (0.13.1-1) - Fix failing tests & new upstream version.
• ruby-rubocop-packaging (0.1.0-1) - NEW (#963016).
• batalert (0.4.0-1) - fixes against RuboCop.
• json-schema-test-suite (2.0.0-1.1) - NMU for a source-only upload on request.
• ruby-ahoy-email (1.1.0-1) - Disable tests temporarily (#959060).
• micro (2.0.6-1) - fix crashing at startup (#961853).
• golang-github-zyedidia-tcell (1.4.8-1) - new upstream version.
• micro (2.0.6-1~bpo10+1) - backport the fix for (#961853).
• micro (2.0.6-2) - fix the reintroduced versioning issue (#953400).
• ruby-whitequark-parser (2.7.1.4-1) - new upstream version.

Other $things:

• Hosted Ruby team meeting. Logs here.
• Mentoring for newcomers.
• FTP Trainee reviewing.
• Moderation of -project mailing list.
• Sponsored ruby-ast for Abraham, libexif for Hugh, djangorestframework-gis and karlseguin-ccache for Nilesh, and twig-extensions, twig-i18n-extension, and mariadb-mysql-kbs for William.

---

GSoC Phase 1, Part 2! Last month, I got selected as a Google Summer of Code student for Debian again! \o/
I am working on the Upstream-Downstream Cooperation in Ruby project. The first half of the first month is blogged here, titled, GSoC Phase 1.
Also, I log daily updates at gsocwithutkarsh2102.tk. Whilst the daily updates are available at the above site^, I ll breakdown the important parts of the later half of the first month here:

• Documented the first cop, GemspecGit via PR #2.
• Made an initial release, v0.1.0!
• Spread the word/usage about this tool/library via adding them in the official RuboCop docs.
• We had our third weekly meeting where we discussed the next steps and the things that are supposed to be done for the next set of cops.
• Wrote more tests so as to cover different aspects of the GemspecGit cop.
• Opened PR #4 for the next Cop, RequireRelativeToLib.
• Introduced rubocop-packaging to the outer world and requested other upstream projects to use it! It is being used by 6 other projects already
• Had our fourth weekly meeting where we pair-programmed (and I sucked :P) and figured out a way to make the second cop work.
• Found a bug, reported at issue #5 and raised PR #6 to fix it.
• And finally, people loved the library/tool (and it s outcome):
  
  
  
  (for those who don t know, @bbatsov is the author of RuboCop, @lienvdsteen is an amazing fullstack engineer at GitLab, and @pboling is the author of some awesome Ruby tools and libraries!)

---

Continuation of GSoC for other Ruby related stuff!

Whilst I have already mentioned it multiple times but it s still not enough to stress how amazing Antonio Terceiro and David Rodr guez are!
They re more than just mentors to me! Well, only they know how much I trouble them with different things, which are not only related to my GSoC project but also extends to the projects they maintain! :P
David maintains rubygems and bundler and Antonio maintains debci. So on days when I decide to hack on rubygems or debci, only I know how kind and nice David and Anotonio are to me!
They very patiently walk me through with whatever I am stuck on, no matter what and no matter when. Thus, with them around, I contributed to these two projects and more, with regards to working on rubocop-packaging.
Following are a few things that I raised:

• PR #3731 for rubygems/bundler to ship default .rubocop.yml file.
• PR #2140 for pry to fix bundler_spec test.
• PR #3740 for rubygems/bundler to fix all RuboCop offenses.
• MR #114 for debci to show package details on the retry page.
• Issue #356 against rake to request to support all gitignore rule patterns in rake/file_list.
• PR #9 for fast_ignore to use fast_ignore instead of git ls-files.
• PR #3748 for rubygems/bundler to add actions to automatically bump man page month.
• PR #8160 for rubocop to add rubocop-packaging as a known extension.
• PR #3754 for rubygems/bundler to constrain the shipped RuboCop s version.
• Issue #8 against fast_ignore to clarify the strange behavior of include_files.
• PR #3765 for rubygems/bundler to fix remaining RuboCop issues and add tests.

---

Debian LTS

Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success. This was my ninth month as a Debian LTS paid contributor. I was assigned 30.00 hours and worked on the following things:

CVE Fixes and Announcements:

• Issued DLA 2215-1, fixing CVE-2020-3327 and CVE-2020-3341, for clamav.
  For Debian 8 Jessie , these problems have been fixed in version 0.101.5+dfsg-0+deb8u2.
• Issued DLA 2216-1, fixing CVE-2020-8161, for ruby-rack.
  For Debian 8 Jessie , this problem has been fixed in version 1.5.2-3+deb8u3.
• Issued DLA 2234-1, fixing CVE-2005-1513, CVE-2005-1514, CVE-2005-1515, CVE-2020-3811, and CVE-2020-3812, for netqmail.
  For Debian 8 Jessie , these problems have been fixed in version 1.06-6.2~deb8u1.
• Uploaded a fix for CVE-2020-8162, CVE-2020-8164, CVE-2020-8165, CVE-2020-8166, and CVE-2020-8167, for rails. This upload was for Sid and Bullseye and these CVE(s) were fixed in version 2:5.2.4.3+dfsg-1.
• Uploaded a fix for CVE-2020-11082, for ruby-kaminari. This upload was for Sid and Bullseye and this CVE was fixed in version 1.0.1-6.
• Uploaded a fix for CVE-2020-10663, for ruby-json, ruby2.1, and ruby2.5. These uploads were for Stretch and Buster and were fixed in the version 2.3.3-1+deb9u8, 2.1.0+dfsg-2+deb10u1, 2.3.3-1+deb9u8, and 2.5.5-3+deb10u2.
• Issued DLA 2237-1, fixing CVE-2019-8842 and CVE-2020-3898, for cups.
  For Debian 8 Jessie , these problems have been fixed in version 1.7.5-11+deb8u8.
• Issued DLA 2246-1, fixing CVE-2020-13696, for xawtv.
  For Debian 8 Jessie , this problem has been fixed in version 3.103-3+deb8u1.
• Issued DLA 2248-1, fixing CVE-2020-0543, CVE-2020-0548, and CVE-2020-0549, for intel-microcode.
  For Debian 8 Jessie , these problems have been fixed in version 3.20200609.2~deb8u1.
• Issued DLA 2249-1, fixing CVE-2020-0182 and CVE-2020-0198, for libexif.
  For Debian 8 Jessie , these problems have been fixed in version 0.6.21-2+deb8u4.

Other LTS Work:

• Triaged sympa, apache2, qemu, and coturn.
• Add fix for CVE-2020-0198/libexif.
• Requested CVE for bug#60251 against apache2 and prodded further.
• Raised issue #947 against sympa reporting an incomplete patch for CVE-2020-10936. More discussions internally.
• Created the LTS Survey on the self-hosted LimeSurvey instance.
• Attended the third LTS meeting. Logs here.
• General discussion on LTS private and public mailing list.

---

Other(s)

Sometimes it gets hard to categorize work/things into a particular category.
That s why I am writing all of those things inside this category.
This includes two sub-categories and they are as follows.

Personal: This month I did the following things:

• Wrote and published v0.1.0 of rubocop-packaging on RubyGems!
  It s open-sourced and the repository is here.
  Bug reports and pull requests are welcomed!
• Integrated a tiny (yet a powerful) hack to align images in markdown for my blog.
  Commit here.
• Released v0.4.0 of batalert on RubyGems!

Open Source: Again, this contains all the things that I couldn t categorize earlier.
Opened several issues and PRs:

• Issue #9 against 100daysof, reporting some broken CSS.
• PR #10 for 100daysof, fixing the above issue^.
• Issue #133 against djangorestframework-api-key, asking to fix copyright years.
• PR #5 for rspec-stubbed_env, dropping git ls-files in gemspec.
• PR #70 for rspec-pending_for, dropping git ls-files in gemspec.
• Issue #74, issue #143, issue #164, and issue #767 against multiple projects, asking them to use RuboCop.
• PR #212 for arbre, dropping git ls-files in gemspec.
• Issue #1749 and issue #1750 against micro, asking for help as the Debian package fails to build in buster-backports.
• PR #1751 for micro, fixing the above issue^.
• Issue #348 against hugo-coder, clarifying the weird timing issue in the blog posts.
• Issue #356 against hugo-coder, reporting the weird display of images and missing twitter cards.
• MR #12 for linter, dropping git ls-files in gemspec.
• MR #13 for linter, doing so minor refactoring.

---

Thank you for sticking along for so long :) Until next time.
:wq for today.

Hello, Earlier last month, I got selected as a Google Summer of Code student for Debian again! \o/
And as Chandler would say,

Could I be any more happier?

Well, this time, my project is basically to write a linter (an extension to RuboCop). This tool is mostly to help the Debian Ruby team. And that is the best part, I love working in/for/with the Ruby team!
(I ve been an active part of the team for 18 months now :)) More details about the project can be found here, on the wiki.
And also, I have got the best mentors I could ve possibly asked for: Antonio Terceiro and David Rodr guez So, the program began on 1st June and I ve been working since then. I log my daily updates at gsocwithutkarsh2102.tk. Whilst the daily updates are available at the above site^, I ll breakdown the important parts here:

• During the first three days, I looked for a potential solution to the usage of git ls-files in the gemspec files. This has been the most problematic thing for us.
  • Apart from the option of using Dir or Dir.glob, the best (closest) possible solution (right now) is to use Rake::FileList which tries to respects the .gitignore file.
  • I stumbled upon this interesting gem, fast_ignore. It is the exact thing which we want to use but unfortunately, to use it inside other gemspec files, it should be vendored inside bundler s code.
• We had our first meeting on the fourth day and we decided to hold meetings every Thursday for the next 12 weeks.
• For the next five days, I learned more of Ruby and figured out what to do and how to do it.
  If you d like to know what exactly I did in these 5 days, I d suggest you to read the daily logs for those respective days.
• During the next to two days, the first part of the project, the GemspecGit Cop, was implemented.
  This cop will correctly determine the usage of git in the gemspec files and would tell the developers and the maintainers to replace them with pure Ruby alternatives with giving them a proper reason to do so. Much thanks to Dana for her help she took out time to pair-program with me!
• We had our second weekly meeting where I finally told Antonio and David that the first part is already done (\o/) and we discussed some things and even pair-programmed :D
• I took the weekend off (and something terrible happened) but anyways, I managed to get together some time and energy to document the source code and raised this PR #2.
• And here I am on the 15th day :)

It has been a lot of fun so far! Though I am little worried on how to implement the next part of the project as I am not sure how to check only a particalar directory for some relative require calls.
But I think, that s okay, somehow, something will work out. And I can always ask around others and check other cops to see how it s done! \( )/

---

Until next time.
:wq for today.

Here s my (eighth) monthly update about the activities I ve done in the F/L/OSS world.

Debian

This month marks my 15 months of contributing to Debian. And 6th month as a DD! \o/ Whilst I love doing Debian stuff, I have started spending more time on the programming side now. And I hope to keep it this for some time now.
Of course, I ll keep doing the Debian stuff, but just lesser in amount. Anyway, the following are the things I did in May.

Uploads:

• ruby-aggregate (0.2.3-1) - got patches merged upstream.
• ruby-whenever (1.0.0-1) - new upstream version + take over maintenance.
• polybar (3.4.3-1) - fix GCC 10 compilation.
• ruby-dbus (0.16.0-1) - new upstream version + fix FTBFS (temporarily).
• ruby-rack (2.1.1-5) - use Dir.entries instead of Dir[glob]. Fixes CVE-2020-8161.
• ruby-espeak (1.0.4-2) - fix FTBFS (#952587).
• ruby-libnotify (0.9.4-1) - NEW (#961577). Needed by batalert.
• batalert (0.3.0-1) - NEW (#961580).
• golang-github-zyedidia-tcell (1.4.5-1) - fix tcell ID for micro.
• micro (2.0.4-1) - new release features + change in build path.

Other $things:

• Hosted Ruby team meeting. Logs here.
• Attended Debian Perl Sprints. Report here.
• Sponsored git-repo-updater and mplcursors for Sudip.
• Mentoring for newcomers.
• FTP Trainee reviewing.
• Moderation of -project mailing list.
• Got selected for GSoC 20 for Debian!

---

Experimenting and improving Ruby libraries FTW!

I have been very heavily involved with the Debian Ruby team for over an year now.
Thanks to Antonio Terceiro (and GSoC), I ve started experimenting and taking more interest in upstream development and improvement of these libraries. This has the sole purpose of learning. It has gotten fun since I ve started doing Ruby.
And I hope it stays this way. This month, I opened some issues and proposed a few pull requests. They are:

• Issue #802 against whenever for Ruby2.7 test failures.
• Issue #8 against aggregate asking upstream for a release on rubygems.
• Issue #104 against irb for asking more about Array.join("\n").
• Issue #1391 against mail asking upstream to cut a new release.
• Issue #1655 against rack reporting test failures in the CVE fix.
• Issue #84 against ruby-dbus for help with Debian bug #836296.
• Issue #85 against ruby-dbus asking if they still use rDoc for doc generation.
• PR #9 against aggregate for dropping git from gemspec.
• PR #804 against whenever for dropping git from gemspec.
• Packaged ruby-cmath as it was split from Ruby2.7; cf: (#961213).

---

Debian LTS

Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success. This was my eighth month as a Debian LTS paid contributor. I was assigned 17.25 hours and worked on the following things:

CVE Fixes and Announcements:

• Issued DLA 2191-1, fixing CVE-2020-10683, for dom4j.
  For Debian 8 Jessie , this problem has been fixed in version 1.6.1+dfsg.3-2+deb8u2.
• Issued DLA 2192-1, fixing CVE-2020-10663, for ruby2.1.
  For Debian 8 Jessie , this problem has been fixed in version 2.1.5-2+deb8u10.
• Issued DLA 2208-1, fixing CVE-2020-11026, CVE-2020-11027, CVE-2020-11028, and CVE-2020-11029, for wordpress.
  For Debian 8 Jessie , these problems have been fixed in version 4.1.30+dfsg-0+deb8u1.
• Issued DLA 2210-1, fixing CVE-2020-3810, for apt.
  This update was prepared by the maintainer, Julian. I just took care of the paperwork.
  For Debian 8 Jessie , this problem has been fixed in version 1.0.9.8.6.

Other LTS Work:

• Triaged tika, freerdp, and apache2.
• Mark CVE-2020-12105/openconnect as no-dsa not-affected for Jessie.
• Mark CVE-2020-9489/tika as no-dsa ignored for Jessie.
• Mark CVE-2020-11025/wordpres as not-affected for Jessie.
• Add fix for Add fix for CVE-2019-18823/condor.
• Requested CVE for bug#60251 against apache2.
• Raised issue #947 against sympa reporting an incomplete patch for CVE-2020-10936.
• Created the LTS Survey on the self-hosted LimeSurvey instance.
• Attended the second LTS meeting. Logs here.
• General discussion on LTS private and public mailing list.

---

Other(s)

Sometimes it gets hard to categorize work/things into a particular category.
That s why I am writing all of those things inside this category.
This includes two sub-categories and they are as follows.

Personal: This month I could get the following things done:

• Wrote and published my first Ruby gem/library/tool on RubyGems!
  It s open-sourced and the repository is here.
  Bug reports and pull requests are welcomed!
• Wrote a small Ruby script (available here) to install Ruby gems from Gemfile(.lock).
  Needed this when I hit a bug while using ruby-standalone, which Antonio fixed pretty quickly!
• Had a coffee chat with John Coghlan!
  Tweet here.

Open Source: Again, this contains all the things that I couldn t categorize earlier.
Opened several issues and did a PR review:

• Issue #15434 against phantomjs, asking to look into CVE-2019-17221. Still no action :/
• Issue #947 against sympa, reporting an incomplete patch for CVE-2020-10936.
• Issue #2102 against polybar, mentioning that the build is not reproducible.
• Issue #5521 against libgit2, mentioning that the build is not reproducible.
• Reviewed PR #5523 for polybar, which was a fix for the above issue.

---

Until next time.
:wq for today.

I know most Debian people know about this already But in case you don t follow the usual Debian communications channels, this might interest you! Given most of the world is still under COVID-19 restrictions, and that we want to work on Debian, given there is no certainty as to what the future holds in store for us Our DPL fearless as they always are had the bold initiative to make this weekend into the first-ever miniDebConf Online (MDCO)! So, we are already halfway through DebCamp (which means, you can come and hang out with us in the debian.social DebCamp Jitsi lounge, where some impromptu presentations might happen (or not). Starting tomorrow morning (11AM UTC), we will have a quite interesting set of talks. I am reproducing the schedule here:

Saturday 2020.05.30

Time (UTC)	Speaker	Talk
11:00 - 11:10	MDCO team members	Hello + Welcome
11:30 - 11:50	Wouter Verhelst	Extrepo
12:00 - 12:45	JP Mengual	Debian France, trust european organization
13:00 - 13:20	Arnaud Ferraris	Bringing Debian to mobile phones, one package at a time
13:30 - 15:00	Lunch Break	A chance for the teams to catch some air
15:00 - 15:45	JP Mengual	The community team, United Nations Organizations of Debian?
16:00 - 16:45	Christoph Biedl	Clevis and tang - overcoming the disk unlocking problem
17:00 - 17:45	Antonio Terceiro	I m a programmer, how can I help Debian

Sunday 2020.05.31

Time (UTC)	Speaker	Talk
11:00 - 11:45	Andreas Tille	The effect of Covid-19 on the Debian Med project
12:00 - 12:45	Paul Gevers	BoF: running autopkgtest for your package
13:00 - 13:20	Ben Hutchings	debplate: Build many binary packages with templates
13:30 - 15:00	Lunch break	A chance for the teams to catch some air
15:00 - 15:45	Holger Levsen	Reproducing bullseye in practice
16:00 - 16:45	Jonathan Carter	Striving towards excellence
17:00 - 17:45	Delib*	Organizing Peer-to-Peer Debian Facilitation Training
18:00 - 18:15	MDCO team members	Closing

• subject to confirmation

Timezone Remember this is an online event, meant for all of the world! Yes, the chosen times seem quite Europe-centric (but they are mostly a function of the times the talk submitters requested). Talks are 11:00 18:00UTC, which means, 06:00 13:00 Mexico (GMT-5), 20:00 03:00 Japan (GMT+9), 04:00 11:00 Western Canada/USA/Mexico (GMT-7) and the rest of the world, somewhere in between. (No, this was clearly not optimized for our dear usual beer team. Sorry! I guess we need you to be fully awake at beertime!)

[update] Connecting! Of course, I didn t make it clear at first how to connect to the Online miniDebConf, silly me!

• The video streams are available at: https://video.debconf.org/
• Suggested: tune in to the #minidebconf-online IRC channel in OFTC.

That should be it. Hope to see you there! (Stay home, stay safe )

I know most Debian people know about this already But in case you don t follow the usual Debian communications channels, this might interest you! Given most of the world is still under COVID-19 restrictions, and that we want to work on Debian, given there is no certainty as to what the future holds in store for us Our DPL fearless as they always are had the bold initiative to make this weekend into the first-ever miniDebConf Online (MDCO)! So, we are already halfway through DebCamp (which means, you can come and hang out with us in the debian.social DebCamp Jitsi lounge, where some impromptu presentations might happen (or not). Starting tomorrow morning (11AM UTC), we will have a quite interesting set of talks. I am reproducing the schedule here:

Saturday 2020.05.30

Time (UTC)	Speaker	Talk
11:00 - 11:10	MDCO team members	Hello + Welcome
11:30 - 11:50	Wouter Verhelst	Extrepo
12:00 - 12:45	JP Mengual	Debian France, trust european organization
13:00 - 13:20	Arnaud Ferraris	Bringing Debian to mobile phones, one package at a time
13:30 - 15:00	Lunch Break	A chance for the teams to catch some air
15:00 - 15:45	JP Mengual	The community team, United Nations Organizations of Debian?
16:00 - 16:45	Christoph Biedl	Clevis and tang - overcoming the disk unlocking problem
17:00 - 17:45	Antonio Terceiro	I m a programmer, how can I help Debian

Sunday 2020.05.31

Time (UTC)	Speaker	Talk
11:00 - 11:45	Andreas Tille	The effect of Covid-19 on the Debian Med project
12:00 - 12:45	Paul Gevers	BoF: running autopkgtest for your package
13:00 - 13:20	Ben Hutchings	debplate: Build many binary packages with templates
13:30 - 15:00	Lunch break	A chance for the teams to catch some air
15:00 - 15:45	Holger Levsen	Reproducing bullseye in practice
16:00 - 16:45	Jonathan Carter	Striving towards excellence
17:00 - 17:45	Delib*	Organizing Peer-to-Peer Debian Facilitation Training
18:00 - 18:15	MDCO team members	Closing

• subject to confirmation

Timezone Remember this is an online event, meant for all of the world! Yes, the chosen times seem quite Europe-centric (but they are mostly a function of the times the talk submitters requested). Talks are 11:00 18:00UTC, which means, 06:00 13:00 Mexico (GMT-5), 20:00 03:00 Japan (GMT+9), 04:00 11:00 Western Canada/USA/Mexico (GMT-7) and the rest of the world, somewhere in between. (No, this was clearly not optimized for our dear usual beer team. Sorry! I guess we need you to be fully awake at beertime!)

[update] Connecting! Of course, I didn t make it clear at first how to connect to the Online miniDebConf, silly me!

• The video streams are available at: https://video.debconf.org/
• Suggested: tune in to the #minidebconf-online IRC channel in OFTC.

That should be it. Hope to see you there! (Stay home, stay safe )

Introduction pristine-tar is a tool that is present in the workflow of a lot of Debian people. I adopted it last year after it has been orphaned by its creator Joey Hess. A little after that Tomasz Buchert joined me and we are now a functional two-person team. pristine-tar goals are to import the content of a pristine upstream tarball into a VCS repository, and being able to later reconstruct that exact same tarball, bit by bit, based on the contents in the VCS, so we don t have to store a full copy of that tarball. This is done by storing a binary delta files which can be used to reconstruct the original tarball from a tarball produced with the contents of the VCS. Ultimately, we want to make sure that the tarball that is uploaded to Debian is exactly the same as the one that has been downloaded from upstream, without having to keep a full copy of it around if all of its contents is already extracted in the VCS anyway. The current state of the art, and perspectives for the future pristine-tar solves a wicked problem, because our ability to reconstruct the original tarball is affected by changes in the behavior of tar and of all of the compression tools (gzip, bzip2, xz) and by what exact options were used when creating the original tarballs. Because of this, pristine-tar currently has a few embedded copies of old versions of compressors to be able to reconstruct tarballs produced by them, and also rely on a ever-evolving patch to tar that is been carried in Debian for a while. So basically keeping pristine-tar working is a game of Whac-A-Mole. Joey provided a good summary of the situation when he orphaned pristine-tar. Going forward, we may need to rely on other ways of ensuring integrity of upstream source code. That could take the form of signed git tags, signed uncompressed tarballs (so that the compression doesn t matter), or maybe even a different system for storing actual tarballs. Debian bug #871806 contains an interesting discussion on this topic. Recent improvements Even if keeping pristine-tar useful in the long term will be hard, too much of Debian work currently relies on it, so we can t just abandon it. Instead, we keep figuring out ways to improve. And I have good news: pristine-tar has recently received updates that improve the situation quite a bit. In order to be able to understand how better we are getting at it, I created a "visualization of the regression test suite results. With the help of data from there, let s look at the improvements made since pristine-tar 1.38, which was the version included in stretch. pristine-tar 1.39: xdelta3 by default. This was the first release made after the stretch release, and made xdelta3 the default delta generator for newly-imported tarballs. Existing tarballs with deltas produced by xdelta are still supported, this only affects new imports. The support for having multiple delta generator was written by Tomasz, and was already there since 1.35, but we decided to only flip the switch after using xdelta3 was supported in a stable release. pristine-tar 1.40: improved compression heuristics pristine-tar uses a few heuristics to produce the smaller delta possible, and this includes trying different compression options. In the release Tomasz included a contribution by Lennart Sorensen to also try the --gnu, which gretly improved the support for rsyncable gzip compressed files. We can see an example of the type of improvement we got in the regression test suite data for delta sizes for faad2_2.6.1.orig.tar.gz: pristine-tar 1.41: support for signatures This release saw the addition of support for storage and retrieval of upstream signatures, contributed by Chris Lamb. pristine-tar 1.42: optionally recompressing tarballs I had this idea and wanted to try it out: most of our problems reproducing tarballs come from tarballs produced with old compressors, or from changes in compressor behavior, or from uncommon compression options being used. What if we could just recompress the tarballs before importing then? Yes, this kind of breaks the pristine bit of the whole business, but on the other hand, 1) the contents of the tarball are not affected, and 2) even if the initial tarball is not bit by bit the same that upstream release, at least future uploads of that same upstream version with Debian revisions can be regenerated just fine. In some cases, as the case for the test tarball util-linux_2.30.1.orig.tar.xz, recompressing is what makes it possible to reproduce the tarball (and thus import it with pristine-tar) possible at all: In other cases, if the current heuristics can t produce a reasonably small delta, recompressing makes a huge difference. It s the case for mumble_1.1.8.orig.tar.gz: Recompressing is not enabled by default, and can be enabled by passing the --recompress option. If you are using pristine-tar via a wrapper tool like gbp-buildpackage, you can use the $PRISTINE_TAR environment variable to set options that will affect any pristine-tar invocations. Also, even if you enable recompression, pristine-tar will only try it if the delta generations fails completely, of if the delta produced from the original tarball is too large. You can control what too large means by using the --recompress-threshold-bytes and --recompress-threshold-percent options. See the pristine-tar(1) manual page for details.

I m back from Debconf17. I gave a talk entitled Patterns for Testing Debian Packages , in which I presented a collection of 7 patterns I documented while pushing the Debian Continuous Integration project, and were published in a 2016 paper. Video recording and a copy of the slides are available. I also hosted the ci/autopkgtest BoF session, in which we discussed issues around the usage of autopkgtest within Debian, the CI system, etc. Video recording is available. Kudos for the Debconf video team for making the recordings available so quickly!

Since 2003, the Debian project has been running a server called Alioth to host source code version control systems. The server will hit the end of life of the Debian LTS release (Wheezy) next year; that deadline raised some questions regarding the plans for the server over the coming years. Naturally, that led to a discussion regarding possible replacements. In response, the current Alioth maintainer, Alexander Wirt, announced a sprint to migrate to pagure, a free-software "Git-centered forge" written in Python for the Fedora project, which LWN covered last year. Alioth currently runs FusionForge, previously known as GForge, which is the free-software fork of the SourceForge code base when that service closed its source in 2001. Alioth hosts source code repositories, mainly Git and Subversion (SVN) and, like other "forge" sites, also offers forums, issue trackers, and mailing list services. While other alternatives are still being evaluated, a consensus has emerged on a migration plan from FusionForage to a more modern and minimal platform based on pagure.

Why not GitLab? While this may come as a surprise to some who would expect Debian to use the more popular GitLab project, the discussion and decision actually took place a while back. During a lengthy debate last year, Debian contributors discussed the relative merits of different code-hosting platforms, following the initiative of Debian Developer "Pirate" Praveen Arimbrathodiyil to package GitLab for Debian. At that time, Praveen also got a public GitLab instance running for Debian (gitlab.debian.net), which was sponsored by GitLab B.V. the commercial entity behind the GitLab project. The sponsorship was originally offered in 2015 by the GitLab CEO, presumably to counter a possible move to GitHub, as there was a discussion about creating a GitHub Organization for Debian at the time. The deployment of a Debian-specific GitLab instance then raised the question of the overlap with the already existing git.debian.org service, which is backed by Alioth's FusionForge deployment. It then seemed natural that the new GitLab instance would replace Alioth. But when Praveen directly proposed to move to GitLab, Wirt stepped in and explained that a migration plan was already in progress. The plan then was to migrate to a simpler gitolite-based setup, a decision that was apparently made in corridor discussions surrounding the Alioth Git replacement BoF held during Debconf 2015. The first objection raised by Wirt against GitLab was its "huge number of dependencies". Another issue Wirt identified was the "open core / enterprise model", preferring a "real open source system", an opinion which seems shared by other participants on the mailing list. Wirt backed his concerns with an hypothetical example:

Debian needs feature X but it is already in the enterprise version. We make a patch and, for commercial reasons, it never gets merged (they already sell it in the enterprise version). Which means we will have to fork the software and keep those patches forever. Been there done that. For me, that isn't acceptable.

This concern was further deepened when GitLab's Director of Strategic Partnerships, Eliran Mesika, explained the company's stewardship policy that explains how GitLab decides which features end up in the proprietary version. Praveen pointed out that:

[...] basically it boils down to features that they consider important for organizations with less than 100 developers may get accepted. I see that as a red flag for a big community like debian.

Since there are over 600 Debian Developers, the community seems to fall within the needs of "enterprise" users. The features the Debian community may need are, by definition, appropriate only to the "Enterprise Edition" (GitLab EE), the non-free version, and are therefore unlikely to end up in the "Community Edition" (GitLab CE), the free-software version. Interestingly, Mesika asked for clarification on which features were missing, explaining that GitLab is actually open to adding features to GitLab CE. The response from Debian Developer Holger Levsen was categorical: "It's not about a specific patch. Free GitLab and we can talk again." But beyond the practical and ethical concerns, some specific features Debian needs are currently only in GitLab EE. For example, debian.org systems use LDAP for authentication, which would obviously be useful in a GitLab deployment; GitLab CE supports basic LDAP authentication, but advanced features, like group or SSH-key synchronization, are only available in GitLab EE. Wirt also expressed concern about the Contributor License Agreement that GitLab B.V. requires contributors to sign when they send patches, which forces users to allow the release of their code under a non-free license. The debate then went on going through a exhaustive inventory of different free-software alternatives:

• GitLab, a Ruby-based GitHub replacement, dual-licensed MIT/Commercial
• Gogs, Go, MIT
• Gitblit, Java, Apache-licensed
• Kallithea, in Python, also supports Mercurial, GPLv3
• and finally, pagure, also written Python, GPLv2

A feature comparison between each project was created in the Debian wiki as well. In the end, however, Praveen gave up on replacing Alioth with GitLab because of the controversy and moved on to support the pagure migration, which resolved the discussion in July 2016. More recently, Wirt admitted in an IRC conversation that "on the technical side I like GitLab a lot more than pagure" and that "as a user, GitLab is much nicer than pagure and it has those nice CI [continuous integration] features". However, as he explained in his blog "GitLab is Opencore, [and] that it is not entirely opensource. I don't think we should use software licensed under such a model for one of our core services" which leaves pagure as the only stable candidate. Other candidates were excluded on technical grounds, according to Wirt: Gogs "doesn't scale well" and a quick security check didn't yield satisfactory results; "Gitblit is Java" and Kallithea doesn't have support for accessing repositories over SSH (although there is a pending pull request to add the feature). In an email interview, Sid Sijbrandij, CEO of GitLab, did say that "we want to make sure that our open source edition can be used by open source projects". He gave examples of features liberated following requests by the community, such as branded login pages for the VLC project and GitLab Pages after popular demand. He stressed that "There are no artificial limits in our open source edition and some organizations use it with more than 20.000 users." So if the concern of the Debian community is that features may be missing from GitLab CE, there is definitely an opening from GitLab to add those features. If, however, the concern is purely ethical, it's hard to see how an agreement could be reached. As Sijbrandij put it:

On the mailinglist it seemed that some Debian maintainers do not agree with our open core business model and demand that there is no proprietary version. We respect that position but we don't think we can compete with the purely proprietary software like GitHub with this model.

Working toward a pagure migration The issue of Alioth maintenance came up again last month when Boyuan Yang asked what would happen to Alioth when support for Debian LTS (Wheezy) ends next year. Wirt brought up the pagure migration proposal and the community tried to make a plan for the migration. One of the issues raised was the question of the non-Git repositories hosted on Alioth, as pagure, like GitLab, only supports Git. Indeed, Ben Hutchings calculated that while 90% (\~19,000) of the repositories currently on Alioth are Git, there are 2,400 SVN repositories and a handful of Mercurial, Bazaar (bzr), Darcs, Arch, and even CVS repositories. As part of an informal survey, however, most packaging teams explained they either had already migrated away from SVN to Git or were in the process of doing so. The largest CVS user, the web site team, also explained it was progressively migrating to Git. Mattia Rizzolo then proposed that older repository services like SVN could continue running even if FusionForge goes down, as FusionForge is, after all, just a web interface to manage those back-end services. Repository creation would be disabled, but older repositories would stay operational until they migrate to Git. This would, effectively, mean the end of non-Git repository support for new projects in the Debian community, at least officially. Another issue is the creation of a Debian package for pagure. Ironically, while Praveen and other Debian maintainers have been working for 5 years to package GitLab for Debian, pagure isn't packaged yet. Antonio Terceiro, another Debian Developer, explained this isn't actually a large problem for debian.org services: "note that DSA [Debian System Administrator team] does not need/want the service software itself packaged, only its dependencies". Indeed, for Debian-specific code bases like ci.debian.net or tracker.debian.org, it may not make sense to have the overhead of maintaining Debian packages since those tools have limited use outside of the Debian project directly. While Debian derivatives and other distributions could reuse them, what usually happens is that other distributions roll their own software, like Ubuntu did with the Launchpad project. Still, Paul Wise, a member of the DSA team, reasoned that it was better, in the long term, to have Debian packages for debian.org services:

Personally I'm leaning towards the feeling that all configuration, code and dependencies for Debian services should be packaged and subjected to the usual Debian QA activities but I acknowledge that the current archive setup (testing migration plus backporting etc) doesn't necessarily make this easy.

Wise did say that "DSA doesn't have any hard rules/policy written down, just evaluation on a case-by-case basis" which probably means that pagure packaging will not be a blocker for deployment. The last pending issue is the question of the mailing lists hosted on Alioth, as pagure doesn't offer mailing list management (nor does GitLab). In fact, there are three different mailing list services for the Debian project:

• the main service, lists.debian.org, running ~~Mailman 2~~ SmartList and managed by hand
• the Alioth service, lists.alioth.debian.org, running Mailman 2 and managed by FusionForge
• the Debconf service, lists.debconf.org, also running Mailman 2

Wirt, with his "list-master hat" on, explained that the main mailing list service is "not really suited as a self-service" and expressed concern at the idea of migrating the large number mailing lists hosted on Alioth. Indeed, there are around 1,400 lists on Alioth while the main service has a set of 300 lists selected by the list masters. No solution for those mailing lists was found at the time of this writing. In the end, it seems like the Debian project has chosen pagure, the simpler, less featureful, but also less controversial, solution and will use the same hosting software as their fellow Linux distribution, Fedora. Wirt is also considering using FreeIPA for account management on top of pagure. The plan is to migrate away from FusionForge one bit at a time, and pagure is the solution for the first step: the Git repositories. Lists, other repositories, and additional features of FusionForge will be dealt with later on, but Wirt expects a plan to come out of the upcoming sprint. It will also be interesting to see how the interoperability promises of pagure will play out in the Debian world. Even though the federation features of pagure are still at the early stages, one can already clone issues and pull requests as Git repositories, which allows for a crude federation mechanism. In any case, given the long history and the wide variety of workflows in the Debian project, it is unlikely that a single tool will solve all problems. Alioth itself has significant overlap with other Debian services; not only does it handle mailing lists and forums, but it also has its own issue tracker that overlaps with the Debian bug tracking system (BTS). This is just the way things are in Debian: it is an old project with lots of moving part. As Jonathan Dowland put it: "The nature of the project is loosely-coupled, some redundancy, lots of legacy cruft, and sadly more than one way to do it." Hopefully, pagure will not become part of that "legacy redundant cruft". But at this point, the focus is on keeping the services running in a simpler, more maintainable way. The discussions between Debian and GitLab are still going on as we speak, but given how controversial the "open core" model used by GitLab is for the Debian community, pagure does seem like a more logical alternative.

Note: this article first appeared in the Linux Weekly News.

When I started debci for Debian CI, I went for the simplest thing that could possibly work. One of the design decisions was to use the filesystem directly for file storage. A large part of the Debian CI data is log files and test artifacts (which are just files), and using the filesystem directly for storage makes it a lot easier to handle it. The rest of the data which is structured (test history and status of packages) is stored as JSON files. Another nice benefit of using the filesystem like this is that I get a sort of REST API for free by just exposing the file storage to the web. For example, getting the latest test status of debci itself on unstable/amd64 is as easy as:

$ curl https://ci.debian.net/data/packages/unstable/amd64/d/debci/latest.json
 
  "run_id": "20170528_173652",
  "package": "debci",
  "version": "1.5.1",
  "date": "2017-05-28 17:43:05",
  "status": "pass",
  "blame": [],
  "previous_status": "pass",
  "duration_seconds": "373",
  "duration_human": "0h 6m 13s",
  "message": "Tests passed, but at least one test skipped",
  "last_pass_version": "1.5.1",
  "last_pass_date": "2017-05-28 17:43:05"
 

Now, nothing in life is without compromises. One big disadvantage of the way debci stored its data is that there were a lot of files, which ends up using a large number of inodes in the filesystem. The current Debian CI master has more than 10 million inodes in its filesystem, and almost all of them were being used. This is clearly unsustainable. You will notice that I said stored, because as of version 1.6, debci now implements a data retention policy: log files and test artifacts will now only be kept for a configurable amount of days (default: 180). So there you have it: effective immediately, Debian CI will not provide logs and test artifacts older than 180 days. If you are reporting bugs based on logs from Debian CI, please don t hotlink the log files. Instead, make sure you download the logs in question and attach them to the bug report, because in 6 months they will be gone.

At the and of 2016 I had the pleasure to attend the 11th Latin American Conference on Pattern Languages of Programs, a.k.a SugarLoaf PLoP. PLoP is a series of conferences on Patterns (as in Design Patterns ), a subject that I appreciate a lot. Each of the PLoP conferences but the original main big conference has a funny name. SugarLoaf PLoP is called that way because its very first edition was held in Rio de Janeiro, so the organizers named it after a very famous mountain in Rio. The name stuck even though a long time has passed since it was held in Rio for the last time. 2016 was actually the first time SugarLoaf PLoP was held outside of Brazil, finally justifying the Latin American part of its name. I was presenting a paper I wrote on patterns for testing Debian packages. The Debian project funded my travel expenses through the generous donations of its supporters. PLoP s are very fun conferences with a relaxed atmosphere, and is amazing how many smart (and interesting!) people gather together for them. My paper is titled Patterns for Writing As-Installed Tests for Debian Packages , and has the following abstract:

Large software ecosystems, such as GNU/Linux distributions, demand a large amount of effort to make sure all of its components work correctly invidually, and also integrate correctly with each other to form a coherent system. Automated Quality Assurance techniques can prevent issues from reaching end users. This paper presents a pattern language originated in the Debian project for automated software testing in production-like environments. Such environments are closer in similarity to the environment where software will be actually deployed and used, as opposed to the development environment under which developers and regular Continuous Integration mechanisms usually test software products. The pattern language covers the handling of issues arising from the difference between development and production-like environments, as well as solutions for writing new, exclusive tests for as-installed functional tests. Even though the patterns are documented here in the context of the Debian project, they can also be generalized to other contexts.

In practical terms, the paper documents a set of patterns I have noticed in the last few years, when I have been pushing the Debian Continous Integration project. It should be an interesting read for people interested in the testing of Debian packages in their installed form, as done with autopkgtest. It should also be useful for people from other distributions interested in the subject, as the issues are not really Debian-specific. I have recently finished the final version of the paper, which should be published in the ACM Digital Library at any point now. You can download a copy of the paper in PDF. Source is also available, if you are into markdown, LaTeX, makefiles and this sort of thing. If everything goes according to plan, I should be presenting a talk on this at the next Debconf in Montreal.

What happened in the Reproducible Builds effort between Sunday September 25 and Saturday October 1 2016: Statistics For the first time, we reached 91% reproducible packages in our testing framework on testing/amd64 using a determistic build path. (This is what we recommend to make packages in Stretch reproducible.) For unstable/amd64, where we additionally test for reproducibility across different build paths we are at almost 76% again. IRC meetings We have a poll to set a time for a new regular IRC meeting. If you would like to attend, please input your available times and we will try to accommodate for you. There was a trial IRC meeting on Friday, 2016-09-31 1800 UTC. Unfortunately, we did not activate meetbot. Despite this participants consider the meeting a success as several topics where discussed (eg changes to IRC notifications of tests.r-b.o) and the meeting stayed within one our length. Upcoming events Reproduce and Verify Filesystems - Vincent Batts, Red Hat - Berlin (Germany), 5th October, 14:30 - 15:20 @ LinuxCon + ContainerCon Europe 2016. From Reproducible Debian builds to Reproducible OpenWrt, LEDE & coreboot - Holger "h01ger" Levsen and Alexander "lynxis" Couzens - Berlin (Germany), 13th October, 11:00 - 11:25 @ OpenWrt Summit 2016. Introduction to Reproducible Builds - Vagrant Cascadian will be presenting at the SeaGL.org Conference In Seattle (USA), November 11th-12th, 2016. Previous events GHC Determinism - Bartosz Nitka, Facebook - Nara (Japan), 24th September, ICPF 2016. Toolchain development and fixes Michael Meskes uploaded bsdmainutils/9.0.11 to unstable with a fix for #830259 based on Reiner Herrmann's patch. This fixed locale_dependent_symbol_order_by_lorder issue in the affected packages (freebsd-libs, mmh). devscripts/2.16.8 was uploaded to unstable. It includes a debrepro script by Antonio Terceiro which is similar in purpose to reprotest but more lightweight; specific to Debian packages and without support for virtual servers or configurable variations. Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible in our testing framework after being fixed:

• ara/1.0.32 by Chris Lamb, original patch by Chris Lamb.
• fracplanet/0.4.0-5 by Chris Lamb, original patch by Reiner Herrman.
• gnarwl/3.6.dfsg-8 by Bernhard Schmidt, original patch by Chris Lamb.
• kgb-bot/1.34-1 by gregor herrmann, original patch by gregor herrmann.
• survex/1.2.29-1 by Olly Betts.
• zpaq/1.10-3 by Chris Lamb, original patch by Reiner Herrman.
• fig2dev/1:3.2.6-3 by Roland Rosenfeld.
• luxio/11-1 by Daniel Silverstone.
• monkeysign/2.1.1 by Antoine Beaupr , original patch by Daniel Kahn Gillmor.
• openarena-085-data/0.8.5split-9 by Simon McVittie.
• openarena-088-data/0.8.8-7 by Simon McVittie.
• openarena-data/0.8.5split-9 by Simon McVittie.
• rc/1.7.4-1 by Reiner Herrmann, original patch by Chris Lamb.

The following updated packages appear to be reproducible now for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

• gkrellm/2.3.8-1 by Sandro Tosi
• glassfish/1:2.1.1-b31g+dfsg1-4 by Emmanuel Bourg

Some uploads have addressed some reproducibility issues, but not all of them:

• apache2/2.4.23-5 by Rapha l Hertzog
• freeradius/3.0.11+dfsg-1 by Michael Stapelberg
• libnss-ldap/265-4 by Chris Lamb
• lift/2.5.0-1 by Nicolas Delvaux
• linux/4.8~rc8-1~exp1 by Ben Hutchings
• nose2/0.6.5-2 by Barry Warsaw
• postgresql-9.6/9.6.0-1 Christoph Berg
• strace/4.13-0.1 by Nicolas Braud-Santoni
• yersinia/0.7.3-3 by No l K the

Patches submitted that have not made their way to the archive yet:

• #838888 filed against dh-haskell by Chris Lamb.
• #838971 filed against slang2 by Chris Lamb.
• #839587 filed against sympa by Chris Lamb.
• #839181 filed against transmission-remote-gtk by Chris Lamb.
• #838829 filed against vala by Sebastian Reichel.
• #838970 filed against webkit2pdf by Chris Lamb.
• #831569 filed against websockets by Chris Lamb.
• #839347 filed against xml-core by Lucas Nussbaum.
• #839526 filed against xml-core by Adrian Bunk.

Reviews of unreproducible packages 77 package reviews have been added, 178 have been updated and 80 have been removed in this week, adding to our knowledge about identified issues. 6 issue types have been updated:

• Added clilibs_line_order, records_build_flags and hevea_captures_build_path.
• Removed locale_dependent_symbol_order_by_lorder, fixed in bsdmainutils/9.0.11.
• Updated diffoscope_runs_forever, captures_build_path.

Weekly QA work As part of reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (3)
• Chris Lamb (12)
• Lucas Nussbaum (3)
• Sebastian Reichel (1)

diffoscope development A new version of diffoscope 61 was uploaded to unstable by Chris Lamb. It included contributions from:

• Ximin Luo:
  • Improve the CLI --help text and add an --output-empty option.
• Chris Lamb:
  • Add a progress bar and show it if stdout is a TTY. You can read more about it here. It can also be read by higher-level programs via the --status-fd CLI option.
• Maria Glukhova:
  • Behaviour improvements in the case of OS-level errors.
• Mattia Rizzolo:
  • Testing and packaging improvements.

Post-release there were further contributions from:

• Chris Lamb:
  • Code architecture improvements.
• Maria Glukhova:
  • Testing improvements.

reprotest development A new version of reprotest 0.3.2 was uploaded to unstable by Ximin Luo. It included contributions from:

• Ximin Luo:
  • Add a --diffoscope-arg CLI option to pass extra args to diffoscope.

Post-release there were further contributions from:

• Chris Lamb:
  • Code quality improvements.

tests.reproducible-builds.org

• Hans-Christoph Steiner continued work on setting up reproducible tests for F-Droid.
• Holger cleaned up the script creating the page showing breakages, so that it now also cleans up some of the breakage it finds.
• IRC notifications about diffoscope crashes and artifacts available for investigations have been dropped; instead the breakages page has a permanent pointer. (h01ger)
• IRC notifications from the automatic package scheduler and status changes for packages have been moved -- as a temporary trial -- to #debian-reproducible-changes on irc.oftc.net (Mattia).

Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

debci 1.4 was released just a few days ago. Among general improvements, I would like to highlight:

• pretty much every place in the web UI that mentions a PASS or a FAIL also displays the tested package version. This was suggested to me on IRC by Holger
• I also tried to workaround an instability when setting up the LXC containers used for the tests, where the test bed process setup would finish without failure even though some steps in the middle of it failed. This caused the very final step for the debci-specific setup to fail, so there was no debci user inside the container, which caused tests to fail because that user was missing. Before that was fixed I was always keeping an eye on this issue, fixing the issue by hand, and re-triggering the affected packages by hand, so as far I can tell there is no package whose status has been permanently affected by this.
• Last, but not least, this release brings an interesting contribution by Gordon Ball, which is keeping track of different failure states. debci will now let you know whether a currently failing package has always failed, has passed in a previous version, or if the same version that is currently failing has previously passed.

ci.debian.net has been upgraded to debci 1.4 just after that. At the same time I have also upgraded autodep8 and autopkgtest to their latest versions, available in jessie-backports. This means that it is now safe for Debian packages to assume the changes in autopkgtest 4.0 are available, in special the $AUTOPKGTEST_* environment variables. In other news, for several weeks there were had issues with tests not being scheduled when they should have. I was just assuming that the issue was due to the existing test scheduler, debci-batch, being broken. Today I was working on a new implementation that is going to be a lot faster, I started to hit a similar issue on my local tests, and finally realized what was wrong. The fact is that debci-batch stores the timestamp of the last time a package has been scheduled to run, and it there are no test result after that timestamp, it assumes the package is still in the queue to be tested, and does not schedule it again. It turns out that a few weeks ago, during maintainance work, I had cleared the queue, discarding all jobs that were there, but forgot to reset those timestamps, so when debci-batch came around again, it checked the timestamp of the last request and did not make new requests because there was no test result after that timestamp! I cleared all those timestamps, and the system should now go back to normal. That is it for now. I you want to contribute to the Debian CI project and want to get in touch, you can pop up on the #debci channel on the OFTC IRC network, or mail the autopkgtest-devel mailing list.

What happened in the Reproducible Builds effort between Sunday August 28 and Saturday September 3 2016: Media coverage Antonio Terceiro blogged about testing build reprodubility with debrepro . GSoC and Outreachy updates The next round is being planned now: see their page with a timeline and participating organizations listing. Maybe you want to participate this time? Then please reach out to us as soon as possible! Packages reviewed and fixed, and bugs filed The following packages have addressed reproducibility issues in other packages:

• ruby-ronn/0.7.3-5 by Antonio Terceiro, original patch by Chris Lamb.

The following updated packages have become reproducible in our current test setup after being fixed:

• check-all-the-things/2016.09.03 by Paul Wise, original patch by Chris Lamb.
• e2fsprogs/1.43.2-2 by Theodore Y. Ts'o.
• gnupg1/1.4.21-1 by Daniel Kahn Gillmor, #834755 by Chris Lamb.
• gnupg2/2.1.15-2 by Daniel Kahn Gillmor.
• inform6-compiler/6.33-2 by Ben Finney.
• libhat-trie/0.0~git25f9e946-2 by Sascha Steinbiss.
• mrd6/0.9.6-13 by Thomas Preud'homme, original patch by Reiner Herrmann.
• safecat/1.13-3 by Teemu Hukkanen, original patch by Reiner Herrmann.
• shibboleth-sp2/2.6.0+dfsg1-1 by Ferenc W gner.
• sweethome3d-furniture-editor/1.15-2 by Markus Koschany, original patch by Reiner Herrmann.
• traceroute/1:2.1.0-2 by Laszlo Boszormenyi, original patch by Reiner Herrmann.
• wise/2.4.1-19 by Sascha Steinbiss.
• xmltooling/1.6.0-2 by Ferenc W gner.

The following updated packages appear to be reproducible now, for reasons we were not able to figure out yet. (Relevant changelogs did not mention reproducible builds.)

• comitup/0.5-1 by David Steele.
• dita-ot/1.5.3-2 by Mathieu Malaterre.
• dput/0.10.2 by Ben Finney.
• gnome-settings-daemon/3.21.90-2 by Michael Biebl.
• libkf5kipi/4:16.08.0-1 by Pino Toscano.
• libmpc/2:0.1~r475-2 by James Cowgill.
• taurus/4.0.1+dfsg-1 by Picca Fr d ric-Emmanuel.
• tcpwatch-httpproxy/1.3.1-2 by Toni Mueller.

The following 4 packages were not changed, but have become reproducible due to changes in their build-dependencies:

• djangorestframework
• roarplaylistd
• zope-maildrophost
• zope-replacesupport

Some uploads have addressed some reproducibility issues, but not all of them:

• amanda/1:3.3.9-1 by Jose M Calhariz.
• dispcalgui/3.1.6.0-2 by Christian Marillat, original patch by Chris Lamb.
• fastqtl/2.184+dfsg-4 by Dylan A ssi.
• grass/7.0.5~rc1-1~exp1 by Bas Couwenberg.
• kdevplatform/5.0-1 by Pino Toscano, original patch by Scarlett Clark.
• leveldb/1.19-1 by Alessio Treglia.
• libdevel-cover-perl/1.23-2 by gregor herrmann, original patch by Chris Lamb.
• linux/4.7.2-1 by Ben Hutchings, #830268 by Reiner Herrmann.
• lmms/1.1.3-4 by Javier Serrano.
• mayavi2/4.4.3-2.2 by Anton Gladky.
• opensaml2/2.6.0-2 by Ferenc W gner.
• perl/5.22.2-4 by Dominic Hargreaves, original patch and original patch by Chris Lamb.
• radare2/0.10.5+dfsg-1 by Sebastian Reichel, original patch by Chris Lamb.
• splint/3.1.2.dfsg1-3 by Rapha l Hertzog, original patch by Chris Lamb.

Patches submitted that have not made their way to the archive yet:

• #835673 filed against dacs by Chris Lamb.
• #835805 filed against dh-python by Chris Lamb.
• #835985 filed against nmh by Chris Lamb.
• #836609 filed against nostalgy by Chris Lamb.
• #835807 filed against pygtksourceview by Chris Lamb.
• #836004 filed against python-gflags by Chris Lamb.
• #835816 filed against rt-extension-customfieldsonupdate by Chris Lamb.
• #835834 filed against ruby-compass by Chris Lamb.
• #836605 filed against torque by Chris Lamb.
• #833176 filed against trafficserver by Reiner Herrmann.

Reviews of unreproducible packages 706 package reviews have been added, 22 have been updated and 16 have been removed in this week, adding to our knowledge about identified issues. 5 issue types have been added:

• timestamps_in_documentation_generated_by_ocamldoc
• timestamp_in_enc_files_added_by_texlive_fontinst
• cython_captures_build_path
• timestamps_in_perllocal_pod_manpages_generated_by_perl_extutils_mm_unix
• uname_output_in_python_debugging_symbols_caused_by_sysconfig_getplatform

1 issue type has been updated:

• timestamp_in_enc_files_added_by_texlive_fontinst

Weekly QA work FTBFS bugs have been reported by:

• Chris Lamb (8)
• Lucas Nussbaum (3)

diffoscope development diffoscope development on the next version (60) continued in git, taking in contributions from:

• Mattia Rizzolo:
  • Better and more thorough testing
  • Improvements to packaging
  • Improvements to the ppu comparator

strip-nondeterminism development Mattia Rizzolo uploaded strip-nondeterminism 0.023-2~bpo8+1 to jessie-backports. A new version of strip-nondeterminism 0.024-1 was uploaded to unstable by Chris Lamb. It included contributions from:

• Chris Lamb:
  • Improve code quality of zip, jar, ar, png processors
• AYANOKOUZI, Ryuunosuke:
  • Preserve file attribute information of target file (#836075)

Holger added jobs on jenkins.debian.net to run testsuites on every commit. There is one job for the master branch and one for the other branches. disorderfs development Holger added jobs on jenkins.debian.net to run testsuites on every commit. There is one job for the master branch and one for the other branches. tests.reproducible-builds.org Debian: We now vary the GECOS records of the two build users. Thanks to Paul Wise for providing the patch. Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

Earlier today I was handling a reproducibility bug and decided I had to try a reproducibility test by myself. I tried reprotest, but I was being hit by a disorderfs issue and I was not sure whether the problem was with reprotest or not (at this point I cannot reproduce that anymore). So I decided to hack a simple script to that, and it works. I even included it in devscripts after writing a manpage. Of course reprotest is more complete, extensible, and supports arbitrary virtualization backends for doing the more dangerous/destructive variations (such as changing the hostname and other things that require root) but for quick tests debrepro does the job. Usage examples:

$ debrepro                                 # builds current directory
$ debrepro /path/to/sourcepackage          # builds package there
$ gbp-buildpackage --git-builder=debrepro  # can be used with vcs wrappers as well

debrepro will do two builds with a few variations between them, including $USER, $PATH, timezone, locale, umask, current time, and will even build under disorderfs if available. Build path variation is also performed because by definition the builds are done in different directories. If diffoscope is installed, it will be used for deep comparison of non-matching binaries. If you are interested and don t want to build devscripts from source or wait for the next release, you can just grab the script, save it as debrepro somewhere on your $PATH and make it executable.

As of yesterday, I am the new maintainer of pristine-tar. As it is the case for most of Joey Hess creations, it is an extremely useful tool, and used in a very large number of Debian packages which are maintained in git. My first upload was most of a terrain recognition nature: I did some housekeeping tasks, such as making the build idempotent and making sure all binaries are built with security hardening flags, and wrote a few automated test cases to serve as build-time and run-time regression test suite. No functional changes have been made. As Joey explained when he orphaned it, there are a few technical challenges involved in making sure pristine-tar stays useful in the future. Although I did read some of the code, I am not particularly familiar with the internals yet, and will be more than happy to get co-maintainers. If you are interested, please get in touch. The source git repository is right there.