Search Results: "spa"

22 May 2020

Steve Kemp: Updated my linux-security-modules for the Linux kernel

Almost three years ago I wrote my first linux-security-module, inspired by a comment I read on LWN I did a little more learning/experimentation and actually produced a somewhat useful LSM, which allows you to restrict command-execution via the use of a user-space helper: Because the detection is done in userspace writing your own custom rules is both safe and easy. No need to touch the kernel any further! Yesterday I rebased all the modules so that they work against the latest stable kernel 5.4.22 in #7.
The last time I'd touched them they were built against 5.1, which was itself a big jump forwards from the 4.16.7 version I'd initially used.
Finally I updated the can-exec module to make it gated, which means you can turn it on, but not turn it off without a reboot. That was an obvious omission from the initial implementation #11. Anyway updated code is available here: I'd kinda like to write more similar things, but I lack inspiration.

18 May 2020

Arturo Borrero Gonz lez: A better Toolforge: upgrading the Kubernetes cluster

Logos This post was originally published in the Wikimedia Tech blog, and is authored by Arturo Borrero Gonzalez and Brooke Storm. One of the most successful and important products provided by the Wikimedia Cloud Services team at the Wikimedia Foundation is Toolforge. Toolforge is a platform that allows users and developers to run and use a variety of applications that help the Wikimedia movement and mission from the technical point of view in general. Toolforge is a hosting service commonly known in the industry as a Platform as a Service (PaaS). Toolforge is powered by two different backend engines, Kubernetes and GridEngine. This article focuses on how we made a better Toolforge by integrating a newer version of Kubernetes and, along with it, some more modern workflows. The starting point in this story is 2018. Yes, two years ago! We identified that we could do better with our Kubernetes deployment in Toolforge. We were using a very old version, v1.4. Using an old version of any software has more or less the same consequences everywhere: you lack security improvements and some modern key features. Once it was clear that we wanted to upgrade our Kubernetes cluster, both the engineering work and the endless chain of challenges started. It turns out that Kubernetes is a complex and modern technology, which adds some extra abstraction layers to add flexibility and some intelligence to a very old systems engineering need: hosting and running a variety of applications. Our first challenge was to understand what our use case for a modern Kubernetes was. We were particularly interested in some key features: Soon enough we faced another Kubernetes native challenge: the documentation. For a newcomer, learning and understanding how to adapt Kubernetes to a given use case can be really challenging. We identified some baffling patterns in the docs. For example, different documentation pages would assume you were using different Kubernetes deployments (Minikube vs kubeadm vs a hosted service). We are running Kubernetes like you would on bare-metal (well, in CloudVPS virtual machines), and some documents directly referred to ours as a corner case. During late 2018 and early 2019, we started brainstorming and prototyping. We wanted our cluster to be reproducible and easily rebuildable, and in the Technology Department at the Wikimedia Foundation, we rely on Puppet for that. One of the first things to decide was how to deploy and build the cluster while integrating with Puppet. This is not as simple as it seems because Kubernetes itself is a collection of reconciliation loops, just like Puppet is. So we had to decide what to put directly in Kubernetes and what to control and make visible through Puppet. We decided to stick with kubeadm as the deployment method, as it seems to be the more upstream-standardized tool for the task. We had to make some interesting decisions by trial and error, like where to run the required etcd servers, what the kubeadm init file would look like, how to proxy and load-balance the API on our bare-metal deployment, what network overlay to choose, etc. If you take a look at our public notes, you can get a glimpse of the number of decisions we had to make. Our Kubernetes wasn t going to be a generic cluster, we needed a Toolforge Kubernetes service. This means we don t use some of the components, and also, we add some additional pieces and configurations to it. By the second half of 2019, we were working full-speed on the new Kubernetes cluster. We already had an idea of what we wanted and how to do it. There were a couple of important topics for discussions, for example: We will describe in detail the final state of those pieces in another blog post, but each of the topics required several hours of engineering time, research, tests, and meetings before reaching a point in which we were comfortable with moving forward. By the end of 2019 and early 2020, we felt like all the pieces were in place, and we started thinking about how to migrate the users, the workloads, from the old cluster to the new one. This migration plan mostly materialized in a Wikitech page which contains concrete information for our users and the community. The interaction with the community was a key success element. Thanks to our vibrant and involved users, we had several early adopters and beta testers that helped us identify early flaws in our designs. The feedback they provided was very valuable for us. Some folks helped solve technical problems, helped with the migration plan or even helped make some design decisions. Worth noting that some of the changes that were presented to our users were not easy to handle for them, like new quotas and usage limits. Introducing new workflows and deprecating old ones is always a risky operation. Even though the migration procedure from the old cluster to the new one was fairly simple, there were some rough edges. We helped our users navigate them. A common issue was a webservice not being able to run in the new cluster due to stricter quota limiting the resources for the tool. Another example is the new Ingress layer failing to properly work with some webservices s particular options. By March 2020, we no longer had anything running in the old Kubernetes cluster, and the migration was completed. We then started thinking about another step towards making a better Toolforge, which is introducing the toolforge.org domain. There is plenty of information about the change to this new domain in Wikitech News. The community wanted a better Toolforge, and so do we, and after almost 2 years of work, we have it! All the work that was done represents the commitment of the Wikimedia Foundation to support the technical community and how we really want to pursue technical engagement in general in the Wikimedia movement. In a follow-up post we will present and discuss more in-depth about some technical details of the new Kubernetes cluster, stay tuned! This post was originally published in the Wikimedia Tech blog, and is authored by Arturo Borrero Gonzalez and Brooke Storm.

17 May 2020

Enrico Zini: Art links

Guglielmo Achille Cavellini (11 September 1914 20 November 1990), also known as GAC, was an Italian artist and art collector. After an initial activity as a painter, in the 1940s and 1950s he became one of the major collectors of contemporary Italian abstract art, developing a deep relationship of patronage and friendship with the artists. This experience has its pinnacle in the exhibition Modern painters of the Cavellini collection at the National Gallery of Modern Art in Rome in 1957. In the 1960s Cavellini resumed his activity as an artist, with an ample production spanning from Neo-Dada to performance art to mail art, of which he became one of the prime exponents with the Exhibitions at Home and the Round Trip works. In 1971 he invented autostoricizzazione (self-historicization), upon which he acted to create a deliberate popular history surrounding his existence. He also authored the books Abstract Art (1959), Man painter (1960), Diary of Guglielmo Achille Cavellini (1975), Encounters/Clashes in the Jungle of Art (1977) and Life of a Genius (1989).
Paul Gustave Louis Christophe Dor (/d re /; French: [ ys.tav d . e]; 6 January 1832 23 January 1883[1]) was a French artist, printmaker, illustrator, comics artist, caricaturist, and sculptor who worked primarily with wood-engraving.
Enrico Baj era bravissimo a pij per culo er potere usanno a fantasia. Co quaa sempricit che solo dii granni, raccatta robbe tipo bottoni, pezzi de stoffa, cordoni, passamanerie varie, e l appiccica su a tela insieme aa pittura sua: che pare quasi che sta a gioc ma giocanno giocanno, zitto zitto, riesce a rovesci er monno. >>

Steve Kemp: Some brief sysbox highlights

I started work on sysbox again recently, adding a couple of simple utilities. (The whole project is a small collection of utilities, distributed as a single binary to ease installation.) Imagine you want to run a command for every line of STDIN, here's a good example:
 $ cat input   sysbox exec-stdin "youtube-dl  "
Here you see for every (non-empty) line of input read from STDIN the command "youtube-dl" has been executed. " " gets expanded to the complete line read. You can also access individual fields, kinda like awk. (Yes youtube-dl can read a list of URLs from a file, this is an example!) Another example, run groups for every local user:
$ cat /etc/passwd   sysbox exec-stdin --split=: groups  1 
Here you see we have split the input-lines read from STDIN by the : character, instead of by whitespace, and we've accessed the first field via " 1 ". This is certainly easier for scripting than using a bash loop. On the topic of bash; command-completion for each subcommand, and their arguments, is now present:
$ source <(sysbox bash-completion)
And I've added a text-based UI for selecting files. You can also execute a command, against the selected file:
$ sysbox choose-file -exec "xine  " /srv/tv
This is what that looks like: /2020/05/17-choose-file.png You'll see: As well as choosing files, you can also select from lines read via STDIN, and you can filter the command in the same way as before. (i.e. " " is the selected item.) Other commands received updates, so the calculator now allows storing results in variables:
$ sysbox calc
calc> let a = 3
3
calc> a / 9 * 3
1
calc> 1 + 2 * a
7
calc> 1.2 + 3.4
4.600000

13 May 2020

Russ Allbery: Review: Gideon the Ninth

Review: Gideon the Ninth, by Tamsyn Muir
Series: The Locked Tomb #1
Publisher: Tor
Copyright: September 2019
ISBN: 1-250-31317-1
Format: Kindle
Pages: 448
Despite being raised there, Gideon Nav is an outsider in the Ninth House. Her mother, already dead, fell from the sky with a one-day-old Gideon in tow, leaving her an indentured servant. She's a grumpy, caustic teenager in a world of moldering corpses, animated skeletons, and mostly-dead adults whose parts are falling off. Her world is sword fighting, dirty magazines, a feud with the house heir Harrowhark, and a determination to escape the terms of her indenture. Gideon does get off the planet, but not the way that she expects. She doesn't get accepted into the military. She ends up in the middle of a bizarre test, or possibly an ascension rite, mingling with and competing with the nobility of the empire alongside her worst enemy. I struggled to enjoy the beginning of Gideon the Ninth. Gideon tries to carry the story on pure snark, but it is very, very goth. If you like desiccated crypts, mostly-dead goons, betrayal, frustration, necromancers, black robes, disturbing family relationships, gloom, and bitter despair, the first six chapters certainly deliver, but I was sick of it by the time Gideon gets out. Thankfully, the opening is largely unlike the rest of the book. What starts as an over-the-top teenage goth rebellion turns into a cross between a manor house murder mystery and a competitive escape room. This book is a bit of a mess, but it's a glorious mess. It's also the sort of glorious mess that I don't think would have been written or published twenty years ago, and I have a pet theory that attributes this to the invigorating influence of fanfic and writers who grew up reading and writing it. I read a lot of classic science fiction and epic fantasy as a teenager. Those books have many merits, obviously, but emotional range is not one of them. There are a few exceptions, but on average the genre either focused on puzzles and problem solving (how do we fix the starship, how do we use the magic system to take down the dark god) or on the typical "heroic" (and male-coded) emotions of loyalty, bravery, responsibility, authority, and defiance of evil. Characters didn't have messy breakups, frenemies, anxiety, socially-awkward love affairs, impostor syndrome, self-hatred, or depression. And authors weren't allowed to fall in love with the messiness of their characters, at least on the page. I'm not enough of a scholar to make the argument well, but I suspect there's a case to be made that fanfic exists partially to fill this gap. So much of fanfic starts from taking the characters on the canonical page or screen and letting them feel more, live more, love more, screw up more, and otherwise experience a far wider range of human drama, particularly compared to what made it into television, which was even more censored than what made it into print. Some of those readers and writers are now writing for publication, and others have gone into publishing. The result, in my theory, is that the range of stories that are acceptable in the genre has broadened, and the emotional texture of those stories has deepened. Whether or not this theory is correct, there are now more novels like this in the world, novels full of grudges, deflective banter, squabbling, messy emotional processing, and moments of glorious emotional catharsis. This makes me very happy. To describe the emotional payoff of this book in any more detail would be a huge spoiler; suffice it to say that I unabashedly love fragile competence and unexpected emotional support, and adore this book for containing it. Gideon's voice, irreverent banter, stubborn defiance, and impulsive good-heartedness are the center of this book. At the start, it's not clear whether there will be another likable character in the book. There will be, several of them, but it takes a while for Gideon to find them or for them to become likable. You'll need to like Gideon well enough to stick with her for that journey. I read books primarily for the characters, not for the setting, and Gideon the Ninth struck some specific notes that I will happily read endlessly. If that doesn't match your preferences, I would not be too surprised to hear you bounced off the book. There's a lot here that won't be to everyone's taste. The setting felt very close to Warhammer 40K: an undead emperor that everyone worships, endless war, necromancy, and gothic grimdark. The stage for most of the book is at least more light-filled, complex, and interesting than the Ninth House section at the start, but everything is crumbling, drowning, broken, or decaying. There's quite a lot of body horror, grotesque monsters, and bloody fights. And the ending is not the best part of the book; roughly the last 15% of the novel is composed of two running fight scenes against a few practically unkillable and frankly not very interesting villains. I got exhausted by the fighting long before it was over, and the conclusion is essentially a series cliffhanger. There are also a few too many characters. The collection of characters and the interplay between the houses is one of the strengths of this book, but Muir sets up her story in a way that requires eighteen significant characters and makes the reader want to keep track of all of them. It took me about halfway through the book before I felt like I had my bearings and wasn't confusing one character for another or forgetting a whole group of characters. That said, most of the characters are great, and the story gains a lot from the interplay of their different approaches and mindsets. Palamedes Sextus's logical geekery, in particular, is a great counterpoint to the approaches of most of the other characters. The other interesting thing Muir does in this novel that I've not seen before, and that feels very modern, is to set the book in essentially an escape room. Locking a bunch of characters in a sprawling mansion until people start dying is an old fictional trope, but this one has puzzles, rewards, and a progressive physical structure that provides a lot of opportunities to motivate the characters and give them space to take wildly different problem-solving approaches. I liked this a lot, and I'm looking forward to seeing it in future books. This is not the best book I've read, but I thoroughly enjoyed it, despite some problems with the ending. I've already pre-ordered the sequel. Followed by Harrow the Ninth. Rating: 8 out of 10

12 May 2020

Evgeni Golov: Building a Shelly 2.5 USB to TTL adapter cable

When you want to flash your Shelly 2.5 with anything but the original firmware for the first time, you'll need to attach it to your computer. Later flashes can happen over the air (at least with ESPHome or Tasmota), but the first one cannot. In theory, this is not a problem as the Shelly has a quite exposed and well documented interface: Shelly 2.5 pinout However, on closer inspection you'll notice that your normal jumper wires don't fit as the Shelly has a connector with 1.27mm (0.05in) pitch and 1mm diameter holes. Now, there are various tutorials on the Internet how to build a compatible connector using Ethernet cables and hot glue or with female header socket legs, and you can even buy cables on Amazon for 18 ! But 18 sounded like a lot and the female header socket thing while working was pretty finicky to use, so I decided to build something different. We'll need 6 female-to-female jumper wires and a 1.27mm pitch male header. Jumper wires I had at home, the header I got is a SL 1X20G 1,27 from reichelt.de for 0.61 . It's a 20 pin one, so we can make 3 adapters out of it if needed. Oh and we'll need some isolation tape. SL 1X20G 1,27 The first step is to cut the header into 6 pin chunks. Make sure not to cut too close to the 6th pin as the whole thing is rather fragile and you might lose it. SL 1X20G 1,27 cut into pieces It now fits very well into the Shelly with the longer side of the pins. Shelly 2.5 with pin headers attached Second step is to strip the plastic part of one side of the jumper wires. Those are designed to fit 2.54mm pitch headers and won't work for our use case otherwise. jumper wire with removed plastic As the connectors are still too big, even after removing the plastic, the next step is to take some pliers and gently press the connectors until they fit the smaller pins of our header. Shelly 2.5 with pin headers and a jumper wire attached Now is the time to put everything together. To avoid short circuiting the pins/connectors, apply some isolation tape while assembling, but not too much as the space is really limited. Shelly 2.5 with pin headers and a jumper wire attached and taped And we're done, a wonderful (lol) and working (yay) Shelly 2.5 cable that can be attached to any USB-TTL adapter, like the pictured FTDI clone you get almost everywhere. Shelly 2.5 with full cable and FTDI attached Yes, in an ideal world we would have soldered the header to the cable, but I didn't feel like soldering on that limited space. And yes, shrink-wrap might be a good thing too, but again, limited space and with isolation tape you only need one layer between two pins, not two.

Daniel Silverstone: The Lars, Mark, and Daniel Club

Last night, Lars, Mark, and I discussed Jeremy Kun's The communicative value of using Git well post. While a lot of our discussion was spawned by the article, we did go off-piste a little, and I hope that my notes below will enlighten you all as to a bit of how we see revision control these days. It was remarkably pleasant to read an article where the comments section wasn't a cesspool of horror, so if this posting encourages you to go and read the article, don't stop when you reach the bottom -- the comments are good and useful too.
This was a fairly non-contentious article for us though each of us had points we wished to bring up and chat about it turned into a very convivial chat. We saw the main thrust of the article as being about using the metadata of revision control to communicate intent, process, and decision making. We agreed that it must be possible to do so effectively with Mercurial (thus deciding that the mention of it was simply a bit of clickbait / red herring) and indeed Mark figured that he was doing at least some of this kind of thing way back with CVS. We all discussed how knowing the fundamentals of Git's data model improved our ability to work wih the tool. Lars and I mentioned how jarring it has originally been to come to Git from revision control systems such as Bazaar (bzr) but how over time we came to appreciate Git for what it is. For Mark this was less painful because he came to Git early enough that there was little more than the fundamental data model, without much of the porcelain which now exists. One point which we all, though Mark in particular, felt was worth considering was that of distinguishing between published and unpublished changes. The article touches on it a little, but one of the benefits of the workflow which Jeremy espouses is that of using the revision control system as an integral part of the review pipeline. This is perhaps done very well with Git based workflows, but can be done with other VCSs. With respect to the points Jeremy makes regarding making commits which are good for reviewing, we had a general agreement that things were good and sensible, to a point, but that some things were missed out on. For example, I raised that commit messages often need to be more thorough than one-liners, but Jeremy's examples (perhaps through expedience for the article?) were all pretty trite one-liners which perhaps would have been entirely obvious from the commit content. Jeremy makes the point that large changes are hard to review, and Lars poined out that Greg Wilson did research in this area, and at least one article mentions 200 lines as a maximum size of a reviewable segment. I had a brief warble at this point about how reviewing needs to be able to consider the whole of the intended change (i.e. a diff from base to tip) not just individual commits, which is also missing from Jeremy's article, but that such a review does not need to necessarily be thorough and detailed since the commit-by-commit review remains necessary. I use that high level diff as a way to get a feel for the full shape of the intended change, a look at the end-game if you will, before I read the story of how someone got to it. As an aside at this point, I talked about how Jeremy included a 'style fixes' commit in his example, but I loathe seeing such things and would much rather it was either not in the series because it's unrelated to it; or else the style fixes were folded into the commits they were related to. We discussed how style rules, as well as commit-bisectability, and other rules which may exist for a codebase, the adherence to which would form part of the checks that a code reviewer may perform, are there to be held to when they help the project, and to be broken when they are in the way of good communication between humans. In this, Lars talked about how revision control histories provide high level valuable communication between developers. Communication between humans is fraught with error and the rules are not always clear on what will work and what won't, since this depends on the communicators, the context, etc. However whatever communication rules are in place should be followed. We often say that it takes two people to communicate information, but when you're writing commit messages or arranging your commit history, the second party is often nebulous "other" and so the code reviewer fulfils that role to concretise it for the purpose of communication. At this point, I wondered a little about what value there might be (if any) in maintaining the metachanges (pull request info, mailing list discussions, etc) for historical context of decision making. Mark suggested that this is useful for design decisions etc but not for the style/correctness discussions which often form a large section of review feedback. Maybe some of the metachange tracking is done automatically by the review causing the augmentation of the changes (e.g. by comments, or inclusion of design documentation changes) to explain why changes are made. We discussed how the "rebase always vs. rebase never" feeling flip-flopped in us for many years until, as an example, what finally won Lars over was that he wants the history of the project to tell the story, in the git commits, of how the software has changed and evolved in an intentional manner. Lars said that he doesn't care about the meanderings, but rather about a clear story which can be followed and understood. I described this as the switch from "the revision history is about what I did to achieve the goal" to being more "the revision history is how I would hope someone else would have done this". Mark further refined that to "The revision history of a project tells the story of how the project, as a whole, chose to perform its sequence of evolution." We discussed how project history must necessarily then contain issue tracking, mailing list discussions, wikis, etc. There are exist free software projects where part of their history is forever lost because, for example, the project moved from Sourceforge to Github, but made no effort (or was unable) to migrate issues or somesuch. Linkages between changes and the issues they relate to can easily be broken, though at least with mailing lists you can often rewrite URLs if you have something consistent like a Message-Id. We talked about how cover notes, pull request messages, etc. can thus also be lost to some extent. Is this an argument to always use merges whose message bodies contain those details, rather than always fast-forwarding? Or is it a reason to encapsulate all those discussions into git objects which can be forever associated with the changes in the DAG? We then diverted into discussion of CI, testing every commit, and the benefits and limitations of automated testing vs. manual testing; though I think that's a little too off-piste for even this summary. We also talked about how commit message audiences include software perhaps, with the recent movement toward conventional commits and how, with respect to commit messages for machine readability, it can become very complex/tricky to craft good commit messages once there are multiple disparate audiences. For projects the size of the Linux kernel this kind of thing would be nearly impossible, but for smaller projects, perhaps there's value. Finally, we all agreed that we liked the quote at the end of the article, and so I'd like to close out by repeating it for you all... Hal Abelson famously said:
Programs must be written for people to read, and only incidentally for machines to execute.
Jeremy agrees, as do we, and extends that to the metacommit information as well.

9 May 2020

Dirk Eddelbuettel: ttdo 0.0.5: Reflect tinytest update

A maintenance release of our (still small) ttdo package just arrived on CRAN. As introduced last fall, the ttdo package extends the most excellent (and very minimal / zero depends) unit testing package tinytest by Mark van der Loo with the very clever and well-done diffobj package by Brodie Gaslam to give us test results with visual diffs: ttdo screenshot tinytest has an extension mechanism we use, and as tinytest was just upgraded to version 1.2.0 changing, among other nice extensions, one interface by allowing for a new error class argument, we had to rebuild as well in order to document the new argument. The release was actually prepared three days ago when tinytest itself was updated, but we waited for the binaries at CRAN to be updated and rebuilt to take advantage of the fully automated submission and test process at CRAN. The NEWS entry follow.

Changes in ttdo version 0.0.5 (2020-05-06)
  • Rebuilt under tinytest 1.2.0 to add support for class argument in error-code test predicates

CRANberries provides the usual summary of changes to the previous version. Please use the GitHub repo and its issues for any questions. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 May 2020

Reproducible Builds: Reproducible Builds in April 2020

Welcome to the April 2020 report from the Reproducible Builds project. In our regular reports we outline the most important things that we and the rest of the community have been up to over the past month. What are reproducible builds? One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. But whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into seemingly secure software during the various compilation and distribution processes.

News It was discovered that more than 725 malicious packages were downloaded thousands of times from RubyGems, the official channel for distributing code for the Ruby programming language. Attackers used a variation of typosquatting and replaced hyphens and underscores (for example, uploading a malevolent atlas-client in place of atlas_client) that executed a script that intercepted Bitcoin payments. (Ars Technica report) Bernhard M. Wiedemann launched ismypackagereproducibleyet.org, a service that takes a package name as input and displays whether the package is reproducible in a number of distributions. For example, it can quickly show the status of Perl as being reproducible on openSUSE but not in Debian. Bernhard also improved the documentation of his unreproducible package to add some example patches for hash issues. [ ]. There was a post on Chaos Computer Club s website listing Ten requirements for the evaluation of Contact Tracing apps in relation to the SARS-CoV-2 epidemic. In particular:
4. Transparency and verifiability: The complete source code for the app and infrastructure must be freely available without access restrictions to allow audits by all interested parties. Reproducible build techniques must be used to ensure that users can verify that the app they download has been built from the audited source code.
Elsewhere, Nicolas Boulenguez wrote a patch for the Ada programming language component of the GCC compiler to skip -f.*-prefix-map options when writing Ada Library Information files. Amongst other properties, these .ali files embed the compiler flags used at the time of the build which results in the absolute build path being recorded via -ffile-prefix-map, -fdebug-prefix-map, etc. In the Arch Linux project, kpcyrd reported that they held their first rebuilder workshop . The session was held on IRC and participants were provided a document with instructions on how to install and use Arch s repro tool. The meeting resulted in multiple people with no prior experience of Reproducible Builds validate their first package. Later in the month he also announced that it was now possible to run independent rebuilders under Arch in a hands-off, everything just works solution to distributed package verification. Mathias Lang submitted a pull request against dmd, the canonical compiler for the D programming languageto add support for our SOURCE_DATE_EPOCH environment variable as well the other C preprocessor tokens such __DATE__, __TIME__ and __TIMESTAMP__ which was subsequently merged. SOURCE_DATE_EPOCH defines a distribution-agnostic standard for build toolchains to consume and emit timestamps in situations where they are deemed to be necessary. [ ] The Telegram instant-messaging platform announced that they had updated to version 5.1.1 continuing their claim that they are reproducible according to their full instructions and therefore verifying that its original source code is exactly the same code that is used to build the versions available on the Apple App Store and Google Play distribution platforms respectfully. Lastly, Herv Boutemy reported that 97% of the current development versions of various Maven packages appear to have a reproducible build. [ ]

Distribution work In Debian this month, 89 reviews of Debian packages were added, 21 were updated and 33 were removed this month adding to our knowledge about identified issues. Many issue types were noticed, categorised and updated by Chris Lamb, including: In addition, Holger Levsen filed a feature request against debrebuild, a tool for rebuilding a Debian package given a .buildinfo file, proposing to add --standalone or --one-shot-mode functionality.
In openSUSE, Bernhard M. Wiedemann made the following changes: In Arch Linux, a rebuilder instance has been setup at reproducible.archlinux.org that is rebuilding Arch s [core] repository directly. The first rebuild has led to approximately 90% packages reproducible contrasting with 94% on the Reproducible Build s project own ArchLinux status page on tests.reproducible-builds.org that continiously builds packages and does not verify Arch Linux packages. More information may be found on the corresponding wiki page and the underlying decisions were explained on our mailing list.

Software development

diffoscope Chris Lamb made the following changes to diffoscope, the Reproducible Builds project s in-depth and content-aware diff utility that can locate and diagnose reproducibility issues (including preparing and uploading versions 139, 140, 141, 142 and 143 to Debian which were subsequently uploaded to the backports repository):
  • Comparison improvements:
    • Dalvik .dex files can also serve as APK containers so restrict the narrower identification of .dex files to files ending with this extension and widen the identification of APK files to when file(1) discovers a Dalvik file. (#28)
    • Add support for Hierarchical Data Format (HD5) files. (#95)
    • Add support for .p7c and .p7b certificates. (#94)
    • Strip paths from the output of zipinfo(1) warnings. (#97)
    • Don t uselessly include the JSON similarity percentage if it is 0.0% . [ ]
    • Render multi-line difference comments in a way to show indentation. (#101)
  • Testsuite improvements:
    • Add pdftotext as a requirement to run the PDF test_metadata text. (#99)
    • apktool 2.5.0 changed the handling of output of XML schemas so update and restrict the corresponding test to match. (#96)
    • Explicitly list python3-h5py in debian/tests/control.in to ensure that we have this module installed during a test run to generate the fixtures in these tests. [ ]
    • Correct parsing of ./setup.py test --pytest-args arguments. [ ]
  • Misc:
    • Capitalise Ordering differences only in text comparison comments. [ ]
    • Improve documentation of FILE_TYPE_HEADER_PREFIX and FALLBACK_FILE_TYPE_HEADER_PREFIX to highlight that only the first 16 bytes are used. [ ]
Michael Osipov created a well-researched merge request to return diffoscope to using zipinfo directly instead of piping input via /dev/stdin in order to ensure portability to the BSD operating system [ ]. In addition, Ben Hutchings documented how --exclude arguments are matched against filenames [ ] and Jelle van der Waa updated the LLVM test fixture difference for LLVM version 10 [ ] as well as adding a reference to the name of the h5dump tool in Arch Linux [ ]. Lastly, Mattia Rizzolo also fixed in incorrect build dependency [ ] and Vagrant Cascadian enabled diffoscope to locate the openssl and h5dump packages on GNU Guix [ ][ ], and updated diffoscope in GNU Guix to version 141 [ ] and 143 [ ].

strip-nondeterminism strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. In April, Chris Lamb made the following changes:
  • Add deprecation plans to all handlers documenting how or if they could be disabled and eventually removed, etc. (#3)
  • Normalise *.sym files as Java archives. (#15)
  • Add support for custom .zip filename filtering and exclude two patterns of files generated by Maven projects in fork mode. (#13)

disorderfs disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues. This month, Chris Lamb fixed a long-standing issue by not drop UNIX groups in FUSE multi-user mode when we are not root (#1) and uploaded version 0.5.9-1 to Debian unstable. Vagrant Cascadian subsequently refreshed disorderfs in GNU Guix to version 0.5.9 [ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including: In addition, Bernhard informed the following projects that their packages are not reproducible:
  • acoular (report unknown non-determinism)
  • cri-o (report a date issue)
  • gnutls (report certtool being unable to extend certificates beyond 2049)
  • gnutls (report copyright year variation)
  • libxslt (report a bug about non-deterministic output from data corruption)
  • python-astropy (report a future build failure in 2021)

Project documentation This month, Chris Lamb made a large number of changes to our website and documentation in the following categories:
  • Community engagement improvements:
    • Update instructions to register for Salsa on our Contribute page now that the signup process has been overhauled. [ ]
    • Make it clearer that joining the rb-general mailing list is probably a first step for contributors to take. [ ]
    • Make our full contact information easier to find in the footer (#19) and improve text layout using bullets to separate sections [ ].
  • Accessibility:
    • To improve accessibility, make all links underlined. (#12)
    • Use an enhanced foreground/background contrast ratio of 7.04:1. (#11)
  • General improvements:
  • Internals:
    • Move to using jekyll-redirect-from over manual redirect pages [ ][ ] and add a redirect from /docs/buildinfo/ to /docs/recording/. (#23)
    • Limit the website self-check to not scan generated files [ ] and remove the old layout checker now that I have migrated all them [ ].
    • Move the news archive under the /news/ namespace [ ] and improve formatting of archived news links [ ].
    • Various improvements to the draft template generation. [ ][ ][ ][ ]
In addition, Holger Levsen clarified exactly which month we ceased to do weekly reports [ ] and Mattia Rizzolo adjusted the title style of an event page [ ]. Marcus Hoffman also started a discussion on our website s issue tracker asking for clarification on embedded signatures and Chris Lamb subsequently replied and asked Marcus to go ahead and propose a concrete change.

Testing framework We operate a large and many-featured Jenkins-based testing framework that powers tests.reproducible-builds.org that, amongst many other tasks, tracks the status of our reproducibility efforts as well as identifies any regressions that have been introduced.
  • Chris Lamb:
    • Print the build environment prior to executing a build. [ ]
    • Drop a misleading disorderfs-debug prefix in log output when we change non-disorderfs things in the file and, as it happens, do not run disorderfs at all. [ ]
    • The CSS for the package report pages added a margin to all <a> HTML elements under <li> ones, which was causing a comma/bullet spacing issue. [ ]
    • Tidy the copy in the project links sidebar. [ ]
  • Holger Levsen:
    • General:
    • Debian:
      • Reduce scheduling frequency of the buster distribution on the arm64 architecture, etc.. [ ][ ]
      • Show builds per day on a per-architecture basis for the last year on the Debian dashboard. [ ]
      • Drop the Subgraph OS package set as development halted in 2017 or 2018. [ ]
      • Update debrebuild to version from the latest version of devscripts. [ ][ ]
      • Add or improve various parts of the documentation. [ ][ ][ ]
    • Work on a Debian rebuilder:
      • Integrate sbuild. [ ][ ][ ][ ][ ]
      • Select a random .buildinfo file and attempt to build and compare the result. [ ][ ][ ][ ]
      • Improve output and related output formatting. [ ][ ][ ][ ][ ]
      • Outline next steps for the development of the tool. [ ][ ][ ]
      • Various refactoring and code improvements. [ ][ ][ ]
Lastly, Mattia Rizzolo fixed some log parsing code regarding potentially-harmless warnings from package installation [ ][ ] and the usual build node maintenance was performed by Holger Levsen [ ][ ][ ] and Mattia Rizzolo [ ][ ][ ].

Misc news On our mailing list this month, Santiago Torres asked whether we were still publishing releases of our tools to our website and Chris Lamb replied that this was not the case and fixed the issue. Later in the month Santiago also reported that the signature for the disorderfs package did not pass its GPG verification which was also fixed by Chris Lamb. Hans-Christoph Steiner of the Guardian Project asked whether there would be interest in making our website translatable which resulted in a WIP merge request being filed against the website and a discussion on how to track translation updates.
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

This month s report was written by Bernhard M. Wiedemann, Chris Lamb, Daniel Shahaf, Holger Levsen, Jelle van der Waa, kpcyrd, Mattia Rizzolo and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

4 May 2020

Dima Kogan: Redirection and /dev/stderr

I just hit some unexpected unix-ism (or maybe Linux-ism?) that I'd like to mention here. I found a workaround, but if anybody knows what's actually happening and can enlighten me, please send me an email, and I'll update the post. OK, so I have a program that does stuff, and outputs its progress to stderr. And I'm invoking this from a shell script that also outputs stuff to stderr. A minimized sample:
echo "Outer print 1111" > /dev/stderr
perl -E 'say STDERR ("1111 1111 1111 1111 1111 1111 1111 1111\n")'
echo "Outer print 2222" > /dev/stderr
perl -E 'say STDERR ("2222 2222 2222 2222 2222 2222 2222 2222\n")'
Let's call this dothing.sh. This works just fine. Now let's say I want to log the output to a file. Naturally I do this:
./dothing.sh 2> /tmp/log
Surprisingly, this does not work:
$ bash dothing.sh 2> /tmp/log && cat /tmp/log
Outer print 2222
2222 2222 2222 2222 2222 2222 2222 2222
$ bash dothing.sh 2> /tmp/log && xxd /tmp/log
0000000: 4f75 7465 7220 7072 696e 7420 3232 3232  Outer print 2222
0000010: 0a00 0000 0000 0000 0000 0000 0000 0000  ................
0000020: 0000 0000 0000 0000 0032 3232 3220 3232  .........2222 22
0000030: 3232 2032 3232 3220 3232 3232 2032 3232  22 2222 2222 222
0000040: 3220 3232 3232 2032 3232 3220 3232 3232  2 2222 2222 2222
0000050: 0a0a                                     ..
The 1111 prints are gone, and we now have a block of binary 0 bytes in there for some reason. I had been assuming that writes to stderr block until they're finished. So I can, without thinking about it, mix writing to fd 2 and opening, and then writing to /dev/stderr. Apparently the right way to do this is to write to fd 2 in the script directly, instead of opening /dev/stderr:
echo "Outer print 1111" >&2
perl -E 'say STDERR ("1111 1111 1111 1111 1111 1111 1111 1111\n")'
echo "Outer print 2222" >&2
perl -E 'say STDERR ("2222 2222 2222 2222 2222 2222 2222 2222\n")'
Why?

Update So I poked at it for a bit more, and it turns out that the obvious scapegoat (buffering) is actually not the issue. The problem is this:

$ ls -l /dev/stderr(:A)
-rw-r--r-- 1 dima dima 36 May  4 17:02 /tmp/log
:A is the zsh syntax for recursive link-following. /dev/stderr is thus ultimately a link to /tmp/log. So the file on disk /tmp/log is being opened for writing and written-to twice at the same time
  • once in the outer 2>/tmp/log redirection
  • again in the inner >/dev/stderr redirection
Now we know. Thanks to Larry Doolittle for pointers.

3 May 2020

Iain R. Learmonth: Hetzner Dedicated Server Reverse DNS + Ansible

Continuing on the path towards all my stuff being managed by Ansible, I ve figured out a method of managing the reverse DNS entries for subnets on the Hetzner Dedicated Server. There s a bunch of Ansible modules for handling Hetzner Cloud, but these servers are managed in Robot which the Cloud API doesn t cover. Instead, you need to use the Robot Webservice. Ansible does have a module for doing pretty arbitrary things with web APIs though, so using that I ve got the following playbook figured out to keep the reverse DNS entries in sync:
---
- hosts:
  - vmhost_vm1
  gather_facts: False
  tasks:
  - name: import hetzner webservice credentials
    include_vars:
      file: "hetzner_ws.yml"
  - name: set rdns for hetzner hosts
    block:
    - name: get current rdns entry
      uri:
        user: "  hetzner_ws_user  "
        password: "  hetzner_ws_password  "
        url: "https://robot-ws.your-server.de/rdns/  vmip4  "
        status_code: [200, 404]
      register: rdns_result
    - name: update incorrect rdns entry
      uri:
        user: "  hetzner_ws_user  "
        password: "  hetzner_ws_password  "
        url: "https://robot-ws.your-server.de/rdns/  vmip4  "
        method: "POST"
        body_format: form-urlencoded
        body:
          ptr: "  inventory_hostname  "
        status_code: [200, 201]
      when: '"rdns" not in rdns_result.json or inventory_hostname != rdns_result.json.rdns.ptr'
      changed_when: True
    delegate_to: localhost
The host groups this runs on are currently hardcoded as the VMs that live in the Hetzner Dedicated Server. A future iteration of this might use some inventory plugin to look up the subnets that are managed on Hetzner and create a group for all of those. Right now it won t be setting the reverse DNS for the router interface on that subnet, and won t automatically include new server s subnets. Gathering facts is disabled because all of these run locally. There is at least one VM running on this server that I can t log in to because I host it for someone else, so running locally is a necessity. The webservice credentials are stored in an Ansible Vault encrypted YAML file and loaded explicitly. An important note: the webservice username and password is not the same as your regular Robot username and password. You need to create a webservice user in Robot under Settings; Webservice and app settings . If you attempt to authenticate with an incorrect username and password 3 times in a row, your IP address will be blocked for 10 minutes. There are 6 hosts in this group, so I did this a few times before I realised there was a different user account I needed to create. I d suggest limiting to a single host while you re testing to get the authentication figured out. The actual calls to the webservice take place in a block just to avoid having to specify the delegate_to: localhost twice. The first step looks up the current PTR record, and accepts success if it gives either a 200 or 404 status code (it will be 404 if there is no existing pointer record). The result of this is used to conditionally create/update the PTR record in the next task. If nothing needs to be done, nothing will be changed and the second task will be skipped. If a change is needed, the second step is successful if either the PTR record is updated (status 200) or created (status 201). This was actually a lot easier than I thought it would be, and the uri module looks to be really flexible, so I bet there are other things that I could easily integrate into my Ansible playbooks.

Evgeni Golov: Remote management for OpenWRT devices without opening inbound connections

Everyone is working from home these days and needs a decent Internet connection. That's especially true if you need to do video calls and the room you want to do them has the worst WiFi coverage of the whole flat. Well, that's exactly what happened to my parents in law. When they moved in, we knew that at some point we'll have to fix the WiFi - the ISP provided DSL/router/WiFi combo would not cut it, especially not with the shape of the flat and the elevator shaft in the middle of it: the flat is essentially a big C around said shaft. But it was good enough for email, so we postponed that. Until now. The flat has wired Ethernet, but the users MacBook Air does not. That would have been too easy, right? So let's add another access point and hope the situation improves. Luckily I still had a TP-Link Archer C7 AC1750 in a drawer, which I could quickly flash with a fresh OpenWRT release, disable DHCPd and configure the same SSID and keys as the main/old WiFi. But I didn't know which channels would be best in the destination environment. Under normal circumstances, I'd just take the AP, drive to my parents in law and finish the configuration there. Nope, not gonna happen these days. So my plan was to finish configuration here, put the AP in a box and on the porch where someone can pick it up. But this would leave me without a way to further configure the device once it has been deployed - I was not particularly interested in trying to get port forwarding configured via phone and I was pretty sure UPnP was disabled in the ISP router. Installing a Tor hidden service for SSH was one possibility, setting up a VPN and making the AP a client another. Well, or just creating a reverse tunnel with SSH! sshtunnel Creating a tunnel with OpenSSH is easy: ssh -R127.0.0.1:2222:127.0.0.1:22 server.example.com will forward localhost:2222 on server.example.com to port 22 of the machine the SSH connection originated from. But what happens if the connection dies? Adding a while true; do ; done around it might help, but I would really like not to reinvent the wheel here! Thankfully, somebody already invented that particular wheel and OpenWRT comes with a sshtunnel package that takes care of setting up and keeping up such tunnels and documentation how to do so. Just install the sshtunnel package, edit /etc/config/sshtunnel to contain a server stanza with hostname, port and username and a tunnelR stanza referring said server plus the local and remote sides of the tunnel and you're good to go.
config server home
  option user     user
  option hostname server.example.com
  option port     22
config tunnelR local_ssh
  option server         home
  option remoteaddress  127.0.0.1
  option remoteport     2222
  option localaddress   127.0.0.1
  option localport      22
The only caveat is that sshtunnel needs the OpenSSH client binary (and the package correctly depends on it) and OpenWRT does not ship the ssh-keygen tool from OpenSSH but only the equivalent for Dropbear. As OpenSSH can't read Dropbear keys (and vice versa) you'll have to generate the key somewhere else and deploy it to the OpenWRT box and the target system. Oh, and OpenWRT defaults to enabling password login via SSH, so please disable that if you expose the box to the Internet in any way! Using the tunnel After configuring and starting the service, you'll see the OpenWRT system logging in to the configured remote and opening the tunnel. For some reason that connection would not show up in the output of w -- probably because there was no shell started or something, but logs show it clearly. Now it's just a matter of connecting to the newly open port and you're in. As the port is bound to 127.0.0.1, the connection is only possible from server.example.com or using it as a jump host via OpenSSH's ProxyJump option: ssh -J server.example.com -p 2222 root@localhost. Additionally, you can forward a local port over the tunneled connection to create a tunnel for the OpenWRT webinterface: ssh -J server.example.com -p 2222 -L8080:localhost:80 root@localhost. Yes, that's a tunnel inside a tunnel, and all the network engineers will go brrr, but it works and you can access LuCi on http://localhost:8080 just fine. If you don't want to type that every time, create an entry in your .ssh/config:
Host openwrt
  ProxyJump server.example.com
  HostName localhost
  Port 2222
  User root
  LocalForward 8080 localhost:80
And we're done. Enjoy easy access to the newly deployed box and carry on.

2 May 2020

Jonathan Carter: Free Software Activities for 2020-04

Debian project leader This month contained the first week and a half or so of my term as Debian Project Leader. So far my focus has been getting up to speed and keeping the gears turning with day to day DPL tasks. The updates listed here will also be available on the DPL blog, where I aim to make more frequent updates. During May, Debian Brazil will host Debian talks throughout the month which they will stream to their YouTube channel. You can find the schedule in this git repository, most of the talks will be in Portuguese, but on the 4th of May at 21:00 UTC, I ll be covering some Debian project topics for an hour or so and take some questions if there s time left. 2020-04-19: Handover session with Sam, our outgoing DPL. We covered a lot of questions I had and main areas that the DPL works in. Thanks to Sam for having taken the time to do this. 2020-04-21: First day of term! Thank you to everybody who showed support and have offered to help! 2020-04-21: Request feedback from the trademark team on an ongoing trademark dispute. 2020-04-21: Join the GNOME Advisory Board as a representative from Debian. 2020-04-21: Reply on an ongoing community conflict issue. 2020-04-21: Update Debian project summary for SPI annual report. 2020-04-21: Received a great e-mail introduction from Debian France and followed up on that. 2020-04-21: Posted Bits from the new DPL to debian-devel-announce. 2020-04-22: Became Debian s OSI Affilliate representative. 2020-04-22: Reply to a bunch of media inquiries for interviews, will do them later when initial priorities are on track. 2020-04-23: Resign as Debian FTP team trainee and mailing list moderator. In both these areas there are enough people taking care of it and I intend to maximise available time for DPL and technical areas in the project. 2020-04-25: Review outgoing mail for trademark team. 2020-04-25: Answer some questions in preparation for DAM/Front Desk delegation updates. 2020-04-26: Initiated wiki documentation for delegation updates process. 2020-04-27: Update delegation for the Debian Front Desk team. 2020-04-29: Schedule video call with Debian treasurer team. 2020-04-29: OSI affiliate call. Learned about some Open Source projects including OpenDev, OpenSourceMatters, FOSS Responders and Democracy Labs.

Debian Social Work on the Debian Social project is progressing, we plan to start a separate blog syndicated to Planet Debian that contains progress and status updates. I ve been terrible at tracking the work we ve been doing on this, so for now, here are some micro updates:

MiniDebConf Online In the DebConf video team, we ve been wondering whether we have all the tools required to successfully host a DebConf (or even a mini DebConf) entirely online. There s really just one way to know for sure, so we re going to host MiniDebConf Online from 28 May to 31 May. The first two days will be an online MiniDebCamp, where we can try to hold online spints, meetings and general chit-chat. The last two days will be for talks and lightnight talks, with BoFs likely to take place throughout the 4 days (this will probably be decided once we have a content team). Announcements should go out within the next week, in the meantime, save the dates.

Debian package uploads 2020-04-07: Upload package flask-autoindex (0.6.6-1) to Debian unstable. 2020-04-07: Upload package gamemode (1.5.1-1) to Debian unstable. 2020-04-08: Accept MR#2 for live-tasks (add usbutils to live-task-standard). 2020-04-08: Upload package live-tasks (11.0.2) to Debian unstable (Closes: #955526, #944578, #942837, #942834). 2020-04-08: Close live-config bug #655198 (Only affects squeeze which is no longer supported). 2020-04-08: Upload package live-config (11.0.1) to Debian unstable. 2020-04-08: Upload package calamares (3.2.22-1) to Debian unstable. 2020-04-15: Upload package xabacus (8.2.6-1) to Debian unstable. 2020-04-16: Merge MR#1 for gnome-shell-extension-dashtodock (new upstream release). 2020-04-16: Upload package gnome-shell-extension-dashtodock (67+git20200225-1) to Debian unstable. 2020-04-16: Merge MR#1 for gnome-shell-extension-hard-disk-led (fix some lintian issues). 2020-04-16: Merge MR#1 for gnome-shell-extension-system-monitor (fix some lintian issues). 2020-04-17: Upload package calamares (3.2.23-1) to Debian unstable. 2020-04-17: Upload package catimg (2.6.0-1) to Debian unstable (Closes: #956150). 2020-04-17: Upload package fabulous (0.3.0+dfsg1-7) to Debian unstable (Closes: #952242). 2020-04-17: Upload package gnome-shell-extension-system-monitor (38+git20200414-32cc79e-1) to Debian unstable (Closes: #956656, #956171). 2020-04-17: Upload package gnome-shell-extension-arc-menu (45-1) to Debian unstable (Closes: #956168). 2020-04-18: Upload package toot (0.26.0-1) to Debian unstable. 2020-04-23: Update packaging for gnome-shell-extension-tilix-shortcut, upstream section needs some additional work before release is possible. 2020-04-23: Upload package xabacus (8.2.7-1) to Debian unstable. 2020-04-27: Upload package gnome-shell-extension-dash-to-panel (37-1) to Debian unstable (Closes: #956162, #954978). 2020-04-27: Upload package gnome-shell-extension-dash-to-panel (37-2) to Debian unstable. 2020-04-27: Upload package gnome-shell-extension-dashtodock (68-1) to Debian unstable. 2020-04-30: Merge MR#8 for gamemode (add symbol files) (Closes: #952425). 2020-04-30: Merge MR#9 for gamemode (reduce number of -dev packages generated). 2020-04-30: Merge MR#10 for gamemode (deal better with upgrades on a buggy version). 2020-04-30: Manually merge changes from MR#11 for gamemode (packaging fixes). 2020-04-30: Upload package gamemode (1.5.1-2) to Debian unstable.

Sylvain Beucler: Debian LTS and ELTS - April 2020

Debian LTS Logo Here is my transparent report for my work on the Debian Long Term Support (LTS) and Debian Extended Long Term Support (ELTS), which extend the security support for past Debian releases, as a paid contributor. In April, the monthly sponsored hours were split evenly among contributors depending on their max availability - I was assigned 28.75h for LTS (out of 30 max; all done) and 7.75h for ELTS (out of 20 max; I did 2.75). Escalation procedures were (internally) documented with a focus on discussing issues with team coordinator(s) first. Debian LTS had its first team meeting through IRC and lots of workflow question were discussed. This should help discuss questions that are a bit hard to bring up, and ensure everybody participates. There were lots of topics and it was a bit rushed, but this is something we want to repeat monthly now, possibly with audio/video in a couple months. Remarks from last month's report were discussed, strengthening the Front-Desk role. 10% of the global funding is now reserved for infrastructure work. What kind of work, and who (LTS or external) will do the work, will be discussed further. A fellow DD suggested (in a private conversation) that LTS may be taking time from the Debian Security team, due to additional commits to review. Conversely, this is another opportunity to mention all the global, non-LTS-specific work that LTS provides, which I usually highlight in my reports, and maybe I should be even more ;) ELTS - Wheezy LTS - Jessie Documentation/Scripts

1 May 2020

Paul Wise: FLOSS Activities April 2020

Changes

Issues

Review

Administration
  • myrepos: fix the forum
  • Debian: restart non-responsive tor daemon, restart processes due to OOM, apply debian.net changes for DD with expired key
  • Debian wiki: approve accounts
  • Debian QA services: deploy changes, auto-disable oldoldstable pockets

Communication

Sponsors The purple-discord work was sponsored by my employer. All other work was done on a volunteer basis.

Utkarsh Gupta: FOSS Activites in April 2020

Here s my (seventh) monthly update about the activities I ve done in the F/L/OSS world.

Debian
It s been 14 months since I ve started contributing to Debian. And 4 months since I ve been a Debian Developer. And in this beautiful time, I had this opprotunity to do and learn lots of new and interesting things. And most importantly, meet and interact with lots of lovely people!
Debian is $home.

Uploads:

Other $things:
  • Attended Ruby team meeting. Logs here.
  • Attended Perl team LHF. Report here.
  • Sponsored a lot of uploads for William Desportes and Adam Cecile.
  • Mentoring for newcomers.
  • FTP Trainee reviewing.
  • Moderation of -project mailing list.
  • Applied for DUCI project for Google Summer of Code 2020.

Ruby2.7 Migration:
Ruby2.7 was recently released on 25th December, 2019. Santa s gift. Believe it or not. We, the Debian Ruby team, have been trying hard to make it migrate to testing. And it finally happened. The default version in testing is ruby2.7. Here s the news! \o/
Here s what I worked on this month for this transition.

Upstream: Opened several issues and proposed patches (in the form of PRs):
  • Issue #35 against encryptor for Ruby2.7 test failures.
  • Issue #28 against image_science for removing relative paths.
  • Issue #106 against ffi-yajl for Ruby2.7 test failures.
  • PR #5 against aggregate for simply using require.
  • PR #6 against aggregate for modernizing CI and adding Ruby 2.5 and 2.7 support.
  • Issue #13 against espeak-ruby for Ruby2.7 test failures.
  • Issue #4 against tty-which for test failures in general.
  • Issue #11 against packable for Ruby2.7 test failures. PR #12 has been proposed.
  • Issue #10 against growl for test failures and proposed an initial patch.

Downstream: I fixed and uploaded the following packages in Debian:

Debian LTS
Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success.
This was my seventh month as a Debian LTS paid contributor. I was assigned 24.00 hours and worked on the following things:

CVE Fixes and Announcements:

Other LTS Work:

Other(s)
Sometimes it gets hard to categorize work/things into a particular category.
That s why I am writing all of those things inside this category.
This includes two sub-categories and they are as follows.

Personal: This month I could get the following things done:
  • Most importantly, I finally migrated to a new website. Huge UI imporvement! \o/
    From Jekyll to Hugo, it was not easy. But it was worth it! Many thanks to Luiz for writing hugo-coder, Clement, and Samyak!
    If you find any flaws, issues and pull requests are welcomed at utkarsh2102/utkarsh2102.com
  • Wrote battery-alert, a mini-project of my own to show battery alerts at <10% and >90%.
    Written in shell, it brings me all the satisfaction as it has saved my life on many occasions.
    And guess what? It has more users than just myself!
    Reviews and patches are welcomed \o/
  • Mentored in HackOn Hackathon. Thanks to Manvi for reaching out!
    It was fun to see people developing some really nice projects.
  • Thanks to Ray and John, I became a GitLab Hero!
    (I am yet to figure out my role and responibility though)
  • Atteneded Intro Sec Con and had the most fun!
    Heard Ian s keynote and attended other talks and learned how to use WireShark!

Open Source: Again, this contains all the things that I couldn t categorize earlier.
Opened several issues and pull requests:
  • Issue #297 against hugo-coder, asking to enable RSS feed for blogs.
  • PR #316 for hugo-coder for fixing the above issue myself.
  • Issue #173 against arbre for requesting a release.
  • Issue #104 against combustion, asking to relax dependency on rubocop. Fixed in this commit.
  • Issue #16 against ffi-compiler for requesting to fix homepage and license.
  • Issue #57 against gographviz for requesting a release.
  • Issue #14 against crb-blast, suggesting compatability with bio 2.0.x.
  • Issue #58 against uniform_notifier for asking to drop the use of ruby-growl.
  • PR #2072 for polybar, adding installation instructions on Debian systems.

Until next time.
:wq for today.

30 April 2020

Chris Lamb: Free software activities in April 2020

Here is my monthly update covering what I have been doing in the free software world during April 2020 (previous month's report). Looking it over prior to publishing, I am surprised how much I got done this month I felt that I was not only failing to do all the extra things I had planned, but I was doing far less than normal. But let us go easy on ourselves; nobody is nailing this. In addition, I did more hacking on the Lintian static analysis tool for Debian packages:
Reproducible builds One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter. Elsewhere in our tooling, I made the following changes to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, including preparing and uploading versions 139, 140, 141 and 142 to Debian: Lastly, I made a large number of changes to our website and documentation in the following categories:
Debian LTS This month I have contributed 18 hours to Debian Long Term Support (LTS) and 7 hours on its sister Extended LTS project. You can find out more about the project via the following video:
Debian I only filed three bugs in April, including one against snapshot.debian.org to report that a Content-Type HTTP header is missing when downloading .deb files (#956471) and to report build failures in the macs & ruby-enumerable-statistics packages:

28 April 2020

Antoine Beaupr : Drowned my camera: dealing with liquid spills in electronics

Folks who acutely dig into this website might know that I have been taking more pictures recently, as I got a new camera since January 2018: a beautiful Fujifilm X-T2 that I really like. Recently, I went out on a photo shoot in the rain. It was intermittent, light rain when I left so I figured the "weather proofing" (dpreview.com calls this "environmentally sealed") would keep the camera secure. After an hour of walking outside, however, rain intensified and I was just quickly becoming more and more soaked. Still trusting the camera would function, I carried on. But after about 90 minutes of dutiful work, the camera just turned off and wouldn't power back on. It had drowned. I couldn't believe it; "but this is supposed to be waterproof! This can't be happening!", I thought. I tried swapping out the battery for a fresh one, which was probably a bad idea (even if I was smart enough to do this under cover): still no luck, yet I could still not believe it was dead, so I figured I would look at it later when I was home. I still eventually removed the battery after a while, remembering that it mattered. Turns out the camera was really dead. Even at home, it wouldn't power up, even with fresh batteries. After closer inspection, the camera was as soaked as I was...
Two Sandisk memory cards with water droplets on them ...even the SD cards were wet!
I was filled with despair! My precious camera! I had been waiting for litterally decades to find the right digital camera that was as close to the good old film cameras I was used to. I was even working on black and white "film" to get back to basics, which turned into a project to witness the impact of the coronavirus on city life! All that was lost, or at least stopped: amazingly, the SD cards were just absolutely fine and survived the flooding without problem.
A one-way sign broken, fallen on the side in a gray cityscape The last photo my camera took before it died
A good photographer friend told me that this was actually fairly common: "if you shoot outside, get used to this, it will happen". So I tried "the rice trick": plunge your camera in a pile of rice and let it rest there for a long time. It didn't work so well: I didn't have a big enough container to hold the camera and the rice. I was also worried about rice particles inserting themselves into the camera holes, as I had opened all the ports to let it dry. I could also not keep myself from inserting a battery and trying it out again: amazingly, it powered up, only once, and died again. After shopping in desperation for dessicators (who would have thought you should keep those little bags from the stuff you order online!), I ended up buying silica gel dehumidifier from Lee Valley (13$, the small one!) which comes in a neat little metal box. But that never arrived in time so I had to find another solution. My partner threw the idea out in jest, but the actual solution worked, and it was surprisingly simple!
My camera and lens drying in a food dehydrator, at 30 C with 22 hours left Tada! Turns out you can dehydrate hardware too!
We have a food dehydrator at home (a Sedna Express if you really want to know) since we do a lot of backpacking and canot-camping, but I never thought I would put electronics in there. Turns out a food dehydrator is perfect: it has a per degree temperature control that can go very low and a timer. I set it to 30 C for 24 hours. (I originally set it to 40 C but it smelled like plastic after a while so my partner turned it off thinking it was melting the camera.) And now the camera is back! I was so happy! There is probably some permanent damage to the delicate circuitry in the camera. And I will probably not go back out in heavy rain again with the camera, or at least not without a rainjacket (35$USD at B&H) on the camera. And I am now in a position to tell other people what to do if they suffer the same fate...

Tips for dealing with electronic liquid damage So, lessons learned...
  1. when you have a liquid spill over your electronics: IMMEDIATELY REMOVE ALL ELECTRIC POWER, including the battery! (this is another reason why all batteries should be removable)
  2. if the spill is "sticky" (e.g. coffee, beer, maple syrup, etc) or "salty", do try to wash it with water, yet without flooding it any further (delicate balance, I know) some devices are especially well adapted to this: I have washed a keyboard with a shower head and drowned the thing completely, it worked fine after drying.
  3. do NOT power it back on until you are certain the equipment is dry
  4. let the electronics device dry for 24 to 48 hours with all ports open in a humidity-absorbing environment: a bag of rice works, but a food dehydrator is best. make sure the rice doesn't get stuck inside the machine: use a small mesh bag if necessary
  5. once you are confident the device has dried, fiddle with the controls and see if water comes out: it might not have dried because it was stuck inside a button or dial. if dry, try powering it back on and watch the symptoms. if it's still weird, try drying it for another day.
  6. if you get tired of waiting and the machine doesn't come back up, you will have to send it to the repair shop or open it up yourself to see if there is soldering damage you can fix.
I hope it might help careless people who dropped their coffee or ran out in the rain, believing the hype of waterproof cameras. Amateur tip: waterproof cameras are not waterproof...

Dirk Eddelbuettel: RcppArmadillo 0.9.870.2.0

armadillo image Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language and is widely used by (currently) 705 other packages on CRAN. A new upstream release 9.870.2 of Armadillo was released a few days ago. We had tested two release candidates, and this caught one bug. The release was held up at CRAN for a few days as one package had an overly sensitive test depending on random input data; eventually we all convinced ourselves that there was no (Rcpp)Armadillo issue here. So morale: friends don t let friends have tests depend on random behavior. Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.870.2.0 (2020-04-24)
  • Upgraded to Armadillo release 9.870.2 (Roasted Mocha Retox)
    • faster handling of matrix multiplication expressions by diagvec() and diagmat()
    • added trimatu_ind() and trimatl_ind()
    • more consistent detection of sparse vector expressions

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

27 April 2020

Dirk Eddelbuettel: #26: Upgrading to R 4.0.0

Welcome to the 26th post in the rationally regularized R revelations series, or R4 for short. R 4.0.0 was released two days ago, and a casual glance at some social media conversations appears to suggest quite some confusion, almost certainly some misunderstandings, and possibly also a fair amount of fear, uncertainty, and doubt about the process. So I thought I could show how I upgrade my own main workstation, live and in colour without a safety net. (Almost: I did upgrade my laptop yesterday which went swimmingly, if more slowly.) So here is a fresh video about upgrading to R 4.0.0, with some support slides as usual:

The slides used in the video are at this link. A few quick follow-ups to the live nature of this. The pbdZMQ package did in fact install smoothly once the (Ubuntu) -dev packages for Zero MQ were (re-)installed; then IRkernel also followed. BioConductor completed once I realized that GOSemSim needed the annotation package GO.db to be updated, that allowed MNF to install. So the only bug, really, was the circular depdency between pkgload and testthat. Overall, not bad at all for a quick afternoon session! And as mentioned, if you are interested and have questions concerning use of R on a .deb based system like Debain or Ubuntu (or Mint or ), the r-sig-debian list is a very good and friendly place to ask them. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Next.

Previous.