Search Results: "cts"

25 April 2020

Reinhard Tartler: Building Packages with Buildah in Debian

1 Building Debian Packages with buildah
Building packages in Debian seems to be a solved problem. But is it? At the bottom, installing the dpkg-dev package provides all the basic tools needed. Assuming that you already succeeded with creating the necessary packaging metadata (i.e., debian/changelog, debian/control, debian/copyright, etc., and there are great helper tools for this such ash dh-make, dh-make-golang, etc.,) it should be as simple as invoking the dpkg-buildpackage tool. So what's the big deal here?

The issue is that dpkg-buildpackage expects to be called with an appropriately setup build context, that is, it needs to be called in an environment that satisfies all build dependencies on the system. Let's say you are building a package for Debian unstable on your Debian stable system (this is the common scenario for the official Debian build machines), you would need your build to link against libraries in unstable, not stable. So how to tell the package build process where to find its dependencies?

The answer (in Debian and many other Linux distributions) is you do not at all. This is actually a somewhat surprising answer for software developers without a Linux distribution development background1. Instead, chroots "simulate" an environment that has all dependencies that we want to build against at the system locations, that is. /usr/lib, etc.

Chroots are basically full system installations in a subdirectory that includes system and application libraries. In order to use them, a package build needs to use the chroot(2) system call, which is a privileged operation. Also creating these system installations is a somewhat finicky process. In Debian, we have tools that make this process easier, the most popular ones are probably pbuilder(1)2 and sbuild(1)3. Still, they are somewhat clumsy to use, add significant levels of abstraction in the sense that they do quite a bit of magic behind the scenes to work hide the fact that privileged operations (need root to run chroot(2), etc.) are required. They are also somewhat brittle, for instance, if a build process is aborted (SIGKILL, or system crash), you may end up with temporary directories and files under userids other than your own that again may require root-privileges to cleanup.

What if there was an easy way to do all of the above without any process running as root? Enter rootless buildah.

Modern Linux container technologies allow unprivileged users to "untie" a process from the system (cf. the unshare(2) system call). This means that a regular user may run a process (and its child processes) in an "environment" where system calls behave differently and provide a configurable amount of isolation levels. This article demonstrates a novel build tool buildah, which is:
  • easy to setup build environment from the command line
  • secure, as no process needs to run as root
  • simple in architecture, requires no running daemon like for example docker
  • convenient to use: you can debug your debian/rules interactively
Architecturally, buildah is written in golang and compiled as a (mostly) statically linked executable. It builds on top a number of libraries written in golang, including github.com/containers/storage and github.com/containers/image. The overlay functionality is provided by the fuse-overlayfs(1) utility.

1.1 Preparation:
The Kernel in Debian bullseye (and in buster, and recent Ubuntu kernels to the best of my knowledge) do support usernamespaces, but leave them disabled by default. Here is how to enable them:
echo kernel.unprivileged_userns_clone = 1   sudo tee -a /etc/sysctl.d/containers.conf
systemctl -p /etc/sysctl.d/containers.conf
I have to admit that I'm not entirely sure why the Debian kernels don't enable usernamespaces by default. I've found a reference on stackexchange4 that claims this disables some "hardening" features in the Debian kernel. I also understand this step is not necessary if you chose to compile and run a vanilla upstream kernel. I'd appreciate a better reference and am happy to update this text.
$ c=$(buildah from docker.io/debian:sid)
Getting image source signatures
Copying blob 2bbc6b8c460d done
Copying config 9b90abe801 done
Writing manifest to image destination
Storing signatures
This command downloads the image from docker.io and stores it locally in your home directory. Let's install essential utilities for building Debian packages:
1: buildah run $c apt-get update -qq
2: buildah run $c apt-get install dpkg-dev -y
3: buildah config --workingdir /src $c
4: buildah commit $c dpkg-dev
The command on line 1 and 2 execute the installation of compilers and dpkg development tools such as dpkg-buildpackage, etc. The buildah config command in line 4 arranges that whenever you start a shell in the container, the current working directory in the container is in /src. Don't worry about this location not existing yet, we will make sources from your host system available there. The last command creates an OCI image with the name dpkg-dev. BTW, the name you use for the image in the commit command can be used in podman (but not the "containers"). See 5 and 6 for a comparison between podman and buildah.
buildah images -a
This output might look like this:
  REPOSITORY                 TAG      IMAGE          ID   CREATED   SIZE             
localhost/dpkg-dev latest b85c34f95d3e 16 seconds ago 406 MB
docker.io/library/debian sid 9b90abe801db 11 hours ago 124 MB

1.2 Running a package build
Now we have a working container with the reference in the variable $c. To use it conveniently with source packages that I have stored in /srv/scratch/packages/containers, let's introduce a shell alias r like this:
alias r='buildah run --tty -v /srv/scratch/packages/containers:/src $c '
This allows you to easily execute commands in that container:
r pwd
r bash
The last command will give you an interactive shell that we'll be using for building packages!
siretart@x1:~/scratch/packages/containers/libpod$ r bash

root@x1:/src# cd golang-github-openshift-imagebuilder

root@x1:/src/golang-github-openshift-imagebuilder# dpkg-checkbuilddeps
dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper (>= 11) dh-golang golang-any golang-github-containers-storage-dev (>= 1.11) golang-github-docker-docker-dev (>= 18.09.3+dfsg1) golang-github-fsouza-go-dockerclient-dev golang-glog-dev

root@x1:/src/golang-github-openshift-imagebuilder# apt-get build-dep -y .
Note, using directory '.' to get the build dependencies
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
[...]

root@x1:/src/golang-github-openshift-imagebuilder# dpkg-checkbuilddeps

root@x1:/src/golang-github-openshift-imagebuilder#
Now we have a environment that has all build-dependencies available and we are ready to build the package:
root@x1:/src/golang-github-openshift-imagebuilder# dpkg-buildpackage -us -uc -b
Assuming the package builds, the package build results are placed in /src inside the container, and are visible at ~/scratch/packages/containers on the host. There you can inspect, extract or even install them. The latter part allows you to interactively rebuild packages against updated dependencies, without the need of setting up an apt archive or similar.

2 Availability
The buildah(1) tool is available in debian/bullseye since 2019-12-17. Since is a golang binary, it only links dynamically against system libraries such as libselinux and libseccomp, which are all available in buster-backports. I'd expect buildah to just work on a debian/buster system as well provided you install those system libraries and possibly the backported Linux kernel.

Keeping the package at the latest upstream version is challenging because of its fast development pace that picks up new dependencies on golang libraries and new versions of existing libraries with every new upstream release. In general, this requires updating several source packages, and in many cases also uploading new source packages to the Debian archive that need to be processed by the Debian ftp-masters.

As an exercise, I suggest to install the buildah package from bullseye, 'git clone' the packaging repository from https://salsa.debian.org/go-team/packages/golang-github-containers-buildah and build the latest version yourself. Note, I would expect the above to even work on a Fedora laptop.

The Debian packages have not swept into the Ubuntu distributions yet, but I expect them to be included in the Ubuntu 20.10 release. In the mean time, ubuntu users can install the package that are provided by the upstream maintainers in the "Project Atomic" PPA at https://launchpad.net/~projectatomic/+archive/ubuntu/ppa

3 Related Tools
The buildah tool is accompanied with two "sister" tools:

The Skopeo package provides tooling to work with remote images registries. This allows you download, upload images to remote registries, convert container images between different formats. It has been available in Debian since 2020-04-20.

The podman tool is a 'docker' replacement. It provides a command-line interface that mimics the original docker command to an extend that a user familiar with docker might want to place this in their ~/.bashrc file:
alias docker='podman'
Unfortunately, at the time of writing podman is still being processed by the ftp-masters since 01-03-2020. At this point, I recommend building the package from our salsa repository at https://salsa.debian.org/debian/libpod

4 Conclusion
Building packages in the right build context is a fairly technical issue for which many tools have been written for. They come with different trade-off when it comes to usability. Containers promise a secure alternative to the tried and proven chroot-based approaches, and the buildah makes using this technology very easy to use.

I'd love to get into a conversation with you on how these tools work for you, and would like to encourage participation and assistance with keeping the complicated software stack up-to-date in Debian (and by extension, in derived distribution such as Ubuntu, etc.).

Footnotes:
1
At my day-job, we build millions of lines of C++ code on Solaris10 and AIX6, where concepts such as "chroots" are restricted to the super user 'root' and is therefore not available to developers, not even through wrappers. Instead, libraries and headers are installed into "refroots", that is, subdirectories that mimic the structure of the "sysroot" directories that are used in the embedded Linux community for cross-compiling packages, and we use Makefiles that set include flags (-I rules and -L flags) to tell the compiler and linker where to look.

24 April 2020

Mark Brown: Book club: Our Software Dependency Problem

A short while ago Daniel, Lars and I met to discuss Russ Cox s excellent essay Our Software Dependency Problem. This essay looks at software reuse in general, especially in the context of modern distribution methods like PyPI and NPM which make the whole process much more frictionless than traditional distribution methods used with languages like C. Possibly our biggest conclusion was that the essay is so eminently sensible that we mostly just talked about how much we agreed with it and how comprehensive it was, we particularly admired the clarity with which it explores how to evaluate the quality of free software projects. Next time we ll have to pick something more controversial to discuss!

22 April 2020

Jonathan Dowland: SUPERHOT

Continuing a series of blog posts about casual Nintendo Switch games, next in the series is SUPERHOT. Normally 19.99, I picked it up for 13.99 in a sale. That's a little bit more than I would usually pay for a casual game. SUPERHOT first came on my radar because someone I know from a baby group worked on their VR port in some capacity.
Slow-motion buckshot Slow-motion buckshot
A first-person shooter, SUPERHOT's USP is that time only progresses when you move well, nearly. Time is slowed to a complete crawl when you are not moving. The game's visual style is very distinctive: Almost everything is a washed out grey or white colour and porcelean-like texture, except weapons and objects you can interact with, which are a matt black, and enemies, which are a bright red. It reminds me a lot of the 1992 Amiga game Robocop 3.
Robocop 3 Robocop 3
The play-style is very reminiscent of the "Bullet Time" sequences in The Matrix seemingly impossibly overwhelming odds deftly manoeuvred through thanks to superhuman reaction times. The game has a relatively short campaign of little vignettes, linked together by a cyberpunk narrative. The game is sometimes criticised for the short campaign, but for me that's ideal. And the vignettes being short and quite standalone suits my play requirements very well.
Amiga easter-egg Amiga easter-egg
The narrative interspersed between the play scenarios is a little bit over-long, and you can spend an unreasonable amount of time bashing buttons to get through it. Despite that it's a moderately interesting story. Once you've beaten the campaign, you can go back and play any of the scenarios again, or try the newly unlocked endless mode. I haven't tried that yet. The original prototype for the game is a free-to-play in-browser demo, available here. On Windows PC, there's a sequel-of-sorts in the works called MIND CONTROL DELETE with a lot of new features to add replay value.

21 April 2020

Matthew Garrett: Linux kernel lockdown, integrity, and confidentiality

The Linux kernel lockdown patches were merged into the 5.4 kernel last year, which means they're now part of multiple distributions. For me this was a 7-year journey, which means it's easy to forget that others aren't as invested in the code as I am. Here's what these patches are intended to achieve, why they're implemented in the current form and what people should take into account when deploying the feature.

Root is a user - a privileged user, but nevertheless a user. Root is not identical to the kernel. Processes running as root still can't dereference addresses that belong to the kernel, are still subject to the whims of the scheduler and so on. But historically that boundary has been very porous. Various interfaces make it straightforward for root to modify kernel code (such as loading modules or using /dev/mem), while others make it less straightforward (being able to load new ACPI tables that can cause the ACPI interpreter to overwrite the kernel, for instance). In the past that wasn't seen as a significant issue, since there were no widely deployed mechanisms for verifying the integrity of the kernel in the first place. But once UEFI secure boot became widely deployed, this was a problem. If you verify your boot chain but allow root to modify that kernel, the benefits of the verified boot chain are significantly reduced. Even if root can't modify the on-disk kernel, root can just hot-patch the kernel and then make this persistent by dropping a binary that repeats the process on system boot.

Lockdown is intended as a mechanism to avoid that, by providing an optional policy that closes off interfaces that allow root to modify the kernel. This was the sole purpose of the original implementation, which maps to the "integrity" mode that's present in the current implementation. Kernels that boot in lockdown integrity mode prevent even root from using these interfaces, increasing assurances that the running kernel corresponds to the booted kernel. But lockdown's functionality has been extended since then. There are some use cases where preventing root from being able to modify the kernel isn't enough - the kernel may hold secret information that even root shouldn't be permitted to see (such as the EVM signing key that can be used to prevent offline file modification), and the integrity mode doesn't prevent that. This is where lockdown's confidentiality mode comes in. Confidentiality mode is a superset of integrity mode, with additional restrictions on root's ability to use features that would allow the inspection of any kernel memory that could contain secrets.

Unfortunately right now we don't have strong mechanisms for marking which bits of kernel memory contain secrets, so in order to achieve that we end up blocking access to all kernel memory. Unsurprisingly, this compromises people's ability to inspect the kernel for entirely legitimate reasons, such as using the various mechanisms that allow tracing and probing of the kernel.

How can we solve this? There's a few ways:
  1. Introduce a mechanism to tag memory containing secrets, and only restrict accesses to this. I've tried to do something similar for userland and it turns out to be hard, but this is probably the best long-term solution.
  2. Add support for privileged applications with an appropriate signature that implement policy on the userland side. This is actually possible already, though not straightforward. Lockdown is implemented in the LSM layer, which means the policy can be imposed using any other existing LSM. As an example, we could use SELinux to impose the confidentiality restrictions on most processes but permit processes with a specific SELinux context to use them, and then use EVM to ensure that any process running in that context has a legitimate signature. This is quite a few hoops for a general purpose distribution to jump through.
  3. Don't use confidentiality mode in general purpose distributions. The attacks it protects against are mostly against special-purpose use cases, and they can enable it themselves.

My recommendation is for (3), and I'd encourage general purpose distributions that enable lockdown to do so only in integrity mode rather than confidentiality mode. The cost of confidentiality mode is just too high compared to the benefits it provides. People who need confidentiality mode probably already know that they do, and should be in a position to enable it themselves and handle the consequences.

comment count unavailable comments

18 April 2020

Jonathan McDowell: Building a very minimal initramfs

I m working on trying to get a Qualcomm IPQ8064 device working properly. I d got as far as an extremely hacked up chain of networking booting and the serial console working, so the next stage was to try and get a basic userland in place to try and make poking things a bit easier. While I have a cross-compiling environment all setup (that s how I m building my kernels) I didn t really want to get into something complicated just to be able to poke around sysfs etc. I figured I could use a static BusyBox in an initramfs. I was right, but there were a couple of hiccups along the way so I m writing it up so I remember next time. First, I cheated and downloaded a prebuilt static binary from https://busybox.net/downloads/binaries/ (very convenient, thanks) - the IPQ8064 is a dual core ARMv7 so I used busybox-armv7l. There were various sites with init scripts to go along with this; I ended up with a variation of Star Brilliant s gist:
Minimal init script
#!/bin/busybox sh
busybox mkdir -p /dev /etc /proc /root /sbin /sys /usr/bin /usr/sbin
busybox mount -t proc proc /proc
busybox mount -t sysfs sys /sys
busybox mount -t devtmpfs dev /dev
echo Starting up...
echo ttyMSM0::respawn:/sbin/getty -L ttyMSM0 115200 vt100 >> /etc/inittab
echo Login with root and no password. > /etc/issue
echo >> /etc/issue
echo root::0:0:root:/root:/bin/sh > /etc/passwd
busybox mdev -s
busybox --install
hostname localhost
ip link set lo up
echo 5 > /proc/sys/kernel/printk
exec /linuxrc
busybox and the init script aren t sufficient. If you have any problems then you re going to want a console device. Otherwise you ll do what I did, forget to change one of the tty references in the script to the right device, and spend too long trying to figure out why you re not getting any output as soon as you start running userspace. It turns out that CONFIG_DEVTMPFS_MOUNT isn t sufficient for the initramfs (the Kconfig help even tells you that). So I need to create a /dev/console device file inside the initramfs too. And I don t build as root (and you shouldn t either) which makes creating a device node hard. It s ok though, because the Linux kernel doesn t need you to be root to build it, and it has a default initramfs with a console device in it. This is helpfully generated using usr/gen_init_cpio, which takes a file defining what you want to put in the cpio file that becomes your initramfs. I used the following:
Minimal initramfs creation file
# Simple busybox based initramfs
dir /dev 0755 0 0
nod /dev/console 0600 0 0 c 5 1
dir /root 0700 0 0
dir /bin 0755 0 0
dir /sbin 0755 0 0
file /init basic-init 0755 0 0
file /bin/busybox busybox-armv7l 0755 0 0
All that then needs to be done is linux/usr/gen_init_cpio initramfs.list > initramfs.cpio and the resulting initramfs.cpio is suitable for inclusion in your kernel. The precompiled binaries have a lot of stuff enabled, which makes for a pretty useful debugging environment with fairly minimal work.

17 April 2020

Julien Danjou: Being in Charge

Being in ChargeIf you never heard of the 10x engineer myth, it's a pretty great concept. It boils down to the idea where an engineer could be 10x more efficient than a random engineer. I find this fantastically twisted.Last week, I sat and chat with Alexis Monville in Le Podcast a podcast that equips you to make positive change in your organization. We talked about that 10x Engineer myth, and from there we digressed on how to grow your career and handle the different aspects of it.This was a very interesting exchange. Alexis is actually going to publish a new book next month (May 2020) entitled I am a Software Engineer and I am in charge.
Being in Charge
Lucky me, this week, I had the chance to be able to read the book before everybody else which means I actually read after our recording. I understood why Alexis said that a lot of what I was talking about during our podcast resonated with him. I send a detailed review of the book to Alexis and Michael if you're curious. I'm definitely recommending this book if you want to stop complaining about your job and start understanding how to pull the strings.I wish I had this book available 10 years ago!

15 April 2020

Ian Jackson: Adapter to use camera tripod as a microphone stand

People complained that my laptop sound was buzzy, so I bought a proper microphone (thanks to mdw for advice, and loan of some kit). Proper microphones come with a holder that accepts a screw from your microphone stand. In my case, 3/8" "BSW" - similar to 3/8" US standard "coarse" (or if you unscrew a supplied insert, a special 27tpi 5/8" UNS). I don't have a microphone stand and I didn't want to buy one. I have two camera tripods - a small one and a big one. My camera tripods have 1/4" coarse (20tpi) UNC screws. Also, we have a 3D printer at home. And nowadays you can download a configurable screw thread from the internet. So I made myself an adapter. After a few iterations I have a pretty good object which can be used to fit my new microphone to either camera tripod. (I found that a UNC thread was close enough to fit the microphone's BSW.) Of coure, this is Open Hardware and the (short) source code is available. (To build it you'll want to git clone to get the included files.) Pictures below the cut. The flange at the bottom is because tripods usually have a soft top which is supposed to stop scratching the camera; a narrow hexagonal base would make gouges, hence the wide base. ( Read more... )

comment count unavailable comments

14 April 2020

Dirk Eddelbuettel: RcppArmadillo 0.9.860.2.0

armadillo image Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language and is widely used by (currently) 706 other packages on CRAN. A new upstream release 9.860.2 of Armadillo was just released. The theme of convergence continues; the previous release increased the minor from 800 to 850, now we are at 860. We first wrapped this up as version 0.9.859.1.0, but it turned out to have been held back by a buglet between R 4.0.0 and Rcpp which the recent patch release fixed (along with other woes on old R or non-CRAN-alike macOS). It then turns out that the new (upstream) version 9.860.1 had a minor bug which I missed as I reverse-depends checked the prior version. Doh. My thanks, as always, to CRAN for spotting this. The fix was added upstream and we have 9.860.2 as RcppArmadillo 0.9.860.2.0. Changes in the new release are noted below.

Changes in RcppArmadillo version 0.9.860.2.0 (2020-04-13)
  • Upgraded to Armadillo release 9.860.2 (Roasted Mocha Fix)
    • Added powmat()
    • Faster access to columns in sparse submatrix views
    • Faster handling of relational expressions by accu()
    • Faster handling of sympd matrices by expmat(), logmat(), sqrtmat()
    • Workaround for save/load issues with HDF5 v1.12
  • Vignettes are now pre-made and include (#285)
  • Two test files are now skipped on 32-bit Windows

Courtesy of CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

13 April 2020

Shirish Agarwal: Migrant worker woes and many other stories

I was gonna use this blog post to share about the migrant worker woes as there has been multiple stories doing the rounds. For e.g. a story which caught the idea of few people but most of us, i.e. middle-class people are so much into our own thing that we care a fig leaf about what happens to migrants. This should not be a story coming from a humane society but it seems India is no different than any other country of the world and in not a good way. Allow me to share
Or for those who don t like youtube, here s an alternative link https://www.invidio.us/watch?v=JGEgZq_1jmc Now the above two editorial shares two stories, one of Trump retaliatory threat to India in the Q&A of the journalist. In fact, Trump has upped the ante on visa sanctions as India buckled so easily under pressure. There have been other stories doing the rounds how people who have illnesses who need HCQ in India are either dying or are close to death because of unavailability of HCQ in the medicine shop. There have been reports in Pune as well as South Mumbai (one of the poshest localities in Mumbai/Bombay) that medicine shops are running empty or emptier. There have been so many stories on that, with reporters going to shops and asking owners of the medicine shops and shop-owners being clueless. I think the best article which vividly describes the Government of India (GOI) response to the pandemic is the free-to-read article shared by Arundhati Roy in Financial Times. It has reduced so much of my work or sharing that it s unbelievable. And she has shared it with pictures and all so I can share other aspects of how the pandemic has been affecting India and bringing the worst out in the Government in its our of need. In fact, not surprisingly though, apparently there was also a pro-Israel similar thing which happened in Africa too . As India has too few friends now globally, hence it decided to give a free pass to them.

Government of India, news agencies and paid News One of the attempts the state tried to do, although very late IMHO is that it tried to reach out to the opposition i.e. Congress party and the others. Mrs. Sonia Gandhi, who is the Congress president asked that the Government should not run any of its ads on private television channels for a period of two years. There had been plenty of articles, both by medianama and others who have alleged that at least from the last 6 odd years, Government ads. comprise of almost 50-60% advertising budget of a channel advertising budget. This has been discussed also in medianama s roundtable on online content which happened few months back. While an edited version is out there on YT, this was full two day s event which happened across two different cities.
or the alternative to youtube https://www.invidio.us/watch?v=c1PhWR1-Urs It was as if the roundtable discussions were not enough, Mrs. Gandhi clarion call was answered by News Broadcaster s Association (NBA) and this is what they had to say
News Broadcasters Association reply to Mrs. Gandhi
To put it simply, NBA deplored the suggestion by Mrs. Gandhi and even called the economy in recession and all they had were the Government s own advertising budget to justify their existence. The statements in themselves are highly pregnant and reveal both the relationship that the media, print or mainstream news channels have with the Government of India. Now if you see that, doesn t it make sense that media always slants the story from the Government s perspective rather than remaining neutral. If my bread basket were on the onus of me siding with the Govt. that is what most sane persons would do, otherwise they would resign and leave which many reporters who had a conscience did. Interestingly enough, the NBA statement didn t just end there but also used the word recession , this is the term that Government of India (GOI) hates and has in turn has been maintaining the word, terminology slowdown . While from a layman s perspective the two terms may seem to be similar, if India has indeed been in recession then the tools and the decisions that should have been taken by GOI should have been much different than what they took. Interestingly, enough GOI has refrained from saying anything on the matter which only reveals their own interests in the matter. Also if an association head is making the statement, it is more than likely that he consulted a lawyer or two and used application of mind while drafting the response. In other words, or put more simply, this was a very carefully drafted letter because they know that tomorrow the opposition party may come into power so they don t want to upset the power dynamics too much.

Privacy issues arising due to the Pandemic On the same Financial Times, two stories which dealt with the possible privacy violations due to the Pandemic have been doing the rounds. The first one, by Yuval Noah Harari is more exploratory by nature and makes some very good points without going far too deep into specific instances of recent times but rather goes into history and past instances where Governments have used the pandemics to exert more control over their populace and drive their agenda. I especially liked the last few lines which he shared in his op-ed Even if the current administration eventually changes tack and comes up with a global plan of action, few would follow a leader who never takes responsibility, who never admits mistakes, and who routinely takes all the credit for himself while leaving all the blame to others. Yuval Noah Harari . The whole statement could right fit onto the American President which he was talking about while at the same time, fits right into the current Indian Prime Minister, Boris Johnson of UK and perhaps Jair Bolsanaro of Brazil. All these three-four individuals have in common is that most of them belong to right-wing and hence cater only to the rich industrialist s agenda. While I don t know about Jair Bolsanaro much, at least three out of four had to turn to socialism and had to give some bailout packages to the public at large, even though continuing to undermine their own actions. More on this probably a bit down the line. The second story shared by Nic Fildes and Javier Espinoza who broke the story of various surveillance attempts and the privacy concerns that people have. Even the Indian PMO has asked this data and because there was no protest by the civil society, a token protest was done by COAI (Cellular Operator Association of India) but beyond that nothing, I am guessing because the civil society didn t make much noise as everybody is busy with their own concerns of safety and things going on, it s possible that such data may have gone to the Government. There is not much new here that people who had been working on the privacy issues know, it s just how easy Governments are finding to do it. The part of informed consent is really a misnomer . Governments lie all the time, for e.g. in the UK, did the leave party and people take informed consent, no they pushed their own agenda. This is and will be similar in many countries of the world.

False Socialism by RW parties In at least the three countries I have observing, simply due to available time, that lot of false promises are being made by our leaders and more often than not, the bailouts will be given to already rich industrialists. An op-ed by Vivek Kaul, who initially went by his handle which means somebody who is educated but unemployed. While Vivek has been one-man army in revealing most of the Government s mischiefs especially as fudging numbers are concerned among other things, there have been others too. As far as the US is concerned, an e-zine called free press (literally) has been sharing Trump s hollowness and proclamations for U.S. . Far more interestingly, I found New York times investigated and found a cache of e-mails starting from early January, which they are calling Red Dawn . The cache is undeniable proof that medical personnel in the U.S. were very much concerned since January 2020 but it was only after other countries started lock-down that U.S. had to follow suit. I am sure Indian medical professionals may have done similar mail exchanges but we will never know as the Indian media isn t independent enough.

Domestic violence and Patriarchy There have been numerous reports of domestic violence against women going up, in fact two prominent publications have shared pieces about how domestic violence has gone up in India since the lockdown but the mainstream press is busy with its own tropes, the reasons already stated above. In fact, interestingly enough, most women can t wear loose fitting clothes inside the house because of the near ones being there 24 7 . This was being shared as India is going through summer where heat waves are common and most families do not have access to A/C s and rely on either a fan or just ventilation to help them out. I can t write more about this as simply I m not a woman so I haven t had to face the pressures that they have to every day. Interestingly though, there was a piece shared by arre. Interestingly, also arre whose content I have shared a few times on my blog has gone from light, funny to be much darker and more serious tone. Whether this is due to the times we live in is something that a social scientist or a social anthropologist may look into in the times to come. One of the good things though, there hasn t been any grid failures as no industrial activity is happening (at all). In fact SEB s (State Electricity Boards) has shown a de-growth in electricity uptake as no industrial activity has been taken. While they haven t reduced any prices (which they ideally should have) as everybody is suffering.

Loot and price rise Again, don t think it is an Indian issue but perhaps may be the same globally. Because of broken supply chains, there are both real and artificial shortages happening which is leading to reasonable and unreasonable price hikes in the market. Fresh veggies which were normally between INR 10/- to INR 20/- for 250 gm have reached INR 40/- 50/- and even above. Many of the things that we have to become depend upon are not there anymore. The shortage of plastic bottles being case in point.
Aryan Plastic bottle
This and many others like these pictures have been shared on social media but it seems the Government is busy doing something else. The only thing we know for sure is that the lock-down period is only gonna increase, no word about PPE s (Personal Protection Equipment) or face masks or anything else. While India has ordered some, those orders are being diverted to US or EU. In fact, many doctors who have asked for the same have been arrested, sacked or suspended for asking such inconvenient questions, although whether in BJP ruled states or otherwise. In fact, the Centre has suspended MPLADS funds , members of parliament get funds which they can use to provide relief work or whatever they think the money is best to spend upon.

Conditions of Labor in the Pandemic Another sort of depressing story has been how the Supreme Court CJI Justice SA Bobde has made statements and refrained from playing any role in directing the Center to provide relief to the daily wage laborers. In fact, Mr. Bobde made statements such as why they need salaries if they are getting food. This was shared by barandbench, a site curated by lawyers and reporters alike. Both livelaw as well as barandbench have worked to enhance people s awareness about the legal happenings in our High Courts and Supreme Court. And while sadly, they cannot cover all, they at least do attempt to cover a bit of what s hot atm. The Chief Justice who draws a salary of INR 250,000 per month besides other perks is perhaps unaware or doesn t care about fate of millions of casual workers, 400 460 million workers who will face abject poverty and by extension even if there are 4 members of the family so probably 1.2 billion people will fall below the poverty line. Three, four major sectors are going to be severely impacted, namely Agriculture, Construction and then MSME (Micro, small and medium enterprises) which cover everything from autos, industrial components, FMCG, electronics, you name it, it s done by the MSME sector. We know that the Rabi crop, even though it was gonna be a bumper crop this year will rot away in the fields. Even the Kharif crop whose window for sowing is at the most 2-3 weeks will not be able to get it done in time. In fact, with the extended lockdown of another 21 days, people will probably return home after 2 months by which time they would have nothing to do there as well as here in the cities. Another good report was done by the wire, the mainstream media has already left the station.

Ministry of Public Health There was an article penned by Dr. Edmond Fernandes which he published last year. The low salary along with the complexities that Indian doctors are and may face in the near future are just mind-boggling.

The Loss Losses have already started pouring in. Just today Air Deccan has ceased all its operations. I had loved Mr. Gopinath s airline which was started in the early 2000 s. While I won t bore you with the history, most of it can be seen from simplify Deccan . This I believe is just the start and it s only after the few months after the lock-down has been lifted would we really know the true extent of losses everywhere. And the more lenghthier the lockdown, the more difficult it would be businesses to ramp back. People have already diagnosed at the very least 15-20 sectors of the economy which would be hit and another similar or more number of sectors which will have first and second-order of losses and ramp-downs. While some guesses are being made, many are wildly optimistic and many are wildly pessimistic, as shared we would only know the results when the lockdown is opened up.

Predictions for the future While things are very much in the air, some predictions can be made or rationally deduced. For instance, investments made in automation and IT would remain and perhaps even accelerate a little. Logistics models would need to be re-worked and maybe, just maybe there would be talk and action in making local supply chains a bit more robust. Financing is going to be a huge issue for at least 6 months to a year. Infrastructure projects which require huge amount of cash upfront will either have to be re-worked or delayed, how they will affect projects like Pune Metro and other such projects only time will tell.

Raghuram Rajan Raghuram Rajan was recently asked if he would come back and let bygones be bygones. Raghuram in his own roundabout way said no. He is right now with Chicago Booth doing the work that he always love. Why would he leave that and be right in the middle of the messes other people have made. He probably gets more money, more freedom and probably has a class full of potential future economists. Immigration Control, Conferences and thought experiment There are so many clueless people out there, who don t know why it takes so long for any visa to be processed. From what little I know, it is to verify who you say you are and you have valid reason to enter the country. The people from home ministry verify credentials, as well as probably check with lists of known criminals and their networks world-wide. They probably have programs for such scenarios and are part and parcel of their everyday work. The same applies to immigration control at Airports. there has been a huge gap at immigration counters and the numbers of passengers who were flying internationally to and fro from India. While in India, we call them as Ministry of Home Affairs, in U.S. it s Department of Homeland security, other countries using similar jargons. Now even before this pandemic happened, the number of people who are supposed to do border control and check people was way less and there have been scenes of Air rage especially in Indian airports after people came after a long-distance flight. Now there are couple of thought experiments, just day before yesterday scientists discovered six new coronaviruses in bats and scientists in Iceland found 40 odd mutations of the virus on people. Now are countries going to ban people from Iceland as in time the icelandic people probably would have anti-bodies on all the forty odd mutations. Now if and when they come in contact onto others who have not, what would happen ? And this is not specifically about one space or ethnicity or whatever, microbes and viruses have been longer on earth than we have. In our greed we have made viruses resistant to antibiotics. While Mr. Trump says as he discovered it today, this has been known to the medical fraternity since tht 1950 s. CDC s own chart shows it. We cannot live in fear of a virus, the only way we can beat it is by understanding it and using science. Jon Cohen shared some of the incredible ways science is looking to beat this thing
or as again an alternative to youtube https://www.invidio.us/watch?v=MPVG_n3w_vM One of the most troubling question is how the differently-abled communities which don t have media coverage at the best of times, haven t had any media coverage at all during the pandemic. What are their stories and what they are experiencing ? How are they coping ? Are there anyways we could help each other ? By not having those stories, we perhaps have left them more vulnerable than we intend. And what does that speak about us, as people or as a community or a society ?

Silver Linings While there is not a lot to be positive about, one interesting project I came about is openbreath.tech . This is an idea, venture started by IISER (Indian Institute of Science Education and Research) , IUCAA (Inter-University Centre for Astronomy and Astrophysics). They are collaborating with octogeneraian Capt (Retd) Rustom Barucha from Barucha Instrumentation and Control, besides IndoGenius, New Delhi, and King s College, London. The first two institutes are from my home town, Pune. While I don t know much of the specifics of this idea other than that there is an existing Barucha ventilator which they hope to open-source and make it easier for people to produce their own. While I have more questions than answers at this point, this is something hopefully to watch out for in the coming days and weeks. The other jolly bit of good news has come from Punjab where after several decades, people in Northern Punjab are finally able to see the Himalayas or the Himalayan mountain range.
Dhauladhar range Northern Punjab Copyright CNN.Com
There you have it, What I have covered is barely scratching the surface. As a large section of the media only focuses on one narrative, other stories and narratives are lost. Be safe, till later.

10 April 2020

Dirk Eddelbuettel: Rcpp 1.0.4.6: Bug fix interim version

rcpp logo Rcpp 1.0.4 was released on March 17, following the usual sequence of fairly involved reverse-depends check along with a call for community testing issued weeks before the release. In that email I specifically pleaded with folks to pretty-please test non-standard setups:
It would be particularly beneficial if those with unsual build dependencies tested it as we would increase overall coverage beyond what I get from testing against 1800+ CRAN packages. BioConductor would also be welcome.
Alas, you can t always get what you want. Shortly after the release we were made aware that the two (large) pull request at the book ends of the 1.0.3 to 1.0.4 release period created trouble. Of these two, the earliest PR in the 1.0.4 release upset older-than-CRAN-tested installation, i.e. R 3.3.0 or before. (Why you d want to run R 3.3.* when R 3.6.3 is current is something I will never understand, but so be it.) This got addressed in two new PRs. And the matching last PR had a bit of sloppyness leaving just about everyone alone, but not all those macbook-wearing data scientists when using newer macOS SDKs not used by CRAN. In other words, unsual setups. But boy, do those folks have an ability to complain. Again, two quick PRs later that was addressed. Along came a minor PR with two more Rcpp::Shield<> uses (as life is too short to manually count PROTECT and UNPROTECT). And then a real issue between R 4.0.0 and Rcpp first noticed with RcppParallel builds on Windows but then also affecting RcppArmadillo. Another quickly issued fix. So by now the count is up to six, and we arrived at Rcpp 1.0.4.6. Which is now on CRAN, after having sat there for nearly a full week, and of course with no reason given. Because the powers that be move in mysterious ways. And don t answer to earthlings like us. As may transpire here, I am little tired from all this. I think we can do better, and I think we damn well should, or I may as well throw in the towel and just release to the drat repo where each of the six interim versions was available for all to take as soon as it materialized. Anyway, here is the state of things. Rcpp has become the most popular way of enhancing R with C or C++ code. As of today, 1897 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 191 in BioConductor. And per the (partial) logs of CRAN downloads, we are running steasy at one millions downloads per month. The changes for this interim version are summarized below.

Changes in Rcpp patch release version 1.0.4.6 (2020-04-02)
  • Changes in Rcpp API:
    • The exception handler code in #1043 was updated to ensure proper include behavior (Kevin in #1047 fixing #1046).
    • A missing Rcpp_list6 definition was added to support R 3.3.* builds (Davis Vaughan in #1049 fixing #1048).
    • Missing Rcpp_list 2,3,4,5 definition were added to the Rcpp namespace (Dirk in #1054 fixing #1053).
    • A further updated corrected the header include and provided a missing else branch (Mattias Ellert in #1055).
    • Two more assignments are protect with Rcpp::Shield (Dirk in #1059)
  • Changes in Rcpp Attributes:
    • Empty strings are not passed to R CMD SHLIB which was seen with R 4.0.0 on Windows (Kevin in #1062 fixing #1061).
  • Changes in Rcpp Deployment:
    • Travis CI unit tests now run a matrix over the versions of R also tested at CRAN (rel/dev/oldrel/oldoldrel), and coverage runs in parallel for a net speed-up (Dirk in #1056 and #1057).

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues); questions are also welcome under rcpp tag at StackOverflow which also allows searching among the (currently) 2356 previous questions. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

9 April 2020

Antoine Beaupr : Mumble dreams

With everyone switching to remote tools for social distancing, I've been using Mumble more and more. That's partly by choice -- I don't like videoconferencing much, frankly -- and partly by necessity: sometimes my web browser fails and Mumble is generally more reliable. Some friend on a mailing list recently asked "shouldn't we make Mumble better?" and opened the door for me to go on a long "can I get a pony?" email. Because I doubt anyone on that mailing list has the time or capacity to actually fix those issues, I figured I would copy this to a broader audience in the hope that someone else would pick it up.

Why Mumble rocks Before I go on with the UI critique, I should show why care: Mumble is awesome. When you do manage to configure it correctly, Mumble just works; it's highly reliable. It uses little CPU, both on the client and the server side, and can have rooms with tens if not hundreds of participants. The server can be easily installed and configured: there's a Debian package and resource requirements are minimal. It's basically network-bound. There are at least three server implementations, the official one called Murmur, the minimalist umurmur and Grumble, a Go rewrite. It has great quality: echo canceling, when correctly configured, is solid and latency is minimal. It has "overlays" so you can use it while gaming or demo'ing in full screen while still having an idea of who's talking. It also supports positional audio for gaming that integrates with popular games like Counterstrike or Half-Life. It's moderately secure: it doesn't support end-to-end encryption, but client/server communication is encrypted with TLS. It supports a server password and some moderation mechanisms.

UI improvements Mumble should be smarter about a bunch of things. Having all those settings is nice for geeky control freaks, but it makes the configuration absolutely unusable for most people. Hide most settings by default, and make better defaults. Specifically, those should be on by default:
  • RNNoise
  • echo cancellation (the proper "monitor" channels)
  • pre-configured shortcut for PTT (Push To Talk) -- right-shift is my favorite
  • "double-PTT" to hold it enabled
  • be more silent by default (I understand why it would want to do voice synthesis, but it would need to be much better at it before it's default)
The echo test should be more accessible, one or two clicks away from the main UI. I have only found out about that feature when someone told me where to find it. This basically means to take it out of the settings page and into its own dialog. The basic UI should be much simpler. It could look something like Jitsi: just one giant mute button with a list of speakers. Basically:
  1. Take that status bar and make it use the entire space of the main window
  2. Push the chat and room list dialog to separate, optional dialog (e.g. the room list could be a popup on login, but we don't need to continuously see the damn thing)
  3. Show the name of the person talking in the main UI, along with other speakers (Big Blue Button does this well: just a label that fades away with time after a person talks)
Some features could be better explained. For example, the "overlay" feature makes no sense at all for most users. It only makes sense when you're a gamer and use Mumble alongside another full-screen program, to show you who's talking. Improved authentication. The current authentication systems in Mumble are somewhat limited: the server can have a shared password to get access to it, and from there it's pretty much free-for-all. There are client certificates but those are hard to understand and the most common usage scenario is that someone manages to configure it once, forgets about it and then cannot login again with the same username. It should be easier to get the audio right. Now, to be fair, this is hard to do in any setup, and Mumble is only a part of this. There are way too many moving parts in Linux for this to be easy: between your hardware, ALSA drivers, Pulseaudio mixers and Mumble, too many things can go wrong. So this is a general problem when doing multimedia in general, and the Linux ecosystem in particular, but Mumble is especially hard to configure in there. Improved speaker stats. When you right-click on a user in Mumble, you get detailed stats about the user: packet loss, latency, bandwidth, codecs... It's pretty neat. But that is hard to parse for a user. Jitsi, in contrast, shows a neat little "bar graph" (similar to what you get on a cell phone) with a color code to show network conditions for that user. Then you can drill down to show more information. Having that info for the user would be really useful to figure out which user is causing that echo or latency. Heck, while I'm dreaming, we could do the same thing Jitsi and tell the user when we detect too much noise on their side and suggest muting! There's probably more UI issues, but at that point you have basically rebuilt the entire user interface. This problem is hard to fix because UX people are unlikely to have the skills required to hack at an (old) Qt app, and C++ hackers are unlikely to have the best UX skills...

Missing features Video. It has been on the roadmap since 2011, so I'm not holding my breath. It is, obviously, the key feature missing from the software when compared to other conferencing tools and it's nice to see they are considering it. Screensharing and whiteboarding would also be a nice addition. Unfortunately, all that is a huge undertaking and it's unlikely to happen in the short term. And even if it does, it's possible hard-core Mumble users would be really upset at the change... A good web app -- a major blocker to the adoption of Mumble is the need for that complex app. If users could join just with a web browser, adoption would be much easier. There is a web app called mumble-web out there, but it seems to work only for listening as there are numerous problems with recording: quality issues, audio glitches, voice activation, voice activation.. The CCC seems to be using that app to stream talk translation, so that part supposedly works correctly. Dial-in -- allow plain old telephones to call into conferences. There seems to be a program called mumsi that can do this, but it's unmaintained and it's unclear if any of the forks work at all. Update: according to samba, mumsi works, but sometimes freezes and needs to be restarted. Each SIP account shows up as a bot that comes up when someone calls the number. It supports multiple callers, although apparently mumsi crashes after a while with 4 callers. A comment here also mentioned there's a fork that mentions using a "pin" as well for dialing in.

Caveats Now the above will probably not happen soon. Unfortunately, Mumble has had trouble with their release process recently. It took them a long time to even agree on releasing 1.3, and when they did agree, it took them a long time again to actually do the release. There has been much more activity on the Mumble client and web app recently, so hopefully I will be proven wrong. The 1.3.1 release actually came out recently(correction:) is actually being worked on, which is encouraging. All in all, mumble has some deeply ingrained UI limitations. it's built like an app from the 1990, all the way down to the menu system and "status bar" buttons. It's definitely not intuitive for a new user and while there's an audio wizard that can help you get started, it doesn't always work and can be confusing in itself. I understand that I'm just this guy saying "please make this for me ktxbye". I'm not writing this as a critic of Mumble: I love the little guy, the underdog. Mumble has been around forever and it kicks ass. I'm writing this in a spirit of solidarity, in the hope the feedback can be useful and to provide useful guidelines on how things could be improved. I wish I had the time to do this myself and actually help the project beyond just writing, but unfortunately the reality is I'm a poor UI designer and I have little time to contribute to more software projects. So hopefully someone could take those ideas and make Mumble even greater. And if not, we'll just have to live with it. Thanks to all the Mumble developers who, over all those years, managed to make and maintain such an awesome product. You rock!

Update: a chat with Mumble folks Update: it seems the idea of simplifying the Mumble interface will take some time to sink in. After presenting this article in the #mumble Freenode IRC channel, it became obvious that having a more usable interface is not a priority. To put it in the words of a participant in the channel:
If someone is too stupid to use a piece of software than they should probably use something else or have someone capable of critical thinking set it up for them. -- meep
So I guess "I'm with stupid": I do not believe we should make software only for "smart" people because that leads to overly complicated and over-engineered user interfaces that are needlessly hard to use and configure. Mumble is actually an excellent example of such programs: powerful software, but too hard to use to actually reach critical mass. Thankfully, meep is not a Mumble developer and does not actually represent the official position of the Mumble project. A core developer (not representing the project either) was more nuanced:
The UI being not up to date is a matter of taste. The kind of UI you are describing is the kind of UI I certainly don't want. More gernally though I agree that the UI needs some rework and especially: Flexibility so that each user can (ideally) make it appear as (s)he wants. Before this can happen though, we'll have to rewrite a fair bit of code as we'll have to separate backend and frontend from one another. -- krzmbrzl
So this will take time. The fact that Qt seems to be moving back to closed-source does not help the situation with Mumble's GUI development, unfortunately. But the other hand, maybe it will provide the necessary "kick" to write a new, simpler GUI.

7 April 2020

Shirish Agarwal: GMRT 2020 and lots of stories

First of all, congratulations to all those who got us 2022 Debconf, so we will finally have a debconf in India. There is of course, lot of work to be done between now and then. For those who would be looking forward to visit India and especially Kochi I would suggest you to hear this enriching tale
I am sorry I used youtube link but it is too good a podcast not to be shared. Those who don t want youtube can use the invidio.us link for the same as shared below. https://www.invidio.us/watch?v=BvjgKuKmnQ4 I am sure there are lot more details, questions, answers etc. but would direct them gently to Praveen, Shruti, Balasankar and the rest who are from Kochi to answer if you have any questions about that history.

National Science Day, GMRT 2020 First, as always, we are and were grateful to both NCRA as well as GMRT for taking such good care of us. Even though Akshat was not around, probably getting engaged, a few of us were there. About 6-7 from the Mozilla Nasik while the rest representing the foss community. Here is a small picture which commentrates the event
National Science Day, GMRT 2020
While there is and was a lot to share about the event. For e.g. Akshay had bought RPI- Zero as well as RPI-2 (Raspberry Pi s ) and showed some things. He had also bought up a Debian stable live drive with persistence although the glare from the sun was too much that we couldn t show it to clearly to students. This was also the case with RPI but still we shared what and how much we could. Maybe next year, we either ask them to have double screens or give us dark room so we can showcase things much better. We did try playing with contrast and all but it didn t have much of an effect  . Of course in another stall few students had used RPI s as part of their projects so at times we did tell some of the newbies to go to those stalls and see and ask about those projects so they would have a much wider experience of things. The Mozilla people were pushing VR as well as Mozilla lite the browser for the mobile. We also gossiped quite a bit. I shared about indicatelts , a third-party certificate extension although I dunno if I should file a wnpp about it or not. We didn t have a good experience of when I had put an RFP (Request for Package) which was accepted for an extension which had similar functionality which we later come to know was sharing the sites people were using the extension to call home and share both the URL and the IP Address they were using it from. Sadly, didn t leave a good taste in mouth

Delhi Riots One thing I have been disappointed with is the lack of general awareness about things especially in the youth. We have people who didn t know that for e.g. in the Delhi riots which happened recently the law and order (Police) lies with Home Minister of India, Amit Shah. This is perhaps the only capital in the world which has its own Chief Minister but doesn t have any say on its law and order. And this has been the case for last 70 years i.e. since independance. The closest I know so far is the UK but they too changed their tune in 2012. India and especially Delhi seems to be in a time-capsule which while being dysfunctional somehow is made to work. In many ways, it s three body or a body split into three personalities which often makes governance a messy issue but that probably is a topic for another day. In fact, scroll had written a beautiful editorial that full statehood for Delhi was not only Arvind Kejriwal s call (AAP) but also something that both BJP as well as Congress had asked in the past. In fact, nothing about the policing is in AAP s power. All salaries, postings, transfers of police personnel everything is done by the Home Ministry, so if any blame has to be given it has to be given to the Home Ministry for the same.

American Capitalism and Ventilators America had been having a history of high cost healthcare as can be seen in this edition of USA today from 2017 . The Affordable Care Act was signed as a law by President Obama in 2010 which Mr. Trump curtailed when he came into power couple of years back. An estimated 80,000 people died due to seasonal flu in 2018-19 . Similarly, anywhere between 24-63,000 have supposed to have died from Last October to February-March this year. Now if the richest country can t take care of their population which is 1/3rd of the population of this country while at the same time United States has thrice the area that India has. This I am sharing as seasonal flu also strikes the elderly as well as young children more than adults. So in one senses, the vulnerable groups overlap although from some of the recent stats, for Covid-19 even those who are 20+ are also vulnerable but that s another story altogether. If you see the CDC graph of the seasonal flu it is clear that American health experts knew about it. One another common factor which joins both the seasonal flu and covid is both need ventilators for the most serious cases. So, in 2007 it was decided that the number of ventilators needed to be ramped up, they had approximately 62k ventilators at that point in time all over U.S. The U.S. in 2010, asked for bids and got bid from a small californian company called Newport Medic Instruments. The price of the ventilators was approximately INR 700000 at 2010 prices, while Newport said they would be able to mass-produce at INR 200000 at 2010 prices. The company got the order and they started designing the model which needed to be certified by FDA. By 2011, they got the product ready when a big company called Covidgen bought Newport Medic and shutdown the project. This was shared in a press release in 2012. The whole story was broken by New York Times again, just a few days ago which highlighted how America s capitalism rough shod over public health and put people s life unnecessarily in jeopardy. If those new-age ventilators would have been a reality then not just U.S. but India and many other countries would have bought the ventilators as every county has same/similar needs but are unable to pay the high cost which in many cases would be passed on to their citizens either as price of service, or by raising taxes or a mixture of both with public being none the wiser. Due to dearth of ventilators and specialized people to operate it and space, there is possibility that many countries including India may have to make tough choices like Italian doctors had to make as to who to give ventilator to and have the mental and emotional guilt which would be associated with the choices made.

Some science coverage about diseases in wire and other publications Since Covid coverage broke out, the wire has been bringing various reports of India s handling of various epidemics, mysteries, some solved, some still remaining unsolved due to lack of interest or funding or both. The Nipah virus has been amply discussed in the movie Virus (2019) which I shared in the last blog post and how easily it could have been similar to Italy in Kerala. Thankfully, only 24 people including a nurse succumbed to that outbreak as shared in the movie. I had shared about Kerala nurses professionalism when I was in hospital couple of years back. It s no wonder that their understanding of hygeine and nursing procedures are a cut above the rest hence they are sought after not just in India but world-over including US and UK and the middle-east. Another study on respitory illness was bought to my attention by my friend Pavithran.

Possibility of extended lockdown in India There was talk in the media of extended lockdown or better put an environment is being created so that an extended lockdown can be done. This is probably in part due to a mathematical model and its derivatives shared by two Indian-origin Cambridge scholars who predict that a minimum 49 days lockdown may be necessary to flatten the covid curve about a week back.
Predictions of the outcome of the current 21-day lockdown (Source: Rajesh Singh, R. Adhikari, Cambridge University)
Alternative lockdown strategies suggested by the Cambridge model (Source: Rajesh Singh, R. Adhikari, Cambridge University)
India caving to US pressure on Hydroxychloroquine While there has been lot of speculation in U.S. about Hydroxychloroquine as the wonder cure, last night Mr. Trump threatened India in a response to a reporter that Mr. Modi has to say no for Hydroxychloroquine and there may be retaliations.
As shared before if youtube is not your cup you can see the same on invidio.us https://www.invidio.us/watch?v=YP-ewgoJPLw Now while there have been several instances in the past of U.S. trying to bully India, going all the way back to 1954. In fact, in recent memory, there were sanctions on India by US under Atal Bihari Vajpayee Government (BJP) 1998 but he didn t buckle under the pressure and now we see our current PM taking down our own notification from a day ago and not just sharing Hydroxychloroquine but also Paracetemol to other countries so it would look as if India is sharing with other countries. Keep in mind, that India, Brazil haven t seen eye to eye on trade agreements of late and Paracetemol prices have risen in India. The price rise has been because the API (Active Pharmaceutical Ingredients) for the same come from China where the supply chain will take time to be fixed and we would also have to open up, although should we, should we not is another question altogether. I talk about supply chains as lean supply chains were the talk since late 90 s when the Japanese introduced Just-in-time manufacturing which lead to lean supply chains as well as lot of outsourcing as consequence. Of course, the companies saved money but at the cost of flexibility and how this model was perhaps flawed was shared by a series of articles in Economist as early as 2004 when there were lot of shocks to that model and would be exaberated since then. There have been frequent shocks to these fragile ecosystem more since 2008 after the financial meltdown and this would put more companies out of business than ever before. The MSME sector in India had already been severely impacted first by demonetization and then by the horrendous implementation of GST whose cries can be heard from all sectors. Also the frequent changing of GST taxes has made markets jumpy and investors unsure. With judgements such as retrospective taxes, AGR (Adjusted Gross Revenue) etc. it made not only the international investors scared, but also domestic investors. The flight of the capital has been noticeable. This I had shared before when Indian Government shared about LRS report which it hasn t since then. In fact Outlook Business had an interesting article about it where incidentally it talked about localcircles, a community networking platform where you get to know of lot of things and whom I am also a member of. At the very end I apologize for not sharing the blog post before but then I was feeling down but then I m not the only one.

Reproducible Builds: Reproducible Builds in March 2020

Welcome to the March 2020 report from the Reproducible Builds project. In our reports we outline the most important things that we have been up to over the past month and some plans for the future.
What are reproducible builds? One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes.

News The report from our recent summit in Marrakesh was published and is now available in both PDF and HTML formats. A sincere thank you to all of the Reproducible Builds community for the input to the event a sincere thank you to Aspiration for preparing and collating this report. Harmut Schorrig published a detailed document on how to compile Java applications in such as way that the .jar build artefact is reproducible across builds. A practical and hands-on guide, it details how to avoid unnecessary differences between builds by explicitly declaring an encoding as the default value differs across Linux and MS Windows systems and ensuring that the generated .jar a variant of a .zip archive does not embed any nondeterministic filesystem metadata, and so on. Janneke gave a quick presentation on GNU Mes and reproducible builds during the lighting talk session at LibrePlanet 2020. [ ] Vagrant Cascadian presented There and Back Again, Reproducibly! video at SCaLE 18x in Pasadena in California which generated some attention on Twitter. Herv Boutemy mentioned on our mailing list in a thread titled Rebuilding and checking Reproducible Builds from Maven Central repository that since the update of a central build script (the parent POM ) every Apache project using the Maven build system should build reproducibly. A follow-up discussion regarding how to perform such rebuilds was also started on the Apache mailing list. The Telegram instant-messaging platform announced that they had updated their iOS and Android OS applications and claim that they are reproducible according to their full instructions, verifying that its original source code is exactly the same code that is used to build the versions available on the Apple App Store and Google Play distribution platforms respectfully. Herv Boutemy also reported about a new project called reproducible-central which aims to allow anyone to rebuild a component from the Maven Central Repository that is expected to be reproducible and check that the result is as expected. In last month s report we detailed Omar Navarro Leija s work in and around an academic paper titled Reproducible Containers which describes in detail the workings of a user-space container tool called dettrace (PDF). Since then, the PhD student from the University Of Pennsylvania presented on this tool at the ASPLOS 2020 conference in Lausanne, Switzerland. Furthermore, there were contributions to dettrace from the Reproducible Builds community itself. [ ][ ]

Distribution work

openSUSE In openSUSE, Bernhard M. Wiedemann published his monthly Reproducible Builds status update as well as made the following changes within the distribution itself:

Debian Chris Lamb further refined his merge request for the debian-installer component to allow all arguments from sources.list files (such as [check-valid-until=no] ) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure. (#13) Holger Levsen filed a number of bug reports against the debrebuild tool that attempts to rebuild a Debian package given a .buildinfo file as input, including: 48 reviews of Debian packages were added, 17 were updated and 34 were removed this month adding to our knowledge about identified issues. Many issue types were noticed, categorised and updated by Chris Lamb, including: Finally, Holger opened a bug report against the software running tracker.debian.org, a service for Debian Developers to follow the evolution of packages via web and email interfaces to request that they integrate information from buildinfos.debian.net (#955434) and Chris Lamb kept isdebianreproducibleyet.com up to date. [ ]

Software development

diffoscope Chris Lamb made the following changes to diffoscope, the Reproducible Builds project s in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, including preparing and uploading version 138 to Debian:
  • Improvements:
    • Don t allow errors with R script deserialisation cause the entire operation to fail, for example if an external library cannot be loaded. (#91)
    • Experiment with memoising output from expensive external commands, eg. readelf. (#93)
    • Use dumppdf from the python3-pdfminer if we do not see any other differences from pdftext, etc. (#92)
    • Prevent a traceback when comparing two R .rdx files directly as the get_member method will return a file even if the file is missing. [ ]
  • Reporting:
    • Display the supported file formats into the package long description. (#90)
    • Print a potentially-helpful message if the PyPDF2 module is not installed. [ ]
    • Remove any duplicate comparator descriptions when formatting in the --help output or in the package long description. [ ]
    • Weaken Install the X package to get a better output message to may produce a better output as the former is not actually guaranteed. [ ]
  • Misc:
    • Ensure we only parse the recommended packages from --list-debian-substvars when we want them for debian/tests/control generation. [ ]
    • Add upstream metadata file [ ] and add a Lintian override for upstream-metadata-in-native-source as we are upstream. [ ]
    • Inline the RequiredToolNotFound.get_package method s functionality as it is only used once. [ ]
    • Drop the deprecated py36 = [..] argument in the pyproject.toml file. [ ]
In addition, Vagrant Cascadian updated diffoscope in GNU Guix to version 138 [ ], as well as updating reprotest our end-user tool to build same source code twice in widely differing environments and then checks the binaries produced by each build for any differences to version 0.7.14 [ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month we wrote a large number of such patches, including:

Project documentation There was further work performed on our documentation and website this month including Alex Wilson adding a section regarding using Gradle for reproducible builds in JVM projects [ ] and Holger Levsen added the report from our recent summit in Marrakesh [ ][ ]. In addition, Chris Lamb made a number of changes, including correcting the syntax of some CSS class formatting [ ], improved some filed against copy a little better [ ] and corrected a reference to calendar.monthrange Python method in a utility function. [ ]

Testing framework We operate a large and many-featured Jenkins-based testing framework that powers tests.reproducible-builds.org that, amongst many other tasks, tracks the status of our reproducibility efforts as well as identifies any regressions that have been introduced. This month, Chris Lamb reworked the web-based package rescheduling tool to:
  • Require a HTTP POST method in the web-based scheduler as not only should HTTP GET requests be idempotent but this will allow many future improvements in the user interface. [ ][ ][ ]
  • Improve the authentication error message in said rescheduler to suggest that the developer s SSL certificate may have expired. [ ]
In addition, Holger Levsen made the following changes:
  • Add a new ath97 subtarget for the OpenWrt distribution.
  • Revisit ordering of Debian suites; sort the experimental distribution last and reverse the ordering of suites to prioritise the suites in development. [ ][ ][ ]
  • Schedule Debian buster and bullseye a little less in order to allow unstable to catch up on the i386 architecture. [ ][ ]
  • Various cosmetic changes to the web-based scheduler. [ ][ ][ ][ ]
  • Improve wordings in the node health maintenance output. [ ]
Lastly, Vagrant Cascadian updated a link to the (formerly) weekly news to our reports page [ ] and kpcyrd fixed the escaping in an Alpine Linux inline patch [ ]. The usual build nodes maintenance was performed by Holger Levsen [ ][ ], Mattia Rizzolo [ ] and Vagrant Cascadian [ ][ ].
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

This month s report was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

6 April 2020

Joachim Breitner: A Telegram bot in Haskell on Amazon Lambda

I just had a weekend full of very successful serious geekery. On a whim I thought: Wouldn't it be nice if people could interact with my game Kaleidogen also via a telegram bot? This led me to learn about how I write a Telegram bot in Haskell and how I can deploy such a Haskell program to Amazon Lambda. In particular the latter bit might be interesting to some of my readers, so here is how went about it.

Kaleidogen Kaleidogen is a little contemplative game (or toy game where, starting from just unicolored disks, you combine abstract circular patterns to breed more interesting patterns. See my FARM 2019 talk for more details, or check out the source repository. BTW, I am looking for help turning it into an Android app!
KaleidogenBot in action

KaleidogenBot in action

Amazon Lambda Amazon Lambda is the Function as a service offering of Amazon Web Services. The idea is that you don t rent a server, where you have to deal with managing the whole system and that you are paying for constantly, but you just upload the code that responds to outside requests, and AWS takes care of the rest: Starting and stopping instances, providing a secure base system etc. When nobody is using the service, no cost occurs. This sounds ideal for hosting a toy Telegram bot: Most of the time nobody will be using it, and I really don't want to have to baby sit yet another service on my server. On Amazon Lambda, I can probably just forget about it. But Haskell is not one of the officially supported languages on Amazon Lambda. So to run Haskell on Lambda, one has to solve two problems:
  • how to invoke the Haskell code on the server, and
  • how to build Haskell so that it runs on the Amazon Linux distribution

A Haskell runtime for Lambda For the first we need a custom runtime. While this sounds complicated, it is actually a pretty simple concept: A runtime is an executable called bootstrap that queries the Lambda Runtime Interface for the next request to handle. The Lambda documentation is phrased as if this runtime has to be a dispatcher that calls the separate function s handler. But it could just do all the things directly. I found the Haskell package aws-lambda-haskell-runtime which provides precisely that: A function
runLambda :: (LambdaOptions -> IO (Either String LambdaResult)) -> IO ()
that talks to the Lambda Runtime API and invokes its argument on each message. The package also provides Template Haskell magic to collect handlers of any JSON-able type and generates a dispatcher, like you might expect from other, more dynamic languages. But that was too much magic for me, so I ignored that and just wrote the handler manually:
main :: IO ()
main = runLambda run
  where
   run ::  LambdaOptions -> IO (Either String LambdaResult)
   run opts = do
    result <- handler (decodeObj (eventObject opts)) (decodeObj (contextObject opts))
    either (pure . Left . encodeObj) (pure . Right . LambdaResult . encodeObj) result
data Event = Event
    path :: T.Text
  , body :: Maybe T.Text
    deriving (Generic, FromJSON)
data Response = Response
    statusCode :: Int
  , headers :: Value
  , body :: T.Text
  , isBase64Encoded :: Bool
    deriving (Generic, ToJSON)
handler :: Event -> Context -> IO (Either String Response)
handler Event body, path  context =
 
I expose my Lambda function to the world via Amazon s API Gateway, configured to just proxy the HTTP requests. This means that my code receives a JSON data structure describing the HTTP request (here called Event, listing only the fields I care about), and it will respond with a Response, again as JSON. The handler can then simply pattern-match on the path to decide what to do. For example this code handles URLs like /img/CAFFEEFACE.png, and responds with an image.
handler :: TC -> Event -> Context -> IO (Either String Response)
handler Event body, path  context
      Just bytes <- isImgPath path >>= T.decodeHex = do
        let pngData = genPurePNG bytes
        pure $ Right Response
              statusCode = 200
            , headers = object [ "Content-Type" .= ("image/png" :: String) ]
            , isBase64Encoded = True
            , body = T.decodeUtf8 $ LBS.toStrict $ Base64.encode pngData
             
     
isImgPath :: T.Text -> Maybe T.Text
isImgPath  = T.stripPrefix "/img/" >=> T.stripSuffix ".png"
If this program would grow more, then one should probably use something more structured for routing here; maybe servant, or bridging towards wai apps (amost like wai-lamda, but that still assumes an existing runtime, instead of simply being the runtime). But for my purposes, no extra layers of indirection or abstraction are needed!

Deploying Haskell to Lambda Building Haskell locally and deploying to different machiens is notoriously tricky; you often end up depending on a shared library that is not available on the other platform. The aws-lambda-haskell-runtime package, and similar projects like serverless-haskell, solve this using stack and Docker two technologies that are probably great, but I never warmed up to them. So instead adding layers and complexities, can I solve this instead my making things simpler? If I compiler my bootstrap into a static Linux binary, it should run on any Linux, including Amazon Linux. Unfortunately, building Haskell programs statically is also notoriously tricky. But it is made much simpler by the work of Niklas Hamb chen and others in the context of the Nix package manager, coordinated in the static-haskell-nix project. The promise here is that once you have set up building your project with Nix, then getting a static version is just one flag away. The support is not completely upstreamed into nixpkgs proper yet, but their repository has a nix file that contains a nixpkgs set with their patches:
let pkgs = (import (sources.nixpkgs-static + "/survey/default.nix")  ).pkgs; in
This, plus a fairly standard nix setup to build the package, yields what I was hoping for:
$ nix-build -A kaleidogen
/nix/store/ppwyq4d964ahd6k56wsklh93vzw07ln0-kaleidogen-0.1.0.0
$ file result/bin/kaleidogen-amazon-lambda
result/bin/kaleidogen-amazon-lambda: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped
$ ls -sh result/bin/kaleidogen-amazon-lambda
6,7M result/bin/kaleidogen-amazon-lambda
If we put this file, named bootstrap, into a zip file and upload it to Amazon Lambda, then it just works! Creating the zip file is easily scripted using nix:
  function-zip = pkgs.runCommandNoCC "kaleidogen-lambda"  
    buildInputs = [ pkgs.zip ];
    ''
    mkdir -p $out
    cp $ kaleidogen /bin/kaleidogen-amazon-lambda bootstrap
    zip $out/function.zip bootstrap
  '';
So to upload this, I use this one-liner (line-wrapped for your convenience):
nix-build -A function-zip &&
aws lambda update-function-code --function-name kaleidogen \
  --zip-file fileb://result/function.zip
Thanks to how Nix pins all dependencies, I am fairly confident that I can return to this project in 4 months and still be able to build it. Of course, I want continuous integration and deployment. So I build the project with GitHub Actions, using a cachix nix cache to significantly speed up the build, and auto-deploy to Lambda using aws-lambda-deploy; see my workflow file for details.

The Telegram part The above allows me to run basically any stateless service, and a Telegram bot is nothing else: When configured to act as a WebHook, Telegram will send a request with a message to our Lambda function, where we can react on it. The telegram-api package provides bindigs for the Telegram Bot API (although I had to use the repository version, as the version on Hackage has some bitrot). Slightly simplified I can write a handler for an Update:
handleUpdate :: Update -> TelegramClient ()
handleUpdate Update  message = Just m   = do
  let c = ChatId (chat_id (chat m))
  liftIO $ printf "message from %s: %s\n" (maybe "?" user_first_name (from m)) (maybe "" T.unpack (text m))
  if "/start"  T.isPrefixOf  fromMaybe "" (text m)
  then do
    rm <- sendMessageM $ sendMessageRequest c "Hi! I am @KaleidogenBot.  "
    return ()
  else do
    m1 <- sendMessageM $ sendMessageRequest c "One moment "
    withPNGFile  $ \pngFN -> do
      m2 <- uploadPhotoM $ uploadPhotoRequest c
        (FileUpload (Just "image/png") (FileUploadFile pngFN))
      return ()
handleUpdate _ u =
  liftIO $ putStrLn $ "Unhandled message: " ++ show u
and call this from the handler that I wrote above:
     
      path == "/telegram" =
      case eitherDecode (LBS.fromStrict (T.encodeUtf8 (fromMaybe "" body))) of
        Left err ->  
        Right update -> do
          runTelegramClient token manager $ handleUpdate Nothing update
          pure $ Right Response
              statusCode = 200
            , headers = object [ "Content-Type" .= ("text/plain" :: String) ]
            , isBase64Encoded = False
            , body = "Done"
             
     
Note that the Lambda code receives the request as JSON data structure with a body that contains the original HTTP request body. Which, in this case, is itself JSON, so we have to decode that. All that is left to do is to tell Telegram where this code lives:
curl --request POST \
  --url https://api.telegram.org/bot<token>/setWebhook
  --header 'content-type: application/json'
  --data ' "url": "https://api.kaleidogen.nomeata.de/telegram" '
As a little add on, I also created a Telegram game for Kaleidogen. A Telegram game is nothing but a webpage that runs inside Telegram, so it wasn t much work to wrap the Web version of Kaleidogen that way, but the resulting Telegram game (which you can access via https://core.telegram.org/bots/games) still looks pretty neat.

No /dev/dri/renderD128 I am mostly happy with this setup: My game is now available to more people in more ways. I don t have to maintain any infrastructure. When nobody is using this bot no resources are wasted, and the costs of the service are neglectible -- this is unlikely to go beyond the free tier, and even if it would, the cost per generated image is roughly USD 0.000021. There is one slight disappointment, though. What I find most intersting about Kaleidogen from a technical point of view is that when you play it in the browser, the images are not generated by my code. Instead, my code creates a WebGL shader program on the fly, and that program generates the image on your graphics card. I even managed to make the GL rendering code work headlessly, i.e. from a command line program, using EGL and libgbm and a helper written in C. But it needs access to a graphics card via /dev/dri/renderD128. Amazon does not provide that to Lambda code, and neither do the other big Function-as-a-service providers. So I had to swallow my pride and reimplement the rendering in pure Haskell. So if you think the bot is kinda slow, then that s why. Despite properly optimizing the pure implementation (the inner loop does not do allocations and deals only with unboxed Double# values), the GL shader version is still three times as fast. Maybe in a few years GPU access will be so ubiquitous that it s even on Amazon Lambda; then I can easily use that.

Jonathan Carter: Free Software Activities for 2020-03

DPL Campaign 2020 On the 12th of March, I posted my self-nomination for the Debian Project Leader election. This is the second time I m running for DPL, and you can read my platform here. The campaign period covered the second half of the month, where I answered a bunch of questions on the debian-vote list. The voting period is currently open and ends on 18 April.

Debian Social

This month we finally announced the Debian Social project. A project that hosts a few websites with the goal to improve communication and collaboration within the Debian project, improve visibility on the work that people do and make it easier for general users to interact with the community and feel part of the project. Some History This has been a long time in the making. From my side I ve been looking at better ways to share/play our huge DebConf video archives for the last 3 years or so. Initially I was considering either some sort of script or small server side app that combined the archives and the metadata into a player, or using something like MediaDrop (which I was using on my highvoltage.tv website for a while). I ran into a lot of MediaDrop s limitations early on. It was fine for a very small site but I don t think it would ever be the right solution for a Debian-wide video hosting platform, and it didn t seem all that actively maintained either. Wouter went ahead and implemented a web player option for the video archives. His solution is good because it doesn t rely on any server side software, so it s easy to mirror and someone who lives on an island could download it and view it offline in that player. It still didn t solve all our problems though. Popular videos (by either views or likes) weren t easily discoverable, and the site itself isn t that easy to discover. Then PeerTube came along. PeerTube provides a similar type of interface such as MediaDrop or YouTube that gives you likes, viewcount and comments. But what really set it apart from previous things that we looked at was that it s a federated service. Not only does it federate with other PeerTube instances, but the protocols it uses means that it can connect to all kinds of other services that makes up an interconnected platform called the Fediverse. This was especially great since independent video sites tend to become these lonely islands on the web that become isolated and forgotten. With PeerTube, video sites can subscribe to similar sites on the Fediverse, which makes videos and other video sites significantly more discoverable and attracts more eyeballs. At DebConf19 I wanted to ramp up the efforts to make a Debian PeerTube instance a reality. I spoke to many people about this and discovered that some Debianites are already making all kinds of Debian videos in many different languages. Some were even distributing them locally on DVD and have never uploaded them. I thought that the Debian PeerTube instance could not only be a good platform for DebConf videos, but it could be a good home for many free software content creators, especially if they create Debian specific content. I spoke to Rhonda about it, who s generally interested in the Fediverse and wanted to host a instances of Pleroma (microblogging service) and PixelFed (free image hosting service that resembles the Instagram site), but needed a place to host them. We decided to combine efforts, and since a very large amount of fediverse services end with .social in their domain names, we ended up calling this project Debian Social. We re also hosting some non-fediverse services like a WordPress multisite and a Jitsi instance for video chatting. Current Status Currently, we have a few services in a beta/testing state. I think we have most of the kinks sorted out to get them to a phase where they re ready for wider use. Authentication is a bit of a pain point right now. We don t really have a single sign-on service in Debian, that guest users can use, or that all these services integrate with. So for now, if you re a Debian Developer who wants an account on one of these services, you can request a new account by creating a ticket on salsa.debian.org and selecting the New account template. Not all services support having dashes (or even any punctuation in the username whatsoever), so to keep it consistent we re currently appending just guest to salsa usernames for guest users, and team at the end of any Debian team accounts or official accounts using these services Stefano finished uploading all the Debconf videos to the PeerTube instance. Even though it s largely automated, it ended up being quite a big job fixing up some old videos, their metadata and adding support for PeerTube to the DebConf video scripts. This also includes some videos from sprints and MiniDebConfs that had video coverage, currently totaling 1359 videos.
Future plans This is still a very early phase for the project. Here are just some ideas that might develop over time on the Debian Social sites: If you d like to join this initiative and help out, please join #debian-social on oftc. We re also looking for people who can help moderate posts on these sites.

Debian packaging I had the sense that there were fewer upstream releases this month. I suspect that everyone was busy figuring out how to cope during Covid-19 lockdowns taking place all over the world. 2020-03-02: Upload package calamares (3.2.10-1) to Debian unstable. 2020-03-10: Upload package gnome-shell-extension-dash-to-panel (29-1) to Debian unstable. 2020-03-10: Upload package gnome-shell-extension-draw-on-your-screen (5.1-1) to Debian unstable. 2020-03-28: Upload package gnome-shell-extension-dash-to-panel (31-1) to Debian unstable. 2020-03-28: Upload package gnome-shell-extension-draw-on-your-screen (6-1) to Debian unstable. 2020-03-28: Update package python3-flask-autoindexing packaging, not releasing due to licensing change that needs further clarification. (GitHub issue #55). 2020-03-28: Upload package gamemode (1.5.1-1) to Debian unstable. 2020-03-28: Upload package calamares (3.2.21-1) to Debian unstable.

Debian mentoring 2020-03-03: Sponsor package python-jaraco.functools (3.0.0-1) (Python team request). 2020-03-03: Review python-ftputil (3.4-1) (Needs some more work) (Python team request). 2020-03-04: Sponsor package pythonmagick (0.9.19-6) for Debian unstable (Python team request). 2020-03-23: Sponsor package bitwise (0.41-1) for Debian unstable (Email request). 2020-03-23: Sponsor package gpxpy (1.4.0-1) for Debian unstable (Python team request). 2020-03-28: Sponsor package gpxpy (1.4.0-2) for Debian unstable (Python team request). 2020-03-28: Sponsor package celery (4.4.2-1) for Debian unstable (Python team request). 2020-03-28: Sponsor package buildbot (2.7.0-1) for Debian unstable (Python team request).

5 April 2020

Vincent Bernat: Safer SSH agent forwarding

ssh-agent is a program to hold in memory the private keys used by SSH for public-key authentication. When the agent is running, ssh forwards to it the signature requests from the server. The agent performs the private key operations and returns the results to ssh. It is useful if you keep your private keys encrypted on disk and you don t want to type the password at each connection. Keeping the agent secure is critical: someone able to communicate with the agent can authenticate on your behalf on remote servers. ssh also provides the ability to forward the agent to a remote server. From this remote server, you can authenticate to another server using your local agent, without copying your private key on the intermediate server. As stated in the manual page, this is dangerous!
Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent s UNIX-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent. A safer alternative may be to use a jump host (see -J).
As mentioned, a better alternative is to use the jump host feature: the SSH connection to the target host is tunneled through the SSH connection to the jump host. See the manual page and this blog post for more details.
If you really need to use SSH agent forwarding, you can secure it a bit through a dedicated agent with two main attributes: The following alias around the ssh command will spawn such an ephemeral agent:
alias assh="ssh-agent ssh -o AddKeysToAgent=confirm -o ForwardAgent=yes"
With the -o AddKeysToAgent=confirm directive, ssh adds the unencrypted private key to the agent but each use must be confirmed.1 Once connected, you get a password prompt for each signature request:2
ssh-agent prompt confirmation with fingerprint and yes/no buttons
Request for the agent to use the specified private key
But, again, avoid using agent forwarding!

Update (2020-04) In a previous version of this article, the wrapper around the ssh command was a more complex function. Alexandre Oliva was kind enough to point me to the simpler solution above.

Update (2020-04) Guardian Agent is an even safer alternative: it shows and ensures the usage (target and command) of the requested signature. There is also a wide range of alternative solutions to this problem. See for example SSH-Ident, Wikimedia solution and solo-agent.


  1. Alternatively, you can add the keys with ssh-add -c.
  2. Unfortunately, the dialog box default answer is Yes.

3 April 2020

Jonathan Dowland: Opinionated IkiWiki

For various personal projects and things, past and present (including my personal site) I use IkiWiki, which (by modern standards) is a bit of a pain to set up and maintain. For that reason I find it hard to recommend to people. It would be nice to fire up a snapshot of an existing IkiWiki instance to test what the outcome of some changes might be. That's cumbersome enough at the moment that I haven't bothered to do it more than once. Separately, some months ago I did a routine upgrade of Debian for the web server running this site, and my IkiWiki installation broke for the first time in ten years. I've never had issues like this before. For all of these reasons I've just dusted off an old experiment of mine now renamed Opinionated IkiWiki. It's IkiWiki in a container, configured to be usable out-of-the-box, with some opinionated configuration decisions made for you. The intention is you should be able to fire up this container and immediately have a useful IkiWiki instance to work from. It should hopefully be easier to clone an existing wiki content, configuration and all for experimentation. You can check out the source at GitHub, and grab container images from quay.io. Or fire one up immediately at http://127.0.0.1:8080 with something like
podman run --rm -ti -p 8080:8080 \
quay.io/jdowland/opinionated-ikiwiki:latest
This was a good excuse to learn about multi-stage container builds and explore quay.io. Feedback gratefully received: As GitHub issues, comments here, or mail.

2 April 2020

Dirk Eddelbuettel: RQuantLib 0.4.12: Small QuantLib 1.18 update

A new release 0.4.12 of RQuantLib arrived on CRAN today, and was uploaded to Debian as well. QuantLib is a very comprehensice free/open-source library for quantitative finance; RQuantLib connects it to the R environment and language. This version does relatively little. When QuantLib 1.18 came out, I immediately did my usual bit of packaging it for Debian as well creating binaries via my Ubuntu PPA so that I could test the package against it. And a few call from RQuantLib are now hitting interface functions marked as deprecated leading to compiler nags. So I fixed that in PR #146. And today CRAN sent me email to please fix in the released version so I rolled this up as 0.4.12. Not other changes.

Changes in RQuantLib version 0.4.12 (2020-04-01)
  • Changes in RQuantLib code:
    • Calls deprecated-in-QuantLib 1.18 were updated (Dirk in #146).

Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the new rquantlib-devel mailing list. Issue tickets can be filed at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Ulrike Uhlig: Breaking the chain reaction of reactions to reactions

Sometimes, in our day-to-day-interactions, communication becomes disruptive, resembling a chain of reactions to reactions to reactions. Sometimes we lose the capacity to express our ideas and feelings. Sometimes communication just gets stuck, maybe conflict breaks out. When we see these same patterns over and over again, this might be due to the ever same roles that we adopt and play. Learnt in childhood, these roles are deeply ingrained in our adult selves, and acted out as unconscious scripts. Until we notice and work on them. This is a post inspired by contents from my mediation training. In the 1960s, Stephen Karpman has thought of a model of human communication that maps the destructive interactions which occur between people. This map is known as the drama triangle. Karpman defined three roles that interact with each other. We can play one role at work, and a different one at home, and another one with our children. Or we can switch from one role to the other in just one conversation. The three roles are: Does this sound familiar? "Involvement in an unhealthy drama triangle is not something another person is doing to you. It's something you are doing with another person or persons." Well, to be more precise, it's something that we are all doing to each other: "Drama triangles form when participants who are predispositioned to adopt the roles of a drama triangle come together over an issue." (quoted from: Escaping conflict and the Karpman Drama Triangle.) People act out these roles to meet personal (often unconscious) needs. But each of these roles is toxic in that it sees others as problems to react to. In not being able to see that we take on these roles, we keep the triangle going, like in a dispute in which one word provokes another until someone leaves, slamming the door. This is drama. When we are stuck in the drama triangle, no one wins because all three roles "cause pain", "perpetuate shame [and] guilt", and "keep people caught in dysfunctional behavior" (quoted from Lynne Namka: The Drama Triangle, Three Roles of Victim-hood). How to get out of the drama triangle Awareness. To get out of the triangle, it is foremost suggested to be aware of its existence. I agree, it helps. I see it everywhere now. Identifying one's role and starting to act differently. While we switch roles, we generally take on a preferred role that we act out most of the time, and that was learnt in childhood. (I found a test to identify one's common primary role in German.) But how do we act differently? We need to take another look at that uncanny triangle. From the drama triangle to the winner triangle I found it insightful to ask what benefit each role could potentially bring into the interaction.
Acey Choy has created the Winner triangle, in 1990, as an attempt to transform social interactions away from drama. Her winner triangle shifts our perceptions of the roles: the Victim becomes the Vulnerable, the Rescuer becomes the Caring, the Persecutor becomes the Assertive.
Persecutor            Rescuer       Assertive              Caring
I'm right.            I'm good.     I have needs.          I'm listening.
    ----------------------              ----------------------
    \                    /              \                    /
     \                  /                \                  /
      \                /                  \                /
       \              /                    \              /
        \            /                      \            /
         \          /                        \          /
          \        /                          \        /
           \      /                            \      /
            \    /                              \    /
             \  /                                \  /
              \/                                  \/
            Victim                            Vulnerable
            I'm blameless.                    I'm struggling.
Karpman Dreaded Drama Triangle            Choy's Winner Triangle
The Assertive "I have needs." has a calling, aims at change, initiates, and gives feedback. Skills to learn: The Assertive needs to learn to identify their needs, communicate them, and negotiate with others on eye level without shaming, punishing, or belittling them. The Assertive needs to learn to give constructive feedback, without dismissing others. (In the workplace, it could be helpful to have a space for this.) The Assertive could benefit from learning to use I-Statements. The Caring "I'm listening." shows good will and sensitivity, cares, is empathic and supportive. Skills to learn: The Caring needs to learn to respect the boundaries of others: trusting their abilities to think, problem solve and talk for themselves. Therefore, the Caring could benefit from improving their active listening skills. Furthermore the Caring needs to learn to identify and respect their own boundaries and not to do things only because it makes them feel better about themselves. The Vulnerable "I'm struggling." has the skill of seeing and naming problems. Skills to learn: The Vulnerable needs to learn to acknowledge their feelings and needs, practice self-awareness, and self-compassion. They need to untie their self-esteem from the validation of other people. They need to learn to take care of themselves, and to strengthen their problem solving and decision making skills. What has this got to do with autonomy and power structures? Each of these interactions is embedded in larger society, and, as said above, we learn these roles from childhood. Therefore, we perpetually reproduce power structures, and learnt behavior. I doubt that fixing this on an individual level is sufficient to transform our interactions outside of small groups, families or work places. Although that would be a good start. We can see that the triangle holds together because the Victim, seemingly devoid of a way to handle their own needs, transfers care of their needs to the Rescuer, thereby giving up on their autonomy. The Rescuer is provided by the Victim with a sense of autonomy, knowledge, and power, that only works while denying the Victim their autonomy. At the same time, the Persecutor denies everyone else's needs and autonomy, and feels powerful by dismissing others. I've recently mentioned the importance of autonomy in order to avoid burnout, and as a means to control one's own life. If the Rescuer can acknowledge being in the triangle, and give the Victim autonomy, by supporting them with compassion, empathy, and guidance, and at the same time respecting their own boundaries, we could find even more ways to escape the drama triangle. Notes My description of the roles was heavily inspired by the article Escaping Conflict and the Karpman Drama Triangle that has a lot more detail on how to escape the triangle, and how to recognize when we're moving into one of the roles. While the article is informing families living with a person suffering from a spectrum of Borderline Personality Disorder, the content applies to any dysfunctional interaction.

1 April 2020

Sylvain Beucler: Debian LTS and ELTS - March 2020

Debian LTS Logo Here is my transparent report for my work on the Debian Long Term Support (LTS) and Debian Extended Long Term Support (ELTS), which extend the security support for past Debian releases, as a paid contributor. In March, the monthly sponsored hours were split evenly among contributors depending on their max availability - I was assigned 30h for LTS (out of 30 max; all done) and 20h for ELTS (out of 20 max; I did 0). Most contributors claimed vulnerabilities by performing early CVE monitoring/triaging on their own, making me question the relevance of the Front-Desk role. It could be due to a transient combination of higher hours volume and lower open vulnerabilities. Working as a collective of hourly paid freelancers makes it more likely to work in silos, resulting in little interaction when raising workflow topics on the mailing list. Maybe we're reaching a point where regular team meetings will be benefical. As previously mentioned, I structure my work keeping the global Debian security in mind. It can be stressful though, and I believe current communication practices may deter such initiatives. ELTS - Wheezy LTS - Jessie Documentation/Scripts

Next.

Previous.