Search Results: "rul"

17 May 2020

Russ Allbery: krb5-strength 3.2

krb5-strength provides password strength checking for Kerberos KDCs (either MIT or Heimdal), and also provides a password history implementation for Heimdal. This release adds a check-only mode to the heimdal-history command to interrogate history without modifying it and increases the default hash iterations used when storing old passwords. explicit_bzero is now used, where available, to clear the memory used for passwords after processing. krb5-strength can now optionally be built without CrackLib support at all, if you only want to use the word list, edit distance, or length and character class rules. It's been a few years since the previous release, so this release also updates all the portability code, overhauls valgrind testing, and now passes tests when built with system CrackLib (by skipping tests for passwords that are rejected by the stronger rules of the embedded CrackLib fork). You can get the latest release from the krb5-strength distribution page. New packages will be uploaded to Debian unstable shortly (as soon as a Perl transition completes enough to make the package buildable in unstable).

16 May 2020

Sven Hoexter: Quick and Dirty: masquerading / NAT with nftables

Since nftables is now the new default, a short note to myself on how to setup masquerading, like the usual NAT setup you use on a gateway. 
nft add table nat
nft add chain nat postrouting   type nat hook postrouting priority 100 \;  
nft add rule nat postrouting ip saddr 192.168.1.0/24 oif wlan0 masquerade
In this case the wlan0 is basically the "WAN" interface, because I use an old netbook as a wired to wireless network adapter.

14 May 2020

Iustin Pop: New internet provider

Note: all this is my personal experience, on my personal machines, and I don t claim it to be the absolute truth. Also, I don t directly call out names, although if you live in Switzerland it s pretty easy to guess who the old provider is from some hints. For a long time, I wanted to move away from my current (well, now past) provider, for a multitude of reasons. The main being that the company is very classic company, with classic support system, that doesn t work very well - I had troubles with their billing system that left me out cold without internet for 15 days, but for the recent few years, they were mostly OK, and changing to a different provider would have meant me routing a really long Ethernet cable around the apartment, so I kept postponing it. Yes, self-inflicted pain, I know. Until the entire work-from-home thing, when the usually stable connection start degrading in a very linear fashion day-by-day (this is a graph that basically reflects download bandwidth):
1+ month download bandwidth test 1+ month download bandwidth test
At first, I didn t realise this, as even 100Mbps is still fast enough. But once the connection went below (I believe) 50Mbps, it became visible in day to day work. And since I work daily from home yeah. Not fun. So I started doing speedtest.net - and oh my, ~20Mbps was a good result, usually 12-14Mbps. On a wired connection. On a connection that officially is supposed to be 600Mbps down. The upload speed was spot on, so I didn t think it was my router, but: Nothing helped. Once in a blue moon, speedtest would give me 100Mbps, but like once every two days, then it would be back and showing 8Mbps. Eight! It ended up as even apt update was tediously slow, and kernel downloads took ages The official instructions for dealing with bad internet are a joke: And the best part: If you are not satisfied with the results, read our internet optimisation guide. If you are still not happy, use our community forums or our social platforms. Given that it was a decline over 3-weeks, that I don t know of any computer component that would degrade this steadily but not throw any other errors, and that my upload speed was all good, I assumed it s the provider. Maybe I was wrong, but I wanted to do this anyway for a long while, so I went through the find how to route cable, check if other provider socket is good, order service, etc. dance, and less than a week later, I had the other connection. Now, of course, bandwidth works as expected:
1+ month updated bandwidth test 1+ month updated bandwidth test
Both download and upload are fine (the graph above is just download). Latency is also much better, towards many parts of the internet that matter. But what is shocking is the difference in jitter to some external hosts I care about. On the previous provider, a funny thing was that both outgoing and incoming pings had both more jitter and packet loss when done directly (IPv4 to IPv4) than when done over a VPN. This doesn t make sense, since VPN is just overhead over IPv4, but the graphs show it, and what I think happens is that a VPN flow is cached in the provider s routers, whereas a simple ping packet not. But, the fact that there s enough jitter for a ping to a not-very-far host doesn t make me happy. Examples, outgoing:
Outgoing smokeping to public IPv4 Outgoing smokeping to public IPv4
Outgoing smokeping over VPN Outgoing smokeping over VPN
And incoming:
Incoming smokeping to public IPv4 Incoming smokeping to public IPv4
Incoming smokeping over VPN Incoming smokeping over VPN
Both incoming and outgoing show this weirdness - more packet loss and more jitter over VPN. Again, this is not a problem in practice, or not much, but makes me wonder what other shenanigans happen behind the scenes. You can also see clearly when the work from home traffic entered the picture and started significantly degrading my connection, even over the magically better VPN connection. Switching to this week s view shows the (in my opinion) dramatic improvement in consistency of the connection:
Outgoing current week smokeping to public IPv4 Outgoing current week smokeping to public IPv4
Outgoing current week smokeping over VPN Outgoing current week smokeping over VPN
No more packet loss, no more jitter. You can also see my VPN being temporarily down during provider switchover because my firewall was not quite correct for a moment. And the last drill down, at high resolution, one day before and one day after switchover. Red is VPN, blue is plain IPv4, yellow is the missing IPv6 connection :) Incoming old:
Incoming 1-day smokeping, old provider Incoming 1-day smokeping, old provider
and new:
Incoming 1-day smokeping, new provider Incoming 1-day smokeping, new provider
Outgoing old:
Outgoing 1-day smokeping, old provider Outgoing 1-day smokeping, old provider
and new:
Outgoing 1-day smokeping, new provider Outgoing 1-day smokeping, new provider
This is what I expect, ping-over-VPN should of course be slower than plain ping. Note that incoming and outgoing have slightly different consistency, but that is fine for me :) The endpoints doing the two tests are different, so this is expected. Reading the legend on the graphs for the incoming connection (similar story for outgoing): So to my previous provider: it can be done better. Or at least, allow people easier ways to submit performance issue problems. For me, the moral of the story is that I should have switched a couple of years ago, instead of being lazy. And that I m curious to see how IPv6 traffic will differ, if at all :) Take care, everyone! And thanks for looking at these many graphs :)

Norbert Preining: Switching from NVIDIA to AMD (including tensorflow)

I have been using my Geforce 1060 extensively for deep learning, both with Python and R. But the always painful play with the closed source drivers and kernel updates, paired with the collapse of my computer s PSU and/or GPU, I decided to finally do the switch to AMD graphic card and open source stack. And you know what, within half a day I had everything, including Tensorflow running. Yeah to Open Source!
Preliminaries So what is the starting point: I am running Debian/unstable with a AMD Radeon 5700. First of all I purged all NVIDIA related packages, and that are a lot I have to say. Be sure to search for nv and nvidia and get rid of all packages. For safety I did reboot and checked again that no kernel modules related to NVIDIA are loaded. Firmware Debian ships the package amd-gpu-firmware but this is not enough for the current kernel and current hardware. Better is to clone git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git and copy everything from the amdgpu directory to /lib/firmware/amdgpu. I didn t do that at first, and then booting the kernel did hang during the switch to AMD framebuffer. If you see this behaviour, your firmwares are too old. Kernel The advantage of having open source driver that is in the kernel is that you don t have to worry about incompatibilities (like every time a new kernel comes out the NVIDIA driver needs patching). For recent AMD GPUs you need a rather new kernel, I have 5.6.0 and 5.7.0-rc5 running. Make sure that you have all the necessary kernel config options turned on if you compile your own kernels. In my case this is 
CONFIG_DRM_AMDGPU=m
CONFIG_DRM_AMDGPU_USERPTR=y
CONFIG_DRM_AMD_ACP=y
CONFIG_DRM_AMD_DC=y
CONFIG_DRM_AMD_DC_DCN=y
CONFIG_HSA_AMD=y
When installing the kernel, be sure that the firmware is already updated so that the correct firmware is copied into the initrd. Support programs and libraries All the following is more or less an excerpt from the ROCm Installation Guide! AMD provides a Debian/Ubuntu APT repository for software as well as kernel sources. Put the following into /etc/apt/sources.list.d/rocm.list: 
deb [arch=amd64] http://repo.radeon.com/rocm/apt/debian/ xenial main
and also put the public key of the rocm repository into /etc/apt/trusted.d/rocm.asc. After that apt-get update should work. I did install rocm-dev-3.3.0, rocm-libs-3.3.0, hipcub-3.3.0, miopen-hip-3.3.0 (and of course the dependencies), but not rocm-dkms which is the kernel module. If you have a sufficiently recent kernel (see above), the source in the kernel itself is newer. The libraries and programs are installed under /opt/rocm-3.3.0, and to make the libraries available to Tensorflow (see below) and other programs, I added /etc/ld.so.conf.d/rocm.conf with the following content: 
/opt/rocm-3.3.0/lib/
and run ldconfig as root. Last but not least, add a udev rule that is normally installed by rocm-dkms, put the following into /etc/udev/rules.d/70-kfd.rules: 
SUBSYSTEM=="kfd", KERNEL=="kfd", TAG+="uaccess", GROUP="video"
This allows users from the video group to access the GPU.
Up to here you should be able to boot into the system and have X running on top of AMD GPU, including OpenGL acceleration and direct rendering: 
$ glxinfo
ame of display: :0
display: :0  screen: 0
direct rendering: Yes
server glx vendor string: SGI
server glx version string: 1.4
...
client glx vendor string: Mesa Project and SGI
client glx version string: 1.4
...
Tensorflow Thinking about how hard it was to get the correct libraries to get Tensorflow running on GPUs (see here and here), it is a pleasure to see that with open source all this pain is relieved. There is already work done to make Tensorflow run on ROCm, the tensorflow-rocm project. The provide up to date PyPi packages, so a simple 
pip3 install tensorflow-rocm
is enough to get Tensorflow running with Python: 
>> import tensorflow as tf
>> tf.add(1, 2).numpy()
2020-05-14 12:07:19.590169: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libhip_hcc.so
...
2020-05-14 12:07:19.711478: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7444 MB memory) -> physical GPU (device: 0, name: Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT], pci bus id: 0000:03:00.0)
3
>>
Tensorflow for R Installation is trivial again since there is a tensorflow for R package, just run (as a user that is in the group staff, which normally own /usr/local/lib/R) 
$ R
...
> install.packages("tensorflow")
..
Do not call the R function install_tensorflow() since Tensorflow is already installed and functional! With that done, R can use the AMD GPU for computations: 
$ R
...
> library(tensorflow)
> tf$constant("Hellow Tensorflow")
2020-05-14 12:14:24.185609: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libhip_hcc.so
...
2020-05-14 12:14:24.277736: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7444 MB memory) -> physical GPU (device: 0, name: Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT], pci bus id: 0000:03:00.0)
tf.Tensor(b'Hellow Tensorflow', shape=(), dtype=string)
>
AMD Vulkan From the Vulkan home page:
Vulkan is a new generation graphics and compute API that provides high-efficiency, cross-platform access to modern GPUs used in a wide variety of devices from PCs and consoles to mobile phones and embedded platforms.
Several games are using the Vulkan API if available and it is said to be more efficient. There are Vulkan libraries for Radeon shipped in with mesa, in the Debian package mesa-vulkan-drivers, but they look a bit outdated is my guess. The AMDVLK project provides the latest version, and to my surprise was rather easy to install, again by following the advice in their README. The steps are basically (always follow what is written for Ubuntu): All as described in the linked README. Just to make sure, I removed the JSON files /usr/share/vulkan/icd.d/radeon* shipped by Debians mesa-vulkan-drivers package. Finally I deviated a bit by not editing the file /usr/share/X11/xorg.conf.d/10-amdgpu.conf, but instead copying to /etc/X11/xorg.conf.d/10-amdgpu.conf and adding there the section: 
Section "Device"
        Identifier "AMDgpu"
        Option  "DRI" "3"
EndSection
. To be honest, I did not follow the Copy driver and JSON files literally, since I don t want to copy self-made files into system directories under /usr/lib. So what I did is: With this in place, I don t pollute the system directories, and still the new Vulkan driver is available. But honestly, I don t really know whether it is used and is working, because I don t know how to check.
With all that in place, I can run my usual set of Steam games (The Long Dark, Shadow of the Tomb Raider, The Talos Principle, Supraland, ) and I don t see any visual problem till now. As a bonus, KDE/Plasma is now running much better, since NVIDIA and KDE has traditionally some incompatibilities. The above might sound like a lot of stuff to do, but considering that most of the parts are not really packaged within Debian, and all this is rather new open source stack, I was surprised that in half a day I got all working smoothly. Thanks to all the developers who have worked hard to make this all possible.

12 May 2020

Daniel Silverstone: The Lars, Mark, and Daniel Club

Last night, Lars, Mark, and I discussed Jeremy Kun's The communicative value of using Git well post. While a lot of our discussion was spawned by the article, we did go off-piste a little, and I hope that my notes below will enlighten you all as to a bit of how we see revision control these days. It was remarkably pleasant to read an article where the comments section wasn't a cesspool of horror, so if this posting encourages you to go and read the article, don't stop when you reach the bottom -- the comments are good and useful too.
This was a fairly non-contentious article for us though each of us had points we wished to bring up and chat about it turned into a very convivial chat. We saw the main thrust of the article as being about using the metadata of revision control to communicate intent, process, and decision making. We agreed that it must be possible to do so effectively with Mercurial (thus deciding that the mention of it was simply a bit of clickbait / red herring) and indeed Mark figured that he was doing at least some of this kind of thing way back with CVS. We all discussed how knowing the fundamentals of Git's data model improved our ability to work wih the tool. Lars and I mentioned how jarring it has originally been to come to Git from revision control systems such as Bazaar (bzr) but how over time we came to appreciate Git for what it is. For Mark this was less painful because he came to Git early enough that there was little more than the fundamental data model, without much of the porcelain which now exists. One point which we all, though Mark in particular, felt was worth considering was that of distinguishing between published and unpublished changes. The article touches on it a little, but one of the benefits of the workflow which Jeremy espouses is that of using the revision control system as an integral part of the review pipeline. This is perhaps done very well with Git based workflows, but can be done with other VCSs. With respect to the points Jeremy makes regarding making commits which are good for reviewing, we had a general agreement that things were good and sensible, to a point, but that some things were missed out on. For example, I raised that commit messages often need to be more thorough than one-liners, but Jeremy's examples (perhaps through expedience for the article?) were all pretty trite one-liners which perhaps would have been entirely obvious from the commit content. Jeremy makes the point that large changes are hard to review, and Lars poined out that Greg Wilson did research in this area, and at least one article mentions 200 lines as a maximum size of a reviewable segment. I had a brief warble at this point about how reviewing needs to be able to consider the whole of the intended change (i.e. a diff from base to tip) not just individual commits, which is also missing from Jeremy's article, but that such a review does not need to necessarily be thorough and detailed since the commit-by-commit review remains necessary. I use that high level diff as a way to get a feel for the full shape of the intended change, a look at the end-game if you will, before I read the story of how someone got to it. As an aside at this point, I talked about how Jeremy included a 'style fixes' commit in his example, but I loathe seeing such things and would much rather it was either not in the series because it's unrelated to it; or else the style fixes were folded into the commits they were related to. We discussed how style rules, as well as commit-bisectability, and other rules which may exist for a codebase, the adherence to which would form part of the checks that a code reviewer may perform, are there to be held to when they help the project, and to be broken when they are in the way of good communication between humans. In this, Lars talked about how revision control histories provide high level valuable communication between developers. Communication between humans is fraught with error and the rules are not always clear on what will work and what won't, since this depends on the communicators, the context, etc. However whatever communication rules are in place should be followed. We often say that it takes two people to communicate information, but when you're writing commit messages or arranging your commit history, the second party is often nebulous "other" and so the code reviewer fulfils that role to concretise it for the purpose of communication. At this point, I wondered a little about what value there might be (if any) in maintaining the metachanges (pull request info, mailing list discussions, etc) for historical context of decision making. Mark suggested that this is useful for design decisions etc but not for the style/correctness discussions which often form a large section of review feedback. Maybe some of the metachange tracking is done automatically by the review causing the augmentation of the changes (e.g. by comments, or inclusion of design documentation changes) to explain why changes are made. We discussed how the "rebase always vs. rebase never" feeling flip-flopped in us for many years until, as an example, what finally won Lars over was that he wants the history of the project to tell the story, in the git commits, of how the software has changed and evolved in an intentional manner. Lars said that he doesn't care about the meanderings, but rather about a clear story which can be followed and understood. I described this as the switch from "the revision history is about what I did to achieve the goal" to being more "the revision history is how I would hope someone else would have done this". Mark further refined that to "The revision history of a project tells the story of how the project, as a whole, chose to perform its sequence of evolution." We discussed how project history must necessarily then contain issue tracking, mailing list discussions, wikis, etc. There are exist free software projects where part of their history is forever lost because, for example, the project moved from Sourceforge to Github, but made no effort (or was unable) to migrate issues or somesuch. Linkages between changes and the issues they relate to can easily be broken, though at least with mailing lists you can often rewrite URLs if you have something consistent like a Message-Id. We talked about how cover notes, pull request messages, etc. can thus also be lost to some extent. Is this an argument to always use merges whose message bodies contain those details, rather than always fast-forwarding? Or is it a reason to encapsulate all those discussions into git objects which can be forever associated with the changes in the DAG? We then diverted into discussion of CI, testing every commit, and the benefits and limitations of automated testing vs. manual testing; though I think that's a little too off-piste for even this summary. We also talked about how commit message audiences include software perhaps, with the recent movement toward conventional commits and how, with respect to commit messages for machine readability, it can become very complex/tricky to craft good commit messages once there are multiple disparate audiences. For projects the size of the Linux kernel this kind of thing would be nearly impossible, but for smaller projects, perhaps there's value. Finally, we all agreed that we liked the quote at the end of the article, and so I'd like to close out by repeating it for you all... Hal Abelson famously said:
Programs must be written for people to read, and only incidentally for machines to execute.
Jeremy agrees, as do we, and extends that to the metacommit information as well.

6 May 2020

Reproducible Builds: Reproducible Builds in April 2020

Welcome to the April 2020 report from the Reproducible Builds project. In our regular reports we outline the most important things that we and the rest of the community have been up to over the past month. What are reproducible builds? One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. But whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into seemingly secure software during the various compilation and distribution processes.

News It was discovered that more than 725 malicious packages were downloaded thousands of times from RubyGems, the official channel for distributing code for the Ruby programming language. Attackers used a variation of typosquatting and replaced hyphens and underscores (for example, uploading a malevolent atlas-client in place of atlas_client) that executed a script that intercepted Bitcoin payments. (Ars Technica report) Bernhard M. Wiedemann launched ismypackagereproducibleyet.org, a service that takes a package name as input and displays whether the package is reproducible in a number of distributions. For example, it can quickly show the status of Perl as being reproducible on openSUSE but not in Debian. Bernhard also improved the documentation of his unreproducible package to add some example patches for hash issues. [ ]. There was a post on Chaos Computer Club s website listing Ten requirements for the evaluation of Contact Tracing apps in relation to the SARS-CoV-2 epidemic. In particular:
4. Transparency and verifiability: The complete source code for the app and infrastructure must be freely available without access restrictions to allow audits by all interested parties. Reproducible build techniques must be used to ensure that users can verify that the app they download has been built from the audited source code.
Elsewhere, Nicolas Boulenguez wrote a patch for the Ada programming language component of the GCC compiler to skip -f.*-prefix-map options when writing Ada Library Information files. Amongst other properties, these .ali files embed the compiler flags used at the time of the build which results in the absolute build path being recorded via -ffile-prefix-map, -fdebug-prefix-map, etc. In the Arch Linux project, kpcyrd reported that they held their first rebuilder workshop . The session was held on IRC and participants were provided a document with instructions on how to install and use Arch s repro tool. The meeting resulted in multiple people with no prior experience of Reproducible Builds validate their first package. Later in the month he also announced that it was now possible to run independent rebuilders under Arch in a hands-off, everything just works solution to distributed package verification. Mathias Lang submitted a pull request against dmd, the canonical compiler for the D programming languageto add support for our SOURCE_DATE_EPOCH environment variable as well the other C preprocessor tokens such __DATE__, __TIME__ and __TIMESTAMP__ which was subsequently merged. SOURCE_DATE_EPOCH defines a distribution-agnostic standard for build toolchains to consume and emit timestamps in situations where they are deemed to be necessary. [ ] The Telegram instant-messaging platform announced that they had updated to version 5.1.1 continuing their claim that they are reproducible according to their full instructions and therefore verifying that its original source code is exactly the same code that is used to build the versions available on the Apple App Store and Google Play distribution platforms respectfully. Lastly, Herv Boutemy reported that 97% of the current development versions of various Maven packages appear to have a reproducible build. [ ]

Distribution work In Debian this month, 89 reviews of Debian packages were added, 21 were updated and 33 were removed this month adding to our knowledge about identified issues. Many issue types were noticed, categorised and updated by Chris Lamb, including: In addition, Holger Levsen filed a feature request against debrebuild, a tool for rebuilding a Debian package given a .buildinfo file, proposing to add --standalone or --one-shot-mode functionality.
In openSUSE, Bernhard M. Wiedemann made the following changes: In Arch Linux, a rebuilder instance has been setup at reproducible.archlinux.org that is rebuilding Arch s [core] repository directly. The first rebuild has led to approximately 90% packages reproducible contrasting with 94% on the Reproducible Build s project own ArchLinux status page on tests.reproducible-builds.org that continiously builds packages and does not verify Arch Linux packages. More information may be found on the corresponding wiki page and the underlying decisions were explained on our mailing list.

Software development

diffoscope Chris Lamb made the following changes to diffoscope, the Reproducible Builds project s in-depth and content-aware diff utility that can locate and diagnose reproducibility issues (including preparing and uploading versions 139, 140, 141, 142 and 143 to Debian which were subsequently uploaded to the backports repository):
  • Comparison improvements:
    • Dalvik .dex files can also serve as APK containers so restrict the narrower identification of .dex files to files ending with this extension and widen the identification of APK files to when file(1) discovers a Dalvik file. (#28)
    • Add support for Hierarchical Data Format (HD5) files. (#95)
    • Add support for .p7c and .p7b certificates. (#94)
    • Strip paths from the output of zipinfo(1) warnings. (#97)
    • Don t uselessly include the JSON similarity percentage if it is 0.0% . [ ]
    • Render multi-line difference comments in a way to show indentation. (#101)
  • Testsuite improvements:
    • Add pdftotext as a requirement to run the PDF test_metadata text. (#99)
    • apktool 2.5.0 changed the handling of output of XML schemas so update and restrict the corresponding test to match. (#96)
    • Explicitly list python3-h5py in debian/tests/control.in to ensure that we have this module installed during a test run to generate the fixtures in these tests. [ ]
    • Correct parsing of ./setup.py test --pytest-args arguments. [ ]
  • Misc:
    • Capitalise Ordering differences only in text comparison comments. [ ]
    • Improve documentation of FILE_TYPE_HEADER_PREFIX and FALLBACK_FILE_TYPE_HEADER_PREFIX to highlight that only the first 16 bytes are used. [ ]
Michael Osipov created a well-researched merge request to return diffoscope to using zipinfo directly instead of piping input via /dev/stdin in order to ensure portability to the BSD operating system [ ]. In addition, Ben Hutchings documented how --exclude arguments are matched against filenames [ ] and Jelle van der Waa updated the LLVM test fixture difference for LLVM version 10 [ ] as well as adding a reference to the name of the h5dump tool in Arch Linux [ ]. Lastly, Mattia Rizzolo also fixed in incorrect build dependency [ ] and Vagrant Cascadian enabled diffoscope to locate the openssl and h5dump packages on GNU Guix [ ][ ], and updated diffoscope in GNU Guix to version 141 [ ] and 143 [ ].

strip-nondeterminism strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. In April, Chris Lamb made the following changes:
  • Add deprecation plans to all handlers documenting how or if they could be disabled and eventually removed, etc. (#3)
  • Normalise *.sym files as Java archives. (#15)
  • Add support for custom .zip filename filtering and exclude two patterns of files generated by Maven projects in fork mode. (#13)

disorderfs disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues. This month, Chris Lamb fixed a long-standing issue by not drop UNIX groups in FUSE multi-user mode when we are not root (#1) and uploaded version 0.5.9-1 to Debian unstable. Vagrant Cascadian subsequently refreshed disorderfs in GNU Guix to version 0.5.9 [ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including: In addition, Bernhard informed the following projects that their packages are not reproducible:
  • acoular (report unknown non-determinism)
  • cri-o (report a date issue)
  • gnutls (report certtool being unable to extend certificates beyond 2049)
  • gnutls (report copyright year variation)
  • libxslt (report a bug about non-deterministic output from data corruption)
  • python-astropy (report a future build failure in 2021)

Project documentation This month, Chris Lamb made a large number of changes to our website and documentation in the following categories:
  • Community engagement improvements:
    • Update instructions to register for Salsa on our Contribute page now that the signup process has been overhauled. [ ]
    • Make it clearer that joining the rb-general mailing list is probably a first step for contributors to take. [ ]
    • Make our full contact information easier to find in the footer (#19) and improve text layout using bullets to separate sections [ ].
  • Accessibility:
    • To improve accessibility, make all links underlined. (#12)
    • Use an enhanced foreground/background contrast ratio of 7.04:1. (#11)
  • General improvements:
  • Internals:
    • Move to using jekyll-redirect-from over manual redirect pages [ ][ ] and add a redirect from /docs/buildinfo/ to /docs/recording/. (#23)
    • Limit the website self-check to not scan generated files [ ] and remove the old layout checker now that I have migrated all them [ ].
    • Move the news archive under the /news/ namespace [ ] and improve formatting of archived news links [ ].
    • Various improvements to the draft template generation. [ ][ ][ ][ ]
In addition, Holger Levsen clarified exactly which month we ceased to do weekly reports [ ] and Mattia Rizzolo adjusted the title style of an event page [ ]. Marcus Hoffman also started a discussion on our website s issue tracker asking for clarification on embedded signatures and Chris Lamb subsequently replied and asked Marcus to go ahead and propose a concrete change.

Testing framework We operate a large and many-featured Jenkins-based testing framework that powers tests.reproducible-builds.org that, amongst many other tasks, tracks the status of our reproducibility efforts as well as identifies any regressions that have been introduced.
  • Chris Lamb:
    • Print the build environment prior to executing a build. [ ]
    • Drop a misleading disorderfs-debug prefix in log output when we change non-disorderfs things in the file and, as it happens, do not run disorderfs at all. [ ]
    • The CSS for the package report pages added a margin to all <a> HTML elements under <li> ones, which was causing a comma/bullet spacing issue. [ ]
    • Tidy the copy in the project links sidebar. [ ]
  • Holger Levsen:
    • General:
    • Debian:
      • Reduce scheduling frequency of the buster distribution on the arm64 architecture, etc.. [ ][ ]
      • Show builds per day on a per-architecture basis for the last year on the Debian dashboard. [ ]
      • Drop the Subgraph OS package set as development halted in 2017 or 2018. [ ]
      • Update debrebuild to version from the latest version of devscripts. [ ][ ]
      • Add or improve various parts of the documentation. [ ][ ][ ]
    • Work on a Debian rebuilder:
      • Integrate sbuild. [ ][ ][ ][ ][ ]
      • Select a random .buildinfo file and attempt to build and compare the result. [ ][ ][ ][ ]
      • Improve output and related output formatting. [ ][ ][ ][ ][ ]
      • Outline next steps for the development of the tool. [ ][ ][ ]
      • Various refactoring and code improvements. [ ][ ][ ]
Lastly, Mattia Rizzolo fixed some log parsing code regarding potentially-harmless warnings from package installation [ ][ ] and the usual build node maintenance was performed by Holger Levsen [ ][ ][ ] and Mattia Rizzolo [ ][ ][ ].

Misc news On our mailing list this month, Santiago Torres asked whether we were still publishing releases of our tools to our website and Chris Lamb replied that this was not the case and fixed the issue. Later in the month Santiago also reported that the signature for the disorderfs package did not pass its GPG verification which was also fixed by Chris Lamb. Hans-Christoph Steiner of the Guardian Project asked whether there would be interest in making our website translatable which resulted in a WIP merge request being filed against the website and a discussion on how to track translation updates.
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

This month s report was written by Bernhard M. Wiedemann, Chris Lamb, Daniel Shahaf, Holger Levsen, Jelle van der Waa, kpcyrd, Mattia Rizzolo and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

3 May 2020

Sean Whitton: Processing mail from discussion lists differently

I realised this week that for some years I have been applying Inbox Zero indiscriminately to all e-mail that I receive, but this does not make sense, and has some downsides. My version of Inbox Zero works very well when applied to mail addressed directly to me, and for mail to certain mailing lists, where each post to the list might as well be addressed directly to me, in addition to the list. However, I also receive by e-mail the following things to which I now believe Inbox Zero should not be applied: I believe that applying Inbox Zero to these sorts of things is not only incorrect but is actually harming my engagement with these mediums. Let us distinguish
  1. processing e-mail this means applying the Inbox Zero decision procedure to incoming messages, at set times during the day (I do it once, around 4pm)
  2. browsing and sometimes catching up RSS articles and list mail looking through unread items, replying if I think I have something useful to say, leaving things for later, and occasionally marking all as read if I ve not had time for that group of lists lately.
These should be kept apart. The easy case to see is why you shouldn t apply the browsing/catching up approach to e-mail which should rather be processed that s just the original wisdom of Inbox Zero. And clearly you don t want ever to have to resort to just marking all mail addressed directly to you as read. What goes wrong, then, if you misapply the processing mentality to e-mail which should, rather, be browsed/caught up? Well, the core of Inbox Zero is deciding whether to read and reply to something right now, add it to your todo list to handle later, or decide the mail cannot be dealt with quickly but is not important enough to go on that todo list. If you apply this to RSS articles and discussion list mail, then the third option is basically ruled out, because almost nothing on mailing lists or on blogs that I follow is important enough to go on my list of Real Tasks. But then you re faced with either reading the article/post right now, or discarding it. There is no option to leave the item marked as unread and then maybe come back to it, or postpone discarding it until you ve decided to catch up the group. Applying Inbox Zero to discussion lists and RSS feeds creates a false sense of urgency. I ve realised that my implicit response to this has been reluctance to subscribe to new mailing lists and feeds, because I don t have things set up to allow me to read them in a leisurely way. But then I m missing out on discussions and writing that might be relevant and beneficial to me if I could only approach them when I m in a frame of mind other than time to get my inbox down to zero . This also means that subscribing to new mailing lists just in order to post something has an unreasonably high cost: introducing all mail on those lists into my Inbox Zero processing window. So, what s needed is to make the virtual folder views which I use to read new mail correspond cleanly to the distinction between mail to be processed and mail to be browsed. I.e. for each virtual folder it should be clear whether mail there is to be processed or to be browsed. Then I can continue to use my processing views once per day, and access the browsing views at leisure. (Indeed, I ve created a keybinding to cycle through the new browsing views, and another to catch them up.) As I mentioned, some list mail ought to be processed. For example, I want to process rather than browse the debian-policy mailing list, as one of the Policy Editors. With notmuch, it s easy to include this mail in the relevant processing views (I ve long had one processing view for weekdays and another for weekends). Additionally, I ve used a bit of Emacs Lisp to create a dynamic uncategorised unread view which catches mail to be browsed which (i) doesn t show up in one of the other virtual folders for mail to be browsed, and (ii) doesn t show up in one of the processing views. So now subscribing to a mailing list is cheap: mail will end up in the uncategorised view, and I can decide whether to leave it there for a low traffic list; create a new virtual folder for the list (trivial as it s all just more Emacs Lisp, no actual moving of files required); or unsubscribe. I ve been experiencing nostalgia for my time in secondary school when my friends and I had all sorts of interesting stuff flowing into our mailstores, from RSS feeds to each other s blogs where we d post both our own content and links elsewhere, RSS feeds to stranger s blogs, and e-mails sent to each other with links. We didn t have to worry about processing anything back then, as our e-mail accounts were used for intellectual enrichment but not the completion of tasks. I think I can recapture something of that with my new virtual folders for browsing.

30 April 2020

Chris Lamb: Free software activities in April 2020

Here is my monthly update covering what I have been doing in the free software world during April 2020 (previous month's report). Looking it over prior to publishing, I am surprised how much I got done this month I felt that I was not only failing to do all the extra things I had planned, but I was doing far less than normal. But let us go easy on ourselves; nobody is nailing this. In addition, I did more hacking on the Lintian static analysis tool for Debian packages:
Reproducible builds One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter. Elsewhere in our tooling, I made the following changes to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, including preparing and uploading versions 139, 140, 141 and 142 to Debian: Lastly, I made a large number of changes to our website and documentation in the following categories:
Debian LTS This month I have contributed 18 hours to Debian Long Term Support (LTS) and 7 hours on its sister Extended LTS project. You can find out more about the project via the following video:
Debian I only filed three bugs in April, including one against snapshot.debian.org to report that a Content-Type HTTP header is missing when downloading .deb files (#956471) and to report build failures in the macs & ruby-enumerable-statistics packages:

29 April 2020

Ian Jackson: subdirmk 1.0 - ergonomic preprocessing assistant for non-recursive make

I have made the 1.0 release of subdirmk. subdirmk is a tool to help with writing build systems in make, without use of recursive make. Why Peter Miller's 1997 essay Recursive Make Considered Harmful persuasively argues that it is better to arrange to have a single make invocation with the project's complete dependency tree, rather than the conventional $(MAKE) -C subdirectory approach. This has become much more relevant with modern projects which tend to be large and have deep directory trees. Invoking make separately for each of these subdirectories can be very slow. Nowadays everyone needs to run a parallel build, but with the recursive make approach great discipline is needed to avoid introducing races which cause the build to sometimes fail. There are various new systems which aim to replace make. My general impression of these is that they mostly threw away the good parts of make (often, they discard the flexibility, and the use of the shell command as the basic unit of execution, making them hard to extend), or make other unfortunate assumptions. And there are a lot of programming-language-specific systems - a very unsatisfactory development. Having said all that, I admit I haven't properly evaluated every make competitor. Other reasons for staying with make including that it is widely available, relatively widely understood, and has a model relatively free of high-level abstract concepts. (I like my languages with high-level concepts, but not my build systems.) But, with make, I found that actually writing a project's build system in non-recursive make was not very ergonomic. So with some help and prompting from Mark Wooding, I have made a tool to help. What subdirmk is a makefile preprocessor and aggregator, typically run from autoconf. subdirmk provides convenience syntaxes for references to per-directory variables and pathnames. It also helps by providing a little syntactic sugar for GNU make's macro facilities, which are awkward to use in raw make. subdirmk's features are triggered by the sigil &. The syntax is carefully designed to avoid getting in the way of makefile programming (and programming of shell commands in make rules). subdirmk is fully documented in the README. There is a demo in the example directory (which also serves as part of the test suite). What's new The version number. I have not felt the need to make any changes since releasing 0.4 in mid-February. The last non-docs change was a (backwards-compatible) extension, in late January, to pass through unaltered GNU make's new grouped multiple targets syntax. Advantages and disadvantages of subdirmk Compared to recursive make, subdirmk is easier and simpler, although you do have to decorate a lot of your variables and filenames with & to indicate that they are directory-local. It is much easier to avoid writing parallel make bugs. You naturally get properly working per-subdirectory targets. subdirmk-based nonrecursive make is much, much faster than recursive make. Compared to many other recent build system tools, subdirmk retains all the flexibility and extensibility of make, and operates at a fairly low level of abstraction. subdirmk-based makefiles can easily invoke other build systems. make knows it's not the only thing in the universe. You can adopt subdirmk incrementally or partially, gradually bringing your recursive submakefiles into the unified build. The build system code in subdirmk's Dir.sd.mk files will be readily navigable by most readers; much will be familiar. Because subdirmk is a small collection of (fairly simple) scripting and makefile code, there is no need to build it; you can simply ship it with your project using git-subtree. For an autoconf-based project, there need be no change to how your users and downstreams invoke your build. On the other hand the price you (continue to) pay is make's punctation soup, which subdirmk adds a new sigil to. subdirmk-based makefiles are terse and help you use make's facilities to abstract away repetition, but that can make them dense. The new & sigil will faze some readers. Currently, the provided mechanism for incorporating subdirmk into your project assumes you are using autoconf but not automake. It would be possible to use subdirmk with autoconf-less projects, or with automake-based ones, but I haven't done the glue work to make that easy. subdirmk does require GNU make and it assumes you have perl installed. But GNU make is very portable, and perl is very widely available. (The perl used is very conservative.) The make competitors are, themselves, even less standard build tools. I don't think a build-dependency on GNU make, or perl, is a significant barrier nowadays, for most projects. Note about comment moderation I have deliberately been vague about other build systems and avoided specific criticisms or references. I don't want the comments to become a build system advocacy debate. Comments may be screened and moderated accordingly. Pointers to other obscure build system tools are very welcome. If you want to write a survey of build tools, or a critique of subdirmk, please do so on your own blog; I would be happy to consider linking to it.

comment count unavailable comments

25 April 2020

Reinhard Tartler: Building Packages with Buildah in Debian

1 Building Debian Packages with buildah
Building packages in Debian seems to be a solved problem. But is it? At the bottom, installing the dpkg-dev package provides all the basic tools needed. Assuming that you already succeeded with creating the necessary packaging metadata (i.e., debian/changelog, debian/control, debian/copyright, etc., and there are great helper tools for this such ash dh-make, dh-make-golang, etc.,) it should be as simple as invoking the dpkg-buildpackage tool. So what's the big deal here?

The issue is that dpkg-buildpackage expects to be called with an appropriately setup build context, that is, it needs to be called in an environment that satisfies all build dependencies on the system. Let's say you are building a package for Debian unstable on your Debian stable system (this is the common scenario for the official Debian build machines), you would need your build to link against libraries in unstable, not stable. So how to tell the package build process where to find its dependencies?

The answer (in Debian and many other Linux distributions) is you do not at all. This is actually a somewhat surprising answer for software developers without a Linux distribution development background1. Instead, chroots "simulate" an environment that has all dependencies that we want to build against at the system locations, that is. /usr/lib, etc.

Chroots are basically full system installations in a subdirectory that includes system and application libraries. In order to use them, a package build needs to use the chroot(2) system call, which is a privileged operation. Also creating these system installations is a somewhat finicky process. In Debian, we have tools that make this process easier, the most popular ones are probably pbuilder(1)2 and sbuild(1)3. Still, they are somewhat clumsy to use, add significant levels of abstraction in the sense that they do quite a bit of magic behind the scenes to work hide the fact that privileged operations (need root to run chroot(2), etc.) are required. They are also somewhat brittle, for instance, if a build process is aborted (SIGKILL, or system crash), you may end up with temporary directories and files under userids other than your own that again may require root-privileges to cleanup.

What if there was an easy way to do all of the above without any process running as root? Enter rootless buildah.

Modern Linux container technologies allow unprivileged users to "untie" a process from the system (cf. the unshare(2) system call). This means that a regular user may run a process (and its child processes) in an "environment" where system calls behave differently and provide a configurable amount of isolation levels. This article demonstrates a novel build tool buildah, which is:
  • easy to setup build environment from the command line
  • secure, as no process needs to run as root
  • simple in architecture, requires no running daemon like for example docker
  • convenient to use: you can debug your debian/rules interactively
Architecturally, buildah is written in golang and compiled as a (mostly) statically linked executable. It builds on top a number of libraries written in golang, including github.com/containers/storage and github.com/containers/image. The overlay functionality is provided by the fuse-overlayfs(1) utility.

1.1 Preparation:
The Kernel in Debian bullseye (and in buster, and recent Ubuntu kernels to the best of my knowledge) do support usernamespaces, but leave them disabled by default. Here is how to enable them:
echo kernel.unprivileged_userns_clone = 1   sudo tee -a /etc/sysctl.d/containers.conf
systemctl -p /etc/sysctl.d/containers.conf
I have to admit that I'm not entirely sure why the Debian kernels don't enable usernamespaces by default. I've found a reference on stackexchange4 that claims this disables some "hardening" features in the Debian kernel. I also understand this step is not necessary if you chose to compile and run a vanilla upstream kernel. I'd appreciate a better reference and am happy to update this text.
$ c=$(buildah from docker.io/debian:sid)
Getting image source signatures
Copying blob 2bbc6b8c460d done
Copying config 9b90abe801 done
Writing manifest to image destination
Storing signatures
This command downloads the image from docker.io and stores it locally in your home directory. Let's install essential utilities for building Debian packages:
1: buildah run $c apt-get update -qq
2: buildah run $c apt-get install dpkg-dev -y
3: buildah config  --workingdir /src $c
4: buildah commit $c dpkg-dev
The command on line 1 and 2 execute the installation of compilers and dpkg development tools such as dpkg-buildpackage, etc. The buildah config command in line 4 arranges that whenever you start a shell in the container, the current working directory in the container is in /src. Don't worry about this location not existing yet, we will make sources from your host system available there. The last command creates an OCI image with the name dpkg-dev. BTW, the name you use for the image in the commit command can be used in podman (but not the "containers"). See 5 and 6 for a comparison between podman and buildah.
buildah images -a
This output might look like this:
  REPOSITORY                 TAG      IMAGE          ID   CREATED   SIZE             
  localhost/dpkg-dev         latest   b85c34f95d3e   16   seconds   ago    406   MB  
  docker.io/library/debian   sid      9b90abe801db   11   hours     ago    124   MB

1.2 Running a package build
Now we have a working container with the reference in the variable $c. To use it conveniently with source packages that I have stored in /srv/scratch/packages/containers, let's introduce a shell alias r like this:
alias r='buildah run --tty -v /srv/scratch/packages/containers:/src $c '
This allows you to easily execute commands in that container:
r pwd
r bash
The last command will give you an interactive shell that we'll be using for building packages!
siretart@x1:~/scratch/packages/containers/libpod$ r bash

root@x1:/src# cd golang-github-openshift-imagebuilder

root@x1:/src/golang-github-openshift-imagebuilder# dpkg-checkbuilddeps
dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper (>= 11) dh-golang golang-any golang-github-containers-storage-dev (>= 1.11) golang-github-docker-docker-dev (>= 18.09.3+dfsg1) golang-github-fsouza-go-dockerclient-dev golang-glog-dev

root@x1:/src/golang-github-openshift-imagebuilder# apt-get build-dep -y .
Note, using directory '.' to get the build dependencies
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
[...]

root@x1:/src/golang-github-openshift-imagebuilder# dpkg-checkbuilddeps

root@x1:/src/golang-github-openshift-imagebuilder#
Now we have a environment that has all build-dependencies available and we are ready to build the package:
root@x1:/src/golang-github-openshift-imagebuilder# dpkg-buildpackage -us -uc -b
Assuming the package builds, the package build results are placed in /src inside the container, and are visible at ~/scratch/packages/containers on the host. There you can inspect, extract or even install them. The latter part allows you to interactively rebuild packages against updated dependencies, without the need of setting up an apt archive or similar.

2 Availability
The buildah(1) tool is available in debian/bullseye since 2019-12-17. Since is a golang binary, it only links dynamically against system libraries such as libselinux and libseccomp, which are all available in buster-backports. I'd expect buildah to just work on a debian/buster system as well provided you install those system libraries and possibly the backported Linux kernel.

Keeping the package at the latest upstream version is challenging because of its fast development pace that picks up new dependencies on golang libraries and new versions of existing libraries with every new upstream release. In general, this requires updating several source packages, and in many cases also uploading new source packages to the Debian archive that need to be processed by the Debian ftp-masters.

As an exercise, I suggest to install the buildah package from bullseye, 'git clone' the packaging repository from https://salsa.debian.org/go-team/packages/golang-github-containers-buildah and build the latest version yourself. Note, I would expect the above to even work on a Fedora laptop.

The Debian packages have not swept into the Ubuntu distributions yet, but I expect them to be included in the Ubuntu 20.10 release. In the mean time, ubuntu users can install the package that are provided by the upstream maintainers in the "Project Atomic" PPA at https://launchpad.net/~projectatomic/+archive/ubuntu/ppa

3 Related Tools
The buildah tool is accompanied with two "sister" tools:

The Skopeo package provides tooling to work with remote images registries. This allows you download, upload images to remote registries, convert container images between different formats. It has been available in Debian since 2020-04-20.

The podman tool is a 'docker' replacement. It provides a command-line interface that mimics the original docker command to an extend that a user familiar with docker might want to place this in their ~/.bashrc file:
alias docker='podman'
Unfortunately, at the time of writing podman is still being processed by the ftp-masters since 01-03-2020. At this point, I recommend building the package from our salsa repository at https://salsa.debian.org/debian/libpod

4 Conclusion
Building packages in the right build context is a fairly technical issue for which many tools have been written for. They come with different trade-off when it comes to usability. Containers promise a secure alternative to the tried and proven chroot-based approaches, and the buildah makes using this technology very easy to use.

I'd love to get into a conversation with you on how these tools work for you, and would like to encourage participation and assistance with keeping the complicated software stack up-to-date in Debian (and by extension, in derived distribution such as Ubuntu, etc.).

Footnotes:
1
At my day-job, we build millions of lines of C++ code on Solaris10 and AIX6, where concepts such as "chroots" are restricted to the super user 'root' and is therefore not available to developers, not even through wrappers. Instead, libraries and headers are installed into "refroots", that is, subdirectories that mimic the structure of the "sysroot" directories that are used in the embedded Linux community for cross-compiling packages, and we use Makefiles that set include flags (-I rules and -L flags) to tell the compiler and linker where to look.
2
http://wiki.debian.org/pbuilder
3
http://wiki.debian.org/sbuild
4
https://security.stackexchange.com/questions/209529/what-does-enabling-kernel-unprivileged-userns-clone-do
5
https://developers.redhat.com/blog/2019/02/21/podman-and-buildah-for-docker-users/
6
https://podman.io/blogs/2018/10/31/podman-buildah-relationship.html

13 April 2020

Shirish Agarwal: Migrant worker woes and many other stories

I was gonna use this blog post to share about the migrant worker woes as there has been multiple stories doing the rounds. For e.g. a story which caught the idea of few people but most of us, i.e. middle-class people are so much into our own thing that we care a fig leaf about what happens to migrants. This should not be a story coming from a humane society but it seems India is no different than any other country of the world and in not a good way. Allow me to share
Or for those who don t like youtube, here s an alternative link https://www.invidio.us/watch?v=JGEgZq_1jmc Now the above two editorial shares two stories, one of Trump retaliatory threat to India in the Q&A of the journalist. In fact, Trump has upped the ante on visa sanctions as India buckled so easily under pressure. There have been other stories doing the rounds how people who have illnesses who need HCQ in India are either dying or are close to death because of unavailability of HCQ in the medicine shop. There have been reports in Pune as well as South Mumbai (one of the poshest localities in Mumbai/Bombay) that medicine shops are running empty or emptier. There have been so many stories on that, with reporters going to shops and asking owners of the medicine shops and shop-owners being clueless. I think the best article which vividly describes the Government of India (GOI) response to the pandemic is the free-to-read article shared by Arundhati Roy in Financial Times. It has reduced so much of my work or sharing that it s unbelievable. And she has shared it with pictures and all so I can share other aspects of how the pandemic has been affecting India and bringing the worst out in the Government in its our of need. In fact, not surprisingly though, apparently there was also a pro-Israel similar thing which happened in Africa too . As India has too few friends now globally, hence it decided to give a free pass to them.

Government of India, news agencies and paid News One of the attempts the state tried to do, although very late IMHO is that it tried to reach out to the opposition i.e. Congress party and the others. Mrs. Sonia Gandhi, who is the Congress president asked that the Government should not run any of its ads on private television channels for a period of two years. There had been plenty of articles, both by medianama and others who have alleged that at least from the last 6 odd years, Government ads. comprise of almost 50-60% advertising budget of a channel advertising budget. This has been discussed also in medianama s roundtable on online content which happened few months back. While an edited version is out there on YT, this was full two day s event which happened across two different cities.
or the alternative to youtube https://www.invidio.us/watch?v=c1PhWR1-Urs It was as if the roundtable discussions were not enough, Mrs. Gandhi clarion call was answered by News Broadcaster s Association (NBA) and this is what they had to say
News Broadcasters Association reply to Mrs. Gandhi
To put it simply, NBA deplored the suggestion by Mrs. Gandhi and even called the economy in recession and all they had were the Government s own advertising budget to justify their existence. The statements in themselves are highly pregnant and reveal both the relationship that the media, print or mainstream news channels have with the Government of India. Now if you see that, doesn t it make sense that media always slants the story from the Government s perspective rather than remaining neutral. If my bread basket were on the onus of me siding with the Govt. that is what most sane persons would do, otherwise they would resign and leave which many reporters who had a conscience did. Interestingly enough, the NBA statement didn t just end there but also used the word recession , this is the term that Government of India (GOI) hates and has in turn has been maintaining the word, terminology slowdown . While from a layman s perspective the two terms may seem to be similar, if India has indeed been in recession then the tools and the decisions that should have been taken by GOI should have been much different than what they took. Interestingly, enough GOI has refrained from saying anything on the matter which only reveals their own interests in the matter. Also if an association head is making the statement, it is more than likely that he consulted a lawyer or two and used application of mind while drafting the response. In other words, or put more simply, this was a very carefully drafted letter because they know that tomorrow the opposition party may come into power so they don t want to upset the power dynamics too much.

Privacy issues arising due to the Pandemic On the same Financial Times, two stories which dealt with the possible privacy violations due to the Pandemic have been doing the rounds. The first one, by Yuval Noah Harari is more exploratory by nature and makes some very good points without going far too deep into specific instances of recent times but rather goes into history and past instances where Governments have used the pandemics to exert more control over their populace and drive their agenda. I especially liked the last few lines which he shared in his op-ed Even if the current administration eventually changes tack and comes up with a global plan of action, few would follow a leader who never takes responsibility, who never admits mistakes, and who routinely takes all the credit for himself while leaving all the blame to others. Yuval Noah Harari . The whole statement could right fit onto the American President which he was talking about while at the same time, fits right into the current Indian Prime Minister, Boris Johnson of UK and perhaps Jair Bolsanaro of Brazil. All these three-four individuals have in common is that most of them belong to right-wing and hence cater only to the rich industrialist s agenda. While I don t know about Jair Bolsanaro much, at least three out of four had to turn to socialism and had to give some bailout packages to the public at large, even though continuing to undermine their own actions. More on this probably a bit down the line. The second story shared by Nic Fildes and Javier Espinoza who broke the story of various surveillance attempts and the privacy concerns that people have. Even the Indian PMO has asked this data and because there was no protest by the civil society, a token protest was done by COAI (Cellular Operator Association of India) but beyond that nothing, I am guessing because the civil society didn t make much noise as everybody is busy with their own concerns of safety and things going on, it s possible that such data may have gone to the Government. There is not much new here that people who had been working on the privacy issues know, it s just how easy Governments are finding to do it. The part of informed consent is really a misnomer . Governments lie all the time, for e.g. in the UK, did the leave party and people take informed consent, no they pushed their own agenda. This is and will be similar in many countries of the world.

False Socialism by RW parties In at least the three countries I have observing, simply due to available time, that lot of false promises are being made by our leaders and more often than not, the bailouts will be given to already rich industrialists. An op-ed by Vivek Kaul, who initially went by his handle which means somebody who is educated but unemployed. While Vivek has been one-man army in revealing most of the Government s mischiefs especially as fudging numbers are concerned among other things, there have been others too. As far as the US is concerned, an e-zine called free press (literally) has been sharing Trump s hollowness and proclamations for U.S. . Far more interestingly, I found New York times investigated and found a cache of e-mails starting from early January, which they are calling Red Dawn . The cache is undeniable proof that medical personnel in the U.S. were very much concerned since January 2020 but it was only after other countries started lock-down that U.S. had to follow suit. I am sure Indian medical professionals may have done similar mail exchanges but we will never know as the Indian media isn t independent enough.

Domestic violence and Patriarchy There have been numerous reports of domestic violence against women going up, in fact two prominent publications have shared pieces about how domestic violence has gone up in India since the lockdown but the mainstream press is busy with its own tropes, the reasons already stated above. In fact, interestingly enough, most women can t wear loose fitting clothes inside the house because of the near ones being there 24 7 . This was being shared as India is going through summer where heat waves are common and most families do not have access to A/C s and rely on either a fan or just ventilation to help them out. I can t write more about this as simply I m not a woman so I haven t had to face the pressures that they have to every day. Interestingly though, there was a piece shared by arre. Interestingly, also arre whose content I have shared a few times on my blog has gone from light, funny to be much darker and more serious tone. Whether this is due to the times we live in is something that a social scientist or a social anthropologist may look into in the times to come. One of the good things though, there hasn t been any grid failures as no industrial activity is happening (at all). In fact SEB s (State Electricity Boards) has shown a de-growth in electricity uptake as no industrial activity has been taken. While they haven t reduced any prices (which they ideally should have) as everybody is suffering.

Loot and price rise Again, don t think it is an Indian issue but perhaps may be the same globally. Because of broken supply chains, there are both real and artificial shortages happening which is leading to reasonable and unreasonable price hikes in the market. Fresh veggies which were normally between INR 10/- to INR 20/- for 250 gm have reached INR 40/- 50/- and even above. Many of the things that we have to become depend upon are not there anymore. The shortage of plastic bottles being case in point.
Aryan Plastic bottle
This and many others like these pictures have been shared on social media but it seems the Government is busy doing something else. The only thing we know for sure is that the lock-down period is only gonna increase, no word about PPE s (Personal Protection Equipment) or face masks or anything else. While India has ordered some, those orders are being diverted to US or EU. In fact, many doctors who have asked for the same have been arrested, sacked or suspended for asking such inconvenient questions, although whether in BJP ruled states or otherwise. In fact, the Centre has suspended MPLADS funds , members of parliament get funds which they can use to provide relief work or whatever they think the money is best to spend upon.

Conditions of Labor in the Pandemic Another sort of depressing story has been how the Supreme Court CJI Justice SA Bobde has made statements and refrained from playing any role in directing the Center to provide relief to the daily wage laborers. In fact, Mr. Bobde made statements such as why they need salaries if they are getting food. This was shared by barandbench, a site curated by lawyers and reporters alike. Both livelaw as well as barandbench have worked to enhance people s awareness about the legal happenings in our High Courts and Supreme Court. And while sadly, they cannot cover all, they at least do attempt to cover a bit of what s hot atm. The Chief Justice who draws a salary of INR 250,000 per month besides other perks is perhaps unaware or doesn t care about fate of millions of casual workers, 400 460 million workers who will face abject poverty and by extension even if there are 4 members of the family so probably 1.2 billion people will fall below the poverty line. Three, four major sectors are going to be severely impacted, namely Agriculture, Construction and then MSME (Micro, small and medium enterprises) which cover everything from autos, industrial components, FMCG, electronics, you name it, it s done by the MSME sector. We know that the Rabi crop, even though it was gonna be a bumper crop this year will rot away in the fields. Even the Kharif crop whose window for sowing is at the most 2-3 weeks will not be able to get it done in time. In fact, with the extended lockdown of another 21 days, people will probably return home after 2 months by which time they would have nothing to do there as well as here in the cities. Another good report was done by the wire, the mainstream media has already left the station.

Ministry of Public Health There was an article penned by Dr. Edmond Fernandes which he published last year. The low salary along with the complexities that Indian doctors are and may face in the near future are just mind-boggling.

The Loss Losses have already started pouring in. Just today Air Deccan has ceased all its operations. I had loved Mr. Gopinath s airline which was started in the early 2000 s. While I won t bore you with the history, most of it can be seen from simplify Deccan . This I believe is just the start and it s only after the few months after the lock-down has been lifted would we really know the true extent of losses everywhere. And the more lenghthier the lockdown, the more difficult it would be businesses to ramp back. People have already diagnosed at the very least 15-20 sectors of the economy which would be hit and another similar or more number of sectors which will have first and second-order of losses and ramp-downs. While some guesses are being made, many are wildly optimistic and many are wildly pessimistic, as shared we would only know the results when the lockdown is opened up.

Predictions for the future While things are very much in the air, some predictions can be made or rationally deduced. For instance, investments made in automation and IT would remain and perhaps even accelerate a little. Logistics models would need to be re-worked and maybe, just maybe there would be talk and action in making local supply chains a bit more robust. Financing is going to be a huge issue for at least 6 months to a year. Infrastructure projects which require huge amount of cash upfront will either have to be re-worked or delayed, how they will affect projects like Pune Metro and other such projects only time will tell.

Raghuram Rajan Raghuram Rajan was recently asked if he would come back and let bygones be bygones. Raghuram in his own roundabout way said no. He is right now with Chicago Booth doing the work that he always love. Why would he leave that and be right in the middle of the messes other people have made. He probably gets more money, more freedom and probably has a class full of potential future economists. Immigration Control, Conferences and thought experiment There are so many clueless people out there, who don t know why it takes so long for any visa to be processed. From what little I know, it is to verify who you say you are and you have valid reason to enter the country. The people from home ministry verify credentials, as well as probably check with lists of known criminals and their networks world-wide. They probably have programs for such scenarios and are part and parcel of their everyday work. The same applies to immigration control at Airports. there has been a huge gap at immigration counters and the numbers of passengers who were flying internationally to and fro from India. While in India, we call them as Ministry of Home Affairs, in U.S. it s Department of Homeland security, other countries using similar jargons. Now even before this pandemic happened, the number of people who are supposed to do border control and check people was way less and there have been scenes of Air rage especially in Indian airports after people came after a long-distance flight. Now there are couple of thought experiments, just day before yesterday scientists discovered six new coronaviruses in bats and scientists in Iceland found 40 odd mutations of the virus on people. Now are countries going to ban people from Iceland as in time the icelandic people probably would have anti-bodies on all the forty odd mutations. Now if and when they come in contact onto others who have not, what would happen ? And this is not specifically about one space or ethnicity or whatever, microbes and viruses have been longer on earth than we have. In our greed we have made viruses resistant to antibiotics. While Mr. Trump says as he discovered it today, this has been known to the medical fraternity since tht 1950 s. CDC s own chart shows it. We cannot live in fear of a virus, the only way we can beat it is by understanding it and using science. Jon Cohen shared some of the incredible ways science is looking to beat this thing
or as again an alternative to youtube https://www.invidio.us/watch?v=MPVG_n3w_vM One of the most troubling question is how the differently-abled communities which don t have media coverage at the best of times, haven t had any media coverage at all during the pandemic. What are their stories and what they are experiencing ? How are they coping ? Are there anyways we could help each other ? By not having those stories, we perhaps have left them more vulnerable than we intend. And what does that speak about us, as people or as a community or a society ?

Silver Linings While there is not a lot to be positive about, one interesting project I came about is openbreath.tech . This is an idea, venture started by IISER (Indian Institute of Science Education and Research) , IUCAA (Inter-University Centre for Astronomy and Astrophysics). They are collaborating with octogeneraian Capt (Retd) Rustom Barucha from Barucha Instrumentation and Control, besides IndoGenius, New Delhi, and King s College, London. The first two institutes are from my home town, Pune. While I don t know much of the specifics of this idea other than that there is an existing Barucha ventilator which they hope to open-source and make it easier for people to produce their own. While I have more questions than answers at this point, this is something hopefully to watch out for in the coming days and weeks. The other jolly bit of good news has come from Punjab where after several decades, people in Northern Punjab are finally able to see the Himalayas or the Himalayan mountain range.
Dhauladhar range Northern Punjab Copyright CNN.Com
There you have it, What I have covered is barely scratching the surface. As a large section of the media only focuses on one narrative, other stories and narratives are lost. Be safe, till later.

7 April 2020

Steve Kemp: A busy few days

Over the past few weeks things have been pretty hectic. Since I'm not working at the moment I'm mostly doing childcare instead. I need a break, now and again, so I've been sending our child to p iv koti two days a week with him home the rest of the time. I love taking care of the child, because he's seriously awesome, but it's a hell of a lot of work when most of our usual escapes are unavailable. For example we can't go to the (awesome) Helsinki Central Library as that is closed. Instead of doing things outdoors we've been baking bread together, painting, listening to music and similar. He's a big fan of any music with drums and shouting, so we've been listening to Rammstein, The Prodigy, and as much Queen as I can slip in without him complaining ("more bang bang!"). I've also signed up for some courses at the Helsinki open university, including Devops with Docker so perhaps I have a future career working with computers? I'm hazy. Finally I saw a fun post the other day on reddit asking about the creation of a DSL for server-setup. I wrote a reply which basically said two things: Anyway I had an idea of just expressing things in a simple fashion, borrowing Puppet syntax (which I guess is just Ruby hash literals). So a module to do stuff with files would just look like this: 
file   name   => "This is my rule",
       target => "/tmp/blah",
       ensure => "absent"
The next thing to do is to allow that to notify another rule, when it results in a change. So you add in: 
notify => "Name of rule"
# or
notify => [ "Name of rule", "Name of another rule" ]
You could also express dependencies the other way round: 
shell   name => "Do stuff",
        command => "wc -l /etc/passwd > /tmp/foo",
        requires => [ "Rule 1", "Rule 2"]
Anyway the end result is a simple syntax which allows you to do things; I wrote a file to allow me to take a clean system and configure it to run a simple golang application in an hour or so. The downside? Well the obvious one is that there's no support for setting up cron jobs, setting up docker images, MySQL usernames/passwords, etc. Just a core set of primitives. Adding new things is easy, but also an endless job. So I added the ability to run external/binary plugins stored outside the project. To support that is simple with the syntax we have: All good. People can write modules, if they like, and they can do that in any language they like. Fun times. We'll call it marionette since it's all puppet-inspired: And that concludes this irregular update.

6 April 2020

Russ Allbery: Review: Thick

Review: Thick, by Tressie McMillan Cottom
Publisher: The New Press
Copyright: 2019
ISBN: 1-62097-437-1
Format: Kindle
Pages: 247
Tressie McMillan Cottom is an associate professor of sociology at Virginia Commonwealth University. I first became aware of her via retweets and recommendations from other people I follow on Twitter, and she is indeed one of the best writers on that site. Thick: And Other Essays is an essay collection focused primarily on how American culture treats black women. I will be honest here, in part because I think much of the regular audience for my book reviews is similar to me (white, well-off from working in tech, and leftist but privileged) and therefore may identify with my experience. This is the sort of book that I always want to read and then struggle to start because I find it intimidating. It received a huge amount of praise on release, including being named as a finalist for the National Book Award, and that praise focused on its incisiveness, its truth-telling, and its depth and complexity. Complex and incisive books about racism are often hard for me to read; they're painful, depressing, and infuriating, and I have to fight my tendency to come away from them feeling more cynical and despairing. (Despite loving his essays, I'm still procrastinating reading Ta-Nehisi Coates's books.) I want to learn and understand but am not good at doing anything with the information, so this reading can feel like homework. If that's also your reaction, read this book. I regret having waited as long as I did. Thick is still, at times, painful, depressing, and infuriating. It's also brilliantly written in a way that makes the knowledge being conveyed easier to absorb. Rather than a relentless onslaught of bearing witness (for which, I should stress, there is an important place), it is a scalpel. Each essay lays open the heart of a subject in a few deft strokes, points out important features that the reader has previously missed, and then steps aside, leaving you alone with your thoughts to come to terms with what you've just learned. I needed this book to be an essay collection, with each thought just long enough to have an impact and not so long that I became numb. It's the type of collection that demands a pause at the end of each essay, a moment of mental readjustment, and perhaps a paging back through the essay again to remember the sharpest points. The essays often start with seeds of the personal, drawing directly on McMillan Cottom's own life to wrap context around their point. In the first essay, "Thick," she uses advice given her younger self against writing too many first-person essays to talk about the writing form, its critics, and how the backlash against it has become part of systematic discrimination because black women are not allowed to write any other sort of authoritative essay. She then draws a distinction between her own writing and personal essays, not because she thinks less of that genre but because that genre does not work for her as a writer. The essays in Thick do this repeatedly. They appear to head in one direction, then deepen and shift with the added context of precise sociological analysis, defying predictability and reaching a more interesting conclusion than the reader had expected. And, despite those shifts, McMillan Cottom never lost me in a turn. This is a book that is not only comfortable with complexity and nuance, but helps the reader become comfortable with that complexity as well. The second essay, "In the Name of Beauty," is perhaps my favorite of the book. Its spark was backlash against an essay McMillan Cottom wrote about Miley Cyrus, but the topic of the essay wasn't what sparked the backlash.
What many black women were angry about was how I located myself in what I'd written. I said, blithely as a matter of observable fact, that I am unattractive. Because I am unattractive, the argument went, I have a particular kind of experience of beauty, race, racism, and interacting with what we might call the white gaze. I thought nothing of it at the time I was writing it, which is unusual. I can usually pinpoint what I have said, written, or done that will piss people off and which people will be pissed off. I missed this one entirely.
What follows is one of the best essays on the social construction of beauty I've ever read. It barely pauses at the typical discussion of unrealistic beauty standards as a feminist issue, instead diving directly into beauty as whiteness, distinguishing between beauty standards that change with generations and the more lasting rules that instead police the bounds between white and not white. McMillan Cottom then goes on to explain how beauty is a form of capital, a poor and problematic one but nonetheless one of the few forms of capital women have access to, and therefore why black women have fought to be included in beauty despite all of the problems with judging people by beauty standards. And the essay deepens from there into a trenchant critique of both capitalism and white feminism that is both precise and illuminating.
When I say that I am unattractive or ugly, I am not internalizing the dominant culture's assessment of me. I am naming what has been done to me. And signaling who did it. I am glad that doing so unsettles folks, including the many white women who wrote to me with impassioned cases for how beautiful I am. They offered me neoliberal self-help nonsense that borders on the religious. They need me to believe beauty is both achievable and individual, because the alternative makes them vulnerable.
I could go on. Every essay in this book deserves similar attention. I want to quote from all of them. These essays are about racism, feminism, capitalism, and economics, all at the same time. They're about power, and how it functions in society, and what it does to people. There is an essay about Obama that contains the most concise explanation for his appeal to white voters that I've read. There is a fascinating essay about the difference between ethnic black and black-black in U.S. culture. There is so much more.
We do not share much in the U.S. culture of individualism except our delusions about meritocracy. God help my people, but I can talk to hundreds of black folks who have been systematically separated from their money, citizenship, and personhood and hear at least eighty stories about how no one is to blame but themselves. That is not about black people being black but about people being American. That is what we do. If my work is about anything it is about making plain precisely how prestige, money, and power structure our so-called democratic institutions so that most of us will always fail.
I, like many other people in my profession, was always more comfortable with the technical and scientific classes in college. I liked math and equations and rules, dreaded essay courses, and struggled to engage with the mandatory humanities courses. Something that I'm still learning, two decades later, is the extent to which this was because the humanities are harder work than the sciences and I wasn't yet up to the challenge of learning them properly. The problems are messier and more fluid. The context required is broader. It's harder to be clear and precise. And disciplines like sociology deal with our everyday lived experience, which means that we all think we're entitled to an opinion. Books like this, which can offer me a hand up and a grounding in the intellectual rigor while simultaneously being engaging and easy to read, are a treasure. They help me fill in the gaps in my education and help me recognize and appreciate the depth of thought in disciplines that don't come as naturally to me. This book was homework, but the good kind, the kind that exposes gaps in my understanding, introduces topics I hadn't considered, and makes the time fly until I come up for air, awed and thinking hard. Highly recommended. Rating: 9 out of 10

31 March 2020

Chris Lamb: Free software activities in March 2020

Here is my monthly update covering what I have been doing in the free software world during March 2020 (previous month): In addition, I did even more hacking on the Lintian static analysis tool for Debian packages, including:
Reproducible builds One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter. This month, I: In our tooling, I also made the following changes to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, including preparing and uploading version 138 to Debian: The Reproducible Builds project also operates a fully-featured and comprehensive Jenkins-based testing framework that powers tests.reproducible-builds.org. This month, I reworked the web-based package rescheduling tool to:
Debian LTS This month I have worked 18 hours on Debian Long Term Support (LTS) and 8 hours on its sister Extended LTS project. You can find out more about the Debian LTS project via the following video:
Debian Uploads For the Debian Privacy Maintainers team I requested that the pyptlib package be removed from the archive (#953429) as well as uploading onionbalance (0.1.8-6) to fix test failures under Pytest 3.x (#953535) and a new upstream release of nautilus-wipe. Finally, I sponsored an upload of bilibop (0.6.1) on behalf of Yann Amar.

Russell Coker: Links March 2020

Rolling Stone has an insightful article about why the Christian Right supports Trump and won t stop supporting him no matter what he does [1]. Interesting article about Data Oriented Architecture [2]. Quarantine Will normalise WFH and Recession will Denormalise Jobs [3]. I guess we can always hope that after a disaster we can learn to do things better than before. Tyre wear is worse than exhaust for small particulate matter [4]. We need better tyres and legal controls over such things. Scott Santens wrote an insightful article about the need for democracy and unconditional basic income [5]. In ancient Greece, work was regarded as a curse is an extreme position but strongly supported by evidence. In his essay In Praise of Idleness, Bertrand Russell wrote Modern methods of production have given us the possibility of ease and security for all; we have chosen, instead, to have overwork for some and starvation for others. Hitherto we have continued to be as energetic as we were before there were machines; in this we have been foolish, but there is no reason to go on being foolish forever. Cory Doctorow wrote an insightful article for Locus titled A Lever Without a Fulcrum Is Just a Stick about expansions to copyright laws [6]. One of his analogies is that giving a bullied kid more lunch money just allows the bullies to steal more money, with artists being bullied kids and lunch money being the rights that are granted under copyright law. The proposed solution includes changes to labor and contract law, presumably Cory will write other articles in future giving the details of his ideas in this regard. The Register has an amusing article about the trial of a former CIA employee on trial for being the alleged vault 7 leaker [7]. Both the prosecution and the defence are building their cases around the defendent being a jerk. The article exposes poor security and poor hiring practices in the CIA. CNN has an informative article about Finland s war on fake news [8]. As Finland has long standing disputes with Russia they have had more practice at dealing with fake news than most countries. The Times of Israel has an interesting article about how the UK used German Jews to spy on German prisoners of war [9]. Cory Doctorow wrote an insightful article Data is the New Toxic Waste about how collecting personal data isn t an asset, it s a liability [10]. Ulrike Uhlig wrote an insightful article about Control Freaks , analysing the different meanings of control, both positive and negative [11]. 538 has an informative article about the value of statistical life [12]. It s about $9M per person in the US, which means a mind-boggling amount of money should be spent to save the millions of lives that will be potentially lost in a natural disaster (like Coronavirus). NPR has an interesting interview about Crypto AG, the Swiss crypto company owned by the CIA [13]. I first learned of this years ago, it s not new, but I still learned a lot from this interview.
Related posts:
  1. Links January 2020 C is Not a Low Level Language [1] is an...
  2. Links February 2020 Truthout has an interesting summary of the US Wars Without...
  3. Links March 2013 Russ Allbery wrote an informative post about how to determine...

30 March 2020

Shirish Agarwal: Covid 19 and the Indian response.

There have been lot of stories about Coronavirus and with it a lot of political blame-game has been happening. The first step that India took of a lockdown is and was a good step but without having a plan as to how especially the poor and the needy and especially the huge migrant population that India has (internal migration) be affected by it. A 2019 World Economic Forum shares the stats. as 139 million people. That is a huge amount of people and there are a variety of both push and pull factors which has displaced these huge number of people. While there have been attempts in the past and probably will continue in future they will be hampered unless we have trust-worthy data which is where there is lots that need to be done. In the recent few years, both the primary and secondary data has generated lot of controversies within India as well as abroad so no point in rehashing all of that. Even the definition of who is a migrant needs to be well-established just as who is a farmer . The simplest lucanae in the later is those who have land are known as farmers but the tenant farmers and their wives are not added as farmers hence the true numbers are never known. Is this an India-specific problem or similar definition issues are there in the rest of the world I don t know.

How our Policies fail to reach the poor and the vulnerable The sad part is most policies in India are made in castles in the air . An interview by the wire shares the conundrum of those who are affected and the policies which are enacted for them (it s a youtube video, sorry)
If one with an open and fresh mind sees the interview it is clear that why there was a huge reverse migration from Indian cities to villages. The poor and marginalized has always seen the Indian state as an extortive force so it doesn t make sense for them to be in the cities. The Prime Minister s annoucement of food for 3 months was a clear indication for the migrant population that for 3 months they will have no work. Faced with such a scenario, the best option for them was to return to their native places. While videos of huge number of migrants were shown of Delhi, this was the scenario of most states and cities, including Pune, my own city . Another interesting point which was made is most of the policies will need the migrants to be back in the villages. Most of these are tied to the accounts which are opened in villages, so even if they want to have the benefits they will have to migrate to villages in order to use them. Of course, everybody in India knows how leaky the administration is. The late Shri Rajiv Gandhi had famously and infamously remarked once how leaky the Public Distribution system and such systems are. It s only 10 paise out of rupee which reaches the poor. And he said this about 30 years ago. There have been numerous reports of both IPS (Indian Police Services) reforms and IAS (Indian Administrative Services) reforms over the years, many of the committee reports have been in public domain and in fact was part of the election manifesto of the ruling party in 2014 but no movement has happened on that part. The only thing which has happened is people from the ruling party have been appointed on various posts which is same as earlier governments. I was discussing with a friend who is a contractor and builder about the construction labour issues which were pointed in the report and if it is true that many a times the migrant labour is not counted. While he shared a number of cases where he knew, a more recent case in public memory was when some labourers died while building Amanora mall which is perhaps one of largest malls in India. There were few accidents while constructing the mall. Apparently, the insurance money which should have gone to the migrant laborer was taken by somebody close to the developers who were building the mall. I have a friend in who lives in Jharkhand who is a labour officer. She has shared with me so many stories of how the labourers are exploited. Keep in mind she has been a labor officer appointed by the state and her salary is paid by the state. So she always has to maintain a balance of ensuring worker s rights and the interests of the state, private entities etc. which are usually in cahoots with the state and it is possible that lot of times the State wins over the worker s rights. Again, as a labour officer, she doesn t have that much power and when she was new to the work, she was often frustrated but as she remarked few months back, she has started taking it easy (routinized) as anyways it wasn t helping her in any good way. Also there have been plenty of cases of labor officers being murdered so its easier to understand why one tries to retain some sanity while doing their job.

The Indian response and the World Response The Indian response has been the lockdown and very limited testing. We seem to be following the pattern of UK and U.S. which had been slow to respond and slow to testing. In the past Kerala showed the way but this time even that is not enough. At the end of the day we need to test, test and test just as shared by the WHO chairman. India is trying to create its own cheap test kits with ICMR approval, for e.g. a firm from my own city Pune MyLab has been given approval. We will know how good or bad they are only after they have been field-tested. For ventilators we have asked Mahindra and Mahindra even though there are companies like Allied Medical and others who have exported to EU and others which the Govt. is still taking time to think through. This is similar to how in UK some companies who are with the Govt. but who have no experience in making ventilators are been given orders while those who have experience and were exporting to Germany and other countries are not been given orders. The playbook is errily similar. In India, we don t have the infrastructure for any new patients, period. Heck only a couple of states have done something proper for the anganwadi workers. In fact, last year there were massive strikes by anganwadi workers all over India but only NDTV showed a bit of it along with some of the news channels from South India. Most mainstream channels chose to ignore it. On the world stage, some of the other countries and how they have responded perhaps need sharing. For e.g. I didn t know that Cuba had so many doctors and the politics between it and Brazil. Or the interesting stats. shared by Andreas Backhaus which seems to show how distributed the issue (age-wise) is rather than just a few groups as has been told in Indian media. What was surprising for me is the 20-29 age group which has not been shared so much in the Indian media which is the bulk of our population. The HBR article also makes a few key points which I hope both the general public and policymakers both in India as well as elsewhere take note of. What is worrying though that people can be infected twice or more as seems to be from Singapore or China and elsewhere. I have read enough of Robin Cook and Michael Crichton books to be aware that viruses can do whatever. They will over time mutate, how things will happen then is anybody s guess. What I found interesting is the world economic forum article which hypothesis that it may be two viruses which got together as well as research paper from journal from poteome research which has recently been published. The biggest myth flying around is that summer will halt or kill the spread which even some of my friends have been victim of . While a part of me wants to believe them, a simple scientific fact has been viruses have probably been around us and evolved over time, just like we have. In fact, there have been cases of people dying due to common cold and other things. Viruses are so prevalent it s unbelivable. What is and was interesting to note is that bat-borne viruses as well as pangolin viruses had been theorized and shared by Chinese researchers going all the way back to 90 s . The problem is even if we killed all the bats in the world, some other virus will take its place for sure. One of the ideas I had, dunno if it s feasible or not that at least in places like Airports, we should have some sort of screenings and a labs working on virology. Of course, this will mean more expenses for flying passengers but for public health and safety maybe it would worth doing so. In any case, virologists should have a field day cataloging various viruses and would make it harder for viruses to spread as fast as this one has. The virus spread also showed a lack of leadership in most of our leaders who didn t react fast enough. While one hopes people do learn from this, I am afraid the whole thing is far from over. These are unprecedented times and hope that all are maintaining social distancing and going out only when needed.

29 March 2020

Enrico Zini: Politics links

How tech's richest plan to save themselves after the apocalypse
politics privilege archive.org
2020-03-30
Silicon Valley s elite are hatching plans to escape disaster and when it comes, they ll leave the rest of us behind
Life of Privilege Explained in a $100 Race
privilege
2020-03-30
Hierarchy problems in autiautoritarian groups, and how to face them
chart politics archive.org
2020-03-30
Abstruse Goose The Story of the Cheese Maze
comics politics privilege archive.org
2020-03-30
Heteronomy - Wikipedia
language politics archive.org
2020-03-30
Heteronomy refers to action that is influenced by a force outside the individual, in other words the state or condition of being ruled, governed, or under the sway of another, as in a military occupation.
Poster - Early Warning Signs of Fascism
fascism politics archive.org
2020-03-30
Poster P590CW $9.00 Early Warning Signs Of Fascism Laurence W. Britt wrote about the common signs of fascism in April, 2003, after researching seven fascist regimes: Hitler's Nazi Germany; Mussolini's Italy; Franco's Spain; Salazar's Portugal; Papadopoulos' Greece; Pinochet's Chile; Suharto's Indonesia. Get involved! Text: Early Warning Signs of Fascism Powerful and Continuing Nationalism Disdain For Human Rights Identification of Enemies As a unifying cause Supremacy of the military Rampant Sexism Controlled Mass Media Obsession With National Security
Mobilizing in Times of Social Media
politics archive.org
2020-03-30
Political and social scientist Stefania Milan writes about social movements, mobilization and organized collective action. On the one hand, interactions and networks achieve more visibility and become a proxy for a collective we . On the other hand: Law enforcement can exercise preemptive monitorin
The Billion-Dollar Disinformation Campaign to Reelect the President
perception politics archive.org
2020-03-30
How new technologies and techniques pioneered by dictators will shape the 2020 election
An Italian Flash Mob Just Pushed Back Europe s Populist Tide
perception politics archive.org
2020-03-30
A regional election offers lessons on combatting the rise of the far right, both across the Continent and in the United States.
Italian diaspora - Wikipedia
italy politics archive.org
2020-03-30
The Italian diaspora is the large-scale emigration of Italians from Italy. There are two major Italian diasporas in Italian history. The first diaspora began more or less around 1880, a decade or so after the Unification of Italy (with most leaving after 1880), and ended in the 1920s to early-1940s with the rise of Fascism in Italy. The second diaspora started after the end of World War II and roughly concluded in the 1970s. These together constituted the largest voluntary emigration period in documented history. Between 1880-1980, about 15,000,000 Italians left the country permanently. By 1980, it was estimated that about 25,000,000 Italians were residing outside Italy. A third wave is being reported in present times, due to the socio-economic problems caused by the financial crisis of the early twenty-first century, especially amongst the youth. According to the Public Register of Italian Residents Abroad (AIRE), figures of Italians abroad rose from 3,106,251 in 2006 to 4,636,647 in 2015, growing by 49.3% in just ten years.

28 March 2020

Fran ois Marier: How to get a direct WebRTC connections between two computers

WebRTC is a standard real-time communication protocol built directly into modern web browsers. It enables the creation of video conferencing services which do not require participants to download additional software. Many services make use of it and it almost always works out of the box. The reason it just works is that it uses a protocol called ICE to establish a connection regardless of the network environment. What that means however is that in some cases, your video/audio connection will need to be relayed (using end-to-end encryption) to the other person via third-party TURN server. In addition to adding extra network latency to your call that relay server might overloaded at some point and drop or delay packets coming through. Here's how to tell whether or not your WebRTC calls are being relayed, and how to ensure you get a direct connection to the other host.

Testing basic WebRTC functionality Before you place a real call, I suggest using the official test page which will test your camera, microphone and network connectivity. Note that this test page makes use of a Google TURN server which is locked to particular HTTP referrers and so you'll need to disable privacy features that might interfere with this:
  • Brave: Disable Shields entirely for that page (Simple view) or allow all cookies for that page (Advanced view).
  • Firefox: Ensure that http.network.referer.spoofSource is set to false in about:config, which it is by default.
  • uMatrix: The "Spoof Referer header" option needs to be turned off for that site.

Checking the type of peer connection you have Once you know that WebRTC is working in your browser, it's time to establish a connection and look at the network configuration that the two peers agreed on. My favorite service at the moment is Whereby (formerly Appear.in), so I'm going to use that to connect from two different computers:
  • canada is a laptop behind a regular home router without any port forwarding.
  • siberia is a desktop computer in a remote location that is also behind a home router, but in this case its internal IP address (192.168.1.2) is set as the DMZ host.

Chromium For all Chromium-based browsers, such as Brave, Chrome, Edge, Opera and Vivaldi, the debugging page you'll need to open is called chrome://webrtc-internals. Look for RTCIceCandidatePair lines and expand them one at a time until you find the one which says:
  • state: succeeded (or state: in-progress)
  • nominated: true
  • writable: true
Then from the name of that pair (N6cxxnrr_OEpeash in the above example) find the two matching RTCIceCandidate lines (one local-candidate and one remote-candidate) and expand them. In the case of a direct connection, I saw the following on the remote-candidate:
  • ip shows the external IP address of siberia
  • port shows a random number between 1024 and 65535
  • candidateType: srflx
and the following on local-candidate:
  • ip shows the external IP address of canada
  • port shows a random number between 1024 and 65535
  • candidateType: prflx
These candidate types indicate that a STUN server was used to determine the public-facing IP address and port for each computer, but the actual connection between the peers is direct. On the other hand, for a relayed/proxied connection, I saw the following on the remote-candidate side:
  • ip shows an IP address belonging to the TURN server
  • candidateType: relay
and the same information as before on the local-candidate.

Firefox If you are using Firefox, the debugging page you want to look at is about:webrtc. Expand the top entry under "Session Statistics" and look for the line (should be the first one) which says the following in green:
  • ICE State: succeeded
  • Nominated: true
  • Selected: true
then look in the "Local Candidate" and "Remote Candidate" sections to find the candidate type in brackets.

Firewall ports to open to avoid using a relay In order to get a direct connection to the other WebRTC peer, one of the two computers (in my case, siberia) needs to open all inbound UDP ports since there doesn't appear to be a way to restrict Chromium or Firefox to a smaller port range for incoming WebRTC connections. This isn't great and so I decided to tighten that up in two ways by:
  • restricting incoming UDP traffic to the IP range of siberia's ISP, and
  • explicitly denying incoming to the UDP ports I know are open on siberia.
To get the IP range, start with the external IP address of the machine (I'll use the IP address of my blog in this example: 66.228.46.55) and pass it to the whois command: 
$ whois 66.228.46.55   grep CIDR
CIDR:           66.228.32.0/19
To get the list of open UDP ports on siberia, I sshed into it and ran nmap: 
$ sudo nmap -sU localhost
Starting Nmap 7.60 ( https://nmap.org ) at 2020-03-28 15:55 PDT
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000015s latency).
Not shown: 994 closed ports
PORT      STATE         SERVICE
631/udp   open filtered ipp
5060/udp  open filtered sip
5353/udp  open          zeroconf
Nmap done: 1 IP address (1 host up) scanned in 190.25 seconds
I ended up with the following in my /etc/network/iptables.up.rules (ports below 1024 are denied by the default rule and don't need to be included here): 
# Deny all known-open high UDP ports before enabling WebRTC for canada
-A INPUT -p udp --dport 5060 -j DROP
-A INPUT -p udp --dport 5353 -j DROP
-A INPUT -s 66.228.32.0/19 -p udp --dport 1024:65535 -j ACCEPT

24 March 2020

Russ Allbery: Review: Lost in Math

Review: Lost in Math, by Sabine Hossenfelder
Publisher: Basic
Copyright: June 2018
ISBN: 0-465-09426-0
Format: Kindle
Pages: 248
Listening to experts argue can be one of the better ways to learn about a new field. It does require some basic orientation and grounding or can be confusing or, worse, wildly misleading, so some advance research or Internet searches are warranted. But it provides some interesting advantages over reading multiple popular introductions to a field. First, experts arguing with each other are more precise about their points of agreement and disagreement because they're trying to persuade someone who is well-informed. The points of agreement are often more informative than the points of disagreement, since they can provide a feel for what is uncontroversial among experts in the field. Second, internal arguments tend to be less starry-eyed. One of the purposes of popularizations of a field is to get the reader excited about it, and that can be fun to read. But to generate that excitement, the author has a tendency to smooth over disagreements and play up exciting but unproven ideas. Expert disagreements pull the cover off of the uncertainty and highlight the boundaries of what we know and how we know it. Lost in Math (subtitled How Beauty Leads Physics Astray) is not quite an argument between experts. That's hard to find in book form; most of the arguments in the scientific world happen in academic papers, and I rarely have the energy or attention span to read those. But it comes close. Hossenfelder is questioning the foundations of modern particle physics for the general public, but also for her fellow scientists. High-energy particle physics is facing a tricky challenge. We have a solid theory (the standard model) which explains nearly everything that we have currently observed. The remaining gaps are primarily at very large scales (dark matter and dark energy) or near phenomena that are extremely difficult to study (black holes). For everything else, the standard model predicts our subatomic world to an exceptionally high degree of accuracy. But physicists don't like the theory. The details of why are much of the topic of this book, but the short version is that the theory does not seem either elegant or beautiful. It relies on a large number of measured constants that seem to have no underlying explanation, which is contrary to a core aesthetic principle that physicists use to judge new theories. Accompanying this problem is another: New experiments in particle physics that may be able to confirm or disprove alternate theories that go beyond the standard model are exceptionally expensive. All of the easy experiments have been done. Building equipment that can probe beyond the standard model is incredibly expensive, and thus only a few of those experiments have been done. This leads to two issues: Particle physics has an overgrowth of theories (such as string theory) that are largely untethered from experiments and are not being tested and validated or disproved, and spending on new experiments is guided primarily by a sense of scientific aesthetics that may simply be incorrect. Enter Lost in Math. Hossenfelder's book picks up a thread of skepticism about string theory (and, in Hossenfelder's case, supersymmetry as well) that I previously read in Lee Smolin's The Trouble with Physics. But while Smolin's critique was primarily within the standard aesthetic and epistemological framework of particle physics, Hossenfelder is questioning that framework directly. Why should nature be beautiful? Why should constants be small? What if the universe does have a large number of free constants? And is the dislike of an extremely reliable theory on aesthetic grounds a good basis for guiding which experiments we fund?
Do you recall the temple of science, in which the foundations of physics are the bottommost level, and we try to break through to deeper understanding? As I've come to the end of my travels, I worry that the cracks we're seeing in the floor aren't really cracks at all but merely intricate patterns. We're digging in the wrong places.
Lost in Math will teach you a bit less about physics than Smolin's book, although there is some of that here. Smolin's book was about two-thirds physics and one-third sociology of science. Lost in Math is about two-thirds sociology and one-third physics. But that sociology is engrossing. It's obvious in retrospect, but I hadn't thought before about the practical effects of running out of unexplained data on a theoretical field, or about the transition from more data than we can explain to having to spend billions of dollars to acquire new data. And Hossenfelder takes direct aim at the human tendency to find aesthetically appealing patterns and unified explanations, and scores some palpable hits.
I went into physics because I don't understand human behavior. I went into physics because math tells it how it is. I liked the cleanliness, the unambiguous machinery, the command math has over nature. Two decades later, what prevents me from understanding physics is that I still don't understand human behavior. "We cannot give exact mathematical rules that define if a theory is attractive or not," says Gian Francesco Giudice. "However, it is surprising how the beauty and elegance of a theory are universally recognized by people from different cultures. When I tell you, 'Look, I have a new paper and my theory is beautiful,' I don't have to tell you the details of my theory; you will get why I'm excited. Right?" I don't get it. That's why I am talking to him. Why should the laws of nature care what I find beautiful? Such a connection between me and the universe seems very mystical, very romantic, very not me. But then Gian doesn't think that nature cares what I find beautiful, but what he finds beautiful.
The structure of this book is half tour of how physics judges which theories are worthy of investigation and half personal quest to decide whether physics has lost contact with reality. Hossenfelder approaches this second thread with multiple interviews of famous scientists in the field. She probes at their bases for preferring one theory over another, at how objective those preferences can or should be, and what it means for physics if they're wrong (as increasingly appears to be the case for supersymmetry). In so doing, she humanizes theory development in a way that I found fascinating. The drawback to reading about ongoing arguments is the lack of a conclusion. Lost in Math, unsurprisingly, does not provide an epiphany about the future direction of high-energy particle physics. Its conclusion, to the extent that it has one, is a plea to find a way to put particle physics back on firmer experimental footing and to avoid cognitive biases in theory development. Given the cost of experiments and the nature of humans, this is challenging. But I enjoyed reading this questioning, contrarian take, and I think it's valuable for understanding the limits, biases, and distortions at the edge of new theory development. Rating: 7 out of 10

19 March 2020

John Goerzen: COVID-19 is serious for all ages. Treat it like WWII

Today I d like to post a few updates about COVID-19 which I have gathered from credible sources, as well as some advice also gathered from credible sources. Summary
  1. Coronavirus causes health impacts requiring hospitalization in a significant percentage of all adult age groups.
  2. Coronavirus also can cause no symptoms at all in many, especially children.
  3. Be serious about social distancing.
COVID-19 is serious for young adults too According to this report based on a CDC analysis, between 14% and 20% of people aged 20 to 44 require hospitalization due to COVID-19. That s enough to be taken seriously. See also this CNN story. Act as if you are a carrier because you may be infected and not even know it, even children Information on this is somewhat preliminary, but it is certainly known that a certain set of cases is asymptomatic. This article discusses manifestations in children, while this summary of a summary (note: not original research) suggests that 17.9% of people may not even know they are infected. How serious is this? Serious. This excellent article by Daniel W. Johnson, MD, is a very good read. Among the points it makes: Advice I m going to just copy Dr. Johnson s advice here:
  1. You and your kids should stay home. This includes not going to church, not going to the gym, not going anywhere.
  2. Do not travel for enjoyment until this is done. Do not travel for work unless your work truly requires it.
  3. Avoid groups of people. Not just crowds, groups. Just be around your immediate family. I think kids should just play with siblings at this point no play dates, etc.
  4. When you must leave your home (to get groceries, to go to work), maintain a distance of six feet from people. REALLY stay away from people with a cough or who look sick.
  5. When you do get groceries, etc., buy twice as much as you normally do so that you can go to the store half as often. Use hand sanitizer immediately after your transaction, and immediately after you unload the groceries.
I m not saying people should not go to work. Just don t leave the house for anything unnecessary, and if you can work from home, do it. Everyone on this email, besides Mom and Dad, are at low risk for severe disease if/when they contract COVID-19. While this is great, that is not the main point. When young, well people fail to do social distancing and hygiene, they pick up the virus and transmit it to older people who are at higher risk for critical illness or death. So everyone needs to stay home. Even young people. Tell every person over 60, and every person with significant medical conditions, to avoid being around people. Please do not have your kids visit their grandparents if you can avoid it. FaceTime them. Our nation is the strongest one in the world. We have been through other extreme challenges and succeeded many times before. We WILL return to normal life. Please take these measures now to flatten the curve, so that we can avoid catastrophe.
I d also add that many supermarkets offer delivery or pickup options that allow you to get your groceries without entering the store. Some are also offering to let older people shop an hour before the store opens to the general public. These could help you minimize your exposure. Other helpful links Here is a Reddit megathread with state-specific unemployment resources. Scammers are already trying to prey on people. Here are some important tips to avoid being a victim. Although there are varying opinions, some are recommending avoiding ibuprofen when treating COVID-19. Bill Gates had some useful advice. Here s a summary emphasizing the need for good testing.

Next.

Previous.