Search Results: "ag"

27 June 2021

Russ Allbery: New Big Eight signing key

For various reasons, I haven't had any time to work on Usenet (or most other personal hobbies other than a few book reviews) for the past few months, so it took me longer than expected to finish this work. But I have finally switched the official, preferred Big Eight signing key to a newly-generated OpenPGP key that isn't incredibly weak. I put up a basic web page with the new control information and links to the signing keys for the Big Eight hierarchy. Duplicate control messages will continue to be issued with the old signing key for the indefinite future, but be aware that the old key is quite weak and could probably be brute-forced if someone cared enough. You should switch to only honoring the new key if you are running a news server that honors Big Eight control messages. The PGP Moose key is almost as ancient as the old control message key and is probably nearly as weak, but we've not yet replaced it. When that happens, that will be noted here and on that web page.

Louis-Philippe V ronneau: Writing QA Scripts for Debian Teams

Since I joined the Debian Python Team, I have had a lot of fun working on different QA issues. Although I'm still a Perl illiterate1, I've for example contributed to a few Lintian tags. There are multiple ways to make mass QA changes to team-managed packages. Projects like the Debian Janitor are more than fantastic: they make for a robust, thorough and automated way to fix QA issues in the archive and I don't have enough good words to describe the amazing work of Jelmer Vernooij on the toolsuite the Janitor uses. But with robustness comes complexity. The Janitor is currently based on 10 different subtools (silver-platter, ognibuild, lintian-brush, ...) and if you want to use it to fix a bug, you first need to make sure there's a Lintian tag that flags the issue you're working on. Then you need to write a lintian-brush fixer to fix said issue. Sadly, sometimes writing a new Lintian tag to flag a trivial changes is not the appropriate course of action and only creates clutter. All this to say until now, I was a missing a "quick and somewhat dirty2" way to make simple one-off changes to a bunch of packages. 200 lines of Python later, I'm happy to report I have a simple way to replace the old Clojure Team email in d/control by the new one for all of our packages. Even better, although this script doesn't aim to be a versatile tool like the Janitor is, most of the functions can be reused for other similar one-off scripts. Many thanks to Felix Lechner showing me the very handy Lintian Query JSON interface!
  1. I don't really enjoy coding in Perl, but it makes up so much of the current Debian infrastructure that I wish I did. I keep telling myself I should buy an "Introduction to Perl" book...
  2. A quick and dirty way to make those changes would've been to write a shell script, but one of my 2021 resolution is to use Python for all my scripting needs.

Fran ois Marier: Removing unsafe-inline from Ikiwiki's style-src directive

After moving my Ikiwiki blog to my own server and enabling a basic CSP policy, I decided to see if I could tighten up the policy some more and stop relying on style-src 'unsafe-inline'. This does require that OpenID logins be disabled, but as a bonus, it also removes the need for jQuery to be present on the server.

Revised CSP policy First of all, I visited all of my pages in a Chromium browser and took note of the missing hashes listed in the developer tools console (Firefox doesn't show the missing hashes):
  • 'sha256-4Su6mBWzEIFnH4pAGMOuaeBrstwJN4Z3pq/s1Kn4/KQ='
  • 'sha256-j0bVhc2Wj58RJgvcJPevapx5zlVLw6ns6eYzK/hcA04='
  • 'sha256-j6Tt8qv7z2kSc7fUs0YHbrxawwsQcS05fVaX1r2qrbk='
  • 'sha256-p4cncjf0hAIeTSS5tXecf7qTUanDC27KdlKhT9eOsZU='
  • 'sha256-Y6v8OCtFfMmI5mbpwqCreLofmGZQfXYK7jJHCoHvn7A='
  • 'sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU='
which took care of all of the inline styles. Note that I kept unsafe-inline in the directive since it will be automatically ignored by browsers who understand hashes, but will be honored and make the site work on older browsers. Next I added the new unsafe-hashes source expression along with the hash of the CSS fragment (clear: both) that is present on all pages related to comments in Ikiwiki: 
$ echo -n "clear: both"   openssl dgst -sha256 -binary   openssl base64 -A
matwEc6givhWX0+jiSfM1+E5UMk8/UGLdl902bjFBmY=
My final style-src directive is therefore the following: 
style-src 'self' 'unsafe-inline' 'unsafe-hashes' 'sha256-4Su6mBWzEIFnH4pAGMOuaeBrstwJN4Z3pq/s1Kn4/KQ=' 'sha256-j0bVhc2Wj58RJgvcJPevapx5zlVLw6ns6eYzK/hcA04=' 'sha256-j6Tt8qv7z2kSc7fUs0YHbrxawwsQcS05fVaX1r2qrbk=' 'sha256-p4cncjf0hAIeTSS5tXecf7qTUanDC27KdlKhT9eOsZU=' 'sha256-Y6v8OCtFfMmI5mbpwqCreLofmGZQfXYK7jJHCoHvn7A=' 'sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=' 'sha256-matwEc6givhWX0+jiSfM1+E5UMk8/UGLdl902bjFBmY='

Browser compatibility While unsafe-hashes is not yet implemented in Firefox, it happens to work just fine due to a bug (i.e. unsafe-hashes is always enabled whether or not the policy contains it). It's possible that my new CSP policy won't work in Safari, but these CSS clears don't appear to be needed anyways and so it's just going to mean extra CSP reporting noise.

Removing jQuery Since jQuery appears to only be used to provide the authentication system selector UI, I decided to get rid of it. I couldn't find a way to get Ikiwiki to stop pulling it in and so I put the following hack in my Apache config file: 
# Disable jQuery.
Redirect 204 /ikiwiki/jquery.fileupload.js
Redirect 204 /ikiwiki/jquery.fileupload-ui.js
Redirect 204 /ikiwiki/jquery.iframe-transport.js
Redirect 204 /ikiwiki/jquery.min.js
Redirect 204 /ikiwiki/jquery.tmpl.min.js
Redirect 204 /ikiwiki/jquery-ui.min.css
Redirect 204 /ikiwiki/jquery-ui.min.js
Redirect 204 /ikiwiki/login-selector/login-selector.js
Replacing the files on disk with an empty reponse seems to work very well and removes a whole lot of code that would otherwise be allowed by the script-src directive of my CSP policy. While there is a slight cosmetic change to the login page, I think the reduction in the attack surface is well worth it.

26 June 2021

Dirk Eddelbuettel: RcppRedis 0.1.11: Minor Update

A new minor release of RcppRedis arrived on CRAN, the first update since the last release in January of last year. RcppRedis is one of several packages connecting R to the fabulous Redis in-memory datastructure store (and much more). RcppRedis does not pretend to be feature complete, but it may do some things faster than the other interfaces, and also offers an optional coupling with MessagePack binary (de)serialization via RcppMsgPack. The package has carried production loads for several years now. This release updates CI to using r-ci, adds a quit() methods, and updates the windows library in builds thanks to a PR by Jeroen which also enables builds under the experimemtal UCRT windows flavor.

Changes in version 0.1.11 (2021-06-26)
  • The CI setup was updated to use run.sh from r-ci (Dirk).
  • A new function quit can be used to close a connection (Dirk).
  • The windows build was updated to libhiredis 1.0.0, and UCRT support was added (Jeroen in #42).

Courtesy of CRANberries, there is also a diffstat report for this release. More information is on the RcppRedis page. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Enrico Zini: Ansible conditionals in Transilience

This is part of a series of posts on ideas for an ansible-like provisioning system, implemented in Transilience. I thought a lot of what I managed to do so far with Transilience would be impossible, but then here I am. How about Ansible conditionals? Those must be impossible, right? Let's give it a try. A quick recon of Ansible sources Looking into Ansible's sources, when expressions are lists of strings AND-ed together. The expressions are Jinja2 expressions that Ansible pastes into a mini-template, renders, and checks the string that comes out. A quick recon of Jinja2 Jinja2 has a convenient function (jinja2.Environment.compile_expression) that compiles a template snippet into a Python function. It can also parse a template into an AST that can be inspected in various ways. Evaluating Ansible conditionals in Python Environment.compile_expression seems to really do precisely what we need for this, straight out of the box. There is an issue with the concept of "defined": for Ansible it seems to mean "the variable is present in the template context". In Transilience instead, all variables are fields in the Role dataclass, and can be None when not set. This means that we need to remove variables that are set to None before passing the parameters to the compiled Jinjae expression: 
class Conditional:
    """
    An Ansible conditional expression
    """
    def __init__(self, engine: template.Engine, body: str):
        # Original unparsed expression
        self.body: str = body
        # Expression compiled to a callable
        self.expression: Callable = engine.env.compile_expression(body)
    def evaluate(self, ctx: Dict[str, Any]):
        ctx =  name: val for name, val in ctx.items() if val is not None 
        return self.expression(**ctx)
Generating Python code Transilience does not only support running Ansible roles, but also converting them to Python code. I can keep this up by traversing the Jinja2 AST generating Python expressions. The code is straightforward enough that I can throw in a bit of pattern matching to make some expressions more idiomatic for Python: 
class Conditional:
    def __init__(self, engine: template.Engine, body: str):
    ...
        parser = jinja2.parser.Parser(engine.env, body, state='variable')
        self.jinja2_ast: nodes.Node = parser.parse_expression()
    def get_python_code(self) -> str:
        return to_python_code(self.jinja2_ast
def to_python_code(node: nodes.Node) -> str:
    if isinstance(node, nodes.Name):
        if node.ctx == "load":
            return f"self. node.name "
        else:
            raise NotImplementedError(f"jinja2 Name nodes with ctx= node.ctx!r  are not supported:  node!r ")
    elif isinstance(node, nodes.Test):
        if node.name == "defined":
            return f" to_python_code(node.node)  is not None"
        elif node.name == "undefined":
            return f" to_python_code(node.node)  is None"
        else:
            raise NotImplementedError(f"jinja2 Test nodes with name= node.name!r  are not supported:  node!r ")
    elif isinstance(node, nodes.Not):
        if isinstance(node.node, nodes.Test):
            # Special case match well-known structures for more idiomatic Python
            if node.node.name == "defined":
                return f" to_python_code(node.node.node)  is None"
            elif node.node.name == "undefined":
                return f" to_python_code(node.node.node)  is not None"
        elif isinstance(node.node, nodes.Name):
            return f"not  to_python_code(node.node) "
        return f"not ( to_python_code(node.node) )"
    elif isinstance(node, nodes.Or):
        return f"( to_python_code(node.left)  or  to_python_code(node.right) )"
    elif isinstance(node, nodes.And):
        return f"( to_python_code(node.left)  and  to_python_code(node.right) )"
    else:
        raise NotImplementedError(f"jinja2  node.__class__  nodes are not supported:  node!r ")
Scanning for variables Lastly, I can implement scanning conditionals for variable references to add as fields to the Role dataclass: 
class FindVars(jinja2.visitor.NodeVisitor):
    def __init__(self):
        self.found: Set[str] = set()
    def visit_Name(self, node):
        if node.ctx == "load":
            self.found.add(node.name)
class Conditional:
    ...
    def list_role_vars(self) -> Sequence[str]:
        fv = FindVars()
        fv.visit(self.jinja2_ast)
        return fv.found
The result in action Take this simple Ansible task: 
---
 - name: Example task
   file:
      state: touch
      path: /tmp/test
   when: (is_test is defined and is_test) or debug is defined
Run it through ./provision --ansible-to-python test and you get: 
from __future__ import annotations
from typing import Any
from transilience import role
from transilience.actions import builtin, facts
@role.with_facts([facts.Platform])
class Role(role.Role):
    # Role variables used by templates
    debug: Any = None
    is_test: Any = None
    def all_facts_available(self):
        if ((self.is_test is not None and self.is_test)
                or self.debug is not None):
            self.add(
                builtin.file(path='/tmp/test', state='touch'),
                name='Example task')
Besides one harmless set of parentheses too much, what I wasn't sure would be possible is there, right there, staring at me with a mischievous grin. Next: Building a Transilience playbook in a zipapp.

Bastian Venthur: The State of Python Packaging in 2021

Every year or so, I revisit the current best practices for Python packaging. I.e. the way you re supposed to distribute your Python packages. The main source is packaging.python.org where the official packaging guidelines are. It is worth noting that the way you re supposed to package your Python applications is not defined by Python or its maintainers, but rather delegated to a separate entity, the Python Packaging Authority (PyPA). PyPA PyPA does an excellent job providing us with information, best practices and tutorials regarding Python packaging. However, there s one thing that irritates me every single time I revisit the page and that is the misleading recommendation of their own tool pipenv. Quoting from the tool recommendations section of the packaging guidelines:
Use Pipenv to manage library dependencies when developing Python applications. See Managing Application Dependencies for more details on using pipenv.
PyPA recommends pipenv as the standard tool for dependency management, at least since 2018. A bold statement, given that pipenv only started in 2017, so the Python community cannot have had not enough time to standardize on the workflow around that tool. There have been no releases of pipenv between 2018-11 and 2020-04, that s 1.5 years for the standard tool. In the past, pipenv also hasn t been shy in pushing breaking changes in a fast-paced manner. PyPA still advertises pipenv all over the place and only mentions poetry a couple of times, although poetry seems to be the more mature product. I understand that pipenv lives under the umbrella of PyPA, but I still expect objectiveness when it comes to tool recommendation. Instead of making such claims, they should provide a list of competing tools and provide a fair feature comparison. Distributions You would expect exactly one distribution for Python packages, but here in Python land, we have several ones. The most popular ones being PyPI the official one and Anaconda. Anaconda is more geared towards data-scientists. The main selling point for Anaconda back then was that it provided pre-compiled binaries. This was especially useful for data-science related packages which depend on libatlas, -lapack, -openblas, etc. and need to be compiled for the target system. This problem has mostly been solved with the wide adoption of wheels, but you still encounter some source-only uploads to PyPI that require you to build stuff locally on pip install. Of course there s also the Python packages distributed by the Operating System, Debian in my case. While I was a firm believer in only using those packages provided by the OS in the very past, I moved to the opposite end of the spectrum throughout the years, and am only using the minimal packages provided by Debian to bootstrap my virtual environments (i.e. pip, setuptools and wheel). The main reason is outdated or missing libraries, which is expected Debian cannot hope to keep up with all the upstream changes in the ecosystem, and that is by design and fine. However, with the recent upgrade of manylinux, even the pip provided by Debian/unstable was too outdated, so you basically had to pip install --upgrade pip for a while otherwise you d end up compiling every package you d try to install via pip. So I m sticking to the official PyPI distribution wherever possible. However, compared to the Debian distribution it feels immature. In my opinion, there should be compiled wheels for all packages available that need it, built and provided by PyPI. Currently, the wheels provided are the ones uploaded by the upstream maintainers. This is not enough, as they usually build wheels only for one platform. Sometimes they don t upload wheels in the first place, relying on the users to compile during install. Then you have manylinux, an excellent idea to create some common ground for a portable Linux build distribution. However, sometimes when a new version of manylinux is released some upstream maintainers immediately start supporting only that version, breaking a lot of systems. A setup similar to Debian s where the authors only do a source-upload and the wheels are compiled on PyPI infrastructure for all available platforms, is probably the way to go. setup.py, setup.cfg, requirements.txt. Pipfile, pyproject.toml oh my! This is the part I m revisiting the documentation every year, to see what s the current way to go. The main point of packaging your Python application is to define the package s meta data and (build-) dependencies. setup.py + requirements.txt For the longest time, the setup.py and requirements.txt were (and, spoiler alert: still is) the backbone of your packaging efforts. In setup.py you define the meta data of your package, including its dependencies. If your project is a deployable application (vs. a library) you ll very often provide an additional requirements.txt with pinned dependencies. Usually the list of requirements is the same as defined in setup.py but with pinned versions. The reason why you avoid version pinning in setup.py is that it would interfere with other pinned dependencies from other dependencies you try to install. setup.cfg setup.cfg is a configuration file that is used by many standard tools in the Python ecosystem. Its format is ini-style and each tools configuration lives in its own stanza. Since 2016 setuptools supports configuring setup() using setup.cfg files. This was exciting news back then, however, it does not completely replace the setup.py file. While you can move most of the setup.py configuration into setup.cfg, you ll still have to provide that file with an empty setup() in order to allow for editable pip installs. In my opinion, that makes this feature useless and I rather stick to setup.py with a properly populated setup() until that file can be completely replaced with something else. Pipfile + Pipflie.lock Pipfile and Pipfile.lock are supposed to replace requirements.txt some day. So far they are not supported by pip or mentioned in any PEP. I think only pipenv supports them, so I d ignore them for now. pyproject.toml PEP 518 introduces the pyproject.toml file as a way to specify build requirements for your project. PEP 621 defines how to store project meta data in it. pip and setuptools support pyproject.toml to some extent, but not to a point where it completely replaces setup.py yet. Many of Python s standard tools allow already for configuration in pyproject.toml so it seems this file will slowly replace the setup.cfg and probably setup.py and requirements.txt as well. But we re not there yet. poetry has an interesting approach: it will allow you to write everything into pyproject.toml and generate a setup.py for you at build-time, so it can be uploaded to PyPI. Ironically, Python settled for the TOML file format here, although there is currently no support for reading TOML files in Python s standard library. Summary While some alternatives exist, in 2021 I still stick to setup.py and requirements.txt to define the meta data and dependencies of my projects. Regarding the tooling, pip and twine are sufficient and do their job just fine. Alternatives like pipenv and poetry exist. The scope of poetry seems to be better aligned with my expectations, and it seems the more mature project compared to pipenv but in any case I ll ignore both of them until I revisit this issue in 2022. Closing Thoughts While the packaging in Python has improved a lot in the last years, I m still somewhat put off how such a core aspect of a programming language is treated within Python. With some jealousy, I look over to the folks at Rust and how they seemed to get this aspect right from the start. What would in my opinion improve the situation?

25 June 2021

Enrico Zini: Parsing YAML

This is part of a series of posts on ideas for an Ansible-like provisioning system, implemented in Transilience. The time has come for me to try and prototype if it's possible to load some Transilience roles from Ansible's YAML instead of Python. The data models of Transilience and Ansible are not exactly the same. Some of the differences that come to mind: To simplify the work, I'll start from loading a single role out of Ansible, not an entire playbook. TL;DR: scroll to the bottom of the post for the conclusion! Loading tasks The first problem of loading an Ansible task is to figure out which of the keys is the module name. I have so far failed to find precise reference documentation about what keyboards are used to define a task, so I'm going by guesswork, and if needed a look at Ansible's sources. My first attempt goes by excluding all known non-module keywords: 
        candidates = []
        for key in task_info.keys():
            if key in ("name", "args", "notify"):
                continue
            candidates.append(key)
        if len(candidates) != 1:
            raise RoleNotLoadedError(f"could not find a known module in task  task_info!r ")
        modname = candidates[0]
        if modname.startswith("ansible.builtin."):
            name = modname[16:]
        else:
            name = modname
This means that Ansible keywords like when or with will break the parsing, and it's fine since they are not supported yet. args seems to carry arguments to the module, when the module main argument is not a dict, as may happen at least with the command module. Task parameters One can do all sorts of chaotic things to pass parameters to Ansible tasks: for example string lists can be lists of strings or strings with comma-separated lists, and they can be preprocesed via Jinja2 templating, and they can be complex data structures that might contain strings that need Jinja2 preprocessing. I ended up mapping the behaviours I encountered in an AST-like class hierarchy which includes recursive complex structures. Variables Variables look hard: Ansible has a big free messy cauldron of global variables, and Transilience needs a predefined list of per-role variables. However, variables are mainly used inside Jinja2 templates, and Jinja2 can parse to an Abstract Syntax Tree and has useful methods to examine its AST. Using that, I managed with resonable effort to scan an Ansible role and generate a list of all the variables it uses! I can then use that list, filter out facts-specific names like ansible_domain, and use them to add variable definition to the Transilience roles. That is exciting! Handlers Before loading tasks, I load handlers as one-action roles, and index them by name. When an Ansible task notifies a handler, I can then loop up by name the roles I generated in the earlier pass, and I have all that I need. Parsed Abstract Syntax Tree Most of the results of all this parsing started looking like an AST, so I changed the rest of the prototype to generate an AST. This means that, for a well defined subset of nsible's YAML, there exists now a tool that is able to parse it into an AST and raeson with it. Transilience's playbooks gained a --ansible-to-ast option to parse an Ansible role and dump the resulting AST as JSON: 
$ ./provision --help
usage: provision [-h] [-v] [--debug] [-C] [--ansible-to-python role]
                 [--ansible-to-ast role]
Provision my VPS
optional arguments:
[...]
  -C, --check           do not perform changes, but check if changes would be
                        needed
  --ansible-to-ast role
                        print the AST of the given Ansible role as understood
                        by Transilience
The result is extremely verbose, since every parameter is itself a node in the tree, but I find it interesting. Here is, for example, a node for an Ansible task which has a templated parameter: 
     
      "node": "task",
      "action": "builtin.blockinfile",
      "parameters":  
        "path":  
          "node": "parameter",
          "type": "scalar",
          "value": "/etc/aliases"
         ,
        "block":  
          "node": "parameter",
          "type": "template_string",
          "value": "root:  postmaster \n % for name, dest in aliases.items() % \n name :  dest \n % endfor % \n"
         
       ,
      "ansible_yaml":  
        "name": "configure /etc/aliases",
        "blockinfile":  ,
        "notify": "reread /etc/aliases"
       ,
      "notify": [
        "RereadEtcAliases"
      ]
     ,
Here's a node for an Ansible template task converted to Transilience's model: 
     
      "node": "task",
      "action": "builtin.copy",
      "parameters":  
        "dest":  
          "node": "parameter",
          "type": "scalar",
          "value": "/etc/dovecot/local.conf"
         ,
        "src":  
          "node": "parameter",
          "type": "template_path",
          "value": "dovecot.conf"
         
       ,
      "ansible_yaml":  
        "name": "configure dovecot",
        "template":  ,
        "notify": "restart dovecot"
       ,
      "notify": [
        "RestartDovecot"
      ]
     ,
Executing The first iteration of prototype code for executing parsed Ansible roles is a little execise in closures and dynamically generated types: 
    def get_role_class(self) -> Type[Role]:
        # If we have handlers, instantiate role classes for them
        handler_classes =  
        for name, ansible_role in self.handlers.items():
            handler_classes[name] = ansible_role.get_role_class()
        # Create all the functions to start actions in the role
        start_funcs = []
        for task in self.tasks:
            start_funcs.append(task.get_start_func(handlers=handler_classes))
        # Function that calls all the 'Action start' functions
        def role_main(self):
            for func in start_funcs:
                func(self)
        if self.uses_facts:
            role_cls = type(self.name, (Role,),  
                "start": lambda host: None,
                "all_facts_available": role_main
             )
            role_cls = dataclass(role_cls)
            role_cls = with_facts(facts.Platform)(role_cls)
        else:
            role_cls = type(self.name, (Role,),  
                "start": role_main
             )
            role_cls = dataclass(role_cls)
        return role_cls
Now that the parsed Ansible role is a proper AST, I'm considering redesigning that using a generic Role class that works as an AST interpreter. Generating Python I maintain a library that can turn an invoice into Python code, and I have a convenient AST. I can't not generate Python code out of an Ansible role! 
$ ./provision --help
usage: provision [-h] [-v] [--debug] [-C] [--ansible-to-python role]
                 [--ansible-to-ast role]
Provision my VPS
optional arguments:
[...]
  --ansible-to-python role
                        print the given Ansible role as Transilience Python
                        code
  --ansible-to-ast role
                        print the AST of the given Ansible role as understood
                        by Transilience
And will you look at this annotated extract: 
$ ./provision --ansible-to-python mailserver
from __future__ import annotations
from typing import Any
from transilience import role
from transilience.actions import builtin, facts
# Role classes generated from Ansible handlers!
class ReloadPostfix(role.Role):
    def start(self):
        self.add(
            builtin.systemd(unit='postfix', state='reloaded'),
            name='reload postfix')
class RestartDovecot(role.Role):
    def start(self):
        self.add(
            builtin.systemd(unit='dovecot', state='restarted'),
            name='restart dovecot')
# The role, including a standard set of facts
@role.with_facts([facts.Platform])
class Role(role.Role):
    # These are the variables used by Jinja2 template files and strings. I need
    # to use Any, since Ansible variables are not typed
    aliases: Any = None
    myhostname: Any = None
    postmaster: Any = None
    virtual_domains: Any = None
    def all_facts_available(self):
        ...
        # A Jinja2 string inside a string list!
        self.add(
            builtin.command(
                argv=[
                    'certbot', 'certonly', '-d',
                    self.render_string('mail. ansible_domain '), '-n',
                    '--apache'
                ],
                creates=self.render_string(
                    '/etc/letsencrypt/live/mail. ansible_domain /fullchain.pem'
                )),
            name='obtain mail.* letsencrypt certificate')
        # A converted template task!
        self.add(
            builtin.copy(
                dest='/etc/dovecot/local.conf',
                src=self.render_file('templates/dovecot.conf')),
            name='configure dovecot',
            # Notify referring to the corresponding Role class!
            notify=RestartDovecot)
        # Referencing a variable collected from a fact!
        self.add(
            builtin.copy(dest='/etc/mailname', content=self.ansible_domain),
            name='configure /etc/mailname',
            notify=ReloadPostfix)
        ...
Conclusion Transilience can load a (growing) subset of Ansible syntax, one role at a time, which contains: The role loader in Transilience now looks for YAML when it does not find a Python module, and runs it pipelined and fast! There is code to generate Python code from an Ansible module: you can take an Ansible role, convert it to Python, and then work on it to add more complex logic, or clean it up for adding it to a library of reusable roles! Next: Ansible conditionals

24 June 2021

Louis-Philippe V ronneau: Hardening Weechat Relays Against RCE on Bullseye

I've been using weechat to connect to IRC since late 2016 and one of its killer feature is relays. They let use other frontends like the Weechat Android app or the amazing Glowing Bear (packaged in Debian Bullseye by yours truly). Sadly, relays also used to be somewhat of a security risk: anyone with access to a relay1 could run scripts on the machine running weechat by using commands such as /exec or /script. Not great. Since version 2.5 (Buster had version 2.3), you can mitigate this risk by setting a command allowlist for relays. Later versions implemented a sane default by blocking the following commands: Sadly, this default didn't make in into Bullseye. If you are running weechat and are using the relays feature, after upgrading to Bullseye, I would recommend you run the following commands in the weechat TUI: 
/set relay.weechat.commands *,!exec,!fset,!set,!unset,!plugin,!script,!python,!perl,!ruby,!lua,!tcl,!guile,!javascript,!php,!secure,!upgrade,!quit
/save
  1. For example, someone steals your phone and connects to IRC via the Weechat app...

23 June 2021

Dirk Eddelbuettel: RcppGSL 0.3.9: Polish and More Builds

Release 0.3.9 of the RcppGSL package arrived at CRAN today, pretty much exactly one year since the last upload. The RcppGSL package provides an interface from R to the GNU GSL by relying on the Rcpp package. This release brings some small documentation and CI polish, and enables builds on the newer (and still experimental) windows UCRT flavor (which will bring native utf-8 chars to Windows, see this and this write-up) thanks to a PR by Jeroen.

Changes in version 0.3.9 (2021-06-23)
  • The pdf vignette was extended by a small subsection (Dirk).
  • The CI setup was updated to use run.sh from r-ci (Dirk).
  • The windows was updated to GSL 2.7, and UCRT support was added (Jeroen in #28).

Courtesy of CRANberries, a summary of changes to the most recent release is also available. More information is on the RcppGSL page. Questions, comments etc should go to the issue tickets at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Enrico Zini: Transilience check mode

This is part of a series of posts on ideas for an ansible-like provisioning system, implemented in Transilience. I added check mode to Transilience, to do everything except perform changes, like Ansible does: 
$ ./provision --help
usage: provision [-h] [-v] [--debug] [-C] [--to-python role]
Provision my VPS
optional arguments:
  -h, --help        show this help message and exit
  -v, --verbose     verbose output
  --debug           verbose output
  -C, --check       do not perform changes, but check if changes would be    NEW!
                    needed                                                   NEW!
It was quite straightforwad to add a new field to the base Action class, and tweak the implementations not to perform changes if it is True: 
# Shortcut function to annotate dataclass fields with documentation metadata
def doc(default: Any, doc: str, **kw):
    return field(default=default, metadata= "doc": doc )
@dataclass
class Action:
    ...
    check: bool = doc(False, "when True, check if the action would perform changes, but do nothing")
Like with Ansible, check mode takes about the same time as a normal run which does not perform changes. Unlike Ansible, with Transilience this is actually pretty fast! ;) Next step: parsing YAML!

22 June 2021

Lisandro Dami n Nicanor P rez Meyer: Creating an app with QML: a heater control

Last week I took the ICS course "Building an Embedded Application with Qt" and now it's time to put the gained knowledge into action. I decided to create an application to (simulate?) a heater control. Why? Because I have a very basic one at home, and I always dreamed of getting something better. So time to implement it.
Requirements General Try to do the business logic in C++ as much as possible. Thermostat Temperature profiles Profile selection The system should be able to configure the desired profile in slots of 15' for each day (Monday to Sunday). So 96 slots per day. Manual button Sometimes I want to override the predefined configuration for a special situation. This button should allow me to set a new target temperature in a time slot from 15 minutes to 3 hours. Heater on indicator We want to know when the heater is being commanded to turn on. Temperature sensor This is not strictly decided, so the ability to use different kinds of sensors, one at a time, would be just nice.
User interface I did some sketches on what I would want as a UI. But I'm not a graphic designer, so I will first do a very simple but yet functional UI and then try to switch to a better designed UI. I dream on emulating an horizontal disc gauge, those where the user sees the border of the disc and the center of it shows the current temperature, perhaps even with a magnifier in the middle. Something like this, but with the needle fixed in the center.
Implementing the idea The backend Code repository at GitLab. My first steps where to implement the C++ backend: a Settings class and a TemperatureEngine class. For the temperature sensor I decided to make a very simple AbstractTemperatureSensor class and also implement a FakeTemperatureSensor. The later will come handy in order to be able to run tests. Later on I can implement other temperature sources like reading an analog voltage from some GPIO, getting the data trough MQTT, etc. The GUI That's definitely Work In Progress :-)

Jamie McClelland: How to Meet Online with Simultaneous Interpretation

May First Movement Technology has been running a public Jitsi Meet instance since well before the pandemic to support Internet-based, video meetings for folks who don't want to rely on corporate and proprietary infrastructure. However (until this week - see below), we haven't been using it for our own meetings for one main reason: simultaneous interpretation. We're an international organization with roots in the US and Mexico and we are committed to building a bi-national leadership with a movement strategy that recongizes the symbolic and practical disaster of the US/Mexico border. As a result, we simply can't hold a meeting without simultaneous interpretation between english and spanish. Up to now, we've worked out a creative way to have mumble meetings with simultaneous interpretation. In short: we have a room for interpretation. If you move into the interpretation room, you hear the interpreter. If you move into the main room, you hear the live voices of the participants. You can switch between rooms as needed. This approach is rock solid, and we benefit from mumble's excellent performance in low bandwidth situations and the availability of mumble clients on both Android and iPhones. However, there are limitations, which include: After years of reviewing the many dead-end threads and issue requests around simultaneous interpretation on the Jitsi boards (and the Big Blue Button boards for that matter) I finally came across the thread that led to the pull request that changed everything. With the ability to control local volume via the Jitsi Meet API, I was able to pull together a very small amount of code to produce Jitsi Simultaneous Interpretation (JSI) - a way to run your Jitsi Meet server with an interpretation slider at the top allowing you to set the volume of the interpreter at any time during the meeting. It's still not perfect - the main problem is that you can't use any of the Jitsi Meet apps - so it runs well on most desktops, but when it comes to cell phones, it only runs (in browser) on modern android phones.

Jamie McClelland: How to Meet Online with Simultaneous Interpretation

May First Movement Technology has been running a public Jitsi Meet instance since well before the pandemic to support Internet-based, video meetings for folks who don t want to rely on corporate and proprietary infrastructure.

21 June 2021

Shirish Agarwal: Accessibility, Freenode and American imperialism.

Accessibility This is perhaps one of the strangest ways and yet also perhaps the straightest way to start the blog post. For the past weeks/months, a strange experience has been there. I am using a Logitech wireless keyboard and mouse for almost a decade. Now, for the past few months and weeks we observed a somewhat rare phenomena . While in-between us we have a single desktop computer. So me and mum take turns to be on the Desktop. At times, however, the system would sit idle and after some time it goes to low-power mode/sleep mode after 30 minutes. Then, when you want to come back, you obviously have to give your login credentials. At times, the keyboard refuses to input any data in the login screen. Interestingly, the mouse still functions. Much more interesting is the fact that both the mouse and the keyboard use the same transceiver sensor to send data. And I had changed batteries to ensure it was not a power issue but still no input :(. While my mother uses and used the power switch (I did teach her how to hold it for few minutes and then let it go) but for self, tried another thing. Using the mouse I logged of the session thinking perhaps some race condition or something might be in the session which was not letting the keystrokes be inputted into the system and having a new session might resolve it. But this was not to be Luckily, on the screen you do have the option to reboot or power off. I did a reboot and lo, behold the system was able to input characters again. And this has happened time and again. I tried to find GOK and failed to remember that GOK had been retired. I looked up the accessibility page on Debian wiki. Very interesting, very detailed but sadly it did not and does not provide the backup I needed. I tried out florence but found that the app. is buggy. Moreover, the instructions provided on the lightdm screen does not work. I do not get the on-screen keyboard while I followed the instructions. Just to be clear this is all on Debian testing which is gonna be Debian stable soonish I even tried the same with xvkbd but no avail. I do use mate as my desktop-manager so maybe the instructions need some refinement ???? $ cat /etc/lightdm/lightdm-gtk-greeter.conf grep keyboard
# a11y-states = states of accessibility features: name save state on exit, -name
disabled at start (default value for unlisted), +name enabled at start. Allowed names: contrast, font, keyboard, reader.
keyboard=xvkbd no-gnome focus &
# keyboard-position = x y[;width height] ( 50%,center -0;50% 25% by default) Works only for onboard
#keyboard= Interestingly, Debian does provide two more on-screen keyboards, matchbox as well as onboard which comes from Ubuntu. While I have both of them installed. I find xvkbd to be enough for my work, the only issue seems to be I cannot get it from the drop-down box of accessibility at the login screen. Just to make sure that I have not gone to Gnome-display manager, I did run

$ sudo dpkg-reconfigure gdm3 Only to find out that I am indeed running lightdm. So I am a bit confused why it doesn t come up as an option when I have the login window/login manager running. FWIW I do run metacity as the window manager as it plays nice with all the various desktop environments I have, almost all of them. So this is where I m stuck. If I do get any help, I probably would also add those instructions to the wiki page, so it would be convenient to the next person who comes with the same issue. I also need to figure out some way to know whether there is some race-condition or something which is happening, have no clue how would I go about it without having whole lot of noise. I am sure there are others who may have more of an idea. FWIW, I did search unix.stackexchange as well as reddit/debian to see if I could see any meaningful posts but came up empty.

Freenode I had not been using IRC for quite some time now. The reasons have been multiple issues with Riot (now element) taking the whole space on my desktop. I did get alerted to the whole thing about a week after the whole thing went down. Somebody messaged me DM. I *think* I put up a thread or a mini-thread about IRC or something in response to somebody praising telegram/WhatsApp or one of those apps. That probably triggered the DM. It took me a couple of minutes to hit upon this. I was angry and depressed, seeing the behavior of the new overlords of freenode. I did see that lot of channels moved over to Libera. It was also interesting to see that some communities were thinking of moving to some other obscure platform, which again could be held hostage to the same thing. One could argue one way or the other, but that would be tiresome and fact is any network needs lot of help to be grown and nurtured, whether it is online or offline. I also saw that Libera was also using a software Solanum which is ircv3 compliant. Now having done this initial investigation, it was time to move to an IRC client. The Libera documentation is and was pretty helpful in telling which IRC clients would be good with their network. So I first tried hexchat. I installed it and tried to add Libera server credentials, it didn t work. Did see that they had fixed the bug in sid/unstable and now it s in testing. But at the time it was in sid, the bug-fixed and I wanted to have something which just ran the damn thing. I chanced upon quassel. I had played around with quassel quite a number of times before, so I knew I could play/use it. Hence, I installed it and was able to use it on the first try. I did use the encrypted server and just had to tweak some settings before I could use it with some help with their documentation. Although, have to say that even quassel upstream needs to get its documentation in order. It is just all over the place, and they haven t put any effort into streamlining the documentation, so that finding things becomes easier. But that can be said of many projects upstream. There is one thing though that all of these IRC clients lack. The lack of a password manager. Now till that isn t fixed it will suck because you need another secure place to put your password/s. You either put it on your desktop somewhere (insecure) or store it in the cloud somewhere (somewhat secure but again need to remember that password), whatever you do is extra work. I am sure there will be a day when authenticating with Nickserv will be an automated task and people can just get on talking on channels and figuring out how to be part of the various communities. As can be seen, even now there is a bit of a learning curve for both newbies and people who know a bit about systems to get it working. Now, I know there are a lot of things that need to be fixed in the anonymity, security place if I put that sort of hat. For e.g. wouldn t it be cool if either the IRC client or one of its add-on gave throwaway usernames and passwords. The passwords would be complex. This would make it easier who are paranoid about security and many do and would have. As an example we can see of Fuchs. Now if the gentleman or lady is working in a professional capacity and would come to know of their real identity and perceive rightly or wrongly the role of that person, it will affect their career. Now, should it? I am sure a lot of people would be divided on the issue. Personally, as far as I am concerned, I would say no because whether right or wrong, whatever they were doing they were doing on their own time. Not on company time. So it doesn t concern the company at all. If we were to let companies police the behavior outside the time, individuals would be in a lot of trouble. Although, have to say that is a trend that has been seen in companies that are firing people either on the left or right. A recent example that comes to mind is Emily Wilder who was fired by Associated Press. Interestingly, she was interviewed by Democracy now, and it did come out that she is a Jew. As can be seen and understood there is a lot of nuance to her story and not the way she was fired. It doesn t give a good taste in the mouth, but then getting fired nobody does. On few forums, people did share of people getting fired of their job because they were dancing (cops). Again, it all depends, for me again, hats off to anybody who feels like dancing or whatever because there are just so many depressing stories all around.

Banned and FOE On few forums I was banned because I was talking about Brexit and American imperialism, both of which are seem to ruffle a few feathers in quite a few places. For instance, many people for obvious reasons do not like this video

Now I m sorry I am not able to and have not been able to give invidious links for the past few months. The reason being invidious itself went through some changes and the changes are good and bad. For e.g. now you need to share your google id with a third-party which at least to my mind is not a good idea. But that probably is another story altogether and it probably will need its own place. Coming back to the video itself, this was shared by Anthony hazard and the Title is The Atlantic slave trade: What too few textbooks told you . I did see this video quite a few years ago and still find it hard to swallow that tens of millions of Africans were bought as slaves to the Americas, although to be fair it does start with the Spanish settlement in the land which would be called the U.S. but they bought slaves with themselves. They even got the American natives, i.e. people from different tribes which made up America at that point. One point to note is that U.S. got its independence on July 4, 1776 so all the people before that were called as European settlers for want of a better word. Some or many of these European settlers would be convicts who were sent from UK. But as shared in the article, that would only happen with U.S. itself is mature and open enough for that discussion. Going back to the original point though, these European or American settlers bought lot of slaves from Africa. The video does also shed some of the cruelty the Europeans or Americans did on the slaves, men and women in different ways. The most revelatory part though which I also forget many a times that because lot of people were taken from Africa and many of them men, it did lead to imbalances in the African societies not just in weddings but economics in general. It also developed a theory called Critical Race theory in which it tries to paint the Africans as an inferior race otherwise how would Christianity work where their own good book says All men are born equal . That does in part explain why the African countries are still so far behind their European or American counterparts. But Africa can still be proud as they are richer than us, yup India. Sadly, I don t think America is ready to have that conversation anytime soon or if ever. And if it were to do, it would have to out-do any truth and reconciliation Committee which the world has seen. A mere apology or two would not just cut it. The problems of America sadly are not limited to just Africans but the natives of the land, for e.g. the Lakota people. In 1868, they put a letter stating we will give the land back to the Lakota people forever, but then the gold rush happened. In 2007, when the Lakota stated their proposal for independence, the U.S. through its force denied. So much for the paper, it was written on. Now from what I came to know over the years, the American natives are called First nations . Time and time again the American Govt. has tried or been foul towards them. Some of the examples include The Yucca Mountain nuclear waste repository . The same is and was the case with The Keystone pipeline which is now dead. Now one could say that it is America s internal matter and I would fully agree but when they speak of internal matters of other countries, then we should have the same freedom. But this is not restricted to just internal matters, sadly. Since the 1950 s i.e. the advent of the cold war, America s foreign policy made Regime changes all around the world. Sharing some of the examples from the Cold War

Iran 1953
Guatemala 1954
Democratic Republic of the Congo 1960
Republic of Ghana 1966
Iraq 1968
Chile 1973
Argentina 1976
Afghanistan 1978-1980s
Grenada
Nicaragua 1981-1990
1. Destabilization through CIA assets
2. Arming the Contras
El Salvador 1980-92
Philippines 1986 Even after the Cold War ended the situation was anonymolus, meaning they still continued with their old behavior. After the end of Cold War

Guatemala 1993
Serbia 2000
Iraq 2003-
Afghanistan 2001 ongoing There is a helpful Wikipedia article titled History of CIA which basically lists most of the covert regime changes done by U.S. The abvoe is merely a sub-set of the actions done by U.S. Now are all the behaviors above of a civilized nation ? And if one cares to notice, one would notice that all the above countries in the list which had the regime change had either Oil or precious metals. So U.S. is and was being what it accuses China, a profiteer. But this isn t just the U.S. China story but more about the American abuse of its power. My own country, India paid IMF loans till 1991 and we paid through the nose. There were economic sanctions against India. But then, this is again not just about U.S. India. Even with Europe or more precisely Norway which didn t want to side with America because their intelligence showed that no WMD were present in Iraq, the relationship still has issues.

Pandemic and the World So I do find that this whole blaming of China by U.S. quite theatrical and full of double-triple standards. Very early during the debates, it came to light that the Spanish Flu actually originated in Kensas, U.S.

What was also interesting as I found in the Pentagon Papers much before The Watergate scandal came out that U.S. had realized that China would be more of a competitor than Russia. And this itself was in 1960 s itself. This shows the level of intelligence that the Americans had. From what I can recollect from whatever I have read of that era, China was still mostly an agri-based economy. So, how the U.S. was able to deduce that China will surpass other economies is beyond me even now. They surely must have known something that even we today do not. One of the other interesting observations and understanding that I got while researching that every year we transfer an average of 7500 diseases from animal to humans and that should be a scary figure. I think more than anything else, loss of habitat and use of animals from food to clothing to medicine is probably the reason we are getting such diseases. I am also sure that there probably are and have been similar number of transfer of diseases from humans to animals as well but for well-known biases and whatnot those studies are neither done or are under-funded. There are and have been reports of something like 850,000 undiscovered viruses which various mammals and birds have. Also I did find that most of such pandemics are hard to identify, for e.g. SARS 1 took about 15 years, Ebola we don t know till date from where it came. Even HIV has questions for us. Hell, even why does hearing go away is a mystery to us. In all of this, we want to say China is culpable. And while China may or may not be culpable, only time will tell, this is surely the opportunity for all countries to spend and make capacities in public health. Countries which will take lessons from it and improve their public healthcare models will hopefully will not suffer as those who will suffer and are continuing to suffer now To those who feel that habitat loss of animals is untrue, I would suggest them to see Sherni which depicts the human/animal conflict in all its brutality. I am gonna warn in advance that the ending is not nice but what can you expect from a country in which forest area cover has constantly declined and the Govt. itself is only interested in headline management

The only positive story I can share from India is that finally the Modi Govt. has said we will do free vaccine immunization for everybody. Although the pace is nothing to write home about. One additional thing they relaxed was instead of going to Cowin or any other portal, people could simply walk in using their identity papers. Although, given the pace of vaccinations, it is going to take anywhere between 13-18 months or more depending on availability of vaccines.

Looking forward to all and any replies have a virtual keyboard, preferably xvkbd as that is good enough for my use-case.

Lisandro Dami n Nicanor P rez Meyer: Firsts steps into QML

After years of using and maintaining Qt there was a piece of the SDK that I never got to use as a developer: QML. Thanks to ICS I've took the free (in the sense of cost) QML Programming Fundamentals and Beyond. It consists of seven sessions, which can be easily done in a few days. I did them all in 4 days, but with enough time available you can do them even faster. Of course some previous knowledge of Qt comes handy. The only drawback was the need of a corporate e-mail in order to register (or at least the webpage says so). Apart from that it is really worth the effort. So, if you are planning into getting into QML this is definitely a nice way to start.

Russ Allbery: Review: Demon Lord of Karanda

Review: Demon Lord of Karanda, by David Eddings
Series: The Malloreon #3
Publisher: Del Rey
Copyright: September 1988
Printing: February 1991
ISBN: 0-345-36331-0
Format: Mass market
Pages: 404
This is the third book of the Malloreon, which in turn is a sequel trilogy to The Belgariad. Eddings, unlike most series authors, does a great job of reminding you what's happening with prologues in each book, but you definitely do not want to start reading here. When we last left our heroes, they had been captured. (This is arguably a spoiler for King of the Murgos, but it's not much of one, nor one that really matters.) This turns out to be an opportunity to meet the Emperor of Mallorea, the empire from which their adversary Zandramas (and, from the earlier trilogy, the god Torak) comes. This goes much better than one might expect, continuing the trend in this series of showing the leaders of the enemy countries as substantially similar to the leaders of the supposedly good countries. This sounds like open-mindedness on Eddings's part, and I suppose it partly is. It's at least a change from the first series, in which the bad guys were treated more like orcs. But the deeper I read into this series, the more obvious how invested Eddings is in a weird sort of classism. Garion and the others get along with Zakath in part because they're all royalty, or at least run in those circles. They just disagree about how to be a good ruler (and not as much as one might think, or hope). The general population of any of the countries is rarely of much significance. Zakath is a bit more cynical than Garion and company and has his own agenda, but he's not able to overcome the strong conviction of this series that the Prophecy and the fight between the Child of Light and the Child of Dark is the only thing of importance that's going on, and other people matter only to the extent that they're involved in that story. When I first read these books as a teenager, I was one of the few who liked the second series and didn't mind that its plot was partly a rehash of the first. I found, and still find, the blatantness with which Eddings manipulates the plot by making prophecy a character in the novel amusing. What I had forgotten, however, was how much of a slog the middle of this series is. It takes about three quarters of this book before there are any significant plot developments, and that time isn't packed with interesting diversions. It's mostly the heroes having conversations with each other or with Zakath, being weirdly sexist, rehashing their personality quirks, or shrugging about horrific events that don't matter to them personally. The last is a reference to the plague that appears in this book, and which I had completely forgotten. To be fair to my memory, that's partly because none of the characters seem to care much about it either. They're cooling their heels in a huge city, a plague starts killing people, they give Zakath amazingly brutal and bloodthirsty advice to essentially set fire to all the parts of the city with infected people, and then they blatantly ignore all the restrictions on movement because, well, they're important unlike all those other people and have places to go. It's rather stunningly unempathetic under the best of circumstances and seems even more vile in a 2021 re-read. Eddings also manages to make Ce'Nedra even more obnoxious than she has been by turning her into a walking zombie with weird fits where she's obsessed with her child, and adding further problems (which would be a spoiler) on top of that. I have never been a Ce'Nedra fan (that Garion's marriage ever works at all appears to be by authorial decree), but in this book she's both useless and irritating while supposedly being a tragic figure. You might be able to tell that I'm running sufficiently low on patience for Eddings's character quirks that I'm losing my enthusiasm for re-reading this bit of teenage nostalgia. This is the third book of a five-book series, so while there's a climax of sorts just like there was in the third book of the Belgariad, it's a false climax. One of the secondary characters is removed, but nothing is truly resolved; the state of the plot isn't much different at the end of this book than it was at the start. And to get there, one has to put up with Garion being an idiot, Ce'Nedra being a basket case, the supposed heroes being incredibly vicious about a plague, and one character pretending to have an absolutely dreadful Irish accent for pages upon pages upon pages. (His identity is supposedly a mystery, but was completely obvious a hundred pages before Garion figured it out. Garion isn't the sharpest knife in the drawer.) The one redeeming merit to this series is the dry voice in Garion's head and the absurd sight of the prophecy telling all the characters what to do, and we barely get any of that in this book. When it wasn't irritating or offensive, it was just a waste of time. The worst book of the series so far. Followed by Sorceress of Darshiva. Rating: 3 out of 10

20 June 2021

Mike Gabriel: BBB Packaging for Debian, a short Heads-Up

Over the past days, I have received tons of positive feedback on my previous blog post about forming the Debian BBB Packaging Team [1]. Feedback arrived via mail, IRC, [matrix] and Mastodon. Awesome. Thanks for sharing your thoughts, folks... Therefore, here comes a short ... Heads-Up on the current Ongoings ... around packaging BigBlueButton for Debian: Credits light+love
Mike Gabriel

[1] https://sunweavers.net/blog/node/133
[2] https://bigbluebutton.org/event-page/
[3] https://docs.google.com/document/d/1kpYJxYFVuWhB84bB73kmAQoGIS59ari1_hn2...

Julian Andres Klode: Migrating away from apt-key

This is an edited copy of an email I sent to provide guidance to users of apt-key as to how to handle things in a post apt-key world. The manual page already provides all you need to know for replacing apt-key add usage:
Note: Instead of using this command a keyring should be placed directly in the /etc/apt/trusted.gpg.d/ directory with a descriptive name and either gpg or asc as file extension
So it s kind of surprising people need step by step instructions for how to copy/download a file into a directory. I ll also discuss the alternative security snakeoil approach with signed-by that s become popular. Maybe we should not have added signed-by, people seem to forget that debs still run maintainer scripts as root. Aside from this email, Debian users should look into extrepo, which manages curated external repositories for you.

Direct translation Assume you currently have: 
wget -qO- https://myrepo.example/myrepo.asc   sudo apt-key add
To translate this directly for bionic and newer, you can use: 
sudo wget -qO /etc/apt/trusted.gpg.d/myrepo.asc https://myrepo.example/myrepo.asc
or to avoid downloading as root: 
wget -qO-  https://myrepo.example/myrepo.asc   sudo tee -a /etc/apt/trusted.gpg.d/myrepo.asc
Older (and all) releases only support unarmored files with an extension .gpg. If you care about them, provide one, and use 
sudo wget -qO /etc/apt/trusted.gpg.d/myrepo.gpg https://myrepo.example/myrepo.gpg
Some people will tell you to download the .asc and pipe it to gpg --dearmor, but gpg might not be installed, so really, just offer a .gpg one instead that is supported on all systems. wget might not be available everywhere so you can use apt-helper: 
sudo /usr/lib/apt/apt-helper download-file https://myrepo.example/myrepo.asc /etc/apt/trusted.gpg.d/myrepo.asc
or, to avoid downloading as root: 
/usr/lib/apt/apt-helper download-file https://myrepo.example/myrepo.asc /tmp/myrepo.asc && sudo mv /tmp/myrepo.asc /etc/apt/trusted.gpg.d

Pretending to be safer by using signed-by People say it s good practice to not use trusted.gpg.d and install the file elsewhere and then refer to it from the sources.list entry by using signed-by=<path to the file>. So this looks a lot safer, because now your key can t sign other unrelated repositories. In practice, security increase is minimal, since package maintainer scripts run as root anyway. But I guess it s better for publicity :) As an example, here are the instructions to install signal-desktop from signal.org. As mentioned, gpg --dearmor use in there is not a good idea, and I d personally not tell people to modify /usr as it s supposed to be managed by the package manager, but we don t have an /etc/apt/keyrings or similar at the moment; it s fine though if the keyring is installed by the package. You can also just add the file there as a starting point, and then install a keyring package overriding it (pretend there is a signal-desktop-keyring package below that would override the .gpg we added). 
# NOTE: These instructions only work for 64 bit Debian-based
# Linux distributions such as Ubuntu, Mint etc.
# 1. Install our official public software signing key
wget -O- https://updates.signal.org/desktop/apt/keys.asc   gpg --dearmor > signal-desktop-keyring.gpg
cat signal-desktop-keyring.gpg   sudo tee -a /usr/share/keyrings/signal-desktop-keyring.gpg > /dev/null
# 2. Add our repository to your list of repositories
echo 'deb [arch=amd64 signed-by=/usr/share/keyrings/signal-desktop-keyring.gpg] https://updates.signal.org/desktop/apt xenial main'  \
  sudo tee -a /etc/apt/sources.list.d/signal-xenial.list
# 3. Update your package database and install signal
sudo apt update && sudo apt install signal-desktop
I do wonder why they do wget gpg --dearmor, pipe that into the file and then cat sudo tee it, instead of having that all in one pipeline. Maybe they want nicer progress reporting.

Scenario-specific guidance We have three scenarios: For system image building, shipping the key in /etc/apt/trusted.gpg.d seems reasonable to me; you are the vendor sort of, so it can be globally trusted. Chrome-style debs and repository config debs: If you ship a deb, embedding the sources.list.d snippet (calling it $myrepo.list) and shipping a $myrepo.gpg in /usr/share/keyrings is the best approach. Whether you ship that in product debs aka vscode/chromium or provide a repository configuration deb (let s call it myrepo-repo.deb) and then tell people to run apt update followed by apt install <package inside the repo> depends on how many packages are in the repo, I guess. Manual instructions (signal style): The third case, where you tell people to run wget themselves, I find tricky. As we see in signal, just stuffing keyring files into /usr/share/keyrings is popular, despite /usr supposed to be managed by the package manager. We don t have another dir inside /etc (or /usr/local), so it s hard to suggest something else. There s no significant benefit from actually using signed-by, so it s kind of extra work for little gain, though.

Addendum: Future work This part is new, just for this blog post. Let s look at upcoming changes and how they make things easier.

Bundled .sources files Assuming I get my merge request merged, the next version of APT (2.4/2.3.something) will do away with all the complexity and allow you to embed the key directly into a deb822 .sources file (which have been available for some time now): 
Types: deb
URIs: https://myrepo.example/ https://myotherrepo.example/
Suites: stable not-so-stable
Components: main
Signed-By:
 -----BEGIN PGP PUBLIC KEY BLOCK-----
 .
 mDMEYCQjIxYJKwYBBAHaRw8BAQdAD/P5Nvvnvk66SxBBHDbhRml9ORg1WV5CvzKY
 CuMfoIS0BmFiY2RlZoiQBBMWCgA4FiEErCIG1VhKWMWo2yfAREZd5NfO31cFAmAk
 IyMCGyMFCwkIBwMFFQoJCAsFFgIDAQACHgECF4AACgkQREZd5NfO31fbOwD6ArzS
 dM0Dkd5h2Ujy1b6KcAaVW9FOa5UNfJ9FFBtjLQEBAJ7UyWD3dZzhvlaAwunsk7DG
 3bHcln8DMpIJVXht78sL
 =IE0r
 -----END PGP PUBLIC KEY BLOCK-----
Then you can just provide a .sources files to users, they place it into sources.list.d, and everything magically works Probably adding a nice apt add-source command for it I guess. Well, python-apt s aptsources package still does not support deb822 sources, and never will, we ll need an aptsources2 for that for backwards-compatibility reasons, and then port software-properties and other users to it.

OpenPGP vs aptsign We do have a better, tighter replacement for gpg in the works which uses Ed25519 keys to sign Release files. It s temporarily named aptsign, but it s a generic signer for single-section deb822 files, similar to signify/minisign. We believe that this solves the security nightmare that our OpenPGP integration is while reducing complexity at the same time. Keys are much shorter, so the bundled sources file above will look much nicer.

Russ Allbery: Review: The Magician's Nephew

Review: The Magician's Nephew, by C.S. Lewis
Illustrator: Pauline Baynes
Series: Chronicles of Narnia #6
Publisher: Collier Books
Copyright: 1955
Printing: 1978
ISBN: 0-02-044230-0
Format: Mass market
Pages: 186
The Magician's Nephew is the sixth book of the Chronicles of Narnia in the original publication order, but it's a prequel, set fifty years before The Lion, the Witch and the Wardrobe. It's therefore put first in the new reading order. I have always loved world-building and continuities and, as a comics book reader (Marvel primarily), developed a deep enjoyment of filling in the pieces and reconstructing histories from later stories. It's no wonder that I love reading The Magician's Nephew after The Lion, the Witch and the Wardrobe. The experience of fleshing out backstory with detail and specifics makes me happy. If that's also you, I recommend the order in which I'm reading these books. Reading this one first is defensible, though. One of the strongest arguments for doing so is that it's a much stronger, tighter, and better-told story than The Lion, the Witch and the Wardrobe, and therefore might start the series off on a better foot for you. It stands alone well; you don't need to know any of the later events to enjoy this, although you will miss the significance of a few things like the lamp post and you don't get the full introduction to Aslan. The Magician's Nephew is the story of Polly Plummer, her new neighbor Digory Kirke, and his Uncle Andrew, who fancies himself a magician. At the start of the book, Digory's mother is bed-ridden and dying and Digory is miserable, which is the impetus for a friendship with Polly. The two decide to explore the crawl space of the row houses in which they live, seeing if they can get into the empty house past Digory's. They don't calculate the distances correctly and end up in Uncle Andrew's workroom, where Digory was forbidden to go. Uncle Andrew sees this as a golden opportunity to use them for an experiment in travel to other worlds. MAJOR SPOILERS BELOW. The Magician's Nephew, like the best of the Narnia books, does not drag its feet getting started. It takes a mere 30 pages to introduce all of the characters, establish a friendship, introduce us to a villain, and get both of the kids into another world. When Lewis is at his best, he has an economy of storytelling and a grasp of pacing that I wish was more common. It's also stuffed to the brim with ideas, one of the best of which is the Wood Between the Worlds. Uncle Andrew has crafted pairs of magic rings, yellow and green, and tricks Polly into touching one of the yellow ones, causing her to vanish from our world. He then uses her plight to coerce Digory into going after her, carrying two green rings that he thinks will bring people back into our world, and not incidentally also observing that world and returning to tell Uncle Andrew what it's like. But the world is more complicated than he thinks it is, and the place where the children find themselves is an eerie and incredibly peaceful wood, full of grass and trees but apparently no other living thing, and sprinkled with pools of water. This was my first encounter with the idea of a world that connects other worlds, and it remains the most memorable one for me. I love everything about the Wood: the simplicity of it, the calm that seems in part to be a defense against intrusion, the hidden danger that one might lose one's way and confuse the ponds for each other, and even the way that it tends to make one lose track of why one is there or what one is trying to accomplish. That quiet forest filled with pools is still an image I use for infinite creativity and potential. It's quiet and nonthreatening, but not entirely inviting either; it's magnificently neutral, letting each person bring what they wish to it. One of the minor plot points of this book is that Uncle Andrew is wrong about the rings because he's wrong about the worlds. There aren't just two worlds; there are an infinite number, with the Wood as a nexus, and our reality is neither the center nor one of an important pair. The rings are directional, but relative to the Wood, not our world. The kids, who are forced to experiment and who have open minds, figure this out quickly, but Uncle Andrew never shifts his perspective. This isn't important to the story, but I've always thought it was a nice touch of world-building. Where this story is heading, of course, is the creation of Narnia and the beginning of all of the stories told in the rest of the series. But before that, the kids's first trip out of the Wood is to one of the best worlds of children's fantasy: Charn. If the Wood is my mental image of a world nexus, Charn will forever be my image of a dying world: black sky, swollen red sun, and endless abandoned and crumbling buildings as far as the eye can see, full of tired silences and eerie noises. And, of course, the hall of statues, with one of the most memorable descriptions of history and empire I've ever read (if you ignore the racialized description):
All of the faces they could see were certainly nice. Both the men and women looked kind and wise, and they seemed to come of a handsome race. But after the children had gone a few steps down the room they came to faces that looked a little different. These were very solemn faces. You felt you would have to mind your P's and Q's, if you ever met living people who looked like that. When they had gone a little farther, they found themselves among faces they didn't like: this was about the middle of the room. The faces here looked very strong and proud and happy, but they looked cruel. A little further on, they looked crueller. Further on again, they were still cruel but they no longer looked happy. They were even despairing faces: as if the people they belonged to had done dreadful things and also suffered dreadful things.
The last statue is of a fierce, proud woman that Digory finds strikingly beautiful. (Lewis notes in an aside that Polly always said she never found anything specially beautiful about her. Here, as in The Silver Chair, the girl is the sensible one and things would have gone better if the boy had listened to her, a theme that I find immensely frustrating because Susan was the sensible one in the first two books of the series but then Lewis threw that away.) There is a bell in the middle of this hall, and the pillar that holds that bell has an inscription on it that I think every kid who grew up on Narnia knows by heart.
Make your choice, adventurous Stranger;
Strike the bell and bide the danger,
Or wonder, till it drives you mad,
What would have followed if you had.
Polly has no intention of striking the bell, but Digory fights her and does it anyway, waking Jadis from where she sat as the final statue in the hall and setting off one of the greatest reimaginings of a villain in children's literature. Jadis will, of course, become the White Witch who holds Narnia in endless winter some thousand Narnian years later. But the White Witch was a mediocre villain at best, the sort of obvious and cruel villain common in short fairy tales where the author isn't interested in doing much characterization. She exists to be evil, do bad things, and be defeated. She has a few good moments in conflict with Aslan, but that's about it. Jadis in this book is another matter entirely: proud, brilliant, dangerous, and creative. The death of everything on Charn was Jadis's doing: an intentional spell, used to claim a victory of sorts from the jaws of defeat by her sister in a civil war. (I find it fascinating that Lewis puts aside his normally sexist roles here.) Despite the best attempts of the kids to lose her both in Charn and in the Wood (which is inimical to her, in another nice bit of world-building), she manages to get back to England with them. The result is a remarkably good bit of villain characterization. Jadis is totally out of her element, used to a world-spanning empire run with magic and (from what hints we get) vaguely medieval technology. Her plan to take over their local country and eventually the world should be absurd and is played somewhat for laughs. Her magic, which is her great weapon, doesn't even work in England. But Jadis learns at a speed that the reader can watch. She's observant, she pays attention to things that don't fit her expectations, she changes plans, and she moves with predatory speed. Within a few hours in London she's stolen jewels and a horse and carriage, and the local police seem entirely overmatched. There's no way that one person without magic should be a real danger to England around the turn of the 20th century, but by the time the kids manage to pull her back into the Wood, you're not entirely sure England would have been safe. A chaotic confrontation, plus the ability of the rings to work their magic through transitive human contact, ends up with the kids, Uncle Andrew, Jadis, a taxicab driver and his horse all transported through the Wood to a new world. In this case, literally a new world: Narnia at the point of its creation. Here again, Lewis translates Christian myth, in this case the Genesis creation story, into a more vivid and in many ways more beautiful story than the original. Aslan singing the world into existence is an incredible image, as is the newly-created world so bursting with life that even things that normally could not grow will do so. (Which, of course, is why there is a lamp post burning in the middle of the western forest of Narnia for the Pevensie kids to find later.) I think my favorite part is the creation of the stars, but the whole sequence is great. There's also an insightful bit of human psychology. Uncle Andrew can't believe that a lion is singing, so he convinces himself that Aslan is not singing, and thus prevents himself from making any sense of the talking animals later.
Now the trouble about trying to make yourself stupider than you really are is that you very often succeed.
As with a lot in Lewis, he probably meant this as a statement about faith, but it generalizes well beyond the religious context. What disappointed me about the creation story, though, is the animals. I didn't notice this as a kid, but this re-read has sensitized me to how Lewis consistently treats the talking animals as less than humans even though he celebrates them. That happens here too: the newly-created, newly-awakened animals are curious and excited but kind of dim. Some of this is an attempt to show that they're young and are just starting to learn, but it also seems to be an excuse for Aslan to set up a human king and queen over them instead of teaching them directly how to deal with the threat of Jadis who the children inadvertently introduced into the world. The other thing I dislike about The Magician's Nephew is that the climax is unnecessarily cruel. Once Digory realizes the properties of the newly-created world, he hopes to find a way to use that to heal his mother. Aslan points out that he is responsible for Jadis entering the world and instead sends him on a mission to obtain a fruit that, when planted, will ward Narnia against her for many years. The same fruit would heal his mother, and he has to choose Narnia over her. (It's a fairly explicit parallel to the Garden of Eden, except in this case Digory passes.) Aslan, in the end, gives Digory the fruit of the tree that grows, which is still sufficient to heal his mother, but this sequence made me angry when re-reading it. Aslan knew all along that what Digory is doing will let him heal his mother as well, but hides this from him to make it more of a test. It's cruel and mean; Aslan could have promised to heal Digory's mother and then seen if he would help Narnia without getting anything in return other than atoning for his error, but I suppose that was too transactional for Lewis's theology or something. Meh. But, despite that, the only reason why this is not the best Narnia book is because The Voyage of the Dawn Treader is the only Narnia book that also nails the ending. The Magician's Nephew, up through Charn, Jadis's rampage through London, and the initial creation of Narnia, is fully as good, perhaps better. It sags a bit at the end, partly because it tries to hard to make the Narnian animals humorous and partly because of the unnecessary emotional torture of Digory. But this still holds up as the second-best Narnia book, and one I thoroughly enjoyed re-reading. If anything, Jadis and Charn are even better than I remembered. Followed by the last book of the series, the somewhat notorious The Last Battle. Rating: 9 out of 10

Sean Whitton: transient-caps-lock

If you re writing a lot of Common Lisp and you want to follow the convention of using all uppercase to refer to symbols in docstrings, comments etc., you really need something better than the shift key. Similarly if you re writing C and you have VARIOUS_LONG_ENUMS. The traditional way is a caps lock key. But that means giving up a whole keyboard key, all of the time, just for block capitalisation, which one hardly uses outside of programming. So a better alternative is to come up with some Emacs thing to get block capitalisation, as Emacs key binding is much more flexible than system keyboard layouts, and can let us get block capitalisation without giving up a whole key. The simplest thing would be to bind some sequence of keys to just toggle caps lock. But I came up with something a bit fancier. With the following, you can type M-C, and then you get block caps until the point at which you ve probably finished typing your symbol or enum name. 
(defun spw/transient-caps-self-insert (&optional n)
  (interactive "p")
  (insert-char (upcase last-command-event) n))
(defun spw/activate-transient-caps ()
  "Activate caps lock while typing the current whitespace-delimited word(s).
This is useful for typing Lisp symbols and C enums which consist
of several all-uppercase words separated by hyphens and
underscores, such that M-- M-u after typing will not upcase the
whole thing."
  (interactive)
  (let* ((map (make-sparse-keymap))
     (deletion-commands &apos(delete-backward-char
                          paredit-backward-delete
                          backward-kill-word
                          paredit-backward-kill-word
                          spw/unix-word-rubout
                          spw/paredit-unix-word-rubout))
     (typing-commands (cons &aposspw/transient-caps-self-insert
                            deletion-commands)))
     (substitute-key-definition &aposself-insert-command
                                #&aposspw/transient-caps-self-insert
                                map
                                (current-global-map))
    (set-transient-map
     map
     (lambda ()
       ;; try to determine whether we are probably still about to try to type
       ;; something in all-uppercase
       (and (member this-command typing-commands)
            (not (and (eq this-command &aposspw/transient-caps-self-insert)
                      (= (char-syntax last-command-event) ?\ )))
            (not (and (or (bolp) (= (char-syntax (char-before)) ?\ ))
                      (member this-command deletion-commands))))))))
(global-set-key "\M-C" #&aposspw/activate-transient-caps)
A few notes:

Next.

Previous.