VIRTFS="-virtfs local,path=/vmstore/virtfs,security_model=mapped-xattr,id=zz,writeout=immediate,fmode=0600,dmode=0700,mount_tag=zz"
VIRTFS="-virtfs local,path=/vmstore/virtfs,security_model=passthrough,id=zz,writeout=immediate,mount_tag=zz"

• [1] https://wiki.qemu.org/Documentation/9psetup

int foo[5];
...
foo[8] = bar;

/* 1-element array */
struct foo  
    ...
    char contents[1];
 ;
/* GNU C extension: 0-element array */
struct foo  
    ...
    char contents[0];
 ;
/* C standard: flexible array */
struct foo  
    ...
    char contents[];
 ;
instance = kmalloc(sizeof(struct foo) + content_size);

int foo[5];
...
for (i = 0; i < some_argument; i++)  
    ...
    foo[i] = bar;
    ...
 

/* safe if strlen(foo) + 1 < sizeof(string) */
wrote  = sprintf(string, "Foo: %s\n", foo);
/* overflows if strlen(foo) + strlen(bar) > sizeof(string) */
wrote += sprintf(string + wrote, "Bar: %s\n", bar);
/* writing way beyond the end of "string" now ... */
wrote += sprintf(string + wrote, "Baz: %s\n", baz);

/* safe, assuming available <= sizeof(string), and for this example
 * assume strlen(foo) < sizeof(string) */
wrote  = snprintf(string, available, "Foo: %s\n", foo);
/* if (strlen(bar) > available - wrote), this is still safe since the
 * write into "string" will be truncated, but now "wrote" has been
 * incremented by how much snprintf() *would* have written, so "wrote"
 * is now larger than "available". */
wrote += snprintf(string + wrote, available - wrote, "Bar: %s\n", bar);
/* string + wrote is beyond the end of string, and availabe - wrote wraps
 * around to a giant positive value, making the write effectively 
 * unbounded. */
wrote += snprintf(string + wrote, available - wrote, "Baz: %s\n", baz);

2020, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 License.

TL;DR: IKR! HOW DO I FIX THIS!? Make sure you have SUBSCRIBE_FORM_SECRET set in your mailman configuration:

SECRET=$(tr -dc 'A-Za-z0-9' < /dev/urandom   head -c 30)'
echo "SUBSCRIBE_FORM_SECRET = '$SECRET'" >> /etc/mailman/mm.cfg

This will add a magic token to all forms in the Mailman web forms that will force the attacker to at least get a token before asking for registration. There are, of course, other ways of performing the attack then, but it's more expensive than a single request for the attacker and keeps most of the junk out.

Other solutions I originally deployed a different fix, using referrer checks and an IP block list:

RewriteMap hosts-deny  txt:/etc/apache2/blocklist.txt
RewriteCond $ hosts-deny:% REMOTE_ADDR NOT-FOUND  !=NOT-FOUND [OR]
RewriteCond $ hosts-deny:% REMOTE_HOST NOT-FOUND  !=NOT-FOUND [OR]
RewriteCond % HTTP_REFERER  !^https://lists.torproject.org/$ [NC]
RewriteRule ^/cgi-bin/mailman/subscribe/ - [F]
# see also https://www.w3.org/TR/referrer-policy/#referrer-policy-origin
Header always set Referrer-Policy "origin"

I kept those restrictions in place because it keeps the spammers from even hitting the Mailman CGI, which is useful to preserve our server resources. But if "they" escalate with smarter crawlers, the block list will still be useful. You can use this query to extract the top 10 IP addresses used for subscription attempts:

awk '  print $NF  ' /var/log/mailman/subscribe   sort   uniq -c   sort -n   tail -10    awk '  print $2 " " $1  '

Note that this might include email-based registration, but in our logs those are extremely rare: only two in three weeks, out of over 73,000 requests. I also use this to keep an eye on the logs:

tail -f  /var/log/mailman/subscribe /var/log/apache2/lists.torproject.org-access.log   grep -v 'GET /pipermail/'

The server-side mitigations might also be useful if you happen to run an extremely old version of Mailman, that is pre-2.1.18, but it's now over 6 years old and part of every supported Debian release out there (all the way back to Debian 8 jessie).

Why does that attack work? Because Mailman 2 doesn't have CSRF tokens in its forms by default, anyone can send a POST request to /mailman/subscribe/LISTNAME to have Mailman send an email to the user. In the old "Internet is for nice people" universe, that wasn't a problem: all it does is ask the victim if they want to subscribe to LISTNAME. Innocuous, right? But in the brave, new, post-Eternal-September, "Internet is for stupid" universe, some assholes think it's a good idea to make a form that collects hundreds of mailing list URLs and spam them through an iframe. To see what that looks like, you can look at the rendered source code behind samedyfreeday.co.uk (not linking to avoid promoting it). That site does what is basically a distributed cross-site scripting attack against Mailman servers. Obviously, CSRF protection should be enabled by default in Mailman, but there you go. Hopefully this will help some folks... (The latest Mailman 3 release doesn't suffer from such idiotic defaults and ships with proper CSRF protection out of the box.)

    spdlog::stopwatch sw;                                   // instantiate a stop watch

    // some other code

    sp->info("Elapsed time:  ", sw);

R> RcppSpdlog::exampleRsink()
[13:03:23.845114] [fromR] [I] [thread 793264] Welcome to spdlog!
[...]
[13:03:23.845205] [fromR] [I] [thread 793264] Elapsed time: 0.000103611
[...]
[13:03:23.845281] [fromR] [I] [thread 793264] Elapsed time: 0.00018203
R> 

Changes in RcppSpdlog version 0.0.2 (2020-09-17)

• Upgraded to upstream release 1.8.0
• Switched Travis CI to using BSPM, also test on macOS
• Added 'stopwatch' use to main R sink example

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

- name: create a device in NetBox
  netbox_device:
    netbox_url: http://netbox.local
    netbox_token: s3cret
    data:
      name: to3-p14.sfo1.example.com
      device_type: QFX5110-48S
      device_role: Compute Switch
      site: SFO1

Code The module has the following signature and it syncs NetBox with the content of the provided YAML file:

netbox_sync:
  source: netbox.yaml
  api: https://netbox.example.com
  token: s3cret

The synchronized objects are:

• sites,
• manufacturers,
• device types,
• device roles,
• devices, and
• IP addresses.

In our environment, the YAML file is generated from our configuration management database and contains a set of devices and a list of IP addresses:

devices:
  ad2-p6.sfo1.example.com:
     datacenter: sfo1
     manufacturer: Cisco
     model: Catalyst 2960G-48TC-L
     role: net_tor_oob_switch
  to1-p6.sfo1.example.com:
     datacenter: sfo1
     manufacturer: Juniper
     model: QFX5110-48S
     role: net_tor_gpu_switch
# [ ]
ips:
  - device: ad2-p6.example.com
    ip: 172.31.115.18/21
    interface: oob
  - device: to1-p6.example.com
    ip: 172.31.115.33/21
    interface: oob
  - device: to1-p6.example.com
    ip: 172.31.254.33/32
    interface: lo0.0
# [ ]

The network team is not the sole tenant in NetBox. While adding new objects or modifying existing ones should be relatively safe, deleting unwanted objects can be risky. The module only deletes objects it did create or modify. To identify them, it marks them with a specific tag, cmdb. Most objects in NetBox accept tags.

Module definition Starting from the skeleton described in the previous article, we define the module:

module_args = dict(
    source=dict(type='path', required=True),
    api=dict(type='str', required=True),
    token=dict(type='str', required=True, no_log=True),
    max_workers=dict(type='int', required=False, default=10)
)
result = dict(
    changed=False
)
module = AnsibleModule(
    argument_spec=module_args,
    supports_check_mode=True
)

It contains an additional optional arguments defining the number of workers to talk to NetBox and query the existing objects in parallel to speedup the execution.

Abstracting synchronization We need to synchronize different object types, but once we have a list of objects we want in NetBox, the grunt work is always the same:

• check if the objects already exist,
• retrieve them and put them in a form suitable for comparison,
• retrieve the extra objects we don t want anymore,
• compare the two sets, and
• add missing objects, update existing ones, delete extra ones.

We code these behaviours into a Synchronizer abstract class. For each kind of object, a concrete class is built with the appropriate class attributes to tune its behaviour and a wanted() method to provide the objects we want. I am not explaining the abstract class code here. Have a look at the source if you want.

Synchronizing tags and tenants As a starter, here is how we define the class synchronizing the tags:

class SyncTags(Synchronizer):
    app = "extras"
    table = "tags"
    key = "name"
    def wanted(self):
        return  "cmdb": dict(
            slug="cmdb",
            color="8bc34a",
            description="synced by network CMDB") 

The app and table attributes defines the NetBox objects we want to manipulate. The key attribute is used to determine how to lookup for existing objects. In this example, we want to lookup tags using their names. The wanted() method is expected to return a dictionary mapping object keys to the list of wanted attributes. Here, the keys are tag names and we create only one tag, cmdb, with the provided slug, color and description. This is the tag we will use to mark the objects we create or modify. If the tag does not exist, it is created. If it exists, the provided attributes are updated. Other attributes are left untouched. We also want to create a specific tenant for objects accepting such an attribute (devices and IP addresses):

class SyncTenants(Synchronizer):
    app = "tenancy"
    table = "tenants"
    key = "name"
    def wanted(self):
        return  "Network": dict(slug="network",
                                description="Network team") 

Synchronizing sites We also need to synchronize the list of sites. This time, the wanted() method uses the information provided in the YAML file: it walks the devices and builds a set of datacenter names.

class SyncSites(Synchronizer):
    app = "dcim"
    table = "sites"
    key = "name"
    only_on_create = ("status", "slug")
    def wanted(self):
        result = set(details["datacenter"]
                     for details in self.source['devices'].values()
                     if "datacenter" in details)
        return  k: dict(slug=k,
                        status="planned")
                for k in result 

Thanks to the use of the only_on_create attribute, the specified attributes are not updated if they are different. The goal of this synchronizer is mostly to collect the references to the different sites for other objects.

>>> pprint(SyncSites(**sync_args).wanted())
 'sfo1':  'slug': 'sfo1', 'status': 'planned' ,
 'chi1':  'slug': 'chi1', 'status': 'planned' ,
 'nyc1':  'slug': 'nyc1', 'status': 'planned' 

Synchronizing manufacturers, device types and device roles The synchronization of manufacturers is pretty similar, except we do not use the only_on_create attribute:

class SyncManufacturers(Synchronizer):
    app = "dcim"
    table = "manufacturers"
    key = "name"
    def wanted(self):
        result = set(details["manufacturer"]
                     for details in self.source['devices'].values()
                     if "manufacturer" in details)
        return  k:  "slug": slugify(k) 
                for k in result 

Regarding the device types, we use the foreign attribute linking a NetBox attribute to the synchronizer handling it.

class SyncDeviceTypes(Synchronizer):
    app = "dcim"
    table = "device_types"
    key = "model"
    foreign =  "manufacturer": SyncManufacturers 
    def wanted(self):
        result = set((details["manufacturer"], details["model"])
                     for details in self.source['devices'].values()
                     if "model" in details)
        return  k[1]: dict(manufacturer=k[0],
                           slug=slugify(k[1]))
                for k in result 

The wanted() method refers to the manufacturer using its key attribute. In this case, this is the manufacturer name.

>>> pprint(SyncManufacturers(**sync_args).wanted())
 'Cisco':  'slug': 'cisco' ,
 'Dell':  'slug': 'dell' ,
 'Juniper':  'slug': 'juniper' 
>>> pprint(SyncDeviceTypes(**sync_args).wanted())
 'ASR 9001':  'manufacturer': 'Cisco', 'slug': 'asr-9001' ,
 'Catalyst 2960G-48TC-L':  'manufacturer': 'Cisco',
                           'slug': 'catalyst-2960g-48tc-l' ,
 'MX10003':  'manufacturer': 'Juniper', 'slug': 'mx10003' ,
 'QFX10002-36Q':  'manufacturer': 'Juniper', 'slug': 'qfx10002-36q' ,
 'QFX10002-72Q':  'manufacturer': 'Juniper', 'slug': 'qfx10002-72q' ,
 'QFX5110-32Q':  'manufacturer': 'Juniper', 'slug': 'qfx5110-32q' ,
 'QFX5110-48S':  'manufacturer': 'Juniper', 'slug': 'qfx5110-48s' ,
 'QFX5200-32C':  'manufacturer': 'Juniper', 'slug': 'qfx5200-32c' ,
 'S4048-ON':  'manufacturer': 'Dell', 'slug': 's4048-on' ,
 'S6010-ON':  'manufacturer': 'Dell', 'slug': 's6010-on' 

The device roles are defined like this:

class SyncDeviceRoles(Synchronizer):
    app = "dcim"
    table = "device_roles"
    key = "name"
    def wanted(self):
        result = set(details["role"]
                     for details in self.source['devices'].values()
                     if "role" in details)
        return  k: dict(slug=slugify(k),
                        color="8bc34a")
                for k in result 

Synchronizing devices A device is mostly a name with references to a role, a model, a datacenter and a tenant. These references are declared as foreign keys using the synchronizers defined previously.

class SyncDevices(Synchronizer):
    app = "dcim"
    table = "devices"
    key = "name"
    foreign =  "device_role": SyncDeviceRoles,
               "device_type": SyncDeviceTypes,
               "site": SyncSites,
               "tenant": SyncTenants 
    remove_unused = 10
    def wanted(self):
        return  name: dict(device_role=details["role"],
                           device_type=details["model"],
                           site=details["datacenter"],
                           tenant="Network")
                for name, details in self.source['devices'].items()
                if  "datacenter", "model", "role"  <= set(details.keys()) 

The remove_unused attribute is a safety implemented to fail if we have to delete more than 10 devices: this may be the indication there is a bug somewhere, unless one of your datacenter suddenly caught fire.

>>> pprint(SyncDevices(**sync_args).wanted())
 'ad2-p6.sfo1.example.com':  'device_role': 'net_tor_oob_switch',
                             'device_type': 'Catalyst 2960G-48TC-L',
                             'site': 'sfo1',
                             'tenant': 'Network' ,
 'to1-p6.sfo1.example.com':  'device_role': 'net_tor_gpu_switch',
                             'device_type': 'QFX5110-48S',
                             'site': 'sfo1',
                             'tenant': 'Network' ,
[ ]

Synchronizing IP addresses The last step is to synchronize IP addresses. We do not attach them to a device.² Instead, we specify the device names in the description of the IP address:

class SyncIPs(Synchronizer):
    app = "ipam"
    table = "ip-addresses"
    key = "address"
    foreign =  "tenant": SyncTenants 
    remove_unused = 1000
    def wanted(self):
        wanted =  
        for details in self.source['ips']:
            if details['ip'] in wanted:
                wanted[details['ip']]['description'] = \
                    f" details['device']  (and others)"
            else:
                wanted[details['ip']] = dict(
                    tenant="Network",
                    status="active",
                    dns_name="",        # information is present in DNS
                    description=f" details['device'] :  details['interface'] ",
                    role=None,
                    vrf=None)
        return wanted

There is a slight difficulty: NetBox allows duplicate IP addresses, so a simple lookup is not enough. In case of multiple matches, we choose the best by preferring those tagged with cmdb, then those already attached to an interface:

def get(self, key):
    """Grab IP address from NetBox."""
    # There may be duplicate. We need to grab the "best".
    results = super(Synchronizer, self).get(key)
    if len(results) == 0:
        return None
    if len(results) == 1:
        return results[0]
    scores = [0]*len(results)
    for idx, result in enumerate(results):
        if "cmdb" in result.tags:
            scores[idx] += 10
        if result.interface is not None:
            scores[idx] += 5
    return sorted(zip(scores, results),
                  reverse=True, key=lambda k: k[0])[0][1]

Getting the current and wanted states Each synchronizer is initialized with a reference to the Ansible module, a reference to a pynetbox s API object, the data contained in the provided YAML file and two empty dictionaries for the current and expected states:

source = yaml.safe_load(open(module.params['source']))
netbox = pynetbox.api(module.params['api'],
                      token=module.params['token'])
sync_args = dict(
    module=module,
    netbox=netbox,
    source=source,
    before= ,
    after= 
)
synchronizers = [synchronizer(**sync_args) for synchronizer in [
    SyncTags,
    SyncTenants,
    SyncSites,
    SyncManufacturers,
    SyncDeviceTypes,
    SyncDeviceRoles,
    SyncDevices,
    SyncIPs
]]

Each synchronizer has a prepare() method whose goal is to compute the current and wanted states. It returns True in case of a difference:

# Check what needs to be synchronized
try:
    for synchronizer in synchronizers:
        result['changed']  = synchronizer.prepare()
except AnsibleError as e:
    result['msg'] = e.message
    module.fail_json(**result)

Applying changes Back to the skeleton described in the previous article, the last step is to apply the changes if there is a difference between these states. Each synchronizer registers the current and wanted states in sync_args["before"][table] and sync_args["after"][table] where table is the name of the table for a given NetBox object type. The diff object is a bit elaborate as it is built table by table. This enables Ansible to display the name of each table before the diff representation:

# Compute the diff
if module._diff and result['changed']:
    result['diff'] = [
        dict(
            before_header=table,
            after_header=table,
            before=yaml.safe_dump(sync_args["before"][table]),
            after=yaml.safe_dump(sync_args["after"][table]))
        for table in sync_args["after"]
        if sync_args["before"][table] != sync_args["after"][table]
    ]
# Stop here if check mode is enabled or if no change
if module.check_mode or not result['changed']:
    module.exit_json(**result)

Each synchronizer also exposes a synchronize() method to apply changes and a cleanup() method to delete unwanted objects. Order is important due to the relation between the objects.

# Synchronize
for synchronizer in synchronizers:
    synchronizer.synchronize()
for synchronizer in synchronizers[::-1]:
    synchronizer.cleanup()
module.exit_json(**result)

---

The complete code is available on GitHub. Compared to using netbox.netbox collection, the logic is written in Python instead of trying to glue Ansible tasks together. I believe this is both more flexible and easier to read, notably when trying to delete outdated objects. While I did not test it, it should also be faster. An alternative would have been to reuse code from the netbox.netbox collection, as it contains similar primitives. Unfortunately, I didn t think of it until now.

---

1. In my opinion, a good option for a source of truth is to use YAML files in a Git repository. You get versioning for free and people can get started with a text editor.
2. This limitation is mostly due to laziness: we do not really care about this information. Our main motivation for putting IP addresses in NetBox is to keep track of the used IP addresses. However, if an IP address is already attached to an interface, we leave this association untouched.

! Google - remove cookie-consent-popup and restore scroll functionality
google.*##.wwYr3.aID8W.bErdLd
google.*##.aID8W.m114nf.t7xA6
google.*##div[jsname][jsaction^="dg_close"]
google.*##html:style(overflow: visible !important;)
google.*##.widget-consent-fullscreen.widget-consent

// this portmanteau include also defines the r_sink we use below, and which
// diverts all logging to R via the Rcpp::Rcout replacement for std::cout
#include <RcppSpdlog>

// [[Rcpp::export]]
void exampleRsink()  

    std::string logname = "fromR";                          // fix a name for this logger
    auto sp = spdlog::get(logname);                         // retrieve existing one
    if (sp == nullptr) sp = spdlog::r_sink_mt(logname);     // or create new one if needed

    // change log pattern (changed from [%H:%M:%S %z] [%n] [%^---%L---%$] )
    sp->set_pattern("[%H:%M:%S.%f] [%n] [%^%L%$] [thread %t] %v");

    sp->info("Welcome to spdlog!");
    sp->error("Some error message with arg:  ", 1);

    sp->warn("Easy padding in numbers like  :08d ", 12);
    sp->critical("Support for int:  0:d ;  hex:  0:x ;  oct:  0:o ; bin:  0:b ", 42);
    sp->info("Support for floats  :03.2f ", 1.23456);
    sp->info("Positional args are  1   0 ..", "too", "supported");
    sp->info(" :<30 ", "left aligned");

 

Changes in RcppSpdlog version 0.0.1 (2020-09-08)

• Initial release with added R/Rcpp logging sink example

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

• Abhijith PA did 10.0h (out of 10h assigned).
• Adrian Bunk did 31h (out of 21.75h assigned and 9.25h from July).
• Ben Hutchings did 6.25h (out of 16h assigned), thus carrying over 9.75h to September.
• Brian May did 10h (out of 10h assigned).
• Chris Lamb did 18h (out of 18h assigned).
• Emilio Pozuelo Monfort did 21.75h (out of 21.75h assigned).
• Holger Levsen did 7h coordinating/managing the LTS team.
• Markus Koschany did 20h (out of 21.75h assigned and 10.25h from July), thus carrying over 12h to September.
• Mike Gabriel did 16.0h (out of 8h assigned and 8h from July).
• Ola Lundqvist did 6.9h (out of 8h assigned and 16h from July), and gave back 5.1h, thus carrying over 12h to September.
• Roberto C. S nchez did 21.75h (out of 21.75h assigned).
• Sylvain Beucler did 21.75h (out of 21.75h assigned).
• Thorsten Alteholz did 21.75h (out of 21.75h assigned).
• Utkarsh Gupta did 32.75h (out of 21.75h assigned and anticipating 11h from September).

• Platinum sponsors:
• TOSHIBA (for 60 months)
• GitHub (for 50 months)
• Civil Infrastructure Platform (CIP) (for 27 months)
• Gold sponsors:
• Blablacar (for 75 months)
• Roche Diagnostics International AG (for 70 months)
• Linode (for 65 months)
• Babiel GmbH (for 54 months)
• Plat Home (for 53 months)
• University of Oxford (for 9 months)
• Silver sponsors:
• The Positive Internet Company (for 76 months)
• Domeneshop AS (for 75 months)
• Nantes M tropole (for 69 months)
• Univention GmbH (for 61 months)
• Universit Jean Monnet de St Etienne (for 61 months)
• Ribbon Communications, Inc. (for 54 months)
• Exonet B.V. (for 44 months)
• Leibniz Rechenzentrum (for 38 months)
• CINECA (for 28 months)
• Minist re de l Europe et des Affaires trang res (for 22 months)
• Cloudways Ltd (for 11 months)
• Dinahosting SL (for 9 months)
• Bauer Xcel Media Deutschland KG (for 3 months)
• Platform.sh (for 3 months)
• Bronze sponsors:
• Seznam.cz, a.s. (for 76 months)
• Evolix (for 75 months)
• Linuxhotel GmbH (for 73 months)
• Intevation GmbH (for 72 months)
• Daevel SARL (for 71 months)
• Bitfolk LTD (for 70 months)
• Megaspace Internet Services GmbH (for 70 months)
• Greenbone Networks GmbH (for 69 months)
• NUMLOG (for 69 months)
• WinGo AG (for 68 months)
• Ecole Centrale de Nantes LHEEA (for 64 months)
• Entr ouvert (for 59 months)
• Adfinis AG (for 57 months)
• GNI MEDIA (for 51 months)
• Laboratoire LEGI UMR 5519 / CNRS (for 51 months)
• Tesorion (for 51 months)
• Bearstech (for 43 months)
• LiHAS (for 43 months)
• People Doc (for 39 months)
• Catalyst IT Ltd (for 37 months)
• Supagro (for 33 months)
• Demarcq SAS (for 31 months)
• Universit Grenoble Alpes (for 17 months)
• TouchWeb SAS (for 9 months)
• SPiN AG (for 6 months)
• CoreFiling

No comment Liked this article? Click here. My blog is Flattr-enabled.

• [1] https://etbe.coker.com.au/2020/09/13/dell-t710-idrac6/

• I packaged a new upstream release of teeworlds, the well-known 2D multiplayer shooter with cute characters called tees to resolve a Python 2 bug (although teeworlds is actually a C++ game). The update also fixed a severe remote denial-of-service security vulnerability, CVE-2020-12066. I prepared a patch for Buster and will send it to the security team later today.
• I sponsored updates of mgba, a Game Boy Advance emulator, for Ryan Tandy, and osmose-emulator for Carlos Donizete Froes.
• I worked around a RC GCC 10 bug in megaglest by compiling with -fcommon.
• Thanks to Gerardo Ballabio who packaged a new upstream version of galois which I uploaded for him.
• Also thanks to Reiner Herrmann and Judit Foglszinger who fixed a regression (crash) in monsterz due to the earlier port to Python 3. Reiner also made fans of supertuxkart happy by packaging the latest upstream release version 1.2.

• I packaged new upstream versions of jboss-modules, libpdfbox2-java and intellij-annotations.

• I was contacted by the upstream maintainer of privacybadger, a privacy addon for Firefox and Chromium, who dislikes the idea of having a stable and unchanging version in Debian stable releases. Obviously I can t really do much about it although I believe the release team would be open-minded for regular point updates of browser addons though. However I don t intend to do regular updates for all of my packages in stable unless there is a really good reason to do so. At the moment I m willing to make an exception for ublock-origin and https-everywhere because I feel these addons should be core browser functionality anyway. I talked about this on our Debian Mozilla Extension Maintainers mailinglist and it seems someone is interested to take over privacybadger and prepare regular stable point updates. Let s see how it turns out.
• Finally this month saw the release of ublock-origin 1.29.0 and the creation of two different browser-specific binary packages for Firefox and Chromium. I have talked about it before and I believe two separate packages for ublock-origin are more aligned to upstream development and make the whole addon easier to maintain which benefits users, upstream and maintainers.
• imlib2, an image library, and binaryen also got updated this month.

• DLA-2303-1. Issued a security update for libssh fixing 1 CVE.
• DLA-2327-1. Issued a security update for lucene-solr fixing 1 CVE.
• DLA-2369-1. Issued a security update for libxml2 fixing 8 CVE.
• Triaged CVE-2020-14340, jboss-xnio as not-affected for Stretch.
• Triaged CVE-2020-13941, lucene-solr as no-dsa because the security impact was minor.
• Triaged CVE-2019-17638, jetty9 as not-affected for Stretch and Buster.
• squid3: I backported the patches for CVE-2020-15049, CVE-2020-15810, CVE-2020-15811 and CVE-2020-24606 from squid 4 to squid 3.

• ELA-271-1. Issued a security update for squid3 fixing 19 CVE. Most of the work was already done before ELTS started, only the patch for CVE-2019-12529 had to be adjusted for the nettle version in Jessie.
• ELA-273-1. Issued a security update for nss fixing 1 CVE.
• ELA-276-1. Issued a security update for libjpeg-turbo fixing 2 CVE.
• ELA-277-1. Issued a security update for graphicsmagick fixing 1 CVE.
• ELA-279-1. Issued a security update for imagemagick fixing 3 CVE.
• ELA-280-1. Issued a security update for libxml2 fixing 4 CVE.

One hiccup we ve encountered in SecureDrop development is that not all Python wheels can be built reproducibly. We ship multiple (Python) projects in Debian packages, with Python dependencies included in those packages as wheels. In order for our Debian packages to be reproducible, we need that wheel build process to also be reproducible

Reproducible builds at DebConf20 There were a number of talks at the recent online-only DebConf20 conference on the topic of reproducible builds. Holger gave a talk titled Reproducing Bullseye in practice , focusing on independently verifying that the binaries distributed from ftp.debian.org are made from their claimed sources. It also served as a general update on the status of reproducible builds within Debian. The video (145 MB) and slides are available. There were also a number of other talks that involved Reproducible Builds too. For example, the Malayalam language mini-conference had a talk titled , ? ( I want to join Debian, what should I do? ) presented by Praveen Arimbrathodiyil, the Clojure Packaging Team BoF session led by Elana Hashman, as well as Where is Salsa CI right now? that was on the topic of Salsa, the collaborative development server that Debian uses to provide the necessary tools for package maintainers, packaging teams and so on. Jonathan Bustillos (Jathan) also gave a talk in Spanish titled Un camino verificable desde el origen hasta el binario ( A verifiable path from source to binary ). (Video, 88MB)

Development work After many years of development work, the compiler for the Rust programming language now generates reproducible binary code. This generated some general discussion on Reddit on the topic of reproducibility in general. Paul Spooren posted a request for comments to OpenWrt s openwrt-devel mailing list asking for clarification on when to raise the PKG_RELEASE identifier of a package. This is needed in order to successfully perform rebuilds in a reproducible builds context. In openSUSE, Bernhard M. Wiedemann published his monthly Reproducible Builds status update. Chris Lamb provided some comments and pointers on an upstream issue regarding the reproducibility of a Snap / SquashFS archive file. [ ]

Debian Holger Levsen identified that a large number of Debian .buildinfo build certificates have been tainted on the official Debian build servers, as these environments have files underneath the /usr/local/sbin directory [ ]. He also filed against bug for debrebuild after spotting that it can fail to download packages from snapshot.debian.org [ ]. This month, several issues were uncovered (or assisted) due to the efforts of reproducible builds. For instance, Debian bug #968710 was filed by Simon McVittie, which describes a problem with detached debug symbol files (required to generate a traceback) that is unlikely to have been discovered without reproducible builds. In addition, Jelmer Vernooij called attention that the new Debian Janitor tool is using the property of reproducibility (as well as diffoscope when applying archive-wide changes to Debian:

New merge proposals also include a link to the diffoscope diff between a vanilla build and the build with changes. Unfortunately these can be a bit noisy for packages that are not reproducible yet, due to the difference in build environment between the two builds. [ ]

56 reviews of Debian packages were added, 38 were updated and 24 were removed this month adding to our knowledge about identified issues. Specifically, Chris Lamb added and categorised the nondeterministic_version_generated_by_python_param and the lessc_nondeterministic_keys toolchain issues. [ ][ ] Holger Levsen sponsored Lukas Puehringer s upload of the python-securesystemslib pacage, which is a dependency of in-toto, a framework to secure the integrity of software supply chains. [ ] Lastly, Chris Lamb further refined his merge request against the debian-installer component to allow all arguments from sources.list files (such as [check-valid-until=no]) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure and sent a ping to the team that maintains that code.

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of these patches, including:

• Bernhard M. Wiedemann:
  • asymptote (shell/Perl date)
  • getfem (embeds datetime and user, submitted via email)
  • getdp (hostname and user)
  • getdp (user)
  • guix (disable parallelism)
  • httpcomponents-client (Java documentation generator readdir order)
  • kuberlr (date)
  • lal (date and time issue, submitted via email)
  • libmesh (host)
  • OBS (discuss how to track old build prjconf metadata in buildinfo)
  • openblas (disable CPU detection)
  • openfoam-selector (date)
  • perl (toolchain, date)
  • python-blosc (CPU detection)
  • python-eventlet (fails to build far in the future)
  • rna-star (date and hostname)
  • trilinos (date)
  • xz/b4 (workaround CPU count influencing output, reported upstream)
• Benjamin Hof:
  • flit
• Chris Lamb:
  • #966657 filed against json-c (forwarded upstream).
  • #967238 filed against nmh.
  • #968045 filed against golang-gonum-v1-plot.
  • #968183 filed against chirp.
  • #968185 filed against pixelmed-codec.
  • #968187 filed against debhelper.
  • #968189 filed against muroar.
  • #968278 filed against serd.
  • #968344 filed against pencil2d.
  • #968557 filed against tpot.
  • #968700 filed against evolution.
  • #969320 filed against aflplusplus.
• Herv Boutemy:
  • plexus-archiver (timezone/DST issue)
• Vagrant Cascadian:
  • #968627 filed against libjpeg-turbo.
  • #968641 filed against jack-audio-connection-kit.
  • #968652 filed against glusterfs.

diffoscope diffoscope is our in-depth and content-aware diff utility that can not only locate and diagnose reproducibility issues, it provides human-readable diffs of all kinds. In August, Chris Lamb made the following changes to diffoscope, including preparing and uploading versions 155, 156, 157 and 158 to Debian:

• New features:
  • Support extracting data of PGP signed data. (#214)
  • Try files named .pgp against pgpdump(1) to determine whether they are Pretty Good Privacy (PGP) files. (#211)
  • Support multiple options for all file extension matching. [ ]
• Bug fixes:
  • Don t raise an exception when we encounter XML files with <!ENTITY> declarations inside the Document Type Definition (DTD), or when a DTD or entity references an external resource. (#212)
  • pgpdump(1) can successfully parse some binary files, so check that the parsed output contains something sensible before accepting it. [ ]
  • Temporarily drop gnumeric from the Debian build-dependencies as it has been removed from the testing distribution. (#968742)
  • Correctly use fallback_recognises to prevent matching .xsb binary XML files.
  • Correct identify signed PGP files as file(1) returns data . (#211)
• Logging improvements:
  • Emit a message when ppudump version does not match our file header. [ ]
  • Don t use Python s repr(object) output in Calling external command messages. [ ]
  • Include the filename in the not identified by any comparator message. [ ]
• Codebase improvements:
  • Bump Python requirement from 3.6 to 3.7. Most distributions are either shipping with Python 3.5 or 3.7, so supporting 3.6 is not only somewhat unnecessary but also cumbersome to test locally. [ ]
  • Drop some unused imports [ ], drop an unnecessary dictionary comprehensions [ ] and some unnecessary control flow [ ].
  • Correct typo of output in a comment. [ ]
• Release process:
  • Move generation of debian/tests/control to an external script. [ ]
  • Add some URLs for the site that will appear on PyPI.org. [ ]
  • Update author and author email in setup.py for PyPI.org and similar. [ ]
• Testsuite improvements:
  • Update PPU tests for compatibility with Free Pascal versions 3.2.0 or greater. (#968124)
  • Mark that our identification test for .ppu files requires ppudump version 3.2.0 or higher. [ ]
  • Add an assert_diff helper that loads and compares a fixture output. [ ][ ][ ][ ]
• Misc:
  • Duplicate docker instructions in the Get diffoscope section of the diffoscope website. [ ]

In addition, Mattia Rizzolo documented in setup.py that diffoscope works with Python version 3.8 [ ] and Frazer Clews applied some Pylint suggestions [ ] and removed some deprecated methods [ ].

Website This month, Chris Lamb updated the main Reproducible Builds website and documentation to:

• Clarify & fix a few entries on the who page [ ][ ] and ensure that images do not get to large on some viewports [ ].
• Clarify use of a pronoun re. Conservancy. [ ]
• Use View all our monthly reports over View all monthly reports . [ ]
• Move a is a suffix out of the link target on the SOURCE_DATE_EPOCH age. [ ]

In addition, Javier Jard n added the freedesktop-sdk project [ ] and Kushal Das added SecureDrop project [ ] to our projects page. Lastly, Michael P hn added internationalisation and translation support with help from Hans-Christoph Steiner [ ].

Testing framework The Reproducible Builds project operate a Jenkins-based testing framework to power tests.reproducible-builds.org. This month, Holger Levsen made the following changes:

• System health checks:
  • Improve explanation how the status and scores are calculated. [ ][ ]
  • Update and condense view of detected issues. [ ][ ]
  • Query the canonical configuration file to determine whether a job is disabled instead of duplicating/hardcoding this. [ ]
  • Detect several problems when updating the status of reporting-oriented metapackage sets. [ ]
  • Detect when diffoscope is not installable [ ] and failures in DNS resolution [ ].
• Debian:
  • Update the URL to the Debian security team bug tracker s Git repository. [ ]
  • Reschedule the unstable and bullseye distributions often for the arm64 architecture. [ ]
  • Schedule buster less often for armhf. [ ][ ][ ]
  • Force the build of certain packages in the work-in-progress package rebuilder. [ ][ ]
  • Only update the stretch and buster base build images when necessary. [ ]
• Other distributions:
  • For F-Droid, trigger jobs by commits, not by a timer. [ ]
  • Disable the Archlinux HTML page generation job as it has never worked. [ ]
  • Disable the alternative OpenWrt rebuilder jobs. [ ]
• Misc;
  • Improve monitoring, such as number of mounts, disk, memory, etc.. [ ][ ][ ][ ]
  • Install the ruby-jekyll-polyglot package to needed for the recently-added internationalisation and translation support on the Reproducible Builds website. [ ]
  • Update link to report potential issues. [ ][ ]

Many other changes were made too, including:

• Chris Lamb:
  • Use <pre> HTML tags when dumping fixed-width debugging data in the self-serve package scheduler. [ ]
• Mattia Rizzolo:
  • For Alpine and ArchLinux, make the cleanup routines in the event of an error more robust. [ ]
  • Update the sudo configuration to permit Jenkins itself to unmount more directories. [ ]
  • Setup automatic renewal of our Let s Encrypt certificates for all domains served by us (including jenkins.debian.net, www.reproducible-builds.org, diffoscope.org, buildinfos.debian.net, etc.). [ ][ ][ ][ ][ ]
• Vagrant Cascadian:
  • Mark that the u-boot Universal Boot Loader should not build architecture independent packages on the arm64 architecture anymore. [ ]

Finally, build node maintenance was performed by Holger Levsen [ ], Mattia Rizzolo [ ][ ] and Vagrant Cascadian [ ][ ][ ][ ]

---

Mailing list On our mailing list this month, Leo Wandersleb sent a message to the list after he was wondering how to expand his WalletScrutiny.com project (which aims to improve the security of Bitcoin wallets) from Android wallets to also monitor Linux wallets as well:

If you think you know how to spread the word about reproducibility in the context of Bitcoin wallets through WalletScrutiny, your contributions are highly welcome on this PR [ ]

Julien Lepiller posted to the list linking to a blog post by Tavis Ormandy titled You don t need reproducible builds. Morten Linderud (foxboron) responded with a clear rebuttal that Tavis was only considering the narrow use-case of proprietary vendors and closed-source software. He additionally noted that the criticism that reproducible builds cannot prevent against backdoors being deliberately introduced into the upstream source ( bugdoors ) are decidedly (and deliberately) outside the scope of reproducible builds to begin with. Chris Lamb included the Reproducible Builds mailing list in a wider discussion regarding a tentative proposal to include .buildinfo files in .deb packages, adding his remarks regarding requiring a custom tool in order to determine whether generated build artifacts are identical in a reproducible context. [ ] Jonathan Bustillos (Jathan) posted a quick email to the list requesting whether there was a list of To do tasks in Reproducible Builds. Lastly, Chris Lamb responded at length to a query regarding the status of reproducible builds for Debian ISO or installation images. He noted that most of the technical work has been performed but there are at least four issues until they can be generally advertised as such . He pointed that the privacy-oriented Tails operation system, which is based directly on Debian, has had reproducible builds for a number of years now. [ ]

---

If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Twitter: @ReproBuilds
• Mastodon: @reproducible_builds@fosstodon.org
• Mailing list: rb-general@lists.reproducible-builds.org

Changes in inline version 0.3.16 (2020-09-06)

• Maintenance updates to README.md standardizing badges (Dirk).
• Maintenance update to Travis CI setup (Dirk).
• Switch to using system2() for better error diagnostics (Ben Goodrich in #12).

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

• not be under the control of a single, central commercial entity running proprietary code, so their privacy is safeguarded, and they are protected from disaster, should it become unattractive to remain on the platform;
• have a vibrant and supportive open source third party extension development community;
• support our image-rich content and effortless sharing of URLs with previews automatically provided from the page s content (e.g. via OpenGraph tags);
• be effortless to join regardless of what platform/device each user uses;
• keep a history of messages so that future members joining the community can benefit from past conversations, and existing members can catch up on conversations they missed;
• but above all else: be acceptable and compelling to the existing community to move over onto it.