Debian I ve prepared and tested backports for 3 packages in the tasktools packaging team: tasksh, bugwarrior and powerline-taskwarrior. Unfortunately I am not currently in the backports ACLs and so I can t upload these but I m hoping this to be resolved soon. Once these are uploaded, the latest upstream release for all packages in the tasktools team will be available either in the stable suite or in the stable backports suite. In preparation for the shutdown of Alioth mailing lists, I ve set up a new mailing list for the tasktools team and have already updated the maintainer fields for all the team s packages in git. I ve subscribed the old mailing list s user to the new mailing list in DDPO so there will still be a comprehensive view there during the migration. I am currently in the process of reaching out to the admins of git.tasktools.org with a view to moving our git repositories there. I ve also continued to review the scapy package and have closed a couple more bugs that were already fixed in the latest upstream release but had been missed in the changelog. Bugs closed (fixed/wontfix): #774962, #850570

Tor Project I ve deployed a small fix to an update from last week where the platform field on Atlas had been pulled across to the left column. It has now been returned to the right hand column and is not pushed down the page by long family lists. I ve been thinking about the merge of Compass functionality into a future Atlas and this is being tracked in #23517. Tor Project has approved expenses (flights and hotel) for me to attend an in-person meeting of the Metrics Team. This meeting will occur in Berlin on the 28th September and I will write up a report detailing outcomes relevant to my work after the meeting. I have spent some time this week preparing for this meeting. Bugs closed (fixed/wontfix): #22146, #22297, #23511

Sustainability I believe it is important to be clear not only about the work I have already completed but also about the sustainability of this work into the future. I plan to include a short report on the current sustainability of my work in each weekly report. The loss of my primary development machine was a setback, however, I have been donated a new workstation which should hopefully arrive soon. The hard drives in my NAS can now also be replaced as I have budget available for this now. I do not see any hardware failures being imminent at this time, however should they occur I would not have budget to replace hardware, I only have funds to replace the hardware that has already failed.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

sudo apt install npm
sudo npm install -g netlify-cli

mkdir repository
cp /path/to/*.deb repository/
cd repository
apt-ftparchive packages . > Packages
apt-ftparchive release . > Release
gpg --clearsign -o InRelease Release
netlify deploy

gpg --export -a YOURKEYID   sudo apt-key add -
echo "deb https://SUBDOMAIN.netlify.com/ /"   sudo tee -a /etc/apt/sources.list
sudo apt update

• Holger Levsen wrote and published details about our upcoming Berlin summit. Expect a more detailed announced soon and consider planning your travel!

• Chris Lamb:
  • #875700 filed against gtk+3.0.
  • #875704 filed against gdk-pixbuf.
  • #875792 filed against doit.
• Vagrant Cascadian:
  • #875711 filed against qemu.

• Holger Levsen:
  • fonts-dustin/20030517-11 uploaded, fixing #815723 with patch by Scarlett Clark.

• Bernhard M. Wiedemann:
  • cadabra2 build timestamp
  • jimtcl build timestamp, fixed another way
  • itpp build timestamp, merged
  • dunst file list ordering, merged
  • HSAIL-Tools hash table ordering, merged
  • kubernetes hash set ordering, no patch

• Add build_path_captured_in_assembly_objects

• Juliana Oliveira Rodrigues:
  • Fix comparisons between different container types not comparing inside files. It was caused by falling back to binary comparison for different file types even for unextracted containers.
  • Add many tests for the fixed behaviour.
  • Other code quality improvements.
• Chris Lamb:
  • Various code quality and style improvements, some of it using Flake8.
• Mattia Rizzolo:
  • Add a check to prevent installation with python < 3.4

• Ximin Luo:
  • Split up the very large __init__.py and remove obsolete earlier code.
  • Extend the syntax for the --variations flag to support parameters to certain variations like user_group, and document examples in README.
  • Add a --vary flag for the new syntax and deprecate --dont-vary.
  • Heavily refactor internals to support > 2 builds.
  • Support >2 builds using a new --extra-build flag.
  • Properly sanitize artifact_pattern to avoid arbitrary shell execution.

• Chris Lamb:
  • Packaging maintenance updates.
  • Developer documentation updates.

• Holger Levsen:
  • Add a page for the Reproducible Builds World Summit 3 in Berlin 2017.
• Chris Lamb:
  • Moved isdebianreproducibleyet.com to HTTPS.
  • Updated the SSL certificate for buildinfo.debian.net.

• Vagrant Cascadian and Holger Levsen:
  • Added two armhf boards to the build farm. #874682
• Holger also:
  • use timeout to limit the diffing of the two build logs to 30min, which greatly reduced jenkins load again.

Many voices arise now and then against risks linked to the Github use by Free Software projects. Yet the infatuation for the collaborative forge of the Octocat Californian start-ups doesn t seem to fade away.

These recent years, Github and its services take an important role in software engineering as they are seen as easy to use, efficient for a daily workload with interesting functions in enterprise collaborative workflow or amid a Free Software project. What are the arguments against using its services and are they valid? We will list them first, then we ll examine their validity.

1. Critical points

1.1 Centralization

The Github application belongs to a single entity, Github Inc, a US company which manage it alone. So, a unique company under US legislation manages the access to most of Free Software application code sources, which may be a problem with groups using it when a code source is no longer available, for political or technical reason.

The Octocat, the Github mascot

This centralization leads to another trouble: as it obtained critical mass, it becomes more and more difficult not having a Github account. People who don t use Github, by choice or not, are becoming a silent minority. It is now fashionable to use Github, and not doing so is seen as out of date . The same phenomenon is a classic, and even the norm, for proprietary social networks (Facebook, Twitter, Instagram).

1.2 A Proprietary Software

When you interact with Github, you are using a proprietary software, with no access to its source code and which may not work the way you think it is. It is a problem at different levels. First, ideologically, but foremost in practice. In the Github case, we send them code we can control outside of their interface. We also send them personal information (profile, Github interactions). And mostly, Github forces any project which goes through the US platform to use a crucial proprietary tools: its bug tracking system.

Windows, the epitome of proprietary software, even if others took the same path

1.3 The Uniformization

Working with Github interface seems easy and intuitive to most. Lots of companies now use it as a source repository, and many developers leaving a company find the same Github working environment in the next one. This pervasive presence of Github in free software development environment is a part of the uniformization of said developers working space.

Uniforms always bring Army in my mind, here the Clone army

2 Critical points cross-examination

2.1 Regarding the centralization

2.1.1 Service availability rate

As said above, nowadays, Github is the main repository of Free Software source code. As such it is a favorite target for cyberattacks. DDOS hit it in March and August 2015. On December 15, 2015, an outage led to the inaccessibility of 5% of the repositories. The same occurred on November 15. And these are only the incident reported by Github itself. One can imagine that the mean outage rate of the platform is underestimated.

2.1.2 Chain reaction could block Free Software development

Today many dependency maintenance tools, as npm for javascript, Bundler for Ruby or even pip for Python can access an application source code directly from Github. Free Software projects getting more and more linked and codependents, if one component is down, all the developing process stop.

One of the best examples is the npmgate. Any company could legally demand that Github take down some source code from its repository, which could create a chain reaction and blocking the development of many Free Software projects, as suffered the Node.js community from the decisions of Npm, Inc, the company managing npm.

2.2 A historical precedent: SourceForge

Github didn t appear out of the blue. In his time, its predecessor, SourceForge, was also extremely popular.

Heavily centralized, based on strong interaction with the community, SourceForge is now seen as an aging SAAS (Software As A Service) and sees most of its customers fleeing to Github. Which creates lots of hurdles for those who stayed. The Gimp project suffered from spams and terrible advertising, which led to the departure of the VLC project, then from installers corrupted with adwares instead of the official Gimp installer for Windows. And finally, the Project Gimp s SourceForge account was hacked by SourceForge team itself!

These are very recent examples of what can do a commercial entity when it is under its stakeholders pressure. It is vital to really understand what it means to trust them with data and exchange centralization, where it could have tremendous repercussion on the day-to-day life and the habits of the Free Software and open source community.

2.3. Regarding proprietary software

2.3.1 One community, several opinions on proprietary software

Mostly based on ideology, this point deals with the definition every member of the community gives to Free Software and open source. Mostly about one thing: is it viral or not? Or GPL vs MIT/BSD.

Those on the side of the viral Free Software will have trouble to use a proprietary software as this last one shouldn t even exist. It must be assimilated, to quote Star Trek, as it is a connected black box, endangering privacy, corrupting for profit our uses and restrain our freedom to use as we re pleased what we own, etc.

Those on the side of complete freedom have no qualms using proprietary software as their very existence is a consequence of freedom without restriction. They even agree that code they developed may be a part of proprietary software, which is quite a common occurrence. This part of the Free Software community has no qualm using Github, which is well within their ideology parameters. Just take a look at the Janson amphitheater during Fosdem and check how many Apple laptops running on macOS are around.

FreeBSD, the main BSD project under the BSD license

2.3.2 Data loss and data restrictions linked to proprietary software use

Even without ideological consideration, and just focusing on Github infrastructure, the bug tracking system is a major problem by itself.

Bug report builds the memory of Free Software projects. It is the entrance point for new contributors, the place to find bug reporting, requests for new functions, etc. The project history can t be limited only to the code. It s very common to find bug reports when you copy and paste an error message in a search engine. Not their historical importance is precious for the project itself, but also for its present and future users.

Github gives the ability to extract bug reports through its API. What would happen if Github is down or if the platform doesn t support this feature anymore? In my opinion, not that many projects ever thought of this outcome. How could they move all the data generated by Github into a new bug tracking system? One old example now is Astrid, a TODO list bought by Yahoo a few years ago. Very popular, it grew fast until it was closed overnight, with only a few weeks for its users to extract their data. It was only a to-do list. The same situation with Github would be tremendously difficult to manage for several projects if they even have the ability to deal with it. Code would still be available and could still live somewhere else, but the project memory would be lost. A project like Debian has today more than 800,000 bug reports, which are a data treasure trove about problems solved, function requests and where the development stand on each. The developers of the Cpython project have anticipated the problem and decided not to use Github bug tracking systems.

Issues, the Github proprietary bug tracking system

Another thing we could lose if Github suddenly disappear: all the work currently done regarding the push requests (aka PRs). This Github function gives the ability to clone one project s Github repository, to modify it to fit your needs, then to offer your own modification to the original repository. The original repository s owner will then review said modification, and if he or she agrees with them will fuse them into the original repository. As such, it s one of the main advantages of Github, since it can be done easily through its graphic interface.

However reviewing all the PRs may be quite long, and most of the successful projects have several ongoing PRs. And this PRs and/or the proprietary bug tracking system are commonly used as a platform for comment and discussion between developers.

Code itself is not lost if Github is down (except one specific situation as seen below), but the peer review works materialized in the PRs and the bug tracking system is lost. Let s remember than the PR mechanism let you clone and modify projects and then generate PRs directly from its proprietary web interface without downloading a single code line on your computer. In this particular case, if Github is down, all the code and the work in progress is lost. Some also use Github as a bookmark place. They follow their favorite projects activity through the Watch function. This technological watch style of data collection would also be lost if Github is down.

Debian, one of the main Free Software projects with at least a thousand official contributors

2.4 Uniformization

The Free Software community is walking a thigh rope between normalization needed for an easier interoperability between its products and an attraction for novelty led by a strong need for differentiation from what is already there.

Github popularized the use of Git, a great tool now used through various sectors far away from its original programming field. Step by step, Git is now so prominent it s almost impossible to even think to another source control manager, even if awesome alternate solutions, unfortunately not as popular, exist as Mercurial.

A new Free Software project is now a Git repository on Github with README.md added as a quick description. All the other solutions are ostracized? How? None or very few potential contributors would notice said projects. It seems very difficult now to encourage potential contributors into learning a new source control manager AND a new forge for every project they want to contribute. Which was a basic requirement a few years ago. It s quite sad because Github, offering an original experience to its users, cut them out of a whole possibility realm. Maybe Github is one of the best web versioning control systems. But being the main one doesn t let room for a new competitor to grow. And it let Github initiate development newcomers into a narrow function set, totally unrelated to the strength of the Git tool itself.

3. Centralization, uniformization, proprietary software What s next? Laziness?

Fight against centralization is a main part of the Free Software ideology as centralization strengthens the power of those who manage it and who through it control those who are managed by it. Uniformization allergies born against main software companies and their wishes to impose a closed commercial software world was for a long time the main fuel for innovation thirst and intelligent alternative development. As we said above, part of the Free Software community was built as a reaction to proprietary software and their threat. The other part, without hoping for their disappearance, still chose a development model opposite to proprietary software, at least in the beginning, as now there s more and more bridges between the two.

The Github effect is a morbid one because of its consequences: at least centralization, uniformization, proprietary software usage as their bug tracking system. But some years ago the Dear Github buzz showed one more side effect, one I ve never thought about: laziness. For those who don t know what it is about, this letter is a complaint from several spokespersons from several Free Software projects which demand to Github team to finally implement, after years of polite asking, new functions. Since when Free Software project facing a roadblock request for clemency and don t build themselves the path they need? When Torvalds was involved in the Bitkeeper problem and the Linux kernel development team couldn t use anymore their revision control software, he developed Git. The mere fact of not being able to use one tool or functions lacking is the main motivation to seek alternative solutions and, as such, of the Free Software movement. Every Free Software community member able to code should have this reflex. You don t like what Github offers? Switch to Gitlab. You don t like it Gitlab? Improve it or make your own solution.

The Gitlab logo

Let s be crystal clear. I ve never said that every Free Software developers blocked should code his or her own alternative. We all have our own priorities, and some of us even like their beauty sleep, including me. But, to see that this open letter to Github has 1340 names attached to it, among them some spokespersons for major Free Software project showed me that need, willpower and strength to code a replacement are here. Maybe said replacement will be born from this letter, it would be the best outcome of this buzz.

In the end, Github usage is just another example of Internet usage massification. As Internet users are bound to go to massively centralized social network as Facebook or Twitter, developers are following the same path with Github. Even if a large fraction of developers realize the threat linked this centralized and proprietary organization, the whole community is following this centralization and uniformization trend. Github service is useful, free or with a reasonable price (depending on the functions you need) easy to use and up most of the time. Why would we try something else? Maybe because others are using us while we are savoring the convenience? The Free Software community seems to be quite sleepy to me.

The lion enjoying the hearth warm

About Me Carl Chenet, Free Software Indie Hacker, founder of the French-speaking Hacker News-like Journal du hacker. Follow me on social networks

• My Mastodon account : @carlchenet
• My Diaspora* account : @carlchenet
• My Twitter account : @carl_chenet

Translated from French by St phanie Chaptal. Original article written in 2015.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Debian I have updated txtorcon (a Twisted-based asynchronous Tor control protocol implementation used by ooniprobe, magic-wormhole and tahoe-lafs) to its latest upstream version. I ve also added two new binary packages that are built by the txtorcon source package: python3-txtorcon and python-txtorcon-doc for Python 3 support and generated HTML documentation respectively. I have gone through the scapy (Python module for the forging and dissection of network packets) bugs and closed a couple that seem to have been silently fixed by new upstream releases and not been caught in the BTS. I ve uploaded a minor revision to include a patch that fixes the version number reported by scapy. I have prepared and uploaded a new package for measurement-kit (a portable C++11 network measurement library) from the Open Observatory of Network Interference, which at time of writing is still in the NEW queue. I have also updated ooniprobe (probe for the Open Observatory of Network Interference) to its latest upstream version. I have updated the Swedish debconf strings in the xastir (X Amateur Station Tracking and Information Reporting) package, thanks to the translators. I have updated the direwolf (soundcard terminal node controller for APRS) package to its latest upstream version and fixed the creation of the system user to run direwolf with systemd to happen at the time the package is installed. Unfortunately, it has been necessary to drop the PDF documentation from the package as I was unable to contact the upstream author and acquire the Microsoft Word sources for this release. I have reviewed and sponsored the uploads of the new packages comptext (GUI based tool to compare two text streams), comptty (GUI based tool to compare two radio teletype streams) and flnet (amateur radio net control station software) in the hamradio team. Thanks to Ana Custura for preparing those packages, comptext and comptty are now available in unstable. I have updated the Debian Hamradio Blend metapackages to include cubicsdr (a software defined radio receiver). This build also refreshes the list of packages that can now be included as they had not been packaged at the time of the last build. I have produced and uploaded an initial package for python-azure-devtools (development tools for Azure SDK and CLI for Python) and have updated python-azure (the Azure SDK for Python) to a recent git snapshot. Due to some issues with python-vcr it is currently not possible to run the test suite as part of the build process and I m watching the situation. I have also fixed the auto dependency generation for python3-azure, which had previously been broken. Bugs closed (fixed/wontfix): #873036, #871940, #869566, #873083, #867420, #861753, #855385, #855497, #684727, #683711

Tor Project I have been working through tickets for Atlas (a tool for looking up details about Tor relays and bridges) and have merged and deployed a number of fixes. Some highlights include: bandwidth sorting in search results is now semantically correct (not just an alphanumeric sort ignoring units), added when a relay was first seen to the details page along with the host name if a reverse DNS record has been found for the IP address of the relay and added support for the NoEdConsensus flag (although happily no relays had this flag at the time this support was added). The metrics team has been working on merging projects into the metrics team website to give a unified view of information about the Tor network. This week I have been working towards a prototype of a port of Atlas to the metrics website s style and this work has been published in my personal Atlas git repository. If you d like to have a click around, you can do so. A relay operators meetup will be happening in Montreal on the 14th of October. I won t be present, but I have taken this opportunity to ask operators if there s anything that they would like from Atlas that they are not currently getting. Some feedback has already been received and turned into code and trac tickets. I also attended the weekly metrics team meeting in #tor-dev. Bugs closed (fixed/wontfix): #6787, #9814, #21958, #21636, #23296, #23160

Sustainability I believe it is important to be clear not only about the work I have already completed but also about the sustainability of this work into the future. I plan to include a short report on the current sustainability of my work in each weekly report. I continue to be happy to spend my time on this work, however I do find myself in a position where it may not be sustainable when it comes to hardware. My desktop, a Sun Ultra 24, is now 10 years old and I m starting to see random reboots which so far have not been explained. It is incredibly annoying to have this happen during a long build. Further, the hard drives in my NAS which are used for the local backups and for my local Debian mirror are starting to show SMART errors. It is not currently within my budget to replace any of this hardware. Please contact me if you believe you can help.

This week's energy was provided by Club Mate

Changes in version 0.2.0 (2017-09-07)

• Added support for building on Windows
• Upgraded to MsgPack 2.1.5 (#3)
• New R functions to manipulate MsgPack objects: msgpack_format, msgpack_map, msgpack_pack, msgpack_simplify, mgspack_unpack (#4)
• New R functions also available as msgpackFormat, msgpackMap, msgpackPack, msgpackSimplify, mgspackUnpack (#4)
• New vignette (#4)
• New tests (#4)

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

conn V2-1
  left        = 2001:db8:1::1
  leftsubnet  = 2001:db8:a1::/64
  right       = 2001:db8:2::1
  rightsubnet = 2001:db8:a2::/64
  authby      = psk
  auto        = route

$ ip xfrm policy
src 2001:db8:a1::/64 dst 2001:db8:a2::/64
        dir out priority 399999 ptype main
        tmpl src 2001:db8:1::1 dst 2001:db8:2::1
                proto esp reqid 4 mode tunnel
src 2001:db8:a2::/64 dst 2001:db8:a1::/64
        dir fwd priority 399999 ptype main
        tmpl src 2001:db8:2::1 dst 2001:db8:1::1
                proto esp reqid 4 mode tunnel
src 2001:db8:a2::/64 dst 2001:db8:a1::/64
        dir in priority 399999 ptype main
        tmpl src 2001:db8:2::1 dst 2001:db8:1::1
                proto esp reqid 4 mode tunnel
[ ]

• a direction (out, in or fwd²),
• a selector (source subnet, destination subnet, protocol, ports),
• a mode (transport or tunnel),
• an encapsulation protocol (esp or ah), and
• the endpoint source and destination addresses.

$ ip xfrm state
src 2001:db8:1::1 dst 2001:db8:2::1
        proto esp spi 0xc1890b6e reqid 4 mode tunnel
        replay-window 0 flag af-unspec
        auth-trunc hmac(sha256) 0x5b68[ ]8ba2904 128
        enc cbc(aes) 0x8e0e377ad8fd91e8553648340ff0fa06
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
[ ]

$ tcpdump -pni eth0 -c2 -s0 esp
13:07:30.871150 IP6 2001:db8:1::1 > 2001:db8:2::1: ESP(spi=0xc1890b6e,seq=0x222)
13:07:30.872297 IP6 2001:db8:2::1 > 2001:db8:1::1: ESP(spi=0xcf2426b6,seq=0x204)

conn V1-1-to-V2-1
  left        = 2001:db8:1::1
  leftsubnet  = 2001:db8:a1::/64,2001:db8:a6::cc:1/128,2001:db8:a6::cc:5/128
  right       = 2001:db8:2::1
  rightsubnet = 2001:db8:a2::/64,2001:db8:a6::/64,2001:db8:a8::/64
  authby      = psk
  keyexchange = ikev2
  auto        = route

1. Routing daemons can be used to distribute routes to be protected by the VPN. This decreases the administrative burden when many subnets are present on each side.
2. Encapsulation and decapsulation can be executed in a different routing instance or namespace. This enables a clean separation between a private routing instance (where VPN users are) and a public routing instance (where VPN endpoints are).

Route-based VPN on Juniper Before looking at how to achieve that on Linux, let s have a look at the way it works with a JunOS-based platform (like a Juniper vSRX). This platform as long-standing history of supporting route-based VPNs (a feature already present in the Netscreen ISG platform). Let s assume we want to configure the IPsec VPN from V3-2 to V1-1. First, we need to configure the tunnel interface and bind it to the private routing instance containing only internal routes (with IPv4, they would have been RFC 1918 routes):

interfaces  
    st0  
        unit 1  
            family inet6  
                address 2001:db8:ff::7/127;
             
         
     
 
routing-instances  
    private  
        instance-type virtual-router;
        interface st0.1;
     
 

The second step is to configure the VPN:

security  
    /* Phase 1 configuration */
    ike  
        proposal IKE-P1  
            authentication-method pre-shared-keys;
            dh-group group20;
            encryption-algorithm aes-256-gcm;
         
        policy IKE-V1-1  
            mode main;
            proposals IKE-P1;
            pre-shared-key ascii-text "d8bdRxaY22oH1j89Z2nATeYyrXfP9ga6xC5mi0RG1uc";
         
        gateway GW-V1-1  
            ike-policy IKE-V1-1;
            address 2001:db8:1::1;
            external-interface lo0.1;
            general-ikeid;
            version v2-only;
         
     
    /* Phase 2 configuration */
    ipsec  
        proposal ESP-P2  
            protocol esp;
            encryption-algorithm aes-256-gcm;
         
        policy IPSEC-V1-1  
            perfect-forward-secrecy keys group20;
            proposals ESP-P2;
         
        vpn VPN-V1-1  
            bind-interface st0.1;
            df-bit copy;
            ike  
                gateway GW-V1-1;
                ipsec-policy IPSEC-V1-1;
             
            establish-tunnels on-traffic;
         
     
 

We get a route-based VPN because we bind the st0.1 interface to the VPN-V1-1 VPN. Once the VPN is up, any packet entering st0.1 will be encapsulated and sent to the 2001:db8: 1::1 endpoint. The last step is to configure BGP in the private routing instance to exchange routes with the remote site:

routing-instances  
    private  
        routing-options  
            router-id 1.0.3.2;
            maximum-paths 16;
         
        protocols  
            bgp  
                preference 140;
                log-updown;
                group v4-VPN  
                    type external;
                    local-as 65003;
                    hold-time 6;
                    neighbor 2001:db8:ff::6 peer-as 65001;
                    multipath;
                    export [ NEXT-HOP-SELF OUR-ROUTES NOTHING ];
                 
             
         
     
 

The export filter OUR-ROUTES needs to select the routes to be advertised to the other peers. For example:

policy-options  
    policy-statement OUR-ROUTES  
        term 10  
            from  
                protocol ospf3;
                route-type internal;
             
            then  
                metric 0;
                accept;
             
         
     
 

The configuration needs to be repeated for the other peers. The complete version is available on GitHub. Once the BGP sessions are up, we start learning routes from the other sites. For example, here is the route for 2001:db8: a1::/64:

> show route 2001:db8:a1::/64 protocol bgp table private.inet6.0 best-path
private.inet6.0: 15 destinations, 19 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
2001:db8:a1::/64   *[BGP/140] 01:12:32, localpref 100, from 2001:db8:ff::6
                      AS path: 65001 I, validation-state: unverified
                      to 2001:db8:ff::6 via st0.1
                    > to 2001:db8:ff::14 via st0.2

It was learnt both from V1-1 (through st0.1) and V1-2 (through st0.2). The route is part of the private routing instance but encapsulated packets are sent/received in the public routing instance. No route-leaking is needed for this configuration. The VPN cannot be used as a gateway from internal hosts to external hosts (or vice-versa). This could also have been done with JunOS security policies (stateful firewall rules) but doing the separation with routing instances also ensure routes from different domains are not mixed and a simple policy misconfiguration won t lead to a disaster.

Route-based VPN on Linux Starting from Linux 3.15, a similar configuration is possible with the help of a virtual tunnel interface³. First, we create the private namespace:

# ip netns add private
# ip netns exec private sysctl -qw net.ipv6.conf.all.forwarding=1

Any private interface needs to be moved to this namespace (no IP is configured as we can use IPv6 link-local addresses):

# ip link set netns private dev eth1
# ip link set netns private dev eth2
# ip netns exec private ip link set up dev eth1
# ip netns exec private ip link set up dev eth2

Then, we create vti6, a tunnel interface (similar to st0.1 in the JunOS example):

# ip tunnel add vti6 \
   mode vti6 \
   local 2001:db8:1::1 \
   remote 2001:db8:3::2 \
   key 6
# ip link set netns private dev vti6
# ip netns exec private ip addr add 2001:db8:ff::6/127 dev vti6
# ip netns exec private sysctl -qw net.ipv4.conf.vti6.disable_policy=1
# ip netns exec private sysctl -qw net.ipv4.conf.vti6.disable_xfrm=1
# ip netns exec private ip link set vti6 mtu 1500
# ip netns exec private ip link set vti6 up

The tunnel interface is created in the initial namespace and moved to the private one. It will remember its original namespace where it will process encapsulated packets. Any packet entering the interface will temporarily get a firewall mark of 6 that will be used only to match the appropriate IPsec policy⁴ below. The kernel sets a low MTU on the interface to handle any possible combination of ciphers and protocols. We set it to 1500 and let PMTUD do its work. We can then configure strongSwan⁵:

conn V3-2
  left        = 2001:db8:1::1
  leftsubnet  = ::/0
  right       = 2001:db8:3::2
  rightsubnet = ::/0
  authby      = psk
  mark        = 6
  auto        = route
  keyexchange = ikev2
  keyingtries = %forever
  ike         = aes256gcm16-prfsha384-ecp384!
  esp         = aes256gcm16-prfsha384-ecp384!
  mobike      = no

The IKE daemon configures the following policies in the kernel:

$ ip xfrm policy
src ::/0 dst ::/0
        dir out priority 399999 ptype main
        mark 0x6/0xffffffff
        tmpl src 2001:db8:1::1 dst 2001:db8:3::2
                proto esp reqid 1 mode tunnel
src ::/0 dst ::/0
        dir fwd priority 399999 ptype main
        mark 0x6/0xffffffff
        tmpl src 2001:db8:3::2 dst 2001:db8:1::1
                proto esp reqid 1 mode tunnel
src ::/0 dst ::/0
        dir in priority 399999 ptype main
        mark 0x6/0xffffffff
        tmpl src 2001:db8:3::2 dst 2001:db8:1::1
                proto esp reqid 1 mode tunnel
[ ]

Those policies are used for any source or destination as long as the firewall mark is equal to 6, which matches the mark configured for the tunnel interface. The last step is to configure BGP to exchange routes. We can use BIRD for this:

router id 1.0.1.1;
protocol device  
   scan time 10;
 
protocol kernel  
   persist;
   learn;
   import all;
   export all;
   merge paths yes;
 
protocol bgp IBGP_V3_2  
   local 2001:db8:ff::6 as 65001;
   neighbor 2001:db8:ff::7 as 65003;
   import all;
   export where ifname ~ "eth*";
   preference 160;
   hold time 6;
 

Once BIRD is started in the private namespace, we can check routes are learned correctly:

$ ip netns exec private ip -6 route show 2001:db8:a3::/64
2001:db8:a3::/64 proto bird metric 1024
        nexthop via 2001:db8:ff::5  dev vti5 weight 1
        nexthop via 2001:db8:ff::7  dev vti6 weight 1

The above route was learnt from both V3-1 (through vti5) and V3-2 (through vti6). Like for the JunOS version, there is no route-leaking between the private namespace and the initial one. The VPN cannot be used as a gateway between the two namespaces, only for encapsulation. This also prevent a misconfiguration (for example, IKE daemon not running) from allowing packets to leave the private network. As a bonus, unencrypted traffic can be observed with tcpdump on the tunnel interface:

$ ip netns exec private tcpdump -pni vti6 icmp6
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vti6, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
20:51:15.258708 IP6 2001:db8:a1::1 > 2001:db8:a3::1: ICMP6, echo request, seq 69
20:51:15.260874 IP6 2001:db8:a3::1 > 2001:db8:a1::1: ICMP6, echo reply, seq 69

You can find all the configuration files for this example on GitHub. The documentation of strongSwan also features a page about route-based VPNs.

---

1. Everything in this post should work with Libreswan.
2. fwd is for incoming packets on non-local addresses. It only makes sense in transport mode and is a Linux-only particularity.
3. Virtual tunnel interfaces (VTI) were introduced in Linux 3.6 (for IPv4) and Linux 3.12 (for IPv6). Appropriate namespace support was added in 3.15. KLIPS, an alternative out-of-tree stack available since Linux 2.2, also features tunnel interfaces.
4. The mark is set right before doing a policy lookup and restored after that. Consequently, it doesn t affect other possible uses (filtering, routing). However, as Netfilter can also set a mark, one should be careful for conflicts.
5. The ciphers used here are the strongest ones currently possible while keeping compatibility with JunOS. The documentation for strongSwan contains a complete list of supported algorithms as well as security recommendations to choose them.

• isdebianreproducibleyet.com was released and subsequently updated.

• Chris Lamb filed #874102 filed against texlive-bin to incorporate a proposed upstream change to fix reproducibility issues in generated PDF files.

• fastforward (#776972) (lamby)
• dtc-xen (#777322) (lamby)
• dhcpping (#777320) (lamby)
• vimoutliner (#776369) (lamby)

• The Linux kernel announced support for the randstruct GCC plugin.
• "Please make the output of gio-querymodules deterministic" was merged upstream. (lamby)

• Bernhard M. Wiedemann:
  • gcin: (merged) Uninitialized stack memory
  • html5-parser (merged): Sorting
  • gromacs (merged): Date
  • crawl (merged): Date
  • GCompris-gtk: Date
  • heimdal: Date, hostname
• Chris Lamb:
  • Numpy (#872459) (PR #9652) build path (merged)

• Adrian Bunk:
  • #874186 filed against svgpp.

• Adrian Bunk (15)

• Chris Lamb:
  • Add support for "binwalking" to find (eg.) concatenated CPIO archives. (Closes: #820631)
  • Loosen matching of file(1)'s output to ensure we correctly also match TTF files under file 5.32.
  • Check we identify all CPIO fixtures in tests
  • Make failing a some flake8 tests cause the testsuite to fail. (Currently just "undefined name")
  • Countless style fixups, eg. remove unused imports, removing blank lines from end of flies, etc.
  • Compare None using identity, not equality.
  • diffoscope.diff: Correct reference to self.buf.
  • comparators.utils.file: Correct reference to path_apparent_size.
• Juliana Rodrigues:
  • Sip html_visuals test if 'sng' binary is not available
• Mattia Rizzolo:
  • Numerous PEP8 fixes (eg. E122, E302, E713, etc.)

• Santiago Torres:
  • Correct string formatting in get_all_servers
• Ximin Luo:
  • Heavy refactoring towards supporting running more than 2 builds
  • Rename append_command to append_to_build_command

• h01ger:
  • Don't update the stretch package sets anymore
  • Update URL for Tails packages list
  • Disabled the OpenWrt at the request of lynxis as they were broken; if no-one shows up to fix them, we'll probably remove them in the future, as all current development happens within LEDE.
  • Renewed the Let's Encrypt SSL certificates.
• Mattia Rizzolo:
  • Avoid stale temporary files from reproducible_build.sh
  • Avoid removing the temporary directory twice
  • Fix IRC notifications when somebody asks for artifacts
  • Simplify 'if' conditions

• There were a lot of GCC-7 bugs to fix this month which claimed most of my games related time.
• Bug fixes: torcs (RC #853685), berusky2 (RC #853325), lordsawar (RC #853529), simutrans(#869029), gngb (RC #853425), libclaw (RC #853488), funguloids (RC #853408), plee-the-bear (RC #853618), alien-arena (RC #871218), fretsonfire (RC #872934), minetest, (RC #873324) widelands, (RC #871114), pingus (RC #853614)
• I released version 2.1 of the Debian Games Blend.
• I completely overhauled the gngb package, a color gameboy emulator.
• I packaged new upstream versions of freeciv, peg-e and blockattack.
• I backported the memory leak fix for unknown-horizons and fife to Stretch (#871037).
• I investigated some graphical glitches in Neverball which appear to be related to OpenGL and the graphic stack in general but I couldn t find an immediate solution. (#871223)

• I sponsored libimglib2-java for Ghislain Vaillant.
• New upstream releases: apktool, jboss-modules, jboss-logging-tools, jboss-logmanager.
• For jboss-xnio I packaged two new build-dependencies which are wildfly-common and wildfly-client-config and they are currently waiting in the NEW queue.
• The last build-dependency for PDFsam was accepted this month and I was able to upload the new version to experimental. Unfortunately the program is currently not really usable due to a bug in libhibernate-validator-java (#874579)

• From 31. July until 06. August I was in charge of our LTS frontdesk. I triaged bugs in tinyproxy, mantis, sox, timidity, ioquake3, varnish, libao, clamav, binutils, smplayer, libid3tag, mpg123 and shadow.
• DLA-1064-1. Issued a security update for freeradius fixing 6 CVE.
• DLA-1068-1. Issued a security update for git fixing 1 CVE.
• DLA-1077-1. Issued a security update for faad2 fixing 11 CVE.
• DLA-1083-1. Issued a security update for openexr fixing 3 CVE.
• DLA-1095-1. Issued a security update for freerdp fixing 5 CVE.

• I uploaded a security fix for openexr (#864078) to fix CVE-2017-9110, CVE-2017-9112 and CVE-2017-9116.

Requirements:

• Synology NAS running DSM >= 6.0
• Sub/domain name with ability to update DNS records
• SSH access to your Synology NAS

DSM: Control Panel > Security > Certificates > Add

Add a new certificate > Get a certificate from Let's Encrypt

Name: SynoCommunity
Location: http://packages.synocommunity.com

DSM: Control Panel > Application Portal > Reverse Proxy > Create

/usr/syno/etc/certificate/_archive/

drwx------ 2 root root 4096 Sep 10 12:57 JeRh3Y

johndoe/freenode:password

• XMPP chatroom
• email: tails-dev@boum.org (public) or tails@boum.org (private)

1. Install the build dependencies:

   sudo apt install \
       git \
       rake \
       libvirt-daemon-system \
       dnsmasq-base \
       ebtables \
       qemu-system-x86 \
       qemu-utils \
       vagrant \
       vagrant-libvirt \
       vmdebootstrap && \
   sudo systemctl restart libvirtd
   

2. Ensure your user is in the relevant groups:

   for group in kvm libvirt libvirt-qemu ; do
      sudo adduser "$(whoami)" "$group"
   done
   

3. Logout and log back in to apply the new group memberships.

git clone https://git-tails.immerda.ch/tails && \
cd tails && \
git checkout 3.2-alpha2 && \
git submodule update --init && \
rake build

sudo apt install apt-show-versions && \
(
  for f in /etc/issue /proc/cpuinfo
  do
    echo "--- File: $ f  ---"
    cat "$ f "
    echo
  done
  for c in free locale env 'uname -a' '/usr/sbin/libvirtd --version' \
            'qemu-system-x86_64 --version' 'vagrant --version'
  do
    echo "--- Command: $ c  ---"
    eval "$ c "
    echo
  done
  echo '--- APT package versions ---'
  apt-show-versions qemu:amd64 linux-image-amd64:amd64 vagrant \
                    libvirt0:amd64
)   bzip2 > system-info.txt.bz2

bzless system-info.txt.bz2

• set Category to Build system;
• paste the output of rake build;
• attach system-info.txt.bz2 (this will publish that file).

sha512sum tails-amd64-3.2~alpha2.iso

9b4e9e7ee7b2ab6a3fb959d4e4a2db346ae322f9db5409be4d5460156fa1101c23d834a1886c0ce6bef2ed6fe378a7e76f03394c7f651cc4c9a44ba608dda0bc

1. Install diffoscope version 83 or higher and all the packages it recommends. For example, if you're using Debian Stretch:

   sudo apt remove diffoscope && \
   echo 'deb http://ftp.debian.org/debian stretch-backports main' \
       sudo tee /etc/apt/sources.list.d/stretch-backports.list && \
   sudo apt update && \
   sudo apt -o APT::Install-Recommends="true" \
            install diffoscope/stretch-backports
   

2. Download the official Tails 3.2~alpha2 ISO image.
3. Compare the official Tails 3.2~alpha2 ISO image with yours:

   diffoscope \
          --text diffoscope.txt \
          --html diffoscope.html \
          --max-report-size 262144000 \
          --max-diff-block-lines 10000 \
          --max-diff-input-lines 10000000 \
          path/to/official/tails-amd64-3.2~alpha2.iso \
          path/to/your/own/tails-amd64-3.2~alpha2.iso
   bzip2 diffoscope. txt,html 
   

4. Send an email to tails-dev@boum.org (public) or tails@boum.org (private) with the subject "Reproduction of Tails 3.2~alpha2 failed", attaching:
   • system-info.txt.bz2;
   • the smallest file among diffoscope.txt.bz2 and diffoscope.html.bz2, except if they are larger than 100 KiB, in which case better upload the file somewhere (e.g. share.riseup.net and share the link in your email.

• The main container is using the (MS-CFB), which is notably used by MS Office 97-2003 .doc files. It contains several streams:
  • Metadata stream: in OLE Property Set format (MS-OLEPS), contains 2 blobs in a specific Type-Length-Value (TLV) format:
    • _ctlfile: global archive properties and access list
      It is obfuscated by means of static-key AES encryption.
      The properties include archive initial filename and a global IV.
      A global encryption key is itself encrypted in each user entry.
    • _catalog: file list
      Contains each file metadata indexed with a 15-bytes identifier.
      Directories are supported.
      Full filename is encrypted using AES.
      File extension is (redundantly) stored in clear, and so are file metadata such as modification time.
  • Each file in the archive compressed with zlib and encrypted with the standard AES algorithm, in a separate stream.
    Several encryption schemes and key sizes are supported.
    The file stream is split in chunks of 512 bytes, individually encrypted.
  • Optional streams, contain additional metadata as well as pictures to display in the application background ("watermarks"). They are not discussed here.

+----------------------------------------------------------------------------------------------------+
  .zed archive (MS-CBF)                                                                               
                                                                                                      
   stream #1                         stream #2                       stream #3...                     
  +------------------------------+  +---------------------------+  +---------------------------+      
    metadata (MS-OLEPS)               encryption (AES)               encryption (AES)                 
                                      512-bytes chunks               512-bytes chunks                 
    +--------------------------+                                                                      
      obfuscation (static key)        +-----------------------+      +-----------------------+        
      +----------------------+       -  compression (zlib)     -    -  compression (zlib)     -       
       _ctlfile (TLV)                                                                            ...  
      +----------------------+          +---------------+              +---------------+               
    +--------------------------+          file contents                  file contents                
                                                                                                      
    +--------------------------+     -  +---------------+      -    -  +---------------+      -       
      _catalog (TLV)                                                                                  
    +--------------------------+      +-----------------------+      +-----------------------+        
  +------------------------------+  +---------------------------+  +---------------------------+      
+----------------------------------------------------------------------------------------------------+

• obfuscated metadata encrypted with static AES key
• filenames and files in archives with 'encryption_mode' set to "AES-CBC-STREAM"
• any AES ciphertext of size < 16 bytes, regardless of encryption mode

• the second-to-last block of the ciphertext is encrypted in AES-ECB mode (i.e. block cipher encryption only, without XORing with the IV)
• then XOR-ed with the last partial block (hence truncated to the length of the partial block)

• filenames and files in archives with 'encryption_mode' set to "AES-CBC-CTS".
  • exception: if the size of the ciphertext is < 16 bytes, then "STREAM" is used instead.

• static delimiter 0765921A2A0774534752073361719300 (hexadecimal) followed by 0100 (hexadecimal) (18 bytes total)
• 16-byte IV
• ciphertext
• 1 uint32be representing the length of all the above
• static delimiter 0765921A2A0774534752073361719300 (hexadecimal) followed by "ZoneCentral (R)" (ASCII) and a NUL byte (32 bytes total)

• global archive properties as a 'fileprops' structure,
• extra archive properties as a 'archive_extraprops' structure
• users access list as a series of 'passworduser' and 'rsauser entries.

• 4 bytes for Type (specified as a 4-bytes hexadecimal below)
• 4 bytes for value Length (uint32be)
• Value

• 80110600: fileprops, used for the file list as well as for the global archive properties
• 001b0600: archive_extraprops
• 80140600: accesslist

• 80110600: fileprops (TLV structure): global archive properties
  • 00230400: archive_pathname (UTF-16LE string): initial archive filename (past versions also leaked the full pathname of the initial archive)
  • 80270200: encryption_mode (utf32be): 103 for "AES-CBC-STREAM", 104 for "AES-CBC-CTS"
  • 80260200: encryption_strength (utf32be): AES key size, in bytes (e.g. 32 means AES with a 256-bit key)
  • 80280500: files_iv (sequence of bytes): global IV for all filenames and file contents
• 001b0600: archive_extraprops (TLV structure): additionnal archive properties (optional)
  • 00c40500: archive_creationtime (FILETIME): date and time when archive was initially created (optional)
  • 00c00400: archive_createdwith (UTF-16LE string): uuid-like structure describing the application that initialized the archive (optional)
    00000188-1000-3CA8-8868-36F59DEFD14D is Zed! Free 1.0.188.
• 80140600: accesslist (TLV structure): describe the users, their key encryption and their permissions
  • 80610600: passworduser (TLV structure): user identified by password (0 or more)
  • 80620600: rsauser (TLV structure): user identified by RSA key (via file or PKCS#11 token) (0 or more)
  • Fields common to passworduser and rsauser:
    • 80710400: login (UTF-16LE string): user name
    • 80720300: login_md5 (sequence of bytes): used by the application to search for a user name
    • 807e0100: priv1 (uchar): user privileges; present and set to 1 when user is admin (optional)
    • 00830200: priv2 (uint32be): user privileges; present and set to 2 when user is admin, present and set to 5 when user is a marked as mandatory, e.g. for recovery keys (optional)
    • 80740500: files_key_ciphertext (sequence of bytes): the archive encryption key, itself encrypted
    • 00840500: user_creationtime (FILETIME): date and time when the user was added to the archive
  • passworduser-specific fields:
    • 80760500: pbe_salt (sequence of bytes): salt for PBE
    • 80770200: pbe_iter (uint32be): number of iterations for PBE
    • 80780200: pkcs12_hashfunc (uint32be): hash function used for PBE and PBA key derivation
    • 80790500: pba_checksum (sequence of bytes): password derived with PBA to check for password validity
    • 807a0500: pba_salt (sequence of bytes): salt for PBA
    • 807b0200: pba_iter (uint32be): number of iterations for PBA
  • rsauser-specific fields:
    • 807d0500: certificate (sequence of bytes): user X509 certificate in DER format

• 80110600: fileprops (TLV structure): describe the archive files (0 or more)
  • 80300500: file_id (sequence of bytes): a 16-byte unique identifier
  • 80310400: filename_halfanon (UTF-16LE string): half-anonymized filename, e.g. File1.txt (leaking filename extension)
  • 00380500: filename_ciphertext (sequence of bytes): encrypted filename; may have a trailing NUL byte once decrypted
  • 80330500: file_size (uint64le): decompressed file size in bytes
  • 80340500: file_creationtime (FILETIME): file creation date and time
  • 80350500: file_lastwritetime (FILETIME): file last modification date and time
  • 80360500: file_lastaccesstime (FILETIME): file last access date and time
  • 00370500: parent_directory_id (sequence of bytes): file_id of the parent directory, 0 is top-level
  • 80320100: is_dir (uint32be): 1 if entry is directory (optional)

• 21: SHA-1
• 22: SHA-256

• ID: 3
• 'pba_salt': the salt, typically an 8-byte random sequence
• 'pba_iter': the iteration count, typically 200000

• 'pbe_salt': the salt, typically an 8-bytes random sequence
• 'pbe_iter': the iteration count, typically 100000

• ID: 1
• size: 32

• ID: 2
• size: 16

Initial  offset: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Shuffled offset: 3 2 1 0 5 4 7 6 8 9 10 11 12 13 14 15

• expressing the current chunk number as little endian on 16 bytes
• XORing it with the global IV
• encrypting with the global AES key in ECB mode (without IV).

• uint32le: unsigned int on 4 bytes, little endian.
• uint32be: unsigned int on 4 bytes, big endian.
• uint64le: unsigned int on 8 bytes, little endian.
• uchar: unsigned char on 1 byte.
• (hexadecimal): represents 1 byte with 2 hexadecimal numbers.
  e.g. 4142 (hexadecimal) means decimal values 65 and then 66.
• (ascii): represents 1 byte with the matching 1-byte ASCII value.
  e.g. 'AB\0' means decimal values 65 then 66 and then 0.
• UTF-16LE: text encoded using 2-bytes Unicode, little-endian, without Byte Order Mark (BOM), as commonly used under MS Windows.
• IV: initialization vector for AES encryption and decryption.
• CBC: Cipher Block Chaining, see Recommendation for Block Cipher Modes of Operation and Wikipedia.
• CTS: CipherText Stealing, one type of end-of-stream handling, see Recommendation for Block Cipher Modes of Operation: Three Variants of Ciphertext Stealing for CBC Mode and Wikipedia.
• PBA, PBE: password-based authentication, password-based encryption; see PKCS #11 Cryptographic Token Interface Current Mechanisms Specification.
• MS-CFB: Compound File Binary Format, documented at MSDN, libolecf, Wikipedia.
  Libraries that can manipulate this format include olefile, libolecf and libforensics.
• MS-OLEPS: OLE Property Set, documented at MSDN and libfole.
  The 'oleps.py' demo from libforensics displays detailed information on MS-OLEPS streams; olefile and libolecf have basic support.
• TLV : Type-Length-Value encoding ; 4 bytes for type, 4 bytes for length (uint32be), and a variable-sized value; see Type-length-value at Wikipedia for the general principle.
• FILETIME: timestamp as 1/10000000s since January 1, 1601 (UTC), see FILETIME structure at MSDN.

• The main container is using the (MS-CFB), which is notably used by MS Office 97-2003 .doc files. It contains several streams:
  • Metadata stream: in OLE Property Set format (MS-OLEPS), contains 2 blobs in a specific Type-Length-Value (TLV) format:
    • _ctlfile: global archive properties and access list
      It is obfuscated by means of static-key AES encryption.
      The properties include archive initial filename and a global IV.
      A global encryption key is itself encrypted in each user entry.
    • _catalog: file list
      Contains each file metadata indexed with a 15-bytes identifier.
      Directories are supported.
      Full filename is encrypted using AES.
      File extension is (redundantly) stored in clear, and so are file metadata such as modification time.
  • Each file in the archive compressed with zlib and encrypted with the standard AES algorithm, in a separate stream.
    Several encryption schemes and key sizes are supported.
    The file stream is split in chunks of 512 bytes, individually encrypted.
  • Optional streams, contain additional metadata as well as pictures to display in the application background ("watermarks"). They are not discussed here.

+----------------------------------------------------------------------------------------------------+
  .zed archive (MS-CBF)                                                                               
                                                                                                      
   stream #1                         stream #2                       stream #3...                     
  +------------------------------+  +---------------------------+  +---------------------------+      
    metadata (MS-OLEPS)               encryption (AES)               encryption (AES)                 
                                      512-bytes chunks               512-bytes chunks                 
    +--------------------------+                                                                      
      obfuscation (static key)        +-----------------------+      +-----------------------+        
      +----------------------+       -  compression (zlib)     -    -  compression (zlib)     -       
       _ctlfile (TLV)                                                                            ...  
      +----------------------+          +---------------+              +---------------+               
    +--------------------------+          file contents                  file contents                
                                                                                                      
    +--------------------------+     -  +---------------+      -    -  +---------------+      -       
      _catalog (TLV)                                                                                  
    +--------------------------+      +-----------------------+      +-----------------------+        
  +------------------------------+  +---------------------------+  +---------------------------+      
+----------------------------------------------------------------------------------------------------+

• obfuscated metadata encrypted with static AES key
• filenames and files in archives with 'encryption_mode' set to "AES-CBC-STREAM"
• any AES ciphertext of size < 16 bytes, regardless of encryption mode

• the second-to-last block of the ciphertext is encrypted in AES-ECB mode (i.e. block cipher encryption only, without XORing with the IV)
• then XOR-ed with the last partial block (hence truncated to the length of the partial block)

• filenames and files in archives with 'encryption_mode' set to "AES-CBC-CTS".
  • exception: if the size of the ciphertext is < 16 bytes, then "STREAM" is used instead.

• static delimiter 0765921A2A0774534752073361719300 (hexadecimal) followed by 0100 (hexadecimal) (18 bytes total)
• 16-byte IV
• ciphertext
• 1 uint32be representing the length of all the above
• static delimiter 0765921A2A0774534752073361719300 (hexadecimal) followed by "ZoneCentral (R)" (ASCII) and a NUL byte (32 bytes total)

• global archive properties as a 'fileprops' structure,
• extra archive properties as a 'archive_extraprops' structure
• users access list as a series of 'passworduser' and 'rsauser entries.

• 4 bytes for Type (specified as a 4-bytes hexadecimal below)
• 4 bytes for value Length (uint32be)
• Value

• 80110600: fileprops, used for the file list as well as for the global archive properties
• 001b0600: archive_extraprops
• 80140600: accesslist

• 80110600: fileprops (TLV structure): global archive properties
  • 00230400: archive_pathname (UTF-16LE string): initial archive filename (past versions also leaked the full pathname of the initial archive)
  • 80270200: encryption_mode (utf32be): 103 for "AES-CBC-STREAM", 104 for "AES-CBC-CTS"
  • 80260200: encryption_strength (utf32be): AES key size, in bytes (e.g. 32 means AES with a 256-bit key)
  • 80280500: files_iv (sequence of bytes): global IV for all filenames and file contents
• 001b0600: archive_extraprops (TLV structure): additionnal archive properties (optional)
  • 00c40500: archive_creationtime (FILETIME): date and time when archive was initially created (optional)
  • 00c00400: archive_createdwith (UTF-16LE string): uuid-like structure describing the application that initialized the archive (optional)
    00000188-1000-3CA8-8868-36F59DEFD14D is Zed! Free 1.0.188.
• 80140600: accesslist (TLV structure): describe the users, their key encryption and their permissions
  • 80610600: passworduser (TLV structure): user identified by password (0 or more)
  • 80620600: rsauser (TLV structure): user identified by RSA key (via file or PKCS#11 token) (0 or more)
  • Fields common to passworduser and rsauser:
    • 80710400: login (UTF-16LE string): user name
    • 80720300: login_md5 (sequence of bytes): used by the application to search for a user name
    • 807e0100: priv1 (uchar): user privileges; present and set to 1 when user is admin (optional)
    • 00830200: priv2 (uint32be): user privileges; present and set to 2 when user is admin, present and set to 5 when user is a marked as mandatory, e.g. for recovery keys (optional)
    • 80740500: files_key_ciphertext (sequence of bytes): the archive encryption key, itself encrypted
    • 00840500: user_creationtime (FILETIME): date and time when the user was added to the archive
  • passworduser-specific fields:
    • 80760500: pbe_salt (sequence of bytes): salt for PBE
    • 80770200: pbe_iter (uint32be): number of iterations for PBE
    • 80780200: pkcs12_hashfunc (uint32be): hash function used for PBE and PBA key derivation
    • 80790500: pba_checksum (sequence of bytes): password derived with PBA to check for password validity
    • 807a0500: pba_salt (sequence of bytes): salt for PBA
    • 807b0200: pba_iter (uint32be): number of iterations for PBA
  • rsauser-specific fields:
    • 807d0500: certificate (sequence of bytes): user X509 certificate in DER format

• 80110600: fileprops (TLV structure): describe the archive files (0 or more)
  • 80300500: file_id (sequence of bytes): a 16-byte unique identifier
  • 80310400: filename_halfanon (UTF-16LE string): half-anonymized filename, e.g. File1.txt (leaking filename extension)
  • 00380500: filename_ciphertext (sequence of bytes): encrypted filename; may have a trailing NUL byte once decrypted
  • 80330500: file_size (uint64le): decompressed file size in bytes
  • 80340500: file_creationtime (FILETIME): file creation date and time
  • 80350500: file_lastwritetime (FILETIME): file last modification date and time
  • 80360500: file_lastaccesstime (FILETIME): file last access date and time
  • 00370500: parent_directory_id (sequence of bytes): file_id of the parent directory, 0 is top-level
  • 80320100: is_dir (uint32be): 1 if entry is directory (optional)

• 21: SHA-1
• 22: SHA-256

• ID: 3
• 'pba_salt': the salt, typically an 8-byte random sequence
• 'pba_iter': the iteration count, typically 200000

• 'pbe_salt': the salt, typically an 8-bytes random sequence
• 'pbe_iter': the iteration count, typically 100000

• ID: 1
• size: 32

• ID: 2
• size: 16

Initial  offset: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Shuffled offset: 3 2 1 0 5 4 7 6 8 9 10 11 12 13 14 15

• expressing the current chunk number as little endian on 16 bytes
• XORing it with the global IV
• encrypting with the global AES key in ECB mode (without IV).

• uint32le: unsigned int on 4 bytes, little endian.
• uint32be: unsigned int on 4 bytes, big endian.
• uint64le: unsigned int on 8 bytes, little endian.
• uchar: unsigned char on 1 byte.
• (hexadecimal): represents 1 byte with 2 hexadecimal numbers.
  e.g. 4142 (hexadecimal) means decimal values 65 and then 66.
• (ascii): represents 1 byte with the matching 1-byte ASCII value.
  e.g. 'AB\0' means decimal values 65 then 66 and then 0.
• UTF-16LE: text encoded using 2-bytes Unicode, little-endian, without Byte Order Mark (BOM), as commonly used under MS Windows.
• IV: initialization vector for AES encryption and decryption.
• CBC: Cipher Block Chaining, see Recommendation for Block Cipher Modes of Operation and Wikipedia.
• CTS: CipherText Stealing, one type of end-of-stream handling, see Recommendation for Block Cipher Modes of Operation: Three Variants of Ciphertext Stealing for CBC Mode and Wikipedia.
• PBA, PBE: password-based authentication, password-based encryption; see PKCS #11 Cryptographic Token Interface Current Mechanisms Specification.
• MS-CFB: Compound File Binary Format, documented at MSDN, libolecf, Wikipedia.
  Libraries that can manipulate this format include olefile, libolecf and libforensics.
• MS-OLEPS: OLE Property Set, documented at MSDN and libfole.
  The 'oleps.py' demo from libforensics displays detailed information on MS-OLEPS streams; olefile and libolecf have basic support.
• TLV : Type-Length-Value encoding ; 4 bytes for type, 4 bytes for length (uint32be), and a variable-sized value; see Type-length-value at Wikipedia for the general principle.
• FILETIME: timestamp as 1/10000000s since January 1, 1601 (UTC), see FILETIME structure at MSDN.

tls paranoid
tls_verify_certificate yes
tls_private_key /etc/munin/master.key
tls_certificate /etc/munin/master.crt
tls_ca_certificate /etc/munin/ca.crt
tls_verify_depth 1

[node1.stormbind.net]
    address [2001:db8::]
    use_node_name yes

• "tls_certificate" is a Web Client Authentication certificate. The master connects to the nodes as a client.
• "tls_ca_certificate" is the root CA certificate.
• If you'd like to disable TLS connections, for example for localhost, set "tls disabled" in the node block.

./easyrsa init-pki
./easyrsa build-ca
./easrsa gen-req master
./easyrsa sign-req client master
./easyrsa set-rsa-pass master nopass

tls paranoid
tls_verify_certificate yes
tls_private_key /etc/munin/node1.key
tls_certificate /etc/munin/node1.crt
tls_ca_certificate /etc/munin/ca.crt
tls_verify_depth 1

./easyrsa gen-req node1
./easyrsa sign-req server node1
./easyrsa set-rsa-pass node1 nopass

• "tls_certificate" on the node must be a server certificate.
• You've to provide the CA here as well so we can verify the client certificate provided by the munin master.

2017, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.