Search Results: "cas"

6 May 2017

Vasudev Kamath: Magic: Attach a HTML file to mail and get different on other side

It's been a long time I did not write any new blog posts, and finally I got something which is interesting enough for a new post and so here it is. I actually wanted some bills from my ISP for some work and I could not find mail from the ISP which had bills for some specific months in my mailbox. Problem with my ISP is bills are accessible in their account which can be only accessed from their own network. They do have a mobile app and that does not seem to work especially for the billing section. I used mobile app and selected month for which I did not have bill and clicked Send Mail button. App happily showed message saying it sent the bill to my registered mail address but that was a fancy lie. After trying several time I gave up and decided I will do it once I get back home. Fast forward few hours, I'm back home from office and then I sat in front of my laptop and connected to ISP site, promptly downloaded the bills, then used notmuch-emacs and fire up a mail composer, attach those HTML file (yeah they produce bill in HTML file :-/) send it to my gmail and forget about it. Fast forward few more days, I just remember I need those bills. I got hold of mail I sent earlier in gmail inbox and clicked on attachment and when browser opened the attachment I was shocked/surprised . Did you ask why?. See what I saw when I opened attachment.

Well I was confused for moment on what happened, I tried downloading the file and opened it an editor, and I see Chinese characters here also. After reaching home I checked the Sent folder in my laptop where I keep copy of mails sent using notmuch-emacs's notmuch-fcc-dirs variable. I open the file and open the attachement and I see same character as I saw in browser!. I open the original bills I downloaded and it looks fine. To check if I really attached the correct files I again drafted a mail this time to my personal email and sent it. Now I open the file from Sent folder and voila again there is different content inside my attachment!. Same happens when I receive the mail and open the attachment everything inside is Chinese characters!. Totally lost I finally opened the original file using emacs, since I use spacemacs it asked me for downloading of html layer and after which I was surprised because everything inside the editor is in Chinese!. OK finally I've something now so I opened same file at same time in nano from terminal and there it looks fine!. OK that is weird again I read carefully content and saw this line in beginning of file

<?xml version="1.0" encoding="UTF-16" ?>
<html xmlns="http://www.w3.org/1999/xhtml>....

OK that is new XML tag had encoding declared as UTF-16, really?. And just for checking I changed it to UTF-8 and voila file is back to normal in Emacs window!. To confirm this behavior I created a sample file with following content

<?xml version="1.0" encoding="utf-8"?><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /><title> Something weird </title><body>blablablablablalabkabakba</body></html>

Yeah with long line because ISP had similar case and now I opened the same file in nano using terminal and changed encoding="UTF-8" to encoding="UTF-16" and the behavior repeated I see Chinese character in the emacs buffer which has also opened the same file. Below is the small gif showing this happening in real time, see what happens in emacs buffer when I change the encoding in terminal below. Fun right?.

I made following observation from above experiments.

When I open original file in browser or my sample file with encoding="UTF-16" its normal, no Chinese characters.
When I attach the mail and send it out the attachment some how gets converted into HTML file with Chinese characters and viewing source from browser shows header containing xml declarations and original <html> declarations get ripped off and new tags show up which I've pasted below.
If I download the same file and open in editor only Chinese characters are present and no HTML tags inside it. So definitely the new tags which I saw by viewing source in browser is added by Firefox.
I create similar HTML file and change encoding I can see the characters changing back and forth in emacs web-mode when I change encoding of the file.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
  <head><META http-equiv="Content-Type"  content="text/html; charset=utf-8">

So if I'm understanding this correctly, Emacs due to encoding declaration interprets the actual contents differently. To see how Emacs really interpreted the content I opened the sent mail I had in raw format and saw following header lines for attachment.

Content-Type: text/html; charset=utf-8
Content-Disposition: attachment;
             filename=SOA-102012059675-FEBRUARY-2017.html
Content-Transfer-Encoding: base64

This was followed with base64 encoded data. So does this mean emacs interpreted the content as UTF-16 and encoded the content using UTF-8?. Again I've no clue, so I changed the encoding in both the files to be as UTF-8 and sent the mail by attaching these files again to see what happens. And my guess was right I could get the attachment as is on the receiving side. And inspecting raw mail the attachment headers now different than before.

Content-Type: text/html
Content-Disposition: attachment;
             filename=SOA-102012059675-DECEMBER-2016.html
Content-Transfer-Encoding: quoted-printable

See how Content-Type its different now also see the Content-Transfer-Encoding its now quoted-printable as opposed to base64 earlier. Additionally I can see HTML content below the header. When I opened the attachment from mail I get the actual bill. As far as I understand base64 encoding is used when the data to be attached is base64. So I guess basically due to wrong encoding declared inside the file Emacs interpreted the content as a binary data and encoded it differently than what it really should be. Phew that was a lot of investigation to understand the magic but it was worth it. Do let me know your thoughts on this.

5 May 2017

Joey Hess: announcing debug-me

Today I'm excited to release debug-me, a program for secure remote debugging.

Debugging a problem over email/irc/BTS is slow, tedious, and hard. The developer needs to see your problem to understand it. Debug-me aims to make debugging fast, fun, and easy, by letting the developer access your computer remotely, so they can immediately see and interact with the problem. Making your problem their problem gets it fixed fast. debug-me session is logged and signed with the developer's GnuPG key, producing a chain of evidence of what they saw and what they did. So the developer's good reputation is leveraged to make debug-me secure.

I've recorded a short screencast demoing debug-me.

And here's a screencast about debug-me's chain of evidence.

The idea for debug-me came from Re: Debugging over email, and then my Patreon supporters picked debug-me in a poll as a project I should work on. It's been a fun month, designing the evidence chain, building a custom efficient protocol with space saving hacks, using websockets and protocol buffers and ed25519 for the first time, and messing around with low-level tty details. The details of debug-me's development are in my devblog. Anyway, I hope debug-me makes debugging less of a tedious back and forth, at least some of the time. PS: Since debug-me's protocol lets multiple people view the same shell session, and optionally interact with it, there could be uses for it beyond debugging, including live screencasting, pair programming, etc. PPS: There needs to be a debug-me server not run by me, so someone please run one..

Daniel Silverstone: Yarn architecture discussion

Recently Rob and I visited Soile and Lars. We had a lovely time wandering around Helsinki with them, and I also spent a good chunk of time with Lars working on some design and planning for the Yarn test specification and tooling. You see, I wrote a Rust implementation of Yarn called rsyarn "for fun" and in doing so I noted a bunch of missing bits in the understanding Lars and I shared about how Yarn should work. Lars and I filled, and re-filled, a whiteboard with discussion about what the 'Yarn specification' should be, about various language extensions and changes, and also about what functionality a normative implementation of Yarn should have. This article is meant to be a write-up of all of that discussion, but before I start on that, I should probably summarise what Yarn is.

Yarn is a mechanism for specifying tests in a form which is more like documentation than code. Yarn follows the concept of BDD story based design/testing and has a very Cucumberish scenario language in which to write tests. Yarn takes, as input, Markdown documents which contain code blocks with Yarn tests in them; and it then runs those tests and reports on the scenario failures/successes. As an example of a poorly written but still fairly effective Yarn suite, you could look at Gitano's tests or perhaps at Obnam's tests (rendered as HTML). Yarn is not trying to replace unit testing, nor other forms of testing, but rather seeks to be one of a suite of test tools used to help validate software and to verify integrations. Lars writes Yarns which test his server setups for example. As an example, lets look at what a simple test might be for the behaviour of the /bin/true tool:

SCENARIO true should exit with code zero
WHEN /bin/true is run with no arguments
THEN the exit code is 0
 AND stdout is empty
 AND stderr is empty

Anyone ought to be able to understand exactly what that test is doing, even though there's no obvious code to run. Yarn statements are meant to be easily grokked by both developers and managers. This should be so that managers can understand the tests which verify that requirements are being met, without needing to grok python, shell, C, or whatever else is needed to implement the test where the Yarns meet the metal. Obviously, there needs to be a way to join the dots, and Yarn calls those things IMPLEMENTS, for example:

IMPLEMENTS WHEN (\S+) is run with no arguments
set +e
"$ MATCH_1 " > "$ DATADIR /stdout" 2> "$ DATADIR /stderr"
echo $? > "$ DATADIR /exitcode"

As you can see from the example, Yarn IMPLEMENTS can use regular expressions to capture parts of their invocation, allowing the test implementer to handle many different scenario statements with one implementation block. For the rest of the implementation, whatever you assume about things will probably be okay for now.

Given all of the above, we (Lars and I) decided that it would make a lot of sense if there was a set of Yarn scenarios which could validate a Yarn implementation. Such a document could also form the basis of a Yarn specification and also a manual for writing reasonable Yarn scenarios. As such, we wrote up a three-column approach to what we'd need in that test suite. Firstly we considered what the core features of the Yarn language are:

Scenario statements themselves (SCENARIO, GIVEN, WHEN, THEN, ASSUMING, FINALLY, AND, IMPLEMENTS, EXAMPLE, ...)
Whitespace normalisation of statements
Regexp language and behaviour
IMPLEMENTS current directory, data directory, home directory, and also environment.
Error handling for the statements, or for missing IMPLEMENTS
File (and filename) encoding
Labelled code blocks (since commonmark includes the backtick code block kind)
Exactly one IMPLEMENTS per statement

We considered unusual (or corner) cases and which of them needed defining in the short to medium term:

Statements before any SCENARIO or IMPLEMENTS
Meaning of split code blocks (concatenation?)
Meaning of code blocks not at the top level of a file (ignore?)
Meaning of HTML style comments in markdown files
Odd scenario ordering (e.g. ASSUMING at the end, or FINALLY at the start)
Meaning of empty lines in code blocks or between them.

All of this comes down to how to interpret input to a Yarn implementation. In addition there were a number of things we felt any "normative" Yarn implementation would have to handle or provide in order to be considered useful. It's worth noting that we don't specify anything about an implementation being a command line tool though...

Interpreter for IMPLEMENTS (and arguments for them)
"Library" for those implementations
Ability to require that failed ASSUMING statements lead to an error
A way to 'stop on first failure'
A way to select a specific scenario to run, from a large suite.
Generation of timing reports (per scenario and also per statement)
A way to 'skip' missing IMPLEMENTS
A clear way to identify the failing step in a scenario.
Able to treat multiple input files as a single suite.

There's bound to be more, but right now with the above, we believe we have two roughly conformant Yarn implementations. Lars' Python based implementation which lives in cmdtest (and which I shall refer to as pyyarn for now) and my Rust based one (rsyarn).

One thing which rsyarn supports, but pyyarn does not, is running multiple scenarios in parallel. However when I wrote that support into rsyarn I noticed that there were plenty of issues with running stuff in parallel. (A problem I'm sure any of you who know about threads will appreciate). One particular issue was that scenarios often need to share resources which are not easily sandboxed into the $ DATADIR provided by Yarn. For example databases or access to limited online services. Lars and I had a good chat about that, and decided that a reasonable language extension could be:

USING database foo

with its counterpart

RESOURCE database (\S+)
LABEL database-$1
GIVEN a database called $1
FINALLY database $1 is torn down

The USING statement should be reasonably clear in its pairing to a RESOURCE statement. The LABEL statement I'll get to in a moment (though it's only relevant in a RESOURCE block, and the rest of the statements are essentially substituted into the calling scenario at the point of the USING. This is nowhere near ready to consider adding to the specification though. Both Lars and I are uncomfortable with the $1 syntax though we can't think of anything nicer right now; and the USING/RESOURCE/LABEL vocabulary isn't set in stone either. The idea of the LABEL is that we'd also require that a normative Yarn implementation be capable of specifying resource limits by name. E.g. if a RESOURCE used a LABEL foo then the caller of a Yarn scenario suite could specify that there were 5 foos available. The Yarn implementation would then schedule a maximum of 5 scenarios which are using that label to happen simultaneously. At bare minimum it'd gate new users, but at best it would intelligently schedule them. In addition, since this introduces the concept of parallelism into Yarn proper, we also wanted to add a maximum parallelism setting to the Yarn implementation requirements; and to specify that any resource label which was not explicitly set had a usage limit of 1.

Once we'd discussed the parallelism, we decided that once we had a nice syntax for expanding these sets of statements anyway, we may as well have a syntax for specifying scenario language expansions which could be used to provide something akin to macros for Yarn scenarios. What we came up with as a starter-for-ten was:

CALLING write foo

paired with

EXPANDING write (\S+)
GIVEN bar
WHEN $1 is written to
THEN success was had by all

Again, the CALLING/EXPANDING keywords are not fixed yet, nor is the $1 type syntax, though whatever is used here should match the other places where we might want it.

Finally we discussed multi-line inputs in Yarn. We currently have a syntax akin to:

GIVEN foo
... bar
... baz

which is directly equivalent to:

GIVEN foo bar baz

and this is achieved by collapsing the multiple lines and using the whitespace normalisation functionality of Yarn to replace all whitespace sequences with single space characters. However this means that, for example, injecting chunks of YAML into a Yarn scenario is a pain, as would be including any amount of another whitespace-sensitive input language. After a lot of to-ing and fro-ing, we decided that the right thing to do would be to redefine the ... Yarn statement to be whitespace preserving and to then pass that whitespace through to be matched by the IMPLEMENTS or whatever. In order for that to work, the regexp matching would have to be defined to treat the input as a single line, allowing . to match \n etc. Of course, this would mean that the old functionality wouldn't be possible, so we considered allowing a \ at the end of a line to provide the current kind of behaviour, rewriting the above example as:

GIVEN foo \
bar \
baz

It's not as nice, but since we couldn't find any real uses of ... in any of our Yarn suites where having the whitespace preserved would be an issue, we decided it was worth the pain.

None of the above is, as of yet, set in stone. This blog posting is about me recording the information so that it can be referred to; and also to hopefully spark a little bit of discussion about Yarn. We'd welcome emails to our usual addresses, being poked on Twitter, or on IRC in the common spots we can be found. If you're honestly unsure of how to get hold of us, just comment on this blog post and I'll find your message eventually. Hopefully soon we can start writing that Yarn suite which can be used to validate the behaviour of pyyarn and rsyarn and from there we can implement our new proposals for extending Yarn to be even more useful.

Arturo Borrero Gonz lez: New in Debian stable Stretch: nftables

Debian Stretch stable includes the nftables framework, ready to use. Created by the Netfilter project itself, nftables is the firewalling tool that replaces the old iptables, giving the users a powerful tool. Back in October 2016, I wrote a small post about the status of ntables in Debian Stretch. Since then, several things have improved even further, so this clearly deserves a new small post :-) Yes, nftables replaces iptables. You are highly encouraged to migrate from iptables to nftables. The version of nftables in Debian stable Stretch is v0.7, and the kernel couterpart is v4.9. This is clearly a very recent release of both components. In the case of nftables, is the last released version by the time of this writting. Also, after the Debian stable release, both kernel and nftables will likely get backports of future releases. Yes, you will be able to easily run a newer release of the framework after the stable release. In case you are migrating from iptables, you should know that there are some tools in place to help you in this task. Please read the official netfilter docs: Moving from iptables to nftables. By the way, the nftables docs are extensive, check the whole wiki. In case you don t know about nftables yet, here is a quick reference:

it s the tool/framework that replaces iptables (also ip6tables, arptables and ebtables)
it integrates advanced structures which allow to arrange your ruleset for optimal performance
all the system is more configurable than in iptables
the syntax is much better than in iptables
several actions in a single rule
simplified IPv4/IPv6 dual stack
less kernel updates required
great support for incremental, dynamic and atomic ruleset updates

To run nftables in Debian Stretch you need several components:

nft: the command line interface
libnftnl: the nftables-netlink library
linux kernel: a least 4.9 is recommended

A simple aptitude run will put your system ready to go, out of the box, with nftables:

root@debian:~# aptitude install nftables

Once installed, you can start using the nft command:

root@debian:~# nft list ruleset

A good starting point is to copy a simple workstation firewall configuration:

root@debian:~# cp /usr/share/doc/nftables/examples/syntax/workstation /etc/nftables.conf

And load it:

root@debian:~# nft -f /etc/nftables.conf

Your nftables ruleset is now firewalling your network:

root@debian:~# nft list ruleset
table inet filter  
        chain input  
                type filter hook input priority 0;
                iif lo accept
                ct state established,related accept
                ip6 nexthdr icmpv6 icmpv6 type   nd-neighbor-solicit,  nd-router-advert, nd-neighbor-advert   accept
                counter drop

Several examples can be found at /usr/share/doc/nftables/examples/. A simple systemd service is included to load your ruleset at boot time, which is disabled by default. nft

Did you know that the nano editor includes nft syntax highlighting? Starting with Debian stable Stretch and nftables, packet filtering and network policing will never be the same.

3 May 2017

Vincent Bernat: VXLAN: BGP EVPN with Cumulus Quagga

VXLAN is an overlay network to encapsulate Ethernet traffic over an existing (highly available and scalable, possibly the Internet) IP network while accomodating a very large number of tenants. It is defined in RFC 7348. For an uncut introduction on its use with Linux, have a look at my VXLAN & Linux post. VXLAN deployment

In the above example, we have hypervisors hosting a virtual machines from different tenants. Each virtual machine is given access to a tenant-specific virtual Ethernet segment. Users are expecting classic Ethernet segments: no MAC restrictions¹, total control over the IP addressing scheme they use and availability of multicast.

In a large VXLAN deployment, two aspects need attention:

discovery of other endpoints (VTEPs) sharing the same VXLAN segments, and
avoidance of BUM frames (broadcast, unknown unicast and multicast) as they have to be forwarded to all VTEPs.

A typical solution for the first point is using multicast. For the second point, this is source-address learning.

Introduction to BGP EVPN BGP EVPN (RFC 7432 and draft-ietf-bess-evpn-overlay for its application to VXLAN) is a standard control protocol to efficiently solves those two aspects without relying on multicast nor source-address learning. BGP EVPN relies on BGP (RFC 4271) and its MP-BGP extensions (RFC 4760). BGP is the routing protocol powering the Internet. It is highly scalable and interoperable. It is also extensible and one of its extension is MP-BGP. This extension can carry reachability information (NLRI) for multiple protocols (IPv4, IPv6, L3VPN and in our case EVPN). EVPN is a special family to advertise MAC addresses and the remote equipments they are attached to. There are basically two kinds of reachability information a VTEP sends through BGP EVPN:

the VNIs they have interest in (type 3 routes), and

for each VNI, the local MAC addresses (type 2 routes).

The protocol also covers other aspects of virtual Ethernet segments (L3 reachability information from ARP/ND caches, MAC mobility and multi-homing²) but we won t describe them here. To deploy BGP EVPN, a typical solution is to use several route reflectors (both for redundancy and scalability), like in the picture below. Each VTEP opens a BGP session to at least two route reflectors, sends its information (MACs and VNIs) and receives others . This reduces the number of BGP sessions to configure. Compared to other solutions to deploy VXLAN, BGP EVPN has three main advantages:

interoperability with other vendors (notably Juniper and Cisco),

proven scalability (a typical BGP routers handle several millions of routes), and

possibility to enforce fine-grained policies.

On Linux, Cumulus Quagga is a fairly complete implementation of BGP EVPN (type 3 routes for VTEP discovery, type 2 routes with MAC or IP addresses, MAC mobility when a host changes from one VTEP to another one) which requires very little configuration. This is a fork of Quagga and currently used in Cumulus Linux, a network operating system based on Debian powering switches from various brands. At some point, BGP EVPN support will be contributed back to FRR, a community-maintained fork of Quagga³. It should be noted the BGP EVPN implementation of Cumulus Quagga currently only supports IPv4.

Route reflector setup Before configuring each VTEP, we need to configure two or more route reflectors. There are many solutions. I will present three of them:

using Cumulus Quagga,

using GoBGP, an implementation of BGP in Go,

using Juniper JunOS.

For reliability purpose, it s possible (and easy) to use one implementation for some route reflectors and another implementation for the other ones. The proposed configurations are quite minimal. However, it is possible to centralize policies on the route reflectors (e.g. routes tagged with some community can only be readvertised to some group of VTEPs).

Using Quagga The configuration is pretty simple. We suppose the configured route reflector has 203.0.113.254 configured as a loopback IP.

router bgp 65000
  bgp router-id 203.0.113.254
  bgp cluster-id 203.0.113.254
  bgp log-neighbor-changes
  no bgp default ipv4-unicast
  neighbor fabric peer-group
  neighbor fabric remote-as 65000
  neighbor fabric capability extended-nexthop
  neighbor fabric update-source 203.0.113.254
  bgp listen range 203.0.113.0/24 peer-group fabric
  !
  address-family evpn
   neighbor fabric activate
   neighbor fabric route-reflector-client
  exit-address-family
  !
  exit
!

A peer group fabric is defined and we leverage the dynamic neighbor feature of Cumulus Quagga: we don t have to explicitely define each neighbor. Any client from 203.0.113.0/24 and presenting itself as part of AS 65000 can connect. All sent EVPN routes will be accepted and reflected to the other clients. You don t need to run Zebra, the route engine talking with the kernel. Instead, start bgpd with the --no_kernel flag.

Using GoBGP GoBGP is a clean implementation of BGP in Go⁴. It exposes an RPC API for configuration (but accepts a configuration file and comes with a command-line client). It doesn t support dynamic neighbors, so you ll have to use the API, the command-line client or some templating language to automate their declaration. A configuration with only one neighbor is like this:

global:
  config:
    as: 65000
    router-id: 203.0.113.254
    local-address-list:
      - 203.0.113.254
neighbors:
  - config:
      neighbor-address: 203.0.113.1
      peer-as: 65000
    afi-safis:
      - config:
          afi-safi-name: l2vpn-evpn
    route-reflector:
      config:
        route-reflector-client: true
        route-reflector-cluster-id: 203.0.113.254

More neighbors can be added from the command line:

$ gobgp neighbor add 203.0.113.2 as 65000 \
>         route-reflector-client 203.0.113.254 \
>         --address-family evpn

GoBGP won t try to interact with the kernel which is fine as a route reflector.

Using Juniper JunOS A variety of Juniper products can be a BGP route reflector, notably:

the Juniper QFX5100⁵, a top-of-the-rack switch,
the Juniper MX240⁶, a high-range router,
the Juniper vRR⁷, a virtual appliance running JunOS without a data-plane.

The main factor is the CPU and the memory. The QFX5100 is low on memory and won t support large deployments without some additional policing. Here is a configuration similar to the Quagga one:

interfaces  
    lo0  
        unit 0  
            family inet  
                address 203.0.113.254/32;
             
         
     
 
protocols  
    bgp  
        group fabric  
            family evpn  
                signaling  
                    /* Do not try to install EVPN routes */
                    no-install;
                 
             
            type internal;
            cluster 203.0.113.254;
            local-address 203.0.113.254;
            allow 203.0.113.0/24;
         
     
 
routing-options  
    router-id 203.0.113.254;
    autonomous-system 65000;

VTEP setup The next step is to configure each VTEP/hypervisor. Each VXLAN is locally configured using a bridge for local virtual interfaces, like illustrated in the below schema. The bridge is taking care of the local MAC addresses (notably, using source-address learning) and the VXLAN interface takes care of the remote MAC addresses (received with BGP EVPN). Bridged VXLAN device

VXLANs can be provisioned with the following script. Source-address learning is disabled as we will rely solely on BGP EVPN to synchronize FDBs between the hypervisors.

for vni in 100 200; do
    # Create VXLAN interface
    ip link add vxlan$ vni  type vxlan
        id $ vni  \
        dstport 4789 \
        local 203.0.113.2 \
        nolearning
    # Create companion bridge
    brctl addbr br$ vni 
    brctl addif br$ vni  vxlan$ vni 
    brctl stp br$ vni  off
    ip link set up dev br$ vni 
    ip link set up dev vxlan$ vni 
done
# Attach each VM to the appropriate segment
brctl addif br100 vnet10
brctl addif br100 vnet11
brctl addif br200 vnet12

The configuration of Cumulus Quagga is similar to the one used for a route reflector, except we use the advertise-all-vni directive to publish all local VNIs.

router bgp 65000
  bgp router-id 203.0.113.2
  no bgp default ipv4-unicast
  neighbor fabric peer-group
  neighbor fabric remote-as 65000
  neighbor fabric capability extended-nexthop
  neighbor fabric update-source dummy0
  ! BGP sessions with route reflectors
  neighbor 203.0.113.253 peer-group fabric
  neighbor 203.0.113.254 peer-group fabric
  !
  address-family evpn
   neighbor fabric activate
   advertise-all-vni
  exit-address-family
  !
  exit
!

If everything works as expected, the instances sharing the same VNI should be able to ping each other. If IPv6 is enabled on the VMs, the ping command shows if everything is in order:

$ ping -c10 -w1 -t1 ff02::1%eth0
PING ff02::1%eth0(ff02::1%eth0) 56 data bytes
64 bytes from fe80::5254:33ff:fe00:8%eth0: icmp_seq=1 ttl=64 time=0.016 ms
64 bytes from fe80::5254:33ff:fe00:b%eth0: icmp_seq=1 ttl=64 time=4.98 ms (DUP!)
64 bytes from fe80::5254:33ff:fe00:9%eth0: icmp_seq=1 ttl=64 time=4.99 ms (DUP!)
64 bytes from fe80::5254:33ff:fe00:a%eth0: icmp_seq=1 ttl=64 time=4.99 ms (DUP!)
--- ff02::1%eth0 ping statistics ---
1 packets transmitted, 1 received, +3 duplicates, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.016/3.745/4.991/2.152 ms

Verification Step by step, let s check how everything comes together.

Getting VXLAN information from the kernel On each VTEP, Quagga should be able to retrieve the information about configured VXLANs. This can be checked with vtysh:

# show interface vxlan100
Interface vxlan100 is up, line protocol is up
  Link ups:       1    last: 2017/04/29 20:01:33.43
  Link downs:     0    last: (never)
  PTM status: disabled
  vrf: Default-IP-Routing-Table
  index 11 metric 0 mtu 1500
  flags: <UP,BROADCAST,RUNNING,MULTICAST>
  Type: Ethernet
  HWaddr: 62:42:7a:86:44:01
  inet6 fe80::6042:7aff:fe86:4401/64
  Interface Type Vxlan
  VxLAN Id 100
  Access VLAN Id 1
  Master (bridge) ifindex 9 ifp 0x56536e3f3470

The important points are:

the VNI is 100, and
the bridge device was correctly detected.

Quagga should also be able to retrieve information about the local MAC addresses :

# show evpn mac vni 100
Number of MACs (local and remote) known for this VNI: 2
MAC               Type   Intf/Remote VTEP      VLAN
50:54:33:00:00:0a local  eth1.100
50:54:33:00:00:0b local  eth2.100

BGP sessions Each VTEP has to establish a BGP session to the route reflectors. On the VTEP, this can be checked by running vtysh:

# show bgp neighbors 203.0.113.254
BGP neighbor is 203.0.113.254, remote AS 65000, local AS 65000, internal link
 Member of peer-group fabric for session parameters
  BGP version 4, remote router ID 203.0.113.254
  BGP state = Established, up for 00:00:45
  Neighbor capabilities:
    4 Byte AS: advertised and received
    AddPath:
      L2VPN EVPN: RX advertised L2VPN EVPN
    Route refresh: advertised and received(new)
    Address family L2VPN EVPN: advertised and received
    Hostname Capability: advertised
    Graceful Restart Capabilty: advertised
[...]
 For address family: L2VPN EVPN
  fabric peer-group member
  Update group 1, subgroup 1
  Packet Queue length 0
  Community attribute sent to this neighbor(both)
  8 accepted prefixes

  Connections established 1; dropped 0
  Last reset never
Local host: 203.0.113.2, Local port: 37603
Foreign host: 203.0.113.254, Foreign port: 179

The output includes the following information:

the BGP state is Established,
the address family L2VPN EVPN is correctly advertised, and
8 routes are received from this route reflector.

The state of the BGP sessions can also be checked from the route reflectors. With GoBGP, use the following command:

# gobgp neighbor 203.0.113.2
BGP neighbor is 203.0.113.2, remote AS 65000, route-reflector-client
  BGP version 4, remote router ID 203.0.113.2
  BGP state = established, up for 00:04:30
  BGP OutQ = 0, Flops = 0
  Hold time is 9, keepalive interval is 3 seconds
  Configured hold time is 90, keepalive interval is 30 seconds
  Neighbor capabilities:
    multiprotocol:
        l2vpn-evpn:     advertised and received
    route-refresh:      advertised and received
    graceful-restart:   received
    4-octet-as: advertised and received
    add-path:   received
    UnknownCapability(73):      received
    cisco-route-refresh:        received
[...]
  Route statistics:
    Advertised:             8
    Received:               5
    Accepted:               5

With JunOS, use the below command:

> show bgp neighbor 203.0.113.2
Peer: 203.0.113.2+38089 AS 65000 Local: 203.0.113.254+179 AS 65000
  Group: fabric                Routing-Instance: master
  Forwarding routing-instance: master
  Type: Internal    State: Established
  Last State: OpenConfirm   Last Event: RecvKeepAlive
  Last Error: None
  Options: <Preference LocalAddress Cluster AddressFamily Rib-group Refresh>
  Address families configured: evpn
  Local Address: 203.0.113.254 Holdtime: 90 Preference: 170
  NLRI evpn: NoInstallForwarding
  Number of flaps: 0
  Peer ID: 203.0.113.2     Local ID: 203.0.113.254     Active Holdtime: 9
  Keepalive Interval: 3          Group index: 0    Peer index: 2
  I/O Session Thread: bgpio-0 State: Enabled
  BFD: disabled, down
  NLRI for restart configured on peer: evpn
  NLRI advertised by peer: evpn
  NLRI for this session: evpn
  Peer supports Refresh capability (2)
  Stale routes from peer are kept for: 300
  Peer does not support Restarter functionality
  NLRI that restart is negotiated for: evpn
  NLRI of received end-of-rib markers: evpn
  NLRI of all end-of-rib markers sent: evpn
  Peer does not support LLGR Restarter or Receiver functionality
  Peer supports 4 byte AS extension (peer-as 65000)
  NLRI's for which peer can receive multiple paths: evpn
  Table bgp.evpn.0 Bit: 20000
    RIB State: BGP restart is complete
    RIB State: VPN restart is complete
    Send state: in sync
    Active prefixes:              5
    Received prefixes:            5
    Accepted prefixes:            5
    Suppressed due to damping:    0
    Advertised prefixes:          8
  Last traffic (seconds): Received 276  Sent 170  Checked 276
  Input messages:  Total 61     Updates 3       Refreshes 0     Octets 1470
  Output messages: Total 62     Updates 4       Refreshes 0     Octets 1775
  Output Queue[1]: 0            (bgp.evpn.0, evpn)

If a BGP session cannot be established, the logs of each BGP daemon should mention the cause.

Sent routes From each VTEP, Quagga needs to send:

one type 3 route for each local VNI, and
one type 2 route for each local MAC address.

The best place to check the received routes is on one of the route reflectors. If you are using JunOS, the following command will display the received routes from the provided VTEP:

> show route table bgp.evpn.0 receive-protocol bgp 203.0.113.2
bgp.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  2:203.0.113.2:100::0::50:54:33:00:00:0a/304 MAC/IP
*                         203.0.113.2                  100        I
  2:203.0.113.2:100::0::50:54:33:00:00:0b/304 MAC/IP
*                         203.0.113.2                  100        I
  3:203.0.113.2:100::0::203.0.113.2/304 IM
*                         203.0.113.2                  100        I
  3:203.0.113.2:200::0::203.0.113.2/304 IM
*                         203.0.113.2                  100        I

There is one type 3 route for VNI 100 and another one for VNI 200. There are also two type 2 routes for two MAC addresses on VNI 100. To get more information, you can add the keyword extensive. Here is a type 3 route advertising 203.0.113.2 as a VTEP for VNI 100⁸:

> show route table bgp.evpn.0 receive-protocol bgp 203.0.113.2 extensive
bgp.evpn.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
* 3:203.0.113.2:100::0::203.0.113.2/304 IM (1 entry, 1 announced)
     Accepted
     Route Distinguisher: 203.0.113.2:100
     Nexthop: 203.0.113.2
     Localpref: 100
     AS path: I
     Communities: target:65000:268435556 encapsulation:vxlan(0x8)
[...]

Here is a type 2 route announcing the location of the 50:54:33:00:00:0a MAC address for VNI 100:

> show route table bgp.evpn.0 receive-protocol bgp 203.0.113.2 extensive
bgp.evpn.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
* 2:203.0.113.2:100::0::50:54:33:00:00:0a/304 MAC/IP (1 entry, 1 announced)
     Accepted
     Route Distinguisher: 203.0.113.2:100
     Route Label: 100
     ESI: 00:00:00:00:00:00:00:00:00:00
     Nexthop: 203.0.113.2
     Localpref: 100
     AS path: I
     Communities: target:65000:268435556 encapsulation:vxlan(0x8)
[...]

With Quagga, you can get a similar output with vtysh:

# show bgp evpn route
BGP table version is 0, local router ID is 203.0.113.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 203.0.113.2:100
*>i[2]:[0]:[0]:[48]:[50:54:33:00:00:0a]
                    203.0.113.2                   100      0 i
*>i[2]:[0]:[0]:[48]:[50:54:33:00:00:0b]
                    203.0.113.2                   100      0 i
*>i[3]:[0]:[32]:[203.0.113.2]
                    203.0.113.2                   100      0 i
Route Distinguisher: 203.0.113.2:200
*>i[3]:[0]:[32]:[203.0.113.2]
                    203.0.113.2                   100      0 i
[...]

With GoBGP, use the following command:

# gobgp global rib -a evpn   grep rd:203.0.113.2:200
    Network  Next Hop             AS_PATH              Age        Attrs
*>  [type:macadv][rd:203.0.113.2:100][esi:single-homed][etag:0][mac:50:54:33:00:00:0a][ip:<nil>][labels:[100]]203.0.113.2                               00:00:17   [ Origin: i   LocalPref: 100   Extcomms: [VXLAN], [65000:268435556] ]
*>  [type:macadv][rd:203.0.113.2:100][esi:single-homed][etag:0][mac:50:54:33:00:00:0b][ip:<nil>][labels:[100]]203.0.113.2                               00:00:17   [ Origin: i   LocalPref: 100   Extcomms: [VXLAN], [65000:268435556] ]
*>  [type:macadv][rd:203.0.113.2:200][esi:single-homed][etag:0][mac:50:54:33:00:00:0a][ip:<nil>][labels:[200]]203.0.113.2                               00:00:17   [ Origin: i   LocalPref: 100   Extcomms: [VXLAN], [65000:268435656] ]
*>  [type:multicast][rd:203.0.113.2:100][etag:0][ip:203.0.113.2]203.0.113.2                               00:00:17   [ Origin: i   LocalPref: 100   Extcomms: [VXLAN], [65000:268435556] ]
*>  [type:multicast][rd:203.0.113.2:200][etag:0][ip:203.0.113.2]203.0.113.2                               00:00:17   [ Origin: i   LocalPref: 100   Extcomms: [VXLAN], [65000:268435656] ]

Received routes Each VTEP should have received the type 2 and type 3 routes from its fellow VTEPs, through the route reflectors. You can check with the show bgp evpn route command of vtysh. Does Quagga correctly understand the received routes? The type 3 routes are translated to an assocation between the remote VTEPs and the VNIs:

# show evpn vni
Number of VNIs: 2
VNI        VxLAN IF              VTEP IP         # MACs   # ARPs   Remote VTEPs
100        vxlan100              203.0.113.2     4        0        203.0.113.3
                                                                   203.0.113.1
200        vxlan200              203.0.113.2     3        0        203.0.113.3
                                                                   203.0.113.1

The type 2 routes are translated to an association between the remote MACs and the remote VTEPs:

# show evpn mac vni 100
Number of MACs (local and remote) known for this VNI: 4
MAC               Type   Intf/Remote VTEP      VLAN
50:54:33:00:00:09 remote 203.0.113.1
50:54:33:00:00:0a local  eth1.100
50:54:33:00:00:0b local  eth2.100
50:54:33:00:00:0c remote 203.0.113.3

FDB configuration The last step is to ensure Quagga has correctly provided the received information to the kernel. This can be checked with the bridge command:

# bridge fdb show dev vxlan100   grep dst
00:00:00:00:00:00 dst 203.0.113.1 self permanent
00:00:00:00:00:00 dst 203.0.113.3 self permanent
50:54:33:00:00:0c dst 203.0.113.3 self
50:54:33:00:00:09 dst 203.0.113.1 self

All good! The two first lines are the translation of the type 3 routes (any BUM frame will be sent to both 203.0.113.1 and 203.0.113.3) and the two last ones are the translation of the type 2 routes.

Interoperability One of the strength of BGP EVPN is the interoperability with other network vendors. To demonstrate it works as expected, we will configure a Juniper vMX to act as a VTEP. First, we need to configure the physical bridge⁹. This is similar to the use of ip link and brctl with Linux. We only configure one physical interface with two old-school VLANs paired with matching VNIs.

interfaces  
    ge-0/0/1  
        unit 0  
            family bridge  
                interface-mode trunk;
                vlan-id-list [ 100 200 ];
             
         
     
 
routing-instances  
    switch  
        instance-type virtual-switch;
        interface ge-0/0/1.0;
        bridge-domains  
            vlan100  
                domain-type bridge;
                vlan-id 100;
                vxlan  
                    vni 100;
                    ingress-node-replication;
                 
             
            vlan200  
                domain-type bridge;
                vlan-id 200;
                vxlan  
                    vni 200;
                    ingress-node-replication;

Then, we configure BGP EVPN to advertise all known VNIs. The configuration is quite similar to the one we did with Quagga:

protocols  
    bgp  
        group fabric  
            type internal;
            multihop;
            family evpn signaling;
            local-address 203.0.113.3;
            neighbor 203.0.113.253;
            neighbor 203.0.113.254;
         
     
 
routing-instances  
    switch  
        vtep-source-interface lo0.0;
        route-distinguisher 203.0.113.3:1; #  
        vrf-import EVPN-VRF-VXLAN;
        vrf-target  
            target:65000:1;
            auto;
         
        protocols  
            evpn  
                encapsulation vxlan;
                extended-vni-list all;
                multicast-mode ingress-replication;
             
         
     
 
routing-options  
    router-id 203.0.113.3;
    autonomous-system 65000;
 
policy-options  
    policy-statement EVPN-VRF-VXLAN  
        then accept;

We also need a small compatibility patch for Cumulus Quagga¹⁰. The routes sent by this configuration are very similar to the routes sent by Quagga. The main differences are:

on JunOS, the route distinguisher is configured statically (in ), and
on JunOS, the VNI is also encoded as an Ethernet tag ID.

Here is a type 3 route, as sent by JunOS:

> show route table bgp.evpn.0 receive-protocol bgp 203.0.113.3 extensive
bgp.evpn.0: 13 destinations, 13 routes (13 active, 0 holddown, 0 hidden)
* 3:203.0.113.3:1::100::203.0.113.3/304 IM (1 entry, 1 announced)
     Accepted
     Route Distinguisher: 203.0.113.3:1
     Nexthop: 203.0.113.3
     Localpref: 100
     AS path: I
     Communities: target:65000:268435556 encapsulation:vxlan(0x8)
     PMSI: Flags 0x0: Label 6: Type INGRESS-REPLICATION 203.0.113.3
[...]

Here is a type 2 route:

> show route table bgp.evpn.0 receive-protocol bgp 203.0.113.3 extensive
bgp.evpn.0: 13 destinations, 13 routes (13 active, 0 holddown, 0 hidden)
* 2:203.0.113.3:1::200::50:54:33:00:00:0f/304 MAC/IP (1 entry, 1 announced)
     Accepted
     Route Distinguisher: 203.0.113.3:1
     Route Label: 200
     ESI: 00:00:00:00:00:00:00:00:00:00
     Nexthop: 203.0.113.3
     Localpref: 100
     AS path: I
     Communities: target:65000:268435656 encapsulation:vxlan(0x8)
[...]

We can check that the vMX is able to make sense of the routes it receives from its peers running Quagga:

> show evpn database l2-domain-id 100
Instance: switch
VLAN  DomainId  MAC address        Active source                  Timestamp        IP address
     100        50:54:33:00:00:0c  203.0.113.1                    Apr 30 12:46:20
     100        50:54:33:00:00:0d  203.0.113.2                    Apr 30 12:32:42
     100        50:54:33:00:00:0e  203.0.113.2                    Apr 30 12:46:20
     100        50:54:33:00:00:0f  ge-0/0/1.0                     Apr 30 12:45:55

On the other end, if we look at one of the Quagga-based VTEP, we can check the received routes are correctly understood:

# show evpn vni 100
VNI: 100
 VxLAN interface: vxlan100 ifIndex: 9 VTEP IP: 203.0.113.1
 Remote VTEPs for this VNI:
  203.0.113.3
  203.0.113.2
 Number of MACs (local and remote) known for this VNI: 4
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
# show evpn mac vni 100
Number of MACs (local and remote) known for this VNI: 4
MAC               Type   Intf/Remote VTEP      VLAN
50:54:33:00:00:0c local  eth1.100
50:54:33:00:00:0d remote 203.0.113.2
50:54:33:00:00:0e remote 203.0.113.2
50:54:33:00:00:0f remote 203.0.113.3

Get in touch if you have some success with other vendors!

For example, they may use bridges to connect containers together.
Such a feature can replace proprietary implementations of MC-LAG allowing several VTEPs to act as a endpoint for a single link aggregation group. This is not needed on our scenario where hypervisors act as VTEPs.
The development of Quagga is slow and closed . New features are often stalled. FRR is placed under the umbrella of the Linux Foundation, has a GitHub-centered development model and an election process. It already has several interesting enhancements (notably, BGP add-path, BGP unnumbered, MPLS and LDP).
I am unenthusiastic about projects whose the sole purpose is to rewrite something in Go. However, while being quite young, GoBGP is quite valuable on its own (good architecture, good performance).
The 48-port version is around $10,000 with the BGP license.
An empty chassis with a dual routing engine (RE-S-1800X4-16G) is around $30,000.
I don t know how pricey the vRR is. For evaluation purposes, it can be downloaded for free if you are a customer.
The value 100 used in the route distinguishier (203.0.113.2:100) is not the one used to encode the VNI. The VNI is encoded in the route target (65000:268435556), in the 24 least signifiant bits (268435556 & 0xffffff equals 100). As long as VNIs are unique, we don t have to understand those details.
For some reason, the use of a virtual switch is mandatory. This is specific to this platform: a QFX doesn t require this.
The encoding of the VNI into the route target is being standardized in draft-ietf-bess-evpn-overlay. Juniper already implements this draft.

Vincent Bernat: VXLAN & Linux

VXLAN is an overlay network to carry Ethernet traffic over an existing (highly available and scalable) IP network while accommodating a very large number of tenants. It is defined in RFC 7348. Starting from Linux 3.12, the VXLAN implementation is quite complete as both multicast and unicast are supported as well as IPv6 and IPv4. Let s explore the various methods to configure it. VXLAN setup

To illustrate our examples, we use the following setup:

an underlay IP network (highly available and scalable, possibly the Internet),
three Linux bridges acting as VXLAN tunnel endpoints (VTEP),
four servers believing they share a common Ethernet segment.

A VXLAN tunnel extends the individual Ethernet segments accross the three bridges, providing a unique (virtual) Ethernet segment. From one host (e.g. H1), we can reach directly all the other hosts in the virtual segment:

$ ping -c10 -w1 -t1 ff02::1%eth0
PING ff02::1%eth0(ff02::1%eth0) 56 data bytes
64 bytes from fe80::5254:33ff:fe00:8%eth0: icmp_seq=1 ttl=64 time=0.016 ms
64 bytes from fe80::5254:33ff:fe00:b%eth0: icmp_seq=1 ttl=64 time=4.98 ms (DUP!)
64 bytes from fe80::5254:33ff:fe00:9%eth0: icmp_seq=1 ttl=64 time=4.99 ms (DUP!)
64 bytes from fe80::5254:33ff:fe00:a%eth0: icmp_seq=1 ttl=64 time=4.99 ms (DUP!)
--- ff02::1%eth0 ping statistics ---
1 packets transmitted, 1 received, +3 duplicates, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.016/3.745/4.991/2.152 ms

Basic usage The reference deployment for VXLAN is to use an IP multicast group to join the other VTEPs:

# ip -6 link add vxlan100 type vxlan \
>   id 100 \
>   dstport 4789 \
>   local 2001:db8:1::1 \
>   group ff05::100 \
>   dev eth0 \
>   ttl 5
# brctl addbr br100
# brctl addif br100 vxlan100
# brctl addif br100 vnet22
# brctl addif br100 vnet25
# brctl stp br100 off
# ip link set up dev br100
# ip link set up dev vxlan100

The above commands create a new interface acting as a VXLAN tunnel endpoint, named vxlan100 and put it in a bridge with some regular interfaces¹. Each VXLAN segment is associated to a 24-bit segment ID, the VXLAN Network Identifier (VNI). In our example, the default VNI is specified with id 100. When VXLAN was first implemented in Linux 3.7, the UDP port to use was not defined. Several vendors were using 8472 and Linux took the same value. To avoid breaking existing deployments, this is still the default value. Therefore, if you want to use the IANA-assigned port, you need to explicitely set it with dstport 4789. As we want to use multicast, we have to specify a multicast group to join (group ff05::100), as well as a physical device (

dev
eth0

). With multicast, the default TTL is 1. If your multicast network leverages some routing, you ll have to increase the value a bit, like here with ttl 5. The vxlan100 device acts as a bridge device with remote VTEPs as virtual ports:

it sends broadcast, unknown unicast and multicast (BUM) frames to all VTEPs using the multicast group, and
it discovers the association from Ethernet MAC addresses to VTEP IP addresses using source-address learning.

The following figure summarizes the configuration, with the FDB of the Linux bridge (learning local MAC addresses) and the FDB of the VXLAN device (learning distant MAC addresses): Bridged VXLAN device

The FDB of the VXLAN device can be observed with the bridge command. If the destination MAC is present, the frame is sent to the associated VTEP (unicast). The all-zero address is only used when a lookup for the destination MAC fails.

# bridge fdb show dev vxlan100   grep dst
00:00:00:00:00:00 dst ff05::100 via eth0 self permanent
50:54:33:00:00:0b dst 2001:db8:3::1 self
50:54:33:00:00:08 dst 2001:db8:1::1 self

If you are interested to get more details on how to setup a multicast network and build VXLAN segments on top of it, see my Network virtualization with VXLAN article.

Without multicast Using VXLAN over a multicast IP network has several benefits:

automatic discovery of other VTEPs sharing the same multicast group,

good bandwidth usage (packets are replicated as late as possible),

decentralized and controller-less design².

However, multicast is not available everywhere and managing it at scale can be difficult. In Linux 3.8, the DOVE extensions have been added to the VXLAN implementation, removing the dependency on multicast.

Unicast with static flooding We can replace multicast by head-end replication of BUM frames to a statically configured lists of remote VTEPs³:

# ip -6 link add vxlan100 type vxlan \
>   id 100 \
>   dstport 4789 \
>   local 2001:db8:1::1
# bridge fdb append 00:00:00:00:00:00 dev vxlan100 dst 2001:db8:2::1
# bridge fdb append 00:00:00:00:00:00 dev vxlan100 dst 2001:db8:3::1

The VXLAN is defined without a remote multicast group. Instead, all the remote VTEPs are associated with the all-zero address: a BUM frame will be duplicated to all those destinations. The VXLAN device will still learn remote addresses automatically using source-address learning. It is a very simple solution. With a bit of automation, you can keep the default FDB entries up-to-date easily. However, the host will have to duplicate each BUM frame (head-end replication) as many times as there are remote VTEPs. This is quite reasonable if you have a dozen of them. This may become out-of-hand if you have thousands of them. Cumulus vxfld daemon is an example of use of this strategy (in the head-end replication mode).

Unicast with static L2 entries When the associations of MAC addresses and VTEPs are known, it is possible to pre-populate the FDB and disable learning:

# ip -6 link add vxlan100 type vxlan \
>   id 100 \
>   dstport 4789 \
>   local 2001:db8:1::1 \
>   nolearning
# bridge fdb append 00:00:00:00:00:00 dev vxlan100 dst 2001:db8:2::1
# bridge fdb append 00:00:00:00:00:00 dev vxlan100 dst 2001:db8:3::1
# bridge fdb append 50:54:33:00:00:09 dev vxlan100 dst 2001:db8:2::1
# bridge fdb append 50:54:33:00:00:0a dev vxlan100 dst 2001:db8:2::1
# bridge fdb append 50:54:33:00:00:0b dev vxlan100 dst 2001:db8:3::1

Thanks to the nolearning flag, source-address learning is disabled. Therefore, if a MAC is missing, the frame will always be sent using the all-zero entries. The all-zero entries are still needed for broadcast and multicast traffic (e.g. ARP and IPv6 neighbor discovery). This kind of setup works well to provide virtual L2 networks to virtual machines (no L3 information available). You need some glue to update the FDB entries. BGP EVPN with Cumulus Quagga is an example of use of this strategy (see VXLAN: BGP EVPN with Cumulus Quagga for additional information).

Unicast with static L3 entries In the previous example, we had to keep the all-zero entries for ARP and IPv6 neighbor discovery to work correctly. However, Linux can answer to neighbor requests on behalf of the remote nodes⁴. When this feature is enabled, the default entries are not needed anymore (but you could keep them):

# ip -6 link add vxlan100 type vxlan \
>   id 100 \
>   dstport 4789 \
>   local 2001:db8:1::1 \
>   nolearning \
>   proxy
# ip -6 neigh add 2001:db8:ff::11 lladdr 50:54:33:00:00:09 dev vxlan100
# ip -6 neigh add 2001:db8:ff::12 lladdr 50:54:33:00:00:0a dev vxlan100
# ip -6 neigh add 2001:db8:ff::13 lladdr 50:54:33:00:00:0b dev vxlan100
# bridge fdb append 50:54:33:00:00:09 dev vxlan100 dst 2001:db8:2::1
# bridge fdb append 50:54:33:00:00:0a dev vxlan100 dst 2001:db8:2::1
# bridge fdb append 50:54:33:00:00:0b dev vxlan100 dst 2001:db8:3::1

This setup totally eliminates head-end replication. However, protocols relying on multicast won t work either. With some automation, this is a setup that should work well with containers: if there is a registry keeping a list of all IP and MAC addresses in use, a program could listen to it and adjust the FDB and the neighbor tables. The VXLAN backend of Docker s libnetwork is an example of use of this strategy (but it also uses the next method).

Unicast with dynamic L3 entries Linux can also notify a program an (L2 or L3) entry is missing. The program queries some central registry and dynamically adds the requested entry. However, for L2 entries, notifications are issued only if:

the destination MAC address is not known,
there is no all-zero entry in the FDB, and
the destination MAC address is not a multicast or broadcast one.

Those limitations prevent us to do a unicast with dynamic L2 entries scenario. First, let s create the VXLAN device with the l2miss and l3miss options⁵:

ip -6 link add vxlan100 type vxlan \
   id 100 \
   dstport 4789 \
   local 2001:db8:1::1 \
   nolearning \
   l2miss \
   l3miss \
   proxy

Notifications are sent to programs listening to an AF_NETLINK socket using the NETLINK_ROUTE protocol. This socket needs to be bound to the RTNLGRP_NEIGH group. The following is doing exactly that and decodes the received notifications:

# ip monitor neigh dev vxlan100
miss 2001:db8:ff::12 STALE
miss lladdr 50:54:33:00:00:0a STALE

The first notification is about a missing neighbor entry for the requested IP address. We can add it with the following command:

ip -6 neigh replace 2001:db8:ff::12 \
    lladdr 50:54:33:00:00:0a \
    dev vxlan100 \
    nud reachable

The entry is not permanent so that we don t need to delete it when it expires. If the address becomes stale, we will get another notification to refresh it. Once the host receives our proxy answer for the neighbor discovery request, it can send a frame with the MAC we gave as destination. The second notification is about the missing FDB entry for this MAC address. We add the appropriate entry with the following command⁶:

bridge fdb replace 50:54:33:00:00:0a \
    dst 2001:db8:2::1 \
    dev vxlan100 dynamic

The entry is not permanent either as it would prevent the MAC to migrate to the local VTEP (a dynamic entry cannot override a permanent entry). This setup works well with containers and a global registry. However, there is small latency penalty for the first connections. Moreover, multicast and broadcast won t be available in the underlay network. The VXLAN backend for flannel, a network fabric for Kubernetes, is an example of this strategy.

Decision There is no one-size-fits-all solution. You should consider the multicast solution if:

you are in an environment where multicast is available,

you are ready to operate (and scale) a multicast network,

you need multicast and broadcast inside the virtual segments,

you don t have L2/L3 addresses available beforehand.

The scalability of such a solution is pretty good if you take care of not putting all VXLAN interfaces into the same multicast group (e.g. use the last byte of the VNI as the last byte of the multicast group). When multicast is not available, another generic solution is BGP EVPN: BGP is used as a controller to ensure distribution of the list of VTEPs and their respective FDBs. As mentioned earlier, an implementation of this solution is Cumulus Quagga. I explore this option in a separate post: VXLAN: BGP EVPN with Cumulus Quagga. If you operate in a container-like environment where L2/L3 addresses are known beforehand, a solution using static and/or dynamic L2 and L3 entries based on a central registry and no source-address learning would also fit the bill. This provides a more security-tight solution (bound resources, MiTM attacks dampened down, inability to amplify bandwidth usage through excessive broadcast). Various environment-specific solutions are available⁷ or you can build your own.

Other considerations Independently of the chosen strategy, here are a few important points to keep in mind when implementing a VXLAN overlay.

Isolation While you may expect VXLAN interfaces to only carry L2 traffic, Linux doesn t disable IP processing. If the destination MAC is a local one, Linux will route or deliver the encapsulated IP packet. Check my post about the proper isolation of a Linux bridge.

Encryption VXLAN enforces isolation between tenants, but the traffic is totally unencrypted. The most direct solution to provide encryption is to use IPsec. Some container-based solutions may come with IPsec support out-of-the box (notably Docker s libnetwork, but flannel has plan for it too). This is quite important for a deployment over a public cloud.

Overhead The format of a VXLAN-encapsulated frame is the following: VXLAN adds a fixed overhead of 50 bytes. If you also use IPsec, the overhead depends on many factors. In transport mode, with AES and SHA256, the overhead is 56 bytes. With NAT traversal, this is 64 bytes (additional UDP header). In tunnel mode, this is 72 bytes. See Cisco IPsec Overhead Calculator Tool. Some users will expect to be able to use an Ethernet MTU of 1500 for the overlay network. Therefore, the underlay MTU should be increased. If it is not possible, ensure the inner MTU (inside the containers or the virtual machines) is correctly decreased⁸.

IPv6 While all the examples above are using IPv6, the ecosystem is not quite ready yet. The multicast L2-only strategy works fine with IPv6 but every other scenario currently needs some patches (1, 2, 3). On top of that, IPv6 may not have been implemented in VXLAN-related tools:

Cumulus Quagga and Cumulus vxfld daemon do not support IPv6 as a transport layer,

flannel has absolutely no IPv6 support,

Docker s libnetwork has no IPv6 support for the VXLAN backend.

Multicast Linux VXLAN implementation doesn t support IGMP snooping. Multicast traffic will be broadcasted to all VTEPs unless multicast MAC addresses are inserted into the FDB.

This is one possible implementation. The bridge is only needed if you require some form of source-address learning for local interfaces. Another strategy is to use MACVLAN interfaces.
The underlay multicast network may still need some central components, like rendez-vous points for PIM-SM protocol. Fortunately, it s possible to make them highly available and scalable (e.g. with Anycast-RP, RFC 4610).

For this example and the following ones, a patch is needed for the ip command (to be included in 4.11) to use IPv6 for transport. In the meantime, here is a quick workaround:

# ip -6 link add vxlan100 type vxlan \
>   id 100 \
>   dstport 4789 \
>   local 2001:db8:1::1 \
>   remote 2001:db8:2::1
# bridge fdb append 00:00:00:00:00:00 \
>   dev vxlan100 dst 2001:db8:3::1

You may have to apply an IPv6-related patch to the kernel (to be included in 4.12).
You have to apply an IPv6-related patch to the kernel (to be included in 4.12) to get appropriate notifications for missing IPv6 addresses.
Directly adding the entry after the first notification would have been smarter to avoid unnecessary retransmissions.
flannel and Docker s libnetwork were already mentioned as they both feature a VXLAN backend. There are also some interesting experiments like BaGPipe BGP for Kubernetes which leverages BGP EVPN and is therefore interoperable with other vendors.
There is no such thing as MTU discovery on an Ethernet segment.

Reproducible builds folks: Reproducible Builds: week 105 in Stretch cycle

Here's what happened in the Reproducible Builds effort between Sunday April 23 and Saturday April 29 2017: Past and upcoming events On April 26th Chris Lamb gave a talk at foss-north 2017 in Gothenburg, Sweden on Reproducible Builds. Between May 5th-7th the Reproducible Builds Hackathon 2017 will take place in Hamburg, Germany. Then on May 26th Bernhard M. Wiedemann will give a talk titled reproducible builds in openSUSE (2017) at the openSUSE Conference 2017 in N rnberg, Germany. Media coverage Already on April 19th Sylvain Beucler wrote a yet another follow-up post Practical basics of reproducible builds 3, after part 1 and part 2 of his series. Toolchain development and fixes Michael Woerister of the Rust project has implemented file maps that affect all path-related compiler information, including "error messages, metadata, debuginfo, and the file!() macro alike". Ximin Luo with support from some other Rust developers and contributors helped steer the final result into something that was compatible with reproducible builds. Many thanks to all involved, especially for the patience of discussing this over several months. Ximin wrote a first-attempt patch to fix R build-path issues. It made 460/477 R packages reproducible, but also caused 3 of these to FTBFS. See randomness_in_r_rdb_rds_databases for details. Bugs filed and patches sent upstream Chris Lamb:

#861133 filed against tf.
#861443 filed against ora2pg.

Bernhard M. Wiedemann filed a number of patches upstream:

intltool
xine-ui
tboot
javapackages
mxml, which got merged already.
calibre, also merged.
wammu, merged as well.

Reviews of unreproducible packages 102 package reviews have been added, 64 have been updated and 24 have been removed in this week, adding to our knowledge about identified issues. 3 issue types have been updated:

Added captures_build_path_in_beam_files, recategorise some erlang packages with captures_build_path into this issue instead.
Removed timestamps_in_beam_files.
Holger started reviewing blacklisted_on_jenkins and blacklisted_on_jenkins_armhf_only and found quite some packages which don't need to be blacklisted anymore.

Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

Aaron M. Ucko (1)
Adrian Bunk (1)
Chris Lamb (4)
Santiago Vila (2)

diffoscope development diffoscope 82 was uploaded to experimental by Chris Lamb. It included contributions from:

Chris Lamb:
- Add support for Ogg Vorbis files.
Vagrant Cascadian:
- Add support for .dtb (device tree blob) files. (Closes: #861109).

Changes from previous weeks that were also released with 82:

Ximin Luo
- Add support for R .rds and .rdb object files.
Chris Lamb
- Add support for comparing Pcap files.
- Add support for .docx and .odt files via docx2txt & odt2txt.
- Add support for PGP files via pgpdump.
- Various documentation and test improvements.
- Various bug fixes and code quality improvements.
Sylvain Beucler
- Display differences in zip platform-specific timestamps.

Misc. This week's edition was written by Ximin Luo, Chris Lamb and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Reproducible builds folks: Reproducing R packages

In the past couple of weeks, Ximin Luo worked on making R generate reproducible output. This is now mostly complete, and we're waiting on feedback from upstream about our patch. In the meantime, there are a few packages that remain unreproducible, but the issue probably lies in those specific packages rather than the R toolchain. Perhaps you can help out with them, after reading this!

R packages compile into a .rdb database format that contains the package's definitions, plus a .rdx index file for easy lookup in the .rdb file. Usually, there is a main rdb with the package contents, plus another rdb that stores the help data. There is also a paths.rds (same format as .rdx) that contains some more stuff. One can actually read these files by hand using Rscript, see the diffoscope code in rdata.py. If you run that, you can see that path.rds contains some obvious paths, but the other files contain less obvious stuff, and in fact these scripts give identical output for the .rdb files, even though they are bitwise different. To get to the bottom of this, we'll have to use the R debugger. Attached to this post is a script that smooths this process. I ran this against Debian's R packages, but it probably also works with other distros' R packages - try it and see. You run it like ./r-mini-repro-test.sh $pkgdir $builddir and it will output some hashes for you; make sure to install the build dependencies first. You should manually vary both $pkgdir and $builddir to introduce the build-path variations; $builddir can be an arbitrary string but $pkgdir should point to the actual R package's source directory, so I just copy that to two locations and point the script at each of them in turn. Now, we can begin debugging. Before I did this, we had 478 unreproducible R packages so the biggest problem was likely with R itself. I downloaded the source code of both R and a small example package (r-cran-tensor), then figured out how the R packages were actually being built. This resulted in me writing the script above, to speed up debugging. You can read about R's debugger here (no HTTPS), it's a bit primitive but it supports the basic stuff (step, continue, where, print/eval) and that was adequate for me. Debugging R core So let's take a small example with Debian R 3.4.0-1:

$ ../r-mini-repro-test.sh r-cran-tensor-1.5 123
[..]
* installing *source* package  tensor  ...
** package  tensor  successfully unpacked and MD5 sums checked
** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (tensor)
[..]
c71a766429151a13e46039f3c37a14edc704004f410c34f8f37a2f098506421d  ./debian/123/usr/lib/R/site-library/tensor/R/tensor.rdx
736b5b4a885db71fb3fe2a538faeefe43c500c4080dbf2c75b97135b98401acd  ./debian/123/usr/lib/R/site-library/tensor/R/tensor.rdb
8a3f9616de7ee78a83b07395dd6ec43c5259d3f1b1b653d397808fc9fc5d96c2  ./debian/123/usr/lib/R/site-library/tensor/help/tensor.rdx
d1ac634987b24606a5da627ec073f45edce2bddf071391dd17c246a0a01eba63  ./debian/123/usr/lib/R/site-library/tensor/help/tensor.rdb
80796cc1a2cfcac40c394153f67617d575e103a405430b8483c2261a903db046  ./debian/123/usr/lib/R/site-library/tensor/help/paths.rds
$ ../r-mini-repro-test.sh r-cran-tensor-1.5 1234
[..]
5f081f4d6f9da1fbe06e374067e6636d926de4cdf09886058b15c5928c636c0d  ./debian/1234/usr/lib/R/site-library/tensor/R/tensor.rdx
7145669891e1b766ca0d8d89fa9c317af716e61a62650abe3a11a848240f1d57  ./debian/1234/usr/lib/R/site-library/tensor/R/tensor.rdb
8a3f9616de7ee78a83b07395dd6ec43c5259d3f1b1b653d397808fc9fc5d96c2  ./debian/1234/usr/lib/R/site-library/tensor/help/tensor.rdx
d1ac634987b24606a5da627ec073f45edce2bddf071391dd17c246a0a01eba63  ./debian/1234/usr/lib/R/site-library/tensor/help/tensor.rdb
80796cc1a2cfcac40c394153f67617d575e103a405430b8483c2261a903db046  ./debian/1234/usr/lib/R/site-library/tensor/help/paths.rds

The files are created by the ** preparing package for lazy loading and

**
help

steps. In retrospect this seems somewhat obvious from the output, but I had never done R before so I spent a while using strace to be sure; one can see that those files are indeed written to after those strings are printed. Then we can grep the R source code for these strings.

r-base-3.4.0$ grep -Ri "preparing package for lazy loading" src
src/library/tools/R/install.R:                starsmsg(stars, "preparing package for lazy loading")

Reading the file, we see that this happens in the tools:::.install_packages function, and reading it further we see that it calls makeLazyLoading then code2LazyLoadDB then makeLazyLoadDB. Seems promising, let's confirm it before chasing potential wild geese. Important: In the rest of these command-line outputs I'll prepend what I input with >>> but this doesn't actually get printed by Rscript. So don't be surprised that you don't see these, when you're trying to recreate my steps. Also I am a total R noob so possibly there are more elegant ways to do what I'm about to show you; this is what I came up with after 2-3 hours of research.

$ DEBUG=1 ../r-mini-repro-test.sh r-cran-tensor-1.5 123
enter (some or all of) the following into R:
[.. helpful commands ..]
+ R_DEFAULT_PACKAGES= LC_COLLATE=C /usr/lib/R/bin/R --no-restore --slave --args nextArg-lnextArgdebian/123/usr/lib/R/site-librarynextArg-dnextArg.nextArg--built-timestamp="Thu, 01 Jan 1970 00:00:00 +0000"
>>> debug(tools:::makeLazyLoadDB)
>>> tools:::.install_packages()
processing  . 
[..]
** preparing package for lazy loading
debugging in: makeLazyLoadDB(ns, dbbase, compress = compress)
debug:  
[.. source code of makeLazyLoadDB, not yet run ..]

OK so we just entered the function and it's paused, in a separate shell let's check what it's doing:

$ find r-cran-tensor-1.5 -name '*.rdb'
[ no output ]

So, no files have been written yet, which means none of the parent functions did anything, we are in the right function (or it's in a child). If we continue the function, we get:

>>> c
exiting from: makeLazyLoadDB(ns, dbbase, compress = compress)
** help
debugging in: makeLazyLoadDB(db, file.path(manOutDir, basename(outDir)))
debug:  
[.. source code of makeLazyLoadDB, not yet run ..]
 
$ find r-cran-tensor-1.5 -name '*.rdb'
r-cran-tensor-1.5/debian/123/usr/lib/R/site-library/tensor/R/tensor.rdb

Then continuing again:

>>> c
exiting from: makeLazyLoadDB(db, file.path(manOutDir, basename(outDir)))
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (tensor)
$ find r-cran-tensor-1.5 -name '*.rdb'
r-cran-tensor-1.5/debian/123/usr/lib/R/site-library/tensor/help/tensor.rdb
r-cran-tensor-1.5/debian/123/usr/lib/R/site-library/tensor/R/tensor.rdb

So it seems likely this function is the correct one. After reading the source code of makeLazyLoadDB, even though I don't understand the whole thing, it looks like possibly this part is the one that does the actual writing:

    for (i in seq_along(vars))  
        key <- if (is.null(from)   is.environment(from))
            lazyLoadDBinsertVariable(vars[i], from, datafile,
                                     ascii, compress,  envhook)
        else lazyLoadDBinsertListElement(from, i, datafile, ascii,
                                         compress, envhook)
        assign(vars[i], key, envir = varenv)

So let's step through it and see if the data contains any paths.

$ DEBUG=1 ../r-mini-repro-test.sh r-cran-tensor-1.5 123
[..]
>>> debug(tools:::makeLazyLoadDB)
>>> tools:::.install_packages()
processing  . 
[..]
** preparing package for lazy loading
debugging in: makeLazyLoadDB(ns, dbbase, compress = compress)
debug:  
[.. source code of makeLazyLoadDB, not yet run ..]
 
>>>                                                                             # pressing <Enter> just means "step"
debug: ascii <- as.logical(ascii)                                               # this is the first line of the function
>>>                                                                             # keep stepping
[..]
>>>
debug: for (i in seq_along(vars))  
[.. source code of the inside of the block, not yet run ..]
 
>>>
debug: key <- if (is.null(from)   is.environment(from)) lazyLoadDBinsertVariable(vars[i],
    from, datafile, ascii, compress, envhook) else lazyLoadDBinsertListElement(from,
    i, datafile, ascii, compress, envhook)
>>>
debug: lazyLoadDBinsertVariable(vars[i], from, datafile, ascii, compress,
    envhook)                                                                    # not yet run

If we look at the source code we see that it will try to insert the value of .Internal(getVarsFromFrame(vars[i], from, FALSE))[[1L]] into the file, so let's evaluate it:

>>> .Internal(getVarsFromFrame(vars[i], from, FALSE))[[1L]]
function (x, y)
tensor(x, y, 2, 2)
<environment: namespace:tensor>
>>> typeof(.Internal(getVarsFromFrame(vars[i], from, FALSE))[[1L]])
[1] "closure"
>>> environment(.Internal(getVarsFromFrame(vars[i], from, FALSE))[[1L]])        # get the environment of a closure
<environment: namespace:tensor>
>>> envlist(environment(.Internal(getVarsFromFrame(vars[i], from, FALSE))[[1L]])) # environment as an envlist, more detailed output
$ %*t% 
function (x, y)
tensor(x, y, 2, 2)
<environment: namespace:tensor>
$ %t*% 
function (x, y)
tensor(x, y, 1, 1)
<environment: namespace:tensor>
$ %t*t% 
function (x, y)
tensor(x, y, 1, 2)
<environment: namespace:tensor>
$.__NAMESPACE__.
<environment: 0x55eb55d00be0>
$.__S3MethodsTable__.
<environment: 0x55eb55d166e8>
$.packageName
[1] "tensor"
$tensor
function (A, B, alongA = integer(0), alongB = integer(0))
 
[.. function definition ..]
 
<environment: namespace:tensor>

Nothing obviously containing paths so far, let's keep stepping and dumping the objects

>>>
debug: assign(vars[i], key, envir = varenv)
>>>                                                                             # next iteration now
debug: key <- if (is.null(from)   is.environment(from)) lazyLoadDBinsertVariable(vars[i],
    from, datafile, ascii, compress, envhook) else lazyLoadDBinsertListElement(from,
    i, datafile, ascii, compress, envhook)
>>>
debug: lazyLoadDBinsertVariable(vars[i], from, datafile, ascii, compress,
    envhook)
>>> .Internal(getVarsFromFrame(vars[i], from, FALSE))[[1L]]
function (x, y)
tensor(x, y, 1, 1)
<environment: namespace:tensor>
>>>
[.. more stepping ..]
debug: lazyLoadDBinsertVariable(vars[i], from, datafile, ascii, compress,
    envhook)
>>> .Internal(getVarsFromFrame(vars[i], from, FALSE))[[1L]]
<environment: 0x55eb55d00be0>
>>> envlist(.Internal(getVarsFromFrame(vars[i], from, FALSE))[[1L]])
$S3methods
     [,1] [,2] [,3]
$dynlibs
NULL
$exports
<environment: 0x55eb55d0a4f8>
$imports
$imports$base
[1] TRUE
$lazydata
<environment: 0x55eb55d0a178>
attr(,"name")
[1] "lazydata:tensor"
$path
[1] "/mystupidbuildpath/r-tests/r-cran-tensor-1.5/debian/123/usr/lib/R/site-library/tensor"
$spec
    name  version
"tensor"    "1.5"

Oh, here we go. (In fact we could have got this earlier, it was part of the .__NAMESPACE__. key of the first environment that we examined.) Naturally, you'll sometimes miss stuff, and sometimes go down dead ends when doing something like this, perseverence is important. The names are also a hint, after grepping the R source code for names like "S3methods" and "lazydata" we see that src/library/base/R/namespace.R is the only one that contains all of them, then reading that code in more depth reveals this. After patching it, rebuilding r-base, installing this patched version, then running our tests again, we see that we've eliminated differences in the main rdb file but not the help rdb file.

In fact, after some more time stepping through the loop repeatedly and printing out all the objects, nothing obvious shows up. Time to re-read the source code of makeLazyLoadDB. We notice that all the lazyLoadDBinsert* functions have a hook parameter that we haven't been stepping through because it's in a separate function. So let's do that. Since it's an inner function, one has to call debug() inside makeLazyLoadDB:

$ DEBUG=1 ../r-mini-repro-test.sh r-cran-tensor-1.5 123
[..]
>>> debug(tools:::makeLazyLoadDB)
>>> tools:::.install_packages()
processing  . 
[..]
** preparing package for lazy loading
debugging in: makeLazyLoadDB(ns, dbbase, compress = compress)
debug:  
[.. source code of makeLazyLoadDB, not yet run ..]
 
>>> c
exiting from: makeLazyLoadDB(ns, dbbase, compress = compress)
** help
debugging in: makeLazyLoadDB(db, file.path(manOutDir, basename(outDir)))
debug:  
[.. source code of makeLazyLoadDB, not yet run ..]
 
[.. stepping through the function ..]
debug: envhook <- function(e)  
[.. function definition ..]
 
>>>                                                                             # step next, to define that function
[.. next statement, we don't care about it ..]
>>> debug(envhook)
[.. then keep stepping through makeLazyLoadDB ..]
>>>
debug: lazyLoadDBinsertVariable(vars[i], from, datafile, ascii, compress,
    envhook)
[.. then keep stepping through makeLazyLoadDB ..]
debug: lazyLoadDBinsertVariable(vars[i], from, datafile, ascii, compress,
    envhook)
[.. hook wasn't run for these previous iterations for some reason, but keep stepping ..]
[.. eventually we get ..]
debugging in: (function (e)
 
[.. function definition ..]
 )(<environment>)                                                               # "(..)(<environment>)" is R's name for that function (actually a closure)
debug:  
[.. source code, not yet run ..]
 
>>> e
/mystupidbuildpath/r-tests/r-cran-tensor-1.5/man/tensor.Rd
>>> typeof(e)
[1] "environment"
>>> envlist(e)
$Enc
[1] "UTF-8"
$encoding
[1] ""
$filename
[1] "/mystupidbuildpath/r-tests/r-cran-tensor-1.5/man/tensor.Rd"
$timestamp
[1] "2002-01-16 11:51:14 CET"
$wd
[1] "/mystupidbuildpath/r-tests/r-cran-tensor-1.5"

After doing the same thing as before, i.e. grepping the R source code for "Enc" "timestamp" etc, we spot a few different places that are setting the wd field. Then after some experimenting, we find that overwriting this in a different file (src/library/tools/R/parseRd.R) seems to work to get rid of this irreproducibility. (This may not be the best; our exact strategy is still being discussed with upstream, but at least we know which areas of the code are responsible.) Exercise for the reader to figure out how to get rid of the paths.rds difference. After doing the previous two, this one is fairly easy. These three issues allowed us to write a preliminary patch that enabled us to reproduce r-cran-tensor.

$ ./r-mini-repro-test.sh r-cran-tensor-1.5 123
[..]
bca9de1351ab265475cf6f1af3f6aff49bde1d209e8fafb33954caa8d90f3462  ./debian/123/usr/lib/R/site-library/tensor/R/tensor.rdx
ac7fffeb751ffc2d13c5317704b19787a7078446d97d1c6f259c93f282e04cef  ./debian/123/usr/lib/R/site-library/tensor/R/tensor.rdb
d989603340d1e8f14c35551ef379956653bf3b6e7b1868b4bf9a66d85820998e  ./debian/123/usr/lib/R/site-library/tensor/help/tensor.rdx
30e543cda9f97c202dc9c0291933940dfe71b94d352f66d8ed6c035c1a722b41  ./debian/123/usr/lib/R/site-library/tensor/help/tensor.rdb
e9f78ba2cca39a6face1ed88fce4bb718d911d567d01db49bc00a3011720ac50  ./debian/123/usr/lib/R/site-library/tensor/help/paths.rds
$ ./r-mini-repro-test.sh r-cran-tensor-1.5 1234
[..]
bca9de1351ab265475cf6f1af3f6aff49bde1d209e8fafb33954caa8d90f3462  ./debian/1234/usr/lib/R/site-library/tensor/R/tensor.rdx
ac7fffeb751ffc2d13c5317704b19787a7078446d97d1c6f259c93f282e04cef  ./debian/1234/usr/lib/R/site-library/tensor/R/tensor.rdb
d989603340d1e8f14c35551ef379956653bf3b6e7b1868b4bf9a66d85820998e  ./debian/1234/usr/lib/R/site-library/tensor/help/tensor.rdx
30e543cda9f97c202dc9c0291933940dfe71b94d352f66d8ed6c035c1a722b41  ./debian/1234/usr/lib/R/site-library/tensor/help/tensor.rdb
e9f78ba2cca39a6face1ed88fce4bb718d911d567d01db49bc00a3011720ac50  ./debian/1234/usr/lib/R/site-library/tensor/help/paths.rds
$ ./r-mini-repro-test.sh r-cran-tensor 1234
[..]
bca9de1351ab265475cf6f1af3f6aff49bde1d209e8fafb33954caa8d90f3462  ./debian/1234/usr/lib/R/site-library/tensor/R/tensor.rdx
ac7fffeb751ffc2d13c5317704b19787a7078446d97d1c6f259c93f282e04cef  ./debian/1234/usr/lib/R/site-library/tensor/R/tensor.rdb
d989603340d1e8f14c35551ef379956653bf3b6e7b1868b4bf9a66d85820998e  ./debian/1234/usr/lib/R/site-library/tensor/help/tensor.rdx
30e543cda9f97c202dc9c0291933940dfe71b94d352f66d8ed6c035c1a722b41  ./debian/1234/usr/lib/R/site-library/tensor/help/tensor.rdb
e9f78ba2cca39a6face1ed88fce4bb718d911d567d01db49bc00a3011720ac50  ./debian/1234/usr/lib/R/site-library/tensor/help/paths.rds

Testing the patch Of course r-cran-tensor is just 1 package, so then we tested this patch on all 478 tagged R packages. Before the patch, all of these were unreproducible. Now 463/478 are reproducible, hurray! The first version of the patch made 2 packages FTBFS ("fail to build from source"), r-bioc-biobase and r-cran-shinybs; this was fixed in a subsequent version of the patch. (Some other packages, r-cran-randomfields r-cran-randomfieldsutils, FTBFS even with an unpatched r-base, due to differences between 3.3.3 and 3.4.0 (#861333), and not because of our patch.) Given the overwhelming proportion of packages that did reproduce, the other 14 packages that are still unreproducible, are quite probably due to issues in those specific packages. Only 2 of these are certainly due to build-path differences: r-cran-runit r-cran-rinside. This is because, we currently see they are reproducible in Debian testing but not Debian unstable. This is slightly misleading, the real reason is not testing vs unstable directly; rather the fact that we (at the time of writing) use the same build path when trying to reproduce Debian testing packages, and different paths for unstable. The other 12 are unreproducible in both testing and unstable, so it either suffers from a non-build-path issue as well as a build-path issue, or only a non-build-path issue. So, further investigation of each package is needed. This is where you can help.

Debugging R packages To investigate why (for example) r-cran-runit doesn't reproduce despite our patches, we just do the above process again:

$ DEBUG=1 ../r-mini-repro-test.sh r-cran-runit
[..]
>>> debug(tools:::makeLazyLoadDB)
>>> tools:::.install_packages()
processing  . 
a directory
* build_help_types=
* DBG: 'R CMD INSTALL' now doing do_install()
* created lock directory  /mystupidbuildpath/r-tests/r-cran-runit/debian/r-cran-runit/usr/lib/R/site-library/00LOCK-r-cran-runit 
* installing *source* package  RUnit  ...
** package  RUnit  successfully unpacked and MD5 sums checked
** backing up earlier installation
** R
** inst
** preparing package for lazy loading
debugging in: makeLazyLoadDB(ns, dbbase, compress = compress)
debug:  
[.. source code of makeLazyLoadDB ..]

We know our irreproducibility is with the help files and not the main package, so c to skip this part:

>>> c
exiting from: makeLazyLoadDB(ns, dbbase, compress = compress)
** help
debugging in: makeLazyLoadDB(db, file.path(manOutDir, basename(outDir)))
debug:  
[.. source code of makeLazyLoadDB ..]

Examine the "from" object, grep the output for our builddir

>>> from
$ RUnit-internal 
[..]
$ RUnit-intro 
[..]
[.. etc ..]
[..]
$checkFuncs
\title RUnit check [..]
[..]
  For a simple example see the provided test cases in
  /mystupidbuildpath/r-tests/r-cran-runit/debian/r-cran-runit/usr/lib/R/site-library/RUnit/examples/runitVirtualClassTest.r.
[..]

Found it, now we quit, patch the source code, and try again.

>>> Q
[..]
exit code 1
[.. hack hack hack ..]
$ ../r-mini-repro-test.sh r-cran-runit-0.4.31/ 123
[..]
01347305351a45bb88dab909c56942498e97f5edbe583cf6ab1327138914e315  ./debian/123/usr/lib/R/site-library/RUnit/R/RUnit.rdx
ecde40bdfac5b070acc30a0d5889d9d7f65418844d14312aceff6af5b9d0bf7a  ./debian/123/usr/lib/R/site-library/RUnit/R/RUnit.rdb
182d38ac5adc160eb9edd40d5bd7e27b95b756b8f14cee7d912ba4745ae81bdc  ./debian/123/usr/lib/R/site-library/RUnit/help/RUnit.rdx
f8a9f22acc9f937cb47a92f3979d619e83a791ad734796c2fcd9d0a34e1f11ca  ./debian/123/usr/lib/R/site-library/RUnit/help/RUnit.rdb
5c37672fdc1f325b8a3846f4eafd2624b246ec70ce01d197134fa1b97a9ee849  ./debian/123/usr/lib/R/site-library/RUnit/help/paths.rds
$ ../r-mini-repro-test.sh r-cran-runit-0.4.31/ 1234
[..]
01347305351a45bb88dab909c56942498e97f5edbe583cf6ab1327138914e315  ./debian/1234/usr/lib/R/site-library/RUnit/R/RUnit.rdx
ecde40bdfac5b070acc30a0d5889d9d7f65418844d14312aceff6af5b9d0bf7a  ./debian/1234/usr/lib/R/site-library/RUnit/R/RUnit.rdb
182d38ac5adc160eb9edd40d5bd7e27b95b756b8f14cee7d912ba4745ae81bdc  ./debian/1234/usr/lib/R/site-library/RUnit/help/RUnit.rdx
f8a9f22acc9f937cb47a92f3979d619e83a791ad734796c2fcd9d0a34e1f11ca  ./debian/1234/usr/lib/R/site-library/RUnit/help/RUnit.rdb
5c37672fdc1f325b8a3846f4eafd2624b246ec70ce01d197134fa1b97a9ee849  ./debian/1234/usr/lib/R/site-library/RUnit/help/paths.rds

Works! All finished.

If yours doesn't, just repeat the above loop; hopefully eventually you'll get there.

Axel Beckert: Upcoming Hacker/FLOSS Events in Switzerland: Debian BSP, LPD, Hackerfunk 10th Anniversary, ZeTeCo

There are quite some events and dates ahead for hackers, makers, debianers and hackerfunk listeners: Crowdfunding for ZeTeCo Camp in July Ends in Two Days! You might have heard of the ZeTeCo Camp near Schaffhausen in July. If you want ot come or at least support that event, please contribute to their crowdfunding campaign. They have more than 90% of their goal funded and there s less only about two days left to reach their funding goal. If it doesn t get funded in time, the event may be be on a knife edge. Debian Bug Squashing Party in Zurich this Weekend One week before the More-than-a-BSP at the Mozilla office in Paris there will also be a Debian Bug Squashing Party (BSP) in Z rich at the CCCZH Hackerspace R schtibach . We ll start on Friday, the 5th of May 2017 in the late afternoon, probably around 4pm or 5pm, and will end on Sunday, the 7th of May 2017 also in the late afternoon. Please add yourself to the according section on the BSP s wiki page if you want to join us to squash the hopefully not that many left over bugs in testing. Unfortunately we didn t notice two date clashes when we set the date for the BSP during the Annual General Meeting (AGM) of the Debian.ch Association earlier this year: Linux Presentation Day this Saturday, 6th of May 2017 On the one hand there will be the Swiss Edition of the Linux Presentation Day (LPD) on Saturday, the 6th of May 2017 including the LPD in Z rich. The latter will take place in the same building as the BSP, just on a different floor: The BSP will be at floor B3 in the CCCZH Hackerspace and the LPD will be on the ground floor at Revamp-IT in the former ZKB foyer. 10 Years Hackerfunk: Show on 6th, Party on 13th of May 2017 And on the other hand, Venty s and my (German dialect) radio show and podcast Hackerfunk will have it s 10th anniversary show also on that Saturday. So I ll vanish from BSP for a few hours on Saturday evening for broadcasting this very special Hackerfunk episode on Radio Radius. But since Venty and me didn t want to make yet another big event at R schtibach on the same weekend, we ll do the Hackerfunk 10th Anniversary Party one weekend later on Saturday the 13th of May 2017 also at the CCCZH Hackerspace R schtibach . A separate announcement on https://www.hackerfunk.ch/ (also in the RSS feed there) will follow.

Raphaël Hertzog: My Free Software Activities in April 2017

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donors (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS I was allocated 10 hours to work on security updates for Debian 7 Wheezy and had 1.5 hours remaining from March. During this time I did the following:

I released DLA-905-1 on ghostscript fixing 3 CVE. I also triaged two other ghostscript CVE that were not relevant to the version in wheezy.
I started to look into CVE-2016-10209 for libarchive but was not able to reproduce the segfault and marked it as not worth an update (same decision as security team).
After many tries to get more details from upstream of libxml-twig-perl on CVE-2016-9180, I decided that the low severity of the issue was not worth spending more time on it (same decision as RedHat and Debian security team).
I released DLA-921-1 on slurm-llnl fixing 1 high-severity CVE.
I investigated CVE-2016-8686 on potrace and marked it as not requiring an update because the impact is very low. I documented the fact that it s fixed in unstable and asked the upstream author for the specific patch (no answer yet though).

Kali and pkg-security I updated the britney instance that we are using in Kali and spotted two small documentation mistakes that I fixed. We had a long-standing bug in Kali where extensions would stay visible on the lock screen. It was hard to reproduce and this month we finally managed to nail down the conditions required to reproduce it. It turns out that EasyScreenCast was the culprit. We paid Emilio Pozuelo Monfort to work on a patch and he fixed the problem in EasyScreenCast and also in gnome-shell, as a buggy extension should not have resulted in this behavior. I responded to multiple queries of new contributors in the pkg-security team. The team is rather active and it would be great if we could have a few more Debian developers to help review and sponsor the work our enthusiastic new members. Thanks See you next month for a new summary of my activities. Hopefully, I will be more active again between kids vacations, French elections and Zelda Breadth of the Wild, I got very much distracted from Debian last month.

One comment Liked this article? Click here. My blog is Flattr-enabled.

2 May 2017

Markus Koschany: My Free Software Activities in April 2017

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Games

I released the final version 2 of debian-games for Stretch, a collection of metapackages that make it easier to install one s favorite games. No surprises here, but two games were removed from Debian and some others didn t make it into Stretch, so an update of the debian/control file was necessary.
I could close three RC bugs in ufoai (#860680) and slashem (#860393), thanks to the analysis and help from Bernhard belacker, and one in tecnoballz. (#861268)
I reviewed and sponsored gnome-games-app and retro-gtk for Jeremy Bicha. The Gnome application is basically a game browser and a unified interface to access various games, engines and emulators.
I adopted torcs, a racing car simulator, updated the package to the latest upstream release and completely overhauled the packaging. This is still work in progress but I think I can upload the new version in May.

Debian Java

I prepared security updates for bouncycastle, logback, activemq, tomcat 7 and tomcat 8 which were later accepted for Stretch and Jessie.
New upstream releases this month: hsqldb and robocode.
We had to deal with a weird bug in libnb-platform18-java respectively libjna-jni (#858876). We are still not sure why the JNA system library was not found by Netbeans but we could find a way to resolve it. I took the opportunity to fix another annoying bug in Netbeans and added the missing dependencies for some symlinked system JAR files.
Last but not least I made the test results in jnr-posix non-fatal (#860691) until we receive more information from upstream why two tests fail on i386. Since the package is arch:all and completes all tests on amd64 successfully, the RC bug was later marked as stretch ignore by the release team.

Debian LTS This was my fourteenth month as a paid contributor and I have been paid to work 23,75 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following:

From 10. April until 16. April I was in charge of our LTS frontdesk. I triaged security issues in wavpack, binutils, tiff, tiff3, imagemagick, wireshark, elfutils, libosip2, feh, freetype, web2py and libplist.
DLA-888-1. Issued a security update for logback fixing 1 CVE.
DLA-893-1. Issued a security update for bouncycastle fixing 1 CVE.
DLA-924-1. Issued a security update for tomcat7 fixing 2 CVE.
DLA-900-1. Issued a security update for freetype. Later it was determined that freetype in Wheezy was not affected and the change was reverted.
DLA-899-1. Issued a security update for feh fixing 1 CVE.
DLA-902-1. Issued a security update for imagemagick fixing 2 CVE.
DLA-911-1. Issued a security update for tiff fixing 11 CVE.
DLA-912-1. Issued a security update for tiff3 fixing 8 CVE.
DLA-913-1. Issued a security update for activemq fixing 1 CVE.
DLA-929-1. Issued a security update for libpodofo fixing 7 CVE.

Misc

I packaged a new major version of Osmo, a personal organizer and calendar application. It has been completely converted to GTK3 and WebKitGtk2 now.

Thanks for reading and see you next time.

1 May 2017

Russ Allbery: Review: Survival

Review: Survival, by Julie E. Czerneda

Series:	Species Imperative #1
Publisher:	DAW
Copyright:	May 2004
ISBN:	0-7564-0180-1
Format:	Hardcover
Pages:	401

Dr. Mackenzie Connor, Mac to everyone she works with, is a biodiversity researcher specializing in salmon. In her future United States, humanity seems to have caught on to the importance of preserving wild places and learning about them, and is willing to invest in good equipment and a semi-permanent research installation. This comes with some occasional drawbacks, since she has to fight to get access to the salmon runs inside a nature preserve, but she wouldn't have it any other way. She wins enough of those fights, won the latest, and is now in position to monitor a run in a way that she's never been able to before. She was not expecting an alien to go diving in the middle of her salmon run. She was certainly not expecting that alien to be accompanied by a bureaucrat insisting that this alien's curiosity is more important than her research (hah). But the accompanying letter she receives is scarily persuasive, if maddeningly unhelpful. Much like the apparently jovial, earnest, eager, and very odd alien. Mac's continued hopes that she can quickly put this bizarre intrusion to rest and go back to her salmon are dashed by an impossible power outage, an alien visitor, and a violent kidnapping. Now her best friend and colleague is missing, the bureaucrat is not who he appears to be, and Mac is getting caught up in something that feels way over her head. SF novels feature a lot of science, but not a lot of scientific research. The research that does appear is often impulsive, wildly compressed, or far too focused on the breakthroughs of single people. The SF novel that everyone points to for accurate portrayal of real scientific research is Benford's Timescape, which I found deeply unexciting. Now I have a new novel to point to for a better treatment, although (somewhat disappointingly) Mac's research gets sidelined relatively early in the story and left behind for the conclusion. Czerneda gives us not just a few scientists and an imaginary research project, but an entire operational field station with a history. The Norcoast Salmon Research Facility is located just off-shore in carefully-designed domes to provide easy access to the sea with minimal intrusion into the local ecology. It's a bustling mix of research scientists, engineers, and the ever-present seasonal grad students, who come and go in all their immature enthusiasm and are viewed with a motherly bemusement that I immediately recognized from years of working at a university. Mac splits administrative duties with another scientist in an arrangement that will be familiar to academics everywhere, and the book opens with a mutually suspicious but mostly scripted turf fight with the guardian of the neighboring wildlife trust, the same fight they've been having every six months for years. I know Czerneda is herself a biologist by training; I'm not sure what her other academic background is, but if she hasn't spent years around academics and field studies, she's at least done some excellent research. A lot of novels have a quotidian background that's interrupted by the arrival of the plot. At the start of the story, the characters often care more about their day-to-day lives than the plot, and are dragged into it reluctantly. But one sign of an excellent writer is their ability to get the reader to care about that quotidian background alongside the character, and to sympathize with the character's reluctance to get pulled into the promised (and generally more exciting) novel plot. Czerneda succeeds in this about as well as any writer I've read since Robin McKinley's Sunshine, and that's high praise. I cared about Mac's salmon, I was nearly as irritated as she was when her research was interrupted, and I still want to go back and see more of the experiments and studies she was hoping to run. Interstellar drama and threats to multiple species are all well and good, but the salmon are running! The actual plot is a mysterious threat that turns into a combination of a biological and cultural puzzle and a sort of first-contact story. Mac is not truly the first human to encounter the Dhryn, but she's certainly the first person they've explained anything to, and the first human to go where she goes. Sadly, it also shares some of the characteristics that sour me a bit on biological SF for personal reasons: a bit too much description of food, eating habits, squishy body parts, digestive processes, and biological discomfort. This is mostly a personal gripe, and won't bother other people as much as it does me, but I could have done without bits like the descriptions of Mac's attempts to figure out how to survive on alien cuisine. I'm also dubious of some of the biology of the Dhryn; given the startling bizarreness of Earth biology, maybe I shouldn't be, but I still think there are a few problems with the square-cube law here. But Mac's irrepressible grumpy curiosity makes this story, even in the bits that made me squeamish. I think I'd read any book in which she's the main character. I will warn that the ending is surprisingly dark and wasn't what I was expecting, and Survival doesn't resolve its central mysteries. This is clearly the first book of a trilogy and should be read with that expectation. But I thoroughly enjoyed it, and hopefully the next book will have more salmon. Followed by Migration. Rating: 8 out of 10

30 April 2017

Paul Wise: FLOSS Activities April 2017

Changes

myrepos: webcheckout considers https URLs as anonymous, switch URLs from http to https and fall back on terminal tools when X11 is gone

fdupes: NUL separator option

whohas: add more TODO sites

apt: more flexibility in APT::Periodic interval options

foxtrotgps: fix maps-for-free.com, httpsify website links (1, 2) and remove website comment

lm-sensors: workaround RRD format inflexibility

check-all-the-things: terminal fixes (1, 2, 3), Perl fixes (1, 2, 3, 4, 5, 6), grep fixes, more key checks, wrapping fixes (1, 2), release (1, 2) and TODO items for chap/Devel::Plumber

youtube-dl: video downloader list

devscripts: add support for running aptitude to chdist, add support for cgit URLs to debcheckout and fix issues when $HOME is unset (1, 2)

Spelling:

codespell: sort dictionary, explot(ing), menue(s), mimimise, intall, cas, overrided, icluding, pleaes, interract(ing), interracts, deffered, upsteram, componet(s), depracated, annoncement, settting, pasword, post-morten, converion, verifyied and achive

lintian: explot(ing), menue(s), mimimise, intall, cas, icluding, interract(ing), interracts, deffered, upsteram, componet(s), depracated, annoncement, adminstration, inappropiate, pasword, post-morten and verifyied

Debian wiki: Andrewsomething/Fonts/PackagingPolicy, AptitudeTodo, AudioVideo, BogdanPurcareata/PluggableAptBackend, BTS/NewcomerTag, DebianColombia (Eventos, MiniDebconf2017), DebianEdu/Status/Stretch, DebianEvents/br, (2017/MiniDebconfCuritiba), Debian_GNU/kFreeBSD/Jessie, DebianIndia/PackagingSessions, DebianInstaller/Qemu, DebianKernel/UserspaceTools, DebianMentorsFaq, DebianOnHandhelds/OpenPandora, DebianPackage (es, fr, ru, sv), DebianReleaseFAQ, DebianRepository (Setup), DebianScience/Debci, DebianSingleSignOn, DebianWomen/Projects/NewbieTasks, Derivatives/Census (QA, AstraLinux, CumulusLinux, CumulusLinux, Kali, Netrunner, Parsix, Tanglu, TurnKeyLinux, Ubuntu, Wazo), FreedomBox/LeavingTheCloud, fr (genisoimage, Quota), hal, Hardware/Wanted, HelpDebian (FAQ (it), TODO), how-can-i-help, (Ideas, fr), HP/ProLiant, Init, (fr), Initrd, (fr), InstallingDebianOn, (InstallingDebianOn/HP/EliteBook 820 G1, Razer/BladeStealth, it/Init, it/Initrd, Mempo/mempo-deb/tar, motd, (it, fr), MulheresBrasil (1, 2), MultimediaFetchingTools (1, 2), Packaging/BinaryPackage, PaulWise/InterestingSoftware, ReproducibleBuilds (BuildinfoInfrastructure, History), ru/ReposRelease, SecureApt, ServicesSSL (1, 2), SHA-1, Statistics, SuitesAndReposExtension, SystemBuildTools, Teams (Publicity, ReleaseTeam/ReleaseCheckList), UntrustedDebs, VideoDownloaders and WebBrowsers

Debian wiki theme: remove last editor info

Debian website: fix encoding issues, fix year issues and fix DPL update issue (1, 2)

Debian mirrors: fix encoding issues

Debian security tracker: encuentro embeds youtube-dl and add mediawiki issues (1, 2)

Debian derivatives census: workaround apt issue with stretch, reduce max patch size to 5 MiB, document how to get large patches, ignore safe debian/changelog symlinks and add FIXMEs

Debian Planets: Add Wazo, BunsenLabs RSS feed update, https for blog feeds and new Cumulus Linux/Netrunner logos

Debian package uploads: python-debianbts, poppler, cruft, clamav-unofficial-sigs, config-package-dev, libarchive, talloc, tevent, libconfig-crontab-perl 1.42-1~bpo7+1 & 1.42-1~bpo8+1, openchange and check-all-the-things

Debian puppet: switch from deprecated function, drop custom update-ca-certificates on stretch (1, 2) and support syslog-ng 3.8

Debian meta-packages: www.d.o stretch updates and qa.d.o vcswatch deps

Debian LDAP UI: use TLS for direct LDAP access

Debian publicity: prefer verification to SHA-1

Debian release: add latest two release updates

Debian PTS: switch from gift to newcomer tags

Issues

Crashes in remmina, liferea (1, 2) and diffoscope

Non-distributable files in Ubuntu

Debian testing migration unblocks of check-all-the-things

Debian Policy violation in muttdown

Move crash-whitepaper from Debian non-free to main

Normal issues in needrestart (1, 2), liferea and git-crecord

Features in needrestart, github-backup, systemctl, android-permissions, reprepro, freerdp, git-crecord (1, 2, 3), Debian manual pages, Debian sources browser, Debian Maintainer Dashboard and dman

New versions of pyvmomi

Typos in fenrir

Code copy in encuentro

Race conditions in git-hub

Terminal handling in needrestart

Bogus home directory handling in how-can-i-help, ruby, bash, dash and busybox sh

CodePlex closing checks for lintian and duck

Missing hashes in Ubuntu

Weird spaces in evolution

Verbosity in apt

Security issue in nagstamon and a CSS injection attack elsewhere

Minor issues in reportbug and debsources (1, 2)

Review

Spam: reported 5 Debian bug reports and 246 Debian mailing list posts

Debian wiki: RecentChanges for the month

Debian packages: reviewed openscap-daemon, gsignond, qt5ct and sponsored open-ath9k-htc-firmware

Debian screenshots:

approved 4pane, amiwm, btrfs-heatmap, cenon.app, desmume, fonts-atarismall, fonts-goudybookletter, fonts-junicode, fonts-ocr-a, fonts-ocr-b, gnome-paint, grafx2, higan (2), lm-sensors, lshw-gtk, mediathekview, mrboom, munin-plugins-btrfs, nano, openlugaru, project-x, protracker, software-properties-gtk (2), termit, torbrowser-launcher, wmaker, w-scan, caveexpress (2), telegram-desktop (2), gtimelog, taglog, gtimer, osmose-emulator (2), rox-filer, fonts-blankenburg, topcat, balsa (2), gnome-multi-writer (4), colorize, wine, mate-menu and mate-tweak

deleted osmose-emulator (6 screenshots of non-free games)

rejected xserver-xorg-video-amdgpu (drivers have no UI), eom (includes image of offensive historical figure), synaptic (includes non-free background image), osmose-emulator (6 screenshots of non-free games) and libomxil-bellagio0-components-xvideo (borked upload)

approved deletion of xserver-xephyr (includes image of non-free OS), perforate (image of the homepage) and openvas/lv2core (not screenshots)

rejected deletion of vkeybd (junk in reason field)

Administration

Debian systems: quiet a logrotate warning, investigate issue with DNSSEC and alioth, deploy fix on our first stretch buildd, restore alioth git repo after history rewrite, investigate iptables segfaults on buildd and investigate time issues on a NAS

Debian derivatives census: delete patches over 5 MiB, re-enable the service

Debian wiki: investigate some 403 errors, fix alioth KGB config, deploy theme changes, close a bogus bug report, ping 1 user with bouncing email, whitelist 9 email addresses and whitelist 2 domains

Debian QA: deploy my changes

Debian mentors: security upgrades and service restarts

Openmoko: debug mailing list issue, security upgrades and reboots

Communication

Invite Wazo to the Debian derivatives census

Welcome ubilinux, Wazo and Roopa Prabhu (of Cumulus Linux) to the Debian derivatives census

Discuss HP/ProLiant wiki page with HPE folks

Inform git history rewriter about the git mailmap feature

Sponsors The libconfig-crontab-perl backports and pyvmomi issue were sponsored by my employer. All other work was done on a volunteer basis.

Russ Allbery: git-pbuilder 1.48

Not clear that anyone gets this from my web site instead of just using the version included in git-buildpackage, but just in case, this release syncs up the version with patches already applied to git-buildpackage (thank you, Guido!). Previous versions did not check for --basepath in the options passed through the environment by gbp buildpackage and hence would ignore some settings. Fixed via a patch by Kevin Locke. Stop removing *_source.changes files after completion of the build, since pbuilder 0.228 and later no longer create bogus changes files with invalid hashes, and the code in git-pbuilder could remove other files that shouldn't be deleted. Patch from Guido G nther. Log the exact pdebuild command ran by the script. Patch from Guido G nther. You can get the latest version from my scripts page.

Chris Lamb: Free software activities in April 2017

Here is my monthly update covering what I have been doing in the free software world (previous month):

I was elected Debian Project Leader for 2017. I'd like to sincerely thank everyone who voted for me as well as everyone who took part in the election in general especially Mehdi Dogguy for being a worthy opponent. The result was covered on LWN, Phoronix, DistroWatch, iTWire, etc.
Added support for the Monzo banking API in social-core, a Python library to allow web applications to authenticate using third-parties. (#68)
Fixed a HTML injection attack in a demo of Russell Keith-Magee's BeeWare presentation library. (#3)
Updated systemd's documentation to explain why we suggest explicitly calling make all despite the Makefile's "check" target calling it. (#5830)
Updated the documentation of a breadth-first version of find(1) called bfs to refer to the newly-uploaded Debian package. (#23)
Updated the configuration for the ticketbot IRC bot (zwiebelbot on OFTC) to identify #reproducible-builds as a Debian-related channel. This is so that bug Debian bug numbers are automatically expanded by the bot. (#7)

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area. This month I:

Presented at foss-north.se 2017 in Gothenburg, Sweden on Reproducible Builds.
Sent a patch for poti a library to generate paje traces to ensure the build is reproducible when Git is not available. (#7)
Submitted a pull request to patat (a tool to make terminal-based presentations using Pandoc) to make the build reproducible. (#36)
Fixed miniupnp's build system to make the build reproducible. (#237)
Updated sunpinyin, a "statistical model" based Chinese input method, to make the .pc file output reproducible. (#73)
taskflow (an OpenStack library) to ensure the generated documentation is reproducible. (#4)
I also submitted 26 patches to fix specific reproducibility issues in avifile, crac, cyclades-serial-client, dactyl, debirf, drumkv1, forked-daapd, foxeye, golang-github-lunny-log, hp-search-mac, libccrtp, miniupnpd, openigtlink, ora2pg, osinfo-db, poti, python-taskflow, qjackctl, qtractor, samplv1, sunpinyin, synthv1, templayer, tf, viruskiller & xmlrpc-c.
Categorised a large number of packages and issues in the Reproducible Builds notes.git repository.
Worked on publishing our weekly reports. (#101, #102, #103 & #104).
Updated our testing framework to also write a .bz2 version of reproducible.json. (#859254)

I also made the following changes to diffoscope, our recursive and content-aware diff utility used to locate and diagnose reproducibility issues:

New features:
- Add support for comparing Ogg Vorbis files. (0436f9b)
Bug fixes:
- Prevent a traceback when using --new-file with containers. (#861286)
- Don't crash on invalid archives; print a useful error instead. (#833697).
- Don't print error output from bzip2 call. (21180c4)
Cleanups:
- Prevent abstraction-level violations by defining visual diff support on Presenter classes. (7b68309)
- Show Debian packages installed in test output. (c86a9e1)

Debian

Patches contributed

msp430mcu: Invalid paths returned from msp430mcu-config
camping: Fix broken font symlink
dh-buildinfo: Please clarify this is unrelated to .buildinfo handling
g2clib: Fix invalid libdir in grib2c.pc
libccrtp: Missing (optional) Build-Depends on graphviz
lintian: Please add "none were" "none was" grammar correction
sugar-memorize-activity: Fix broken icon symlink
sunpinyin: Correct Homepage field

Debian LTS

This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:

"Frontdesk" duties, triaging CVEs, etc.
Issued DLA 882-1 for the tryton-server general application platform to fix a path suffix injection attack.
Issued DLA 883-1 for curl preventing a buffer read overrun vulnerability.
Issued DLA 884-1 for collectd (a statistics collection daemon) to close a potential infinite loop vulnerability.
Issued DLA 885-1 for the python-django web development framework patching two open redirect & XSS attack issues.
Issued DLA 890-1 for ming, a library to create Flash files, closing multiple heap-based buffer overflows.
Issued DLA 892-1 and DLA 891-1 for the libnl3/libnl Netlink protocol libraries, fixing integer overflow issues which could have allowed arbitrary code execution.

Uploads

redis (4:4.0-rc3-1) New upstream RC release.
adminer:
- 4.3.0-2 Fix debian/watch file.
- 4.3.1-1 New upstream release.
bfs:
- 1.0-1 Initial release.
- 1.0-2 Drop fstype tests as they rely on /etc/mtab being available. (#861471)
python-django:
- 1:1.10.7-1 New upstream security release.
- 1:1.11-1 New upstream stable release to experimental.

I sponsored the following uploads:

wolfssl (3.10.2+dfsg-2) Updating debian/copyright (#860046) and disabling the CRL monitor on all architectures (#860514).
python-aniso8601 (1.2.0-1) New upstream release.

I also performed the following QA uploads:

gtkglext (1.2.0-7) Correct installation location of gdkglext-config.h after "Multi-Archification" in 1.2.0-5. (#860007)

Finally, I made the following non-maintainer uploads (NMUs):

python-formencode (1.3.0-2) Don't ship files in /usr/lib/python 2.7,3 /dist-packages/docs. (#860146)
django-assets (0.12-2) Patch pytest plugin to check whether we are running in a Django context, otherwise we can break unrelated testsuites. (#859916)

RC bugs filed

khal: Please reference Reference-license-from-copyright-file.patch in debian/copyright
libbsd-dev: Trying to overwrite 'explicit_bzero.3.gz', which is also in manpages-dev
node-diffie-hellman: Please clarify security concerns

I also filed 2 bugs for packages that access the internet during build (against fail2ban & ruby-rack-proxy). I also filed 11 FTBFS bugs against bup, golang-github-lunny-nodb, hunspell-dict-ko, icinga-web, nanoc, oggvideotools, polygen, python-dogpile.cache, reapr, tendermint-go-merkle & z88.

FTP Team

As a Debian FTP assistant I ACCEPTed 155 packages: aiohttp-cors, bear, colorize, erlang-p1-xmpp, fenrir, firejail, fizmo-console, flask-ldapconn, flask-socketio, fontmanager.app, fonts-blankenburg, fortune-zh, fw4spl, fzy, gajim-antispam, gdal, getdns, gfal2, gmime, golang-github-go-macaron-captcha, golang-github-go-macaron-i18n, golang-github-gogits-chardet, golang-github-gopherjs-gopherjs, golang-github-jroimartin-gocui, golang-github-lunny-nodb, golang-github-markbates-goth, golang-github-neowaylabs-wabbit, golang-github-pkg-xattr, golang-github-siddontang-goredis, golang-github-unknwon-cae, golang-github-unknwon-i18n, golang-github-unknwon-paginater, grpc, grr-client-templates, gst-omx, hddemux, highwayhash, icedove, indexed-gzip, jawn, khal, kytos-utils, libbloom, libdrilbo, libhtml-gumbo-perl, libmonospaceif, libpsortb, libundead, llvm-toolchain-4.0, minetest-mod-homedecor, mini-buildd, mrboom, mumps, nnn, node-anymatch, node-asn1.js, node-assert-plus, node-binary-extensions, node-bn.js, node-boom, node-brfs, node-browser-resolve, node-browserify-des, node-browserify-zlib, node-cipher-base, node-console-browserify, node-constants-browserify, node-delegates, node-diffie-hellman, node-errno, node-falafel, node-hash-base, node-hash-test-vectors, node-hash.js, node-hmac-drbg, node-https-browserify, node-jsbn, node-json-loader, node-json-schema, node-loader-runner, node-miller-rabin, node-minimalistic-crypto-utils, node-p-limit, node-prr, node-sha.js, node-sntp, node-static-module, node-tapable, node-tough-cookie, node-tunein, node-umd, open-infrastructure-storage-tools, opensvc, openvas, pgaudit, php-cassandra, protracker, pygame, pypng, python-ase, python-bip32utils, python-ltfatpy, python-pyqrcode, python-rpaths, python-statistics, python-xarray, qtcharts-opensource-src, r-cran-cellranger, r-cran-lexrankr, r-cran-pwt9, r-cran-rematch, r-cran-shinyjs, r-cran-snowballc, ruby-ddplugin, ruby-google-protobuf, ruby-rack-proxy, ruby-rails-assets-underscore, rustc, sbt, sbt-launcher-interface, sbt-serialization, sbt-template-resolver, scopt, seqsero, shim-signed, sniproxy, sortedcollections, starjava-array, starjava-connect, starjava-datanode, starjava-fits, starjava-registry, starjava-table, starjava-task, starjava-topcat, starjava-ttools, starjava-util, starjava-vo, starjava-votable, switcheroo-control, systemd, tilix, tslib, tt-rss-notifier-chrome, u-boot, unittest++, vc, vim-ledger, vis, wesnoth-1.13, wolfssl, wuzz, xandikos, xtensor-python & xwallpaper. I additionally filed 14 RC bugs against packages that had incomplete debian/copyright files against getdns, gfal2, grpc, mrboom, mumps, opensvc, python-ase, sniproxy, starjava-topcat, starjava-ttools, unittest++, wolfssl, xandikos & xtensor-python.

Matthew Garrett: Looking at the Netgear Arlo home IP camera

Another in the series of looking at the security of IoT type objects. This time I've gone for the Arlo network connected cameras produced by Netgear, specifically the stock Arlo base system with a single camera. The base station is based on a Broadcom 5358 SoC with an 802.11n radio, along with a single Broadcom gigabit ethernet interface. Other than it only having a single ethernet port, this looks pretty much like a standard Netgear router. There's a convenient unpopulated header on the board that turns out to be a serial console, so getting a shell is only a few minutes work.

Normal setup is straight forward. You plug the base station into a router, wait for all the lights to come on and then you visit arlo.netgear.com and follow the setup instructions - by this point the base station has connected to Netgear's cloud service and you're just associating it to your account. Security here is straightforward: you need to be coming from the same IP address as the Arlo. For most home users with NAT this works fine. I sat frustrated as it repeatedly failed to find any devices, before finally moving everything behind a backup router (my main network isn't NATted) for initial setup. Once you and the Arlo are on the same IP address, the site shows you the base station's serial number for confirmation and then you attach it to your account. Next step is adding cameras. Each base station is broadcasting an 802.11 network on the 2.4GHz spectrum. You connect a camera by pressing the sync button on the base station and then the sync button on the camera. The camera associates with the base station via WPS and now you're up and running.

This is the point where I get bored and stop following instructions, but if you're using a desktop browser (rather than using the mobile app) you appear to need Flash in order to actually see any of the camera footage. Bleah.

But back to the device itself. The first thing I traced was the initial device association. What I found was that once the device is associated with an account, it can't be attached to another account. This is good - I can't simply request that devices be rebound to my account from someone else's. Further, while the serial number is displayed to the user to disambiguate between devices, it doesn't seem to be what's used internally. Tracing the logon traffic from the base station shows it sending a long random device ID along with an authentication token. If you perform a factory reset, these values are regenerated. The device to account mapping seems to be based on this random device ID, which means that once the device is reset and bound to another account there's no way for the initial account owner to regain access (other than resetting it again and binding it back to their account). This is far better than many devices I've looked at.

Performing a factory reset also changes the WPA PSK for the camera network. Newsky Security discovered that doing so originally reset it to 12345678, which is, uh, suboptimal? That's been fixed in newer firmware, along with their discovery that the original random password choice was not terribly random.

All communication from the base station to the cloud seems to be over SSL, and everything validates certificates properly. This also seems to be true for client communication with the cloud service - camera footage is streamed back over port 443 as well.

Most of the functionality of the base station is provided by two daemons, xagent and vzdaemon. xagent appears to be responsible for registering the device with the cloud service, while vzdaemon handles the camera side of things (including motion detection). All of this is running as root, so in the event of any kind of vulnerability the entire platform is owned. For such a single purpose device this isn't really a big deal (the only sensitive data it has is the camera feed - if someone has access to that then root doesn't really buy them anything else). They're statically linked and stripped so I couldn't be bothered spending any significant amount of time digging into them. In any case, they don't expose any remotely accessible ports and only connect to services with verified SSL certificates. They're probably not a big risk.

Other than the dependence on Flash, there's nothing immediately concerning here. What is a little worrying is a family of daemons running on the device and listening to various high numbered UDP ports. These appear to be provided by Broadcom and a standard part of all their router platforms - they're intended for handling various bits of wireless authentication. It's not clear why they're listening on 0.0.0.0 rather than 127.0.0.1, and it's not obvious whether they're vulnerable (they mostly appear to receive packets from the driver itself, process them and then stick packets back into the kernel so who knows what's actually going on), but since you can't set one of these devices up in the first place without it being behind a NAT gateway it's unlikely to be of real concern to most users. On the other hand, the same daemons seem to be present on several Broadcom-based router platforms where they may end up being visible to the outside world. That's probably investigation for another day, though.

Overall: pretty solid, frustrating to set up if your network doesn't match their expectations, wouldn't have grave concerns over having it on an appropriately firewalled network.

(Edited to replace a mistaken reference to WDS with WPS)

comments

29 April 2017

Shirish Agarwal: India and the Agricultural Economy

I was in two minds when I read Ritesh s blog post about the Indian Economy. I was angry with Ritesh as he seemed to selectively take facts and present it rather than taking it whole. Even if he had searched even a little bit, he would have got much more better material and everybody would have been the gainer. I have to also admit, I feel very much like a hypocrite as I have never slaved in a farm so my understanding and conclusions are a mix of media and limited interaction with farmers some years ago. There is also lots of local customs and politics that come into the picture and it s not as straight-forward as Ritesh thinks. What he has failed to share/account for is the far worse bad and stressed debts for the industry so just saying farm loan waivers are bad without sharing any of the context makes it seem much more worse. This is when our Current Chief Economic Adviser states about loan waivers to corporates You need to be able to forgive those debts because this is how capitalism works. People make mistakes, those have to be forgiven to some extent Let me start though with words from a book I read sometime back On a peasant uprising erupted in . The farmers were angry with high interest rates, high taxes, high inflation and low-government prices for their crops. The system had let them into debt, and debt had meant foreclosure and loss of their fields to the land barons. I intentionally have made a fill in the blanks as both the dates and places was true in India 100 years back and even today, the only difference between the two is the absence of taxes. Many people would think I m talking about Champaran whose tale while well-known in India is sadly a stub-class article in wikipedia

with quite a few citation needed tags as well

but is also true today as will be seen below. Interestingly, there is/was a remark by some unknown person who said gora sahab gays, bhura sahab aaya meaning the white officer has gone, in his place the brown officer has come. The evidence of this is very much in the Telegraph Act and the story about its usage and its place in Indian politics Surprisingly, sadly and coincidentally, the quote minus the dates and place didn t happen in India but also in Cambodia. The above quote has been taken from for the sake of all living things by John M. Del Vecchio. The quote itself appears in the first 10 odd pages (historical summation) of the somewhat 1200 odd pages book. I actually got an old edition which tops out at 900 pages so probably some more updated input/news isn t available but it still packs a powerful wallop. I want to dedicate a separate blog post for the book itself so will not say more on that book and what it shares. Sadly and coincidentally, there were news reports yesterday itself of farmers agitating for better prices just yesterday. Some of the interesting work if you want to understand the farmer s indebtedness is to study the Income, Expenditure, Productive Assets and Indebtedness of Agricultural Households in India done by NSSO. Again, one does not need to read the whole report, there were some of the analysis shared by the Hindu here and here. This was also echoed by Logical Indian It really boggles the mind to know than an average farming household earns around INR 200 per day. Even if you take a family of four people that comes to INR 50/- per person. Most rural joint families at the very least have 3-4 kids at the very least. Sadly Agricultural incomes do not keep sync even with the inflation index as there is no fair minimum age and wage for the Indian farmer, the concept does not exist for her(im)

. Just me and mum going onto a restaurant and having one dish each easily can run anywhere between INR 200~250 easily . Cooking in the house is the same if you add/input the labor (which is usually not calculated) used to make lunch/dinner. There was the idea that contract farming might be a solution but even that was corrupted by Multi-national companies such as Pepsi and others that the Government is showing movement to have a model contracting law . There are loads of stories on downtoearth magazine which deals with the above and all sorts of issues the farmer faces. I should talk about Maharashtra and even here there was/has been an irrigation scam with figures given from 35k to 70k crores or 350 to 700 billion INR . The latest finding by the PAC has been shared here. I will cut the blog post short as I find the whole thing personally very depressing. As far as local customs go this was from one of the farmers interaction some years ago where me and some other friends had gone across a village and came to know that all of them grew the same crop with some minor variances. When asked why is it so, while many said its fate, one of the elderly gentleman shared an experience where a farmer had planted some other thing. The gentleman prospered while the other villagers were suffering from glut of whatever they produced. Knowing he prospered, the other villagers damaged his crops and all sorts of unlucky things started happening for the farmer. In the end he realized his best bet is to follow the ways of the other villagers, at least they would be in peace. What I have shared isn t either unique or even unknown, even Toronto Star of Canada reported on the issue some years ago. At the end of it all, the story is one of no education and limited skill-set and I don t see it changing any time soon. There are some who are earning big figures, but majority of the farmers will always be in the red
Filed under: Miscellenous Tagged: #Agricultal Economy, #Champaran, #Contract farming, #Corruption, #farmer suicide, #irrigation scam, #Loan waivers, #planet-debian, poverty

Antoine Beaupr : My free software activities, April 2017

Debian Long Term Support (LTS) This is my monthly Debian LTS report. My time this month was spent working on various hairy security issues, most notably XBMC (now known as Kodi) and yaml-cpp.

Kodi directory transversal I started by looking in CVE-2017-5982, a "directory traversal" vulnerability in XBMC (now known as Kodi) which is a technical term for "allow attackers to read any world-readable file on your computer from the network". It's a serious vulnerability which has no known fix. When you enable the "remote control" interface in Kodi, it allows anyone with the password (which is disabled by default) to download any files Kodi has read access to on the machine it's running. Considering Kodi is often connected to multiple services, this may mean elevated compromise and more nasty stuff. I furthered the investigation done with my own analysis which showed the problem is difficult to solve: Kodi internally uses the facility to show thumbnails and media to the user, and there are no clear way of restricting which paths Kodi should have access to. Indeed, Kodi is designed to access mounted file systems and paths in arbitrary locations. In Debian bug #855225, I further confirmed confirmed wheezy and jessie-backports as vulnerable and therefore showed with good certainty that stretch and sid are vulnerable as well. I also suggested possible workaround, but at this point, it's in upstream's hands, as the changes will be intrusive. The file transfer mechanism need to be revamped all over Kodi, or authentication (with a proper password policy), need to be enforced.

Squirrelmail Next I looked at that old webmail software, Squirrelmail, which suffers from a remote code execution vulnerability (CVE-2017-7692) when sending mails with sendmail on the commandline. This is arguably an edge case, but considering the patch was simple, I figured I would provide an update to the LTS community. I tried to get a coordinated release for jessie, since the code is the same, but this wasn't completed at the time of writing. A patch is available and will hopefully be picked up by another LTS worker soon.

Fop and Batik Those issues (CVE-2017-5661 and CVE-2017-5662) were more difficult. The patches weren't clearly documented and there were no upstream references other than security advisories for the first release in years (in the case of batik) or months (in the case of fop), which made it hard to track down the issues. Fortunately, I was able to track down the upstream issues (FOP-2668 and BATIK-1139) where I got confirmation on what the proper fixes were. I could then release DLA-927-1 and DLA-926-1 with the backported patches. I do not use fop or batik. In fact, even after reading the homepage of both products, I couldn't quite figure out what use people could possibly have for that thing. Before uploading the packages, I therefore made packages available for testing for fop and batik.

libsndfile Next up was libsndfile which a bunch of overflows when parsing various audio files. I backported a patch for CVE-2017-7585 CVE-2017-7586 and CVE-2017-7741 which all seemed to be fixed by a single patch usptream. CVE-2017-7742 was also fixed, although with a separate patch. In all of those, i could only test CVE-2017-7741 and CVE-2017-7742, as the others were missing test cases. I provided a test package for a few days then I also figured it would be best to incorporate the security fixes done in stable, which brought in fixes for CVE-2015-7805, CVE-2014-9756 and CVE-2014-9496. So in the end, I ported patches from wheezy to jessie and uploaded the jessie version (reverting certain build changes) into wheezy and uploaded DLA-928-1 with the results.

yaml-cpp I then turned to yaml-cpp, a C++ parser for YAML. This one didn't have a known upstream fix, but I figured I would give it a shot anyways. I ended up writing my first C++ code in years which is still pending review and merge upstream. It's not an easy problem to fix: this is basically an excessive recursion problem that can be used to smash the stack. I figured I could introduce a recursion limit, but as the discussion showed, this is a limited approach: stack size varies on different platforms and it's not easy to find the right limit. The real solution is to rewrite the code to avoid recursion but that's a major code refactoring I didn't feel belong in a LTS update. Besides, this could be better handled by upstream, so I will leave things at that for now. It does make you wonder how much code out there is recursing on untrusted data structures...

kedpm Finally, a friend over at Koumbit.org reported Debian bug #860817, as information leak in kedpm, a password manager I previously maintained. I requested and got assigned CVE-2017-8296 and provided a fix for wheezy and jessie. For unstable and the coming stretch release, I have requested kedpm to be completely removed from Debian (Debian bug #860817) which involved a release notes update (Debian bug #861277). It's unfortunate to see software go, but kedpm wasn't maintained. I wasn't the original author: I just gave a few patches and ended up maintaining that software and not using it. It's a bad situation to be in, as you don't really know what's working and not with the tools you are supposed to be responsible for. There are more modern alternatives available now and I encourage everyone to switch.

Triage Looking for more work, I peeked a bit in the secretary tasks to triage some pending issues. I found that trafficserver could be crashed with simple requests (CVE-2017-5659) so I looked into that issue. My analysis showed that the patch is long and complex and could be difficult to backport to the old version available in wheezy. I also couldn't reproduce the issue in wheezy, so it may be a bug introduced only later, although I couldn't confirm that directly. I also triaged wireshark, where I just noted the maintainer expressed concern that we were taking up issues too fast and will probably take care of this one. I also postponed various issues in GnuTLS (marked "no-dsa") as they affect only a (unfortunately) rarely used part of GnuTLS that has been removed in later version: OpenPGP support.

Other free software work

Debiman I finally got around contributing to the debiman project. I worked on ensuring that there is a dman compatibility in debiman, by shipping dman in the debian-goodies package (Debian bug #860920). I also submitted a pull request to fix the fix about page title, document the custom assets repository, fix a stray bracket and link to the link to venerable man7.org project After a discussion on IRC, I also filed a few more issues:

redirect bpf to bpf.2, not bpf.4freebsd

redirect to unstable by default

arrows missing in table of contents

automatically collapse long links

old ?query= parameter ignored/failing

I'm happy to be able to contribute to this important service and I hope the new FAQ I created will be online soon!

XMonad and Emacs I started using writeroom-mode again as part of my work on LWN. As it turns out, my setup was not exactly working: I had to port my config to the new version and windows weren't "sticky" as they should be, a known issue with Xmonad. Indeed, Xmonad doesn't obey the "static" or "all desktops" standard directives. Writeroom is a "distraction-free writing" mode for Emacs, so the irony of working on such a deep distraction in establishing a distraction free environment is not lost on me. Needing to scratch that particular itch, and with the help of clever people from the IRC channel, I was able to make Emacs tell Xmonad to show its window (or "frame" as Emacs likes to call it) on all desktops. This involved creating a new function which I think could be useful in the CopyWindow library:

--   Toggle between "copyToAll" or "killAllOtherCopies". Copies to all
-- workspaces, or remove from all other workspaces, depending on
-- previous state (checked with "wsContainingCopies").
copyToAllToggle :: X ()
copyToAllToggle = do
    -- check which workspaces have copies
    copies <- wsContainingCopies
    if null copies
      then windows copyToAll -- no workspaces, make sticky
      else killAllOtherCopies -- already other workspaces, unstick

There are probably better ways of implementing this directly in the CopyWindow code - wsContainingCopies, in particular, is probably overkill. But it's all I can use directly from my xmonad.hs, so that's what I did. The other bit I needed was something to trigger that function from the outside. I rejected the ServerMode hook because it looked a bit too complicated and there is a built-in facility within X that works without this, which, from Emacs' point of view, is the x-send-client-message function. So I made up a new message identifier and wrote a event hook handler to process it:

--   handle X client messages that tell Xmonad to make a window appear
-- on all workspaces
--
-- this should really be using _NET_WM_STATE and
-- _NET_WM_STATE_STICKY. but that's more complicated: then we'd need
-- to inspect a window and figure out the current state and act
-- accordingly. I am not good enough with Xmonad to figure out that
-- part yet.
--
-- Instead, just check for the relevant message and check if the
-- focused window is already on all workspaces and toggle based on
-- that.
--
-- this is designed to interoperate with Emacs's writeroom-mode module
-- and called be called from elisp with:
--
-- (x-send-client-message nil 0 nil "XMONAD_COPY_ALL_SELF" 8 '(0))
myClientMessageEventHook :: Event -> X All
myClientMessageEventHook (ClientMessageEvent  ev_message_type = mt, ev_data = dt ) = do
  dpy <- asks display
  -- the client message we're expecting
  copyAllMsg <- io $ internAtom dpy "XMONAD_COPY_ALL_SELF" False
  -- if the event matches the message we expect, toggle sticky state
  when (mt == copyAllMsg && dt /= []) $ do
    copyToAllToggle
  -- we processed the event completely
  return $ All True

All that was left was to hook that into Emacs, and I was done! Whoohoo! Full screen total domination, distraction free work!

I would love to hear from others what they think of that approach, if they have improvements or if the above copyToAllToggle function could be merged in. Ideally, Xmonad would just parse the STICKY client messages and do the right thing - maybe even directly in CopyWindow - but I have found this enough Haskell for one day. You can see the diff on my home directory to see exactly the changes involved to make this configuration work.

Emacs packaging Speaking of Emacs, after complaining in the noisy `#emacs` IRC channel about the poor TLS configuration of marmelade.org -- and filing a bug (Debian bug #861106) regarding the use of SHA-1 in certificate pinning -- I was told we shouldn't expect trust from third-party ELPA repositories. Marmelade seems to be dead, as the maintainer is "behind the great firewall of China" and MELPA still hasn't figured out how to sign packages. In the end, it seems like there are tons of elpa packages in Debian and that if your favorite one is missing, that's a bug that can be filed and fixed. I first discovered that 6 of the packages I used were already packaged:

elpa-notmuch

elpa-ledger

elpa-use-package

elpa-markdown-mode

elpa-go-mode

elpa-anzu

And so I went ahead and filed a ton more bugs for the packages I am using but that aren't in Debian just yet:

company-go: Debian bug #861177

elpy: Debian bug #861174

markdown-toc: Debian bug #861128

multiple-cursors: Debian bug #861127

writegood-mode: Debian bug #861125

writeroom-mode: Debian bug #861124

Of those, I can't recommend multiple-cursors (MC) enough: I used it at least 4 times just writing this text. It's just awesome. The other ones are also all great in their own right of course, but I feel they are more specific to my workflow whereas MC is just amazing.

ikiwiki I did some more work on ikiwiki the software driving this blog. I created a new plugin to, at least, fix anchors in the table of contents to be human readable. This is something I had done on the MoinMoin wiki almost a decade ago -- which I called then NicerHeadingIds and that I have always found frustrating with Ikiwiki. It turns out the problem was both easier and hairier than I thought. Right from the start, something weird was happening: something was already adding nice headings, but they were somewhat broken. It turns out that multimarkdown already inserts those headers, but I wasn't satisfied with the way they were generated. But even worse, I had the headinganchors plugin enabled, but that plugin wasn't taking effect, because of multimarkdown. And even if it would take effect, it doesn't behave well with non-ASCII characters, which gets turned in their numeric presentation. So I also wrote the i18nheadinganchors plugin that creates better headings and patched the toc plugin so that it can reuse existing anchors if they exist, while keeping backwards compatibility. I hope this gets merged in a future ikiwiki release so I do not have to carry this patch locally too long... In other news, I have upgraded the ikiwiki-hosting package to the latest version and sent a patch upstream to provide HSTS support.

Other stuff I have migrated all my public repos hosted on my home server to either Gitlab.com or Github. I also have repositories on 0xacab and it seemed ludicrous to have 4 different, canonical, places where my code was hosted. I have now about 40 different projects on Gitlab and about 60 on Github, although most of the latter are forks of existing projects. I also made a manpage for stressant and moved the documentation to RTFD which makes it neatly accessible. I also made small incremental improvements (like `--directory` support). I installed Rainloop on this server to give a nice, mobile-friendly webmail. Instructions to replicate this setup are in mail. In the constant git-annex documentation effort, I tried to draft a user guide that could be a basis for restructuring the documentation to be more easily accessible. I also helped a friend put his documentation on the wiki in splitting a repository. Finally, I also looked into Android stuff a little more. I wrote a usability review of the F-Droid privileged extension that will bring good changes, I hope. I also opened the discussion regarding reproducible builds to try and clarify exactly how those worked to help the Wallabag people ship consistently signed alphas. So far, it seems that it will remain a standard practice on F-Droid to ship packages that are not signed by the official upstream signature, unfortunately, unless upstream provides a reproducible build that is publicly available... Switching to such build is also a hairy issue, as, obviously, the signature changes, which raises the alarm we are trying to avoid in the first place.

Russ Allbery: Review: Neverness

Review: Neverness, by David Zindell

Publisher:	Bantam Spectra
Copyright:	May 1988
Printing:	July 1989
ISBN:	0-553-27903-3
Format:	Mass market
Pages:	552

Mallory Ringess is a Pilot, one of the people who can guide a lightship through interstellar space from inside the dark cocoon and biotech interface that allows visualization of the mathematics of interstellar travel. At the start of the book, he's young, arrogant, impulsive, and has a deeply unhealthy relationship with Leopold Soli, the Lord Pilot and supposedly his uncle by marriage (although they share a remarkable physical resemblance). An encounter with his uncle in a bar provokes a rash promise, and Ringess finds himself promising to attempt to map the Solid State Entity in search of the Elder Eddas, a secret of life from the mythical Ieldra that might lead to mankind's immortality. The opening of Neverness is Ringess's initial voyage and brash search, in which he proves to be a capable mathematician who can navigate a region of space twisted and deformed by becoming part of a transcendent machine intelligence. The knowledge he comes away with, though, is scarcely more coherent than the hints Soli relates at the start of the story: the secret of mankind is somehow hidden in its deepest past. That, in turn, provokes a deeply bizarre trip into the ice surrounding his home city of Neverness to attempt to steal biological material from people who have recreated themselves as Neanderthals. Beyond that point, I would say that things get even weirder, but weird still implies some emotional connection with the story. I think a more accurate description is that the book gets more incoherently mystical, more hopelessly pretentious, and more depressingly enthralled by childish drama. It's the sort of thing that one writes if one is convinced that the Oedipal complex is the height of subtle characterization. I loathed this book. I started loathing this book partway through Ringess's trip through the Solid State Entity, when Zindell's prose reached for transcendent complexity, tripped over its own shoelaces, and fell headlong into overwrought babbling. I continued reading every page because there's a perverse pleasure in hate-reading a book one dislikes this intensely, and because I wanted to write a review on the firm foundation of having endured the entire experience. The paperback edition I have has a pull quote from Orson Scott Card on the cover, which includes the phrase "excellent hard science fiction." I'm not sure what book Card read, because if this is hard science fiction, Lord of the Rings is paranormal romance. Even putting aside the idea that one travels through interstellar space by proving mathematical theorems in artificially dilated time (I don't think Zindell really understands what a proof is or why you write one), there's the whole business with stopping time with one's mind, reading other people's minds, and remembering one's own DNA. The technology, such as it is, makes considerably less sense than Star Wars. The hard SF requirement to keep technology consistent with extrapolated science is nowhere to be found here. The back-cover quote from the St. Louis Post-Dispatch is a bit more on-target: "Reminiscent of Gene Wolfe's New Sun novels... really comes to life among the intrigues of Neverness." This is indeed reminiscent of Gene Wolfe, in that it wouldn't surprise me at all if Zindell fell in love with the sense of antiquity, strangeness, and hints of understood technology that Wolfe successfully creates and attempted to emulate Wolfe in his first novel. Sadly, Zindell isn't Wolfe. Almost no one is, which is why attempting to emulate the extremely difficult feat Wolfe pulls off in the Book of the New Sun in your first novel is not a good idea. The results aren't pretty. There is something to be said for resplendent descriptions, rich with detail and ornamental prose. That something is "please use sparingly and with an eye to the emotional swings of the novel." Wolfe does not try to write most of a novel that way, which is what makes those moments of description so effective. Wolfe is also much better at making his mysteries and allusions subtle and unobtrusive, rather than having the first-person protagonist beat the reader over the head with them for pages at a time. This is a case where showing is probably better than telling. Let me quote a bit of description from the start of the book:

She shimmers, my city, she shimmers. She is said to be the most beautiful of all the cities of the Civilized Worlds, more beautiful even than Parpallaix or the cathedral cities of Vesper. To the west, pushing into the green sea like a huge, jewel-studded sleeve of city, the fragile obsidian cloisters and hospices of the Farsider's Quarter gleamed like black glass mirrors. Straight ahead as we skated, I saw the frothy churn of the Sound and their whitecaps of breakers crashing against the cliffs of North Beach and above the entire city, veined with purple and glazed with snow and ice, Waaskel and Attakel rose up like vast pyramids against the sky. Beneath the half-ring of extinct volcanoes (Urkel, I should mention, is the southernmost peak, and though less magnificent than the others, it has a conical symmetry that some find pleasing) the towers and spires of the Academy scattered the dazzling false winter light so that the whole of the Old City sparkled.

That's less than half of that paragraph, and the entire book is written like that, even in the middle of conversations. Endless, constant words piled on words about absolutely everything, whether important or not, whether emotionally significant or not. And much of it isn't even description, but philosophical ponderings that are desperately trying to seem profound. Here's another bit:

Although I knew I had never seen her before, I felt as if I had known her all my life. I was instantly in love with her, not, of course, as one loves another human being, but as a wanderer might love a new ocean or a gorgeous snowy peak he has glimpsed for the first time. I was practically struck dumb by her calmness and her beauty, so I said the first stupid thing which came to mind. "Welcome to Neverness," I told her.

Now, I should be fair: some people like this kind of description, or at least have more tolerance for it than I do. But that brings me to the second problem: there isn't a single truly likable character in this entire novel. Ringess, the person telling us this whole story, is a spoiled man-child, the sort of deeply immature and insecure person who attempts to compensate through bluster, impetuousness, and refusing to ever admit that he made a mistake or needed to learn something. He spends a good portion of the book, particularly the deeply bizarre and off-putting sections with the fake Neanderthals, attempting to act out some sort of stereotyped toxic masculinity and wallowing in negative emotions. Soli is an arrogant, abusive asshole from start to finish. Katherine, Ringess's love interest, is a seer who has had her eyes removed to see the future (I cannot express how disturbing I found Zindell's descriptions of this), has bizarre and weirdly sexualized reactions to the future she never explains, and leaves off the ends of all of her sentences, which might be be the most pointlessly irritating dialogue quirk I've seen in a novel. And Ringess's mother is a man-hating feminist from a separatist culture who turns into a master manipulator (I'm starting to see why Card liked this book). I at least really wanted to like Bardo, Ringess's closest friend, who has a sort of crude loyalty and unwillingness to get pulled too deep into the philosophical quicksand lurking underneath everything in this novel. Alas, Zindell insists on constantly describing Bardo's odious eating, belching, and sexual habits every time he's on the page, thus reducing him to the disgusting buffoon who gets drunk a lot and has irritating verbal ticks. About the only person I could stand by the end of the book was Justine, who at least seems vaguely sensible (and who leaves the person who abuses her), but she's too much of a non-entity to carry sustained interest. (There is potential here for a deeply scathing and vicious retelling of this story from Justine's point of view, focusing on the ways she was belittled, abused, and ignored, but I think Zindell was entirely unaware of why that would be so effective.) Oh, and there's lots of gore and horrific injury and lovingly-described torture, because of course there is. And that brings me back to the second half of that St. Louis Post-Dispatch review quote: "... really comes to life among the intrigues of Neverness." I would love to know what was hiding behind the ellipses in this pull quote, because this half-sentence is not wrong. Insofar as Neverness has any real appeal, it's in the intrigues of the city of Neverness and in the political structure that rules it. What this quote omits is that these intrigues start around page 317, more than halfway through the novel. That's about the point where faux-Wolfe starts mixing with late-career Frank Herbert and we get poet-assassins, some revelations about the leader of the Pilot culture, and some more concrete explanations of what this mess of a book is about. Unfortunately, you have to read through the huge and essentially meaningless Neanderthal scenes to get there, scenes that have essentially nothing to do with the interesting content of this book. (Everything that motivates them turns out to be completely irrelevant to the plot and useless for the characters.) The last 40% of the book is almost passable, and characters I cared about might have even made it enjoyable. Still, a couple of remaining problems detract heavily, chief among them the lack of connection of the great revelation of the story to, well, anything in the story. We learn at the very start of the novel that the stars of the Vild are mysteriously exploding, and much of the novel is driven by uncovering an explanation and solution. The characters do find an explanation, but not through any investigation. Ringess is simply told what is happening, in a wad of exposition, as a reward for something else entirely. It's weirdly disconnected from and irrelevant to everything else in the story. (There are some faint connections to the odd technological rules that the Pilot society lives under, but Zindell doesn't even draw attention to those.) The political intrigue in Neverness is similar: it appears out of nowhere more than halfway through the book, with no dramatic foundation for the motives of the person who has been keeping most of the secrets. And the final climax of the political machinations involves a bunch of mystical nonsense masquerading as science, and more of the Neanderthal bullshit that ruins the first half of the book. This is a thoroughly bad book: poorly plotted, poorly written, clotted and pretentious in style, and full of sociopaths and emotionally stunted children. I read the whole thing because I'm immensely stubborn and make poor life choices, but I was saying the eight deadly words ("I don't care what happens to these people") by a hundred pages in. Don't emulate my bad decisions. (Somehow, this novel was shortlisted for the Arthur C. Clarke award in 1990. What on earth could they possibly have been thinking?) Neverness is a stand-alone novel, but the ending sets up a subsequent trilogy that I have no intention of reading. Followed by The Broken God. Rating: 2 out of 10

27 April 2017

Joey Hess: Exede Surfbeam 2

My new satellite internet connection is from Exede, connecting to the ViaSat 1 bird in geosync orbit. A few technical details that I've observed follow. antagonistic by design The "Surfbeam 2 wifi modem" is a closed proprietary system. That is important because it's part of Exede's bandwidth management system. The Surfbeam tracks data use and sends it periodically to Exede. When a user has gone over their monthly priority data, Exede then throttles the bandwidth in various ways -- this throttling seems to be implemented, at least partially on the Surfbeam itself. (Perhaps by setting QoS flags?) So, if a user could hack their Surfbeam, they could probably bypass the bandwidth caps, or at least some of them. Perhaps Exede would notice eventually. Of course, doing so would surely violate the Exede TOS. If you're renting the modem, like I am, hacking a device you don't own might also subject you to criminal penalties. Needless to say, I don't plan to hack the SurfBeam. But it's been hacked before. So, this is a device that lives in people's homes and is antagonistic to them by design. weird data throttling The way the Surfbeam reports data use back to Exede periodically and gets throttling configured has some odd effects sometimes. For example, the Surfbeam can be in throttled state left-over from the previous billing month. When a new billing month begins, it can remain throttled for some time (up to multiple hours) until it sends an update to Exede and they un-throttle it. Data downloaded at that time might still be counted as priority data even though it was throttled. I've seen some good indications of that happening, but am not sure yet. But, I've decided that the throttling doesn't matter for me. Why? ViaSat 1 has many spot beams, and the working-class beam I'm in (most of it is in eastern Kentucky) does not seem to get a lot of use between 7 am and 4:30 pm weekdays. Even when throttled, I often get 300 kb/s - 1 mb/s speeds during the day, which is not a lot worse than the ~2.5 mb/s peak when unthrottled. And that's the time when I want to use broadband -- when the sun is shining and I'm at home at work/play. I'm probably going to switch to a cheaper plan with less priority data, because the priority data is not buying me much. This is a big change from the old FAP which rendered the satellite no faster than dialup. a whole network in there Looking at the ports open on the Surfbeam, some very strange things turned up. First, there are not one, not two, but three separate IPs used by the device, and there are at least two and perhaps three distinct computers involved. There are a lot of flickering LEDs inside the box; a whole network in there. 192.168.100.1 is the satellite controller. It's a Linux box, fingerprinted as kernel 3.10 or so (so full of security holes presumably), and it's running thttpd/2.25b (doesn't seem to have any known holes). It seems to have ssh and snmp, but with some port filtering that prevents access. (Note that the above exploit video confirms that snmp is running.) Some machine parsable data is exposed at http://192.168.100.1/index.cgi?page=modemStatusData and http://192.168.100.1/index.cgi?page=triaStatusData. (See (SurfStat program) 192.168.1.1 is the wifi router. It has a dns server, an icslap proxy, and nmap thinks it's Linux 3.x with Synology DiskStation Manager (probably the latter is a false positive?) It has its own separate web server for configuration, which is not thttpd. I'm fairly sure this is a separate processor from the other IP address. 192.168.100.2 responds to ICMP, but has no open ports at all. However, it seems to have filtered ssh, telnet, msrpc, microsoft-ds, and port 8090 (probably http), so perhaps it's running all that stuff. This one is definitely a separate processor, located in the Satellite dish's TRIA (transmit receive integrated assembly). Verified by disconnecting the dish's coax cable and being unable to ping it. lack of source code for GPLed software Exede did not provide anything to me about the GPL licensed source code on the Surfbeam 2. I'm not sure if they're legally required to do so in my case, since they're renting it to me? But, they do let you buy the device too, so it's interesting that nothing mentions the licenses of its software and where to get the source code. I'll try to buy it and ask for the source code eventually. power use The Surfbeam pulls as much power as two beefy laptops, but it is beaming signals thousands of miles into space, so I give it a bit of a pass. I want to find a way to power it from DC power, but have not had any success with that so far, so am wasting more power to run it on an inverter. Its power supply is rated at 30v and 2.5 amps. Interestingly, I've seen a photo of another Surfbeam power supply that only pulls 1.5 amps. This and a different case with fewer (external) LEDs makes me think perhaps the installer stuck me with an old and less efficient version. Maybe they integrated some of those processors into a single board in a new version and perhaps made it waste less power as heat. Mine gets pretty warm. I'm currently only able to run it approximately one hour per hour of sun collected by the house. Still on dialup the rest of the time. With the ability to get on broadband when dialup is being painful, being on dialup the rest of the time is perfectly ok. Of course it helps that with tools like git-annex, I'm very well tuned to popping up onto broadband briefly and making it count.

Next.

Previous.