Command line is not complete. Try option "help"

 root@smaug ~ # facter --debug
 Found no suitable resolves of 1 for ec2_metadata
 value for ec2_metadata is still nil
 value for netmask_git is still nil
 value for ipaddress6_lo is still nil
 value for macaddress_lo is still nil
 value for ipaddress_master is still nil
 value for ipaddress6_master is still nil
 Command line is not complete. Try option "help"
 value for netmask_master is still nil
 value for ipaddress_skx_mail is still nil
 ..

 ip link show $interface

  $ ip link show git
  6: git: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT group default qlen 1000

  $ ip link show master
  Command line is not complete. Try option "help"

  ethbond = regex.match(%x /sbin/ip link show '# interface ' )

  ethbond = regex.match(%x /sbin/ip link show dev '# interface ' )

This host runs KVM guests, one of the guests is a puppet-master, with a local name master. Hence the name of the interface. Similarly the interface git is associated with the KVM guest behind git.steve.org.uk.

Reproducibility
---------------
Packages should build reproducibly, which for the purposes of this
document [#]_ means that given
- a version of a source package unpacked at a given path;
- a set of versions of installed build dependencies;
- a set of environment variable values;
- a build architecture; and
- a host architecture,
repeatedly building the source package for the build architecture on
any machine of the host architecture with those versions of the build
dependencies installed and exactly those environment variable values
set will produce bit-for-bit identical binary packages.
It is recommended that packages produce bit-for-bit identical binaries
even if most environment variables and build paths are varied.  It is
intended for this stricter standard to replace the above when it is
easier for packages to meet it.
.. [#]
   This is Debian's precisification of the  reproducible-builds.org
   definition  _.

• Holger Levsen wrote a blog post briefly describing the background and implications of this. To quote him: "we are not 94% done yet, rather more like half done or so. We still need tools and processes to enable anyone to indepently verify that a given binary comes from the sources it is said to be coming, this will involve distributing .buildinfo files and providing user interfaces in APT and elsewhere and probably also systematic rebuilds by us and other parties. And 6% or 7% of the archive is still a lot of packages, eg. in Buster we currently still have 273 unreproducible key packages and for a large part we don't have patches yet so there is still a lot of work ahead."
• There were discussion threads on Hacker News and Reddit.
• Our long-term goal is that Policy mandates that packages "must" be reproducible, but for that we need to show further progress and also reach a consensus on .buildinfo files and much more.

• Bernhard M. Wiedemann:
  • qt5/base, SOURCE_DATE_EPOCH support.

• Adrian Bunk:
  • #872262 filed against jellyfish.
  • #872675 filed against pdp.
  • #872678 filed against gem.
• Chris Lamb:
  • #872453 filed against isa-support.
  • #872459 filed against python-numpy.
  • #872460 filed against gcab, forwarded upstream.
  • #872514 filed against devscripts.
  • #872728 filed against desktop-file-utils.
  • #872729 filed against gtk+2.0, forwarded upstream, found via a reproducibility issue in Tails.
• Federico Brega:
  • #872285 filed against pyqt5.
• Jeremy Bicha:
  • #871963 filed against birdfont.
• Philip Rinn:
  • #872184 filed against quilt.

• Mattia Rizzolo:
  • console-data (for bug #799871).

• Added nondeterministic_output_generated_by_gcab.
• Added build_path_captured_by_python_numpy_misc_util.
• Added captures_build_path_in_sphinx_attr_links.
• Re-added nondeterminstic_ordering_in_gsettings_glib_enums_xml.

• Adrian Bunk (59)
• Bastien Roucari s (1)
• James Clarke (1)
• Jeremy Bicha (1)

• Ximin Luo:
  • presenters: html: Don't traverse children whose parents were already limited (Closes: #871413)
  • On a non-GNU system, prefer tools that start with "g" for certain whitelisted commands. (Closes: #871029)
  • Add a --tool-prefix-binutils CLI flag. (Closes: #869868)
• Chris Lamb:
  • Temporarily revert "Bump Standards-Version to 4.0.1" to avoid spurious CI test failures.
  • comparators.xml: Use name attribute over path to avoid leaking comparison full path in output.
  • Code style fixes.

• Chris Lamb:
  • Add simple autopkgtest.

• Ximin Luo:
  • Choose an existent HOME for the "control" build. (Closes: #860428)
  • Update debian/changelog with Santiago's changes.
• Santiago Torres:
  • Abstract parts of autopkgtest to support running on non-Debian systems.
  • Add a --host-distro flag to support that too.

Changes in version 1.65.0-1 (2017-08-24)

• Upgraded to Boost 1.64 and then 1.65 installed directly from upstream source with several minor tweaks (as before)
• Fourth tweak corrects a misplaced curly brace (see the Boost ublas GitHub repo and its issue #40)
• Added Boost align (as requested in #32)
• Added Boost sort (as requested in #35)
• Added Boost multiprecision by fixing a script typo (as requested in #42)
• Updated Travis CI support via newer run.sh

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Filed under: Debian English SUSE Weblate

When the rumor mill starts Sullivan gave a recent example of the claim of the decline of copyleft in an article on Opensource.com by Jono Bacon from February 2017 that showed a histogram of license usage between 2010 and 2017 (seen below).

From that, Bacon elaborates possible reasons for the apparent decline of the GPL. The graphic used in the article was actually generated by Stephen O'Grady in a January article, The State Of Open Source Licensing, which said:

In Black Duck's sample, the most popular variant of the GPL version 2 is less than half as popular as it was (46% to 19%). Over the same span, the permissive MIT has gone from 8% share to 29%, while its permissive cousin the Apache License 2.0 jumped from 5% to 15%.

Sullivan, however, argued that the methodology used to create both articles was problematic. Neither contains original research: the graphs actually come from the Black Duck Software "KnowledgeBase" data, which was partly created from the old Ohloh web site now known as Open Hub. To show one problem with the data, Sullivan mentioned two free-software projects, GNU Bash and GNU Emacs, that had been showcased on the front page of Ohloh.net in 2012. On the site, Bash was (and still is) listed as GPLv2+, whereas it changed to GPLv3 in 2011. He also claimed that "Emacs was listed as licensed under GPLv3-only, which is a license Emacs has never had in its history", although I wasn't able to verify that information from the Internet archive. Basically, according to Sullivan, "the two projects featured on the front page of a site that was using [the Black Duck] data set were wrong". This, in turn, seriously brings into question the quality of the data:

I reported this problem and we'll continue to do that but when someone is not sharing the data set that they're using for other people to evaluate it and we see glimpses of it which are incorrect, that should give us a lot of hesitation about accepting any conclusion that comes out of it.

Reproducible observations are necessary to the establishment of solid theories in science. Sullivan didn't try to contact Black Duck to get access to the database, because he assumed (rightly, as it turned out) that he would need to "pay for the data under terms that forbid you to share that information with anybody else". So I wrote Black Duck myself to confirm this information. In an email interview, Patrick Carey from Black Duck confirmed its data set is proprietary. He believes, however, that through a "combination of human and automated techniques", Black Duck is "highly confident at the accuracy and completeness of the data in the KnowledgeBase". He did point out, however, that "the way we track the data may not necessarily be optimal for answering the question on license use trend" as "that would entail examination of new open source projects coming into existence each year and the licenses used by them". In other words, even according to Black Duck, its database may not be useful to establish the conclusions drawn by those articles. Carey did agree with those conclusions intuitively, however, saying that "there seems to be a shift toward Apache and MIT licenses in new projects, though I don't have data to back that up". He suggested that "an effective way to answer the trend question would be to analyze the new projects on GitHub over the last 5-10 years." Carey also suggested that "GitHub has become so dominant over the recent years that just looking at projects on GitHub would give you a reasonable sampling from which to draw conclusions".

Indeed, GitHub published a report in 2015 that also seems to confirm MIT's popularity (45%), surpassing copyleft licenses (24%). The data is, however, not without its own limitations. For example, in the above graph going back to the inception of GitHub in 2008, we see a rather abnormal spike in 2013, which seems to correlate with the launch of the choosealicense.com site, described by GitHub as "our first pass at making open source licensing on GitHub easier". In his talk, Sullivan was critical of the initial version of the site which he described as biased toward permissive licenses. Because the GitHub project creation page links to the site, Sullivan explained that the site's bias could have actually influenced GitHub users' license choices. Following a talk from Sullivan at FOSDEM 2016, GitHub addressed the problem later that year by rewording parts of the front page to be more accurate, but that any change in license choice obviously doesn't show in the report produced in 2015 and won't affect choices users have already made. Therefore, there can be reasonable doubts that GitHub's subset of software projects may not actually be that representative of the larger free-software community.

In search of solid evidence So it seems we are missing good, reproducible results to confirm or dispel these claims. Sullivan explained that it is a difficult problem, if only in the way you select which projects to analyze: the impact of a MIT-licensed personal wiki will obviously be vastly different from, say, a GPL-licensed C compiler or kernel. We may want to distinguish between active and inactive projects. Then there is the problem of code duplication, both across publication platforms (a project may be published on GitHub and SourceForge for example) but also across projects (code may be copy-pasted between projects). We should think about how to evaluate the license of a given project: different files in the same code base regularly have different licenses often none at all. This is why having a clear, documented and publicly available data set and methodology is critical. Without this, the assumptions made are not clear and it is unreasonable to draw certain conclusions from the results. It turns out that some researchers did that kind of open research in 2016 in a paper called "The Debsources Dataset: Two Decades of Free and Open Source Software" [PDF] by Matthieu Caneill, Daniel M. Germ n, and Stefano Zacchiroli. The Debsources data set is the complete Debian source code that covers a large history of the Debian project and therefore includes thousands of free-software projects of different origins. According to the paper:

The long history of Debian creates a perfect subject to evaluate how FOSS licenses use has evolved over time, and the popularity of licenses currently in use.

Sullivan argued that the Debsources data set is interesting because of its quality: every package in Debian has been reviewed by multiple humans, including the original packager, but also by the FTP masters to ensure that the distribution can legally redistribute the software. The existence of a package in Debian provides a minimal "proof of use": unmaintained packages get removed from Debian on a regular basis and the mere fact that a piece of software gets packaged in Debian means at least some users found it important enough to work on packaging it. Debian packagers make specific efforts to avoid code duplication between packages in order to ease security maintenance. The data set covers a period longer than Black Duck's or GitHub's, as it goes all the way back to the Hamm 2.0 release in 1998. The data and how to reproduce it are freely available under a CC BY-SA 4.0 license.

Sullivan presented the above graph from the research paper that showed the evolution of software license use in the Debian archive. Whereas previous graphs showed statistics in percentages, this one showed actual absolute numbers, where we can't actually distinguish a decline in copyleft licenses. To quote the paper again:

The top license is, once again, GPL-2.0+, followed by: Artistic-1.0/GPL dual-licensing (the licensing choice of Perl and most Perl libraries), GPL-3.0+, and Apache-2.0.

Indeed, looking at the graph, at most do we see a rise of the Apache and MIT licenses and no decline of the GPL per se, although its adoption does seem to slow down in recent years. We should also mention the possibility that Debian's data set has the opposite bias: toward GPL software. The Debian project is culturally quite different from the GitHub community and even the larger free-software ecosystem, naturally, which could explain the disparity in the results. We can only hope a similar analysis can be performed on the much larger Software Heritage data set eventually, which may give more representative results. The paper acknowledges this problem:

Debian is likely representative of enterprise use of FOSS as a base operating system, where stable, long-term and seldomly updated software products are desirable. Conversely Debian is unlikely representative of more dynamic FOSS environments (e.g., modern Web-development with micro libraries) where users, who are usually developers themselves, expect to receive library updates on a daily basis.

The Debsources research also shares methodology limitations with Black Duck: while Debian packages are reviewed before uploading and we can rely on the copyright information provided by Debian maintainers, the research also relies on automated tools (specifically FOSSology) to retrieve license information. Sullivan also warned against "ascribing reason to numbers": people may have different reasons for choosing a particular license. Developers may choose the MIT license because it has fewer words, for compatibility reasons, or simply because "their lawyers told them to". It may not imply an actual deliberate philosophical or ideological choice. Finally, he brought up the theory that the rise of non-copyleft licenses isn't necessarily at the detriment of the GPL. He explained that, even if there is an actual decline, it may not be much of a problem if there is an overall growth of free software to the detriment of proprietary software. He reminded the audience that non-copyleft licenses are still free software, according to the FSF and the Debian Free Software Guidelines, so their rise is still a positive outcome. Even if the GPL is a better tool to accomplish the goal of a free-software world, we can all acknowledge that the conversion of proprietary software to more permissive and certainly simpler licenses is definitely heading in the right direction.

[I would like to thank the DebConf organizers for providing meals for me during the conference.] Note: this article first appeared in the Linux Weekly News.

• Roundcube with the sqlite3 backend stopped working after the upgrade; fix was to edit /etc/roundcube/debian-db-roundcube.php and change sqlite3:// to sqlite:// in the $config['db_dsnw'] line.
• Dovecot no longer supports SSLv2 so had to remove !SSLv2 from the ssl_protocols list in /etc/dovecot/conf.d/10-ssl.conf
• Duplicity now tries to do a mkdir so I had to change from the scp:// backend to the sftp:// backend in my backup scripts.
• Needed to add needs_root_rights=yes to /etc/X11/Xwrapper.config so Kodi systemd unit could still start it on a new VT. Need to figure out how to get this working without the need for root.
• Upgrading fail2ban would have been easier if I d dropped my additions in /etc/fail2ban/jail.d/ rather than the master config. Fixed for next time.
• ejabberd continues to be a pain; I do wonder if it s worth running an XMPP server these days. I certainly don t end up using it to talk to people myself.
• Upgrading 1200+ packages takes a long time, even when the majority of them don t have any questions to ask during the process.
• PostgreSQL upgrades have got so much easier. pg_upgradecluster 9.4 main chugged away but did exactly what I needed.

Device descriptor Remembering that values which are > 1 byte in length are always stored little-endian, we can construct our device descriptor as:

Our device descriptor

Field Name	Value	Bytes
bLength	18	0x12
bDescriptorType	DEVICE	0x01
bcdUSB	USB 2.0	0x00 0x02
bDeviceClass	0	0x00
bDeviceSubClass	0	0x00
bDeviceProtocol	0	0x00
bMaxPacketSize	64	0x40
idVendor	TEST	0xff 0xff
idProduct	TEST	0xff 0xff
bcdDevice	0.0.1	0x01 0x00
iManufacturer	1	0x01
iProduct	2	0x02
iSerialNumber	3	0x03
bNumConfigurations	1	0x01

We're using the vendor ID and product id 0xffff because at this point we don't have any useful values for this (it costs $5,000 to register a vendor ID). This gives us a final byte array of:

0x12 0x01 0x00 0x02 0x00 0x00 0x00 0x40 (Early descriptor) 0xff 0xff 0xff 0xff 0x01 0x00 0x01 0x02 0x03 0x01 (and the rest)

We're reserving string ids 1, 2, and 3, for the manufacturer string, product name string, and serial number string respectively. I'm deliberately including them all so that we can see it all come out later in lsusb. If you feed the above hex sequence into a USB descriptor decoder then you can check my working.

Endpoint Descriptor We want a single configuration, which covers our one interface, with one endpoint in it. Let's start with the endpoint...

Our bulk IN endpoint

Field Name	Value	Bytes
bLength	7	0x07
bDescriptorType	ENDPOINT	0x05
bEndpointAddress	EP2IN	0x82
bmAttributes	BULK	0x02
wMaxPacketSize	64	0x40 0x00
bInterval	IGNORED	0x00

We're giving a single bulk IN endpoint, since that's the simplest thing to describe at this time. This endpoint will never be ready and so nothing will ever be read into the host. All that gives us:

0x07 0x05 0x82 0x02 0x40 0x00 0x00

Interface Descriptor The interface descriptor prefaces the endpoint set, and thanks to our simple single endpoint, and no plans for alternate interfaces, we can construct the interface simply as:

Our single simple interface

Field Name	Value	Bytes
bLength	9	0x09
bDescriptorType	INTERFACE	0x04
bInterfaceNumber	1	0x01
bAlternateSetting	1	0x01
bNumEndpoints	1	0x01
bInterfaceClass	0	0x00
bInterfaceSubClass	0	0x00
bInterfaceProtocol	0	0x00
iInterface	5	0x05

All that gives us:

0x09 0x04 0x01 0x01 0x01 0x00 0x00 0x00 0x05

Configuration descriptor Finally we can put it all together and get the configuration descriptor...

Our sole configuration, encapsulating the interface and endpoint above

Field Name	Value	Bytes
bLength	9	0x09
bDescriptorType	CONFIG	0x02
wTotalLength	9+9+7	0x19 0x00
bNumInterfaces	1	0x01
bConfigurationValue	1	0x01
iConfiguration	4	0x04
bmAttributes	Bus powered, no wake	0x80
bMaxPower	500mA	0xfa

The wTotalLength field is interesting. It contains the configuration length, the interface length, and the endpoint length, hence 9 plus 9 plus 7 is 25. This gives:

0x09 0x02 0x19 0x00 0x01 0x01 0x04 0x80 0xfa

String descriptors We allowed ourselves a total of five strings, they were iManufacturer, iProduct, iSerial (from the device descriptor), iConfiguration (from the configuration descriptor), and iInterface (from the interface descriptor) respectively. Our string descriptors will therefore be:

String descriptor zero, en_GB only

Field Name	Value	Bytes
bLength	4	0x04
bDescriptorType	STRING	0x03
wLangID[0]	en_GB	0x09 0x08

0x04 0x03 0x09 0x08

...and...

String descriptor one, iManufacturer

Field Name	Value	Bytes
bLength	38	0x26
bDescriptorType	STRING	0x03
bString	"Rusty Manufacturer"	...

0x26 0x03 0x52 0x00 0x75 0x00 0x73 0x00 0x74 0x00 0x79 0x00 0x20 0x00 0x4d 0x00 0x61 0x00 0x6e 0x00 0x75 0x00 0x66 0x00 0x61 0x00 0x63 0x00 0x74 0x00 0x75 0x00 0x72 0x00 0x65 0x00 0x72 0x00

(You get the idea, there's no point me breaking down the rest of the string descriptors here, suffice it to say that the other strings are appropriate for the values they represent - namely product, serial, configuration, and interface.)

Putting it all together Given all the above, we have a device descriptor which is standalone, then a configuration descriptor which encompasses the interface and endpoint descriptors too. Finally we have a string descriptor table with six entries, the first is the language sets available, and the rest are our strings. In total we have:

    // Device descriptor
    const DEV_DESC: [u8; 18] =  
        0x12, 0x01, 0x00, 0x02, 0x00, 0x00, 0x00, 0x40,
        0xff, 0xff, 0xff, 0xff, 0x01, 0x00, 0x01, 0x02,
        0x03, 0x01
     ;
    // Configuration descriptor
    const CONF_DESC: [u8; 25] =  
        0x09, 0x02, 0x19, 0x00, 0x01, 0x01, 0x04, 0x80, 0xfa,
        0x09, 0x04, 0x01, 0x01, 0x01, 0x00, 0x00, 0x00, 0x05,
        0x07, 0x05, 0x82, 0x02, 0x40, 0x00, 0x00
     ;
    // String Descriptor zero
    const STR_DESC_0: [u8; 4] =  0x04, 0x03, 0x09, 0x08 ;
    // String Descriptor 1, "Rusty Manufacturer"
    const STR_DESC_1: [u8; 38] =  
        0x26, 0x03, 0x52, 0x00, 0x75, 0x00, 0x73, 0x00,
        0x74, 0x00, 0x79, 0x00, 0x20, 0x00, 0x4d, 0x00,
        0x61, 0x00, 0x6e, 0x00, 0x75, 0x00, 0x66, 0x00,
        0x61, 0x00, 0x63, 0x00, 0x74, 0x00, 0x75, 0x00,
        0x72, 0x00, 0x65, 0x00, 0x72, 0x00
     ;
    // String Descriptor 2, "Rusty Product"
    const STR_DESC_2: [u8; 28] =  
        0x1c, 0x03, 0x52, 0x00, 0x75, 0x00, 0x73, 0x00,
        0x74, 0x00, 0x79, 0x00, 0x20, 0x00, 0x50, 0x00,
        0x72, 0x00, 0x6f, 0x00, 0x64, 0x00, 0x75, 0x00,
        0x63, 0x00, 0x74, 0x00
     ;
    // String Descriptor 3, "123ABC"
    const STR_DESC_3: [u8; 14] =  
        0x0e, 0x03, 0x31, 0x00, 0x32, 0x00, 0x33, 0x00,
        0x41, 0x00, 0x42, 0x00, 0x43, 0x00
     ;
    // String Descriptor 4, "Rusty Configuration"
    const STR_DESC_4: [u8; 40] =  
        0x28, 0x03, 0x52, 0x00, 0x75, 0x00, 0x73, 0x00,
        0x74, 0x00, 0x79, 0x00, 0x20, 0x00, 0x43, 0x00,
        0x6f, 0x00, 0x6e, 0x00, 0x66, 0x00, 0x69, 0x00,
        0x67, 0x00, 0x75, 0x00, 0x72, 0x00, 0x61, 0x00,
        0x74, 0x00, 0x69, 0x00, 0x6f, 0x00, 0x6e, 0x00
     ;
    // String Descriptor 5, "Rusty Interface"
    const STR_DESC_5: [u8; 32] =  
        0x20, 0x03, 0x52, 0x00, 0x75, 0x00, 0x73, 0x00,
        0x74, 0x00, 0x79, 0x00, 0x20, 0x00, 0x49, 0x00,
        0x6e, 0x00, 0x74, 0x00, 0x65, 0x00, 0x72, 0x00,
        0x66, 0x00, 0x61, 0x00, 0x63, 0x00, 0x65, 0x00
     ;

With the above, we're a step closer to our first prototype which will hopefully be enumerable. Next time we'll look at beginning our prototype low level USB device stack mock-up.

• Remindr on Gitlab.com (would be awesome starring it if you like/use it)
• The official documentation on Readthedocs.io

o en Automatically Send Toots To The Mastodon Social Network https://carlchenet.com/automatically-send-toots-to-the-mastodon-social-network/ #Mastodon
x fr Sur Mastodon, cr er son compte de secours  ou tout perdre https://carlchenet.com/sur-mastodon-creer-son-compte-de-secours-ou-tout-perdre/ #Mastodon
x en Automatically boost cool toots on Mastodon with the Boost bot https://carlchenet.com/automatically-boost-cool-toots-on-mastodon-with-the-boost-bot/ #Mastodon
x en Cryptocurrencies On the New Social Network Mastodon #Mastodon #bitcoin #ethereum #monero

Lookup trie implementation Looking up a prefix in a routing table comes down to find the most specific entry matching the requested destination. A common structure for this task is the trie, a tree structure where each node has its parent as prefix. With IPv4, Linux uses a level-compressed trie (or LPC-trie), providing good performances with low memory usage. For IPv6, Linux uses a more classic radix tree (or Patricia trie). There are three reasons for not sharing:

• The IPv6 implementation (introduced in Linux 2.1.8, 1996) predates the IPv4 implementation based on LPC-tries (in Linux 2.6.13, commit 19baf839ff4a).
• The feature set is different. Notably, IPv6 supports source-specific routing¹ (since Linux 2.1.120, 1998).
• The IPv4 address space is denser than the IPv6 address space. Level-compression is therefore quite efficient with IPv4. This may not be the case with IPv6.

The trie in the below illustration encodes 6 prefixes: For more in-depth explanation on the different ways to encode a routing table into a trie and a better understanding of radix trees, see the explanations for IPv4. The following figure shows the in-memory representation of the previous radix tree. Each node corresponds to a struct fib6_node. When a node has the RTN_RTINFO flag set, it embeds a pointer to a struct rt6_info containing information about the next-hop. The fib6_lookup_1() function walks the radix tree in two steps:

1. walking down the tree to locate the potential candidate, and
2. checking the candidate and, if needed, backtracking until a match.

Here is a slightly simplified version without source-specific routing:

static struct fib6_node *fib6_lookup_1(struct fib6_node *root,
                                       struct in6_addr  *addr)
 
    struct fib6_node *fn;
    __be32 dir;
    /* Step 1: locate potential candidate */
    fn = root;
    for (;;)  
        struct fib6_node *next;
        dir = addr_bit_set(addr, fn->fn_bit);
        next = dir ? fn->right : fn->left;
        if (next)  
            fn = next;
            continue;
         
        break;
     
    /* Step 2: check prefix and backtrack if needed */
    while (fn)  
        if (fn->fn_flags & RTN_RTINFO)  
            struct rt6key *key;
            key = fn->leaf->rt6i_dst;
            if (ipv6_prefix_equal(&key->addr, addr, key->plen))  
                if (fn->fn_flags & RTN_RTINFO)
                    return fn;
             
         
        if (fn->fn_flags & RTN_ROOT)
            break;
        fn = fn->parent;
     
    return NULL;
 

Caching While IPv4 lost its route cache in Linux 3.6 (commit 5e9965c15ba8), IPv6 still has a caching mechanism. However cache entries are directly put in the radix tree instead of a distinct structure. Since Linux 2.1.30 (1997) and until Linux 4.2 (commit 45e4fd26683c), almost any successful route lookup inserts a cache entry in the radix tree. For example, a router forwarding a ping between 2001:db8:1::1 and 2001:db8:3::1 would get those two cache entries:

$ ip -6 route show cache
2001:db8:1::1 dev r2-r1  metric 0
    cache
2001:db8:3::1 via 2001:db8:2::2 dev r2-r3  metric 0
    cache

These entries are cleaned up by the ip6_dst_gc() function controlled by the following parameters:

$ sysctl -a   grep -F net.ipv6.route
net.ipv6.route.gc_elasticity = 9
net.ipv6.route.gc_interval = 30
net.ipv6.route.gc_min_interval = 0
net.ipv6.route.gc_min_interval_ms = 500
net.ipv6.route.gc_thresh = 1024
net.ipv6.route.gc_timeout = 60
net.ipv6.route.max_size = 4096
net.ipv6.route.mtu_expires = 600

The garbage collector is triggered at most every 500 ms when there are more than 1024 entries or at least every 30 seconds. The garbage collection won t run for more than 60 seconds, except if there are more than 4096 routes. When running, it will first delete entries older than 30 seconds. If the number of cache entries is still greater than 4096, it will continue to delete more recent entries (but no more recent than 512 jiffies, which is the value of gc_elasticity) after a 500 ms pause. Starting from Linux 4.2 (commit 45e4fd26683c), only a PMTU exception would create a cache entry. A router doesn t have to handle those exceptions, so only hosts would get cache entries. And they should be pretty rare. Martin KaFai Lau explains:

Out of all IPv6 RTF_CACHE routes that are created, the percentage that has a different MTU is very small. In one of our end-user facing proxy server, only 1k out of 80k RTF_CACHE routes have a smaller MTU. For our DC traffic, there is no MTU exception.

Here is how a cache entry with a PMTU exception looks like:

$ ip -6 route show cache
2001:db8:1::50 via 2001:db8:1::13 dev out6  metric 0
    cache  expires 573sec mtu 1400 pref medium

Performance We consider three distinct scenarios:

Excerpt of an Internet full view

In this scenario, Linux acts as an edge router attached to the default-free zone. Currently, the size of such a routing table is a little bit above 40,000 routes.

/48 prefixes spread linearly with different densities

Linux acts as a core router inside a datacenter. Each customer or rack gets one or several /48 networks, which need to be routed around. With a density of 1, /48 subnets are contiguous.

/128 prefixes spread randomly in a fixed /108 subnet

Linux acts as a leaf router for a /64 subnet with hosts getting their IP using autoconfiguration. It is assumed all hosts share the same OUI and therefore, the first 40 bits are fixed. In this scenario, neighbor reachability information for the /64 subnet are converted into routes by some external process and redistributed among other routers sharing the same subnet².

Route lookup performance With the help of a small kernel module, we can accurately benchmark³ the ip6_route_output_flags() function and correlate the results with the radix tree size: Getting meaningful results is challenging due to the size of the address space. None of the scenarios have a fallback route and we only measure time for successful hits⁴. For the full view scenario, only the range from 2400::/16 to 2a06::/16 is scanned (it contains more than half of the routes). For the /128 scenario, the whole /108 subnet is scanned. For the /48 scenario, the range from the first /48 to the last one is scanned. For each range, 5000 addresses are picked semi-randomly. This operation is repeated until we get 5000 hits or until 1 million tests have been executed. The relation between the maximum depth and the lookup time is incomplete and I can t explain the difference of performance between the different densities of the /48 scenario. We can extract two important performance points:

• With a full view, the lookup time is 450 ns. This is almost ten times the budget for forwarding at 10 Gbps which is about 50 ns.
• With an almost empty routing table, the lookup time is 150 ns. This is still over the time budget for forwarding at 10 Gbps.

With IPv4, the lookup time for an almost empty table was 20 ns while the lookup time for a full view (500,000 routes) was a bit above 50 ns. How to explain such a difference? First, the maximum depth of the IPv4 LPC-trie with 500,000 routes was 6, while the maximum depth of the IPv6 radix tree for 40,000 routes is 40. Second, while both IPv4 s fib_lookup() and IPv6 s ip6_route_output_flags() functions have a fixed cost implied by the evaluation of routing rules, IPv4 has several optimizations when the rules are left unmodified⁵. Those optimizations are removed on the first modification. If we cancel those optimizations, the lookup time for IPv4 is impacted by about 30 ns. This still leaves a 100 ns difference with IPv6 to be explained. Let s compare how time is spent in each lookup function. Here is a CPU flamegraph for IPv4 s fib_lookup(): Only 50% of the time is spent in the actual route lookup. The remaining time is spent evaluating the routing rules (about 30 ns). This ratio is dependent on the number of routes we inserted (only 1000 in this example). It should be noted the fib_table_lookup() function is executed twice: once with the local routing table and once with the main routing table. The equivalent flamegraph for IPv6 s ip6_route_output_flags() is depicted below: Here is an approximate breakdown on the time spent:

• 50% is spent in the route lookup in the main table,
• 15% is spent in handling locking (IPv4 is using the more efficient RCU mechanism),
• 5% is spent in the route lookup of the local table,
• most of the remaining is spent in routing rule evaluation (about 100 ns)⁶.

Why does the evaluation of routing rules is less efficient with IPv6? Again, I don t have a definitive answer.

History The following graph shows the performance progression of route lookups through Linux history: All kernels are compiled with GCC 4.9 (from Debian Jessie). This version is able to compile older kernels as well as current ones. The kernel configuration is the default one with CONFIG_SMP, CONFIG_IPV6, CONFIG_IPV6_MULTIPLE_TABLES and CONFIG_IPV6_SUBTREES options enabled. Some other unrelated options are enabled to be able to boot them in a virtual machine and run the benchmark. There are three notable performance changes:

• In Linux 3.1, Eric Dumazet delays a bit the copy of route metrics to fix the undesirable sharing of route-specific metrics by all cache entries (commit 21efcfa0ff27). Each cache entry now gets its own metrics, which explains the performance hit for the non-/128 scenarios.
• In Linux 3.9, Yoshifuji Hideaki removes the reference to the neighbor entry in struct rt6_info (commit 887c95cc1da5). This should have lead to a performance increase. The small regression may be due to cache-related issues.
• In Linux 4.2, Martin KaFai Lau prevents the creation of cache entries for most route lookups. The most sensible performance improvement comes with commit 4b32b5ad31a6. The second one is from commit 45e4fd26683c, which effectively removes creation of cache entries, except for PMTU exceptions.

Insertion performance Another interesting performance-related metric is the insertion time. Linux is able to insert a full view in less than two seconds. For some reason, the insertion time is not linear above 50,000 routes and climbs very fast to 60 seconds for 500,000 routes. Despite its more complex insertion logic, the IPv4 subsystem is able to insert 2 million routes in less than 10 seconds.

Memory usage Radix tree nodes (struct fib6_node) and routing information (struct rt6_info) are allocated with the slab allocator⁷. It is therefore possible to extract the information from /proc/slabinfo when the kernel is booted with the slab_nomerge flag:

# sed -ne 2p -e '/^ip6_dst/p' -e '/^fib6_nodes/p' /proc/slabinfo   cut -f1 -d:
   name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab>
fib6_nodes         76101  76104     64   63    1
ip6_dst_cache      40090  40090    384   10    1

In the above example, the used memory is 76104 64+40090 384 bytes (about 20 MiB). The number of struct rt6_info matches the number of routes while the number of nodes is roughly twice the number of routes: The memory usage is therefore quite predictable and reasonable, as even a small single-board computer can support several full views (20 MiB for each): The LPC-trie used for IPv4 is more efficient: when 512 MiB of memory is needed for IPv6 to store 1 million routes, only 128 MiB are needed for IPv4. The difference is mainly due to the size of struct rt6_info (336 bytes) compared to the size of IPv4 s struct fib_alias (48 bytes): IPv4 puts most information about next-hops in struct fib_info structures that are shared with many entries.

Conclusion The takeaways from this article are:

• upgrade to Linux 4.2 or more recent to avoid excessive caching,
• route lookups are noticeably slower compared to IPv4 (by an order of magnitude),
• CONFIG_IPV6_MULTIPLE_TABLES option incurs a fixed penalty of 100 ns by lookup,
• memory usage is fair (20 MiB for 40,000 routes).

Compared to IPv4, IPv6 in Linux doesn t foster the same interest, notably in term of optimizations. Hopefully, things are changing as its adoption and use at scale are increasing.

---

1. For a given destination prefix, it s possible to attach source-specific prefixes:

   ip -6 route add 2001:db8:1::/64 \
     from 2001:db8:3::/64 \
     via fe80::1 \
     dev eth0
   

   Lookup is first done on the destination address, then on the source address.
2. This is quite different of the classic scenario where Linux acts as a gateway for a /64 subnet. In this case, the neighbor subsystem stores the reachability information for each host and the routing table only contains a single /64 prefix.
3. The measurements are done in a virtual machine with one vCPU and no neighbors. The host is an Intel Core i5-4670K running at 3.7 GHz during the experiment (CPU governor set to performance). The benchmark is single-threaded. Many lookups are performed and the result reported is the median value. Timings of individual runs are computed from the TSC.
4. Most of the packets in the network are expected to be routed to a destination. However, this also means the backtracking code path is not used in the /128 and /48 scenarios. Having a fallback route gives far different results and make it difficult to ensure we explore the address space correctly.
5. The exact same optimizations could be applied for IPv6. Nobody did it yet.
6. Compiling out table support effectively removes those last 100 ns.
7. There is also per-CPU pointers allocated directly (4 bytes per entry per CPU on a 64-bit architecture). We ignore this detail.

• the Debian wiki page for Suricata
• the official Suricata upstream docs

% sudo aptitude install suricata

% sudo systemctl status suricata
  suricata.service - Suricata IDS/IDP daemon
   Loaded: loaded (/lib/systemd/system/suricata.service; enabled; vendor preset: enabled)
   Active: active (running) since Sat 2017-08-19 12:50:49 CEST; 44min ago
     Docs: man:suricata(8)
           man:suricatasc(8)
           https://redmine.openinfosecfoundation.org/projects/suricata/wiki
 Main PID: 1101 (Suricata-Main)
    Tasks: 8 (limit: 4915)
   CGroup: /system.slice/suricata.service
            1101 /usr/bin/suricata -D --af-packet -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid
ago 19 12:50:44 nostromo systemd[1]: Starting Suricata IDS/IDP daemon...
ago 19 12:50:47 nostromo suricata[1032]: 19/8/2017 -- 12:50:47 - <Notice> - This is Suricata version 4.0.0 RELEASE
ago 19 12:50:49 nostromo systemd[1]: Started Suricata IDS/IDP daemon.

% sudo suricatasc -c uptime
 "message": 3892, "return": "OK" 

[...]
# Linux high speed capture support
af-packet:
  - interface: wlan0
[...]

% sudo systemctl restart suricata
% sudo tail -f /var/log/suricata/fast.log
08/19/2017-14:03:04.025898  [**] [1:2012648:3] ET POLICY Dropbox Client Broadcasting [**] \
	[Classification: Potential Corporate Privacy Violation] [Priority: 1]  UDP  192.168.1.36:17500 -> 255.255.255.255:17500

% sudo aptitude install suricata-oinkmaster

• Antoine Beaupr did 20h (out of 16h allocated + 4 extra hours).
• Ben Hutchings did 14 hours (out of 15h allocated, thus keeping 1 extra hour for August).
• Chris Lamb did 18 hours.
• Emilio Pozuelo Monfort did 18.5 hours (out of 23.5 hours allocated + 8 hours remaining, thus keeping 13 hours for August).
• Guido G nther did 10 hours.
• Hugo Lefeuvre did nothing due to personal problems (out of 2h allocated + 10 extra hours, thus keeping 12 extra hours for August).
• Markus Koschany did 23.5 hours.
• Ola Lundqvist did not publish his report yet (out of 14h allocated + 2 extra hours).
• Rapha l Hertzog did 7 hours (out of 12 hours allocated but he gave back his remaining hours).
• Roberto C. Sanchez did 19.5 hours (out of 23.5 hours allocated + 12 hours remaining, thus keeping 16 extra hours for August).
• Thorsten Alteholz did 23.5 hours.

• Platinum sponsors:
• TOSHIBA (for 22 months)
• GitHub (for 13 months)
• Gold sponsors:
• The Positive Internet (for 38 months)
• Blablacar (for 37 months)
• Linode (for 27 months)
• Babiel GmbH (for 16 months)
• Plat Home (for 16 months)
• Silver sponsors:
• Domeneshop AS (for 37 months)
• Universit Lille 3 (for 37 months)
• Trollweb Solutions (for 35 months)
• Nantes M tropole (for 32 months)
• Dalenys (for 28 months)
• Univention GmbH (for 23 months)
• Universit Jean Monnet de St Etienne (for 23 months)
• Sonus Networks (for 17 months)
• UR Communications BV (for 12 months)
• maxcluster GmbH (for 11 months)
• Exonet B.V. (for 7 months)
• Leibniz Rechenzentrum
• Bronze sponsors:
• David Ayers IntarS Austria (for 38 months)
• Evolix (for 38 months)
• Offensive Security (for 38 months)
• Seznam.cz, a.s. (for 38 months)
• Freeside Internet Service (for 37 months)
• MyTux (for 37 months)
• Intevation GmbH (for 35 months)
• Linuxhotel GmbH (for 35 months)
• Daevel SARL (for 33 months)
• Bitfolk LTD (for 32 months)
• Megaspace Internet Services GmbH (for 32 months)
• NUMLOG (for 32 months)
• Greenbone Networks GmbH (for 31 months)
• WinGo AG (for 31 months)
• Ecole Centrale de Nantes LHEEA (for 27 months)
• Sig-I/O (for 24 months)
• Entr ouvert (for 22 months)
• Adfinis SyGroup AG (for 19 months)
• GNI MEDIA (for 14 months)
• Quarantainenet BV (for 14 months)
• RHX Srl (for 11 months)
• Bearstech (for 5 months)
• LiHAS (for 5 months)
• People Doc
• Catalyst IT Ltd

No comment Liked this article? Click here. My blog is Flattr-enabled.

• Upstream: https://github.com/mate-desktop
• Debian Packaging: https://qa.debian.org/developer.php?login=pkg-mate-team@lists.alioth.deb...
• Ubuntu MATE: http://ubuntu-mate.org/
• Git Packaging Repositories: https://anonscm.debian.org/cgit/pkg-mate/
• IRC (upstream): #mate-dev on Freenode (irc.freenode.net) # IRC (Debian + Ubuntu): #debian-mate on OFTC (irc.debian.org)

• Improve the MATE User Guide, I am quite optimistic that we get help from the person that asked for a better help tool on the MATE Desktop (like it was back in the times of old GNOMEv2)
• Provide several desktop layouts in Debian, like available in Ubuntu MATE, that can be chosen / configured by the MATE Tweak Tool
• Port the Ubuntu MATE Welcome application to Debian and autostart it on first login, like people know it from Ubuntu MATE

• released version 4.0.1.0 of the Policy Manual
• figured out documenting reproducibility in policy. Formulating the wording turned out to be easier than I had expected
• approx. ten years after they were first published, incorporated marga s maintscript flowcharts into policy proper
• converted policy from docbook to rST built with the Sphinx toolchain. Many, many thanks to Hideki Yamane and David Bremner for helping Russ and I get this merged to our master branch
• triage of every single bug against policy, and mass closure of inactive bugs, bringing the total down from more than 200 to around 125
• conversations with Technical Committee members about how the two teams can help each other s work (mainly us helping them to help us!)
• conversations about how we handle disagreement and plans to streamline our overly complex BTS usertags (watch this space)
• very useful input from policy consumers about how the upgrading checklist is formatted, and how we can recruit more people to get patches written

• plans to finally drop our emacsXY binary packages, and just have a single version of Emacs in the archive, so that we no longer have to deal with bugs due to someone still having emacs21 installed (David s idea; Rob s implementation; Sean s mostly-helpful comments)
• other plans to simplify and otherwise improve the Debian Emacsen policy
• finally finished off the work needed to RM emacs24 nine months later including a lot of NMUs
• mentoring a junior team member

• Arbitrary installation attacks. We protect against that with the outer signature and hashes
• Endless data attacks. Yes, we impose a limit on Release files (the sizes of other files are specified in there and this file is signed)
• Extraneous dependencies attacks That s verified by the signed hashes of Packages files
• Fast-forward attacks same
• Indefinite freeze attacks APT has a Valid-Until field that can be used to specify a maximum life time of a release file
• Malicious mirrors preventing updates. Well, the user configures the mirror, so usually not applicable. if the user has multiple mirrors, APT deals with that fine
• Mix-and-match attacks Again, signed Release file and hashes of other files
• Rollback attacks We do not allow Date fields in Release files to go backwards
• Slow retrieval attacks TUF cannot protect against that either. APT has very high timeouts, and there is no reasonable answer to that.
• Vulnerability to key compromises For our purposes where we need all repository signing keys to be online, as we need to sign new releases and metadata fairly often, it does not make it less vulnerable to require a threshold of keys (APT allows repositories to specify concrete key ids they may be signed with though, that has the same effect)
• Wrong software installation. Does not happen, the .deb files are hashed in the Packages files which are signed by the release file