XMir's been delayed from Ubuntu 13.10. The stated reason is that multi-monitor support isn't sufficiently reliable. That's true, but it's far from the only problem that XMir still has:

• It's still broken on some single-monitor systems

Some machines have display corruption. There are writes to freed memory. To be fair, some of the behaviour that's been seen has been down to underlying bugs in the Xorg drivers that were never triggered under normal use but are hit by XMir. Others are down to implicit assumptions made in the drivers that XMir happens to violate. The problem is that there doesn't appear to have been enough room in the schedule to deal with these interactions, presumably because nobody accounted for the inevitable "This thing we thought would be easy turns out to be difficult" part of the project.


• The input driver bug still isn't really fixed

I've mentioned this bug before. It's now marked "Fix released" which is kind of true but not really. Mir now tells XMir that the VT is changing before the VT is changed, but it doesn't wait for that to complete before switching the VTs. Until XMir is scheduled and runs, it's still receiving input events. In most circumstances that window is small, so there's no real risk of triggering it accidentally.

There's one corner case where it might still cause problems. Simply running isn't enough - XMir has to make progress through its event loop. That'll only happen if the X server is processing its wakeup events, and it's possible to effectively stall that by submitting a sufficiently awkward drawing request to the server. The X server will appear to freeze, and if the user then attempts to work out what's going on by switching to a VT and logging in, those input events will still be going to XMir. It's left as an exercise to the reader to construct ways to take advantage of this.

This can't happen in Xorg because the VT switch is blocked until the input devices have been released. Mir could have done the same, but doesn't because of a conscious design decision - in the Ubuntu Phone world, clients stop doing things when they're told to. Ubuntu Desktop is expected to behave the same way.

This is an unfortunate situation to be in. Ubuntu Desktop was told that they were switching to XMir, but Mir development seems to be driven primarily by the needs of Ubuntu Phone. XMir has to do things that no other Mir client will ever cope with, and unless Mir development takes that into account it's inevitably going to suffer breakage like this. Canonical management needs to resolve this if there's any hope of ever shipping XMir as the default desktop environment.


• It's still missing features

XMir doesn't support colour profiles. XRandR properties aren't exposed, so there's no way to control TV output encoding or overscan. There's still no hardware cursor support. Switching to XMir now would reduce functionality without providing any user-visible gain.

It's clear that XMir has turned into a larger project than Canonical had originally anticipated, but that's hardly surprising. There's only one developer with previous X experience working on it full-time, and the announced schedule provided no opportunity to deal with unexpected problems. As if that weren't enough, there's obvious conflict between Ubuntu Desktop and Ubuntu Phone when it comes to developer time and required functionality. The one hardware vendor who's willing to take a public stand has demonstrated that they have no interest in supporting XMir, despite Canonical assuring people that they were already engaging in productive negotiations with GPU manufacturers.

Multiple monitor support may be the single most obvious feature where XMir is lacking, but it's not the only one. Technically and politically, this code is a long way from being ready for prime time. Without significant community contribution, Canonical need to focus on prioritising it appropriately rather than expecting a tiny number of developers to perform miracles. Or, alternatively, they could just drop XMir entirely and focus on Unity 8.

comments

As a user of a shell you have most likely had the need to export a variable to another process; i.e. set/modify an environment variable.

Now, how do you stop exporting an environment variable? can you export anything else?

The bash shell offers the -n option of the export built-in, claiming it "remove[s] the export property". However, this feature is not part of the POSIX:2001 standard and is, therefore, a bashism.

A portable way to stop exporting an environment variable is to unset it. E.g. the effect of "export MY_VAR=foo" can be reverted by calling "unset MY_VAR" - surely enough, this will also destroy the content of the variable.

An equivalent could then be:

# to stop exporting/"unexport" the MY_VAR environment variable:
my_var="$MY_VAR" ; unset MY_VAR ;
MY_VAR="$my_var" ; unset my_var ;

The above code will make a copy of the variable before destroying it and then restoring its content.

How about exporting other things? did you know that you can export shell functions?

With the bash shell, you can export a function with the -f parameter of the export built-in. Needless to say, this is a bashism. Its secret? it's just an environment variable with the name of the function and the rest of the function definition as its value.

Try this:

$ echo="() /bin/echo 'have some bash' ; " bash -c 'echo "Hello World!"'
have some bash

Yes, this means that if you can control the content of an environment variable passed to bash you can probably execute whatever code you want. It comes handy when you want to alter a script's behaviour without modifying the script itself.

Possibilities are endless thanks to bash's support for non-standard characters in function names. Functions with slashes can also be exported, for example:

/sbin/ifconfig()
echo "some people say you should be using ip(1) instead" ;


Are you into bug hunting? export exec='() echo mount this ; '

A new version of Rcpp is now on the CRAN network for GNU R; binaries for Debian have been uploaded as well. Once more, this release brings a large number of exciting changes to Rcpp. Some concern usability, some bring new features, some increase performance; see below for the detailed list. We have now released three updates on a quarterly cycle; if we keep this up the next version ought to be ready at the end of December. As in the past, we tested the release rather rigorously by checking against all packages I could (relatively easily) built on my server: this time it successfully passed \code R CMD check for all 107 packages I can build locally out of a total of 136 packages. (Two failed: one for an error in \code Makevars , and one for the need of an X11 server during tests; this may get addressed in the in test script next time). As all of these 107 packages passed, we do not expect any issues with dependent packages. Should there be issues we would appreciate a note, preferably with reproducible code, to the rcpp-devel mailing list. The complete NEWS entry for 0.10.4 is below; more details are in the ChangeLog file in the package and on the Rcpp Changelog page.

Changes in Rcpp version 0.10.5 (2013-09-28)

• Changes in R code:
  • New R function demangle that calls the DEMANGLE macro.
  • New R function sizeof to query the byte size of a type. This returns an object of S3 class bytes that has a print method showing bytes and bits.
• Changes in Rcpp API:
  • Add defined(__sun) to lists of operating systems to test for when checking for lack of backtrace() needed for stack traces.
  • as<T*>, as<const T*>, as<T&> and as<const T&> are now supported, when T is a class exposed by modules, i.e. with RCPP_EXPOSED_CLASS
  • DoubleVector as been added as an alias to NumericVector
  • New template function is<T> to identify if an R object can be seen as a T. For example is<DataFrame>(x). This is a building block for more expressive dispatch in various places (modules and attributes functions).
  • wrap can now handle more types, i.e. types that iterate over std::pair<const KEY, VALUE> where KEY can be converted to a String and VALUE is either a primitive type (int, double) or a type that wraps. Examples :
    • std::map<int, double> : we can make a String from an int, and double is primitive
    • boost::unordered_map<double, std::vector<double> >: we can make a String from a double and std::vector<double> can wrap itself
    Other examples of this are included at the end of the wrap unit test file (runit.wrap.R and wrap.cpp).
  • wrap now handles containers of classes handled by modules. e.g. if you expose a class Foo via modules, then you can wrap vector<Foo>, ... An example is included in the wrap unit test file.
  • RcppLdFlags(), often used in Makevars files of packages using Rcpp, is now exported from the package namespace.
• Changes in Attributes:
  • Objects exported by a module (i.e. by a RCPP_MODULE call in a file that is processed by sourceCpp) are now directly available in the environment. We used to make the module object available, which was less useful.
  • A plugin for openmp has been added to support use of OpenMP.
  • Rcpp::export now takes advantage of the more flexible as<>, handling constness and referenceness of the input types. For users, it means that for the parameters of function exported by modules, we can now use references, pointers and const versions of them. The file Module.cpp file has an example.
  • No longer call non-exported functions from the tools package
  • No longer search the inline package as a fallback when loading plugins for the the Rcpp::plugins attribute.
• Changes in Modules:
  • We can now expose functions and methods that take T& or const T& as arguments. In these situations objects are no longer copied as they used to be.
• Changes in sugar:
  • is_na supports classes DatetimeVector and DateVector
• Changes in Rcpp documentation:
  • The vignettes have been moved from inst/doc/ to the vignettes directory which is now preferred.
  • The appearance of the vignettes has been refreshed by switching to the Bistream Charter font, and microtype package.
• Deprecation of RCPP_FUNCTION_*:
  • The macros from the preprocessor_generated.h file have been deprecated. They are still available, but they print a message in addition to their expected behavior.
  • The macros will be permanently removed in the first Rcpp release after July 2014.
  • Users of these macros should start replacing them with more up-to-date code, such as using 'Rcpp attributes' or 'Rcpp modules'.

Thanks to CRANberries, you can also look at a diff to the previous release 0.10.4. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

In a response to my blogpost about bashisms in function names, reader Detlef L pointed out in a comment that aliases allow non-standard characters in their names, contrary to functions. They could then be used to, for example, set an alias of the run-parts(1) command (cf. the blog post).

Aliases indeed allow characters such as commas ( , ) to be used in the alias name. However, aliases are an extension to the POSIX:2001 specification and are therefore bashisms. Moreover, the characters set defined by POSIX does not include dashes.

Last but not least, aliases belong to the list of shell features that are usually "disabled" when the shell is run in non-interactive mode. I.e.


$ bash <<EOF
alias true=false;
if true; then echo alias disabled; else echo alias enabled; fi
EOF

alias disabled
$ bash -i <<EOF # force interactive mode
alias true=false;
if true; then echo alias disabled; else echo alias enabled; fi
EOF

$ alias true=false;
$ if true; then echo alias disabled; else echo alias enabled; fi
alias enabled
$ exit


To add to the fun of different behaviours, other shells always expand aliases.

If you decide to play with aliases you should note one thing: they are only enabled from the line after the definition. E.g. note the difference below

$ dash -c 'alias foo="echo foo"; foo'
dash: foo: not found
$ dash -c 'alias foo="echo foo";
foo'
foo


Explanation of umarell

Umarell /uma'r l/ (oo-mah-rell), n; pl. Umarells. People in a community who offer all sorts of comments to those who are trying to get some work done, but who are not doing any work themselves.

Etymology and further details Umarell is a word that entered Italian slang in Bologna and is spreading to nearby towns, occasionally even across Italy. It comes from the Bolognese for "cute/odd little man". "Umarells" are those people, usually retired men, who spend time watching construction works, often holding their hands behind their back, occasionally commenting on what is going on, sometimes trying to tell the workers what to do. It's easy to find examples on the internet; the word was popularised by a blog collecting photos, which has even been published into a book. With some Italian Debian friends, we realised that umarell is the perfect word to describe those people in a community, who offer all sorts of comments to those who are trying to get some work done, but who are not doing any work themselves. I think that it is a word that fits perfectly, and since I'm likely going to use it blissfully anywhere, here is a page that temporarily explains what it means until the Oxford English Dictionary picks it up.

EFI includes support for storing persistent 'variables': it implements a key-value store where keys are (UUID, string) and values are arbitrary blobs. (There are also some flags that restrict when the variables can be accessed.) The physical storage medium is normally flash. The Linux efivars module provides access to variable storage, both through a specific interface in sysfs and through the somewhat abstracted pstore interface. An EFI implementation reads the boot order and (if applicable) Secure Boot configuration from specific EFI variables. An operating system can add arbitrary variables, using its own UUID(s). On Linux, the efibootmgr utility is used to read and write the boot order and the pstore interface is used within the kernel to dump kernel log messages or other information when particular events occur. Since efivars is needed to set the boot order on EFI systems, it's a critical part of installation. But it wasn't being loaded on Debian systems until efibootmgr was run. This doesn't work so well when using a rescue system, as the installation kernel and installed kernel may be different (Debian bug 703363). In future (Debian package version 3.2.41-1) this will be loaded automatically on EFI systems. But now comes a new problem: bugs in the implementation of variable storage can prevent a system from booting, and even worse, they may prevent manual recovery without special equipment (the system is 'bricked'). Matthew Garrett recently talked about this and mentioned that some Samsung laptops are bricked if the variable storage becomes more than 50% full. efivars now has some extra checks to avoid triggering this (both in mainline and in stable branches). This isn't the end of the story, unfortunately. Remember that nothing can be overwritten in flash without erasing a relatively large block, and that writes may fail after a few thousand erase cycles (number depending on the type of flash used). Therefore, deleting a variable usually does not free storage immediately, and overwriting a variable usually reduces the amount of free storage. In the case of the Samsung laptops, unused storage is apparently reclaimed at the next reboot. However, while testing the backported efivars changes I found that my EFI-booting system (Asus P8Z68-V LX motherboard) does not reclaim unused storage until it's very nearly full. Since the firmware itself increments a counter variable on every boot, using 36 more bytes of storage, I was able to observe the available space dropping to under 100 bytes, and then after another boot jumping to about 42K (out of 64K). This means that well over 50% of storage was unused, yet efivars would refuse to write any variables because the firmware didn't reclaim it. This would prevent (re)installation of Linux, as the boot configuration could not be written. On the assumption that both behaviours are common, I've settled on a compromise that I hope minimises overall risk:

• Applied pending changes that allow pstore integration to be disabled at run-time, and made this the default. This means only efibootmgr is likely to write to variables.
• Removed the check that variable writes will not fill storage more than 50%.

But it's essential that we find a way to identify and work safely with both behaviours. Not only do I want to eliminate the possibility of bricking or installaton failure, but I want to make EFI-backed pstore safe to enable. Crash logs are often hard to obtain and pstore can help to solve that problem. Updated 2013-09-15: The free space check has been revised (in Linux 3.2.48, 3.5.7.17, 3.8.13.5, 3.9.7 and 3.10) based on advice from Samsung. The kernel leaves at least 5K (rather than 50%) of the EFI variable store free. If it is about to breach this limit, it attempts to write a dummy variable that is larger than the free space, which will either fail or trigger garbage collection. The normal variable can then be written if garbage collection freed up enough space. This check can be disabled using the kernel parameter efi_no_storage_paranoia, but there should be no need to do so.

The first time I found this cute game, was around 2009, but for a bunch of reasons I haven t been able to get it into the archive until now. Pink Pony is a cute game in which you have to control a pink pony who s main goal is to last in the game more time than the other ponies. You might remember the game Tron, or the film, in which there were light cycles who left a trace behind, with which the other cycles crashed. Well, this is the similar game concept, only with cute ponies instead of light cycles. You can see a video of the game in action, if you want to get a feeling of how it is. Thomas Weber (Ginko), has published a newer version of the game (1.3.1) a couple of days ago, and that is the one that has entered Debian repositories (sid) today. Thanks, Ginko, for this lovely game! Even though the game might not seem too appealing for some of the adult users of Debian or Ubuntu, maybe some of their kids will like it as much as I do. The game is quite quick, though, so very young kids might have trouble controlling the pony. I would suspect it might be all right for kids above 9 years old, but I haven t been able to test my hypothesis, so if you find out, please tell me

Many thanks to Vincent Meurisse who solved my node.js callback woe. Some history of the blogspam service:

Back in 2008 I was annoyed by the many spam-comments that were being submitted to my Debian Administration website. I added some simple anti-spam measures, which reduced the flow, but it was a losing battle. In the end I decided I should test comments, as the users submitted them, via some kind of external service. The intention being that any improvements to that central service would benefit all users. (So I could move to testing comments on my personal blog too, for example). Ultimately I registered the domain-name "blogspam.net", and set up a simple service on it which would test comments and judge them to be "SPAM" or "OK". The current statistics show that this service has stopped 20 million spam comments, since then. (We have to pretend I didn't wipe the counters once or twice.)

I've spent a while now re-implementing most of the old plugins in node.js, and I think I'll be ready to deploy the new service over the weekend. The new service will have to handle two different kinds of requests:

New Requests

These will be submitted via HTTP POSTed JSON data, and will be handled by node.js. These should be nice and fast.

Legacy Requests

These will come in via XML-RPC, and be proxied through the new node.js implementation. Hopefully this will mean existing clients won't even notice the transition.

I've not yet deployed the new code, but it is just a matter of time. Hopefully being node.js based and significantly easier to install, update, and tweak, I'll get more contributions too. The dependencies are happily very minimal:

• A redis-server for maintaining state:
  • The number of SPAM/OK comments for each submitting site.
  • An auto-expiring cache of blacklisted IP adddresses. (I cache the results of various RBL results for 48 hours).
• node.js

The only significant outstanding issue is that I need to pick a node.js library for performing CIDR lookups - "Does 10.11.12.23 lie within 10.11.12.0/24?" - I'm surprised that functionality isn't available out of the box, but it is the only omission I've missed. I've been keeping load & RAM graphs, so it will be interesting to see how the node.js service competes. I expect that if clients were using it, in preference to the XML-RPC version, then I'd get a hell of a lot more throughput, but with it hidden behind the XML-RPC proxy I'm less sure what will happen. I guess I also need to write documentation for the new/preferred JSON-based API... https://github.com/skx/blogspam.js

When it comes to provide redundant services, several options are available:

• The service can be hosted behind a set of load-balancers. They will detect any faulty node. However, you need to ensure that this new layer is also fault-tolerant.
• The nodes providing the service can rely on IP failover to share a set of IP using protocols like VRRP¹ or CARP. The IP address of a faulty node will be assigned to another node. This requires all nodes to be part of the same IP subnet.
• The clients of the service can ask a third-party for available nodes. Usually, this is achieved through round-robin DNS where only working nodes are advertised in a DNS record. The failover can be quite long because of caches.

A common setup is a combination of those solutions: web servers are behind a couple of load-balancers achieving both redundancy and load-balancing. The load-balancers use VRRP to ensure redundancy. The whole setup is replicated to another datacenter and round-robin DNS is used to ensure both redundancy and load-balacing of the datacenters. There is a fourth option which is similar to VRRP but relies on dynamic routing and therefore is not limited to nodes in the same subnet:

• The nodes advertise their availability with BGP to announce the set of service IP addresses they are able to serve. Each address is weighted such that IP addresses are balanced among the available nodes.

We will explore how to implement this fourth option using ExaBGP, the BGP swiss army knife of networking, in a small lab based on KVM. You can grab the complete lab from GitHub. ExaBGP 3.2.5 is needed to run this lab.

Environment We will be working in the following (IPv6-only) environment:

BGP configuration BGP is enabled on ER2 and ER3 to exchange routes with peers and transits (only R1 for this lab). BIRD is used as a BGP daemon. Its configuration is pretty basic. Here is a fragment:

router id 1.1.1.2;
protocol static NETS   #  
  import all;
  export none;
  route 2001:db8::/40 reject;
 
protocol bgp R1   #  
  import all;
  export where proto = "NETS";
  local as 64496;
  neighbor 2001:db8:1000::1 as 64511;
 
protocol bgp ER3   #  
  import all;
  export all;
  next hop self;
  local as 64496;
  neighbor 2001:db8:1::3 as 64496;
 

First, in , we declare the routes that we want to export to our peers. We don t try to summarize them from the IGP. We just unconditonnaly export the networks that we own. Then, in , R1 is defined as a neighbor and we export the static route we declared previously. We import any route that R1 is willing to send us. In , we share everything we know with our pal, ER3, using internal BGP.

OSPF configuration OSPF will distribute routes inside the AS. It is enabled on ER2, ER3, DR6, DR7 and DR8. For example, here is the relevant part of the configuration of DR6:

router id 1.1.1.6;
protocol kernel  
   persist;
   import none;
   export all;
 
protocol ospf INTERNAL  
  import all;
  export none;
  area 0.0.0.0  
    networks  
      2001:db8:1::/64;
      2001:db8:6::/64;
     ;
    interface "eth0";
    interface "eth1"   stub yes;  ;
   ;
 

ER2 and ER3 inject a default route into OSPF:

protocol static DEFAULT  
  import all;
  export none;
  route ::/0 via 2001:db8:1000::1;
 
filter default_route  
  if proto = "DEFAULT" then accept;
  reject;
 
protocol ospf INTERNAL  
  import all;
  export filter default_route;
  area 0.0.0.0  
    networks  
      2001:db8:1::/64;
     ;
    interface "eth1";
   ;
 

Web nodes The web nodes are pretty basic. They have a default static route to the nearest router and that s all. The interesting thing here is that they are each on a separate IP subnet: we cannot share an IP using VRRP². Why are those web servers on different subnets? Maybe they are not in the same datacenter or maybe your network architecture is using a routed access layer. Let s see how to use BGP to enable redundancy of those web nodes.

Redundancy with ExaBGP ExaBGP is a convenient tool to plug scripts into BGP. They can then receive and advertise routes. ExaBGP does the hard work of speaking BGP with your routers. The scripts just have to read routes from standard input or advertise them on standard output.

The big picture Here is the big picture: Let s explain it step by step:

1. Three IP addresses will be allocated for our web service: 2001:db8:30::1, 2001:db8:30::2 and 2001:db8:30::3. Those are distinct from the real IP addresses of W1, W2 and W3.
2. Each web node will advertise all of them to the route servers we added in the network. I will talk more about those route servers later.
3. Each route comes with a metric to help the route server choose where it should be routed. We choose the metrics such that each IP address will be routed to a distinct web node (unless there is a problem).
4. The route servers (which are not routers) will then advertise the best routes they learned to all the routers in our network. This is still done using BGP.
5. Now, for a given IP address, each router knows to which web node the traffic should be directed.

Here are the respective metrics announced routes for W1, W2 and W3 when everything works as expected:

Route	W1	W2	W3	Best	Backup
2001:db8:30::1	102	101	100	W3	W2
2001:db8:30::2	101	100	102	W2	W1
2001:db8:30::3	100	102	101	W1	W3

ExaBGP configuration The configuration of ExaBGP is quite simple:

group rs  
  neighbor 2001:db8:1::4  
    router-id 1.1.1.11;
    local-address 2001:db8:6::11;
    local-as 65001;
    peer-as 65002;
   
  neighbor 2001:db8:8::5  
    router-id 1.1.1.11;
    local-address 2001:db8:6::11;
    local-as 65001;
    peer-as 65002;
   
  process watch-nginx  
      run /usr/bin/python /lab/healthcheck.py -s --config /lab/healthcheck-nginx.conf --start-ip 0;
   
 

A script is entailed to check if the service (an nginx web server) is up and running and advertise the appropriate IP addresses to the two declared route servers. If we run the script manually, we can see the advertised routes:

$ python /lab/healthcheck.py --config /lab/healthcheck-nginx.conf --start-ip 0
INFO[healthcheck] send announces for UP state to ExaBGP
announce route 2001:db8:30::3/128 next-hop self med 100
announce route 2001:db8:30::2/128 next-hop self med 101
announce route 2001:db8:30::1/128 next-hop self med 102
[...]
WARNING[healthcheck] Check command was unsuccessful: 7
INFO[healthcheck] Output of check command:  curl: (7) Failed connect to ip6-localhost:80; Connection refused
WARNING[healthcheck] Check command was unsuccessful: 7
INFO[healthcheck] Output of check command:  curl: (7) Failed connect to ip6-localhost:80; Connection refused
WARNING[healthcheck] Check command was unsuccessful: 7
INFO[healthcheck] Output of check command:  curl: (7) Failed connect to ip6-localhost:80; Connection refused
INFO[healthcheck] send announces for DOWN state to ExaBGP
announce route 2001:db8:30::3/128 next-hop self med 1000
announce route 2001:db8:30::2/128 next-hop self med 1001
announce route 2001:db8:30::1/128 next-hop self med 1002

When the service becomes unresponsive, the healthcheck script detect the situation and retry several times before acknowledging that the service is dead. Then, the IP addresses are advertised with higher metrics and the service will be routed to another node (the one advertising 2001:db8:30::3/128 with metric 101). This healthcheck script is now part of ExaBGP.

Route servers We could have connected our ExaBGP servers directly to each router. However, if you have 20 routers and 10 web servers, you now have to manage a mesh of 200 sessions. The route servers are here for three purposes:

1. Reduce the number of BGP sessions (from 200 to 60) between equipments (less configuration, less errors).
2. Avoid modifying the configuration on routers each time a new service is added.
3. Separate the routing decision (the route servers) from the routing process (the routers).

You may also ask yourself: why not use OSPF? . Good question!

• OSPF could be enabled on each web node and the IP addresses advertised using this protocol. However, OSPF has several drawbacks: it does not scale, there are restrictions on the allowed topologies, it is difficult to filter routes inside OSPF and a misconfiguration will likely impact the whole network. Therefore, it is considered a good practice to limit OSPF to network equipments.
• The routes learned by the route servers could be injected into OSPF. On paper, OSPF has a next-hop field to provide an explicit next-hop. This would be handy as we wouldn t have to configure adjacencies with each router. However, I have absolutely no idea how to inject BGP next-hop into OSPF next-hop. What happens is that BGP next-hop is resolved locally using OSPF routes. For example, if we inject BGP routes into OSPF from RS4, RS4 will know the appropriate next-hop but other routers will route the traffic to RS4.

Let s look at how to configure our route servers. RS4 will use BIRD while RS5 will use Quagga. Using two different implementations will help with resiliency by avoiding a bug to hit the two route servers at the same time.

Bird configuration There are two sides for BGP configuration: the BGP sessions with the ExaBGP nodes and the ones with the routers. Here is the configuration for the later:

template bgp INFRABGP  
  export all;
  import none;
  local as 65002;
  rs client;
 
protocol bgp ER2 from INFRABGP  
  neighbor 2001:db8:1::2 as 65003;
 
protocol bgp ER3 from INFRABGP  
  neighbor 2001:db8:1::3 as 65003;
 
protocol bgp DR6 from INFRABGP  
  neighbor 2001:db8:1::6 as 65003;
 
protocol bgp DR7 from INFRABGP  
  neighbor 2001:db8:1::7 as 65003;
 
protocol bgp DR8 from INFRABGP  
  neighbor 2001:db8:1::8 as 65003;
 

The AS number used by our route server is 65002 while the AS number used for routers is 65003 (the AS numbers for web nodes will be 65001). They are reserved for private use by RFC 6996. All routes known by the route server are exported to the routers but no routes are accepted from them. Let s have a look at the other side:

# Only import loopback IPs
filter only_loopbacks   #  
  if net ~ [ 2001:db8:30::/64 128,128  ] then accept;
  reject;
 
# General template for an EXABGP node
template bgp EXABGP  
  local as 65002;
  import filter only_loopbacks; #  
  export none;
  route limit 10; #  
  rs client;
  hold time 6; #  
  multihop 10;
  igp table internal;
 
protocol bgp W1 from EXABGP  
  neighbor 2001:db8:6::11 as 65001;
 
protocol bgp W2 from EXABGP  
  neighbor 2001:db8:7::12 as 65001;
 
protocol bgp W3 from EXABGP  
  neighbor 2001:db8:8::13 as 65001;
 

To ensure separation of concerns, we are being a bit more picky. With and , we only accept loopback addresses and only if they are contained in the subnet that we reserved for this use. No server should be able to inject arbitrary addresses into our network. With , we also limit the number of routes that a server can advertise. With , we reduce the hold time from 240 to 6. It means that after 6 seconds, the peer is considered dead. This is quite important to be able to recover quickly from a dead machine. The minimal value is 3. We could have use a similar setting with the session with the routers.

Quagga configuration Quagga s configuration is a bit more verbose but should be strictly equivalent:

router bgp 65002 view EXABGP
 bgp router-id 1.1.1.5
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor R peer-group
 neighbor R remote-as 65003
 neighbor R ebgp-multihop 10
 neighbor EXABGP peer-group
 neighbor EXABGP remote-as 65001
 neighbor EXABGP ebgp-multihop 10
 neighbor EXABGP timers 2 6
!
 address-family ipv6
 neighbor R activate
 neighbor R soft-reconfiguration inbound
 neighbor R route-server-client
 neighbor R route-map R-IMPORT import
 neighbor R route-map R-EXPORT export
 neighbor 2001:db8:1::2 peer-group R
 neighbor 2001:db8:1::3 peer-group R
 neighbor 2001:db8:1::6 peer-group R
 neighbor 2001:db8:1::7 peer-group R
 neighbor 2001:db8:1::8 peer-group R
 neighbor EXABGP activate
 neighbor EXABGP soft-reconfiguration inbound
 neighbor EXABGP maximum-prefix 10
 neighbor EXABGP route-server-client
 neighbor EXABGP route-map RSCLIENT-IMPORT import
 neighbor EXABGP route-map RSCLIENT-EXPORT export
 neighbor 2001:db8:6::11 peer-group EXABGP
 neighbor 2001:db8:7::12 peer-group EXABGP
 neighbor 2001:db8:8::13 peer-group EXABGP
 exit-address-family
!
ipv6 prefix-list LOOPBACKS seq 5 permit 2001:db8:30::/64 ge 128 le 128
ipv6 prefix-list LOOPBACKS seq 10 deny any
!
route-map RSCLIENT-IMPORT deny 10
!
route-map RSCLIENT-EXPORT permit 10
  match ipv6 address prefix-list LOOPBACKS
!
route-map R-IMPORT permit 10
!
route-map R-EXPORT deny 10
!

The view is here to not install routes in the kernel³.

Routers Configuring BIRD to receive routes from route servers is straightforward:

# BGP with route servers
protocol bgp RS4  
  import all;
  export none;
  local as 65003;
  neighbor 2001:db8:1::4 as 65002;
  gateway recursive;
 
protocol bgp RS5  
  import all;
  export none;
  local as 65003;
  neighbor 2001:db8:8::5 as 65002;
  multihop 4;
  gateway recursive;
 

It is important to set gateway recursive because most of the time, the next-hop is not reachable directly. In this case, by default, BIRD will use the IP address of the advertising router (the route servers).

Testing Let s check that everything works as expected. Here is the view from RS5:

# show ipv6  bgp  
BGP table version is 0, local router ID is 1.1.1.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*  2001:db8:30::1/128
                    2001:db8:6::11         102             0 65001 i
*>                  2001:db8:8::13         100             0 65001 i
*                   2001:db8:7::12         101             0 65001 i
*  2001:db8:30::2/128
                    2001:db8:6::11         101             0 65001 i
*                   2001:db8:8::13         102             0 65001 i
*>                  2001:db8:7::12         100             0 65001 i
*> 2001:db8:30::3/128
                    2001:db8:6::11         100             0 65001 i
*                   2001:db8:8::13         101             0 65001 i
*                   2001:db8:7::12         102             0 65001 i
Total number of prefixes 3

For example, traffic to 2001:db8:30::2 should be routed through 2001:db8:7::12 (which is W2). Other IP are affected to W1 and W3. RS4 should see the same thing⁴:

$ birdc6 show route 
BIRD 1.3.11 ready.
2001:db8:30::1/128 [W3 22:07 from 2001:db8:8::13] * (100/20) [AS65001i]
                   [W1 23:34 from 2001:db8:6::11] (100/20) [AS65001i]
                   [W2 22:07 from 2001:db8:7::12] (100/20) [AS65001i]
2001:db8:30::2/128 [W2 22:07 from 2001:db8:7::12] * (100/20) [AS65001i]
                   [W1 23:34 from 2001:db8:6::11] (100/20) [AS65001i]
                   [W3 22:07 from 2001:db8:8::13] (100/20) [AS65001i]
2001:db8:30::3/128 [W1 23:34 from 2001:db8:6::11] * (100/20) [AS65001i]
                   [W3 22:07 from 2001:db8:8::13] (100/20) [AS65001i]
                   [W2 22:07 from 2001:db8:7::12] (100/20) [AS65001i]

Let s have a look at DR6:

$ birdc6 show route
2001:db8:30::1/128 via fe80::5054:56ff:fe6e:98a6 on eth0 * (100/20) [AS65001i]
                   via fe80::5054:56ff:fe6e:98a6 on eth0 (100/20) [AS65001i]
2001:db8:30::2/128 via fe80::5054:60ff:fe02:3681 on eth0 * (100/20) [AS65001i]
                   via fe80::5054:60ff:fe02:3681 on eth0 (100/20) [AS65001i]
2001:db8:30::3/128 via 2001:db8:6::11 on eth1 * (100/10) [AS65001i]
                   via 2001:db8:6::11 on eth1 (100/10) [AS65001i]

So, 2001:db8:30::3 is owned by W1 which is behind DR6 while the two others are in another part of the network and will be reached through the appropriate link-local addresses learned by OSPF. Let s kill W1 by stopping nginx. A few seconds later, DR6 learns the new routes:

$ birdc6 show route
2001:db8:30::1/128 via fe80::5054:56ff:fe6e:98a6 on eth0 * (100/20) [AS65001i]
                   via fe80::5054:56ff:fe6e:98a6 on eth0 (100/20) [AS65001i]
2001:db8:30::2/128 via fe80::5054:60ff:fe02:3681 on eth0 * (100/20) [AS65001i]
                   via fe80::5054:60ff:fe02:3681 on eth0 (100/20) [AS65001i]
2001:db8:30::3/128 via fe80::5054:56ff:fe6e:98a6 on eth0 * (100/20) [AS65001i]
                   via fe80::5054:56ff:fe6e:98a6 on eth0 (100/20) [AS65001i]

Demo For a demo, have a look at the following video (it is also available as an Ogg Theora video).

---

1. The primary target of VRRP is to make a default gateway for an IP network highly available by exposing a virtual router which is an abstract representation of multiple routers. The virtual IP address is held by the master router and any backup router can be promoted to master in case of a failure. In practice, VRRP can be used to achieve high availability of services too.
2. However, you could deploy an L2 overlay network using, for example, VXLAN and use VRRP on it.
3. I have been told on the Quagga mailing-list that such a setup is quite uncommon and that it would be better to not use views but use --no_kernel flag for bgpd instead. You may want to look at the whole thread for more details.
4. The output of birdc6 shows both the next-hop as advertised in BGP and the resolved next-hop if the route have to be exported into another protocol. I have simplified the output to avoid confusion.

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (47.50 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Package Tracking System There are only 2-3 weeks left in the summer of code project dedicated to rewrite the package tracking system. We have come a long way during August check it out yourself in pts.debian.net. The rewrite doesn t have all the features of the old PTS yet, but I opted to keep some of the easy and less interesting features for others to re-implement. Instead I asked Marko to work in the coming weeks on new features that will bring more value, like the possibility to have user accounts with the possibility to easily review and tweak all your subscriptions on the web, and like the possibility to subscribe to groups of packages (i.e. those managed by a team). Our main problem right now is that exim has a pretty poor default behavior of forking hundreds of processes if you get hundreds of mails (in a batch) to an address that delivers via a pipe (postfix is saner, it serializes the deliveries on pipes). The new PTS is much more modular and its memory footprint is bigger (about 3 times more for the process that delivers mails, 30Mb instead of 10Mb), and in such a situation we managed to run out of memory for now we worked around the situation with an exim setting that queues mails once the load gets too high but it s a poor workaround IMO. We could obviously implement our own queue and a daemon but I d like to avoid this. So who knows how to tell exim to behave? On the positive side, Marko has gotten some feedback from people who like the new PTS and are using it daily already. And several persons have expressed their interest to work on the new codebase already. On my side, I created a package so that it s easy to deploy for derivatives. In this process, I revamped the way we manage the Django settings (for development and for production). The package is not finished yet, but it s mostly usable already. But I still want to do some cleanup/refactoring in the models before others start deploying it. We must also enable South to make it possible to upgrade easily afterwards. DebConf 13 in Vaumarcus From August 10th to 17th, I was attending DebConf 13. It matched the only week of vacation that my wife had this summer so we went there with the whole family (that is with a 3 years old son, and 6 months old one). Thus I could not immerse myself in Debconf and missed all the nice things that happen outside of the talk rooms. I picked 3-4 interesting talks per day and I spent the rest with my family. On the positive side, I was pleased that my wife could meet (or at least see) some other Debian people. She knows quite a few (of you) by name because I have been telling her Debian stories for years now Debian France Debian France sold quite some merchandise during Debconf but I didn t take care of that. It was supervised by Sylvestre Ledru but fortunately he got the help of multiple persons, both to bring everything there, to sell it, and to bring back the rest. The good news of the month is that the upstream author of galette published a new version with all the features that we ordered him a few months ago. We send now automatic reminders to members who must renew their subscription, we have automatic update of our accounting books (in a ledger file in a git repository) when we people donate or pay their subscription via the paypal form on our website. I was so pleased to finally have this that I took some hours to finalize the packaging of galette, so that it could be uploaded to Debian. It s now waiting in the NEW queue. I also spent multiple hours to write the python script that is executed by galette and that updates the accounting files. Misc Debian stuff Debian Packaging. I did two uploads of logidee-tools to fix bugs #718671 and #718836. I created a package for Dolibarr a PHP-based CRM and ERP software (it doesn t do accounting however), it s sitting in the NEW queue for almost a month already. I forwarded #719000 to the upstream Publican developers. I filed #720393 to request a new upstream version of libphp-mailer. git-multimail. After its deployment on Alioth last month, Niels Thykier reported me a case where it lead to bounces, I filed this as a new upstream ticket and in fact I fixed it myself a few days after. I got the fixed version installed on Alioth. dpkg. I investigated why the the automatic builds of dpkg were no longer happening and asked Michael Prokop if he could install a newer version of gettext in the build chroot. He told me that he would need a backport for that so I asked Santiago Vila if he was willing to provide it and he kindly accepted. A few days after, the package was in backports and I m now again running the latest dpkg out of git thanks to the nice service provided by Michael. Misc discussions. The thread about user planets drifted into a discussion of how to avoid promotional posts on such planets and in that context someone again brought up the Debian Machine Usage Policy as a way to shut down any kind of (self-)promotional content on planet if there s money involved. This always irritates me and this time I opted to ask James Troup about the origin of that clause in the DMUP. So who is willing to work with DSA to fix the DMUP so that people stop abusing it in contexts where it doesn t make sense? I also participated in some discussions concerning dgit. I like the ideas behind the tool, but I m saddened by the behavior of Ian Jackson. I helped him to fill his gap of knowledge about new sources formats but he keeps on bashing about the 3.0 (quilt) source format both in the manual page and in the output of the program. He believes that dgit is no longer an experiment but the truth is that it s still a poorly commented Perl script doing lots of hackish things. Kali Linux Between Debconf and all, I haven t done much for Kali except a couple of fixes. There s a nice story of how I tracked a bug in live-installer on the Kali blog. That fix has been committed to Debian. I also improved live-build to include xfsprogs/jfsutils on the ISO image when you include the debian-installer (so that you don t end up in problems when you pick JFS or XFS as file systems for your installation). Thanks See you next month for a new summary of my activities.

One comment Liked this article? Click here. My blog is Flattr-enabled.

One of the goals I set for myself on returning to Brisbane was running in the Bridge to Brisbane. The City2South run ended up being a longer distance, sooner, so I only stuck with the Bridge to Brisbane so I could say I'd done it. The added variable was I had Zoe, and I was pushing her in a jogging stroller. I did a couple of practice runs up the bridge with Zoe in the weeks leading up to the run. Mostly to get her used to it, and for me to get a feel for what the incline was like. I found I had the stamina to easily do the bridge, but ran out of steam on the return leg. No big deal, I didn't need to do that for the real run. I did a poor job of publicising that I was doing the run, so I didn't end up raising a whole lot of money for my chosen charity, beyondblue, because I had the added variable of Zoe. I knew I had to get her up early to get to the start line in time, and I wasn't sure if she was going to be cooperative or not, or whether she'd have a tantrum half way through the run and I'd have to stop, so I didn't feel good about taking money from people for running a run that I wasn't sure I could complete. My main concern had been getting to the starting line in time. I wanted to take the train from Morningside to Murarrie and then walk to the start line (about 10 minutes) because that way I could take the train back to Morningside from the finish to collect the car and not have to worry about trying to get back to the start. That meant I'd have to get Zoe up before 5am to get to the station, and if she didn't cooperate, I'd miss the train, and it'd all be over. Fortunately, my cousin Renata offered to drop us off, and we did a complicated car switcharoo, which involved her leaving my car at Morningside station for me after dropping us off at Murarrie. Now that I've seen the road closure situation, I feel pretty confident that I could have just driven to Murarrie and ditched the car somewhere there. Interestingly, a train that was packed to the rafters with people was arriving at the station at the time we arrived to leave Renata's car there. I'm not sure if it was the 5:07am train running late, or if I'd mis-interpreted the timetable and there was a later train we could have more comfortably made without Renata's assistance. Anyway, we uneventfully got to Murarrie station by about 5:40am, and walked to the start. It was ridiculously packed. People as far as the eye could see. Apparently there were 29,479 entrants and 24,596 finishers in the 10km run, so that explains it. I was seeded in the "joggers" group, which was a fair way back from the start line, and as it was, technically had to walk around some massive barrier to get in with the runners, but I snuck in the side after the pack started moving. Zoe was really, really good for the whole run. I couldn't have asked for her to be better behaved, really. She was wanting a cuddle at the top of the bridge, and claimed she wanted to go to the toilet the first time she saw some portable toilets, but easily talked out of both. My strategy of having a broad array of snacks for her to eat during the run worked well to keep her happy, and the course was heavily packed the entire way, so she had plenty to look at. It was a nice father/daughter thing to do for Fathers Day. My official time was 57:28, which I'm extremely happy with, given the conditions I ran under. I was going to be happy with anything under an hour. My personal best for 10km (not that course) is 51:52 I'm very grateful to those people who did donate some money on short notice. I've managed to raise $180, which is better than nothing. Now that I've done the run, and seen how it works, I'm game to try it again next year, and I'm game to do more 10km runs with Zoe. My running overall has been going very well, and so I think it's time to step things up a bit and I've decided to set myself a goal of running the half-marathon in the Gold Coast Marathon next year.

Review: An Artificial Night, by Seanan McGuire

Series:	October Daye #3
Publisher:	DAW
Copyright:	September 2010
ISBN:	1-101-44276-X
Format:	Kindle
Pages:	368

This is the third book of the October Daye series, following A Local Habitation. It could be read out of order, as each chapter of Toby's story so far is somewhat standalone, but I wouldn't recommend it. You'd miss the character development and the background behind several friendships. As with the previous book review, a caveat: I am reviewing this book four months after finishing it, rather than the normal couple of weeks, so apologies in advance for any thinness or inaccuracies. The first sign that something is about to go seriously wrong in Toby's life this time is a visitor. She says her name is May Daye, and apparently she's Toby's Fetch: a duplicate in nearly every respect except personality, manifested to guide someone to their death. Their imminent death. The second sign is that her best friend's children are kidnapped. The children who treat her like an aunt. And the cause of that kidnapping is far, far beyond the level of power that Toby could be expected to deal with. I have profound mixed feelings about this book. On one hand, McGuire further expands Toby's web of connections and shows us more of Toby's friends and their background, and that's becoming my favorite part of this series. She introduces May, who I think is a great character: at first apparently much different than Toby, but with a similarity beneath surface impressions that one notices more over time. And I thought the climax of the story, once we finally got there, was very moving and well-done. Also, I just love the way that McGuire uses children's rhymes and doggerel throughout this series as a magical element, and that's particularly well-handled here. Due to the nature of the antagonist, McGuire has a good excuse to bring that front and center, and the process of familiar rhymes expanding into deep magical rules was a fun intellectual experience. Best of all, the Luidaeg plays a significant role in the story again. The Luidaeg is, hands down, my favorite character in McGuire's constructed universe, and she becomes even more so over the course of this story. Her complex relationship with Toby is clearly blossoming into a friendship that's surprising both of them deeply, but a friendship that has to play by an odd set of rules (like so much else in Toby's life). I loved her here. Seeing someone that difficult and abrupt show unexpected emotional depths is one of the story elements I enjoy the most. The problem, though, is that quite a lot of An Artificial Night is very, very dark. And by that, I don't mean that there is a lot of tension, or that Toby is struggling against monsterous things, although both of those statements are also true. Rather, Toby is not only out of her league but ends up in some viciously nasty situations, and they persist for rather more of this book than I would have preferred. Putting one's protagonist through hell has a long tradition in urban fantasy, but I found the level of sheer helplessness that Toby struggled through to be a bit more than I wanted to read about. It was too reminiscent of events from Toby's background (as discussed in Rosemary and Rue) that I still don't like to think about. There is, of course, an eventual payoff, as dark as the interim events seem. One knows that, given the genre and given that this is an ongoing series, and even just given the nature of stories. But as much as I like the growing cast of this series, and adored the scenes with the Luidaeg, I found parts of this journey to be deeply disturbing, and not in an enjoyable dramatic tension sort of way. I don't mind protagonists being seriously hurt (although it's not my favorite plot), or even some amount of despair, but this went beyond that. I think McGuire was going for a story about sacrifice and self-definition, about trying even when one can't possibly win, and I can appreciate the narrative arc. But parts of the book were simply awful to read for me. I'm therefore not quite sure what to say in terms of a recommendation. I'm still reading the series, and will definitely read the next book. I think my reaction is somewhat idiosyncratic; the events here will probably not bother others as much as they bothered me. And I can't disagree with their dramatic effectiveness, or with the way that McGuire builds conflict out of the rules of fairy and is building Toby into a stronger and more capable person than she ever thought she could be. But there were still spots where I wished I were reading a different book. Hopefully the next book in the series will be that book I wanted to read. Followed by Late Eclipses. Rating: 6 out of 10

For me, DebConf13 in Switzerland was a perfect DebConf, and generally an great developer meeting and conference. I've attended several past DebConfs in Canada, Norway, Brazil, Finland, Mexico, Scotland, Spain, NYC, Bosnia, and Nicaragua. I have special memories of each. DebConf13 was ideal in every way. Let's start with the camping. Only a minority of us did it, and my campsite was located in a small field behind a bar that was often noisy late into the night, when I was trying to rest and preserve my European jetlag remedy of inverted sleeping schedule. Still, I love sleeping cocooned in a tent, and going through the routines of camping and spending time alone there was a good counterpoint to DebConf. I generally find myself retreating for a day or two in the middle of a week-long conference, burnt out with social interactions. That didn't happen this time. I need to find way to camp at future DebConfs. (Also special shout out to wonderful people who lent me a warmer sleeping bag, and bought me a pillow while on a trip to town! Upgraded my first backpacking in Europe experience's comfort level a lot!) The next perfect thing for me were the talks I gave at DebConf. I always enjoy presenting at DebConf, and barely remember when I'd get nervous doing it. But I really felt comfortable giving these talks in a way I've not before. I'm pleased with how the Debian Cosmology talk turned out; I'd worried it would be a downer talk, or too light, or not well received, and none of those problems surfaced. I was especially happy with my dh_busfactor session, which I prepared for all of 10 minutes and was structured entirely to benefit me by talking over my thoughts and getting feedback, and in which I probably drove the camerapeople crazy running my own mic into the audience. Despite all that, I think it succeeded being interesting for the participants as well as for me. Hope to follow up on it soon.. But beyond the two talks I submitted, there were a series of ad-hoc scheduled talks. It seemed that every day I spent at least an hour discussing something I'm deeply involved in. This started with the Ikiwiki bof, and then there was an epic series of (unrecorded) BoFs about git stuff, which basically turned into an unofficial git track at DebConf. Much, much discussion of git-annex and other things I'm involved in. BTW, I really like that DebConf, as well as being a formal conference on top, is really an un-conference underneath, with new BoFs and things popping up on the schedule all week long. You can even have completely unscheduled meetings that somehow get just the right mix of people to make something awesome happen.
But it wasn't all about talks. Indeed, I only attended 2 or 3 sessions a day. The DebConf13 venue was controversial, but I found it excellent for getting things done. Rather than one or two massive rooms that get too noisy to think in, this venue had many little nooks and crannies suited for different types of work. There were so many I only discovered some while walking down the hill as I was leaving the venue! Anchoring it all was the excellent view, and I particularly enjoyed the "scenic hacklab". I was particularly impressed with the flexability of the venue's management. Someone asked if we could have cheese for breakfast, and every breakfast thereafter had not one but often three excellent cheeses. One day we'd be complaining that there were not enough vegetables in the food, and the next we'd be eating an excellent vegetarian Swiss lunch. They even apparently relaxed stringent Swiss rules on not allowing tents in sightlines visible from people further up the hill. They were clearly happy to welcome our crazy gathering to Le Camp. They let a DD borrow and use a chain saw and build a bonfire! The venue also encouraged getting out and exploring the surrounding countryside. Too often I end up spending the whole DebConf without seeing much of the area around. This time I at least got down the hill to see the castle in Vaumarcus a few times, and enjoyed many evening walks amoung the grape fields. The conference dinner (on a boat!) tied into this well too. And the daytrip to CERN was a great addition. We got to poke around the control center for a massive particle accellerator! My only regret is we didn't get a chance to actually go inside the LHC tunnels. Back to DebConf, another thing I really liked about this one was that its location allowed several upstream authors of software in Debian (other than the usual suspects who are both upstreams and Debian Developers) to attend. I don't know if Lennart Poettering changed any minds about systemd with his talk, but I was happy for the opportunity to buy him a beer. I didn't get to Monty's talk, but it was excellent to see him attend as well. Finally, I got to meet old friends I'd not seen in years, and new ones I've not met in real life before. We're still not sure where this whole DebConf thing is going -- will it be all about the cheese and wine party and greybeards in 20 years time? But one thing we know is we enjoy our little slice of the year where kilts and coding are equally unremarkable, where problems that seem intractable online somehow solve themselves in person, where we come away with ideas for an entire year and memories for a lifetime.

This is part 5 in a series on organizational design and growth. What it means

Each of us has a job to do, probably more than one, and our teammates should know what they are. Why it s important

Roles are a kind of standing commitment we make to each other. They re a way of dividing up work which is easy to understand and simple to apply. Utilizing this tool will make it easier for us to coordinate our day to day work, and manage the changes and growth we re going through. Old Status Quo

Roles are vague or nonexistent. Management roles in particular are probably not well understood. Many people juggle multiple roles, all of which are implicit. Moving to a new team means learning from scratch what other people do. People take responsibility for tasks and decisions largely on a case-by-case basis, or based on implicit knowledge of what someone does (or doesn t do). In many cases, there is only one person in the company who knows how to perform a certain function. When someone leaves or goes on vacation, gaps are left behind. New Status Quo

Each individual has a clear understanding of the scope of their job. We have a handful of well defined roles, which are used by multiple teams and have written definitions. People who are new or transfer between teams have a relatively easy time understanding what the people around them are doing. Many day to day responsibilities are defined by roles, and more than one person can fill that role. When someone leaves a team, another person can cover their critical roles. Behaviors that help

• Define project roles: when starting something new, make it explicit who will be working on it. Often this will be more than one person, often from different teams. For example, customer-facing product changes should have at least a product owner and an engineering owner. This makes it easy to tell if too many concurrent projects are dependent on a single person, which is a recipe for blockage.

• Define team roles: Most recurring tasks should fall within a defined role. An owner of a technical service is an example of a role. An on-call engineer is an example of a time-limited role. There are many others which will depend on the team and its scope.

• Define job roles: Have a conversation with your teammates and manager about what the scope of your job is, which responsibilities are shared with other members of the team and which are yours alone.

Obstacles that stand in our way

• Getting hung up on titles as ego gratification. Roles are tools, not masters.

• Fear that a role limits your options, locks you into doing one thing forever. Roles can be as flexible as we want them to be.

Confused yet? The popular browser extension AdBlock for Chrome/Chromium and Safari has launched a new crowdsourced campaign - to buy up internet advertising space inviting people to download AdBlock, so they can block ads. Additionally, if they raise enough, they'd like to get additional space for a billboard at Times Square, a full page spread in the New York Times, and even a TV slot at the Superbowl (although the target for this is $4.2M, so I'm not entirely confident they'll make it.) However, this does raise an interesting question, which has been under debate for some time. How do you make money on the internet without adverts? Wikipedia has a few interesting suggestions. Firstly, affiliate programmes. These mean that any products or services that are linked or searched via the original site earn a little bit of commission for that site. Founded in the mid-90s, and most famously used by Amazon, it's a possible method for getting money. However, it should be deployed with careful thought. If a blogger or reviewer is recommending a product, is it simply because they are being paid to do so, or is it a genuine endorsement? The BBC ban this practice, and the Society of Professional Journalists also has strong guidelines on taking gifts. Interestingly, a few search engines now offer this to browsers for them sending their default search traffic to them. Secondly, you could engage in (what I consider to be) the shady practice of data monetisation. People visit a website, and they leave data about their visit. By visiting here, you've told me that you're probably interested in technology issues, what country and city you're from, and where you came from. Other programmes and sites will also be able to wrap up what other sites you've visited, and can build up quite a profile of your browsing habits. Add together a rather unscrupulous vendor, and they can also know your contact details. This metadata can then be sold on. Thirdly, you can essentially ask your readers for money. Donations, via a micropayment link (which I use) or larger donation requests. Another method is to have a membership system, where content is locked up behind a paywall. This does, however, risk making your content less valuable as people will just head elsewhere for news and opinion. To counter this, a couple of sites (LWN, Slashdot) simply delay the latest news for a little while. Alternatively, you can just ask like the fantastic C G P Grey has done. Interestingly, he also has a great video on why advertising can be a bad thing for content creators. Lastly, you can sell things. This could be your authorship services on another site as a guest post, writing a book, offering consultancy services, or simple merchandise. In summary though, which is more honest? An advert may well bend your view on what sort of content is being created, but is at least a lot more open than potentially hiding real motives behind affiliate links, and a lot more open than selling data. Asking for money is fairly straightforward, though could easily backfire if it's viewed as begging. Google's advertising income totalled $43.7 billion in 2012 - so it's quite clear that advertising is big business. Websites cost money to run, can the web survive in an ad-free environment? If so, it certainly won't be the same web as we know today.