Search Results: "ron"

7 September 2013

Ben Armstrong: Wifi roaming on the move redux

It has been nearly six years with a netbook and five since I last wrote about wifi roaming from the bus to stay on irc without a costly celluar link during the daily commute. Since then, some readers have asked me to share my refinements to the method in a followup post. So here it is. The software On the server: On the client: Putting it together: on the client Make sure if you have a wireless manager installed (such as NetworkManager) it is configured to skip your wireless interface, disabled entirely, or if possible, removed. Set up /etc/wpa_supplicant/wpa_supplicant.conf and /etc/network/interfaces for roaming, as per the instructions in /usr/share/doc/wpasupplicant/README.modes.gz. Don t forget to add yourself to the netdev group if you are not in it already. In /etc/wpa_supplicant/wpa_supplicant.conf, list common names of open networks. Normally the catch-all network that associates with any essid, i.e. the first stanza below, works well. However, occasionally the strongest signal is neither one of the common networks nor an easily accessible network (e.g. web portals), so having a list of common open networks helps to quickly select from among those instead. The more you travel, the more of these will discover and add. Just use reconfigure from wpa_cli to reload your edited list each time you add a new one.
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network= 
        key_mgmt=NONE
 
network= 
        ssid="default"
        key_mgmt=NONE
 
network= 
        ssid="linksys"
        key_mgmt=NONE
 
...
Since you ll be using ssh repeatedly to connect and it has to be fast, make sure your server is set up to accept your key and use ssh-add so that you only have to enter your ssh key password once. You can tweak isc-dhcp-client to make connections faster. In /etc/dhcp/dhclient.conf, use:
backoff-cutoff 1;
initial-interval 1;
Here are a few scripts I wrote to facilitate quick roaming from one open AP to another and reconnect to irssi running in screen, to break a connection and try the next one, and to recover from occasional lockups (more about that later). ~/bin/screen_reconnect This is a script to reconnect continuously via ssh to a screen session:
#!/bin/sh
reset
while ! ssh -t 10.9.8.7 'screen -UDr' 2>/dev/null ; do echo -n "." ; sleep .1 ; done
Just substitute the IP of your own server here. Using an IP instead of domain name makes the connection faster because a DNS lookup is not required. ~/bin/wifi_reassociate This script closes any open ssh sessions and informs wpa_supplicant to attempt to connect again.
#/bin/sh
/sbin/wpa_cli rea
killall ssh >/dev/null 2>&1
~/bin/wifi_killall This optional, somewhat ugly script addresses an issue I hope you never have. On my ASUS Eee PC 1001PX, occasionally scanning stops. When this happens, and I have never figured out why, apparently ACPI events are blocked. At this point wifi becomes unusable and ACPI sleep is inhibited. By trial and error I found that if you bring down the interface, kill all network-related processes, and bring it up again, ACPI events are unblocked and wifi is usable once more (and any pending request to sleep will finally happen). The script requires sudo, and to use the openbox key binding, gksudo.
#/bin/sh
sudo ifdown wlan0
# in case any of these are hung
sudo killall dhclient3
sudo killall wpa_cli
sudo killall wpa_action
sudo killall wpa_supplicant
# in case any of these are *really* hung
sleep 1
sudo killall -9 dhclient3
sudo killall -9 wpa_cli
sudo killall -9 wpa_action
sudo killall -9 wpa_supplicant
sudo dhclient -r
sudo ifup wlan0
Openbox Since certain actions need to be performed repeatedly and quickly, it is useful to have hotkeys bound in your window manager to the scripts. In ~/.config/openbox/rc.xml, key bindings for <alt>-r to reassociate and <alt>-d to disconnect a hung connection would look like:
  
<keyboard>
  <!-- My keybindings -->
  <keybind key="A-R">
    <action name="Execute">
        <execute>/home/synrg/bin/wifi_reassociate</execute>
    </action>
  </keybind>
  <keybind key="A-D">
    <action name="Execute">
        <execute>gksudo /home/synrg/bin/wifi_killall</execute>
    </action>
  </keybind>
</keyboard>
Putting it together: on the server There is very little to do here. Just start screen, and start irssi in screen. Running screen on the client as well as the server means you should either bind the screen meta keys to a different key sequence on each system, or else learn to press meta twice to pass through meta to the server screen as needed. I use the latter approach. Alternatively, you could use a tabbed terminal on the client, or separate terminals per client process instead of screen. This is a matter of personal taste. Ready to roam Here is a typical setup for roaming on the bus: In a terminal (I use urxvt), first ssh-add, then start screen with these three processes running in separate virtual terminals: March of the dots Most of the commute, just enjoy watching the dots march by, waiting for a new connection. If you estimate a connection is unusable, press <alt>-r to reassociate immediately, giving the next network a chance. If the connection is already firmly established, this might not work on the first try. If the dots don t resume immediately, wait a bit and press it again. This might take a few tries. Changing selected networks on the fly Use wpa_cli when you need to do some fine-tuning of network selections on the fly. While normally you can just watch the march of the dots until a connection is acquired, sometimes you can improve your chances of connecting to a good network by manually controlling the selected candidate networks here. For example, by watching the speed of the bus relative to known good APs, you can predict which networks are more likely to succeed. Rather than connect to any arbitrary network, you might select a specific one by id, and then later when it goes out of range, revert to the original configuration, e.g.
> select_network 5
...
> reconfigure
You can use tab-completion in wpa_cli to type these commands quickly or else just abbreviate the commands. Another common scenario is when you pass through a business area with many captive portal hotspots. These rarely make good choices because they either require a password not known to you or else you can t click through I agree in time before the bus moves on. In this case, you might just disable the catch-all stanza and let the common open network stanzas you listed ( default , linksys , etc.) do the work:
> disable_network 1
Become a type ahead wizard While running, a continuous stream of periods fills the screen, which provides you with a highly visible cue that no available APs are in range. When the movement stops, you know a connection is being attempted. While waiting to connect, you can type ahead any comments you want to make in the current irssi window (taking care to remember which one you are in!) While having periods interspersed in what you type may be disorienting at first, you get used to it. There is a point when a connection is first established and ssh is accepting input, but anything you type can no longer be seen while you re typing. Depending on whether the connection was completely successful or not, what you type now may or may not finally be sent. For best results, only type ahead before the dots stop moving. Eventually you can become skilled enough at this to type ahead a comment in one channel, switch channels with /win # and continue typing ahead in the new channel, all buffered until the next few seconds (or even fraction of a second) of connection time. Fine-tune antenna direction with wavemon When the bus has come to a standstill, you may find wavemon useful to pull in a weak signal. Because wavemon has continuously updated signal level and link quality bars, you can use it to fine-tune the antenna position. Just turn your laptop until the bars are at their maximum. Captive portals I have not figured out how to do any automation for this, so it really is a crapshoot, as it is likely the bus has moved on by the time you ve managed to manually navigate the login through a captive portal. But in rush hour, you may have the luxury of time to connect to these as you pass them. I have recently learned about the CoovaFX Firefox plugin which automates logins to captive portals. I m going to give it a try to see if it helps. Update: I can t recommend this plugin, as it is not compatible with Iceweasel >= 23.0. Also, the standard it is based on, WISPr, appears to have an uncertain future. That, coupled with the fact that the plugin appears to not be open source means I m still looking for alternatives. Summary If all of this sounds a bit nuts to you, well, it probably is. But after half a decade enjoying free access to irc from the bus, it all seems perfectly natural to me! If you try this method and like it, please let me know in the comments. Likewise, if you have any improvements to the process or scripts, please share them!

6 September 2013

Sylvain Le Gall: OUnit 2.0 progress, August 2013

After a long pause, I have resumed my work on OUnit2. It is going quite well. The basic features: I still need to do the following: There is still a lot of work, but the current results are already quite good. The speed improvement of the processes runner is a good thing to shorten test timing (HINT: tester needed!). Focus on: the new bracket. In OUnit 1, a bracket was very functional:
 bracket 
    (fun () -> "foo")  (* setup *)
    (fun foo -> ())
    (fun foo -> ())  (* tear down *)
So for common bracket, like bracket_tmpfile
   bracket_tmpfile
       (fun (fn, chn) ->
             (* Do something with chn and fn *)
The problem is that if you were using 2 or 3 temporary files, the level of indentation was high. I have decide to switch to a more imperative approach, registering the tear down function inside the test context:
   let (fn1, chn1) = bracket_tmpfile ctxt in
   let (fn2, chn2) = bracket_tmpfile ctxt in
      ....
This is shorter and clearer (albeit less functional). Focus on: non fatal section Sometimes, you want to verify a set of properties but to have a clear vision of what is going wrong, you need to do more than one assert. In OUnit1, you can do:
 assert_equal exp1 v1;
 assert_equal exp2 v2
But if exp1 <> v1, you quit immediately and you'll never know for exp2 and v2. In OUnit2, you can do:
 non_fatal ctxt (fun ctxt -> assert_equal exp1 v1);
 non_fatal ctxt (fun ctxt -> assert_equal exp2 v2)
In this new version, you will test both equality and the result of the test will be the worst failure you get (or success if both of them succeed). Help wanted If you have a long standing issue with OUnit, this is the time to submit a bug. OUnit BTS If you want to try the dev version of OUnit:
 $> darcs get http://forge.ocamlcore.org/anonscm/darcs/ounit/ounit
Patches always welcome. Special thanks to Thomas Wickham who has entirely written OUnitRunnerThreads and kickstarted the processes runner.

Vincent Bernat: High availability with ExaBGP

When it comes to provide redundant services, several options are available: A common setup is a combination of those solutions: web servers are behind a couple of load-balancers achieving both redundancy and load-balancing. The load-balancers use VRRP to ensure redundancy. The whole setup is replicated to another datacenter and round-robin DNS is used to ensure both redundancy and load-balacing of the datacenters. There is a fourth option which is similar to VRRP but relies on dynamic routing and therefore is not limited to nodes in the same subnet: We will explore how to implement this fourth option using ExaBGP, the BGP swiss army knife of networking, in a small lab based on KVM. You can grab the complete lab from GitHub. ExaBGP 3.2.5 is needed to run this lab.

Environment We will be working in the following (IPv6-only) environment: Lab with three web nodes

BGP configuration BGP is enabled on ER2 and ER3 to exchange routes with peers and transits (only R1 for this lab). BIRD is used as a BGP daemon. Its configuration is pretty basic. Here is a fragment:
router id 1.1.1.2;
protocol static NETS   #  
  import all;
  export none;
  route 2001:db8::/40 reject;
 
protocol bgp R1   #  
  import all;
  export where proto = "NETS";
  local as 64496;
  neighbor 2001:db8:1000::1 as 64511;
 
protocol bgp ER3   #  
  import all;
  export all;
  next hop self;
  local as 64496;
  neighbor 2001:db8:1::3 as 64496;
 
First, in , we declare the routes that we want to export to our peers. We don t try to summarize them from the IGP. We just unconditonnaly export the networks that we own. Then, in , R1 is defined as a neighbor and we export the static route we declared previously. We import any route that R1 is willing to send us. In , we share everything we know with our pal, ER3, using internal BGP.

OSPF configuration OSPF will distribute routes inside the AS. It is enabled on ER2, ER3, DR6, DR7 and DR8. For example, here is the relevant part of the configuration of DR6:
router id 1.1.1.6;
protocol kernel  
   persist;
   import none;
   export all;
 
protocol ospf INTERNAL  
  import all;
  export none;
  area 0.0.0.0  
    networks  
      2001:db8:1::/64;
      2001:db8:6::/64;
     ;
    interface "eth0";
    interface "eth1"   stub yes;  ;
   ;
 
ER2 and ER3 inject a default route into OSPF:
protocol static DEFAULT  
  import all;
  export none;
  route ::/0 via 2001:db8:1000::1;
 
filter default_route  
  if proto = "DEFAULT" then accept;
  reject;
 
protocol ospf INTERNAL  
  import all;
  export filter default_route;
  area 0.0.0.0  
    networks  
      2001:db8:1::/64;
     ;
    interface "eth1";
   ;
 

Web nodes The web nodes are pretty basic. They have a default static route to the nearest router and that s all. The interesting thing here is that they are each on a separate IP subnet: we cannot share an IP using VRRP2. Why are those web servers on different subnets? Maybe they are not in the same datacenter or maybe your network architecture is using a routed access layer. Let s see how to use BGP to enable redundancy of those web nodes.

Redundancy with ExaBGP ExaBGP is a convenient tool to plug scripts into BGP. They can then receive and advertise routes. ExaBGP does the hard work of speaking BGP with your routers. The scripts just have to read routes from standard input or advertise them on standard output.

The big picture Here is the big picture: Using ExaBGP to advertise web services Let s explain it step by step:
  1. Three IP addresses will be allocated for our web service: 2001:db8:30::1, 2001:db8:30::2 and 2001:db8:30::3. Those are distinct from the real IP addresses of W1, W2 and W3.
  2. Each web node will advertise all of them to the route servers we added in the network. I will talk more about those route servers later.
  3. Each route comes with a metric to help the route server choose where it should be routed. We choose the metrics such that each IP address will be routed to a distinct web node (unless there is a problem).
  4. The route servers (which are not routers) will then advertise the best routes they learned to all the routers in our network. This is still done using BGP.
  5. Now, for a given IP address, each router knows to which web node the traffic should be directed.
Here are the respective metrics announced routes for W1, W2 and W3 when everything works as expected:
Route W1 W2 W3 Best Backup
2001:db8:30::1 102 101 100 W3 W2
2001:db8:30::2 101 100 102 W2 W1
2001:db8:30::3 100 102 101 W1 W3

ExaBGP configuration The configuration of ExaBGP is quite simple:
group rs  
  neighbor 2001:db8:1::4  
    router-id 1.1.1.11;
    local-address 2001:db8:6::11;
    local-as 65001;
    peer-as 65002;
   
  neighbor 2001:db8:8::5  
    router-id 1.1.1.11;
    local-address 2001:db8:6::11;
    local-as 65001;
    peer-as 65002;
   
  process watch-nginx  
      run /usr/bin/python /lab/healthcheck.py -s --config /lab/healthcheck-nginx.conf --start-ip 0;
   
 
A script is entailed to check if the service (an nginx web server) is up and running and advertise the appropriate IP addresses to the two declared route servers. If we run the script manually, we can see the advertised routes:
$ python /lab/healthcheck.py --config /lab/healthcheck-nginx.conf --start-ip 0
INFO[healthcheck] send announces for UP state to ExaBGP
announce route 2001:db8:30::3/128 next-hop self med 100
announce route 2001:db8:30::2/128 next-hop self med 101
announce route 2001:db8:30::1/128 next-hop self med 102
[...]
WARNING[healthcheck] Check command was unsuccessful: 7
INFO[healthcheck] Output of check command:  curl: (7) Failed connect to ip6-localhost:80; Connection refused
WARNING[healthcheck] Check command was unsuccessful: 7
INFO[healthcheck] Output of check command:  curl: (7) Failed connect to ip6-localhost:80; Connection refused
WARNING[healthcheck] Check command was unsuccessful: 7
INFO[healthcheck] Output of check command:  curl: (7) Failed connect to ip6-localhost:80; Connection refused
INFO[healthcheck] send announces for DOWN state to ExaBGP
announce route 2001:db8:30::3/128 next-hop self med 1000
announce route 2001:db8:30::2/128 next-hop self med 1001
announce route 2001:db8:30::1/128 next-hop self med 1002
When the service becomes unresponsive, the healthcheck script detect the situation and retry several times before acknowledging that the service is dead. Then, the IP addresses are advertised with higher metrics and the service will be routed to another node (the one advertising 2001:db8:30::3/128 with metric 101). This healthcheck script is now part of ExaBGP.

Route servers We could have connected our ExaBGP servers directly to each router. However, if you have 20 routers and 10 web servers, you now have to manage a mesh of 200 sessions. The route servers are here for three purposes:
  1. Reduce the number of BGP sessions (from 200 to 60) between equipments (less configuration, less errors).
  2. Avoid modifying the configuration on routers each time a new service is added.
  3. Separate the routing decision (the route servers) from the routing process (the routers).
You may also ask yourself: why not use OSPF? . Good question!
  • OSPF could be enabled on each web node and the IP addresses advertised using this protocol. However, OSPF has several drawbacks: it does not scale, there are restrictions on the allowed topologies, it is difficult to filter routes inside OSPF and a misconfiguration will likely impact the whole network. Therefore, it is considered a good practice to limit OSPF to network equipments.
  • The routes learned by the route servers could be injected into OSPF. On paper, OSPF has a next-hop field to provide an explicit next-hop. This would be handy as we wouldn t have to configure adjacencies with each router. However, I have absolutely no idea how to inject BGP next-hop into OSPF next-hop. What happens is that BGP next-hop is resolved locally using OSPF routes. For example, if we inject BGP routes into OSPF from RS4, RS4 will know the appropriate next-hop but other routers will route the traffic to RS4.
Let s look at how to configure our route servers. RS4 will use BIRD while RS5 will use Quagga. Using two different implementations will help with resiliency by avoiding a bug to hit the two route servers at the same time.

Bird configuration There are two sides for BGP configuration: the BGP sessions with the ExaBGP nodes and the ones with the routers. Here is the configuration for the later:
template bgp INFRABGP  
  export all;
  import none;
  local as 65002;
  rs client;
 
protocol bgp ER2 from INFRABGP  
  neighbor 2001:db8:1::2 as 65003;
 
protocol bgp ER3 from INFRABGP  
  neighbor 2001:db8:1::3 as 65003;
 
protocol bgp DR6 from INFRABGP  
  neighbor 2001:db8:1::6 as 65003;
 
protocol bgp DR7 from INFRABGP  
  neighbor 2001:db8:1::7 as 65003;
 
protocol bgp DR8 from INFRABGP  
  neighbor 2001:db8:1::8 as 65003;
 
The AS number used by our route server is 65002 while the AS number used for routers is 65003 (the AS numbers for web nodes will be 65001). They are reserved for private use by RFC 6996. All routes known by the route server are exported to the routers but no routes are accepted from them. Let s have a look at the other side:
# Only import loopback IPs
filter only_loopbacks   #  
  if net ~ [ 2001:db8:30::/64 128,128  ] then accept;
  reject;
 
# General template for an EXABGP node
template bgp EXABGP  
  local as 65002;
  import filter only_loopbacks; #  
  export none;
  route limit 10; #  
  rs client;
  hold time 6; #  
  multihop 10;
  igp table internal;
 
protocol bgp W1 from EXABGP  
  neighbor 2001:db8:6::11 as 65001;
 
protocol bgp W2 from EXABGP  
  neighbor 2001:db8:7::12 as 65001;
 
protocol bgp W3 from EXABGP  
  neighbor 2001:db8:8::13 as 65001;
 
To ensure separation of concerns, we are being a bit more picky. With and , we only accept loopback addresses and only if they are contained in the subnet that we reserved for this use. No server should be able to inject arbitrary addresses into our network. With , we also limit the number of routes that a server can advertise. With , we reduce the hold time from 240 to 6. It means that after 6 seconds, the peer is considered dead. This is quite important to be able to recover quickly from a dead machine. The minimal value is 3. We could have use a similar setting with the session with the routers.

Quagga configuration Quagga s configuration is a bit more verbose but should be strictly equivalent:
router bgp 65002 view EXABGP
 bgp router-id 1.1.1.5
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor R peer-group
 neighbor R remote-as 65003
 neighbor R ebgp-multihop 10
 neighbor EXABGP peer-group
 neighbor EXABGP remote-as 65001
 neighbor EXABGP ebgp-multihop 10
 neighbor EXABGP timers 2 6
!
 address-family ipv6
 neighbor R activate
 neighbor R soft-reconfiguration inbound
 neighbor R route-server-client
 neighbor R route-map R-IMPORT import
 neighbor R route-map R-EXPORT export
 neighbor 2001:db8:1::2 peer-group R
 neighbor 2001:db8:1::3 peer-group R
 neighbor 2001:db8:1::6 peer-group R
 neighbor 2001:db8:1::7 peer-group R
 neighbor 2001:db8:1::8 peer-group R
 neighbor EXABGP activate
 neighbor EXABGP soft-reconfiguration inbound
 neighbor EXABGP maximum-prefix 10
 neighbor EXABGP route-server-client
 neighbor EXABGP route-map RSCLIENT-IMPORT import
 neighbor EXABGP route-map RSCLIENT-EXPORT export
 neighbor 2001:db8:6::11 peer-group EXABGP
 neighbor 2001:db8:7::12 peer-group EXABGP
 neighbor 2001:db8:8::13 peer-group EXABGP
 exit-address-family
!
ipv6 prefix-list LOOPBACKS seq 5 permit 2001:db8:30::/64 ge 128 le 128
ipv6 prefix-list LOOPBACKS seq 10 deny any
!
route-map RSCLIENT-IMPORT deny 10
!
route-map RSCLIENT-EXPORT permit 10
  match ipv6 address prefix-list LOOPBACKS
!
route-map R-IMPORT permit 10
!
route-map R-EXPORT deny 10
!
The view is here to not install routes in the kernel3.

Routers Configuring BIRD to receive routes from route servers is straightforward:
# BGP with route servers
protocol bgp RS4  
  import all;
  export none;
  local as 65003;
  neighbor 2001:db8:1::4 as 65002;
  gateway recursive;
 
protocol bgp RS5  
  import all;
  export none;
  local as 65003;
  neighbor 2001:db8:8::5 as 65002;
  multihop 4;
  gateway recursive;
 
It is important to set gateway recursive because most of the time, the next-hop is not reachable directly. In this case, by default, BIRD will use the IP address of the advertising router (the route servers).

Testing Let s check that everything works as expected. Here is the view from RS5:
# show ipv6  bgp  
BGP table version is 0, local router ID is 1.1.1.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*  2001:db8:30::1/128
                    2001:db8:6::11         102             0 65001 i
*>                  2001:db8:8::13         100             0 65001 i
*                   2001:db8:7::12         101             0 65001 i
*  2001:db8:30::2/128
                    2001:db8:6::11         101             0 65001 i
*                   2001:db8:8::13         102             0 65001 i
*>                  2001:db8:7::12         100             0 65001 i
*> 2001:db8:30::3/128
                    2001:db8:6::11         100             0 65001 i
*                   2001:db8:8::13         101             0 65001 i
*                   2001:db8:7::12         102             0 65001 i
Total number of prefixes 3
For example, traffic to 2001:db8:30::2 should be routed through 2001:db8:7::12 (which is W2). Other IP are affected to W1 and W3. RS4 should see the same thing4:
$ birdc6 show route 
BIRD 1.3.11 ready.
2001:db8:30::1/128 [W3 22:07 from 2001:db8:8::13] * (100/20) [AS65001i]
                   [W1 23:34 from 2001:db8:6::11] (100/20) [AS65001i]
                   [W2 22:07 from 2001:db8:7::12] (100/20) [AS65001i]
2001:db8:30::2/128 [W2 22:07 from 2001:db8:7::12] * (100/20) [AS65001i]
                   [W1 23:34 from 2001:db8:6::11] (100/20) [AS65001i]
                   [W3 22:07 from 2001:db8:8::13] (100/20) [AS65001i]
2001:db8:30::3/128 [W1 23:34 from 2001:db8:6::11] * (100/20) [AS65001i]
                   [W3 22:07 from 2001:db8:8::13] (100/20) [AS65001i]
                   [W2 22:07 from 2001:db8:7::12] (100/20) [AS65001i]
Let s have a look at DR6:
$ birdc6 show route
2001:db8:30::1/128 via fe80::5054:56ff:fe6e:98a6 on eth0 * (100/20) [AS65001i]
                   via fe80::5054:56ff:fe6e:98a6 on eth0 (100/20) [AS65001i]
2001:db8:30::2/128 via fe80::5054:60ff:fe02:3681 on eth0 * (100/20) [AS65001i]
                   via fe80::5054:60ff:fe02:3681 on eth0 (100/20) [AS65001i]
2001:db8:30::3/128 via 2001:db8:6::11 on eth1 * (100/10) [AS65001i]
                   via 2001:db8:6::11 on eth1 (100/10) [AS65001i]
So, 2001:db8:30::3 is owned by W1 which is behind DR6 while the two others are in another part of the network and will be reached through the appropriate link-local addresses learned by OSPF. Let s kill W1 by stopping nginx. A few seconds later, DR6 learns the new routes:
$ birdc6 show route
2001:db8:30::1/128 via fe80::5054:56ff:fe6e:98a6 on eth0 * (100/20) [AS65001i]
                   via fe80::5054:56ff:fe6e:98a6 on eth0 (100/20) [AS65001i]
2001:db8:30::2/128 via fe80::5054:60ff:fe02:3681 on eth0 * (100/20) [AS65001i]
                   via fe80::5054:60ff:fe02:3681 on eth0 (100/20) [AS65001i]
2001:db8:30::3/128 via fe80::5054:56ff:fe6e:98a6 on eth0 * (100/20) [AS65001i]
                   via fe80::5054:56ff:fe6e:98a6 on eth0 (100/20) [AS65001i]

Demo For a demo, have a look at the following video (it is also available as an Ogg Theora video).

  1. The primary target of VRRP is to make a default gateway for an IP network highly available by exposing a virtual router which is an abstract representation of multiple routers. The virtual IP address is held by the master router and any backup router can be promoted to master in case of a failure. In practice, VRRP can be used to achieve high availability of services too.
  2. However, you could deploy an L2 overlay network using, for example, VXLAN and use VRRP on it.
  3. I have been told on the Quagga mailing-list that such a setup is quite uncommon and that it would be better to not use views but use --no_kernel flag for bgpd instead. You may want to look at the whole thread for more details.
  4. The output of birdc6 shows both the next-hop as advertised in BGP and the resolved next-hop if the route have to be exported into another protocol. I have simplified the output to avoid confusion.

5 September 2013

Vincent Sanders: Strive for continuous improvement, instead of perfection.


Kim Collins was perhaps thinking more about physical improvement but his advice holds well for software.

A lot has been written about the problems around software engineers wanting to rewrite a codebase because of "legacy" issues. Experience has taught me that refactoring is a generally better solution than rewriting because you always have something that works and can be released if necessary.

Although that observation applies to the whole of a project, sometimes the technical debt in component modules means they need reimplementing. Within NetSurf we have historically had problems when such a change was done because of the large number of supported platforms and configurations.
HistoryA year ago I implemented a Continuous Integration (CI) solution for NetSurf which, combined with our switch to GIT for revision control, has transformed our process. Making several important refactor and rewrites possible while being confident about the overall project stability.

I know it has been a year because the VPS hosting bill from Mythic turned up and we are still a very happy customer. We have taken the opportunity to extend the infrastructure to add additional build systems which is still within the NetSurf projects means.

Over the last twelve months the CI system has attempted over 100,000 builds including the projects libraries and browser. Each commit causes an attempt to build for eight platforms, in multiple configurations with multiple compilers. Because of this the response time to a commit is dependant on the slowest build slave (the mac mini OS X leopard system).

Currently this means a browser build, not including the support libraries, completes in around 450 seconds. The eleven support libraries range from 30 to 330 seconds each. This gives a reasonable response time for most operations. The worst case involves changing the core buildsystem which causes everything to be rebuilt from scratch taking some 40 minutes.

The CI system has gained capability since it was first set up, there are now jobs that:
DownsidesIt has not all been positive though, the administration required to keep the builds running has been more than expected and it has highlighted just how costly supporting all our platforms is. When I say costly I do not just refer to the physical requirements of providing build slaves but more importantly the time required.

Some examples include:
  • Procuring the hardware, installing the operating system and configuring the build environment for the OS X slaves
  • Getting the toolchain built and installed for cross compilation
  • Dealing with software upgrades and updates on the systems
  • Solving version issues with interacting parts, especially limiting is the lack of JAVA 1.6 on PPC OS X preventing jenkins updates
This administration is not interesting to me and consumes time which could otherwise be spent improving the browser. Though the benefits of having the system are considered by the development team to outweigh the disadvantages.

The highlighting of the costs of supporting so many platforms has lead us to reevaluate their future viability. Certainly the PPC mac os X port is in gravest danger of being imminently dropped and was only saved when the build slaves drive failed because there were actual users.

There is also the question of the BeOS platform which we are currently unable to even build with the CI system at all as it cannot be targeted for cross compilation and cannot run a sufficiently complete JAVA implementation to run a jenkins slave.

An unexpected side effect of publishing every CI build has been that many non developer user are directly downloading and using these builds. In some cases we get messages to the mailing list about a specific build while the rest of the job is still ongoing.

Despite the prominent warning on the download area and clear explanation on the mailing lists we still get complaints and opinions about what we should be "allowing" in terms of stability and features with these builds. For anyone else considering allowing general access to CI builds I would recommend a very clear statement of intent and to have a policy prepared for when when users ignore the statement.
Tools
Using jenkins has also been a learning experience. It is generally great but there are some issues I have which, while not insurmountable, are troubling:
Configuration and history cannot easily be stored in a revision control system.
This means our system has to be restored from a backup in case of failure and I cannot simply redeploy it from scratch.

Job filtering, especially for matrix jobs with many combinations, is unnecessarily complicated.
This requires the use of a single text line "combination filter" which is a java expression limiting which combinations are built. An interface allowing the user to graphically select from a grid similar to the output tables showing success would be preferable. Such a tool could even generate the textural combination filter if thats easier.

This is especially problematic of the main browser job which has options for label (platform that can compile the build), javascript enablement, compiler and frontend (the windowing toolkit if you prefer e.g. linux label can build both gtk and framebuffer). The filter for this job is several kilobytes of text which due to the first issue has to be cut and pasted by hand.

Handling of simple makefile based projects is rudimentary.
This has been worked around mainly by creating shell scripts to perform the builds. These scripts are checked into the repositories so they are readily modified. Initially we had the text in each job but that quickly became unmanageable.

Output parsing is limited.
Fortunately several plugins are available which mitigate this issue but I cannot help feeling that they ought to be integrated by default.

Website output is not readily modifiable.
Instead of perhaps providing a default css file and all generated content using that styling someone with knowledge of JAVA must write a plugin to change any of the look and feel of the jenkins tool. I understand this helps all jenkins instances look like the same program but it means integrating jenkins into the rest of our projects web site is not straightforward.
ConclusionIn conclusion I think the CI system is an invaluable tool for almost any non trivial software project but the implementation costs with current tools should not be underestimated.

Gergely Nagy: What's cooking in syslog-ng 3.5?

The first alpha of syslog-ng 3.4 was released more than a year ago, and the first stable release earlier this year, development on syslog-ng 3.5 has been going on steadlily since then, with a lot of interesting features and changes pouring in. While the 3.5 release will not bring as many overwhelming changes as the previous two releases did, the sheer number of improvements is still substantial. Our focus was on smaller developments all over the place this time, instead of big sweeping changes. In this post, we will go over most of these features, with use-cases, examples and a few words about why they are so useful and awesome.Multi-line support, new destinations, new template functions, type hinting, unit suffixes, and other things. Read on, and be amazed!These are only bigger features, there has been a lot of small tweaking going on, and of course, all of the fixes that went into prior versions will also be in 3.5 too, and there's still a chance that something else may find its way in before the feature freeze.Multi-line supportOne of the major new features in the 3.5 release will be support for multi-line messages, a feature that has been available in syslog-ng PE for a good while, and which has been ported to and improved upon to the open source edition. Two variants of multi-line are supported, which will be detailed below. Both of them are available for the file() and pipe() sources only.Indented multi-lineThe easiest variant is indented multi-line, where each line can be followed by others, indented by whitespace, and the message continues until the first non-indented line. This is the format used by the Linux kernel too, from version 3.5, for /dev/log. This type of multi-line can be used as follows:
source s_multiline   file("/path/to/file" multi-line-mode(indented));  ;
Consider that /path/to/file has the following content:
First line
 Continuation 1;
 Continuation 2;
Second line
With the indented multi-line-mode() setting, this would turn into two log messages:
First line\n Continuation 1;\n Continuation 2;
Second line
Regexp-based multilineIf multi-line input is not based on indentation, one can use the regexp multi-line-mode() instead, which makes two new settings available: multi-line-prefix() and multi-line-garbage(). These can be used to define the start and the end of a log message: any string between a the beginning matching prefix and a matching garbage will be considered a single message. That is, the prefix will be included, the garbage will not be: it will be discarded.To illustrate:
source s_multiline  
 file("/path/to/file" multi-line-mode(regexp)
                      multi-line-prefix("^prefix")
                      multi-line-garbage(" garbage$"));
 ;
If the source contains these lines:
prefix message
continuing garbage
This will turn into the following log message:
prefix message\ncontinuing
New destinationsIt would be pretty hard to do a syslog-ng release without new destinations, and with 3.5, we will have three of them!STOMPWhile we had AMQP before, we now have a STOMP destination too, so we can stream logs to any STOMP server, with a few simple lines:
stomp(
 host("localhost")
 port(61613)
 destination("/topic/syslog")
 routing_key("")
 persistent(yes)
 ack(no)
 username("someone")
 password("something")
 body("This is the body! YAY! Here's a message: $MESSAGE!\n")
 value-pairs(scope(nv-pairs, syslog-proto, selected-macros))
)
Of course, none of these settings need to be set, you can just use the defaults, and it will just work!RiemannDid you know that syslog-ng is far more than a log collector and processor? No? You do now. When you have access to a lot of logs, and a tremendous amount of power in parsing them, you can use these tools for monitoring easily! And when we're talking monitoring, riemann is a great asset in our toolbox, and with the new destination, we can easily forward metrics to it:
riemann(
 server("localhost")
 port(5555)
 host("$HOST")
 service("$PROGRAM")
 description("$PROGRAM pids")
 metric(int("$PID"))
 ttl(300)
);
Of course, the above is a very silly example, it would make much more sense to extract data from a log message instead, but this will serve as an example.New template functions
$(upper-case TEXT...), $(lower-case TEXT...)
To turn some text into *upper-* or *lower-case*, these two template functions come in handy. They simply turn all their arguments into the respecitve case.
$(delimit DELIMITERS NEW-DELIMITER TEXT)
Sometimes one has a delimited string, where one wishes to replace the delimiters. This function does just that: give it a list of delimiters (you may need to quote it, if it contains whitespace), a replacement, and a text to replace delimiters in, and it does the rest. For example, to replace tabs and spaces with a vertical bar, one could write a template like this:
template("$(delimit \"\t \" \" \" $MESSAGE)\n")
$(env VARIABLE...)
Have you ever felt the need to check an environment variable from within syslog-ng? I did, and now I can.
Type hintingWhile syslog-ng supported sending log messages into various data stores and message queues for a while (SQL at first, MongoDB, JSON and AMQP later), even when those supported different types of data than strings, we could not do anything else. Until now. It is now possible - at places where it makes sense - to annotate templates with type hints, which the destination driver can optionally use. Type hinting is implemented for the mongodb() destination and the $(format-json) template function for now. When no type hint is specified, syslog-ng defaults to string.To add type hints, simply wrap the respective template with the hinted type, like this:
mongodb(
 value-pairs(pair("date", datetime("$UNIXDATE"))
             pair("pid", int64("$PID")))
);
Currently the following type hints exist: boolean (anything that begins with a t or 1 is true, anything that begins with f or 0 is false, everything else is an error), string, literal (same as string, but not quoted if it would be quoted otherwise), int32 (int is an alias for this), int64, and datetime. Only UNIX timestamps can be type-hinted to datetime, anything else will likely result in a casting error.It is also possible to control what happens when type casting fails: syslog-ng can drop the whole message, drop the property, or fall back to string. It can also do all of these silently:
options  
 typecast(on-error(silently-drop-property));
 ;
Using this feature with $(format-json) is very similary too:
$(format-json date=datetime("$UNIXDATE") pid=int64("$PID"))
With this feature in place, you can now store your non-string values with their proper types!Unit suffixesUnit suffixes make it considerably easier to set limits and describe numbers within the syslog-ng configuration. We no longer need to spell out sizes to the byte precious, it is now enough to write: log-fifo-size(200MiB). Now, syslog-ng will understand suffixes for kilo-, mega-, and giga-bytes, (K, M, G, respectively) either in base-10 or base-2 (with an extra i after the suffix). One can also omit the trailing b from the end.So, to set the log-fifo-size() to 2097152 bytes, one can simply use 2MiB. Or, to set it to 2000000, 2Mb. That's a whole lot easier, isn't it? No more counting zeros, no more silly typos in a ten-digit number, no more pain, but easily readable units!Miscellaneous featuresApart from the features above, there have been a lot of other changes and improvements in the code base:

Keith Packard: Airfest-altimeter-testing

Altimeter Testing at Airfest Bdale and I, along with AJ Towns and Mike Beattie, spent last weekend in Argonia, Kansas, flying rockets with our Kloudbusters friends at Airfest 19. We had a great time! AJ and Mike both arrived a week early at Bdale s to build L3 project airframes, and both flew successful cert flights at Airfest! Airfest was an opportunity for us to test fly prototypes of new flight electronics Bdale and I have spent the last few weeks developing, and I thought I d take a few minutes today to write some notes about what we built and flew. TeleMega We ve been working on TeleMega for quite a while. It s a huge step up in complexity from our original TeleMetrum, as it has a raft of external sensors and six pyro circuits. Bdale flew TeleMega in his new fiberglass 4 airframe on a Loki 75mm blue M demo motor. GPS tracking was excellent; you can see here that GPS altitude tracked the barometric sensor timing exactly: GPS lost lock when the motor lit, but about 3 seconds after motor burnout, it re-acquired the satellite signals and was reporting usable altitude data right away. The GPS reported altitude was higher than the baro sensor, but that can be explained by our approximation of an atmospheric model used to convert pressure into altitude. The rest of the flight was also nominal; TeleMega deployed drogue and main chutes just fine. TeleMetrum We ve redesigned TeleMetrum. The new version uses better sensors (MS5607 baro sensor, MMA6555 accelerometer) and a higher power radio (CC1120 40mW). The board is the same size, all the connectors are in the same places so it s a drop-in replacement, and it s still got two pyro channels and USB for configuration, data download and battery charging. I loaded up my Candy-Cane airframe with a small 5 grain 38mm CTI classic: The flight computer worked perfectly, but GPS reception was not as good as we d like to see: Given how well TeleMega was receiving GPS signals, I m hopeful that we ll be able to tweak TeleMetrum to improve performance. TeleMini We ve also redesigned TeleMini. It s still a two-channel flight computer with logging and telemetry, but we ve replaced the baro sensor with the MS5607, added on-board flash for increased logging space and added on-board screw terminals for an external battery and power switch. You can still use one of our 3.7V batteries, but you can also use another battery providing from 3.7 to 15V. I was hoping to finish up the firmware and fly it, but I ran out of time before the launch. The good news is that all of the components of the board have been tested and work correctly, and the firmware is feature complete , meaning we ve gotten all of the features coded, it s just not quite working yet. EasyMini EasyMini is a new product for us. It s essentially the same as a TeleMini, but without a radio. Two channels, baro-only, with logging. Like TeleMini, it includes an on-board USB connector and can use either one of our 3.7V batteries, or an external battery from 3.7V to 15V. EasyMini and TeleMini are the same size, and have holes in the same places, so you can swap between them easily. I flew EasyMini in my Koala airframe with a 29mm 3 grain CTI blue-streak motor. EasyMini successfully deployed the main chute and logged flight data: We also sent a couple of boards home with Kevin Trojanowski and Greg Rothman for them to play with. TeleGPS TeleGPS is a GPS tracker, incorporating a u-blox Max receiver and a 70cm transmitter. It can send position information via APRS or our usual digital telemetry formats. I was also hoping to have the TeleGPS firmware working, and I spent a couple of nights in the motel coding, but didn t manage to finish up. So, no data from this board either. Production Plans Given the success of the latest TeleMega prototype, we re hoping to have it into production first. We ll do some more RF testing on the bench with the boards to make sure it meets our standards before sending it out for the first production run. The goal is to have TeleMega ready to sell by the end of October. TeleMetrum clearly needs work on the layout to improve GPS RF performance. With the testing equipment that Bdale is in the midst of re-acquiring, it should be possible to finish this up fairly soon. However, the flight firmware looks great, so we re hoping to get these done in time to sell by the end of November. TeleMini is looking great from a hardware perspective, but the firmware needs work. Once the firmware is running, we ll need to make enough test flights to shake out any remaining issues before moving forward with it. EasyMini is also looking finished; I ve got a stack of prototypes and will be getting people to fly them at my local launch in another couple of weeks. The plan here is to build a small batch by hand and get them into the store once we re finished testing, using those to gauge interest before we pay for a larger production run.

2 September 2013

Daniel Pocock: Australia's Labor Party exposed spying on their own members

Australia's Labor Party claims to represent the little guy, advocating the interests of a wide slice of the population over the interests of those who might supposedly exploit them. In this latest blog I'll look at just how the Victorian state branch of the party is emulating the more sinister elements of monitoring and control that you would typically expect from marketers, spammers and the NSA. It makes you wonder, if they do this to their own members, how can Australia's government ever hope to rein in spammers or other undesirable practices? Europe has tried cookie laws and the right to be forgotten, while the ALP has taken these developments almost as a guide to exploiting their own most valuable resource, their membership. The email version of invisible ink Straight out of an email attempting to recruit party members to the federal election campaign, we find that the ALP's logo itself has been turned into a sinister tracking mechanism. Looking underneath the logo, there is a hidden tracking number attached to every message:
http://gallery.mailchimp.com/3021e68df9a7200135725c633/images/Email_Header_ThisisLabor.jpg
There it is, that big long code 3021e68df9a7200135725c633. On it's own, you may think this number can't do much harm. Think again. After all, as the filename says, "ThisisLabor". Reaching the grass roots by long distance email When a party member sees any of the images in the email or clicks any of the links, their email software or web browser sends a HTTP request to the ALP's outsourced tracking server in the United States. Here is an example of what that request may contain:
TCP Headers:
  SOURCE IP ADDRESS: 198.51.100.56
HTTP headers:
  GET /3021e68df9a7200135725c633/images/Email_Header_ThisisLabor.jpg
  HTTP/1.1
  Host: gallery.mailchimp.com
  User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130515 Firefox/17.0 Iceweasel/17.0.6
  Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
  Accept-Language: en-US,en;q=0.5
  Accept-Encoding: gzip, deflate
  X-Forwarded-For: 192.168.1.54
  Connection: keep-alive
These codes are all transmitted from the recipient of the email back to the ALP. Notice the tracking number is in the very first line of the HTTP request? These codes send many details to the tracking server. Just what is revealed?
  • What time the message was read
  • How many times the message was opened
  • Whether the message was read on multiple computers (perhaps it was forwarded)
  • The IP addresses reveal which network was used.
    • For people reading email at work or on wifi, the IP address reveals the name of the employer, it can be easily looked up in this public database. Here is an example querying one of the ALPs own IP addresses, 203.24.72.2
    • For people using an internet connection at home, the IP address reveals the service provider. There is more to it though: if one party member regularly visits another party member and uses their wifi, they will both appear to be using the same IP address and the party will be able to deduce that there is a connection between these two people.
  • The web browser name and version and the operating system name and version.
  • The language normally used on the computer
Crunching the numbers You may think that nobody would be interested in all that data or that it is even too much data for anybody to easily study. Think again. The ALP has outsourced processing of this data to an offshore firm that specializes in bulk email, MailChimp. After lecturing everybody about the dangers of outsourcing, the ALP has literally sold the farm (their own membership data) - to a foreign marketing company. Here are some of the claims MailChimp make on their web site:
"Dig even deeper with Subscriber Activity Reports, which give you a real time report for every subscriber s email activities: when they opened, what they clicked, and when they came back for more. You can even generate a list of people who didn t open, and send them a modified campaign."
That's right: if ALP members don't read their emails fast enough, they will send even more of them. Just who has access to this data? Well, the scope is quite broad. According to these rules, it is not only the dubious officials in the party's state office, but Labor Party regulations also list a whole lot of other officials who have access to some subset of the data: branch and FEA district committees, policy committees and maybe some trade union officials too. Once you start to contemplate this, it becomes apparent that the probability of somebody profiling members at an individual level is very real. On the other hand, those rules clearly do not permit the state branch to process the member's data offshore. Double whammy In fact, even MailChimp is further outsourcing the data. When receiving an image from the booby-trapped link in the email, we receive a response like this:
HTTP/1.1 200 OK
Content-Type: image/jpeg
Content-Length: 407474
Connection: keep-alive
Date: Wed, 10 Jul 2013 06:25:55 GMT
Last-Modified: Tue,  03 Sep 2013 04:05:37 GMT
ETag: "df9a7200135725c6331369a22"
Accept-Ranges: bytes
Server: AmazonS3
Age: 62097
Via: 1.0 e71fe855f29513f54a41e213fe653fb1.cloudfront.net (CloudFront)
X-Cache: Hit from cloudfront
X-Amz-Cf-Id: S2V2aW4gUnVkZCBpcyBhIGJhZCBib3kuCg==
which contains more tracking numbers and clearly indicates that the data is being processed in the massive AmazonS3 (a.k.a Amazon online bookstore) cloud platform. Thanks to cloud computing, there is considerable scope for cross-referencing records across other web sites. The IP address becomes a magic key, allowing a range of sinister forces to join the dots at the click of a button and profile the whole private and professional identity of any individual ALP member. Googling ALP members It doesn't stop there however, after following these links a victim will eventually end up on one of the ALP's own web pages. Looking through the HTML code underneath http://www.viclabor.com.au is very easy, in most web browsers, you can just press Control-U to see what is hidden away in the shadows. Buried in the code, I found this little gem:
  var _gaq = _gaq   [];
  _gaq.push(['_setAccount', 'UA-35325645-1']);
  _gaq.push(['_trackPageview']);
  (function()  
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
   )();
This may be even scarier: it's a JavaScript code that channels all access to the ALP web site into an outsourced tracking system provided by Google. Google can now cross-reference ALP members against their Gmail accounts, their friends and the topics they search for to further profile the type of advertising they should be exposed to. Google, like any private company, has a loyalty to its shareholders and the laws of the country where it is domiciled. Will that coincide with the best interests of ALP members in Victoria? Australia's other parties: samples needed Do you have similar communications from Australia's other political parties? Please send them to me and I'll be happy to examine them for similar shortcomings.

Dmitrijs Ledkovs: My Debian Activities: Multiarch & upstart on kFreeBSD hacking

Looks like there is a recent trend to publicize one's debian activities, so I thought I should join in =) Not so sure if MDA acronym would work for these types of posts...
MultiarchI have been working on multi-arching additional libraries. In particular I am working on multiarching boost libraries. At the moment boost1.53 multiarched libraries are uploaded in the experimental, and I am still working out the quirks with those. At the same time I have submitted a patch to multiarch libicu-dev, which has been accepted by the maintainer and is currently waiting in new queue. This brings us closer to multiarching all of boost libraries. But as it has been pointed out at the Multiarch BOF at Debconf'13, one doesn't have to wait for dependencies (be it libs or lib-devs) to be Multiarched before multi-arching your own library. If you do it right (put all or arch-specifi headers in multiarch location, and place libraries in multiarch location, split utilities into a separate package), you can go and multiarch your libraries ahead of time. Maybe it's time to default to debhelper 9 in dh-make, and default to multiarch locations for all libraries?

Boost also has libraries which depend on mpi, and at the moment that's not multiarched. I'm not sure if there are additional considerations there given the support for alternative mpi implementations in Debian.

Boost also depends on libpython. At the moment it's shipping libboost_python-py2.7.so & libbost-py3.3.so, with a libboost_python.so symlink to libbost_python-py2.7.so. I do wonder if it's a sensible approach, or should the boosty_python library be simply shipped in arch specific & python implementation specific paths e.g. /usr/lib/pythonX.Y/config-X.Y$(abi_tag)-$(DEB_HOST_MULTIARCH)/ that way we'll be able to support python debug interpreter in python_boost. I guess I should seek expert opinion on other multiarch wizards.

In addition to boost, I've now started looking at qmake and how it handles multiarch. At the moment, it doesn't handle it much. All paths are statically copied around and hard-coded in each module qmake provides. I have now been working at abusing qmake's $$system to call into dpkg-architecture, and actually choose native or cross compiler & adjust settings and paths to Qt libraries. There are a couple of bugs to fix, but I should be able to propose a "debian-multiarch" qmake spec soon. Such that in calls dpkg-architecture & properly adjust compilers & locations for the DEB_HOST. qmake by the way has its quirks. For example all variable assignments seemed to default to be "by value" & not "by reference" (i.e. upon assignment all variables on the right hand side get expanded and the literal string is stored, instead of storing $(prefix)/lib/$(arch), and evaluating that later.) Hence my current qmakespec is less elegant than I hoped.
upstart on kFreeBSDFreeBSD 9.2 and up have implemented the wait6 and with it waitid is also exposed. At the moment there are some differences between glibc and FreeBSD waitid usage. I am not well versed in glibc, FreeBSD and POSIX, but the bug at hand & POSIX reference lead me to believe that POSIX is slightly ambiguous about it.

Why is waitid support is interesting? Well, kFreeBSD/libc maintainer in Debian is working on exposing it with/after (e)glibc2.18 is in Debian, which will remove one more blocker from porting Upstart to kFreeBSD. To this extend, I have in the mean time setup FreeBSD & kFreeBSD virtual machines and got libnih to compile on both (sans waitid on kFreeBSD). One of the optional upstart's features is to use inotify to spot & reload configuration files, at the moment I haven't ported that functionality to use native kevent/kqueues (EVFILT_AIO) instead I have cheated and packaged for Debian the Google Summer of Code 2011 project libinotify-kevent which provides inotify compatible API, implemented with kevent. Which is kind of the reverse strategy of libkqueue and/or libevent. It would be interesting to see kernels and/or those libraries to unify/standartise on common APIs. Unfortunately, it still seems like using native API for each kernel is best to avoid dependencies and leverage all features and semantics. Nonetheless I did package it for now. To further progress with actually porting upstart, I'd like to get my hands at (e)glibc 2.18 compiled for kFreeBSD with waitid patches. I got pointers to the patches, but I haven't yet succeeded at compiling (e)glibc with those applied. So no upstart booting kFreeBSD VM yet, but rather work in progress. It's quite fun. It's been years since I have touched a FreeBSD boxes and it's first time me using kFreeBSD. The semantics, differences and benefits of each do shine and it's a promising platform.
Multiarch on kFreeBSD?
Given the excellent multiarch implementation on Debian and sophisticated Linux Emulation Layer on FreeBSD kernel, I was half expecting the following to work:
  • on kFreeBSD enable linux i386/amd64 repository
  • apt-get install hello:i386
  • $ hello
But unfortunately this confused kFreeBSD, since it didn't find ELF tags and linker didn't seem to load the right libraries any more =( Not sure if this issue has been raised before, but I think it would be awesome if FreeBSD Linux Emulation Layer worked transparently on Debian/kFreeBSD multiarch system.

1 September 2013

Eddy Petri&#537;or: HOWTO add a shell script wrapper and preserve quoting for parameters

If you ever find yourself in the situation where you have to add a shell script wrapper over a command, but the parameters' quoting gets lost and you end up with the wrong parameters in the wrapped command/tool, you might want to read this post.

On my system I have some command line tools which are Windows only and, in order to easily use the same build system as on Windows on my Linux machine I added a wrapper script which invokes wine on the commands and made symlinks to the wrapper with the file names as the tools, but without the '.exe' suffix.

Of course, I wanted to properly pass the parameters through the wrapper to the tools so I wrote (note the bold text):
#!/bin/sh
wine $0.exe "$@"
So the answer is: use $@ and quote like I did in the code above and the parameters will be passed correctly.




Update: stbuehler suggested to use exec to replace the shell process with wine with this construct:
Use:
#!/bin/sh
exec wine $0.exe "$@"

Thanks for the suggestion.

Paul Tagliamonte: Recent life in paultagland

I ve been pretty absent from the net over the last 3 weeks, (and for the first time in well over a year, I missed a week of blogging) I ve been pretty busy with real-life-stuff, nothing ultra interesting to report, I ve been working away with the ftpteam more (as I m sure some of you have noticed), and doing Hy work when I can. Nothing major on either front yet :)

Russ Allbery: Review: An Artificial Night

Review: An Artificial Night, by Seanan McGuire
Series: October Daye #3
Publisher: DAW
Copyright: September 2010
ISBN: 1-101-44276-X
Format: Kindle
Pages: 368
This is the third book of the October Daye series, following A Local Habitation. It could be read out of order, as each chapter of Toby's story so far is somewhat standalone, but I wouldn't recommend it. You'd miss the character development and the background behind several friendships. As with the previous book review, a caveat: I am reviewing this book four months after finishing it, rather than the normal couple of weeks, so apologies in advance for any thinness or inaccuracies. The first sign that something is about to go seriously wrong in Toby's life this time is a visitor. She says her name is May Daye, and apparently she's Toby's Fetch: a duplicate in nearly every respect except personality, manifested to guide someone to their death. Their imminent death. The second sign is that her best friend's children are kidnapped. The children who treat her like an aunt. And the cause of that kidnapping is far, far beyond the level of power that Toby could be expected to deal with. I have profound mixed feelings about this book. On one hand, McGuire further expands Toby's web of connections and shows us more of Toby's friends and their background, and that's becoming my favorite part of this series. She introduces May, who I think is a great character: at first apparently much different than Toby, but with a similarity beneath surface impressions that one notices more over time. And I thought the climax of the story, once we finally got there, was very moving and well-done. Also, I just love the way that McGuire uses children's rhymes and doggerel throughout this series as a magical element, and that's particularly well-handled here. Due to the nature of the antagonist, McGuire has a good excuse to bring that front and center, and the process of familiar rhymes expanding into deep magical rules was a fun intellectual experience. Best of all, the Luidaeg plays a significant role in the story again. The Luidaeg is, hands down, my favorite character in McGuire's constructed universe, and she becomes even more so over the course of this story. Her complex relationship with Toby is clearly blossoming into a friendship that's surprising both of them deeply, but a friendship that has to play by an odd set of rules (like so much else in Toby's life). I loved her here. Seeing someone that difficult and abrupt show unexpected emotional depths is one of the story elements I enjoy the most. The problem, though, is that quite a lot of An Artificial Night is very, very dark. And by that, I don't mean that there is a lot of tension, or that Toby is struggling against monsterous things, although both of those statements are also true. Rather, Toby is not only out of her league but ends up in some viciously nasty situations, and they persist for rather more of this book than I would have preferred. Putting one's protagonist through hell has a long tradition in urban fantasy, but I found the level of sheer helplessness that Toby struggled through to be a bit more than I wanted to read about. It was too reminiscent of events from Toby's background (as discussed in Rosemary and Rue) that I still don't like to think about. There is, of course, an eventual payoff, as dark as the interim events seem. One knows that, given the genre and given that this is an ongoing series, and even just given the nature of stories. But as much as I like the growing cast of this series, and adored the scenes with the Luidaeg, I found parts of this journey to be deeply disturbing, and not in an enjoyable dramatic tension sort of way. I don't mind protagonists being seriously hurt (although it's not my favorite plot), or even some amount of despair, but this went beyond that. I think McGuire was going for a story about sacrifice and self-definition, about trying even when one can't possibly win, and I can appreciate the narrative arc. But parts of the book were simply awful to read for me. I'm therefore not quite sure what to say in terms of a recommendation. I'm still reading the series, and will definitely read the next book. I think my reaction is somewhat idiosyncratic; the events here will probably not bother others as much as they bothered me. And I can't disagree with their dramatic effectiveness, or with the way that McGuire builds conflict out of the rules of fairy and is building Toby into a stronger and more capable person than she ever thought she could be. But there were still spots where I wished I were reading a different book. Hopefully the next book in the series will be that book I wanted to read. Followed by Late Eclipses. Rating: 6 out of 10

31 August 2013

Christian Perrier: DebConf 13: hacking outcome

I *did* some hacking at DebConf, as usual. For once, I had no TODO list: after all, I never complete these, so why make one? Still, I participated to discussions around samba packaging, mostly animated by Ivo De Decker and I think we're making good progress towards samba 4.x packages. The road is long, quite complicated, but we now have a stronger team, with a very active Ivo, Jeroen Dekkers who officially joined, Steve Langasek who still cherishes one of his pet packages (and even branched in our git with the Ubuntu packages). Great work and thanks to Ivo for pushing this forward. I also worked quite actively on the migration of fonts packages to git (I now reached the point where I'm more comfortable with git than SVN, yes, everything can happen). These packages were modernized at the same time and checked for new upstream versions (I have to say that few of these fonts had new upstream versions, indeed). We unfortunately found no time to have a good font BoF in Vaumarcus, indeed...but I'm not sure we would have many things to say. The work is done and done well, in this team. Some progress was made, also, for restoring a working "monolithic" build of D-I. This build gathers together all udebs from unstable, which means it offers a D-I image that uses ALL udebs from nustable.....which is not the default of other images. This would be very helpful for translators who want to check their work as soon as possible. In the future, with a Jenkins task that would build each package at each commit and then build a monolithic image, we could have a way to provide a tes snapshot of D-I git repos.....which could help catching more bugs (or more of my stupid mistakes). So, in general, I consider this a quite successful DebConf when it comes at "real" production.

Russell Coker: Links August 2013

Mark Cuban wrote an interesting article titled What Business is Wall Street In about the failure of Wall Street to fulfill it s stated purpose of providing capital for businesses [1]. His proposed solution is extreme, but so is the problem. PopSci has an interesting article by Peter Nonacs about allowing students to collaborate (which is often known as cheating ) in an exam on a Game Theory exam [2]. So the Game Theory exam became a practical exercise. It s more interesting than I can describe in a paragraph. Salon has an interesting article suggesting that mealtime routines and rituals improve flavor [3]. I wonder whether that applies to other things. Does the ritual of compiling software make one enjoy it more? Would the food at a Linux conference taste better if we sang the Free Software Song first?

30 August 2013

Daniel Pocock: Big decision?

Most people have probably heard that elections are on in Australia. I just received my postal ballot paper the other day and in the spirit of transparency, I'm sharing it here on my blog. Spoilt for choice There are two ballots, one for the Lower House and the other for the Senate. The big one is for the Senate. Our cat is on top of it to help give some perspective on how big it is. Like the columns on a bloated spreadsheet, the parties are designated with symbols from A to AM and then there are some leftover candidates too. Animal instincts With so many candidates to choose from, even our kitty may have found an opportunity in the Animal Justice Party - although that little snack in her claws suggests that she is more interested in the law of the jungle than any political debate. She may want to be wary not to fall under the wheel of the party next to them, Motoring Enthusiasts. The Senate, a place for convicted hackers? The next party in the picture is Wikileaks and their number one candidate, of course, is the first person to successfully claim asylum from Australia, Julian Assange. It's no secret that he has a string of convictions for computer hacking in the 90s and that when US authorities catch up with him, their drone won't just be dropping him a letter demanding a $2,100 penalty fine through the window. So how do they compare? As I contemplate my decision, lets look at the numbers:
Party Seats won in 2010
House of Reps, Senate
Criminal charges against members in last 3 years
Labor Party 72, 31 173, mainly fraud
Liberal/National coalition (conservative) 73, 34 3, including shoplifting and assault
Wikileaks 0 None - Assange is not actually charged with any crime
All of a sudden, in a country that is often the butt of British jokes about convicts, Assange starts to look squeaky clean. He may take further comfort from knowing that out of all those crimes, hacking is the only one that appears to have the presidential seal of approval. Conclusion Whichever way I vote may or may not make so much difference. With so many candidates to choose from though, there is little sign of any quality leadership from any corner of Australian politics. Whether Mr Assange is elected or not, many people will have a chuckle when they realise he almost ended up alongside the Australian Sex Party out of 38 other possible parties randomly distributed across the ballot. Please feel free to comment: which party is the most ridiculous?

28 August 2013

Vincent Sanders: Men admire the man who can organize their wishes and thoughts in stone and wood and steel and brass.


I would probably not yet worthy of the admiration Emerson was alluding to but I do like to make things. As anyone who has read previous posts knows I have pretty much embraced the "do things, tell people" idea.

One small wrinkle is doing things needs somewhere to work. Since moving myself and the family to rented accommodation in Cambridge (swampy 3,500 year old English city, not the one in Massachusetts) I have been lacking space to do practical projects.

The main space
To fix this lack I have joined the cambridge makespace which, in addition to somewhere I can work gives me access to some tools I was previously unable to afford. The space gives practical training on the more complex machines (any tool can be dangerous if you do not use it correctly) which recently allowed my induction on the CNC router.

My instructor , Mark Mellors, who was good enough to give up some of his valuable making time to train me (and accidentally get his car stuck in a car park overnight by staying late) suggested that it was a good idea to have a design to try.

I decided to use this opportunity to actually create something useful (though now I re-read Marks message it did suggest a simple design...oopsy). I had been working at the electronics bench in previous weeks and been uncomfortable using the existing stools and chairs as they were either a bit high or unable to be adjusted high enough for the 80cm tall benches. I decided to design a 60cm tall stool for use at this workbench.

My initial idea was for a simple three leg stool, round top, three legs, how hard could it be? Initial research showed that showed that above 30cm the legs needed to be braced to each other. This is because the leg to seat joints simply cannot handle the stress caused by leverage which longer legs introduce.

I looked at the structure of several stools online and was initially drawn towards creating something like the IKEA Dalfred bar stool. It was discovered that the design would be easier to realise if it were made from sheet material which give a smaller challenge to a naive operator of a CNC router. Because of this simpler designs were researched and I eventually found a simple design I liked.

The design could not however be used directly as it was for imperial sized material which is not available in europe. I selected the QCAD open source CAD package and recaptured the design adjusting for the available 12mm plywood sheet material. This resulted in an imperial measurement design for metric materials.

Mark helped me transfer the DXF into the CAM software (vcarve pro 7, after I discovered the demo version of this software generates files which cannot be imported into the full edition!) and generate toolpaths for our machine. Once the toolpaths are saved to a USB stick (no modern conveniences like direct upload here) the job can be run on the machine.

Here I ran into reality, turns out that tolerances in imperial combined with lack of understanding how plywood reacts resulted in excessive play in the joints. This resulted in an unusable stool, which simply tried to rotate around its central axis and become flatpack. I had successfully turned 10 worth of plywood into some sawdust and a selection of useless shapes. On the other hand I did become competent with the router workflow so it was not a complete failure.

I was determined to make the design work so I decided to start from scratch with a similar design but entirely in metric. I performed some material research both online and practically (why yes i did spend an informative hour in several cambridge DIY shops with digital calipers, why do you ask?) it turns out that generally available 12mm thick plywood actually ranges between 11.9 and 12mm thick.

I did some test slot cuts of varying width and determined that the available stock can be "persuaded" to fit into a 11.8mm wide slot. This Interference fit joint is strong and removes the need for adhesive in most cases.

Second cut still attached to the bed. Leftovers of first attempt in the background
The sheet plywood material is readily available with a width of 1220mm and a height of 606mm (a full sheet is 2440mm long which is cut into four with a 4mm wide saw blade) so making the legs fit within a sheet and be close to the 60cm target should be possible.

I selected a suitable seat radius (175mm) and from that determined the minimal gap to the base ring with a 6mm end mill tool (20mm for two toolpaths and some separation) and hence the minimum suitable width of the base ring (50mm) giving a total radius of 245mm.

For the legs allowance was made for two joints of 30mm with 30mm separation between them the legs come out at 90mm width. If a 6mm space and a 6mm toolpath top and bottom of the sheet is accounted for a 582mm height (606 - 24) is available in which to fit the legs. The top of the leg which fits into the seat is 12mm tall leaving 570mm total leg height.

12mm plywood stool design
At this point I selected some arbitrary values for the leg positioning and angle (30mm from seat centre and 15 ) using a right angle triangle triganometry this produced a stool with a base of around 550mm or 100mm outside the radius of the seat. This seemed a pleasing shape and when the values for the holding ring were calculated it is at 104mm height which also seems to work out well.

This design was cut on the machine, lightly finished with some sandpaper and assembled. The legs slotted into the ring first and then the legs eased into the seat slots, the whole thing flipped and the seat hammered home onto the legs.

Anne Harrison volunteering to try the wobbly stool of doom
Success! It physically fit together and if you were brave enough you could sit on it. Unfortunately the plywood seemed to flex around the central axis of rotation in a rather alarming way, fine if you are under 70 kilos but not giving the impression of security most people want from their seating.

Ok, lets try with five legs instead of three (at least we can reuse the existing three legs)...nope still not good enough and another 10 gone.

18mm Plywood stool design
The others in the space suggested a few ideas to improve matters and the one I selected was to use 18mm plywood instead of 12mm, this should improve rigidity. There was a brief pause in proceedings to discover 18mm sheet is actually 17.7mm and needs a 17.5mm slot to make the interference fit work.

Completed 18mm stool
A swift redesign later altering the seat radius, gap, ring width and leg height to accommodate the new material and we have version 4 and it works without caveat. Tested up to 150Kg load without trouble, there is a small amount of flex still but nothing that feels worrying.

I finished the stool by rounding the seat top edge with a ball bearing rounding bit in a manual router and applying a couple of coats of gloss acrylic varnish. Finished stool is now doing service at the space.

In conclusion the final design allows someone with a CNC router to create a useful 580mm high fixed stool for 7.50 in timber plus cutter wear and varnish so maybe 8.50 total.

You can actually get five legs and a seat/ring out of a 1220x606 sheet and with intelligent arrangement a single 1220x2440 sheet will probably yield five or possibly six stools in total

I am making the design files of the proven 18mm version available (heck they are all there...but you have been warned, none of the other solutions produce a satisfactory result) under the MIT licence so anyone can reproduce. More pretty pictures are also available.

27 August 2013

Neil McGovern: Adblock adding ads to block ads

shutterstock_151488752Confused yet? The popular browser extension AdBlock for Chrome/Chromium and Safari has launched a new crowdsourced campaign - to buy up internet advertising space inviting people to download AdBlock, so they can block ads. Additionally, if they raise enough, they'd like to get additional space for a billboard at Times Square, a full page spread in the New York Times, and even a TV slot at the Superbowl (although the target for this is $4.2M, so I'm not entirely confident they'll make it.) However, this does raise an interesting question, which has been under debate for some time. How do you make money on the internet without adverts? Wikipedia has a few interesting suggestions. Firstly, affiliate programmes. These mean that any products or services that are linked or searched via the original site earn a little bit of commission for that site. Founded in the mid-90s, and most famously used by Amazon, it's a possible method for getting money. However, it should be deployed with careful thought. If a blogger or reviewer is recommending a product, is it simply because they are being paid to do so, or is it a genuine endorsement? The BBC ban this practice, and the Society of Professional Journalists also has strong guidelines on taking gifts. Interestingly, a few search engines now offer this to browsers for them sending their default search traffic to them. Secondly, you could engage in (what I consider to be) the shady practice of data monetisation. People visit a website, and they leave data about their visit. By visiting here, you've told me that you're probably interested in technology issues, what country and city you're from, and where you came from. Other programmes and sites will also be able to wrap up what other sites you've visited, and can build up quite a profile of your browsing habits. Add together a rather unscrupulous vendor, and they can also know your contact details. This metadata can then be sold on. Thirdly, you can essentially ask your readers for money. Donations, via a micropayment link (which I use) or larger donation requests. Another method is to have a membership system, where content is locked up behind a paywall. This does, however, risk making your content less valuable as people will just head elsewhere for news and opinion. To counter this, a couple of sites (LWN, Slashdot) simply delay the latest news for a little while. Alternatively, you can just ask like the fantastic C G P Grey has done. Interestingly, he also has a great video on why advertising can be a bad thing for content creators. Lastly, you can sell things. This could be your authorship services on another site as a guest post, writing a book, offering consultancy services, or simple merchandise. In summary though, which is more honest? An advert may well bend your view on what sort of content is being created, but is at least a lot more open than potentially hiding real motives behind affiliate links, and a lot more open than selling data. Asking for money is fairly straightforward, though could easily backfire if it's viewed as begging. Google's advertising income totalled $43.7 billion in 2012 - so it's quite clear that advertising is big business. Websites cost money to run, can the web survive in an ad-free environment? If so, it certainly won't be the same web as we know today.

26 August 2013

Vincent Fourmond: Announcing version 0.5 of ctioga2

I've just released version 0.5 of ctioga2, my plotting program. While there is still quite a lot of improvements I'd like to get in some day, the past few days have seen a lot of activity. Probably the most important change is the new space-separated command file format, much simpler than the old one (which is still supported, although marked clearly as deprecated now). Another important change for the users is the large improvement of error reporting: in most cases, ctioga2 pinpoints the exact location where things go wrong; it also features reporting on LaTeX errors. The last important feature is also the introduction of real contour plots, such as the one below, coming from the documentation:
Now, I feel like ctioga2 is coming close to gnuplot in terms of features and ease of use. Part of my plans for the website is to create a gnuplot mockup gallery, in which I would take as many examples from the gnuplot website as possible (ie all the ones ctioga2 can reproduce) and compare the code and look of both.Among features I'd like to implement for ctioga2, here are the ones that I think are the most important ones:But maybe you have other ideas you'd like to see before that ?

25 August 2013

Jonathan Carter: Still Alive

I ve been really quiet on the blogging front the last few months. There s been a lot happening in my life. In short, I moved back to South Africa and started working for another company. I have around 10 blog drafts that I will probably never finish, so I m just declaring post bankruptcy and starting from scratch (who wants read my old notes from Ubuntu VUDS from March anyway?) For what it s worth, I m still alive and doing very well, possibly better than ever. Over the next few months I want to focus my free software time on the Edubuntu project, my Debian ITPs (which I ve neglected so badly they ve been turned into RFPs) and Flashback. Once I ve caught up there s plenty of other projects where I d like to focus some energy as well (LTSP, Debian-Edu, just to name a few). Thanks for reading! More updates coming soon.

Vincent Bernat: Boilerplate for autotools-based C project

When starting a new HTML project, a common base is to use HTML5 Boilerplate which helps by setting up the essential bits. Such a template is quite useful for both beginners and experienced developers as it is kept up-to-date with best practices and it avoids forgetting some of them. Recently, I have started several little projects written in C for a customer. Each project was bootstrapped from the previous one. I thought it would be useful to start a template that I could reuse easily. Hence, bootstrap.c1, a template for simple projects written in C with the autotools, was born.

Usage A new project can be created from this template in three steps:
  1. Run Cookiecutter, a command-line tool to create projects from project templates, and answer the questions.
  2. Setup Git.
  3. Complete the todo list .

Cookiecutter Cookiecutter is a new tool to create projects from project templates. It uses Jinja2 as a template engine for file names and contents. It is language agnostic: you can use it for Python, HTML, Javascript or C! Cookiecutter is quite simple. You can read an introduction from Daniel Greenfeld. The Debian package is currently waiting in the NEW queue and should be available in a few weeks in Debian Sid. You can also install it with pip. Bootstrapping a new project is super easy:
$ cookiecutter https://github.com/vincentbernat/bootstrap.c.git
Cloning into 'bootstrap.c'...
remote: Counting objects: 90, done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 90 (delta 48), reused 64 (delta 22)
Unpacking objects: 100% (90/90), done.
Checking connectivity... done
full_name (default is "Vincent Bernat")? Alfred Thirsty
email (default is "bernat@luffy.cx")? alfred@thirsty.eu
repo_name (default is "bootstrap")? secretproject
project_name (default is "bootstrap")? secretproject
project_description (default is "boilerplate for small C programs with autotools")? Super secret project for humans
Cookiecutter asks a few questions to instantiate the template correctly. The result has been stored in the supersecret directory:
.
  autogen.sh
  configure.ac
  get-version
  m4
      ax_cflags_gcc_option.m4
      ax_ld_check_flag.m4
  Makefile.am
  README.md
  src
      log.c
      log.h
      Makefile.am
  secretproject.8
  secretproject.c
  secretproject.h
2 directories, 13 files

Remaining steps There are still some steps to be executed manually. You first need to initalize Git, as some features of this template rely on it:
$ git init
Initialized empty Git repository in /home/bernat/tmp/secretproject/.git/
$ git add .
$ git commit -m "Initial import"
[...]
Then, you need to extract the todo list built from the comments contained in source files:
$ git ls-tree -r --name-only HEAD   \
>   xargs grep -nH "T[O]DO:"   \
>   sed 's/\([^:]*:[^:]*\):\(.*\)T[O]DO:\(.*\)/\3 (\1)/'   \
>   sort -ns   \
>   awk '(last != $1)  print ""   last=$1 ; print '
2003 Add the dependencies of your project here. (configure.ac:52)
2003 The use of "Jansson" here is an example, you don't have (configure.ac:53)
2003 to keep it. (configure.ac:54)
2004 Each time you have used  PKG_CHECK_MODULES  macro (src/Makefile.am:12)
2004 in  configure.ac , you get two variables that (src/Makefile.am:13)
2004 you can substitute like above. (src/Makefile.am:14)
3000 It's time for you program to do something. Add anything (src/secretproject.c:76)
3000 you want here. */ (src/secretproject.c:77)
[...]
Only a few minutes are needed to complete those steps.

What do you get? Here are the main features:
  • Minimal configure.ac and Makefile.am.
  • Changelog based on Git logs and automatic version from Git tags2.
  • Manual page skeleton.
  • Logging infrastructure with variadic functions like log_warn(), log_info().
logging output of lldpd

About the use of the autotools The autotools are a suite of tools to provide a build system for a project, including:
  • autoconf to generate a configure script, and
  • automake to generate makefiles using a similar but higher-level language.
Understanding the autotools can be a quite difficult task. There are a lot of bad documentations on the web and the manual does not help by describing corner-cases that would be useful if you wanted your project to compile for HP-UX. So, why do I use it?
  1. I have invested a lot of time in the understanding of this build system. Once you grasp how it should be used, it works reasonably well and can cover most of your needs. Maybe CMake would be a better choice but I have yet to learn it. Moreover, the autotools are so widespread that you have to know how they work.
  2. There are a lot of macros available for autoconf. Many of them are included in the GNU Autoconf Archive and ready to use. The quality of such macros are usually quite good. If you need to correctly detect the appropriate way to compile a program with GNU Readline or something compatible, there is a macro for that.
If you want to learn more about the autotools, do not read the manual. Instead, have a look at Autotools Mythbuster. Start with a minimal configure.ac and do not add useless macros: a macro should be used only if it solves a real problem. Happy hacking!

  1. Retrospectively, I think boilerplate.c would have been a better name.
  2. For more information on those features, have a look at their presentation in a previous post about lldpd.

Francesca Ciceri: Some things I learnt at DebConf13

My self-tagged badge

Next.

Previous.