I was super sad to know that Paco de Lucia passed a few days ago. It was shocking to know that he was so young (only 66 year old, if I am not mistaken). To share some of his fine work with his frequent collaborators John McLaughlin and Al Di Meola to people that may not otherwise know him or his work, I offered on a Facebook post to upload a bootleg of a show of the Guitar Trio (John McLaughlin, Paco De Lucia and Al Di Meola) with the entire performance (possibly out of order, as that is a bootleg, after all) ripped from a broadcast to Dutch TV station. This performance includes the exceptional Paco de Lucia prominently and the show has many songs from the "Friday Night in San Francisco" record, including the magnificent (IMVHO) "Mediterranean Sundance". The credits of the video are, of course, of the musicians. I only ripped, deinterlaced (remember, analog TV it was interlaced), denoised, encoded, and uploaded the video to youtube.

Debian: watch your stats! Over the past few weeks, myself and Matthieu Caneill have worked quite a bit on Debsources. As we have now deployed most of the new features on http://sources.debian.net, it's time for another "What's new with Debsources?" blog post. Here is what's new:

• Debsources now knows about Debian suites, i.e. which package is in which "release" (stable, testing, unstable, ...). This knowledge is already useful for some of the other features below and will be used more in the future.
• since last summer Debsources has been running sloccount on all unpacked source packages, together with ctags and du, but the resulting information wasn't exposed on the Web. This is now fixed. Each package now has an infobox (example) which shows: disk usage, archive area, suites, and sloccount with per-language breakdown. The new infobox also subsumes the old puny list of package links. You can easily embed the infobox in other webapps if you need to (example). Check the URL scheme doc for more info.
• Debsources now gathers and plot accurate Debian sources statistics, both overall and per-suite, in both snapshot and historical trends flavors. (Yeah, I know, the charts are not particularly good looking ATM, but that's easy to change without impacting the rest. So if you're a matplotlib artist and willing to help, please step forward!)
• many changes have been going on also at the plumbing layer to make the service less resource hungry and more maintainable, in view of a migration to the official Debian infrastructure --- which I've in the meantime started discussing with DSA. Some highlights:
  • Debsources now has a rather comprehensive test suite, built using Nose. Most notably, we do test full update runs down to source unpacking (of a small subset of a Debian mirror), DB injection, and plugin execution --- which is quite neat.
  • the updater is now much faster (about 2x) and might require, in pathological cases, 10x less memory than before. Memory usage now caps at around 300MB, even when injecting ctags for large packages such as linux, chromium, and libreoffice.
  • the DB schema went through several refactoring cycles, and now uses a separate file table to index all known source file paths. In the past path information were duplicated across the checksums and ctags tables, not only wasting DB space, but also making the presence of file information conditional on the enablement of at least one of the two corresponding plugins. This is now fixed --- and migrating the full DB has been quite "fun". Unfortunately, we've also added quite a few large-ish indexes, resulting in no significant overall changes in DB size (currently at ~50GB), but at least in much faster queries The next step on this front will be the addition of path-based searches, using the excellent Postgres trigram indexes.

Want more? Sure, we'll be happy to! But it'll happen faster if you help. Speaking of which: we've got Debsources into the new contributors game (see announcement) and we're looking forward to mentor new contributors.

Following my blog post on the topic, I played a bit with various options. But let s explain my use case (which might be quite specific). I need to deal with three main sources of events:

• the Zimbra instance from my lab. It provides a CalDav interface.
• the ICS export from my University s teaching timetable.
• a calendar for personal stuff. I don t want to use my lab s Zimbra for that.

Additionally, I follow some ICS feeds for some colleagues and other events.
I tend to access my calendar mostly on my computer, and sometimes on my N900 phone. None of the web interfaces I looked at enabled me to (1) manage different calendars hosted on different CalDav servers; (2) subscribe to ICS feeds; (3) provide a CalDav interface to synchronize my phone. I ended up using a radicale instance for my personal calendar, which was extremely easy to set up. It s unfortunately a bit slow when there are many events (1600 since 2010 in my case), so I ended up importing only future events, and I will probably have to cleanup from time to time. I switched to using IceDove with the Lightning add-on to manage all my calendars and ICS feeds. It s unfortunately slower and less user-friendly than Google Calendar, but I ll live with it. On my N900, I used syncevolution to synchronize my various CalDav calendars. It works fine, but understanding how to configure it is rather tricky due to the number of concepts involved (templates, databases, servers, contexts, ). The synchronization is quite slow (several minutes for the 400-events Zimbra calendar), but works. I also wanted a way to export my calendars to colleagues (both in a free/busy version, and in a full information version). I quickly hacked something using ruby-agcaldav (which is not packaged in Debian, and required quite a few dependencies, but it was easy to generate packages for all of them using gem2deb the situation with other languages did not look better).
The resulting script is:
require 'agcaldav'
require 'date' cal = AgCalDAV::Client.new(:uri => 'LABCALDAVSERVER', :user => 'xx', :password => "xx")
ev = cal.find_events(:start => '2014-02-01', :end => '2200-01-01') cal = AgCalDAV::Client.new(:uri => 'RADICALESERVER', :user => 'xx', :password => "xx")
ev2 = cal.find_events(:start => '2014-02-01', :end => '2200-01-01') limit = (Time::now - 7*86400).to_datetime # create new empty calendars
ncpriv = Icalendar::Calendar.new
ncpub = Icalendar::Calendar.new (ev + ev2).each do e
next if e.end < limit # drop old events to keep the calendar small # build event for the free/busy calendar
pe = Icalendar::Event.new
pe.start = e.start
pe.end = e.end
pe.klass = "PRIVATE"
pe.transp = e.transp
ncpriv.add(pe) # build event for the calendar with event information
pube = Icalendar::Event.new
pube.start = e.start
pube.end = e.end
pube.transp = e.transp
if not e.klass == "PRIVATE"
pube.summary = e.summary
pube.location = e.location
end
ncpub.add(pube)
end # export free/busy calendar
fd = File::new('xx.ics', 'w')
fd.puts ncpriv.to_ical
fd.close # export calendar with event information
fd = File::new('yy-Zeeh9bie.ics', 'w')
fd.puts ncpub.to_ical
fd.close
So, mostly everything works. The only thing that doesn't is that I haven't found a way to subscribe to an ICS feed on my N900. Any ideas?

It is the first of these which I have recently been attempting and I think Phil Karlton might have a good point.
What are we talking about?Web browsers are pretty complex beasties but the basic concept is pretty easy to understand. They fetch a bunch of files that make up a web page and render those source files into something suitable for human consumption.

It is also intuitive that fetching all the files that make up a web page, every time you browse to a new page, might be wasteful, especially if most of the files had not changed from the previous page.

In order to address this browsers hang onto copies of these files in case your browsing needs them again, this is known as a source file cache. Caches are a widely used technique throughout many aspects of computer technology but the basic idea is that they trade one resource for another.

In this case we are trading local storage space (memory or disc) for access time. This type of trade may give large benefits due to the large differences in access times (sometimes known as latency) between local and network resources.

For example, if the source files that make up a web page are a Megabyte in size accessed with a 2013 era PC with a 10Mbit connection to the Internet:

• Accessing the data from main memory will take approximately 20 s (millionths of a second).
• The same data retrieved from a hard disc drive will take 4,000 s to arrive, around 200 times longer.
• If we were to retrieve the data from a fast web server and assuming the network connection is in perfect working order, the data will roll in 1,200,000 s later, or around 300 times more slowly than disc and a unfathomable 60,000 times slower than memory.

From this example it becomes startlingly obvious why a source cache is so desirable. I have greatly simplified the browsers operation here as they usually implement many layers of additional caching in other areas to make the browsing experience subjectively smoother.

And before the observant reader suggests that we could just make the network faster it ought to be mentioned that due to fundamental physical constraints, like the speed of light, it would be unlikely that the average practical network reply will ever drop much below 150,000 s [1] or some 40 times slower than disc. Still a worthwhile gain.
Knowing the cost of everything and the value of nothingHaving shown the benefits it might be nice to think there are no downsides to caching, this is not the case, as I mentioned we are trading storage for time and as in any trade one must expect there to be overheads.

In the case of a browser we face several obstacles. Firstly not every file that goes to make up a web page can be cached. The HTTP specification contains an entire section on caching which contains numerous rules and operations to ensure that only objects that should be cached are and determines for how long they can be kept before going "stale".

A great deal of this complexity comes from just how desirable caching is to network providers. Many ISPs will have web proxy servers to cache requests for all users to reduce their bandwidth usage, or increasingly, to implement content filtering according to the local government censorship rules. The browsers cache must know how to interact with these proxies without getting erroneous results.

Then comes the problem alluded to in the section title. A browsers cache cannot grow indefinitely. Eventually a system would run out of memory and disc if a browser kept every file. To deal with this the cache is limited sometimes by the number of files it holds, but more commonly by size both in memory and on disc.

The caching structure used by a web browser is hierarchical in that it will first evict (move) files from memory to a backing store (disc) and when the disc cache size limit is exceeded files are evicted to the next tier. Which in this case, will involve deleting the local copy (invalidating) and requiring performing a fetch from the network if the file is needed again.

When the cache size is exceeded the decision is made about what to remove from the cache using a cache strategy or algorithm. The aim of this decision is to discard the data that will not be required again for the longest time.

If we had a blue box with a madman inside, who we could persuade to bring our browsing history back from the future, implementing b l dy's algorithm might be possible. Failing a perfect solution we are reduced to making a judgement based on the information available to us.

This involves computing a cost for replacing each entry in the cache and evicting those with the lowest values. The selected algorithm has a large impact on the effectiveness of the cache and often needs fine tuning for the task at hand.
Never has so little been measured so muchHaving established that caching is useful and that we need to make decisions on what to keep it follows that those decisions must be based on information. This information consists of two main groups:

• A series of metrics about the cache state such as the number of objects and their cumulative size currently being stored in memory.
• Information about each object, known as metadata, this includes things like the objects individual size, how long until it becomes stale and when it was last used.

Maintaining all of this information is a significant part of the overheads involved with caching and a compromise must be struck between having the necessary information to make good caching choices and the cost maintaining that data.
Cache metricsMost cache implementations maintain at least some basic metrics, at the very least how much storage is being used so a decision on if evicting entries from the cache to reclaim space is necessary.

It is also common to keep a record of how well the cache is performing. The cache efficiency (the hit rate) is usually measured as the ratio of successful accesses provided from the cache (a cache hit) to the total number of requests to the cache.

A perfect cache would have all hits and no misses (a hit rate of 1) while using as little storage as possible (but as already noted that requires a special madman). The hit rate can be used to automatically change the behaviour of the eviction algorithm, perhaps using more storage if the hit rate drops too low or changing the eviction frequency.
MetadataEach object must carry with it the data necessary to regain its state when retrieved from the cache. This information is essential for the correct operation of the cache but in of itself provides no direct value.

In the case of browser source data this includes all the headers sent in the original network fetch. These headers are themselves just metadata sent along with the object by the web server which must be preserved.

Two particularly useful value for making a good eviction decision are how long it has been since the object was last used and how many times it has been used. This is because an object that was cached a long time ago and then never used again is much less valuable than one that has recently been repeatedly used.
An ImplementationIt is all very well talking about the general solution but a full understanding can only be gained by examining a real implementation. The NetSurf cache was chosen as it is a suitably small amount of code but still demonstrates all the major design features.
InterfaceThe cache deals with all source data and provides the browser with a single unified object request interface regardless of if the source data can be cached or not. This is useful to hide the implementation details allowing for changes without affecting the rest of the browsers code.

The implementation effectively consists of two lists of objects, those that can be cached and those that cannot. Each object on these lists is reference counted against one or more users and while there is at least one user the source data is maintained in memory.

Objects determined to be non cached are always directly fetched from their source and are immediately released once they have no users remaining.

For cacheable objects an attempt is made to fulfil the request in descending order:

1. from objects already in memory.
2. from the backing store.
3. from a network fetch.

Irrespective of the source of the data the remaining operations on the object (determining freshness etc.) are exactly as if the request had been fulfilled from the network. This approach means the use of the cache is transparent to the object users.
WriteoutObjects are placed in the backing store asynchronously to any other operations. When an object has been fetched from the network a background task is scheduled for when the browser is otherwise idle.

This writeout task constructs a list of all cacheable objects not yet in the backing store, associates a cost value with each object and then proceeds to write those objects to the backing store in highest to lowest value order.
CleaningA periodically scheduled task deals with ensuring the cache is cleaned. This consists of destroying stale objects and arranging fresh objects are not using more memory than the configuration permits.

Memory usage is reduced within the cleaning task by discarding the source data for objects already held in the backing store (where the data can be retrieved relatively cheaply)

Additional memory may be recovered by simply discarding unused objects held only in memory. These objects are usually those which have only recently become unused otherwise the writeout task would have committed them to the backing store.

The most recently used objects are statistically likely to be used again soon, because of this there is a high risk of a cache miss associated with discarding these objects. In order to mitigate this undesirable effect the cleaning heuristic favours using more memory than configured for short periods rather than discarding from memory.
Backing storeThe backing store is a simple key-value database. Each source data object and its metadata (its headers etc.) is associated with a unique key (the URL used to fetch it). The backing store is simply required to store and retrieve the object data associated with the given URL.

The backing store implementation in NetSurf is pluggable, this flexibility is required for dealing with the greatly varying capabilities and limits of the various systems the browser is required to execute on.

A novel aspect of this backing store interface is that if an implementation returns the wrong object it is not considered an error and is simply treated as a cache miss. This is possible because the metadata contains the URL of the stored object enabling object verification.

This behaviour is permitted because techniques which significantly improve key-value store performance (principally key hashing) become available if they are not required to always give the correct answer.

The backing store is also required to manage its size within the configured limits and deal with any filesystem behaviour details.

The reference backing store is trivial in that it performs a hash operation on the input URL, encodes the result with base64url encoding and uses that as the object filename. The length of the hash can be configured allowing use of the reference implementation in any situation where using the filesystem as a database is acceptable.

The reference backing store default hash is SHA-1 yielding almost unique [2] 160 bit key values stored in much the same way as the git DVCS. Note we are not using this hash for its cryptographic properties and at worst a collision will result in the cache being a tiny amount less efficient.
ConclusionHopefully this has shown what a browser source file cache is, why it is useful and a basic understanding of how they are implemented.

I will admit I have glossed over some of the more challenging aspects especially in relation to actually implementing the cache strategy but I hope that the reader will forgive those omissions of detail in a quest for a little more clarity of the general principles.

I would like to thank John-Mark Bell for implementing the NetSurf cache and to Melodie Parry for proof reading and providing feedback.

[1] Speed of light is 3x10^8m/s earth circumference is 4x10^7m, divide circumference by speed gives 133,000 s which gives longest round trip time from anywhere on surface of earth to point furthest from. So assuming a whole megabyte can be transferred in one round trip and allowing for some overhead the lower bound estimate of 150,000 s looks reasonable.

[2] Mathematically speaking this hash would allow us to say that with the current estimated 1x10^11 URLs on the internet the likelihood of a collision is still vanishingly small (at least 1x10^-15)

I m trying to self-host my calendar setup, and I must admit that I m lost between all the different solutions. My requirements are:

• (A) manage my own personal calendar using a reasonably modern web interface (probably on my own CalDAV server)
• (B) display a dozen public ICS calendars in the web interface. Organizing those public calendars in a tree would be great.
• (C) display several caldav calendars (from two different instances of zimbra), preferably in RW mode
• (D) provide ICS links with a secret token that allow me to provide a full view of my calendar to some people (except for private events, where I should just be marked busy )
• (E) provide ICS links with a secret token that allow me to provide a busy/available view of my calendar to some people
• (F) export something usable on my n900. MFE would be great since that is already known to work.
• (G) easy to setup (Debian packages available in wheezy or wheezy-backports, especially for the server part)
• (H) preferably lightweight. I don t need a full groupware application. I can ignore the other bits if really needed.

It does not seem to be possible to find a single framework doing all of the above. AFAIK:

• Owncloud does A, D, G
• Baikal does A. not sure about the rest.
• For (B), an alternative is to script the download of the ics and then upload it to the CalDAV using cadaver. But that sounds quite low-level for such a trivial use case.
• I ve looked at using IceOwl (and Thunderbird+Lightning) with a CalDAV server such as Radicale. That would solve A (using iceowl instead), B, C. But which CalDAV servers support D, E, F ? Radicale does not do any of those, apparently.

What did I miss?

There are times when I consider launching my own company again, most often when it is late at night and the inpetitude of so many other companies gets me too worked up. Then I sit back and think about details and write it off. I've worked for myself in the past a couple of times, and each time it was both more fun and more difficult than expected. Getting a couple of clients is usually easy, getting a ten more is common, but getting "many" is hard and getting "lots" is something I've never done - lots of users for free sites though, along with the associated support burdon! So the though dies away once I sit down and work out the net profit I'd need to live. My expenses are low, so let us pretend I can easily live on 1000 a month. So the "company" has to make more than that, to cover costs, but perhaps not much. Pretend you were offering DNS hosting you'd probably be able to implement that easily on, say 10, virtual machines, net of 150 a month. Imagine clients pay 5 for an unlimited number of domains that means you need to have 1000+150/5 = 230 clients. Not impossible, but also not easy. Pretend instead you're offering backup space, and the numbers get bigger because disk is expensive. Again getting some users would be easy, but getting lots would be hard because your competition is dropbox, skydrive, etc, etc. Once you start thinking of "ideas" they come easily, but the hard part is being realistic about what people would pay for. As always the idea is the easy part, the execution is the hardest part. Realistically if I were to be desperate to work for myself at short notic I'd do the obvious thing - I'd buy a pair of ladders, a bucket, and clean windows. Low overheads, reasonable demand, and I'd be both "fit" and "outdoors". When it comes to paying for online services off the top of my head I personally pay for maybe two things, both of them niche (although profitable for their providers I'm sure), and I know many people who live on the internet but pay for nothing.

For example I'm a VIP member of an online modeling community, which in theory allows me a higher chance of persuading interesting people to pose for me. In practice the turnover on those sites is immense. Lots of cute boys and girls hear constantly "You're so pretty, you should be a model", which is true in perhaps 1% of cases, and the net result is you have a few hard working people who do good things day in day out, and many flighty teenagers who'll pose for two-three people, and then never do it again because they realise it is neither glamourous nor easy money.

Two things I've semi-serously considered recently where hosted "status pages", and hosted "domain parking", but both have many competitors and both I can see a) some people would pay for but b) not very many. I suspect there is no universal "I'd pay for this" online service hwich is both competition free and genuinely trivial to setup, but I'd be curious to see what people are missing, and even more curious to see what people do pay for.

Today was yet another busy day. Zoe seems to have taken to waking up at around 5:30am instead of her normal 6:00am (or a bit later when I'm really lucky). Not sure what that's all about. I did get her to go back to bed until about 6:55am, and then she was happy to just come and snuggle in bed with me for a bit instead of play games on her Nexus 7. After that we had a leisurely start to day, making porridge with strawberries in the Thermomix, and then biked to Kindergarten. Somehow it still managed to be 8:45am by the time we got there. I was expecting it would be closer to 8:30am. Drop off looked like it was going to be clingy, but then she just waved me off, so that was nice. I had an appointment at the podiatrist at 11:40am, and that was back near Kindergarten, but further away from home, so I had this grand plan of cycling over there, and then twiddling my thumbs for a bit before Kindergarten pickup. That left me with only a couple of hours at home to work on my taxes before I had to leave again. I managed to make those couple of hours productive though, so that was good. I biked to the podiatrist, had my appointment, and then it was 1pm and I still needed lunch, so I decided to grab some at a cafe near the Kindergarten. I chilled out there for a while catching up on social media on my phone, and then headed over to the Kindergarten, but was still quite early, so I just hung out on one of the couches they have out by the front door, out of sight of the kids. The Kindergarten has an A-frame whiteboard out the front that they use to broadcast messages to parents at drop off and pick up time. One of the teachers came out to write something on it while I was sitting out there, and another parent reacted rather negatively to it as it was being written, so I had to go see it for myself. Apparently a child had a suspected gastro bug (he'd thrown up a couple of times). It was interesting observing the reactions of a few mothers as they trickled in for pick up. It's been a while since a gastro bug has gone through the household, so I guess the news didn't evoke a particularly powerful reaction in me. I'm going with the "suspected" and that hopefully Zoe didn't come into contact with patient zero. I guess time will tell. I certainly hope she doesn't come down with anything between now and Sunday, with all the travel we're doing. I had an accident report to sign today. Apparently a large wooden block hit Zoe in the head while they were packing up a tower they'd made. She had a bit of a lump on her head, but otherwise seemed fine. Zoe had a nap today, of unknown length, but woke up naturally as I entered the room. She took a fair while to get going though, and was grumpy and uncooperative getting on the bike. I ended up having to do it the "hard way" and we had a fabulous ride back with her doing a tired cry all the way. All the way until we passed a playground, when she snapped out of it and declared she wanted to have a play. I'd booked myself in for a haircut at 3pm, and a make up swim class at 4:45pm (to cover our absence this Saturday due to our Melbourne trip) so playing in the playground seemed like the perfect way to fill the gap. It was even on the way to swim class. So after I had my haircut, we popped home, grabbed her swim gear, and biked back to the playground for a bit. Then we went to swim class. This was her third one at her new level, and her progress continues to go very well. It's really exciting watching her. Zoe took her time leaving the pool, so we didn't get home until after 5:30pm and I hadn't started dinner yet. I whipped it up while Zoe watched some TV, and I managed to recoup all of the lost time after dinner and still get Zoe to bed at a reasonable hour. Bedtime itself went nice and smoothly. I used the canned goodnight video I asked Sarah to make for the first time tonight, and it was well received and I think defused any potential issues. I'm hoping she'll sleep well tonight, as her mosquito bites don't seem to be actively giving her trouble any more.

TL;DR: Do not enable net.ipv4.tcp_tw_recycle. The Linux kernel documentation is not very helpful about what net.ipv4.tcp_tw_recycle does:

Enable fast recycling TIME-WAIT sockets. Default value is 0. It should not be changed without advice/request of technical experts.

Its sibling, net.ipv4.tcp_tw_reuse is a little bit more documented but the language is about the same:

Allow to reuse TIME-WAIT sockets for new connections when it is safe from protocol viewpoint. Default value is 0. It should not be changed without advice/request of technical experts.

The mere result of this lack of documentation is that we find numerous tuning guides advising to set both these settings to 1 to reduce the number of entries in the TIME-WAIT state. However, as stated by tcp(7) manual page, the net.ipv4.tcp_tw_recycle option is quite problematic for public-facing servers as it won t handle connections from two different computers behind the same NAT device, which is a problem hard to detect and waiting to bite you:

Enable fast recycling of TIME-WAIT sockets. Enabling this option is not recommended since this causes problems when working with NAT (Network Address Translation).

I will provide here a more detailed explanation in the hope to teach people who are wrong on the Internet. As a sidenote, despite the use of ipv4 in its name, the net.ipv4.tcp_tw_recycle control also applies to IPv6. Also, keep in mind we are looking at the TCP stack of Linux. This is completely unrelated to Netfilter connection tracking which may be tweaked in other ways¹.

About TIME-WAIT state Let s rewind a bit and have a close look at this TIME-WAIT state. What is it? See the TCP state diagram below²: Only the end closing the connection first will reach the TIME-WAIT state. The other end will follow a path which usually permits to quickly get rid of the connection. You can have a look at the current state of connections with ss -tan:

$ ss -tan   head -5
LISTEN     0  511             *:80              *:*     
SYN-RECV   0  0     192.0.2.145:80    203.0.113.5:35449
SYN-RECV   0  0     192.0.2.145:80   203.0.113.27:53599
ESTAB      0  0     192.0.2.145:80   203.0.113.27:33605
TIME-WAIT  0  0     192.0.2.145:80   203.0.113.47:50685

Purpose There are two purposes for the TIME-WAIT state:

• The most known one is to prevent delayed segments from one connection being accepted by a later connection relying on the same quadruplet (source address, source port, destination address, destination port). The sequence number also needs to be in a certain range to be accepted. This narrows a bit the problem but it still exists, especially on fast connections with large receive windows. RFC 1337 explains in details what happens when the TIME-WAIT state is deficient³. Here is an example of what could be avoided if the TIME-WAIT state wasn t shortened:

• The other purpose is to ensure the remote end has closed the connection. When the last ACK is lost, the remote end stays in the LAST-ACK state⁴. Without the TIME-WAIT state, a connection could be reopened while the remote end still thinks the previous connection is valid. When it receives a SYN segment (and the sequence number matches), it will answer with a RST as it is not expecting such a segment. The new connection will be aborted with an error:

RFC 793 requires the TIME-WAIT state to last twice the time of the MSL. On Linux, this duration is not tunable and is defined in include/net/tcp.h as one minute:

#define TCP_TIMEWAIT_LEN (60*HZ) /* how long to wait to destroy TIME-WAIT
                                  * state, about 60 seconds     */

There have been propositions to turn this into a tunable value but it has been refused on the ground the TIME-WAIT state is a good thing.

Problems Now, let s see why this state can be annoying on a server handling a lot of connections. There are three aspects of the problem:

• the slot taken in the connection table preventing new connections of the same kind,
• the memory occupied by the socket structure in the kernel, and
• the additional CPU usage.

The result of ss -tan state time-wait wc -l is not a problem per se!

Connection table slot A connection in the TIME-WAIT state is kept for one minute in the connection table. This means, another connection with the same quadruplet (source address, source port, destination address, destination port) cannot exist. For a web server, the destination address and the destination port are likely to be constant. If your web server is behind a L7 load-balancer, the source address will also be constant. On Linux, the client port is by default allocated in a port range of about 30,000 ports (this can be changed by tuning net.ipv4.ip_local_port_range). This means that only 30,000 connections can be established between the web server and the load-balancer every minute, so about 500 connections per second. If the TIME-WAIT sockets are on the client side, such a situation is easy to detect. The call to connect() will return EADDRNOTAVAIL and the application will log some error message about that. On the server side, this is more complex as there is no log and no counter to rely on. In doubt, you should just try to come with something sensible to list the number of used quadruplets:

$ ss -tan 'sport = :80'   awk ' print $(NF)" "$(NF-1) '   \
>     sed 's/:[^ ]*//g'   sort   uniq -c
    696 10.24.2.30 10.33.1.64
   1881 10.24.2.30 10.33.1.65
   5314 10.24.2.30 10.33.1.66
   5293 10.24.2.30 10.33.1.67
   3387 10.24.2.30 10.33.1.68
   2663 10.24.2.30 10.33.1.69
   1129 10.24.2.30 10.33.1.70
  10536 10.24.2.30 10.33.1.73

The solution is more quadruplets⁵. This can be done in several ways (in the order of difficulty to setup):

• use more client ports by setting net.ipv4.ip_local_port_range to a wider range,
• use more server ports by asking the web server to listen to several additional ports (81, 82, 83, ),
• use more client IP by configuring additional IP on the load balancer and use them in a round-robin fashion,
• use more server IP by configuring additional IP on the web server⁶.

Of course, a last solution is to tweak net.ipv4.tcp_tw_reuse and net.ipv4.tcp_tw_recycle. Don t do that yet, we will cover those settings later.

Memory With many connections to handle, leaving a socket open for one additional minute may cost your server some memory. For example, if you want to handle about 10,000 new connections per second, you will have about 600,000 sockets in the TIME-WAIT state. How much memory does it represent? Not that much! First, from the application point of view, a TIME-WAIT socket does not consume any memory: the socket has been closed. In the kernel, a TIME-WAIT socket is present in three structures (for three different purposes):

1. A hash table of connections, named the TCP established hash table (despite containing connections in other states) is used to locate an existing connection, for example when receiving a new segment. Each bucket of this hash table contains both a list of connections in the TIME-WAIT state and a list of regular active connections. The size of the hash table depends on the system memory and is printed at boot:

   $ dmesg   grep "TCP established hash table"
   [    0.169348] TCP established hash table entries: 65536 (order: 8, 1048576 bytes)
   

   It is possible to override it by specifying the number of entries on the kernel command line with the thash_entries parameter. Each element of the list of connections in the TIME-WAIT state is a struct tcp_timewait_sock, while the type for other states is struct tcp_sock⁷:

   struct tcp_timewait_sock  
       struct inet_timewait_sock tw_sk;
       u32    tw_rcv_nxt;
       u32    tw_snd_nxt;
       u32    tw_rcv_wnd;
       u32    tw_ts_offset;
       u32    tw_ts_recent;
       long   tw_ts_recent_stamp;
    ;
   struct inet_timewait_sock  
       struct sock_common  __tw_common;
       int                     tw_timeout;
       volatile unsigned char  tw_substate;
       unsigned char           tw_rcv_wscale;
       __be16 tw_sport;
       unsigned int tw_ipv6only     : 1,
                    tw_transparent  : 1,
                    tw_pad          : 6,
                    tw_tos          : 8,
                    tw_ipv6_offset  : 16;
       unsigned long            tw_ttd;
       struct inet_bind_bucket *tw_tb;
       struct hlist_node        tw_death_node;
    ;
   

2. A set of lists of connections, called the death row , is used to expire the connections in the TIME-WAIT state. They are ordered by how much time left before expiration. It uses the same memory space as for the entries in the hash table of connections. This is the struct hlist_node tw_death_node member of struct inet_timewait_sock.
3. A hash table of bound ports, holding the locally bound ports and the associated parameters, is used to determine if it is safe to listen to a given port or to find a free port in the case of dynamic bind. The size of this hash table is the same as the size of the hash table of connections:

   $ dmesg   grep "TCP bind hash table"
   [    0.169962] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
   

   Each element is a struct inet_bind_socket. There is one element for each locally bound port. A TIME-WAIT connection to a web server is locally bound to the port 80 and shares the same entry as its sibling TIME-WAIT connections. On the other hand, a connection to a remote service is locally bound to some random port and does not share its entry.

So, we are only concerned by the space occupied by struct tcp_timewait_sock and struct inet_bind_socket. There is one struct tcp_timewait_sock for each connection in the TIME-WAIT state, inbound or outbound. There is one dedicated struct inet_bind_socket for each outbound connection and none for an inbound connection. A struct tcp_timewait_sock is only 168 bytes while a struct inet_bind_socket is 48 bytes:

$ sudo apt-get install linux-image-$(uname -r)-dbg
[...]
$ gdb /usr/lib/debug/boot/vmlinux-$(uname -r)
(gdb) print sizeof(struct tcp_timewait_sock)
 $1 = 168
(gdb) print sizeof(struct tcp_sock)
 $2 = 1776
(gdb) print sizeof(struct inet_bind_bucket)
 $3 = 48

So, if you have about 40,000 inbound connections in the TIME-WAIT state, it should eat less than 10MB of memory. If you have about 40,000 outbound connections in the TIME-WAIT state, you need to account for 2.5MB of additional memory. Let s check that by looking at the output of slabtop. Here is the result on a server with about 50,000 connections in the TIME-WAIT state, 45,000 of which are outbound connections:

$ sudo slabtop -o   grep -E '(^  OBJS tw_sock_TCP tcp_bind_bucket)'
  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME                   
 50955  49725  97%    0.25K   3397       15     13588K tw_sock_TCP            
 44840  36556  81%    0.06K    760       59      3040K tcp_bind_bucket

There is nothing to change here: the memory used by TIME-WAIT connections is really small. If your server need to handle thousands of new connections per second, you need far more memory to be able to efficiently push data to clients. The overhead of TIME-WAIT connections is negligible.

CPU On the CPU side, searching for a free local port can be a bit expensive. The work is done by the inet_csk_get_port() function which uses a lock and iterate on locally bound ports until a free port is found. A large number of entries in this hash table is usually not a problem if you have a lot of outbound connections in the TIME-WAIT state (like ephemeral connections to a memcached server): the connections usually share the same profile, the function will quickly find a free port as it iterates on them sequentially.

Other solutions If you still think you have a problem with TIME-WAIT connections after reading the previous section, there are three additional solutions to solve them:

• disable socket lingering,
• net.ipv4.tcp_tw_reuse, and
• net.ipv4.tcp_tw_recycle.

Socket lingering When close() is called, any remaining data in the kernel buffers will be sent in the background and the socket will eventually transition to the TIME-WAIT state. The application can continue to work immediatly and assume that all data will eventually be safely delivered. However, an application can choose to disable this behaviour, known as socket lingering. There are two flavors:

1. In the first one, any remaining data will be discarded and instead of closing the connection with the normal four-packet connection termination sequence, the connection will be closed with a RST (and therefore, the peer will detect an error) and will be immediatly destroyed. No TIME-WAIT state in this case.
2. With the second flavor, if there is any data still remaining in the socket send buffer, the process will sleep when calling close() until either all the data is sent and acknowledged by the peer or the configured linger timer expires. It is possible for a process to not sleep by setting the socket as non-blocking. In this case, the same process happens in the background. It permits the remaining data to be sent during a configured timeout but if the data is succesfully sent, the normal close sequence is run and you get a TIME-WAIT state. And on the other case, you ll get the connection close with a RST and the remaining data is discarded.

In both cases, disabling socket lingering is not a one-size-fits-all solution. It may be used by some applications like HAProxy or Nginx when it is safe to use from the upper protocol point of view. There are good reasons to not disable it unconditionnaly.

net.ipv4.tcp_tw_reuse The TIME-WAIT state prevents delayed segments to be accepted in an unrelated connection. However, on certain conditions, it is possible to assume a new connection s segment cannot be misinterpreted with an old connection s segment. RFC 1323 presents a set of TCP extensions to improve performance over high-bandwidth paths. Among other things, it defines a new TCP option carrying two four-byte timestamp fields. The first one is the current value of the timestamp clock of the TCP sending the option while the second one is the most recent timestamp received from the remote host. By enabling net.ipv4.tcp_tw_reuse, Linux will reuse an existing connection in the TIME-WAIT state for a new outgoing connection if the new timestamp is strictly bigger than the most recent timestamp recorded for the previous connection: an outgoing connection in the TIME-WAIT state can be reused after just one second. How is it safe? The first purpose of the TIME-WAIT state was to avoid duplicate segments to be accepted in an unrelated connection. Thanks to the use of timestamps, such a duplicate segments will come with an outdated timestamp and therefore be discarded. The second purpose was to ensure the remote end is not in the LAST-ACK state because of the lost of the last ACK. The remote end will retransmit the FIN segment until:

1. it gives up (and tear down the connection), or
2. it receives the ACK it is waiting (and tear down the connection), or
3. it receives a RST (and tear down the connection).

If the FIN segments are received in a timely manner, the local end socket will still be in the TIME-WAIT state and the expected ACK segments will be sent. Once a new connection replaces the TIME-WAIT entry, the SYN segment of the new connection is ignored (thanks to the timestamps) and won t be answered by a RST but only by a retransmission of the FIN segment. The FIN segment will then be answered with a RST (because the local connection is in the SYN-SENT state) which will allow the transition out of the LAST-ACK state. The initial SYN segment will eventually be resent (after one second) because there was no answer and the connection will be established without apparent error, except a slight delay: It should be noted that when a connection is reused, the TWRecycled counter is increased (despite its name).

net.ipv4.tcp_tw_recycle This mechanism also relies on the timestamp option but affects both incoming and outgoing connections which is handy when the server usually closes the connection first⁸. The TIME-WAIT state is scheduled to expire sooner: it will be removed after the retransmission timeout (RTO) interval which is computed from the RTT and its variance. You can spot the appropriate values for a living connection with the ss command:

$ ss --info  sport = :2112 dport = :4057
State      Recv-Q Send-Q    Local Address:Port        Peer Address:Port   
ESTAB      0      1831936   10.47.0.113:2112          10.65.1.42:4057    
         cubic wscale:7,7 rto:564 rtt:352.5/4 ato:40 cwnd:386 ssthresh:200 send 4.5Mbps rcv_space:5792

To keep the same guarantees the TIME-WAIT state was providing, while reducing the expiration timer, when a connection enters the TIME-WAIT state, the latest timestamp is remembered in a dedicated structure containing various metrics for previous known destinations. Then, Linux will drop any segment from the remote host whose timestamp is not strictly bigger than the latest recorded timestamp, unless the TIME-WAIT state would have expired:

if (tmp_opt.saw_tstamp &&
    tcp_death_row.sysctl_tw_recycle &&
    (dst = inet_csk_route_req(sk, &fl4, req, want_cookie)) != NULL &&
    fl4.daddr == saddr &&
    (peer = rt_get_peer((struct rtable *)dst, fl4.daddr)) != NULL)  
        inet_peer_refcheck(peer);
        if ((u32)get_seconds() - peer->tcp_ts_stamp < TCP_PAWS_MSL &&
            (s32)(peer->tcp_ts - req->ts_recent) >
                                        TCP_PAWS_WINDOW)  
                NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_PAWSPASSIVEREJECTED);
                goto drop_and_release;
         
 

When the remote host is in fact a NAT device, the condition on timestamps will forbid allof the hosts except one behind the NAT device to connect during one minute because they do not share the same timestamp clock. In doubt, this is far better to disable this option since it leads to difficult to detect and difficult to diagnose problems. The LAST-ACK state is handled in the exact same way as for net.ipv4.tcp_tw_recycle.

Summary The universal solution is to increase the number of possible quadruplets by using, for example, more server ports. This will allow you to not exhaust the possible connections with TIME-WAIT entries. On the server side, do not enable net.ipv4.tcp_tw_recycle unless you are pretty sure you will never have NAT devices in the mix. Enabling net.ipv4.tcp_tw_reuse is useless for incoming connections. On the client side, enabling net.ipv4.tcp_tw_reuse is another almost-safe solution. Enabling net.ipv4.tcp_tw_recycle in addition to net.ipv4.tcp_tw_reuse is mostly useless. And a final quote by W. Richard Stevens, in Unix Network Programming:

The TIME_WAIT state is our friend and is there to help us (i.e., to let old duplicate segments expire in the network). Instead of trying to avoid the state, we should understand it.

---

1. Notably, fiddling with net.netfilter.nf_conntrack_tcp_timeout_time_wait won t change anything on how the TCP stack will handle the TIME-WAIT state.
2. This diagram is licensed under the LaTeX Project Public License 1.3. The original file is available on this page.
3. The first work-around proposed in RFC 1337 is to ignore RST segments in the TIME-WAIT state. This behaviour is controlled by net.ipv4.rfc1337 which is not enabled by default on Linux because this is not a complete solution to the problem described in the RFC.
4. While in the LAST-ACK state, a connection will retransmit the last FIN segment until it gets the expected ACK segment. Therfore, it is unlikely we stay long in this state.
5. On the client side, older kernels also have to find a free local tuple (source address and source port) for each outgoing connection. Increasing the number of server ports or IP won t help in this case. Linux 3.2 is recent enough to be able to share the same local tuple for different destinations. Thanks to Willy Tarreau for his insight on this aspect.
6. This last solution may seem a bit dumb since you could just use more ports but some servers are not able to be configured this way. The before last solution can also be quite cumbersome to setup, depending on the load-balancing software, but uses less IP than the last solution.
7. The use of a dedicated memory structure for sockets in the TIME-WAIT is here since Linux 2.6.14. The struct sock_common structure is a bit more verbose and I won t copy it here.
8. When the server closes the connection first, it gets the TIME-WAIT state while the client will consider the corresponding quadruplet free and hence may reuse it for a new connection.

Piston, an Openstack-in-a-box vendor[1] are a sponsor of the Red Hat[2] Summit this year. Last week they briefly ceased to be for no publicly stated reason, although it's been sugggested that this was in response to Piston winning a contract that Red Hat was also bidding on. This situation didn't last for long - Red Hat's CTO tweeted that this was an error and that Red Hat would pay Piston's sponsorship fee for them.

To Red Hat's credit, having the CTO immediately and publicly accept responsibility and offer reparations seems like the best thing they could possibly do in the situation and demonstrates that there are members of senior management who clearly understand the importance of community collaboration to Red Hat's success. But that leaves open the question of how this happened in the first place.

Red Hat is big on collaboration. Workers get copies of the Red Hat Brand Book, an amazingly well-written description of how Red Hat depends on the wider community. New hire induction sessions stress the importance of open source and collaboration. Red Hat staff are at the heart of many vital free software projects. As far as fundamentally Getting It is concerned, Red Hat are a standard to aspire to.

Which is why something like this is somewhat unexpected. Someone in Red Hat made a deliberate choice to exclude Piston from the Summit. If the suggestion that this was because of commercial concerns is true, it's antithetical to the Red Hat Way. Piston are a contributor to upstream Openstack, just as Red Hat are. If Piston can do a better job of selling that code than Red Hat can, the lesson that Red Hat should take away is that they need to do a better job - not punish someone else for doing so.

However, it's not entirely without precedent. The most obvious example is the change to kernel packaging that happened during the RHEL 6 development cycle. Previous releases had included each individual modification that Red Hat made to the kernel as a separate patch. From RHEL 6 onward, all these patches are merged into one giant patch. This was intended to make it harder for vendors like Oracle to compete with RHEL by taking patches from upcoming RHEL point releases, backporting them to older ones and then selling that to Red Hat customers. It obviously also had the effect of hurting other distributions such as Debian who were shipping 2.6.32-based kernels - bugs that were fixed in RHEL had to be separately fixed in Debian, despite Red Hat continuing to benefit from the work Debian put into the stable 2.6.32 point releases.

It's almost three years since that argument erupted, and by and large the community seems to have accepted that the harm Oracle were doing to Red Hat (while giving almost nothing back in return) justified the change. The parallel argument in the Piston case might be that there's no reason for Red Hat to give advertising space to a company that's doing a better job of selling Red Hat's code than Red Hat are. But the two cases aren't really equal - Oracle are a massively larger vendor who take significantly more from the Linux community than they contribute back. Piston aren't.

Which brings us back to how this could have happened in the first place. The Red Hat company culture is supposed to prevent people from thinking that this kind of thing is acceptable, but in this case someone obviously did. Years of Red Hat already having strong standing in a range of open source communities may have engendered some degree of complacency and allowed some within the company to lose track of how important Red Hat's community interactions are in perpetuating that standing. This specific case may have been resolved without any further fallout, but it should really trigger an examination of whether the reality of the company culture still matches the theory. The alternative is that this kind of event becomes the norm rather than the exception, and it takes far less time to lose community goodwill than it takes to build it in the first place.

[1] And, in the spirit of full disclosure, a competitor to my current employer
[2] Furthering the spirit of full disclosure, a former employer

comments

Note Apparently, it's not Debian planet that has problems to deal with MathJax (which makes posts usingn it appear illegible on Debian planet), but the ikiwiki plugin that I am using which generates garbage in the feeds that get consumed by planet.debian.org. If you it know of a better plugin (which doesn't generate such output) please let me know, especially if it is not super computationally expensive for a Core 2 Duo T7250 (which is my notebook).

The real thing As William Stein is now offering a course on Number Theory and he has been posting recorded videos of his lectures, I started watching some of them (mainly, the ones regarding continued factions). In particular, he shows the usual recursive formula for convergents of a continued fraction and that's super nice. For the curious reader, the recurrence relations for the convergents of !!mathjaxbegin-i!! W2FfMCwgYV8xLCBcbGRvdHMsIGFfbiwgXGxkb3RzXQ== !!mathjaxend-i!! are: !!mathjaxbegin-d!! CnBfbiA9IGFfbiBwX3tuLTF9ICsgcF97bi0yfVxcCnFfbiA9IGFfbiBxX3tuLTF9ICsgcV97bi0y fSwK !!mathjaxend-d!! with initial conditions (!!mathjaxbegin-i!! cF97LTJ9ID0gMA== !!mathjaxend-i!!, !!mathjaxbegin-i!! cF97LTF9ID0gMQ== !!mathjaxend-i!!, !!mathjaxbegin-i!! cF8wID0gYV8w !!mathjaxend-i!!, !!mathjaxbegin-i!! cV97LTJ9ID0gMQ== !!mathjaxend-i!!, !!mathjaxbegin-i!! cV97LTF9ID0gMA== !!mathjaxend-i!!, !!mathjaxbegin-i!! cV8wID0gMQ== !!mathjaxend-i!!). He even motivated the use of continued fractions with the golden ratio, which is super nice, given that I like the subject and have been writing a document collecting facts that I know about the Fibonacci numbers (well, this document is horribly incomplete and not even close to something that I would consider proper for public consumption I plan on publishing them soon). OK, after he discussed the basics of it convergents, he noted that the recurrence relations are like those defining the Fibonacci numbers, just that one of the terms is "weighted" by the coefficients of the continued fraction. I missed one thing, though: neither his book nor wikipedia's article on continued fractions mentions a very neat, alternative way to express the convergents of continued fractions (truth be said, I took a quick peek at wikipedia's article and I found that it doesn't mentionat least explicitly, in a 1 min skim it may well be buried somewhere else). I, then, proposed the following exercise for his students (which he apparently liked, as he +1's the suggestion): Prove that the recurrence relation of the !!mathjaxbegin-i!! cF9p !!mathjaxend-i!!'s and !!mathjaxbegin-i!! cV9p !!mathjaxend-i!!'s that we mentioned before can be obtained via matrix multiplication. More precisely, prove that: !!mathjaxbegin-d!! ClxiZWdpbntibWF0cml4fQphXzAgJmFtcDsgMVxcCjEgJmFtcDsgMApcZW5ke2JtYXRyaXh9Clxi ZWdpbntibWF0cml4fQphXzEgJmFtcDsgMVxcCjEgJmFtcDsgMApcZW5ke2JtYXRyaXh9ClxjZG90 cwpcYmVnaW57Ym1hdHJpeH0KYV9uICZhbXA7IDFcXAoxICZhbXA7IDAKXGVuZHtibWF0cml4fQog PQpcYmVnaW57Ym1hdHJpeH0KcF9uICZhbXA7IHBfe24tMX1cXApxX24gJmFtcDsgcV97bi0xfQpc ZW5ke2JtYXRyaXh9Lgo= !!mathjaxend-d!! As a corollary, derive Cassini's identity for the Fibonacci Numbers.
Please, if you any errors on this, please let me know so that I can fix it. Edit: Thanks "noone" for spotting a typographical error.

This morning I noticed some parking bays reserved for car charging in a car park at the corner of Sydney Rd and Glenlyon St in Brunswick (near Aldi). One of the parking spots was occupied by a Plug-in Prius from GoGet [1]. I didn t even realise that you could get a plug-in Prius in Australia. The charging station is run by Charge Point [2]. The charging points are about 1.5m high and the cable is about 3cm thick (about as thick as the pipe used for filling a car with petrol), so it would charge a car much faster than could be done with a regular power point. One big problem with the Charge Point web site is that they don t give any information on pricing. They sell home charge points (which I guess means just an all-weather two-phase power point) but don t give a price for that. They sell charge points that can be used by commercially but don t give a price for them either. Also their infrastructure for billing is apparently based on companies installing charge points and setting a price for the service. Some charge points may offer free service (I guess staff car parks and some government agencies) and others will charge varying rates none of which is available on the web site. Apparently they have an online portal which gives information on such things to registered users so you have to register to discover what it costs. Of course hardly anyone is going to register before discovering the price, not even when registration is free. But while registration is free the web site demands the make and model of the electric car, so presumably one has to spend $40,000 or more on a vehicle before discovering the price and availability of charging it. Charge Point can be used as an example of how not to design a web site that promotes a service, or at least how not to promote a service that is aimed at saving money (electricity is significantly cheaper than petrol so it s of interest to people and organisations that want to save money). The Charge Point site seems to be better suited to showing that the concept can work than convincing people that they should sign up for it. It seems to me that the best thing that they could do would be to prominently display the average cost of all non-free charge points that are open to the public along with an explanation of the price of driving a desirable car (such as a plug-in Prius or a Nissan Leaf) with such an electricity cost. The contact section on the web site only has a link for careers . I don t think it s possible to get widespread use of electric vehicles without getting better information out there. It appears that Charge Point is relying on councils to do the work of promoting their business by installing their stations and reserving car parking as Moreland council has done in this case.

• [1] http://www.goget.com.au/
• [2] http://www.chargepoint.com.au/

I'm not an avid Kickstarter follower (if I were, I'd have gone broke long ago). I tend to wind up backing campaigns that come to my attention via other means (in other words, they're in the process of going viral). That said, I've backed plenty of campaigns over the years, and so I'd like to think I have a little bit of familiarity with how they usually operate. When Ozobot came to my attention, I found it unusual, because they were pre-promoting their Kickstarter campaign before it opened. To me, this looked like a case of them trying to build hype prior to the campaign opening, which was a new one to me. The whole thing seemed incredibly slick, and I was surprised they were "only" seeking $100K. The product looked like it'd be something cool for Zoe to play with, so I decided to back it anyway. Then all the updates started flowing in about how well it was being received at various trade shows and whatnot. Yet the amount of dollars flowing into the Kickstarter campaign didn't seem to be reflecting the external hype. I was watching the campaign's dashboard with great interest, because as time marched on, it was looking more and more likely that it wasn't going to make its funding target. This seemed highly unusual to me, given the slickness of the product and purported external interest in it. And then they pulled the plug on the campaign. Purportedly because they were pursuing equity funding instead. They admitted they'd also read the writing on the wall and it was unlikely they were going to make their funding target. I haven't followed other campaigns to see how much of a last minute funding "pop" they have. Usually I've found they've closed at many multiples of their original target, and hit their target well in advance of their deadline, when they're ridiculously popular. My interpretation of Ozobot's campaign, from a funding perspective, is that Kickstarters gave it a big fat "MEH", which surprised me somewhat. Then the question comes up: was the Kickstarter campaign a ruse all along? Was it just a new way of pitching for venture capital? The videos seemed pretty slick. The product seemed already complete, and $100K didn't seem like enough to take it to manufacturing. It'll be interesting to see what becomes of Ozobot now.

Hi Debian! Just to briefly introduce myself, I'm Vincent (vcheng@d.o, vincent_c on OFTC/Freenode) and I'm a recent graduate of the NM process, having attained DD-ship (is that even a word?) about a month ago in January. Before that, I was a DM since mid-2012, and just another contributor for a few years prior to that. Most of my contributions to the Project are related to packaging, especially packages maintained within the Debian Games Team. Outside of Debian, well, I'm just another 20-year-old, 3rd-year undergrad student studying at the University of British Columbia, working towards a major in computer science. I anticipate that I'll be able to ditch academia for good in 2016; just 2 more years to go... More about my life story in my AM's report. It's fairly boring, if I say so myself. This blog post isn't just about me; I'd like to touch upon another topic that I consider important, i.e. attracting new contributors, and in many cases that involves package sponsorship. That's one of the many things I'd like to work on as a DD, and to that effect I've dedicated a large chunk of the time I currently spend on Debian sponsoring various packages, including packages that I'm not familiar with. I would love to see the sponsorship-requests queue empty one day, but I admit that I find that increasingly unlikely to happen, now that I've gotten a taste of the time commitment involved with sponsoring. Team maintenance and sponsorship seems to work relatively well in Debian; why does that success not translate over to sponsorship-requests / debian-mentors@l.d.o, where a considerable amount of RFS requests don't see any response at all, and merely get closed once those packages get removed from mentors.debian.net (AFAIK they automatically get removed after 20 weeks)? Well, I'm not here to offer an answer for that, but I will point out that there was a relevant thread that originated on debian-devel a month ago: <87ha8lzl7n.fsf@inf-8660.int-evry.fr> (and no, I'm not talking about the giant init-related threads that pop up on debian-devel every other week or so and drown out everything else on the list...). Are DDs in general reluctant to engage in "fly-by sponsoring"? Most packages coming through the sponsorship-requests queue are not team-maintained, and are sent from sponsorees who are not already in contact with a sponsor, and who have no other way of getting in touch with a potential sponsor (assuming that there isn't an existing team or a DD who happens to be interested in that package). Regardless of the merits of fly-by sponsoring (many arguments in favour of, and against, this type of sponsorship have already been discussed in that thread; no need to re-iterate them here), it's discouraging for potential contributors new to Debian, who may just give up and move on to something else. I've now had the opportunity to be a sponsor (rather than just a sponsoree), and I acknowledge that being a sponsor is more work than I imagined it to be a month ago, and it can be quite tedious and not-fun at times. I suppose I can draw a parallel between sponsoring packages and RC bug fixing; neither is what I'd consider to be a highly enjoyable way of spending my leisure time, and both can eat up a large chunk of time and effort; both often involve working on something that you aren't familiar with. But both are beneficial to Debian. No, beneficial isn't a strong enough word; how about "essential" instead? Yeah, that makes more sense. Without dedicated RC bug squashers, Debian would never be able to make a release; without dedicated sponsors, the size of Debian's archive would be smaller and of lesser quality, and its community also a lot smaller. With all that said...I hope this blog post inspires at least a few DDs, possibly more, to dedicate a little bit more of their time towards sponsoring packages. :)

FreeBSD vt(4) -commonly known as Newcons -, which is planned to replace Syscons as the default FreeBSD console, is now available on Debian GNU/kFreeBSD. It is enabled by default, but only in kfreebsd-11 packages from experimental. Newcons provides many interesting new features, such as KMS support, Unicode, double-width CJK characters, etc. More details are available in FreeBSD wiki. Ironically, Debian GNU/kFreeBSD is in a bit more of a hurry to deploy Newcons than FreeBSD is. The reason for this is that Newcons is practically a requirement for using the new KMS drivers. While FreeBSD Ports, in order to ensure a smooth transition, preserve support for UMS (User Mode Setting) in their X11/DRI userland, Debian supports many kernels and (contrary to some ill claims I heard) gives priority to new features on Debian GNU/Linux port, which is the one with most users and developers. In this case, it means KMS is the only option when it comes to X11/DRI userland in Debian. Anyway, we ve been making some nice progress. Here s a screenshot of Newcons in action in a Debian system (running on VirtualBox, thus with VGA backend): The idea is to backport this to kfreebsd 10.0, as FreeBSD plans to merge it into stable/10 anyway. However, it is still in the process of being tested (and being polished on FreeBSD, I hear they plan to merge it on March).

I made some changes on http://llvm.org/apt/ for the last 2 months.

• Added trusty, Ubuntu 14.04, as a new supported distribution (on the request of Michael Larabel, Phoronix)
• Support both the stable and development version. Currently, that means that the release_34 branch and the trunk are built. So, for example, clang-3.4 and clang-3.5 can be installed.
  release_34 are only built when a new commit is submitted in this branch. trunk is built twice a day.
• Add a new package llvm- 3.4,3.5 -tools which contains various tools to build software/packages on top of llvm. Contributed by Martin Nowack in the context of Klee.
• Since a C++ 11 compiler is now mandatory, I had to force the usage of a backported gcc/g++ 4.8 (thanks Doko).
  This is the case for Ubuntu Precise (12.04), Quantal (12.10) and raring (13.04).
  The thing is that it triggers a dependency on the libstdc++ 4.8 causing the PPA to be mandatory.
  deb http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu $DISTRIBUTION main
  
  For now, because of the lack of backport of gcc 4.8, I am not providing support for Debian stable (wheezy).

Did you ever want to be able to tag your files, and use the tags to query and select the files you want? For many sorts of files we use, this is clearly better than being locked into a single hierarchial view of nested directories. Semantic file systems are a way to do things like this. But that's an entire separate filesystem, often implemented as a FUSE layer. I have never wanted tagging enough to take on that layer of complications. A week ago I realized that there was a way to do this without using an entirely separate filesystem. When we use git, we're used to having different views of our files, called branches, that we switch between. What then, if a git branch were generated from tags and other metadata that meet a query, providing a custom view that meet the user's current needs and could be incrementally refined. Here's a short screencast demoing these views.
And here's a walkthrough for setting it up. To build this, I needed a nice way to store metadata in git, and since git-annex happens to have a very nice way of using git as a database, I naturally built it on top of git-annex. This means that the tags and other metadata are automatically synchronized between different clones of the repository. Multiple users can be tagging and setting other metadata at the same time, and their changes will merge in a consistent way. I have a long list of things to do to fully integrate views into git-annex. However, they're already basically usable, and I'm very pleased with how these dymanic views of the contents of a repository are working out. Today's release of git-annex includes views support, so give them a try!

So Whatsapp has been acquired by Facebook and this news is still hot and people are discussing it all over the twitterverse. So I took this opportunity to stop using Whatsapp and removed it from my phone. Possibly I could have deleted my account but who cares. Anyway I've been searching for a secure and FLOSS alternative for Whatsapp for quite some time now, few days ago I found about Telegram but after reading post by Tincho on planet Debian, I decided not to use it. Recently while going through the talk list for fossmeet.in I found link for Kontalk in a privacy awareness talk proposal by Praveen and thought I should give it a try. So below is my first feelings about Kontalk.

Wowsers, today was a pretty full, but laid back day. Zoe has this small wrist-sized ring with a bunch of different coloured long ribbons on it that she got from a birthday party a while ago. She plays with it every now and then, and uses it to torment Smudge. This morning she was dancing around with it, and it made me think of rhythmic gymnastics, so using my trusty Chromecast, I dug up some Youtube coverage from the 2012 Olympics. Zoe loved the sparkly costumes and the routines. The first video alternated between ball and hoop routines. Zoe went out and rummaged around her craft drawers and came back with a styrofoam ball and was using her crown from yesterday as a hoop. It kept her occupied for about an hour. I watched the first three competitors with her and she continued to watch it while I had a shower. I see a hula hoop in her future. We biked over to her Brazilian Jiu Jitsu class and had another really good class. I think Zoe's really enjoying the mix of gymnastics-style activities and game play. She also got to tackle me today, which she thoroughly enjoyed. The instructor suggested we might like to try out one of the 4-7 year old proper martial arts classes, so we'll try that in the coming weeks. We biked home the scenic way again and picked up a few additional supplies for lunch. When we got home, Zoe went off and played with some of her toys on her own. It's great that she's getting more independent and self-directed like that. I'd invited Zoe's great-grandparents (on her mother's side) over for lunch today. I think it's wonderful that Zoe has great-grandparents who are still alive and able to be in her life. We had a nice lunch and they wanted to see the kinetic sand I'd blogged about, so we got that out after lunch for a play. Then Zoe wanted to watch some more rhythmic gymnastics with them, so we did that for a bit. After they left, I swapped over to rhythmic gymnastics with the ribbon and clubs. The ribbon is my personal favourite one to watch, and I think Zoe enjoyed it too, because she immediately started waving her ribbons all over the place. I trialled skipping her nap today, and instead called watching the gymnastics "quiet time". After quiet time, we did our Science Friday activity, which was messing around with conductive play dough. Zoe had about as much fun just messing around with the play dough as she did running electricity through it, and enjoyed mushing the non-conductive white play dough in with the conductive blue play dough. We did a few experiments, notably one with three LEDs in series, and then shorting the play dough at either end of the series to see what it did to the lights. She enjoyed wearing her lab coat and safety glasses. I'd wanted to do bottle rockets today, because we'd built a cardboard box rocket earlier in the week, and that seemed like a good theme to stick with, but I was ill-prepared, and it was too hot anyway. I want to do Diet Coke and Mentos and reuse the Coke bottles for the bottle rockets, so that'll have to be in a few weeks' time. I prepped dinner on and off while Zoe played with the play dough, so it was all ready to pop in the oven nice and early. After I cleaned up the mess from the play dough, we played with her coloured peg board game for a bit, and then she wanted to see how the sofa bed folded out, so we did that I used it as an opportunity to give it a thorough vacuuming. After that she watched a bit more TV while dinner was in the oven. After dinner, we walked down to the Hawthorne Garage for a babyccino. Zoe had to pat every dog that crossed our path. Zoe didn't seem too worse for wear for having dropped her nap (which was to be expected, it's only a recent phenomenon for her to nap at home), but she was showing signs of possibly coming down with a cold (being a bit sniffy, without having obvious signs of a runny nose) so I put her to bed about half an hour early. She went down pretty easily, which was a pleasant change. I did miss the 2 hour opportunity to get things done during the day though. C'est la vie. The thing I liked most today is Zoe's independently going and playing with toys that she's had for ages, but not really shown much interest in. She's had this Melissa and Doug board with interchangeable sheets with animals on them and a bunch of different colour octagonal plastic pegs that you're supposed to match the coloured pegs to whatever colours are on the sheet you put on the board. We did that this afternoon, and this morning she played a lace up game where you thread shoelaces through some thick cardboard outlines of animals. I think she got both for Christmas in 2012.

When I am at home I do not want to be bothered by the screensaver locking my laptop. To solve this, I use a custom PAM configuration which checks if I am authenticated to the local access point. Add this to the top of /etc/pam.d/xscreensaver:

auth sufficient pam_exec.so quiet /usr/local/sbin/pam_auth_xscreensaver

And then use a script like this one to decide when you want the display to be automatically unlocked:

#!/bin/sh -e
# return the ESSID of this interface
current_essid()  
  /sbin/iwconfig $1   sed -nre '/ESSID/s/.*ESSID:"([^"]+)".*/\1/p'
 
# automatically unlock only for these users
case "$PAM_USER" in
  "")   echo "This program must be run by pam_auth.so!"
        exit 1
        ;;
  md)   ;;
  *)    exit 1
        ;;
esac
CURRENT_ESSID=$(current_essid wlan0)
# automatically unlock when connected to these networks
case "$CURRENT_ESSID" in
  MYOWNESSID) exit 0 ;;
esac
exit 6