Russell Coker: Improving Blog Latency to Benefit Readers
I just read an interesting post about latency and how it affects web sites [1]. The post has some good ideas but unfortunately mixed information on some esoteric technologies such as infiniband that are not generally applicable with material that is of wide use (such as ping times).
The post starts by describing the latency requirements of Amazon and stock broking companies. It’s obvious that stock brokers have a great desire to reduce latency, it’s also not surprising that Google and Amazon analyse the statistics of their operations and make changes to increase their results by a few percent. But it seems to be a widely held belief that personal web sites are exempt from such requirements. The purpose of creating content on a web site is to have people read it, if you can get an increase in traffic of a few percent by having a faster site and if those readers refer others then it seems likely to have the potential to significantly improve the result. Note that an increase in readership through a better experience is likely to be exponential, and an exponential increase of a few percent a year will eventually add up (an increase of 4% a year will double the traffic in 18 years).
I have been considering hosting my blog somewhere else for a while. My blog is currently doing about 3G of traffic a month which averages out to just over 1KB/s, peaks will of course be a lot greater than that and the 512Kb/s of the Internet connection would probably be a limit even if it wasn’t for the other sites onn the same link. The link in question is being used for serving about 8G of web data per month and there is some mail server use which also takes bandwidth. So performance is often unpleasantly slow.
For a small site such as mine the most relevant issues seem to be based around available bandwidth, swap space use (or the lack therof), disk IO (for when things don’t fit in cache) and available CPU power exceeding the requirements.
For hosting in Australia (as I do right now) bandwidth is a problem. Internet connectivity is not cheap in any way and bandwidth is always limited. Also the latency of connections from Australia to other parts of the world often is not as good as desired (especially if using cheap hosting as I currently do).
According to Webalizer only 3.14% of the people who access my blog are from Australia, they will get better access to my site if hosted in Australia, and maybe the 0.15% of people who access my blog from New Zealand will also benefit from the locality of sites hosted in Australia. But the 37% of readers who are described as “US Commercial” (presumably .com) and the 6% described as “United States” (presumably .us) will benefit from US hosting, as will most of the 30% who are described as “Network” (.net I guess).
For getting good network bandwidth it seems that the best option is to choose what seems to be the best ISP in the US that I can afford, where determining what is “best” is largely based on rumour.
One of the comments on my post about virtual servers and swap space [2] suggested just not using swap and referenced the Amazon EC2 (Elastic Computing) cloud service and the Gandi.net hosting (which is in limited beta and not generally available).
The Amazon EC2 clound service [3] has a minimum offering of 1.7G of RAM, 1EC2 Compute Unit (equivalent to a 1.0-1.2GHz 2007 Opteron or 2007 Xeon processor), 160G of “instance storage” (local disk for an instance) running 32bit software. Currently my server is using 12% of a Celeron 2.4GHz CPU on average (which includes a mail server with lots of anti-spam measures, Venus, and other things). Running just the web sites on 1EC2 Compute Unit should use significantly less than 25% of a 1.0GHz Opteron. I’m currently using 400M of RAM for my DomU (although the MySQL server is in a different DomU). 1.7G of RAM for my web sites is heaps even when including a MySQL server. Currently a MySQL dump of my blog is just under 10M of data, with 1.7G of RAM the database should stay entirely in RAM which will avoid the disk IO issues. I could probably use about 1/3 of that much RAM and still not swap.
The cost of EC2 is $US0.10 per hour of uptime (for a small server), so that’s $US74.40 per month. The cost for data transfer is 17 cents a GIG for sending and 10 cents a gig for receiving (bulk discounts are available for multiple terabytes per month).
I am not going to pay $74 per month to host my blog. But sharing that cost with other people might be a viable option. An EC2 instance provides up to 5 “Elastic IP addresses” (public addresses that can be mapped to instances) which are free when they are being used (there is a cost of one cent per hour for unused addresses - not a problem for me as I want 24*7 uptime). So it should be relatively easy to divide the costs of an EC2 instance among five people by accounting for data transfer per IP address. Hosting five web sites that use the same software (MySQL and Apache for example) should reduce memory use and allow more effective caching. A small server on EC2 costs about five times more than one of the cheap DomU systems that I have previously investigated [4] but provides ten times the RAM.
While the RAM is impressive, I have to wonder about CPU scheduling and disk IO performance. I guess I can avoid disk IO on the critical paths by relying on caching and not doing synchronous writes to log files. That just leaves CPU scheduling as a potential area where it could fall down.
Here is an interesting post describing how to use EC2 [5].
Another thing to consider is changing blog software. I currently use Wordpress which is more CPU intensive than some other options (due to being written in PHP), is slightly memory hungry (PHP and MySQL), and doesn’t have the best security history. It seems that an ideal blog design would use a language such as Java or PHP for comments and use static pages for the main article (with the comments in a frame or loaded by JavaScript). Then the main article would load quickly and comments (which probably aren’t read by most users) would get loaded later.
- [1] http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it
- [2] http://etbe.coker.com.au/2008/08/27/killing-servers-virtualisation-and-swap/
- [3] http://www.amazon.com/b/?node=201590011
- [4] http://etbe.coker.com.au/2008/05/28/xen-hosting/
- [5] http://www.protocolostomy.com/2008/08/27/more-adventures-in-amazon-ec2-and-ebs/
Tired and under the influence of beer, I tried to remove the
Rebuilding packages fast with distcc and ccache.
I've found that I've neglected ecasound package for a long time.
It's a pretty big C++ package, and one of the reasons I don't want to touch it recently is the
time it takes to build the package.
I played with distcc/ccache a little bit and found that it's pretty easy to set it up
working with debuild thanks to those people who made it possible.
debuild (or dpkg-buildpackage) now takes -j option for parallelism,
and there are some options for ccache/distcc.
I just crafted ~/.devscript, and ran debuild -j5, and
the build went zooming fast. Cool.
The McCain campaign made a great tactical move by choosing
For those who missed my message at #debconf: I had troubles creating a new album at gallery.dc.o so I uploaded some of the photos I took on my trip to DC8 and are available at:
According to the
So,
The other day I had the "pleasure" to debug some PHP code.
It always showed the very useful error message Database Error.
As it turned out, this error message is printed out at over 38 (!)
different places of the PHP script I had to debug. It also included
several other PHP scripts, which printed the very same error message.
Please, if you are to lazy to print useful error messages, at least
make them unique, so the poor soul who got the task to debug your stuff
knows more or less where the error happens. Even numbering all your
Database Errors would have helped a bit...


As I start this blog post, I'm on the train between Amersfoort and
Osnabr ck, on my way to Kiel, where there'll be a meeting of the m68k
porters, kindly organized by m68k kernel maintainer Christian Steigies.
I took the 8:40 train in Mechelen this morning, and if all goes well,
I'll be arriving in Kiel at 17:22 tonight. Some train trip... but I
certainly prefer that to any flight.
Anyway. The days of Debconf were very nice. Debconf is a lot
of things to me. Travel. Hacking. Mao, evolving into a drinking game.
Beer. Attending talks. Giving talks. Meeting people.
Face-to-face non-flaming. Gesturing for an extra knife at luunch.
Hangover. Whiskey. Talking to the person next to you—over IRC.
Getting killed before knowing what the rules of the assassins game are.
Filing bugreports in person. 



Our Belfast office has finally got large enough to have to partake in the
Xorg 1.4 in experimental is supposed to have input device hotplugging.Does anyone have a Howto for Debian? I tried it, but I couldn't get it to
hot-plug my USB mouse, so I'm back to using the regular mouse driver for it
again, using the /dev/input/mice in-kernel-hack for hotplugging.P.S. on a recent kernel, you might want to add
