Search Results: "ag"

30 August 2008

Russell Coker: Improving Blog Latency to Benefit Readers

I just read an interesting post about latency and how it affects web sites [1]. The post has some good ideas but unfortunately mixed information on some esoteric technologies such as infiniband that are not generally applicable with material that is of wide use (such as ping times). The post starts by describing the latency requirements of Amazon and stock broking companies. It’s obvious that stock brokers have a great desire to reduce latency, it’s also not surprising that Google and Amazon analyse the statistics of their operations and make changes to increase their results by a few percent. But it seems to be a widely held belief that personal web sites are exempt from such requirements. The purpose of creating content on a web site is to have people read it, if you can get an increase in traffic of a few percent by having a faster site and if those readers refer others then it seems likely to have the potential to significantly improve the result. Note that an increase in readership through a better experience is likely to be exponential, and an exponential increase of a few percent a year will eventually add up (an increase of 4% a year will double the traffic in 18 years). I have been considering hosting my blog somewhere else for a while. My blog is currently doing about 3G of traffic a month which averages out to just over 1KB/s, peaks will of course be a lot greater than that and the 512Kb/s of the Internet connection would probably be a limit even if it wasn’t for the other sites onn the same link. The link in question is being used for serving about 8G of web data per month and there is some mail server use which also takes bandwidth. So performance is often unpleasantly slow. For a small site such as mine the most relevant issues seem to be based around available bandwidth, swap space use (or the lack therof), disk IO (for when things don’t fit in cache) and available CPU power exceeding the requirements. For hosting in Australia (as I do right now) bandwidth is a problem. Internet connectivity is not cheap in any way and bandwidth is always limited. Also the latency of connections from Australia to other parts of the world often is not as good as desired (especially if using cheap hosting as I currently do). According to Webalizer only 3.14% of the people who access my blog are from Australia, they will get better access to my site if hosted in Australia, and maybe the 0.15% of people who access my blog from New Zealand will also benefit from the locality of sites hosted in Australia. But the 37% of readers who are described as “US Commercial” (presumably .com) and the 6% described as “United States” (presumably .us) will benefit from US hosting, as will most of the 30% who are described as “Network” (.net I guess). For getting good network bandwidth it seems that the best option is to choose what seems to be the best ISP in the US that I can afford, where determining what is “best” is largely based on rumour. One of the comments on my post about virtual servers and swap space [2] suggested just not using swap and referenced the Amazon EC2 (Elastic Computing) cloud service and the Gandi.net hosting (which is in limited beta and not generally available). The Amazon EC2 clound service [3] has a minimum offering of 1.7G of RAM, 1EC2 Compute Unit (equivalent to a 1.0-1.2GHz 2007 Opteron or 2007 Xeon processor), 160G of “instance storage” (local disk for an instance) running 32bit software. Currently my server is using 12% of a Celeron 2.4GHz CPU on average (which includes a mail server with lots of anti-spam measures, Venus, and other things). Running just the web sites on 1EC2 Compute Unit should use significantly less than 25% of a 1.0GHz Opteron. I’m currently using 400M of RAM for my DomU (although the MySQL server is in a different DomU). 1.7G of RAM for my web sites is heaps even when including a MySQL server. Currently a MySQL dump of my blog is just under 10M of data, with 1.7G of RAM the database should stay entirely in RAM which will avoid the disk IO issues. I could probably use about 1/3 of that much RAM and still not swap. The cost of EC2 is $US0.10 per hour of uptime (for a small server), so that’s $US74.40 per month. The cost for data transfer is 17 cents a GIG for sending and 10 cents a gig for receiving (bulk discounts are available for multiple terabytes per month). I am not going to pay $74 per month to host my blog. But sharing that cost with other people might be a viable option. An EC2 instance provides up to 5 “Elastic IP addresses” (public addresses that can be mapped to instances) which are free when they are being used (there is a cost of one cent per hour for unused addresses - not a problem for me as I want 24*7 uptime). So it should be relatively easy to divide the costs of an EC2 instance among five people by accounting for data transfer per IP address. Hosting five web sites that use the same software (MySQL and Apache for example) should reduce memory use and allow more effective caching. A small server on EC2 costs about five times more than one of the cheap DomU systems that I have previously investigated [4] but provides ten times the RAM. While the RAM is impressive, I have to wonder about CPU scheduling and disk IO performance. I guess I can avoid disk IO on the critical paths by relying on caching and not doing synchronous writes to log files. That just leaves CPU scheduling as a potential area where it could fall down. Here is an interesting post describing how to use EC2 [5]. Another thing to consider is changing blog software. I currently use Wordpress which is more CPU intensive than some other options (due to being written in PHP), is slightly memory hungry (PHP and MySQL), and doesn’t have the best security history. It seems that an ideal blog design would use a language such as Java or PHP for comments and use static pages for the main article (with the comments in a frame or loaded by JavaScript). Then the main article would load quickly and comments (which probably aren’t read by most users) would get loaded later.

Martin F. Krafft: Recovering a lost default route

Tired and under the influence of beer, I tried to remove the currently broken IPv6 default route from my primary mailserver via an SSH connection and accidentally executed instead:
ip r d default
# I meant: ip -6 r d default

Unfortunately, I didn t immediately realise what I had done, and then it was too late. Robert Collins told me that I could have saved the session because even without a default route, packets still reach the machine. Unfortunately, due to Nagle s algorithm, you pretty much only have one shot, so the next time this happens to me (or you), quickly type
ip r a default via <gateway>

into a separate window, then copy and paste it into the SSH session and you may get lucky. Update: one lesson I learnt from this is to add specific routes for at least the IPv6 tunnel peer, and possibly another machine (or two) under my control out there. I now have that in place on all machines I own. NP: The Phoenix Foundation: Pegasus

Junichi Uekawa: Rebuilding packages fast with distcc and ccache.

Rebuilding packages fast with distcc and ccache. I've found that I've neglected ecasound package for a long time. It's a pretty big C++ package, and one of the reasons I don't want to touch it recently is the time it takes to build the package. I played with distcc/ccache a little bit and found that it's pretty easy to set it up working with debuild thanks to those people who made it possible. debuild (or dpkg-buildpackage) now takes -j option for parallelism, and there are some options for ccache/distcc. I just crafted ~/.devscript, and ran debuild -j5, and the build went zooming fast. Cool.

Joerg Jaspert: Back in Germany

I’m back in Germany since Thursday already. Without any Jetlag! My flight back was delayed nearly an hour (bah), but other than that went without problems. DebConf was great! As was my vacation after it. Have a lot of pictures to upload in the next few days (and possibly some text to write about it). Apropos pictures - upload your DebCamp/DebConf pictures to the DebConf gallery too please, if you haven’t already. :) Comments: 0

Frank S. Thomas: Converting git-svn tag branches to real tags

Over the last few days I’ve converted some Subversion repositories used for Debian packaging to Git. None of these Subversion repositories contained upstream sources because of Subversion’s storage inefficiency. With Git I wanted to change that and also track the upstream sources in my repositories. To move a Git repository from a debian-only to debian+upstream layout I found zack’s recipe very helpful. The actual conversion to Git was done with git-svn. I cloned the Subversion repositories with git-svn’s --prefix=svn-import/ and --stdlayout options so that the trunk and all branches and tags were imported as remote branches with svn-import/ prepended. Then I created local branches of the previous Subversion branches and removed the remote remnants, for example:
git checkout -b debian/backports/etch svn-import/branches/etch-backports
git branch -d -r svn-import/branches/etch-backports
The Subversion tags are imported as svn-import/tags/<version> where <version> is the Debian version number. I don’t need branches for every version of my packages but I wanted to convert these tags to real Git tags but without losing the actual commit dates and messages. To achieve this I wrote this small script:
#!/bin/bash
for branch in  git branch -r ; do
    if [  echo $branch   egrep "svn-import/tags/.+$"  ]; then
        version= basename $branch 
        subject= git log -1 --pretty=format:"%s" $branch 
        export GIT_COMMITTER_DATE= git log -1 --pretty=format:"%ci" $branch 
        echo "Tag $version [Y/n]?"
        read yesno
        if [ -z $yesno ]   [ $yesno = "Y" ]; then
            git tag -s -f -m "$subject" "debian/$version" "$branch^"
            git branch -d -r $branch
        fi
    fi
done
For each remote branch that contains svn-import/tags/ it gets the version number, the commit message of the tag and exports GIT_COMMITTER_DATE (man git-tag) with the value of the date the version was tagged in Subversion and then asks you if it should tag the parent of the tag branch with the original commit date and message and remove the then unnecessary tag branch. The “safety measure” is there because people sometimes commit directly to Subversion tags (ugh!) and then "$branch^" would not be the commit you want to tag. To decide which Subversion tags are safe to tag with this script, gitk --all can be of great help. P.S.: This is my first post on Planet Debian, so hello Planet! :-)

Jaldhar Vyas: And That Just About Wraps it up For Obama

The McCain campaign made a great tactical move by choosing Sarah Palin. She appeals to all the constituencies where McCain was weak: women, Christians, Libertarians, small state residents, energy policy wonks, corruption fighters, capitalists, gun enthusiasts, and Tina Fey fetishists. Downsides? She is inexperienced in national affairs but I really don't think the average voter will care. After all many of them were willing to overlook a similar level of inexperience in Obama. And there is one minor possible ethics situation which again will only excite the true believers. When it turns out to be overblown it may actually rebound in Palins favor as an example of MSM manipulation. She can also take the change meme away from the Democrats though that was already on the rocks what with the FISA vote and the choice of Neil Kinnock as his VP nominee. No, the lacklustre convention in Denver clearly showed that far from being the post-ideological, audaciously hopeful candidate of the future, Barack Obama is merely John Kerry with rhythm. So what do the Dems have have left? The traditional Democratic themes of "We're not Bush" (that worked sooo well the last two times didn't it?) and "The Earth is about to explode in a fiery cataclysm." As the situation in Iraq improves, the economy climbs upwards from recession and petrol prices continue to come down its going to be harder to sustain the necessary level of gloom and doom. While Republicans shouldn't exhale just yet (and the title of this post is just a teensy bit hyperbolic,) things are looking pretty good for us at the moment.

29 August 2008

Raphael Geissert: Debconf8 photos

For those who missed my message at #debconf: I had troubles creating a new album at gallery.dc.o so I uploaded some of the photos I took on my trip to DC8 and are available at:
http://my.opera.com/atomo64/albums/

Please note that some people already requested me to remove some photos (the funniest ones) :(
Anyway, enjoy them! :)

Aigars Mahinovs: 24th September as a day against software patents - Debian support?

According to the Slashdot article and the StopSoftwarePatents.org website itself and on Digg, the anti-software-patent activists are attempting a world-wide event on the 24th of September as a world-wide day against software patents. USA has them via a weird court ruling, Japan has them as well (not sure why), there have been efforts to force software patents on EU, India, Australia and many other countries either by Microsoft lobbies or even via US trade treaty pressure. While lobbying by local Microsoft branches and their pet companies can be countered locally, like I and many others did in EU a couple years ago (mostly thanks to FFII), trade negotiations are very secretive affairs and it is very hard to lobby there directly. The public needs to be aware of the issue, otherwise the politicians will not be aware of its importance to their voters. Software patents are a threat to free software as it circumvents the power of GPL and other copyright licenses if enough money is thrown at the lawyers. If a software patent is generic enough, it can easily stop development of a whole class of free software applications on a whim of the patent holder. And there are plenty of granted software patents with a very broad scope (progress bar, anyone?). I am aware that to get Debian support for such an initiative, a GR is needed, but how about a personal word of support from the DPL or support from the SPI? It is in direct area of interest for SPI - software patents create an ever present threat of legal action against any and all software in Debian (in USA and Japan, at least) and, as the legal umbrella of Debian in the USA, SPI would be a prime target. What do others think about this?

David Welton: &quot;Blog&quot; vs the English language

I've never been a fan of the word "blog" - I believe it sounds like something one calls a plumber to deal with. Even worse is "blog" as a verb. "I blogged it yesterday". Did you really? Was it expensive to get cleaned up? Here are some alternatives that I came up with: I'm sure I'm missing some good ones...

Christian Perrier: D-I string freeze started

So, the string freeze for Debian Installer was announced today. It will last until Spetember 20th 23:59 UTC. We start with 13 complete languages and 15 meeting the release criteria. 60 (59+English) languages are currently supported. How many will meet the release criteria in 22 days?

Alexander Reichle-Schmehl: How to code... not!

The other day I had the "pleasure" to debug some PHP code. It always showed the very useful error message Database Error. As it turned out, this error message is printed out at over 38 (!) different places of the PHP script I had to debug. It also included several other PHP scripts, which printed the very same error message. Please, if you are to lazy to print useful error messages, at least make them unique, so the poor soul who got the task to debug your stuff knows more or less where the error happens. Even numbering all your Database Errors would have helped a bit...

Meike Reichle: 3rd Congress on Solidary Economy Bremen

Poster for the 3rd congress on solidary economy in Bremen If you happen to be near Bremen, Germany around the 5.-7. September 2008 and have an interest in solidary economy, free software, renewable energies and the like you might want to stop by at the 3rd Congress on Solidary Economy (German website). I haven't been at the first two of these congresses but I was invited as a speaker for this one, so I'll be around at least on Saturday and probably also on Sunday. I'll mainly be on a panel on free and open source software which will take place on Saturday evening at 20:00. (Prime Time :)) But I've also let myself be talked into joining another panel at 17:00 which is somehow ubuntu-related (Communtu). I am not yet sure if I'll have much to contribute there, I am neither a ubuntu specialist nor particularly fond of it, but the organiser said he wouldn't mind and he'd welcome critical remarks just as much, so, we'll see. I am really looking forward to being a speaker on an event again! Ever since I started my PhD I've been much too busy to attend (leave alone prepare talks for) as many events as I'd have liked. I hope the situation will improve next year when things get rolling with my PhD. (Currently still in a "topic orientation/find a project/write proposals" state ...)

Wouter Verhelst: Debconf8, and the m68k meeting in Kiel

As I start this blog post, I'm on the train between Amersfoort and Osnabr ck, on my way to Kiel, where there'll be a meeting of the m68k porters, kindly organized by m68k kernel maintainer Christian Steigies. I took the 8:40 train in Mechelen this morning, and if all goes well, I'll be arriving in Kiel at 17:22 tonight. Some train trip... but I certainly prefer that to any flight. Anyway. The days of Debconf were very nice. Debconf is a lot of things to me. Travel. Hacking. Mao, evolving into a drinking game. Beer. Attending talks. Giving talks. Meeting people. Face-to-face non-flaming. Gesturing for an extra knife at luunch. Hangover. Whiskey. Talking to the person next to you—over IRC. Getting killed before knowing what the rules of the assassins game are. Filing bugreports in person. Pictures. Kilts. Streaming video. Not being sick, hopefully. Name badges. Flying. Sleeping. Yes, sleeping. Most of all, though, this year, Debconf was just great. Thanks. You know who you are. After debconf, I flew to Buenos Aires, where I slept for one night in a Youth Hostel somewhere downtown. They were affordable, but the bed wasn't great—the mattress sorely needed replacement. Since I had a day in BA, and since there were apparently not enough people on the schedule, I'd agreed to hold a talk, and came up with the idea of a 'debian secrets' talk—about Debian-specific commands, such as dpkg-divert and update-alternatives etc—so that people could learn how to use their Debian system more efficiently. As I was sitting in Andreas' talk, who was right before me on the schedule, suddenly Dag walked up to me and said hi. This was unexpected; Dag is a fellow Belgian, who's involved with the CentOS project, and who maintains a positively huge RPM repository at his site—if you maintain an RPM-based system somewhere, you'll probably know about that site. So while I recognized him, I immediately wondered what he was doing there. Turned out he was invited to hold a talk at the Free Software event to which Debianday was attached (and which would not start for another day or two), and that when he saw my name on the schedule, he decided to attend. Fun. He learned a thing or two from my talk, and was talking later about writing some tools for RPM-based systems that would perform the same or similar functionality implemented by some of the Debian tools I talked about. Great. We had a picture together (in front of the DebConf banner—hah) that I still should upload, and then went out for lunch together with some other CentOS guys. The talk itself seems to have been well-received, and I'm glad about that; I only gave it because there was a need for more talks, not because I felt confident I was very knowledgeable about that subject. In fact, I did have to ask on the debconf-discuss mailinglist for some input (which I received) so that I could make sure the talk would actually be useful to people. That helped a lot, I guess. After that, we played some mao in the lobby of the DebianDay venue. While doing so, I overheard Dag talking about CentOS, advocating it to some of the people there, which I must say felt pretty weird on a Debian event. Not that there's anything wrong with it, of course. Except that Dag was supposed to be writing his talk slides. How did that work out, Dag? ;-) Eventually, I got in a cab to the EZE airport, and flew home. That wasn't fully without issue, but I did get there. And now, I've been, eh, overloaded. Still have to follow up on a question a customer asked me, but I've barely been home or at the office, just enough to sleep really. Should do something about that, I guess... Two days ago, I also managed to forget my camera somewhere. For 12 hours, I was worried, even though I had a pretty good idea of where it was; but since I'm getting quite good at taking pictures now, and since I really like doing so as well, I really didn't want to lose my camera. Fortunately, when I called this morning to the place where I thought I'd lost it, they told me it'd been found, and where to get it. In other words, it's safe, it's taken care of, Philip went to get it, and I just need to arrange to get it when I get back. Which is a relief. Let's not be so stupid anymore in the future... Finishing up now, a day later, and I'm at Christian's office, playing with his and my coldfire board. As it happens, it appears that Freescale has brought out a new BSP for these boards, so that's nice. Let's see whether we can get those to boot again...

Kartik Mistry: I think Honeymoon is over..


* I am using Mac OS X at work from last two months. Nice OS with feel of Linux is with you as you can do ssh with same feeling (Btw, I have Debian on same laptop, which is for my night hackings). I was impressed by Mac applications and its look-and-feel, but from some days, following kind of messages are frequent. and this is LOL:

28 August 2008

Clint Adams: In the next phase, astronut teaches the bot

This is the night that the clockwork burns down. This is the night of elbows scraping against iron bars. Tom Stoppard thought he had a glass full of fractals, when really it was an ass full of dactyls. This is the night of candelabras crashing into the shoes of giants. This is the night when the man with the homburg sings a hymn for the hopeless. The novice groped an honored mater outside the slic hut. This is the night that the hunger reached out for the abyss and fell in. This is the night that the crack broke its own mother's back. Sweet Nutty went to the levee and the levee didn't like it one bit. This is the night of the British spellings. This is the night of the sandpiper's dream. Lady Starbucks wondered about the healing powers of the dental-floss bikini. This is the night that nothing lasts forever. This is the night that nothing ever ends. The Q2 Gang tripped the light fantastic while waxing the text unreadable. This is the night of the Magic Mainspring. This is the night of remembering Trufflehunter's name. There's ice cream waiting for me in another world.

Jeff Bailey: Honesty

Roughly transcribed conversation between me and Pizza Hut customer service:

me on phone: Hi! I posted a comment on your website a week and a half ago asking how I add toppings to a pizza that a store location carries, but aren't standard across the franchise. How long does it usually take to respond?

PH rep after asking for contact details: I don't see it here in the system. It's unusual that it takes this long to get keyed in. I can enter it directly though.



PH rep: Well, since we have been experiencing issues with the website for some time, I'll send this to the web team to let them know that this is one of the issues, too. In the meantime, sir, I advice you to place the order over the phone.

Well. +1 for honesty, I suppose. =) I can't think of another way she could've resolved the issue.

Jonathan McDowell: Neither really means neither

Our Belfast office has finally got large enough to have to partake in the NI Equality Commission monitoring. This covers religion and gender. The options for religion are:

Which all seems fine, except for the bit below that says "If you do not complete this questionaire, we are encouraged to use the "residuary" method, which means that we can make a determination on the basis of personal infomation on file/application form". It turns out that also applies to the neither option. So I clarified with the poor guy who's having to deal with this what he'd do if I ticked neither. And he rang the Equality Commission for some advice. And it turns out that although the wording in the step-by-step guide for employers is "we strongly advise" essentially if you end up with more than a certain number of neithers then you put yourself at risk of an investigation.

FFS. How the hell is this country supposed to get any better if even the Equality Commission is forcing us all to be one side or the other?

(I ticked the neither box and told the guy handling it that if he had to apply the residual method then I'd provide him some suitable information to support whichever side was most convenient. It's not his fault some silly buggers want to make me a Protestant or Catholic atheist, or imply I only hang round with one side or the other.)

Erich Schubert: Xorg evdev hotplugging anyone?

Xorg 1.4 in experimental is supposed to have input device hotplugging.Does anyone have a Howto for Debian? I tried it, but I couldn't get it to hot-plug my USB mouse, so I'm back to using the regular mouse driver for it again, using the /dev/input/mice in-kernel-hack for hotplugging.P.S. on a recent kernel, you might want to add
blacklist snd_pcsp
to a custom file in /etc/modutils/, in order to avoid your PC speaker showing up as regular audio device. You don't want your regular apps to consider your legacy PC speaker as audio device usually.P.S. No, my blog doesn't have comments. Just send me an email (you know, 'legacy' email) via erich AT debian org.

Kartik Mistry: foss.in/2008 event announcement


* If you have not read event announcement yet, Please go through this message. I have just applied (and granted) for leave for event. C ya at foss.in/2008!

Russell Coker: Swapping to a Floppy Disk

In the mid 90’s I was part-owner of a small ISP. We had given out Trumpet Winsock [1] to a large number of customers and couldn’t convert them to anything else. Unfortunately a new release of the Linux kernel (from memory I think it was 2.0) happened to not work with Trumpet Winsock. Not wanting to stick to the old kernel I decided to install a Linux machine running a 1.2.x kernel for the sole purpose of proxying connections for the Winsock users. I had a 386 machine with 8M of RAM that was suitable for the purpose. At that time hard disks were moderately expensive, and the servers were stored in a hot place which tended to make drives die more rapidly than they might otherwise. So I didn’t want to use a hard disk for that purpose. I configured the machine to boot from a floppy disk (CD-ROM drives also weren’t cheap then) and use an NFS root filesystem. The problem was that it needed slightly more than 8M of RAM and swapping to NFS was not supported. My solution was to mount the floppy disk read-write and use a swap file on the floppy. The performance difference between floppy disks and hard disks was probably about a factor of 10 or 20 - but they were both glacially slow when compared to main memory. After running for about half an hour the machine achieved a state where about 400K of unused data was paged out and the floppy drive would then hardly ever be used. I had initially expected that the floppy disk would get a lot of use and wear out, I had prepared a few spare disks so that they could be swapped in case of read errors. But in about a year of service I don’t recall having a bad sector on a floppy (I replaced the floppy whenever I upgraded the kernel or rebooted for any other reason as a routine precaution). Does anyone have an anecdote to beat that?

Next.

Previous.