Search Results: "tar"

7 December 2006

John Goerzen: Saving Power with CPU Frequency Scaling

Yesterday I wrote about the climate crisis. Today, let's start doing something about it.

Electricity, especially in the United States and China, turns out to be a pretty dirty energy source. Most of our electricity is generated using coal, which despite promises of "clean coal" to come, burns dirty. Not only does it contribute to global warming, but it also has been shown to have an adverse impact on health.

So let's start simple: reduce the amount of electricity our computers consume. Even for an individual person, this can add up to quite a bit of energy (and money) savings in a year. When you think about multiplying this over companies, server rooms, etc., it adds up fast. This works on desktops, servers, laptops, whatever.

The easiest way to save power is with CPU frequency scaling. This is a technology that lets you adjust how fast a running CPU runs, while it's running. When CPUs run at slower speeds, they consume less power. Most CPUs are set to their maximum speed all the time, even when the system isn't using them. Linux has support for keeping the CPU at maximum speed unless it is idle. By turning on this feature, we can save power at virtually no cost to performance. The Linux feature to handle CPU frequency scaling is called cpufreq.

Set up modules

Let's start by checking to see whether cpufreq support is already enabled in your kernel. These commands will need to be run as root.

# cd /sys/devices/system/cpu/cpu0
# ls -l


If you see an entry called cpufreq, you are good and can skip to the governor selection below.

If not, you'll need to load cpufreq support into your kernel. Let's get a list of available drivers:

# ls /lib/modules/ uname -r /kernel/arch/*/kernel/cpu/cpufreq

Now it's guess time. It doesn't really hurt if you guess wrong; you'll just get a harmless error message. One hint, though: try acpi-cpufreq last; it's the option of last resort.

On my system, I see:

acpi-cpufreq.ko     longrun.ko      powernow-k8.ko         speedstep-smi.ko
cpufreq-nforce2.ko  p4-clockmod.ko  speedstep-centrino.ko
gx-suspmod.ko       powernow-k6.ko  speedstep-ich.ko
longhaul.ko         powernow-k7.ko  speedstep-lib.ko


For each guess, you'll run modprobe with the driver name. I have an Athlon64, which is a K8 machine, so I run:

#modprobe powernow-k8

Note that you leave off the ".ko" bit. If you don't get any error message, it worked.

Once you find a working module, edit /etc/modules and add the module name there (again without the ".ko") so it will be loaded for you on boot.

Governor Selection

Next, we need to load the driver that tells the kernel what governor to use. The governor is the thing that monitors the system and adjusts the speed accordingly.

I'm going to suggest the ondemand governor. This governor keeps the system's speed at maximum unless it is pretty sure that the system is idle. So this will be the one that will let you save power with the least performance impact.

Let's load the module now:

# modprobe cpufreq_ondemand

You should also edit /etc/modules and add a line that says simply cpufreq_ondemand to the end of the file so that the ondemand governor loads at next boot.

Turning It On

Now, back under /sys/devices/system/cpu/cpu0, you should see a cpufreq directory. cd into it.

To turn on the ondemand governor, run this:

# echo echo ondemand > scaling_governor

That's it, your governor is enabled. You can see what it's doing like this:

# cat cpuinfo_min_freq
800000
# cat cpuinfo_max_freq
2200000
# cat cpuinfo_cur_freq
800000


That shows that my CPU can go as low as 800MHz, as high as 2.2GHz, and that at the present moment, it's running at 800MHz presently.

Now, check your scaling governor settings:

# cat scaling_min_freq
800000
# cat scaling_max_freq
800000


This is showing that the system is constraining the governor to only ever operate on an 800MHz to 800MHz range. That's not what I want; I want it to scale over the entire range of the CPU. Since my cpuinfo_max_freq was 2200000, I want to write that out to scaling_max_freq as well:

echo 2200000 > scaling_max_freq

Making This The Default

The last step is to make this happen on each boot. Open up your /etc/sysfs.conf file. If you don't have one, you will want to run a command such as apt-get install sysfsutils (or the appropriate one for your distribution).

Add a line like this:

devices/system/cpu/cpu0/cpufreq/scaling_governor = ondemand
devices/system/cpu/cpu0/cpufreq/scaling_max_freq = 2200000


Remember to replace the 2200000 with your own cpu_max_freq value.

IMPORTANT NOTE: If you have a dual-core CPU, or more than one CPU, you'll need to add a line for each CPU. For instance:

devices/system/cpu/cpu1/cpufreq/scaling_governor = ondemand
devices/system/cpu/cpu1/cpufreq/scaling_max_freq = 2200000


You can see what all CPU devices you have with ls /sys/devices/system/cpu.

Now, save this file, and you'll have CPU frequency scaling saving you money, and helping the environment, every time you boot. And with the ondemand governor, chances are you'll never notice any performance loss.

This article showed you how to save power using CPU frequency scaling on Linux. I have no idea if it's possible to do the same on Windows, Mac, or the various BSDs, but it would be great if someone would leave comments with links to resources for doing that if so.

Updated: added scaling_max_freq info

Lucas Nussbaum: Resurrecting Debian package a day , part two.

After this blog entry, I thought for a while, and decided to take over Debian package a day. I’m well aware that it needs a lot of time, so I’ll try to make this a collaborative work: The publishing rate should depend on the number of ready-to-go entries in the queue. Starting with publishing twice a week should be okay. I have already started to set up a wordpress blog (using drafts to prepare entries + mails to coordinate should be enough for now ; we can always switch to something more complex later). TODO List :
  1. Get control over the debaday.debian.net domain. I couldn’t do it directly since DAMs seem to want to wait until Christmas to deliver my Debian account, so buxy did it for me. Should resolve soon.
  2. Get added to Planet Debian and Planet Ubuntu. If the posting rate stays low, it shouldn’t be a problem, and we can always change this later.
  3. Write the initial pages explaining how to contribute, etc.
  4. Get the first entry online.
  5. Recruit editors to share the load.
  6. Wait for you to submit entries :-)
Update (08/12/06) : Debian Package of the Day won’t be added on Planet Debian: a recent discussion concluded that non-personal blogs should not be on Planet Debian. So you have to subscribe directly to its feed (if you read Planet Ubuntu, you probably don’t need to, since it was added there).

Mario Iseli: Spam in my favourite Linux magazine

Hmmm, good morning. The “Linux Magazin” is more or less my favourite IT magazine. I like it with the following reasons: But today I was really unhappy, because there was an article which really was not neutral to the free software community. What about I’m writing? I talk from Microsoft which has a A4-page advertisement in this free software magazine and there’s written that Windows Servers are much more stable and due to this they are used in the “London stock exchange”. Sorry, is this fair? This magazine is about _FREE_ software and not only about IT-Advertisements… I’m interested how much they paied for this article. :-(

6 December 2006

Mario Iseli: Planning my holidays, stop smoking and other projects

Hello :-) (first of all, there are new pictures of me)
I think it’s time for another blog, because I have at the moment many different projects. My prio 1 project is the new proxy server which I would like to install in the office (about 500 users) which should have complicated ACLs and SquidGuard for content filtering, a virus scanner would also be very nice. I plan to authenticate the users via winbind on Microsoft ActiveDirectory (I already blogged about it some months ago) and the embedded systems should have special rules which don’t require an authentication. I also had a look at dansguardian because the people from the AIO use this solution, but I don’t think that it’s possible to auth against AD with it and I have no experience with dansguardian. The next problem is with the downtime, I should not have any if possible. This means that I will copy the system of the old proxy into a VMWare-Machine in the intranet and NAT the public IP of the proxy to it when it’s ready. And in this time I can easily setup the new proxy on the physical proxy-machine. Then my next project is for the school - we have to write a management tool (including file-databases) for network devices such as switches or routers, I think this will be a quite funny task and 2 months are IMHO enough time. The cool thing is that we need something like that on my workplace so we will be able to use it. My program also will be licensed under a free license and maybe I will build a debian package if the upstream software is in a good shape. Then the next (maybe less important, but important for my health) project is finally stop smoking. I really didn’t think that this is so difficult because I’m not a “i need it every 2 hours”-smoker, I only need it in the evening when I come home to relax a bit and this is really quite difficult. Then my last (really hard ;-) ) project is planning my holidays, I didn’t use them since summer of this year and until next august I can have 5 weeks + the days from christmas until 3rd January, so my plan is the following: 15.12.2006 Last workday 26.12 - 30.12 23c3 in Berlin 31.12 - 1.12 Snowboarding in the mountains and sylvester party with friends Then in february 2007 I will go again snowboarding for a week somewhere And in june 2007 my last holidays for this year where I’d like to visit Debconf7 (depends if I get sponsoring for accomodation and food or not, you know… I’m still a student and edinburgh isn’t really cheap). I hope this will be my last article without the hackergotchi included, I’m not a graphics-guru so I asked in #debian.de some minutes ago and Markus Gloor made one for me, biggest thanks!

Jonathan McDowell: Making DVDs hurts

I bought a Sony DCR-HC24E for the purposes of recording my wedding ceremony. I could have borrowed my parents', but it's an ancient analogue thing and although I don't really have a use for a camcorder I talked it through with Katherine (making it clear that part of the appeal was just sheer toy value) and we went ahead with it. (I should point out the driver for recording any of the wedding came from my mother-in-law and I wasn't that fussed about the whole thing, but having experienced the "day was a blur" thing I'm glad we have it. Though I've watched it too many times as I'm about to explain.) The Sony comes with both USB and Firewire. My desktop doesn't do Firewire, but I figured USB would be fine. First issue is that Linux doesn't seem to support it in USB mode. I don't know if this is easily solved, as reading the manual I discovered that Sony claim USB isn't as good quality and Firewire is better. Huh? It's all digital. What gives? The best I could come up with is that Firewire is better at guaranteeing latency/bandwidth so you don't end up with dropped frames. Anyway. I buy a Firewire cable and dig out a Via EPIA-M board which has Firewire. Some poking about leads me to the conclusion that I want dvgrab. I fire it up, it seems to do the right thing, I end up with a huge file. Point mplayer at it and discover it's dropping frames. It appears raw DV is about 30Mb/s. The HD in the Via is an old 6GB one that only does ATA-33. So I find a newer ATA-133 capable one, double the RAM in the box (on Adam's recommendation) and try again. This goes better. So now I have a bunch of AVI files, all of them about 1G in size. avimerge is employed to combine them (I might have been able to just get dvgrab to output a big file, but I was wary of large file support). Then I did ffmpeg -i bigfile.avi -target dvd smallerfile.mpg resulting in a couple of DVD format MPEGs (one for the ceremony, one for the speeches). My first attempt at a DVD image was with DeVeDe. It seemed to work fine, but I ended up with something that the DVD player didn't like and neither did mplayer or xine. So I fell back to qdvdauthor, which I've used in the past but don't deem the most user friendly of programs. I finally end up with something that mplayer and xine seem happy with in terms of a menu and a pretty picture of us signing the register and working video. And so to burning a DVD. I used growisofs for this. Firstly with a DVD-R, which mostly worked but had annoying pauses on my real DVD player. So I tried with a DVD+R and this seems to be much better, however there are some minor pauses. This might be my ancient DVD player not coping well I guess. I'm sure DVD creation shouldn't be this painful. I bet if I ran Windows I could point and drool my way to a DVD with the software Sony supply with the camcorder. All I wanted was a basic menu with a background picture and some video attached to the menu options. I'm pondering doing a version with a picture gallery as well, but can I be bothered with the hassle?

Gustavo Franco: FLOSS in Brazil turns out to be a joke

Well, forget anything you've heard about how well developed is Brazil in terms of FLOSS. The gov't loves FLOSS, the NGOs and even a lot of companies too, right? Wrong!

With just a few exceptions (and i think i work in one of them and i'm glad), the FLOSS hype in Brazil is a marketing thing. Even the most proprietary software defender needs to agree that FLOSS and every aspect of it, has appeal where there's not enough money or where the money isn't being distributed in a fair manner.

The point is that a lot of gov't institutions, NGOs and companies are using the FLOSS appeal and its arguments with the masses to do what a brazilian does best (after playing soccer): Corruption. The recipe is simple: Tell as much people as possible that you're developing the FLOSS 'foo' and ask for funding - even companies do that here and when they don't, they join a gov't institution. You know, when you haven't compromises with results and the law and have contacts it isn't that hard get some money here and there for your 'foo' project.

Many times nobody out of the project office (when there's a real one) ever see the source code of that and in a lot of times it will be delayed forever or it will suck - but when it sucks they blame the 'community' (they don't know the difference between user and developer community). Wait, but where's all that money? The money is in the same places where part of the 3 trillions of dollars (no, i'm not wrong) that were used by less developed countries during the past years from the north-american and european banks. Go figure...

My hint for these FLOSS advocates in Brazil and those with some money for funding projects here: Hire somebody that have a vague idea what a user community, developer community, software licenses and source code are in the next time.

MJ Ray: Gowers Review

At first glance, the Gowers report could be very good, especially the EUCD (European Copyright Directive, also known as European-DMCA, or "that TPM/DRM crap") changes for 'orphan works' and the fence-sitting on punishment adjustments, but could also be bad with increased numbers of state copyright enforcers and increased use of the confused "IP" jargon (then again, UK Patent Office handles Copyright at the moment, which is confused anyway).
History
Discussion

Ross Burton: Managing an APT Repository via WebDAV

At work we've just bought online another server, so we have a true public/private server split. The private server is only accessible via secure methods, and the public server only has a few users to reduce the possibility of people brute-forcing their way in. However, the public server holds the APT repositories for debian.o-hand.com and maemo.o-hand.com. Before the split this was not a problem, after building the package dput was used with the scp method, and then mini-dinstall was run as the user to update the repository. Now that there are minimal accounts on the new server, we can't scp the packages to the server, or ssh in to run mini-dinstall. Running m-d in daemon mode solves the latter, but there is still the problem of getting the files on the server in the first place. Enter WebDAV. The public server already has a number of WebDAV shares (secure client shares, Subversion, and so on) so it was trivial to export the mini-dinstall incoming queues over WebDAV. With a custom init script to start mini-dinstall in daemon mode, anyone with the right permissions can upload files via WebDAV. One option here is to use Nautilus, but I like the safety of dput. A day of hacking, with lots of screaming at Python's urllib2, produced this patch to dput that adds HTTP uploading. I've been using it for a few days now and it seems to be working well, so hopefully it will be merged into Sid soon. NP: Xmas in Frisko, soma.fm

Adrian von Bidder: We Know Who Takes the Blame!

Sam Hocevar is wrong when he says:
Suppose for instance that you stopped contributing to Debian months ago [...] You are obviously not only directly responsible for Etch being late due to your lack of involvement, ...
“You” are not responsible for Etch being late. The Debian project has picked an official scapegoat for Etch being late and for everything else in Debian that just doesn't work how it should. So everybody can now relax: no Debian developer is personally responsible for anything. It's really convenient if some people group together and do something that other people don't like and then give it a name. You can blame the new thing for everything you don't want to do or don't like, without actually having to name the people behind it or thinking about constructive ways to improve matters. It's always “them”. Yes, I'm pissed by the attitude of some very vocal people. Dunc Tank is not an official Debian project, so I still fail to see how it's different from Canonical, Progeny, Credativ etc. hiring people. Dunc Tank even pays people to work directly on Debian, it doesn't pay them to do Debian-related work that might eventually benefit Debian, too. As a private employer, Dunc Tank is free to pick who to employ and who not to employ. Yes, Dunc Tank and some Debian project roles do overlap, but stuff like this can't be avoided, especially in time consuming volunteer projects doing stuff that can also be done for money. Debian is not a charity, but a project competing directly with commercial products, so its members shouldn't be surprised when the question of money comes up every now and then. (And just to make it clear: I'm not arguing against Sam, who blogged about a totally different topic. The part I quoted just got me started.)

Joerg Jaspert: Hardware for you

As I just wrote to the debian-project lists I have a few machines to give away to Debian people. The machine specs start with some older 486 DX2/66 and go up to Dual Xeons with 3GHz, so contains a good number of interesting systems. Thats all hardware that got sorted out, after we migrated the servers to a complete new set of machines. Lets see where it gets a new use. Update: Please notice that the list contains more than those 4 Dual Xeon machines. Maybe note an alternative P4 you could live with in case other people did love the Xeons too. :)

Daniel Burrows: Things I miss about being an Undergrad.

So, it's been about 4.5 years since I got my ScB, 1.5 years since I got my MSc, and 1 year since I started working at Satori. Seems like an ideal time to look back on the halcyon simplicity of college, right? Here are a few things that I really wish I had appreciated more in 2000.

(1) Getting enough sleep.

I was just about the only well-rested student I knew in college. When other people (who stayed up past midnight regularly) would complain that they seemed to spend lots of time working withotu getting anything done, I occasionally suggested that they try getting more sleep. Hah.

So, in grad school I developed some bad, bad habits -- staying up past midnight, sleeping in erratically, etc. (I like to blame my girlfriend, who refused to get up before 9-10am when she was a student ;-) ) I started to sort of get a handle on that, and then I got a full-time job.

The problem here is this: you know how they say that to figure out how much sleep you need, you should sleep until you wake up naturally and feel rested? If I go to bed at 9PM, I wake up naturally at 7-8AM or so. That's -- count it -- 10-11 HOURS OF SLEEP. As a student, I had the luxury of sleeping 11 hours straight (since I had no social life). As a working drone with an hour-plus commute, that would leave me with 3-4 hours to fit in all my non-work activities (like, say, eating; personal hygeine; etc). The fact that I tend to get heartburn if I lie down within 2-3 hours of a meal, and that I usually have to cook my meal before I can eat it ... well, this just isn't gonna work.

So, to get enough time to actually live a bit outside work, I end up curtailing my sleep schedule to "only" about 7-8 hours. I know that's enough for some people. I wish I was one of them. The result for me, though, is that I go through most days feeling dazed, tired, sleepy, and with an on-and-off headache. I can generally hold it together well enough to get through the work day (although yesterday I collapsed on the couch halfway through), but once I get home I lack the energy to do anything but feed myself and idly browse the Web. Trying to do anything intellectually challenging (hence interesting), like say learn Cantonese or category theory, is right out. (I can just about say I want to eat, and I got as far as power categories before getting horribly tired and confused)


(2) Breaks.

One great feature of college is that you get time off of it. In addition to some random vacations during the semester, you usually break for several whole weeks between semesters, and then break again in the summer. Even if you have a summer job, this is a great way to recharge the batteries.

In the working world, I get exactly 1 day off per month -- if I want a decent-sized break, I have to go several months with no days off at all (except the paid holiday day every other month or so that the company gives us in its beneficience).


(3) The cafeteria.

College students like to make fun of cafeteria food. They don't know how good they have it. You get three hot meals a day, plus you don't have to do ANY cooking or cleaning up? Sign me up!

When I get home at 6:30 PM and I'm starving, the first thing I have to do is get out the raw ingredients for my dinner and toss them in a pot, then gnaw on some cheese or something to take away the hunger pains until they're done. By 7:30, I might have some food in me, hopefully. I'm thinking about switching to a frozen-food diet (ew), although that doesn't help clean the dishes.


(4) There's no commute in college.

To get to work, I have to walk 10 minutes, ride the bus for about 30-40 minutes, then walk another 20 minutes or so. In the evening I have to do the reverse. That's 2-3 hours of my life sucked up by just getting to work and back. If my brain wasn't too fried to study any of the interesting books I've optimistically purchased (see (1) and (2)), it might be bearable...but when I just want to get some food and rest, it really sucks. And it sucks especially when I have no good alternative (living in Seattle is EXPENSIVE, and it would give Kate an even worse commute in the other direction).

(5) Having the time and energy to study topics or write software just for the heck of it.

This really incorporates all of the above. It seems like all these factors conspire to suck my time up, then suck up the energy I have in the little time left to me. Meh. For instance, right now (at 9PM) I am getting very sleepy and should get to bed, yet I've done nothing since I finished eating (at 7:30PM) but skim Lambda the Ultimate and a few linked articles sites (slowly) and write this blog post. Blah.




Lest I give the impression that I completely hate work, let me add that there is one very, very important thing that I enjoy about it relative to my student, free-software-hacking days.

(1) NOT BEING FLAT BROKE.

I believe this one is self-explanatory; and so far it still makes up for the rest (I wonder how long this will continue). It's too bad I can't get paid to just sit around doing whatever I feel like and then giving it away, but then it's also too bad I can't wave a magic wand to get a pony and a few dozen acres to keep it on.

5 December 2006

Evan Prodromou: 14 Frimaire CCXV

Christmas is coming soon, and we'll be off to California again at the end of the month to be with family and friends for the holidays. My immediate family sends out gift suggestion emails at the beginning of December, and I'm always the last one to send mine out. So I'm getting it done today. Probably the most consumer-y thing that I want for X-mas is a portable digital media player that can run RockBox. I really want to get more into podcasts and videocasts and that sort of thing, but I'd like to have a portable player for the stuff. Chaining myself to an iPod is just too unpleasant. Any suggestions? tags:
 

Hampr My friend Ryan Finnie, created the excellent one-disk recovery distro Finnix. He also just started a new bookmarking service called Hampr, complete with its own Hampr FireFox extension. One thing I like about the service is that it uses OpenID, and only OpenID, for login and user profile services. It's the latest on the directory of OpenID-enabled sites. A cool idea, and it works well for me. tags:

The new look OK, it's the old new look, but this article on current style in Web design is interesting and informative. Good job. tags:

Ross Burton: Postr 0.2

Postr Postr 0.2 is finally out. I didn't get around to finished the all new Flickr API, but instead there are a lot of bug fixes. You can follow the development in the Bazaar branch, or get the Postr 0.2 tarball.

Benjamin Mako Hill: Convergence

I recently found out that Simon and Schuster (and some other publishers) have been publishing books under an MTV imprint. In a way that is slightly reminiscent of Street Fighter: The Movie: The Game, MTV has given their project the obvious name: MTV Books. That is to say, Music ... Television ... Books. Of course, MTV Books is hardly constrained to traditional book media and been proactive in releasing MTV Books eBooks (including the intriguingly titled Oh My Goth! and Life as a Poser). Additionally, they have published an MTV Photobooth -- a Music Television photography book. While they've published Sing Like the Stars! (quite a few of their titles are exclamations), a paperback accompanied by an audio CD, they do not seem to have published an audio book. I am looking forward to the day when they do so and finish the process of coming full circle.

Axel Beckert: Gaia resurrection

On Thursday, 24th of November 2006 evening (about one and a half week ago) we got an anonymous Symlink submission about gaia 0.1.0, a simple, but free (GPL‘ed) client for Google Earth, solely based on reverse engineering. Liked the idea of a really free client for such a popular service, so I posted it as Symlink article (German) quite shortly after noticing the submisson. Since it’s under GPL and uses only libraries already being in Debian (OpenGL, SCons, cURL, SDL, libjpeg, libpng, libgps and Doxygen), I really would like to see it in Debian. But since I have never programmed with most of them, I did not try to compile gaia but instead took the easy way for myself and filed an RFP bug for it in Debian. The next day in office, Gürkan, a coworker of mine and currently in NM, told me that he tries to package gaia for Debian and that it doesn’t compile on Sarge. So I gave him access to my Sid chroot and asked him if he has read about the RFP at debian-devel or debian-wnpp. To my surprise he answered that he didn’t read about it on Debian lists at all but on Symlink and therefore didn’t knew about the RFP at all. So I pointed him to the RFP just to notice, that it wasn’t an RFP anymore but already an ITP. Strangely I didn’t get any mail notice about this change although the retitling happend only shortly after the initial RFP… Of couse, Gürkan wasn’t very happy about this, since he planned to maintain gaia himself, but at least he provided his working proof of concept package for Sid to Jordà Polo who intends to maintain an official Debian package of gaia. We then also met Jordà on the #debian-games channel in OFTC. He told us, that he’ll check some more licensing issues before uploading a package of gaia. Well, since I had a working gaia package installed in my Sid chroot, you can probably imagine what I did half the evening? Right: Surfing around the earth with gaia and visiting my favourite places on earth. While virtually visiting places here and there, I listened to Venty’s first podcasts and seem to have downloaded 444 MB of Google Earth images (at least according to du -sh ~/.gaia). Goddamned addiction! ;-) During the night from Friday to Saturday, I got a mail that the RFP/ITP has been closed and that Jordà was right with his suspiciousness: Google has sent a cease and desist letter to the gaia developers and they removed the downloads from their site. (So how was that Google motto? “Don’t be evil”? Well, good joke! Yet again the monopolist clearly shows that it doesn’t really mean what it says.) Since the gaia developers were allowed to post the mail from Google on their website, you can read parts of it there. I really wonder what was written in the left out parts of the mails. Job offers? ;-) But it’s interesting to see that until one week ago gaia 0.1.0 already made it into the FreeBSD ports as well as into the ArchLinux User Repository. Both noticed, too, that gaia version 0.1.0 has been withdrawn by the authors. Today in the morning Gürkan noticed that there’s a new version (0.1.1) of gaia at SourceForge using free NASA WorldWind / OnEarth imagery. Although the imagery is far away from the detailedness of the Google Earth imagery, this has one big advantage: There wasn’t really a X port for the windows-only client from NASA until now. (No real wonder, since they use proprietary and operating specific libraries such as .NET and DirectX…) And now we have gaia, a free client for free imagery on free operating systems. That made my day! I hope that this will revive the packaging of gaia, at least Gürkan has already built a new proof of concept package of 0.1.1 for Sid. (Today in the afternoon, they released 0.1.2.) And if everything goes fine and Dunc-Bank manages to delay Etch until gaia has been 10 days in Unstable without bugs, we’ll have it even in Etch. ;-)

John Goerzen: Review of VPSLink

Back in June, I wrote about my switch to VPSLink for my virtual private server (VPS) host.

Now it's 6 months later, my initial contract is up, and it's time to consider whether to renew it.

Overall, VPSLink has worked out reasonably well. I have their Link-4 plan, which provides 512MB RAM, 20GB of disk storage, and 500GB of bandwidth for $40/mo (or down to $33/mo if you pay for 12 months in advance).

Reliability and Uptime

VPSLink has been reasonably reliable. I wouldn't say that they have been exceptionally reliable, though.

Some outages:



They have generally responded to support requests in a reasonable amount of time, and support has been approximately as helpful as I'd expect.

Overall, there've been a few issues but, aside from that 8-hour outage in July, nothing especially remarkable.

Automated Tools

VPSLink has a "control panel" where you can reboot, start, or stop your virtual machine. You can see how much bandwidth you've used in a month, your IPs, and billing information.

You can also create support requests and view the status of support requests there. You can also view all correspondence on a given ticket, and add new correspondence to it -- a nice touch that's useful if your email is down because your VPS is down.

The reboot/start/stop facility didn't work during one outage, though.

Resources

The 20GB of disk space is nice to have, though I never used it all. No complaints there.

But the memory setup is rather strange. This may be because VPSLink is using OpenVZ instead of its more popular (and featureful) commercial counterpart Virtuozzo, or something such as Xen.

Those that have used OpenVZ/Virtuozzo know there is a /proc/user_beancounters file in the virtual environment that reports on the limits of resources allocated to your virtual server. VZ lets the server admins regulate the amount of resident RAM, swap, inodes, pending IP connections, amount of virtual RAM, etc, etc. About 3 dozen items in total.

Unlike some companies like JohnCompanies, VPSLink was not willing to make any adjustments to this file for me. For instance, my mail server may need more simultaneous open files than they have by default, but they wouldn't work with me to make reasonable adjustments.

But the larger problem was with regard to RAM. This was particularly annoying. VPSLink does not permit any access to swap, so the 512MB is what you get. But tools like top show the entire memory allocation on the host, and are mostly useless in tracking down your own usage.

You can look at the privvmpages entry in /proc/user_beancounters to get an idea of your current usage, but that's it. There is a script on their wiki that will turn this into a more useful number, but again, it's not the most helpful.

The minute you try to allocate anything past 512MB, your processes start getting killed. According to /proc/user_beancounters, even though my system normally hovers around 70% of that 512MB allocated, I've had 290,567 instances where a process has been refused its request to allocate memory since the last reboot. Such instances usually cause a process to crash.

Other Virtuozzo companies like JohnCompanies will give you "burstable RAM", so if others aren't using their full RAM allocation at the moment that your machine goes over your regular limit, you can go over it to a certain extent. That was very effective at preventing crashes during times of higher than usual memory usage, and I never had trouble with this at JohnCompanies.

I've had a lot of trouble with VPSLink because of these hard, no-exceptions limits. Getting hit by the Google bot at the same time as the MSN bot could raise memory usage enough that processes would start getting killed. Depending on which process it is, it could take down cron, apache, the mail server, whatever. And there is no log of which processes die because of this.

In normal circumstances, this wouldn't pose even a noticeable impact to a server; inactive bits could be swapped out, etc.

I am also suspicious of exactly how OpenVZ calculates memory usage. Since moving to a physical Linux box, my memory usage seems to be much lower than the privvmpages would seem to indicate, even though it's running the exact same code.

So I had some outages on my server that were caused by OpenVZ memory limits, which didn't get listed in the overall outages above.

This problem is the #1 reason I'm leaving VPSLink.

Performance

Overall performance has been acceptable. The CPU speed appears to be reasonable and about as expected.

The disk performance has been more problematic, however. It tends to be rather slow. There have been times when I can type "ls" in a directory with about 10 files and it takes 5-10 seconds to respond. And I wasn't even using "-l".

Now that sort of thing is certainly the exception. But it makes databases -- and websites that rely on databases -- very slow.

The performance measurements over at RealMetrics also show VPSLink as being at the bottom of the pack for disk performance.

Overall

VPSLink is a reasonable value, and if monthly cost is important to you, probably a good choice. Don't expect it to be spotless, though. I would call the overall experience pretty average. Nothing too spectacular in either direction in any respect.

Brett Parker: And here goes the update...

Fantastic beginning to the weekend - was in Norwich from Thursday, saw the Dave and the Becca on thursday night - many many thanks to them for floor space thurs and fri and for generally just being fantastic people (with tea *and* coffee, no less - and a silly beepy tivo box for even more fun - and for introducing me to the american chopper type show thingummy - and for letting me watch the a-team!) Then, serious amounts of thanks to Noodles and Kathy (who I appear to not be able to find a link for :( ) for inviting me to the wedding and the reception - was fantastic - you're both such absolutely lovely people! Pictures so far appear to be available from Becca and Dave, Burly and Steve McIntyre - hopefully more soon... specially from Dave Noble (can't find him *bah* he needs more google juice the damned slacker ;) which should be cool! Saturday was absolutely brilliant - including talking about atmosperic physics, failing a-levels, and random universities doing questions (hey, I can't entirely remembere - someone bought a half barrel of nero and some port, and lots of red wine... I wasn't *entirely* sober ;). Saturday morning was a bit of a blur - I remember watching the A-Team and then food in the Glass House with Dave and Becca (thanks guys, you rock!) then I wandered to the Fat Cat and met with Eli and Dave Noble, a while later Rob and James and Kaz arrived and we played Munchkin, which was a very very silly game but quite fun (thanks James and Kaz - you're both mildly insane - but it's always fun when you come to the pub!). Went home with Eli, tried to escape because of the annoying giggly evil students from hell, started wandering back towards the train station and a random bench to sleep on, then got emotional black mail and ended up back at Eli's - ho hum. Went to catch train at around 10am Sunday morning - discovered trains from Norwich weren't running - replacement bus service from Norwich to Diss left Norwich at around 10:25am, got to Diss, had a 10 to 15 minute wait there (mmmmm, bacon roll and a coffee!) then straight to Liverpool Street, wandered straight off to the underground and got the circle line to Victoria, got to Victoria around 13:40 ish, wandered to look at the boards and saw a 14:17 from Victoria round half the planet to Brighton - thought to myself "not in any rush" and wandered off and found food and then wandered round the WHSmith in Victoria station.... got out of there, glanced at the board again, Victoria to Brighton train had been cancelled - wondered what to do - announcement over the tannoy that to get to Brighton the best idea was to travel to East Croyden and the train would meet us there. So, grabbed that train, got to East Croyden, jumped on the train (first 8 carriages Brighton, back four Plymouth... so, jumped on somewhere around the 7th carriage from what I thought was the front - turned out to be the back - wasn't entirely sure what was going on so when it stopped at Gatwick wandered further to the front of the train a lot, found an empty pair of seats and sat down. Got to Three Bridges and there was a delay, it was announced on the train as "We're waiting for the conductor, he's missed a train and is currently in a taxi, there's at least a 10 minute delay before he gets here", so a load of us bundled out of the train on the basis that that was very much long enough for a cigarette break - just as we were finishing the conductor turned up, so we all got back on the train for the "scenic" route to Brighton (yay for going round the coast), finally got in to Brighton at 17:05, so wasn't too bad, only about 6 and a half hours for a 3 and a half or 4 hour train journey ;). Grabbed chinese on the way home, watched some Morse - all was well with the world. Lots of beeping this morning around 8.30am, ignored it, finally re-woke up at about 10:30am (ish) - watched some more Morse, grabbed some lunch and then watched more Morse until going to the pub quiz tonight. Charly, Julius and Pieter turned up for a quick drink and then all left me to fend for my self at the quiz - I came in my usual place - very last (by over 20 points, no less ;) - but had fun, so who cares :) Anyways - it's now tuesday morning, so probably time for some good ol' fashioned sleep type stuff... might put on the next episode of Morse for the hell of it (might even manage to stay awake through it!). Tomorrow is probably more Morse, some work on the django packages (just got the first bug report in for those) and possibly applying some of the patches to the scrobbler packages to close some of those bug reports too. It appears that upstream for xmms-scrobbler has disappeared a bit, so might have to look at that a lot closer over the next few months - have some time off work soon - last day this year being the 15th (also happens to be when work are planning the "christmas do"). Wow - that was a ridiculously longer post than I though it was going to be! Night all!

4 December 2006

Matthew Palmer: Automatic Ordering Systems

Reading about Jamie Zawinski's problems with purchasing plumbing parts (look for 'side-outlet elbows' and read until 'Construction'), I started wondering about the automatic ordering system that shop used. To compensate for the obvious stupidity in the staff, I wondered whether their ordering computer might have a system of automatic adjustments for stock re-ordering quantities. Then I wondered what the thresholds might be to increase the per-restock quantity. Finally, I entertained myself with the idea that, just maybe, jwz's rampant purchases triggered the stock computer to put in an order for some huge number of 4-way pipe joints, which then sat in stock for years and years, because the average consumption rate for those pieces is probably a hell of a lot less than the 10-per-week that the computer saw over the month in question.

Erich Schubert: Software mixer with ALSA

The "high definition audio" in my laptop is rather minimal. Apart from the mic not yet working, it has a line out and a single volume control. Thats it. However, I'd like to have multiple applications use the sound card, for example my music player, eventually some video player or flash, and some system events. ALSA to the rescue. It's not easy to configure, but really powerful: Mixer screenshot with MPD volume control Yes, that is a separate volume slider in my regular mixer for MPD, my music player. Here's how to do that: First we'll need to create a huge /etc/asound.conf:
# the real device
pcm.realdev  
        type hw
        card 0
        device 0
 
# software mixing
pcm.dmixer  
        type dmix
        ipc_key 3129397
        ipc_key_add_uid false
        ipc_perm 0666
        slave.pcm "realdev"
        slave  
                period_time 0
                period_size 1024
                buffer_size 8192
                # we're enforcing 44100, since thats what most apps use
                rate 44100
         
        bindings  
                0 0
                1 1
         
 
# software volume
pcm.softvol  
        type softvol
        slave.pcm "dmixer"
        control  
                name "Software"
                card 0
         
 
pcm.mpdvol  
        type softvol
        slave.pcm "dmixer"
        control  
                name "MPD"
                card 0
         
 
# input
pcm.input  
        type dsnoop
        ipc_key 3129398
        ipc_key_add_uid false
        ipc_perm 0660
        slave.pcm "realdev"
 
# duplex device
pcm.duplex  
        type asym
        playback.pcm "softvol"
        capture.pcm "input"
 
# default devices
pcm.!default  
        type plug
        slave.pcm "duplex"
 
pcm.dsp0  
        type plug
        slave.pcm "duplex"
 
I won't go into all the details (I don't know all of them either), but the basic idea is that I setup the dmixer plugin for software mixing, and attach two software volume controls to it. I'm keeping the permissions to 0666, since my MPD is running as a different user, and I didn't find a way to say I'd like to have the permission set to group "audio". The default device is the regular microphone jack (untested, since there are driver issues with my chipset and recording) and the "Software" volume control. If you have an ~/.asoundrc you might want to move it out of the way. Next I need to configure MPD to use its own volume control via /etc/mpd.conf:
audio_output  
        type            "alsa"
        name            "my ALSA device"
        # 'mpdvol' is the device name I used in asound.conf
        device          "mpdvol"
 
You'll probably need to restart all applications currently accessing your audio devices to have them pick up the new config. The new volume control will also only appear after you've started an application using it AFAICT. Sometimes you need to restart them twice for the new volume control to be effective. When alsa-utils restores the volume controls after reboot it should work right away. The "Software" volume control is optional, but that way I can tune down the default applications volumes below the MPD volume if ever needed. Oh, and I think you can safely keep one or all of the software volume controls to maximum without distortion happening (this doesn't hold for all sound cards, where it's often best to keep the sliders at ~75% max to avoid distortion)

Erich Schubert: Penny Stock Spam

... is rather annoying these days. Apparently this known hacker group with their powerful Windows trojans is behind the latest spam wave: they use image obfuscation that makes the average Captchas look easy. In fact it took me a couple of seconds to understand what they're spamming about this time - the image just contained a highly unreadable stock symbol. I wonder if the big financial web sites (for non-experts) like Google Finance, Yahoo Finance or whatever you use for occassionally checking your stock quotes couldn't just add a warning if a penny stock is being spamvertised. Preventing naive users from falling for these frauds would certainly be a good service for their visitors, wouldn't it? And there are already websites listing spamvertised penny stocks. So dear big-financial-websites, start protecting your customers now (and reduce the amount of spam for all of us that way, since that will hopefully make penny stocks a lot less lucrative).

Next.

Previous.