Search Results: "tar"

I just read an interesting post about latency and how it affects web sites [1]. The post has some good ideas but unfortunately mixed information on some esoteric technologies such as infiniband that are not generally applicable with material that is of wide use (such as ping times). The post starts by describing the latency requirements of Amazon and stock broking companies. It’s obvious that stock brokers have a great desire to reduce latency, it’s also not surprising that Google and Amazon analyse the statistics of their operations and make changes to increase their results by a few percent. But it seems to be a widely held belief that personal web sites are exempt from such requirements. The purpose of creating content on a web site is to have people read it, if you can get an increase in traffic of a few percent by having a faster site and if those readers refer others then it seems likely to have the potential to significantly improve the result. Note that an increase in readership through a better experience is likely to be exponential, and an exponential increase of a few percent a year will eventually add up (an increase of 4% a year will double the traffic in 18 years). I have been considering hosting my blog somewhere else for a while. My blog is currently doing about 3G of traffic a month which averages out to just over 1KB/s, peaks will of course be a lot greater than that and the 512Kb/s of the Internet connection would probably be a limit even if it wasn’t for the other sites onn the same link. The link in question is being used for serving about 8G of web data per month and there is some mail server use which also takes bandwidth. So performance is often unpleasantly slow. For a small site such as mine the most relevant issues seem to be based around available bandwidth, swap space use (or the lack therof), disk IO (for when things don’t fit in cache) and available CPU power exceeding the requirements. For hosting in Australia (as I do right now) bandwidth is a problem. Internet connectivity is not cheap in any way and bandwidth is always limited. Also the latency of connections from Australia to other parts of the world often is not as good as desired (especially if using cheap hosting as I currently do). According to Webalizer only 3.14% of the people who access my blog are from Australia, they will get better access to my site if hosted in Australia, and maybe the 0.15% of people who access my blog from New Zealand will also benefit from the locality of sites hosted in Australia. But the 37% of readers who are described as “US Commercial” (presumably .com) and the 6% described as “United States” (presumably .us) will benefit from US hosting, as will most of the 30% who are described as “Network” (.net I guess). For getting good network bandwidth it seems that the best option is to choose what seems to be the best ISP in the US that I can afford, where determining what is “best” is largely based on rumour. One of the comments on my post about virtual servers and swap space [2] suggested just not using swap and referenced the Amazon EC2 (Elastic Computing) cloud service and the Gandi.net hosting (which is in limited beta and not generally available). The Amazon EC2 clound service [3] has a minimum offering of 1.7G of RAM, 1EC2 Compute Unit (equivalent to a 1.0-1.2GHz 2007 Opteron or 2007 Xeon processor), 160G of “instance storage” (local disk for an instance) running 32bit software. Currently my server is using 12% of a Celeron 2.4GHz CPU on average (which includes a mail server with lots of anti-spam measures, Venus, and other things). Running just the web sites on 1EC2 Compute Unit should use significantly less than 25% of a 1.0GHz Opteron. I’m currently using 400M of RAM for my DomU (although the MySQL server is in a different DomU). 1.7G of RAM for my web sites is heaps even when including a MySQL server. Currently a MySQL dump of my blog is just under 10M of data, with 1.7G of RAM the database should stay entirely in RAM which will avoid the disk IO issues. I could probably use about 1/3 of that much RAM and still not swap. The cost of EC2 is $US0.10 per hour of uptime (for a small server), so that’s $US74.40 per month. The cost for data transfer is 17 cents a GIG for sending and 10 cents a gig for receiving (bulk discounts are available for multiple terabytes per month). I am not going to pay $74 per month to host my blog. But sharing that cost with other people might be a viable option. An EC2 instance provides up to 5 “Elastic IP addresses” (public addresses that can be mapped to instances) which are free when they are being used (there is a cost of one cent per hour for unused addresses - not a problem for me as I want 24*7 uptime). So it should be relatively easy to divide the costs of an EC2 instance among five people by accounting for data transfer per IP address. Hosting five web sites that use the same software (MySQL and Apache for example) should reduce memory use and allow more effective caching. A small server on EC2 costs about five times more than one of the cheap DomU systems that I have previously investigated [4] but provides ten times the RAM. While the RAM is impressive, I have to wonder about CPU scheduling and disk IO performance. I guess I can avoid disk IO on the critical paths by relying on caching and not doing synchronous writes to log files. That just leaves CPU scheduling as a potential area where it could fall down. Here is an interesting post describing how to use EC2 [5]. Another thing to consider is changing blog software. I currently use Wordpress which is more CPU intensive than some other options (due to being written in PHP), is slightly memory hungry (PHP and MySQL), and doesn’t have the best security history. It seems that an ideal blog design would use a language such as Java or PHP for comments and use static pages for the main article (with the comments in a frame or loaded by JavaScript). Then the main article would load quickly and comments (which probably aren’t read by most users) would get loaded later.

The McCain campaign made a great tactical move by choosing Sarah Palin. She appeals to all the constituencies where McCain was weak: women, Christians, Libertarians, small state residents, energy policy wonks, corruption fighters, capitalists, gun enthusiasts, and Tina Fey fetishists. Downsides? She is inexperienced in national affairs but I really don't think the average voter will care. After all many of them were willing to overlook a similar level of inexperience in Obama. And there is one minor possible ethics situation which again will only excite the true believers. When it turns out to be overblown it may actually rebound in Palins favor as an example of MSM manipulation. She can also take the change meme away from the Democrats though that was already on the rocks what with the FISA vote and the choice of Neil Kinnock as his VP nominee. No, the lacklustre convention in Denver clearly showed that far from being the post-ideological, audaciously hopeful candidate of the future, Barack Obama is merely John Kerry with rhythm. So what do the Dems have have left? The traditional Democratic themes of "We're not Bush" (that worked sooo well the last two times didn't it?) and "The Earth is about to explode in a fiery cataclysm." As the situation in Iraq improves, the economy climbs upwards from recession and petrol prices continue to come down its going to be harder to sustain the necessary level of gloom and doom. While Republicans shouldn't exhale just yet (and the title of this post is just a teensy bit hyperbolic,) things are looking pretty good for us at the moment.

For those who missed my message at #debconf: I had troubles creating a new album at gallery.dc.o so I uploaded some of the photos I took on my trip to DC8 and are available at:
http://my.opera.com/atomo64/albums/

Please note that some people already requested me to remove some photos (the funniest ones) :(
Anyway, enjoy them! :)

According to the Slashdot article and the StopSoftwarePatents.org website itself and on Digg, the anti-software-patent activists are attempting a world-wide event on the 24th of September as a world-wide day against software patents. USA has them via a weird court ruling, Japan has them as well (not sure why), there have been efforts to force software patents on EU, India, Australia and many other countries either by Microsoft lobbies or even via US trade treaty pressure. While lobbying by local Microsoft branches and their pet companies can be countered locally, like I and many others did in EU a couple years ago (mostly thanks to FFII), trade negotiations are very secretive affairs and it is very hard to lobby there directly. The public needs to be aware of the issue, otherwise the politicians will not be aware of its importance to their voters. Software patents are a threat to free software as it circumvents the power of GPL and other copyright licenses if enough money is thrown at the lawyers. If a software patent is generic enough, it can easily stop development of a whole class of free software applications on a whim of the patent holder. And there are plenty of granted software patents with a very broad scope (progress bar, anyone?). I am aware that to get Debian support for such an initiative, a GR is needed, but how about a personal word of support from the DPL or support from the SPI? It is in direct area of interest for SPI - software patents create an ever present threat of legal action against any and all software in Debian (in USA and Japan, at least) and, as the legal umbrella of Debian in the USA, SPI would be a prime target. What do others think about this?

So, the string freeze for Debian Installer was announced today. It will last until Spetember 20th 23:59 UTC. We start with 13 complete languages and 15 meeting the release criteria. 60 (59+English) languages are currently supported. How many will meet the release criteria in 22 days?

Poster for the 3rd congress on solidary economy in Bremen

If you happen to be near Bremen, Germany around the 5.-7. September 2008 and have an interest in solidary economy, free software, renewable energies and the like you might want to stop by at the 3rd Congress on Solidary Economy (German website). I haven't been at the first two of these congresses but I was invited as a speaker for this one, so I'll be around at least on Saturday and probably also on Sunday. I'll mainly be on a panel on free and open source software which will take place on Saturday evening at 20:00. (Prime Time :)) But I've also let myself be talked into joining another panel at 17:00 which is somehow ubuntu-related (Communtu). I am not yet sure if I'll have much to contribute there, I am neither a ubuntu specialist nor particularly fond of it, but the organiser said he wouldn't mind and he'd welcome critical remarks just as much, so, we'll see. I am really looking forward to being a speaker on an event again! Ever since I started my PhD I've been much too busy to attend (leave alone prepare talks for) as many events as I'd have liked. I hope the situation will improve next year when things get rolling with my PhD. (Currently still in a "topic orientation/find a project/write proposals" state ...)

As per the original press release:

In Quebec, access to public markets is the rule while contracts attribution without invitation to tender is the exception. A public market should be transparent, fair and most importantly, open to all. The solutions as well as the propositions must be evaluated objectively on known and accepted criteria. Furthermore, the regulation implies that public markets have to enhance the local economic development as well as the Quebec technologies. From February to June 2008, FACIL has noticed sales of proprietary software for more than 25 million dollars. These purchases were made for products offered by large multinational enterprises, with no regard to suppliers in Quebec.

Much to my surprise, news about this lawsuit have spread as far as Vietnam! Something tells me that this court case might have lasting repercussions...

As I start this blog post, I'm on the train between Amersfoort and Osnabr ck, on my way to Kiel, where there'll be a meeting of the m68k porters, kindly organized by m68k kernel maintainer Christian Steigies. I took the 8:40 train in Mechelen this morning, and if all goes well, I'll be arriving in Kiel at 17:22 tonight. Some train trip... but I certainly prefer that to any flight. Anyway. The days of Debconf were very nice. Debconf is a lot of things to me. Travel. Hacking. Mao, evolving into a drinking game. Beer. Attending talks. Giving talks. Meeting people. Face-to-face non-flaming. Gesturing for an extra knife at luunch. Hangover. Whiskey. Talking to the person next to you—over IRC. Getting killed before knowing what the rules of the assassins game are. Filing bugreports in person. Pictures. Kilts. Streaming video. Not being sick, hopefully. Name badges. Flying. Sleeping. Yes, sleeping. Most of all, though, this year, Debconf was just great. Thanks. You know who you are. After debconf, I flew to Buenos Aires, where I slept for one night in a Youth Hostel somewhere downtown. They were affordable, but the bed wasn't great—the mattress sorely needed replacement. Since I had a day in BA, and since there were apparently not enough people on the schedule, I'd agreed to hold a talk, and came up with the idea of a 'debian secrets' talk—about Debian-specific commands, such as dpkg-divert and update-alternatives etc—so that people could learn how to use their Debian system more efficiently. As I was sitting in Andreas' talk, who was right before me on the schedule, suddenly Dag walked up to me and said hi. This was unexpected; Dag is a fellow Belgian, who's involved with the CentOS project, and who maintains a positively huge RPM repository at his site—if you maintain an RPM-based system somewhere, you'll probably know about that site. So while I recognized him, I immediately wondered what he was doing there. Turned out he was invited to hold a talk at the Free Software event to which Debianday was attached (and which would not start for another day or two), and that when he saw my name on the schedule, he decided to attend. Fun. He learned a thing or two from my talk, and was talking later about writing some tools for RPM-based systems that would perform the same or similar functionality implemented by some of the Debian tools I talked about. Great. We had a picture together (in front of the DebConf banner—hah) that I still should upload, and then went out for lunch together with some other CentOS guys. The talk itself seems to have been well-received, and I'm glad about that; I only gave it because there was a need for more talks, not because I felt confident I was very knowledgeable about that subject. In fact, I did have to ask on the debconf-discuss mailinglist for some input (which I received) so that I could make sure the talk would actually be useful to people. That helped a lot, I guess. After that, we played some mao in the lobby of the DebianDay venue. While doing so, I overheard Dag talking about CentOS, advocating it to some of the people there, which I must say felt pretty weird on a Debian event. Not that there's anything wrong with it, of course. Except that Dag was supposed to be writing his talk slides. How did that work out, Dag? ;-) Eventually, I got in a cab to the EZE airport, and flew home. That wasn't fully without issue, but I did get there. And now, I've been, eh, overloaded. Still have to follow up on a question a customer asked me, but I've barely been home or at the office, just enough to sleep really. Should do something about that, I guess... Two days ago, I also managed to forget my camera somewhere. For 12 hours, I was worried, even though I had a pretty good idea of where it was; but since I'm getting quite good at taking pictures now, and since I really like doing so as well, I really didn't want to lose my camera. Fortunately, when I called this morning to the place where I thought I'd lost it, they told me it'd been found, and where to get it. In other words, it's safe, it's taken care of, Philip went to get it, and I just need to arrange to get it when I get back. Which is a relief. Let's not be so stupid anymore in the future... Finishing up now, a day later, and I'm at Christian's office, playing with his and my coldfire board. As it happens, it appears that Freescale has brought out a new BSP for these boards, so that's nice. Let's see whether we can get those to boot again...

This is the night that the clockwork burns down. This is the night of elbows scraping against iron bars. Tom Stoppard thought he had a glass full of fractals, when really it was an ass full of dactyls. This is the night of candelabras crashing into the shoes of giants. This is the night when the man with the homburg sings a hymn for the hopeless. The novice groped an honored mater outside the slic hut. This is the night that the hunger reached out for the abyss and fell in. This is the night that the crack broke its own mother's back. Sweet Nutty went to the levee and the levee didn't like it one bit. This is the night of the British spellings. This is the night of the sandpiper's dream. Lady Starbucks wondered about the healing powers of the dental-floss bikini. This is the night that nothing lasts forever. This is the night that nothing ever ends. The Q2 Gang tripped the light fantastic while waxing the text unreadable. This is the night of the Magic Mainspring. This is the night of remembering Trufflehunter's name. There's ice cream waiting for me in another world.

I was just looking to buy some online classes. I haven’t bought one yet and here’s three ideas I think online course sellers should follow to get more customers:-

Telephone courses - it’s the world-wide web, but that doesn’t necessarily mean I want to make an international telephone call for the course. If the course includes a teleconference, publish the area code(s) so I can include the cost of my calls when budgeting, please. Even better, offer a local (UK) number or Voice-Over-IP (VOIP, or internet phone) access for SIP phones, so it won’t cost me extra.
Web classes - web browsers have plugins, but not all web browsers have all plugins and sometimes I’d prefer to watch your video stream on my TV instead. That’s easy if it’s a video stream I can load in a standalone player (I can even remote-control some players with my bluetooth phone), but it’s a right pain if it requires Adobe Flash, locked down with javascript, DRM and so on. Make video streams that can be used in Videolan or MPlayer, please.
Web classes - if a class uses its own special downloadable program (even if it’s Java), then it’s not a web class. Please make the class usable in a web browser and please publish a small test class so I can make sure my browser works before the class starts.
Ordering systems - one site gave me no fewer than three big red error messages while I was trying to buy. The first said the site requires Javascript. I changed that setting, pressed back and re-ordered, then the second big red error said it requires cookies . I changed that setting, then it said it needed Internet Explorer. I closed the Firefox/Iceweasel window and gave up. Please test your ordering system and show a full list of system requirements as soon as there’s an error, to avoid wasting my time.

Are there any good online course providers out there which do all of the above?

I first read about Y Combinator while browsing Paul Graham s website. I am a huge fan of his and have tried many times to bring him to Brazil for FISL (unsuccessfully so far). By that time I was updating a lecture of mine where I quote something from his Hackers and Painters essay and sudenly I noted an YC link on the left, and decided to click to learn what was that. Y Combinator is a venture capital firm to help start-ups. It was built on the concept that not much money is needed for first-stage start-ups ( from idea to company , as they call it). Quite a concept! I lost track of it until yesterday, when I read this blog post summarizing one of its events for start-ups. I found it amazing how far it came in such little time. They ve already funded about 100 start-ups, some of them really interesting. Of those presented in that post, Posterous is one I found most interesting. They create instant blogs just by sending emails to some address. There s no sign-up procedure Just send the email with any attachments and voil . There you have: a new blog. Many others are really interesting (PollEveryWhere, IDidWork, Frogmetrics, just to name a few), so I really suggest reading that post. I think Y Combinator is another great idea from Paul Graham, and I will be following it more closely, as a way to keep myself updated Who knows maybe I can get them to come to FISL to present that! ;-)

The Debian Live team is pleased to announce the first beta of Debian Lenny's Live images. Although we missed releasing images for Etch along with the installer images, we are now prepared to release live images within the regular Lenny release process. This is the first official release of Debian Live and the whole team has been working hard during the past 2.5 years to make Debian's own live systems become a reality. Nevertheless, we do need your help to find more bugs and improve the live systems, so please try them out. The images are available at:

	http://cdimage.debian.org/cdimage/lenny_live_beta1/

Please also have a look at the known issues listed below. Main features 100% Debian The build process of Debian Live basically consists of creating a Debian chroot, installing one or more kernels along with live-initramfs (a set of hooks into initramfs-tools for handling booting from read-only media) and generating a bootable image from that. This process is handled by live-helper, a collection of shell scripts that allow us to automate and customize this process. Considerable care is taken to ensure that the resulting live system is not tainted by the host system and that installed packages are not modified morethan absolutely necessary. This ensures that Debian Live really is Debian, and not "just another" a Debian-based live system. Flavours Although live-helper is a toolkit to produce your very own live systems with only a few steps, we also provide prebuilt images that are meant to be used as reference systems for end-users. Currently, this consists of the three major desktop environments (GNOME, KDE and Xfce), as well as a small 'standard' image without a graphical environment. For the desktop environments, package selection is performed by 'tasksel' with the respective desktop task, whilst the 'standard' image contains only packages of "Priority: standard" or greater, notwithstanding a handful of live-specific packages (console-common, eject, file, kbd, live-initramfs, locales, sudo and vim-tiny). Image types Debian Live offers prebuilt images for CD/DVD discs, USB sticks (or any HD-media-like device), tarballs (for PXE netboot) as well as a bare squashfs image to boot from the web directly. Live Magic Live Magic is an GUI frontend around the live-helper scripts, offering a subset of the features of live-helper in an easy-to-use graphical user interface. live-magic 1.0 was recently uploaded to sid and is the recommended version. It currently supports 7 languages. Live Installer Live Installer is a special udeb for the Debian Installer that (optionally) replaces a part of d-i in order to install the system from the live image instead of to bootstrapping it from .deb packages. This way, a live system can be easily installed to the harddisk, ensuring that the look and feel of the installation (including preseeding) works the same as the regular installer process. Unfortunately, live-installer does still have a few minor bugs left and is thus not included in our builds yet; we hope to be able to include it in the next beta. Known issues in this release

The prebuilt images for gnome-desktop and kde-desktop are a bit too big to fit on a CD. Although the automatic installation of Recommends was disabled to build the images, they are still too big. This will need further tweaking as they are supposed to fit with the next beta.
The rescue flavour, containing system rescue and forensic related packages, is missing in this beta release.
There will be a DVD image with the next beta that includes all three desktop environments so that you can choose at boot time which system you would like to start.
Due to time constraints, the prebuilt images for Beta1 are only covering i386 and amd64; with the next beta, powerpc and sparc will follow (if you wish to test these architectures earlier, please build them yourself).
The new desktop artwork is not yet included.
The syslinux menu is still the old, prompt-based one. A freshly made new syslinux vgamenu using the official lenny desktop artwork is on the way (the same as d-i media will use in Lenny).
No live-installer included yet (if you wish to test live-installer earlier, please build it yourself).
No win32-loader included yet.
Sid packages used - At the time of building the live images, both live-helper 1.0.0-1 and live-initramfs 1.139.1-1 from sid were used. Since the latter has not yet migrated to testing, it was included manually as a local package. This is undesirable, as the release is supposed to be self-contained - however, it is just a convenient workaround as both were granted freeze exception and will migrate soon.

Plans for next Beta release We are looking forward to upload Beta2 in about two weeks from now (maybefollowed by a third beta) with one final RC after that which should be identical to the final release. Some more details about the open things we would like to address in Beta2 and later can be found at the wiki page. The Debian Live team is still looking for more contributors for new features (post-lenny, though) as well as documentation writers for the manual (always). If you care about live systems, please join and help! Thanks Last but not least, our thanks goes to everyone who has contributed and to all maintainers that have kindly fixed live-specific bugs in their packages.

The Problem: A problem with virtual machines is the fact that one rogue DomU can destroy the performance of all the others by inappropriate resource use. CPU scheduling is designed to allow reasonable sharing of computational resources, it is unfortunately not well documented, the XenSource wiki currently doesn’t document the “credit” scheduler which is used in Debian/Etch and CentOS 5 [1]. One interesting fact is that CPU scheduling in Xen can have a significant effect on IO performance as demonstrated in the paper by Ludmila Cherkasova, Diwaker Gupta and Amin Vahdat [2]. But they only showed a factor of two performance difference (which while bad is not THAT bad). A more significant problem is managing virtual memory, when there is excessive paging performance can drop by a factor of 100 and even the most basic tasks become impossible. The design of Xen is that every DomU is allocated some physical RAM and has it’s own swap space. I have previously written about my experiments to optimise swap usage on Xen systems by using a tmpfs in the Dom0 [3]. The aim was to have every Xen DomU swap data out to a tmpfs so that if one DomU was paging heavily and the other DomUs were not paging then the paging might take place in the Dom0’s RAM and not hit disk. The experiments were not particularly successful but I would be interested in seeing further research in this area as there might be some potential to do some good. I have previously written about the issues related to swap space sizing on Linux [4]. My conclusion is that following the “twice RAM” myth will lead to systems becoming unusable due to excessive swapping in situations where they might otherwise be usable if the kernel killed some processes instead (naturally there are exceptions to my general rule due to different application memory use patterns - but I think that my general rule is a lot better than the “twice RAM” one). One thing that I didn’t consider at the time is the implications of this problem for Xen servers. If you have 10 physical machines and one starts paging excessively then you have one machine to reboot. If you have 10 Xen DomUs on a single host and one starts paging heavily then you end up with one machine that is unusable due to thrashing and nine machines that deliver very poor disk read performance - which might make them unusable too. Read performance can particularly suffer in a situation when one process or VM is writing heavily to disk due to the way that the disk queuing works, it’s not uncommon for an application to read dozens or hundreds of blocks from disk to satisfy a single trivial request from a user, if each of these block read requests has to wait for a large amount of data to be written out from the write-back cache then performance will suck badly (I have seen this in experiments on single disks and on Linux software RAID - but have not had the opportunity to do good tests on a hardware RAID array). Currently for Xen DomUs I am allocating swap spaces no larger than 512M, as anything larger than that is likely to cause excessive performance loss to the rest of the server if it is actually used. A Solution for Similar Problems: A well known optimisation technique of desktop systems is to use a separate disk for swap, in desktop machines people often use the old disk as swap after buying a new larger disk for main storage. The benefit of this is that swap use will not interfere with other disk use, for example the disk reads needed to run the “ps” and “kill” programs won’t be blocked by the memory hog that you want to kill. I believe that similar techniques can be applied to Xen servers and give even greater benefits. When a desktop machine starts paging excessively the user will eventually take a coffee break and let the machine recover, but when an Internet server such as a web server starts paging excessively the requests keep coming in and the number of active processes increases so it seems likely that using a different device for the swap will allow some processes to satisfy requests by reading data from disk while some other processes are waiting to be paged in. Applying it to Xen Servers: The first thing that needs to be considered for such a design is the importance of reliable swap. When it comes to low-end servers there is ongoing discussion about the relative merits of RAID-0 and RAID-1 for swap. The benefit of RAID-0 is performance (at least in perception - I can imagine some OS swapping algorithms that could potentially give better performance on RAID-1 and I am not aware of any research in this area). The benefit of RAID-1 is reliability. Now there are two issues in regard to reliability, one is continuity of service (EG being able to hot-swap a failed disk while the server is running), and the other is the absence of data loss. For some systems it may be acceptable to have a process SEGV (which I presume is the result if a page-in request fails) due to a dead disk (reserving the data loss protection of RAID for files). One issue related to this is the ability to regain control of a server after a problem. For example if the host OS of a machine had non-RAID swap then a disk failure could prevent a page-in of data related to sshd or some similar process and thus make it impossible to recover the machine without hardware access. But if the swap for a virtual machine was on a non-RAID disk and the host had RAID for it’s swap then the sysadmin could login to the host and reboot the DomU after creating a new swap space on a working disk. Now if you have a server with 8 or 12 disks (both of which seem to be reasonably common capacities of modern 2RU servers) and if you decide that RAID is not required for the swap space of DomUs then it would be possible to assign single disks for swap spaces for groups of virtual machines. So if one client had several virtual machines they could have them share the same single disk for the swap, so a thrashing server would only affect the performance of other VMs from the same client. One possible configuration would be a 12 disk server that has a four disk RAID-5 array for main storage and 8 single disks for swap. 8 CPU cores is common for a modern 2RU server, so it would be possible to lock 8 groups of DomUs so that they share CPUs and swap spaces. Another possibility would be to have four groups of DomUs where each group had a RAID-1 array for swap and two CPU cores. I am not sure of the aggregate performance impact of such a configuration, I suspect that a group of single disks would give better performance for swap than a single RAID array and that RAID-1 would outperform RAID-5. For a single DomU it seems most likely that using part of a large RAID array for swap space would give better performance. But the benefit in partitioning the server seems clear. An architecture where each DomU had it’s own dedicated disk for a swap space is something that I would consider a significant benefit if renting a Xen DomU. I would rather have the risk of down-time (which should be short with hot-swap disks and hardware monitoring) in the rare case of a disk failure than have bad performance regularly in the common situation of someone else’s DomU being overloaded. Failing that, having a separate RAID array for swap would be a significant benefit. If every process that isn’t being paged out could deliver full performance while one DomU was thrashing then it would be a significant benefit over the situation where any DomU can thrash and kill the file access performance of all other DomUs. A single RAID-1 array should handle all the swap space requirements for a small or medium size Xen server One thing that I have not tested is the operation of LVM when one disk goes bad. In the case of a disk with bad sectors it’s possible to move the LVs that are not affected to other disks and to remove the LV that was affected and re-create it after removing the bad disk. The case of a disk that is totally dead (IE the PV header can’t be read or written) might cause some additional complications.

[1] http://wiki.xensource.com/xenwiki/Scheduling

[2] http://www.cs.ucsd.edu/~dgupta/papers/per07-3sched-xen.pdf

[3] http://etbe.coker.com.au/2008/05/22/xen-and-swap/

[4] http://etbe.coker.com.au/2007/09/28/swap-space/

Share This

I took a note of the various talks I attended at DebConf, with the aim of writing a few notes about each so that I could try to explain to my work mates exactly what I get out of going. However I think a lot of the benefit is about general face to face contact with fellow DDs and random conversations that happen. Having worked from home for over 3 years I'm finding it quite nice to be back in an office with people working on the same things as me; there are conversations that you will have over a quick cup of coffee or in the corridor that you wouldn't bother typing on IRC or email or picking up the phone for. Debian is a lot like a large organisation full of telecommuters and DebConf provides a vital opportunity for us to have those "minor" conversations that prove quite useful and to remind each other what we're actually like in person. I didn't end up writing a lot of notes during the talks so these are just snippets after the fact. If I'd been more organised I'd have done a write up on a daily basis rather than leaving it all until the week after. A lot of the below is fragmented thoughts, but the longer I leave it the less relevant it gets and the more chance I forget something. It'd have been up faster if I hadn't been fighting Movable Type to try and get some sort of headings. Sunday
Introduction, Marga The "Welcome to DebConf" bit. Marga got up and talked about how at DebConf4 (Brazil) she hadn't been a DD and now she was and how it had been a life changing event. She'd thought she was to blame for DebConf8 in Argentina, but trawling mail archives had shown it was originally Mart n's idea. She hadn't realised at the start quite how much work it would be. Steve's DPL talk, Steve McIntyre The DPL welcomes us to DebConf and talks about Debian's achievements, some of the results of his team survey (the teams largely say they need more manpower. Not really a huge surprise.). NM/User Survey BOF, Paul Wise Paul emailed debian-user, debian-newmaint and various other places (I think) asking about people's experiences as a Debian user or applicant to New Maint. His final results aren't yet published and I got the impression a disappointing number of our users had replied. The NM problems these days seem to be largely down to a lack of AMs, at least according to the stats on nm.debian.org. Meet SPI board - bdale, maulkin, jimmy, michael, others A chance to meet those of the SPI board who were present. The usual "If you're involved with Debian you should seriously consider joining SPI". Some discussion about the fact that although SPI was originally mostly Debian these days other projects are making more and more use of it - OFTC, PostgreSQL and Madwifi for example. Not a huge amount of new information for me. I've been an SPI member for some time and I think they perform a vital role for Debian. They've had issues in the past but it's really good to see them overcome and more projects on board. Monday
http://hp.com/go/debian, bdale Bdale talks about the history of Debian and HP. And makes the point that HP aren't doing any of this to be "nice"; their primary responsibility is to their shareholders. Not surprising, but it's good that even with that criteria at the fore front Debian ends up a good idea for a large multinational. Managing 666 packages, Mart n Ferrari Mostly centered around the pkg-perl experience and the tools/workflow they have to help manage things. Impressive web tool (DEHS) to help track packages with new upstreams, issues or pending uploads. Quality Assurance in Lenny + 1 I can't remember a lot of this, which is a bit bad. I remember a discussion about whether we should release with orphaned packages, or if they should be removed. The argument for not removing them seemed to revolve around a theory that if they weren't buggy then what was the harm, the argument for was that if they were unmaintained and not really used than how did we actually know they weren't buggy, and was it reasonable that they potentially held up library transitions etc? Bugs in large packages Don Armstrong and ways in which the BTS can help people with lots of bugs. This (thankfully) doesn't affect me, but it was interesting to hear Don talk about his work on the BTS and his various ideas. I was sorry I'd missed his talk on SOAP access to it; I shall have to find time to watch the video. Tuesday
Debian and Ubuntu, Mark Shuttleworth The usual interaction with Debian stuff, including how we interact with our upstreams. If a bug is raised with an Ubuntu package then it might well be appropriate to raise it with Debian and upstream as well, but each set of people should be able to use their own bug reporting systems rather than having to go to a foreign system. I don't think it was actually said, but I guess the idea is that launchpad is the thing that will do this gluing together. He also talked about Ubuntu as an "experiment" rather than a fork of Debian; something that has potentially different ideas about how to do things but that these ideas may lead to things that should be fed back into Debian. The point was made that even if you consider Ubuntu a fork they've forked 9 times now; it hasn't been a case of forking once and diverging wildly since thast point. I have a few thoughts on all the whole Debian/Ubuntu stuff which I should really try to form into something coherant and post. I think it's important to take Mark's talk in the context of bdale's "Shareholders come first" point. Derivatives round table Didn't stay for most of this. Ubuntu, Debian-EDU (which I hadn't realised was a derivative rather than a team within Debian, but apparently there are various things it was much easier for them just to patch now and try to work towards a generic solution later), someone from Extramadura and someone from the Munich government Linux stuff. I'm sure I've missed someone off. tdebs/i18n This was a useful BOF. It helped that we said no to the usual video team requirement that if you want to speak you had to wait for the microphone. This works fine for a normal talk, but really impedes the flow of conversation in something like a BOF. tdebs are something that have partly come from a need of Emdebian to reduce installed package size and partly from the desire of the i18n team to be able to update package translations without having to do a full binary NMU of a package. We had some of the FTP team, the release team, the i18n team and Emdebian there and I think some progress has been made. There'd been various conversations had before this meeting which I think helped people think about the issues and objections so that we had answers by the time we all sat down together - an excellent example of the use of DebConf. DebConf9 C ceres Various bits of discussion about next year's DebConf in Extramadura. Temperature is likely to be an issue for me as they reckon it could easily get up to 40C! I'll melt! Venue sounds good; some queries regarding late night hacklabs and net connections to same, but I'm sure the networking team will sort it all out as usual. ;) DebConf10 bids Venezela and New York were both pitching. Lots of people don't want to travel to the US or think they'll have problems doing so (I hadn't realised there was such an issue for people from South America). I suspect various people will have issues with Venezela as well; I heard expressions of doubt over their political stability expressed. However I'd be happy to go to either (and preferably both! DebConf10 + 12 perhaps?) Speaking of which, what ever happened to the Sarajevo bid? If they're reading, have you considered DebConf11? Wednesday
A day trip to an Argentinian ranch and lunch was an asido. Fantastic food and a pleasant afternoon spent sitting in the sun chatting to people. That evening was the formal, but thanks to having stayed up until after 4 on Tuesday I had to wimp out and go to bed fairly early. Not until I'd seen the DPL dance though. :) Thursday
Internationalization in Debian I forget most of this other than pretty graphs from Christian regarding what's translated and what other languages we might have; e.g. you can consider South America to be covered largely by Spanish and Portuguese, or you can consider it to not be covered very much at all by the local indigenous languages. dh-make-webapp: yeah right! Mainly what I took away from this was "It'll never happen thanks to the fact that every web app author uses a different system" and that more work needs to be done on the web apps best practices document. I don't really do web apps, but having them easily installable on my systems is a good thing so I hope some standardisation within Debian comes from this. Best practises in team-maintaining packages Various team members got up to talk about work flow in their teams. Not unsurprisingly different things seem to work for different people. Predictable PRNG in the Vulnerable Debian OpenSSL Package Lucian giving his Black Hat talk about the OpenSSL issues we faced earlier this year. Some queries about automated testing and that it should have found this, which originally I agreed with until I thought about it - you'd have to have run > 65536 tests and compared all the data to find this; something like the FIPS tests against a single run wouldn't have shown any issues. Lucian wanted feedback on ensuring he was presenting the Debian side of things acceptably as well. Keysigning Run by Don Armstrong. Nothing to really remark except I think key signing attendance is dropping off after things like the Helsinki experience. Also I've now managed to get my new RSA key signed by keys other than my own (and I'll try to get sorted out to sign keys myself soon). Friday
LessWatts Intel's drive towards reducing power usage. Odd to hear such things from someone who wasn't Matthew Garrett and I think I'd heard most of it before. Debian technical policy update Sorry Manoj, I should have been paying more attention. Debian Policy is something I think is important, but I'm not enough of a nitpicker to really feel I can contribute a whole lot. Virtualisation in Debian - Present and future This was depressing. What I took away was "We had 2 forms of virtualisation in etch (Xen + vserver) and we're supporting neither for Lenny". As someone running a Xen machine this is problematic. The host does have the hardware support that would allow kvm, but I'm worried about smooth migration (it's a colo box). There was some mention of Xenner, which isn't packaged for Debian. I took a look at doing so but there are some worrying items in the TODO list (like, oh, locking to enable reliable SMP) and I've spectacularly failed to get it compiling so far. I should have another go. Emdebian update Neil Williams about where Emdebian is and waved his Balloon3 about. Some interest from the Openmoko guys and discussion about why it might be better than OpenEmbedded. And equally what it couldn't do that OpenEmbedded can manage (mainly very small installs; a minimal Emdebian install is 24MB+ and with X that goes up to 75MB+ I believe). netconf I'd never actually seen Martin talk about netconf before. Am impressive flow of control diagram. It all sounds quite complex for little benefit at present, but if he pulls it off then I can see it being a much more flexible replacement for the likes of Network Manager. Saturday
Lenny - The Road To Release Neil played with putting the heads of the release team on random images. And tried to tell us all how to behave during the release process in terms of what should and shouldn't be uploaded etc. Most of it is just common sense, but apparently everything he talked about he'd experienced. Also pointed out we were supposed to release in 3 weeks and didn't even have the appropriate kernel in testing yet so it's probably not going to happen. Colour me surprised. Sustainable Computing Low power consumption/affordable computing. Interesting points about the OLPC being used in the middle of nowhere with no chance of an Internet connection being close by compared to neighbours in large cities where people can't necessarily afford top of the line computers or individual connections - in this sort of scenario mesh networking is really viable in terms of getting people connected up. Multi winner voting Manoj needs pretty diagrams, probably drawn by Martin Krafft. ;) Semi-interesting discussion about multi-winner voting (e.g. SPI board elections) and how the current Debian voting system is completely tied to Debian's infrastructure eg the LDAP setup. Manoj would like to make it more generic so it could be used by other groups as well as for different sorts of votes in Debian.

A while ago I started using Google Reader. Reader has a nice social feature where you can share articles with your friends. This is nicer than posted items in an activity stream since you don't have to click on anything, the article is right there. Anyway, I got a shared item in my stream of articles today with the second sentence as "She somehow got her own claw stuck into her paw.". Then it struck me, I've heard this story before. My dinner guests last week told this same story. So it seems I've found Michelle's blog. Thanks Tony.

Search Results: "tar"

30 August 2008

Russell Coker: Improving Blog Latency to Benefit Readers

Jaldhar Vyas: And That Just About Wraps it up For Obama

29 August 2008

Raphael Geissert: Debconf8 photos

Aigars Mahinovs: 24th September as a day against software patents - Debian support?

Christian Perrier: D-I string freeze started

Meike Reichle: 3rd Congress on Solidary Economy Bremen

Martin-Éric Racine: Qu bec government sued for buying Microsoft products without any public call for tender

Wouter Verhelst: Debconf8, and the m68k meeting in Kiel

28 August 2008

Clint Adams: In the next phase, astronut teaches the bot

MJ Ray: Why I Didn?t Buy Some Online Courses

27 August 2008

Pablo Lorenzzoni: Paul Graham and his start-up funding company

Daniel Baumann: Debian Live Lenny Beta1

26 August 2008

Russell Coker: Killing Servers with Virtualisation and Swap

Gerfried Fuchs: scons annoys

MJ Ray: Solved but Why? Belkin F5D7630 to Realtek 8139 Drop-outs

25 August 2008

Jeff Licquia: Standards and Conversations, Part 1

Holger Levsen: One RC bug per week

24 August 2008

Russell Coker: Xen and Linux Memory Assignment Bugs

Jonathan McDowell: DebConf8 writeup

23 August 2008

James Morrison: How not to find a friends blog