Search Results: "ag"

28 August 2008

Bastian Venthur: Finally, a Debian Live CD

That was just about time. In the last two Linux Tag events people asked us for Live-CDs and we had to explain why we don’t have any, while most other distros have. Now we finally can offer a Debian Live-CD. Still beta, but a huge improvement. Thanks Debian Live team!

David Welton: Well, that didn't work

Ouch. Absolutely no takers on the Squeezed Books contest: http://www.squeezedbooks.com/book/contest http://www.squeezedbooks.com/book/leaders I wonder what didn't work out? I mean, a book isn't a huge prize, exactly, but I know I've participated in mini-contests such as "leave a comment on my blog saying why you should win this book". I actually got a copy of Nick Carr's "The Big Switch" that way. And as it turns out, all it would have taken to win a free book in the Squeezed Books contest is an edit of a few words (as long as it wasn't random gibberish), so the 'barrier to entry' was pretty low. Perhaps I wasn't able to publicize it enough? Maybe the 'right' people didn't find it? Maybe people didn't find it credible? Too much of a pain to log in? The instructions aren't clear? It's tough - I am and always will be a programmer at heart, and this marketing stuff is harder than it seems! However, barring any sudden blinding insights, I'm going to try again. This time, whoever contributes the most to new or existing business book summaries by September 30th gets two business books of their choice, since we didn't give one away last time.

MJ Ray: Why I Didn?t Buy Some Online Courses

I was just looking to buy some online classes. I haven’t bought one yet and here’s three ideas I think online course sellers should follow to get more customers:- Are there any good online course providers out there which do all of the above?

27 August 2008

Gerfried Fuchs: Two Days, Apocalyptica

For quite a while I'm already looking forward to this year's Two Days A Week festival. To be precise right from when I saw the first poster about it and me finding the name of one of my most favourite bands on it: Live. I haven't seen the guys from the US for a long time now and am absolutely eager to see them on stage again. But today came the next shock: Some few days ago Slipknot had to cancel their attendance because of the injury of one of their band members, they have found a more than adequate replacement: Apocalyptica!!!! Wiesen, I'm coming!

Pablo Lorenzzoni: Paul Graham and his start-up funding company

I first read about Y Combinator while browsing Paul Graham s website. I am a huge fan of his and have tried many times to bring him to Brazil for FISL (unsuccessfully so far). By that time I was updating a lecture of mine where I quote something from his Hackers and Painters essay and sudenly I noted an YC link on the left, and decided to click to learn what was that. Y Combinator is a venture capital firm to help start-ups. It was built on the concept that not much money is needed for first-stage start-ups ( from idea to company , as they call it). Quite a concept! I lost track of it until yesterday, when I read this blog post summarizing one of its events for start-ups. I found it amazing how far it came in such little time. They ve already funded about 100 start-ups, some of them really interesting. Of those presented in that post, Posterous is one I found most interesting. They create instant blogs just by sending emails to some address. There s no sign-up procedure Just send the email with any attachments and voil . There you have: a new blog. Many others are really interesting (PollEveryWhere, IDidWork, Frogmetrics, just to name a few), so I really suggest reading that post. I think Y Combinator is another great idea from Paul Graham, and I will be following it more closely, as a way to keep myself updated Who knows maybe I can get them to come to FISL to present that! ;-)

Dirk Eddelbuettel: littler 0.1.1 released

The new release 0.1.1 of r (pronounced littler) was just rolled up. The only new feature is due to a suggestion by Paul Gilbert: r now reports the value of the optional status variable when calling q() at the end of a script:
$ r -e'q(status=42)'; echo $?
42
This can be very useful to signal exit codes and branching on those in other scripts or Makefile. We also applied a patch to manual page which adds some examples there (thanks, Seb!) and made some small changes to tests and examples. As usual, our code in our svn archive, on my r page, and in the local directory here. A fresh package is in Debian's incoming queue, and Jeff's littler page at Vanderbilt should reflect the new release soon too.

Adam Rosi-Kessel: Democratic Unity

A sure sign of the long road ahead of us to pull together in this election: Can’t we all just join forces? Technorati Tags: , ,

Daniel Baumann: Debian Live Lenny Beta1

The Debian Live team is pleased to announce the first beta of Debian Lenny's Live images. Although we missed releasing images for Etch along with the installer images, we are now prepared to release live images within the regular Lenny release process. This is the first official release of Debian Live and the whole team has been working hard during the past 2.5 years to make Debian's own live systems become a reality. Nevertheless, we do need your help to find more bugs and improve the live systems, so please try them out. The images are available at:
	http://cdimage.debian.org/cdimage/lenny_live_beta1/
Please also have a look at the known issues listed below. Main features 100% Debian The build process of Debian Live basically consists of creating a Debian chroot, installing one or more kernels along with live-initramfs (a set of hooks into initramfs-tools for handling booting from read-only media) and generating a bootable image from that. This process is handled by live-helper, a collection of shell scripts that allow us to automate and customize this process. Considerable care is taken to ensure that the resulting live system is not tainted by the host system and that installed packages are not modified morethan absolutely necessary. This ensures that Debian Live really is Debian, and not "just another" a Debian-based live system. Flavours Although live-helper is a toolkit to produce your very own live systems with only a few steps, we also provide prebuilt images that are meant to be used as reference systems for end-users. Currently, this consists of the three major desktop environments (GNOME, KDE and Xfce), as well as a small 'standard' image without a graphical environment. For the desktop environments, package selection is performed by 'tasksel' with the respective desktop task, whilst the 'standard' image contains only packages of "Priority: standard" or greater, notwithstanding a handful of live-specific packages (console-common, eject, file, kbd, live-initramfs, locales, sudo and vim-tiny). Image types Debian Live offers prebuilt images for CD/DVD discs, USB sticks (or any HD-media-like device), tarballs (for PXE netboot) as well as a bare squashfs image to boot from the web directly. Live Magic Live Magic is an GUI frontend around the live-helper scripts, offering a subset of the features of live-helper in an easy-to-use graphical user interface. live-magic 1.0 was recently uploaded to sid and is the recommended version. It currently supports 7 languages. Live Installer Live Installer is a special udeb for the Debian Installer that (optionally) replaces a part of d-i in order to install the system from the live image instead of to bootstrapping it from .deb packages. This way, a live system can be easily installed to the harddisk, ensuring that the look and feel of the installation (including preseeding) works the same as the regular installer process. Unfortunately, live-installer does still have a few minor bugs left and is thus not included in our builds yet; we hope to be able to include it in the next beta. Known issues in this release Plans for next Beta release We are looking forward to upload Beta2 in about two weeks from now (maybefollowed by a third beta) with one final RC after that which should be identical to the final release. Some more details about the open things we would like to address in Beta2 and later can be found at the wiki page. The Debian Live team is still looking for more contributors for new features (post-lenny, though) as well as documentation writers for the manual (always). If you care about live systems, please join and help! Thanks Last but not least, our thanks goes to everyone who has contributed and to all maintainers that have kindly fixed live-specific bugs in their packages.

MJ Ray: Forthcoming Attractions: Software Cooperative Events for Late 2008

Some interesting events that I’ve heard about recently, with a bit of a focus on software, cooperatives and the South-West:- I’m pretty sure I’ll go to the *d ones - which others do you think I should go to? Will you be at any of these? Any comments to make on/to any of them? Any more to add to the list?

Ingo Juergensmann: re: Killing Servers with Virtualisation and Swap

Russel Coker blogs about a problem, which concerned me as well: Killing Servers with Virtualisation and Swap, i.e.: what happens when one domU is happily swapping the whole day?

Luckily this didn't happen to me yet, but as Russel do I tend as well to give a small sized swap area to domUs. If they're doing extensive swapping, something is wrong and most probably a task has a memory leakiing problem or the virtual machine is undersized with its RAM.

Anyway, Russel had some thoughts and ideas how to deal with the problem, such as giving dedicated swap areas on single disks to some domUs to seperate disk I/O for the filesystem and the I/O for swap. He proceeds with:

Now if you have a server with 8 or 12 disks (both of which seem to be reasonably common capacities of modern 2RU servers) and if you decide that RAID is not required for the swap space of DomUs then it would be possible to assign single disks for swap spaces for groups of virtual machines. So if one client had several virtual machines they could have them share the same single disk for the swap, so a thrashing server would only affect the performance of other VMs from the same client. One possible configuration would be a 12 disk server that has a four disk RAID-5 array for main storage and 8 single disks for swap. 8 CPU cores is common for a modern 2RU server, so it would be possible to lock 8 groups of DomUs so that they share CPUs and swap spaces. Another possibility would be to have four groups of DomUs where each group had a RAID-1 array for swap and two CPU cores.


I would myself consider a different approach:


Of course you can mix up both approaches or even use swap over NFS or NBD/DRBD. There are many ways and possible solutions, but the best way to deal with swapping is to take care that it doesn't happen, of course... ;)

26 August 2008

Russell Coker: Killing Servers with Virtualisation and Swap

The Problem: A problem with virtual machines is the fact that one rogue DomU can destroy the performance of all the others by inappropriate resource use. CPU scheduling is designed to allow reasonable sharing of computational resources, it is unfortunately not well documented, the XenSource wiki currently doesn’t document the “credit” scheduler which is used in Debian/Etch and CentOS 5 [1]. One interesting fact is that CPU scheduling in Xen can have a significant effect on IO performance as demonstrated in the paper by Ludmila Cherkasova, Diwaker Gupta and Amin Vahdat [2]. But they only showed a factor of two performance difference (which while bad is not THAT bad). A more significant problem is managing virtual memory, when there is excessive paging performance can drop by a factor of 100 and even the most basic tasks become impossible. The design of Xen is that every DomU is allocated some physical RAM and has it’s own swap space. I have previously written about my experiments to optimise swap usage on Xen systems by using a tmpfs in the Dom0 [3]. The aim was to have every Xen DomU swap data out to a tmpfs so that if one DomU was paging heavily and the other DomUs were not paging then the paging might take place in the Dom0’s RAM and not hit disk. The experiments were not particularly successful but I would be interested in seeing further research in this area as there might be some potential to do some good. I have previously written about the issues related to swap space sizing on Linux [4]. My conclusion is that following the “twice RAM” myth will lead to systems becoming unusable due to excessive swapping in situations where they might otherwise be usable if the kernel killed some processes instead (naturally there are exceptions to my general rule due to different application memory use patterns - but I think that my general rule is a lot better than the “twice RAM” one). One thing that I didn’t consider at the time is the implications of this problem for Xen servers. If you have 10 physical machines and one starts paging excessively then you have one machine to reboot. If you have 10 Xen DomUs on a single host and one starts paging heavily then you end up with one machine that is unusable due to thrashing and nine machines that deliver very poor disk read performance - which might make them unusable too. Read performance can particularly suffer in a situation when one process or VM is writing heavily to disk due to the way that the disk queuing works, it’s not uncommon for an application to read dozens or hundreds of blocks from disk to satisfy a single trivial request from a user, if each of these block read requests has to wait for a large amount of data to be written out from the write-back cache then performance will suck badly (I have seen this in experiments on single disks and on Linux software RAID - but have not had the opportunity to do good tests on a hardware RAID array). Currently for Xen DomUs I am allocating swap spaces no larger than 512M, as anything larger than that is likely to cause excessive performance loss to the rest of the server if it is actually used. A Solution for Similar Problems: A well known optimisation technique of desktop systems is to use a separate disk for swap, in desktop machines people often use the old disk as swap after buying a new larger disk for main storage. The benefit of this is that swap use will not interfere with other disk use, for example the disk reads needed to run the “ps” and “kill” programs won’t be blocked by the memory hog that you want to kill. I believe that similar techniques can be applied to Xen servers and give even greater benefits. When a desktop machine starts paging excessively the user will eventually take a coffee break and let the machine recover, but when an Internet server such as a web server starts paging excessively the requests keep coming in and the number of active processes increases so it seems likely that using a different device for the swap will allow some processes to satisfy requests by reading data from disk while some other processes are waiting to be paged in. Applying it to Xen Servers: The first thing that needs to be considered for such a design is the importance of reliable swap. When it comes to low-end servers there is ongoing discussion about the relative merits of RAID-0 and RAID-1 for swap. The benefit of RAID-0 is performance (at least in perception - I can imagine some OS swapping algorithms that could potentially give better performance on RAID-1 and I am not aware of any research in this area). The benefit of RAID-1 is reliability. Now there are two issues in regard to reliability, one is continuity of service (EG being able to hot-swap a failed disk while the server is running), and the other is the absence of data loss. For some systems it may be acceptable to have a process SEGV (which I presume is the result if a page-in request fails) due to a dead disk (reserving the data loss protection of RAID for files). One issue related to this is the ability to regain control of a server after a problem. For example if the host OS of a machine had non-RAID swap then a disk failure could prevent a page-in of data related to sshd or some similar process and thus make it impossible to recover the machine without hardware access. But if the swap for a virtual machine was on a non-RAID disk and the host had RAID for it’s swap then the sysadmin could login to the host and reboot the DomU after creating a new swap space on a working disk. Now if you have a server with 8 or 12 disks (both of which seem to be reasonably common capacities of modern 2RU servers) and if you decide that RAID is not required for the swap space of DomUs then it would be possible to assign single disks for swap spaces for groups of virtual machines. So if one client had several virtual machines they could have them share the same single disk for the swap, so a thrashing server would only affect the performance of other VMs from the same client. One possible configuration would be a 12 disk server that has a four disk RAID-5 array for main storage and 8 single disks for swap. 8 CPU cores is common for a modern 2RU server, so it would be possible to lock 8 groups of DomUs so that they share CPUs and swap spaces. Another possibility would be to have four groups of DomUs where each group had a RAID-1 array for swap and two CPU cores. I am not sure of the aggregate performance impact of such a configuration, I suspect that a group of single disks would give better performance for swap than a single RAID array and that RAID-1 would outperform RAID-5. For a single DomU it seems most likely that using part of a large RAID array for swap space would give better performance. But the benefit in partitioning the server seems clear. An architecture where each DomU had it’s own dedicated disk for a swap space is something that I would consider a significant benefit if renting a Xen DomU. I would rather have the risk of down-time (which should be short with hot-swap disks and hardware monitoring) in the rare case of a disk failure than have bad performance regularly in the common situation of someone else’s DomU being overloaded. Failing that, having a separate RAID array for swap would be a significant benefit. If every process that isn’t being paged out could deliver full performance while one DomU was thrashing then it would be a significant benefit over the situation where any DomU can thrash and kill the file access performance of all other DomUs. A single RAID-1 array should handle all the swap space requirements for a small or medium size Xen server One thing that I have not tested is the operation of LVM when one disk goes bad. In the case of a disk with bad sectors it’s possible to move the LVs that are not affected to other disks and to remove the LV that was affected and re-create it after removing the bad disk. The case of a disk that is totally dead (IE the PV header can’t be read or written) might cause some additional complications.

Clint Adams: Uncle Orbison singing for Costello

The loud girl wasn't there. The dynamic boy missed her only in the most tangential sense. As the radio plays. The flighty girl missed her when she wasn't getting enough attention, for the loud girl was disturbingly good at giving her attention. I just can't face myself alone again. The weary boy did not miss her, for he had peered into her soul and seen the evil fomenting within. Maybe we ain't that young anymore. Italian blared. You ain't a beauty but, hey, you're all right. The euphuistic boy was of three minds about the subject and he made no pr tense about it. Throw roses in the rain. The rude girl flip-flopped like a politician on speed. For a savior to rise from these streets. They maintained their course, with smut, standards, and debauchery. Is beneath this dirty hood. The weary boy invoked the power of pork, and the euphuistic boy invoked Chaim and his band of goyim. Let the wind blow back your hair. The flighty girl grew more insecure. We got one last chance to make it real. Oh, Thunderoos.

Clint Adams: ainda sua boca e sua bunda s o diferentes

[Christian Hug]

Gerfried Fuchs: scons annoys

scons claims to be a better replacement for make, or rather especially autofoo magic. Unfortunately, it isn't. To me a proper build system should definitely be able to clean up behind itself. The reasoning flying around for why scons isn't able to do so are quite hilarious, ranging from that it doesn't know what it generates (how does it generate them in the first place?) to that it's extensible and thus can't be done properly (then the extensions are broken and should add their clean informations in a hook or such, too). I haven't seen any valid reason for why it shouldn't be able to do so—yet we still seem to need to clean up cruft lying aroud like .scons* files and directories, config.log and of course the build/ directory. People, if you really want to do some proper build system, don't forget to make it clean up after itself. It shouldn't be the requirement of application developers to fix that (which doesn't really work because a scons target trying to clean the files makes scons crash).

Jonathan McDowell: I'll have what she's having

pleia2 wants a new smartphone. Me too.The current options seem to be a Nokia E90 or an HTC Touch Pro. Both have disadvantages, but nothing else appears to offer a keyboard and decent screen. I'd quite like to have a play with a Touch Pro to see what it's like and if the interface is fast enough to not be annoying for normal phone use. I've also seen various snippets about the forthcoming HTC Dream Google Android based device suggesting it might be out in October. I'd been expecting it not to happen until next year, but October is tantalisingly close...

Neil Williams: Using Zenity to review (RC) bugs

I've written a simple perl script using SOAP to add some useful data to the RC bug list (or any list of bugs). It can easily be extended but right now it supports simple things like the date the bug was filed, short description of the package, the name of the maintainer, the most recent upload data, the debtags for the package, apt-cache policy data and whether the bug is already tagged pending.
Zenity presents this info in a question dialog, clicking OK loads the bug and the PTS in your browser using 'sensible-browser'.
Clicking cancel offers a new zenity dialog asking for the next bug number. (Click Cancel there to end the script.)
Code: http://linux.codehelp.co.uk/rc-zenity.pl (GPLv3 or later)
POD: http://linux.codehelp.co.uk/rc-zenity.html (easily convertible into a manpage using pod2man)
If you like it, use it.
If lots of people like it, maybe it can go into an existing package.
If I've duplicated another script that I just didn't know about, tell me.
:-)
It's not exclusively for RC bugs, it could work with any bug list or be modified to only look for particular severity settings.
Thanks to Don Armstrong for making debbugs so accessible using SOAP. The debbugs SOAP implementation is just so straightforward with a whole load of useful data returned without needing repeated queries, it makes scripts like this really satisfying.

25 August 2008

Russ Allbery: Debian packaging and Git

I'm now using Git for nearly all of my Debian packaging except for some team-maintained packages and some packages where I'm also upstream and have not yet converted the repository. I've written up my workflow and related notes. I think I mentioned this previously in my journal, but somewhat in passing. Since changing ones packaging workflow and adopting the new revision control systems came up again on Debian Planet, and since I've made various modifications and updates, it's probably time for another mention. What I'm doing still doesn't maintain nice patches for pushing upstream that include complete revision history. I diff the feature branches against upstream to produce patches, which works reasonably well. Something like TopGit would probably be an improvement, but I haven't had a chance to experiment with this yet. The Shibboleth packaging team is using this repository format. The AFS and Kerberos packaging team is using a simplified version of it.

David Watson: Packaging workflow

After watching the Packaging with version control systems talk by Martin F. Krafft ( thanks debconf video team ). I really want to update my packaging workflow, Martin's workflow seems too complicated to me. I'd like to have a simplified version for my packages. The question is though which VCS do I choose, I have experience of bzr but have never used git or mercurial. Changing my packaging workflow is not something I want to be doing too often, so I would like to make the "right" choice from the outset. So to that end I will be trying each major Distributed VCS, and looking at the associated packaging helpers to decide on which to use. If you have any insight or opinions on this, please leave a comment or mail me.

Martin Michlmayr: initramfs-tools MODULES=dep default on ARM

Frans Pop reported recently that the initramfs did not fit in the flash of the QNAP TS-109 when you used LVM. We had problems with initramfs images that were too large before (on the NSLU2). By default, initramfs-tools uses the MODULES=most setting which puts all modules that might be of interest to booting a machine in the initramfs. This is a good idea on a PC where the hardware can change a lot, but it makes less sense on most NAS devices. Frans has now changed debian-installer so the MODULES policy can be chosen during the installation. His change allowed architecture specific defaults, so I've changed ARM to use MODULES=dep. debian-installer will now create the file /etc/initramfs-tools/conf.d/driver-policy with a MODULES setting that will override that from /etc/initramfs-tools/initramfs.conf. On QNAP devices, I intend to only support MODULES=dep installations, so please change your configuration if you've installed Debian already. On the NSLU2, MODULES=most will of course continue to be supported.

Jeff Licquia: Standards and Conversations, Part 1

So it looks like the project I’ve been laboring on has been getting some attention:
Ever thought it was difficult to write software for Linux? For multiple distros? InternetNews reports that the LSB is making a push for their next release (due out later this year) that should help make all that much easier.
They even link to our project status page. Cool! Of course, good publicity invites criticism. This time, there seem to be two themes. William Pitcock seems to have the most succinct summary:
To put things simply, the LSB sucks. Here s why:
  • The LSB spec depends on RPM. I mean, come on. Seriously. Why do they need to require a specific package manager? If package handling is really required, then why not create a simple package format that can be converted on demand into the system package format? Or why care about packages at all?
  • The LSB spec invents things without consulting distros. Like the whole init scripts thing. But that s not as bad as depending on RPM or requiring a specific layout.
(See also Scott James Remnant.) Let’s take this one part at a time. Today’s topic: packaging. Part of William’s problem may be that he doesn’t understand the spec. The LSB doesn’t require a specific package manager, or a specific package format. It doesn’t even require that the distribution be set up using package management at all! The spec only requires that LSB-compliant software be distributed so that any LSB-compliant distribution can install it. That could be tarballs with POSIX-compliant install scripts, an LSB-compliant install binary, a shar archive, a Python script with embedded base64 binaries, whatever. One of the options allowed is an RPM package, with a number of restrictions. The restrictions are key, because they effectively define a subset of RPM that acts as, to quote William again:
…a simple package format that can be converted on demand into the system package format…
The difference being, of course, that we didn’t reinvent the wheel and create our own; we used a popular format as the basis for ours. Scott raises another concern:
While much of the LSB can be hacked into a different distribution through compatibility layers and tools, such as alien, what ISV or other vendor wants to provide a support contract against a distribution that has such kludges?
I’m not sure if he’s referring specifically to packaging or to the standard in general. As regards packaging: the reason we specify a strict subset is because we can test that subset, and we’ve tailored it to the needs of tools such as alien. The theory goes that alien isn’t a kludge when it comes to LSB packages. But, as already mentioned, if vendors aren’t comfortable with supporting RPM, they have a number of other options. As it turns out, most of them are doing just that; the feedback we’re getting from most ISVs is that packaging (whether LSB-subset RPM, full RPM, or Debian) is just not worth the effort. Coming up: part 2…

Next.

Previous.