Search Results: "xam"

23 April 2007

Erich Schubert: SELinux status

If you are trying to setup SELinux, a must-know command for you is "sestatus".Example output:
SELinux status:                 enabled
SELinuxfs mount:                /selinux
Current mode:                   permissive
Mode from config file:          permissive
Policy version:                 20
Policy from config file:        refpolicy-strict
If it doesn't say "enabled", there is no way it will work. :-)If it says "Current mode: permissive", SELinux will not block any access (that's why it's called "permissive"). So if you still get an "access denied" error from your web server, your unix permissions are blocking the access.When asking for help, please include which policy you are running (most likely you'll be running 'refpolicy-targeted', since that is the most usable out of the box). Writing e.g. "sestatus says enabled/permissive/refpolicy-strict" is really helpful, and not much to type, is it?

Erich Schubert: SELinux and object orientation

In a previous post, I claimed that SELinux uses a kind of object-oriented approach, and that this has several advantages over path name based access.Since SELinux policy isn't actually a 'program', the term is somewhat surprising. It's just one fragement of "object oriented programming" that is available in SELinux, and actually a rather simple fragment. Also it's not how it's evaluated; but the OOP 'design pattern' of 'classes' is somewhat present in the SELinux reference policy. The syntax is kind of twisted, too.Let me give you a simple example policy fragment:
type mylog_t;
logging_log_file(mylog_t)
This fragment defines a new type, and 'inherits' from the 'log_file' class. (you can think of "logging_log_file" as com.tresys.oss.refpolicy.system.logging.log_file if you are a Java freak)The logging_log_file 'class' (which technically is a M4 macro, and maps to the SELinux 'typeattribute' 'logfile' - this would be called an abstract class in other programming languages) in turn will make mylog_t also be a "file_type" type attribute.So above line means that the new mylog_t type will 'inherit' certain access rules. Namely those defined via the logfile type attribute or file_type.This means that I can write policy to grant access on "any file" (e.g. for security scanners such as 'samhain' or the SELinux relabeling utilities, backup application, etc.) or just to any log files (e.g. for logrotate). Independend of where they live or who wrote the policy for them.So if I write policy for a custom application, add mylog_t for files living somewhere in /home/myapp/logs, I don't need to modify the logrotate policy, because that policy contains:
logging_manage_all_logs(logrotate_t)
(You can again think of this as logrotate being derived from a "log file manager" class, though it's implemented differently this time. Also note that multiple inheritance is no problem in SELinux policy)Macro names in the reference policy usually start with a namespace prefix. In this example, it's logging_. Since it uses M4 for processing the files, the syntax is largely determined by what is easy to use in M4.

Russell Coker: lemonup and blog license

I have just updated my previous post about licenses and also explicitely licensed my blog. Previously I had used a Creative-Commons share-alike license for lecture notes to allow commercial use and had not specified what the license is for my blog apart from it being free for feeds (you may add it to a planet without seeking permission first).

Unfortunately the operators of a site named lemonup.com decided to mirror many of my blog posts with Google AdWords. The site provides no benefit to users that I can discover and merely takes away AdWords revenue from my site. It has no listed method of contacting the site owner so it seems that blogging about this and letting them read it on their own site is the only way of doing so. :-#

I'm happy for Technorati to mirror my site as they provide significant benefits to users and to me personally. I am also happy for planet installations that include my blog among others to have a Google advert on the page (in which case it's a Google advert for the entire planet not for my blog post).

Also at this time I permit sites to mirror extracts of my articles. So for example the porn blogs that post paragraphs of my posts about topics such as "meeting people" with links to my posts don't bother me. I'm sure that someone who is searching for porn will not be happy to get links to posts about Debian release parties etc - but that's their QA issue not a license issue. I am aware that in some jurisdictions I can not prevent people from using extracts of my posts - but I permit this even in jurisdictions where such use is not mandated by law.

Lemonup: you may post short extracts (10% or one paragraph) of my posts with links to the original posts, or you may mirror my posts with no advertising at all. If those options are not of interest to you then please remove all content I wrote from your site.

22 April 2007

Masayuki Hatta: GNU a2ps 4.13c-rc4 released

GNU sysadmins gave me permission some days ago, so I uploaded my work to alpha.gnu.org for the first time. You can obtain it from:
ftp://alpha.gnu.org/gnu/a2ps/a2ps-4.13c-rc4.tar.gz
I uploaded new Debian a2ps package to ftp-master, too. Let's see what buildds are gonna say... BTW, it seems Gentoo people have already "released" 4.13c (for example, look here) somehow. Hmm...I might have to number the new one as 4.14.

Tag this post with del.icio.us!

Michael Prokop: Booting from USB pen: troubleshooting and pitfalls

As grml supports booting from usb devices (harddisk installation via grml2hd as well as booting from usb pens via grml2usb) and I helped many people at debugging booting problems I think I stumbled upon every existing pitfall. 8-) There are several situations where booting might fail. So here we go with a summary of the most common pitfalls and possible solutions:
  1. The Bootsplash is displayed, the kernel loads but you very soon get a:
  2. ran out of input data
    System halted
    
    Reason: everything OK, except for the filesystem used on your usb device. So instead of fat16 you use for example fat32. Fix: use the appropriate filesystem (fat16 for usb pens usually).
  3. When trying to boot from USB you get something like:
  4. Invalid operating system
    
    Possible reason: the partition layout is not ok. Very probably there’s no primary partition or none has the flag ‘bootable’ set. (Read on for further details.)
  5. When trying to boot from USB you get something like:
  6. No operating system found.
    
    Possible reason: you forgot to set the boot-flag on the partition. Or you did not install an operating system at all. ;-)
  7. When trying to set up the partition(s) on the USB device cfdisk reports something like:
  8. FATAL ERROR: Bad primary partition 0: Partition ends in the final partial cylinder
    
    Reason: the partition layout is not ok. Either use fdisk (requires detailed knowledge about partition setup!) or better: use a fronted of libparted, like gparted. gparted does an excellent job at setting up an appropriate partition layout for booting from USB (read on for details about the correct setup). As long as cfdisk has problems with the layout - it will exit on errors - your usb pen very probably does not have a valid partition setup for booting. Please notice that fdisk very often uses defaults which are not OK, whereas gparted uses the appropriate heads, sectors/track and cylinders settings. So if you really want to use fdisk for setting up the appropriate partition layout you might have to enter the ‘extra functionality (experts only)’ menu of fdisk.
  9. The system never displays the bootsplash of the usb device at all? Checklist:
    • does your system support booting from USB at all? Check settings in BIOS.
    • do you use the correct USB mode in BIOS? Check BIOS again, usually you want to use USB-HDD mode. Not possible? See USB-ZIP mode.
    • is "booting from USB" in the appropriate order of the booting sequence? Check the "boot"-section in your BIOS.
    • is the bootable flag set on the boot-partition? Check for example with cfdisk.
    • is the partition used for booting the first partition and a primary one on your device? Check with cfdisk again.
    • is the partition layout ok? The first partition (the one used for booting) has to start at cylinder 1/sector 63, otherwise booting will fail on most (all?) x86 systems. Check with ‘cfdisk -P s /dev/..’ or ‘fdisk -l -u /dev/…’. Do *not* trust the output of ‘fdisk -l /dev/…’. Really.
  10. syslinux does not work on your system (b0rken BIOS for example)? Possible solution: use grub instead.
  11. Kernel boots but fails to find the root filesystem with something like:
  12. kernel-panic: unable to mount root-fs...
    
    Yes? Then the root= argument in your kernel commandline (specified in the bootloader, usually lilo, grub or isolinux/syslinux) is pointing to the wrong device. Solution: adjust the root=-option in your bootmanager’s configuration. Tip: use root=UUID instead of root=/dev/[hs]d…
  13. All of the above tips don’t work? Maybe your USB pen is even larger than 1GB?
  14. Possible reason: Some BIOSes offer different modes for USB booting. The proper mode to boot a USB stick is USB-HDD. If that doesn’t work or is not supported by your system, you need to format your USB-Stick as USB-ZIP. To do this, the syslinux source distribution contains an utility called mkdiskimage, which you can use to re-format your USB stick in USB-ZIP format. For more information refer to the USB-webpage in the grml-wiki and Bootable USB Key in the knoppix-wiki.
I mentioned the correct partition layout several times. Reminder: a first, primary partition with fat16 (as filesystem as well as partition’s system id!) with bootable-flag set. The partition should start behind MBR at 1st cylinder and 63rd sector (see www.ata-atapi.com/hiwmbr.htm and thestarman.pcministry.com/asm/mbr/index.html for more details about MBR). So how does a correct partition look like? Output in several formats as reference:
# fdisk -l -u /dev/sda
Disk /dev/sda: 1039 MB, 1039663104 bytes
255 heads, 63 sectors/track, 126 cylinders, total 2030592 sectors
Units = sectors of 1 * 512 = 512 bytes
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63     2024189     1012063+   6  FAT16
# fdisk -l /dev/sda
Disk /dev/sda: 1039 MB, 1039663104 bytes
255 heads, 63 sectors/track, 126 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         126     1012063+   6  FAT16
# sfdisk -d /dev/sda
# partition table of /dev/sda
unit: sectors
/dev/sda1 : start=       63, size=  2024127, Id= 6, bootable
/dev/sda2 : start=        0, size=        0, Id= 0
/dev/sda3 : start=        0, size=        0, Id= 0
/dev/sda4 : start=        0, size=        0, Id= 0
Tip: if you have a working stick, store your partition layout using ’sfdisk -d /dev/sda > sfdisk.sda’ for later restore (via ’sfdisk /dev/sda < sfdisk.sda’). This is also very useful for cloning a working setup to other devices (though you might have to adjust cylinder/units later on). And of course you can use it as backup if you ran [sc]fdisk-commands on the wrong device. ;-)

21 April 2007

Erich Schubert: More AppArmor FUD^W PR:

I didn't intend to write (ok, rant...) on this topic again, but a few days ago, this "AppArmor FAQ", was posted to the Linux Kernel Mailing list. Again, this is a lot of 'marketing stuff', and doesn't have the facts quite right...Some quotes:
In the traditional UNIX model, files do have names but not labels, and applications only operate in terms of those names.
No. In the traditional UNIX model, filenames point to inodes, and the inodes contain data such as file modification dates and permissions. This is where SELinux stores the labels it uses for access control. In fact, the "unix owner" and "unix group" are pretty much the same thing as SELinux labels. And also work pretty much the same way.
Contrary to popular belief, AppArmor is *not* "Pathnames R Us", but rather "Use native abstractions to mediate stuff": when you mediate something, you should use the native syntax that users normally use to access the object.
Maybe the appropriate term would have been "naive abstractions"?And, no, security systems shouldn't be designed around what a user can easily grasp, because that might just not work... it should be designed around the way resources are handled and how you can successfully restrict access."Contrary to popular belief", SELinux *does* use path names. Actually in a very convenient way for the users. So I wouldn't claim SELinux is really harder to use than AppArmor because of its use of path names.The difference is that SELinux doesn't rely on the path names, but it still gives the user the possibility to use them. In fact, the only way I know to get a proplerly labeled filesystem is based on path names.If I'd for example keep my web stuff in /home/www, I can tell SELinux that the appropriate label is httpd_sys_content_t by doing
semanage fcontext -a -t httpd_sys_content_t /home/www
restorecon -R /home/www
In AppArmor, I'd need to modify the policy for several applications to achieve this, because of it's dependency on path names. SELinux also uses an abstraction. One that is very convenient for users actually. SELinux does access control on object classes. What I'm doing in above example is to put these files into the "Web site data files" class. So please stop scaring people of "labels". That is just the technical term for what is an "object class", a concept every developer will be familiar with. (SELinux policy development has a couple of OOP-like features, btw., for example it is possible to give an application access to 'all log files'.)There are, however, some true statements hidden in there:
We also acknowledge that pathname based access control requires a way to perform pathname matching in the kernel, and this comes at a cost higher than comparing object labels [...]
So by providing string matching in the kernel, AppArmor trades run-time performance to grant reduced administrative work. It is however not obvious how large the difference in "reduced administrative work" is between AppArmor testing the filename "run-time" compared to SELinux using filenames to relabel files "on demand" (or, with restorecond, even on creation by using inotify).What I still seriously dislike is the FUD spread by AppArmor people such as
Label-based security (exemplified by SELinux, and its predecessors in MLS systems) attaches security policy to the data. As the data flows through the system, the label sticks to the data, and so security policy with respect to this data stays intact. This is a good approach for ensuring secrecy, the kind of problem that intelligence agencies have.
Why should this only be useful to "intelligence agencies"? Some people live in countries with very strict privacy requirements, and we might want to use these techniques to lessen the risk of leaking data. In AppArmor, I can copy a file with private customer data to the web server directory, and it becomes readable by the web server. In SELinux, the file will still be unreadable by the web server. Does that really sound like "only useful to intelligence agencies" to you?Dear AppArmor people, please stop trying to invoke fear of the NSA, just to make people use AppArmor. That makes you look very untrustworthy, you know...The SELinux kernel code is part of the Linux kernel, AppArmor is not. So in theory, that code has been reviewed much better than yours, and is maybe even less likely to contain a "NSA backdoor" than yours. Changes to SELinux are openly discussed on a mailing list, and go through peer review there. They foolow a very strict coding style, which makes review easier; obfuscated code doesn't have a place there. It's not like we were running a binary blob from the NSA we can't have a look at.

Nico Golde: Errm Mark Shuttleworth...

Today I read an interview on Golem and there was a question about Debian's relationship to Ubuntu.

Mark Shuttleworth answer includes the following statement:
"Wir wollen so effizient wie m glich mit Debian zusammenarbeiten und investieren daher auch, beispielsweise indem wir alle an Ubuntu-Paketen durchgef hrten nderungen an den jeweiligen Debian-Maintainer zur ckgeben."

Which says that they want to co-work with Debian as efficient as possible and that's why they also invest in Debian for example with sending all the changes on Ubuntu packages back to the Debian-maintainer.

I am sorry, I never really said my opinion about Ubuntu in public since I think this wouldn't be really productive for the problem but that is just a lie in my opinion.
I won't state any further here but I never ever received a patch for the fetchmail package from Ubuntu.

Mike Hommey: svk shortcuts

J rg, you can significantly reduce the number of commands to type by using some of the useful svk shortcuts. First, you can create the mirror and check it out with a simple one liner:

svk cp svn+ssh://joerg@svn.debian.org/svn/debconf-data/ /chekout/directory
It will ask you the base URI to mirror (in case you’d actually want to mirror svn+ssh://joerg@svn.debian.org/svn/, for example), and then the depot path for the mirror and the local svk copy. The checkout will then be available in /checkout/directory. Then, to get updates from svn:

svk up -sm
And you can also push your changes into svn with

svk push

20 April 2007

Joerg Jaspert: Switching to bzr for &#8220;offline&#8221; edits considered - useless

Bastian describes his move to bzr, away from svn. Fine, use what you want, but please chose a better reason than one can do (small) local checkins and later a big commit or “like to have the whole repo stored locally”. :) You can do that with svn repositories too, using svk, and you wouldn t need to use a version control system that only scares people away. So, install svk. Then run the following commands for the svn repositories you want to sync (using an alioth.d.o one as example, replace the values as needed):
 svk mirror svn+ssh://joerg@svn.debian.org/svn/debconf-data/ //alioth/debconf-data
 svk sync //alioth/debconf-data
 svk copy -p //alioth/debconf-data //local/debconf-data -m "Copy to local"
Change to the dir you want the checkout in, run
 svk co //local/debconf-data
Later on you use the following commands to get updates from svn to your system:
 svk sync //alioth/debconf-data
 svk smerge //alioth/debconf-data //local/debconf-data -m "Merge to local"
 svk up
Obviously the svk up should run in the dir you checked it out earlier. Now work. Have fun, use svk instead of svn and do the stuff you would normally do, like svk add/remove files, commit changes, etc. Whatever. Changes wont go upstream, to get it synced you need to run
 svk smerge -l //local/debconf-data //alioth/debconf-data
which will get your changes propagated upstream, preparing a commit message for you that is based on your local logs. Oh, dont want to have one big commit upstream? Try svk help smerge and use -I. Instead of //alioth you can take whatever you want, just not //local, as we want to keep //local for those stuff that doesnt get committed upstream directly.

19 April 2007

MJ Ray: The Tragedy of Targets

I've started using the phrase "tragedy of targets" recently for those situations where someone does something stupid to meet an ill-specified target, instead of working towards the result in the way the target-setter expected. This gem was published by a Conservatives candidate over in my home county of Northamptonshire and was mentioned in today's Weston and Worle News
"an offender visited approximately 550 domestic properties in Weston-super-Mare with a fake comic relief sponsor form. This person obtained property by deception, (i.e. convinced them to part with money) from no less than 542 separate victims. Following good work from the police, he was caught, arrested, interviewed and I understand he will be cautioned for all the offences. My sources tell me that as a result of the arrest, the North Somerset District was or is planning to claim 542 'detections' for this one job. This will prove to be an amazing result for the bean-counters and has no doubt salvaged the whole 'performance target' result for the area in 2006. Indeed, the local Police Community Support Officers have spent the week going door to door in an attempt to locate every household that was conned, and get them to sign a pro-forma statement of complaint to bolster the clear-up rate"
Do you know other examples where setting a target has had unintended effects? Do you have a suggestion how to set better targets for the police here? Me, I like the suggestion of one anonymous commenter:
"The Police should be measured on the absence of crime not by detecting it."

Russell Coker: a Heartbeat developer comments on my blog post

Alan Robertson (a major contributor to the Heartbeat project) commented on my post failure probability and clusters. His comment deserves wider readership than a comment generally gets so I'm making a post out of it. Here it is:


One of my favorite phrases is "complexity is the enemy of reliability" . This is absolutely true, but not a complete picture, because you don't actually care much about reliability, you care about availability.

Complexity (which reduces MTBF) is only worth it if you can use it to drastically cut MTTR - which in turn raises availability significantly. If your MTTR was 0, then you wouldn't care if you ever had a failure. Of course, it's never zero

But, with normal clustering software, you can significantly improve your availability, AND your maintainability.

Your post makes some assumptions which are more than a little simplistic. To be fair, the real mathematics of this are pretty darn complicated.

First I agree that there are FAR more 2-node clusters than larger clusters. But, I think for a different reason. People understand 2-node clusters. I'm not saying this isn't important, it is important. But, it's not related to reliability.

Second, you assume a particular model of quorum, and there are many. It is true that your model is the most common, but it's hardly the only one - not even for heartbeat (and there are others we want to implement).

Third, if you have redundant networking, and multiple power sources, as it should, then system failures become much less correlated. The normal model which is used is completely uncorrelated failures.

This is obviously an oversimplification as well, but if you have redundant power supplies supplied from redundant power feeds, and redundant networking etc. it's not a bad approximation.

So, if you have an MTTR of 4 hours to repair broken hardware, what you care about is the probability of having additional failures during those four hours.

If your HA software can recover from an error in 60 seconds, then that's your effective MTTR as seen by (a subset) of users. Some won't see it at all. And, of course, that should also go into your computation. This depends on knowing a lot about what kind of protocol is involved, and what the probability of various lengths of failures is to be visible to various kinds of users. And, of course, no one really knows that either in practice.

If you have a hardware failure every 5 years approximately, and a hardware repair MTTR of 4 hours, then the probability of a second failure during that time is about .009%. The probability of two failures occuring during that time is about 8^10-7% - which is a pretty small number.

Probabilities for higher order failures are proportionately smaller.

But, of course, like any calculation, the probabilities of this are calculated using a number of simplifying assumptions.

It assumes, for example, that the probabilities of correlated failures are small. For example, the probability of a flood taking out all the servers, or some other disaster is ignored.

You can add complexity to solve those problems too ;-), but at some point the managerial difficulties (complexity) overwhelms you and you say (regardless of the numbers) that you don't want to go there.

Mangerial complexity is minimized by uniformity in the configuration. So, if all your nodes can run any service, that's good. If they're asymmetric, and very wildly so, that's bad.

I have to go now, I had a family emergency come up while I was writing this. Later...


End quote.

It's interesting to note that there are other models of quorum, I'll have to investigate that. Most places I have worked have had a MTTR that is significantly greater than four hours. But if you have hot-swap hard drives (so drive failure isn't a serious problem) then having machines have an average of one failure per five years should be possible.

Adam Rosi-Kessel: Adam s Super Simple Guide to mbox->maildir conversion

There are a lot of pages out there explaining how to convert your mail from mbox to maildir. I’m going to add one more, resurrecting the “tips” section of my blog. (All the tips from my blosxom days are grouped together on one page — someday I’ll get around to breaking those up properly.) If nothing else, I’ll never have to write these instructions in an email to someone again, instead I’ll just send them the link to this entry. I hope these steps are specific enough so anyone savvy enough to even have this problem in the first place will be able to follow them. One thing I find consistently concerns people when converting to maildir is how to make sure no mail is lost in the transition. The steps below address that concern. It’s really not hard at all once you understand how maildir works. First, I’m assuming you have mb2md installed and procmail support. Also, if you plan to access your mail through IMAP, your IMAP server needs to know to use ~/Maildir if ~/Mail does not exist. This is often default behavior (at least it is for Dovecot). Finally, I assume that your inbox is in /var/mail/username, and your other mail folders are in ~/mail. So, here goes:
  1. Edit your .procmailrc. The first two lines should read:
    MAILDIR=$HOME/Maildir
    DEFAULT=$MAILDIR/

    This will cause all new mail to be delivered (after you save your .procmailrc) to be stored in maildir format under ~/Maildir. The key for procmail to know to use the maildir format is the trailing slash in the “DEFAULT” line. Note that under the new system, your inbox and all your folders are kept in ~/Maildir, rather than having an inbox in /var/mail and your folders under your home directory. (This is nice because it makes it easier to back up all your mail together, among other things).
  2. If you have any specific procmail rules in your .procmailrc, you’ll also need to change the target folders for that. For example, if you have a spam filtering rule like this:
    :0 H
    * ^X-Spam-Level: *****
    spam
    It needs to be changed to this:

    :0 H
    * ^X-Spam-Level: *****
    .spam/

    The trailing slash, again, tells procmail to use maildir rather mbox format for delivery. The leading dot will make the folder appear properly for your IMAP client (there’s a trick I suggest below for making “dot” folders easy to access with a command line client like mutt).
If you have mail being delivered to a nested mail folder, an additional change is needed because maildir has a flat hierarchy. So if your mail was being delivered to “misc/other/spam” the new target should be “.misc.other.spam/”.
  • Save your .procmailrc. From this point forward, all mail will be delivered to your maildir folders. You can now import all your old mail to the new folders without concern. Since maildir stores each individual message as a separate unique file, you don’t need to worry about any overwriting issues while you are making the transition. New mail will be delivered to newly named files; old mail will be imported to other, unique, newly named files.
  • Import your mail folders:
    mb2md -s ~/mail -R
    The -R just means to import recursively through all your folders; if your mail folder is Mail rather than mail, use that instead. No destination is needed because mb2md uses ~/Maildir by default.
  • Import your inbox:
    mb2md -m
  • Move your old inbox out of the way. Most people won’t actually be able to move or delete files in /var/mail (since they don’t have directory write privileges), so the easiest thing is probably to just open the inbox in your IMAP client or with mutt and save or delete all the messages.
  • Move your old mail folder out of the way:mv ~/mail ,.bak Once you’ve confirmed that everything is okay, you should be able to delete ~/mail.bak (or compress it and move it elsewhere.) If you saved your inbox to mail folder in step 6, that will get moved out of the way here as well.
  • That’s it, you’re done.
  • Okay, so you’re not quite done if you access your mail with mutt rather than an IMAP client. (You could just point your mutt at your local IMAP server, of course). I’ve found the most convenient way to access the “dot” folders with mutt is to create a parallel symlinked folder structure without the dots. Here’s a hacked-together shell script that will do that. This could obviously be rewritten to be much more elegant (probably in perl). But we all have our messy scripts, right?

    #!/bin/bash
  • cd ~/Maildir for x in .*
    do
    y=$ x##.
    if ( echo $y grep -q ‘.’ )
    then
    mkdir -p ~/mymail/$ y%.*
    fi
    ln -s ~/Maildir/$x ~/mymail/ echo $y sed “s/.///g”
    done for x in *
    do
    ln -s ~/Maildir/$x ~/mymail
    done
    Then, just add these lines to your .muttrc file:
    set folder="~/mymail"
    set spoolfile=”~/mymail”
    Any questions?

    18 April 2007

    Russell Coker: 2 node vs 3+ node clusters

    A comment on my post about the failure probability of clusters suggested that a six node cluster that has one node fail should become a five node cluster.

    The problem with this is what to do when nodes recover from a failure. For example if a six node cluster had a node fail and became a five node cluster, then became a three node cluster after another two nodes had failed, then you would have half the cluster that was disconnected. If the three nodes that appeared to have failed became active again but unable to see the other three nodes then you would have a split-brain situation.

    As noted in the comment the special case of a two node cluster does have different failure situations. If the connection between nodes goes down and the router can still be pinged then you can have a split brain situation. To avoid this you will generally have a direct connection between the two nodes (either a null-modem cable or a crossover Ethernet cable), such cables are more reliable than networking which involves a switch or hub. Also the network interface which involves the router in question will ideally also be used as a method of maintaining cluster status - it seems unlikely that two nodes will both be able to ping the router but be unable to send data to each other.

    For best reliability you need to use multiple network interfaces between cluster nodes. One way of doing this is to have a pair of Ethernet ports bonded for providing the service (connected to two switches and pinging a router to determine which switch is best to use). The Heartbeat software supports encrypted data so it should be safe to run it on the same interface as used for providing the service (of course if you provide a service to the public Internet then you want a firewall to prevent machines on the net from trying to attack it).

    Heartbeat also supports using multiple interfaces for maintaining the cluster data, so you can have one network dedicated to cluster operations and the network that is used for providing the service can be a backup network for cluster data. The pingd service allows Heartbeat to place services on nodes that have good connectivity to the net. So you could have multiple nodes that each have one Ethernet port for providing the service and one port as a backup for Heartbeat operations, if pingd indicates that the service port was not functioning correctly then the services would be moved to other nodes.

    If you want to avoid having private Heartbeat data going over the service interface then in the two-node case you need a minimum of two Ethernet ports for Heartbeat and one port for providing the service if you use pingd. If you don't use pingd then you need two bonded ports for providing the service and two ports (either bonded or independently configured in Hertbeat) for Heartbeat giving a total of four ports.

    When there are more than two nodes in the cluster the criteria for cluster membership is that a majority of nodes are connected. This makes split-brain impossible and reduces the need to have reliable Ethernet interfaces. A cluster with three or more nodes could have a single service port and a single private port for Heartbeat, or if you trust the service interface you could do it all on one Ethernet port.

    In summary, three nodes is better than two, but requires more hardware. Five nodes is better than three, but as I wrote in my previous post four nodes is not much good. I recommend against any even number of nodes other than two for the same reason.

    17 April 2007

    MJ Ray: Webmastering: How fat is your feed? (2)

    Smigs commented:
    "perhaps the ultimate example of this is the ebay uk announcements feed - the items date back to 2004 and it totals 267.86 kB - so big that the livejournal syndicator refuses to use it at all. I went so far as to contact ebay about it, but they didn't do anything."
    I'm having a little problem getting some money from ebay (which I might write about after I resolve it), so the above comes as no surprise to me. That sounds like the Carlsberg of feeds: probably the fattest feed in the world. Martijn wrote:
    "I think a feed should contain all content the other versions of the page have, it's an alternative (link rel etc.) after all. Imho, parsers should just be made smarter, and drop things they consider to be 'too old'."
    I agree that a feed should be the same content as other versions (or at least the main one of them should... see my site's filtered feeds) but my comment about being too big and too old applies to the non-feed versions too: Recent Changes pages should drop items which aren't recent. I disagree that parsers should drop things that are too old because it would do more harm than good: generally, you can't tell how old an item claims to be until after you've read it; and a lot of publishers have subtly broken clocks (which is why my aggregator uses ordered lists, instead of sorting by timestamps like planet). Related link: Problogger: 34 Reasons Why People Unsubscribe from RSS feeds.

    Gunnar Wolf: On system init schemes

    I was delighted at finding Erich's series of posts regarding init schemes - In fact, when reading the bottom of your third message I got disappointing at you stating that this concludes this series of blog posts. Maybe someone can follow up with some details on upstart (which seems to be the most promising init replacement)? - I expected you to delve a bit into other schemes, such as file-rc (which is, AFAIK, quite similar to the standard sysv-rc, but instead of having directories with symlinks has files describing each runlevel in similar terms, and I understand works in quite a similar logic to sysv-rc), runit seems to work in a way quite similar to BSD's RC...
    I was particularly interested on your point of view on something like Init NG - And, of course, Upstart sounds sexy, and besides, being Scott's brainchild automatically stamps a must be good legend on it, at least for me. Shame that you didn't follow on with the series - but hey, nobody's paying you to do it ;-)
    Anyway... I was thinking on a way we could get at least Debian maintainers work more easily towards having more than one init scheme for our daemons and such - or at least, shipping less cruft for derived distributions to clean up. Specifically, yes, our most notorious descendant goes Upstart, so /etc/init.d/* becomes crufty for them. So, what if:
    This means, yes, a serious explosion of very small packages. But it also means less dependency on a specific maintainer to be sympathetic to yet-another-init-scheme. If I'm pushing superb-init, which does preemptive reverse dependency tracking and invention and works based on the oh-so-3v1l aptsh, and I'm having a hard time convincing anybody to update their packages to support my init scheme before it matures, I can at least come up with all the needed scripts to get my system usable and considerable for the masses. Sounds decent? Did I smoke too much crack? No, I haven't had my morning coffee yet, so I'll go grab some while this hits the planet.

    Stefano Zacchiroli: ocaml 3.10.0 beta in experimental

    OCaml 3.10.0 Beta in Experimental As per title. It isn't actually fully functional: Julien for example just noticed that ocamlc -version returns "3.10.0+beta" while the libdir is (correctly) /usr/lib/ocaml/3.10.0/, this means that practically all packages using the ocaml CDBS class can't be built properly against it. Still you can give it a try (if you love risk) and perhaps document yourselves on the new API of camlp4, which changed completely. For us it will be useful to see the result of experimental autobuilding ... I'm quite scared about that!

    16 April 2007

    Biella Coleman: On the Caveat, Better than Well, and HOT Latino Bodies

    So, yes these three topics, the caveat, the book Better than Well and Hot Latino Bodies are related. You just have to stick with this long post to find out why… *** As I progress slowly but surely with my book manuscript, I am really coming to see how a dissertation and book are quite different creatures. I think one of the most important and noticeable differences is that a book has a lot more short caveats and warrants than necessary in a dissertation. I think there are two main reasons for this. One of which is has to do with your committee members, the primary and (usually only readers) of the dissertation. They are a lot more prepared and adept to ingest complex ideas than lets say undergraduate students, because that is what they are trained to do and because most of them are much more familiar with your topic because they have been with it nearly as long as you have. In a dissertation you are also allowed to (and often expected) to go on and on, ad naseum, with your theoretical explanations that help substantiate what are otherwise shakier, initial claims. For various reasons, for a book, especially if you are not some FFT (Famous French Theorist), you are strongly encouraged to dump most of the theory in favor of providing a streamlined version (which really, is preferable of course, but extraordinarily hard to pull off). I have been thinking a lot of the caveat because I have just finished re-reading a book Better than Well that is not only fascinating in its own right but brings the caveat to a stunning art form. The author, Carl Elliot, is a philosopher/bio-ethicist and the topic of the book, broadly speaking, examines how the rise of new enhancement technologies (prozac, plastic surgery, sex change surgery) is bound tightly with longer-standing, distinctly American ideals, such as the autonomous, self-directed and authentic self. It is one of those rare books that can be read by your father, aunt and uncle, tossed over to a willing teenager, and assigned in all sorts of college courses and still manage to impress all sort of academics in all sorts of fields. Part of the reason for his broad appeal is because the book is thoughtful and clever and so chock-full of really interesting examples that you are hooked and want more of his tasty intellectual Kool-Aid. So while he has one main focus, which largely triangulates between enhancement technologies, selfhood, and consumerism, in the process of exploring them, you learn about a bunch of other really neat topics: suburbia, the history of cosmetics and childhood, odd social phobias, long-gone and culture- bound disorders like dissociative, fugue, amputee wanabee’s, extreme blushing, and so much more. Along with crystal clear writing, he also throws in some classically funny lines, my favorite one currently being: For better or worse, suburbia has come to stand for something than can be survived only with minor tranquilizers. Another reason he manages to pull this Houdini-like feat is because of his judicious and artful use of the caveat, which is really the only way he can bring forth complex ideas, in a fashion that is much more accessible than is usually done in a purely academic book. To take on example, when he introduces the usefulness theories of Thorstein Veblen, an economist usually known (and only barely), by academics, he opens in the following way, because in many ways, if you just decided to pick up a copy of Veblen, his style make strike outdated:
    Reading Veblen nearly a century after he wrote The Theory of the Leisure Class, it is not easy to know which parts of the book to take seriously. It comes off as equal parts intellectual theory, social satire, and crackpot polemic, (and goes on for a full more paragraph) and then says Where Veblen is prescient, however, is his sense that in a consumption economy, consumer goods would become markers of who we are. p. 103
    In this way he can say, ok Veblen is useful because of this specific reason” and yet communicate to his academic readers that he knows the limits of Veblen. No matter how much I love the book, and now matter how I think his use of the caveat is stunning, there are two problems I have with it. In one case, I think he fails to give one of the most important caveats. He paints a picture in which all of American society is ensnared in dominant social codes and mores (which somehow all point back to consumerism and capitalism and a desire to improve the self). While there are points he seems to back away from that sort of statement, and a few rare points where he ascribes his insights to the group I think he should mainly be sticking to white, liberal-leaning middle-class Americans I think there are more instances where he paints a picture of America as far more uniform than it actually is. According to his account, no one is immune to the forces he so eloquently writes of and so in the end the environmentalist activist, is as caught up in the traps of consumer life-style as is the investment banker on Wall Street. It lead hims to say such statements as
    Many Americans today learn who they want to be by listening to a Methodist minister or a civics teacher but by watching advertisements for The Gap.
    Ok while he bit about the civics teacher may be true, any consideration of lets, say… the religious right in this country, which, as we know from recent elections, don’t represent a teeny-tiny itsy-bitsy minority (and for a fascinating glimpse into the world, I would recommend Jesus Camp), would bring holes, and sizable ones, to that sort of statement. Many Americans do in fact listen to their minster. And this does not only help explain the deep divisions in this country, but I bet because they do listen to their pastors, their notions of the good, the self, etc, are going to be pretty distinct from those he describes (and gain see Jesus Camp to get at this point) It is not that the religious right exists outside of the web of consumerism we are all at least partially caught in, and indeed, a lot of the new Protestant religious movements here and elsewhere as Jean and John’s Comaroff’s work has shown can be all about securing a more robust middle class lifestyle. But we must remember that even something as powerful as consumer capitalism or dominant ideals of an authentic, beautiful self though powerful and more often than not work in concert with each other do not quite have the power to efface all meaningful difference between lets say a white, liberal middle class woman and let’s say, many Latinos, who, do, let’s not forget, comprise a huge portion of America. Many Americans have a very different picture of the ideal female body than the picture he explores, which is skinny and lanky and forever youthful (and hence the appeal of botox and lipo). Let me provide just one example drawn from the annual Puerto Rican day parade and this hold true for the one held either in NYC or Chicago. Along with a blizzard of Puerto Rican flags, what you may also notice is the abundance of really bright spandex being donned on ladies that are not by any standard of the word slim. I am sure that any middle class lady (you know, the type who spends 5 days of the week working out at the gym, wishing her thighs were just a little thinner), would feel morally repulsed in seeing that sort of image, that is if they even bothered to go to the parade. But among many Puerto Rican men (not all) a sexylicious and extra-curvy, meaty Puerto Rican woman, decked out in tight & bright spandex, will like bring on a loud HAY MAMITA, ven ac ”………………… which roughly translates into You are HOT… Like I want you NOW. (more…)

    Alastair McKinstry: Looking for Planets, by Alan Boss

    Looking for Earths: The Race to Find New Solar Systems is an excellent introduction book, as a scientific story of exploration and summary of the science of planetary discovery. Alan Boss is a planetary scientist and adviser to NASA on the search for extrasolar planets. He has had a front-row seat in the revolution over the last two decades that has led to the discovery planets outside our Solar system for the first time, leading to 215 known planets at last counting. The book is written in an first-person style that reflects the race to discover the first exoplanets in the 1990s. He describes historical searches for planets in the early 20th Century: the hunt for evidence of planets around Barnards Stara and the meticulous work over decades of Peter Van de Kamp, a good summary of the challenges in proving the case for planets in Astronomy, and explains the techniques used to determine other planets. The heartbreaking difficulties of proving the discovery are well expressed, as well as documenting the leaders of the field at the time. Along the way, he describes the discovery of the first Brown Dwarfs, the border cases between Stars and Planets, and the bureaucratic and other challenges involved in setting up large observatories such as the Keck interferometers capable of doing this work. This is ironic, as the first discoveries were actually made by ground-based observatories. Boss describes the race between the main teams, Geoffrey Marcy and Paul Butler of Lick observatory. The race was finally won by Michel Mayor and Didier Queloz of Geneva Observatory, who discovered the first extrasolar planet around a sun-like star, 51 Pegasi, by looking at radial velocity measurements of nearby stars. This turned out to be a close race, as the planet they had found had a short period of 4.23 days, implying the planet was what we now call a "hot Jupiter", orbiting very close into the Sun. Most teams had been looking for copies of our Solar system, looking for large planets such as Jupiter with orbits of around a decade, and had hence not started to analyse the data they had in hand, expecting to need years of measurements: examining the data showed that, had they analysed data in hand, they could have won. Looking for Earths does a good job of explaining the field of exoplanet research as it was evolving over Boss's careeer and particularly 1995-2005, with theoretical and computer models evolving as data arrived with new puzzles. In some places, the need to give a clear explanation of a topic such as the formation of planets from solar disks slows the pace of the biography, but the book does an excellent job of describing both the science and the life lived by those doing the work. Few books I've read have offered such a good summary of the field to date, along with a good bibliography and I would heartily recommend it both as an introduction to the field and as a model for science writing.

    15 April 2007

    Kai Hendry: Opensource Web applications

    olpc web browser pointing to Webconverger Whilst downloading, installing and running Debian packages is easy, fun and “free”, most Web applications aren’t. I can’t be the only one that thinks there is a problem somewhere as for example, where are the all the Web applications listed on Freshmeat? I have an idea for a Web application though I thought it would be cool to make it as Free software though it can be a pain. I think there is a few more “barriers to entry” problems running a Web application as a free software project. Web hosting is generally quite poor and a lot of the Web centric deployment/packaging tools aren’t up to scratch. Though there are a couple of amazing examples of “open source” Web sites, like the stuff My Society do. Nonetheless I would like to see Debian as an even better Web application platform.

    Kai Hendry: Opensource Web services

    olpc web browser pointing to Webconverger Whilst downloading, installing and running Debian packages is easy, fun and “free”, most Web application services aren’t. I can’t be the only one that thinks there is a problem somewhere as for example, where are the all the free software Web services listed on Freshmeat? I have an idea for a Web application though I thought it would be cool to make it as Free software though it can be a pain.
    1. Getting a reliable host
    2. Host has to be Debian and probably unstable to get the bleeding edge stuff :)
    3. You probably want to have root, hence dedicated
    4. It is probably expensive and you might have to watch bandwidth
    5. You have to create accounts, where your colleagues also want root.
    6. You could use sudo though you’ll probably get frustrated with it and end up with “ALL = NOPASSWD: ALL” and no one checks the logs ;)
    7. You’re probably want Xen and a few IPs
    8. Since Web application packaging with current tools is near impossible with all the different httpds and DBMSes, you’ll probably have to use some odd hand crafted setup.
    9. You will probably want a DBA sooner or later, to figure WTF is going on with your DBMS ;)
    10. You’ll probably end up with a several step build&deployment process (that’s bad)
    11. Testing (if you ever get that far) will probably be a kludge
    12. To lower the barrier of entry, you will probably go for PHP which is great to deploy. Scary to secure.
    13. Though non-PHP avenues will probably require FastCGI. And FastCGI on Apache last time I checked is PAIN.
    14. Your developers probably can’t run the Web application locally as it such a pain to setup (esp. forget offline)
    15. You might have some “privacy policy” that makes sharing log or database data difficult amongst developers.
    16. Your Web application will probably need mail server access and all the pain that comes with that.
    17. Your application will have to deal with difficult abuse problems like spam that ordinary applications don’t usually deal with.
    18. If your project’s site becomes popular you’ll have a ton of maintenance and scalability head aches that will probably require money to be thrown at the problem.
    19. Someone will think your Web design stinks. Then if you use some “designer”, he creates some awful XHTML template and some fixed px style sheet with images replacing text. ;)
    I think there is a few more “barriers to entry” problems running a Web application service as a free software project. Web hosting is generally quite poor and a lot of the Web centric deployment/packaging tools aren’t up to scratch. Though there are a couple of amazing examples of “open source” Web services, like the stuff My Society do. Any other good examples? Nonetheless I would like to see Debian serve as an even better Web application platform. Update: I think “services” makes more sense than “application” in this context… I hope you’ll agree.

    Next.

    Previous.