Search Results: "ed"

6 August 2010

Thomas Koch: Free Software in companies

At work we currently have a discussion: I'd like to develop some components as Free Software. My bosses like the idea, but the client goes nuts only thinking about it.
So as part of the discussion I thought to collect those companies who actively advertise their Free Software. I know about the big ones, but it would be even more interesting to get a list of succesful small companies that share at least part of their inhouse projects. On the other hand it would not be too interesting to list full Free Software companies like Red Hat.
It would also be fine to share this list in some wiki (FSFE?) when it grows larger. For some companies I also list very popular projects from my point of view. big companies
  • Adobe
  • Facebook: Cassandra, Thrift, Scribe, HipHop for PHP, ...
  • Yahoo: YUI, Design Pattern Library, YSlow, Hadoop, ZooKeeper
  • Twitter
  • Google: Chromium Browser, Google Summer of Code, Android, Google Web Toolkit (GWT), Protocol Buffers - data interchange format, Java Collections Library, unladen-swallow - faster implementation of Python, ...
  • IBM: Developerworks (Tutorials), Eclipse, International Components for Unicode (ICU)
middle sized companies small companies

Update: Need to add Danga (Gearman), Liip.ch with Okapi, Flux CMS, Jackalope

David Welton: Geek Dad: "Skywalker Story"

I don't usually talk about personal stuff here, but I'll make an exception: I got tired of telling our two year old daughter the same old stories about children lost in forests, bears, witches, evil stepmothers and so on, so I turned to some other material I know fairly well. I started recounting simplified bits and pieces of the first Star Wars movie over the course of a week. She thought it was great fun, what with wookies and robots and a princess and lots of adventures, and now, every evening, asks for "Skywalker story!". I'm quite proud, although my wife rolls her eyes a bit.

John Goerzen: Really Dark Blue

As we were walking home tonight, Jacob started singing, over and over:
Really dark blue,
really dark blue,
really dark blue
The tune changed a bit, but the words didn t. Terah and I wondered what was really dark blue, until finally we heard:
Really dark blue,
really dark blue,
really dark blue sky. Really dark blue
Then a little while later, we heard:
Really dark blue,
really dark blue,
really dark blue with morning stars.
It was such a cute moment that it s hard to convey it in words. I even got an audio recording of it on my phone, which I ll perhaps post someday. Update 21:05: For Jacob s usual bedtime song tonight, I offered to sing the really dark blue song for him. When I sang really dark blue with morning stars, he said, no, that s not it. It s MOONLIT stars. So I guess my 3-year-old just corrected my blog post. I see a career in copy editing in his future

5 August 2010

Dirk Eddelbuettel: RcppArmadillo 0.2.5

Hot on the heels of RcppArmadillo release 0.2.4 a few days ago comes a new release 0.2.5 which is now on CRAN. RcppArmadillo makes it easy to write highly efficient and highly readable C++ code for linear algebra (based on Armadillo) in R extensions (using Rcpp for the interface). This release upgrades the included Armadillo version to Conrad's just-released version 0.9.60. This overcomes some of minor issues we had with 'older' compilers such as g++ 4.2.x with x being 1 or 2. No other changes were made from our end. The short NEWS file extract follows:
0.2.5   2010-08-05
    o   Upgraded to Armadillo 0.9.60 "Killer Bush Turkey"
More information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

Joey Hess: announcing Branchable

Time to introduce what I've been working on for these past too many months.. Branchable is an Ikiwiki hosting site, making it very easy to create blogs and wikis. The name "Branchable" was chosen because with its git backend, and ikiwiki-hosting frontend, sites hosted there can be easily branched by their owner (or by anyone, if the owner allows). branched site in control panel The ability to take your data and configuration and go home, as well as everything being free software, means Branchable follows the vision of the Franklin Street Statement on Freedom and Network Services, like identi.ca and last.fm. This is very important to me and to my cofounder, Lars Wirzenius. Branchable has a one month free trial, so if you've been meaning to do something with Ikiwiki, I hope you'll sign up. Also, we are offering free hosting for Free Software projects that sign up while Branchable is in beta.

Christian Perrier: +1 to languages of India in Debian Installer

After Arjuna Rao Chavala successfully completed enough of D-I translation, I just activated support for the Telugu language in Debian Installer. Given the currently planned released, that will be available in the next release, so hopefully for Squeeze. Telugu is spoken in Andhra Pradesh, one of the largest states in India, on the south-eastern coast of the country. The main city of Andrha Pradesh is Hyderabad, the 6th largest city of India. Telugu is the third most-spoken language in India, with about 74 million native speakers. As of now, out of the 29 languages in India with over 1 million speakers, we do support 9 of them that represent over 70% of the population of India. We indeed now support the five most spoken languages in India (of course, all this by excluding English).

Andreas Barth: RFH: buildd and multiple builds in parallel

On some machines, we have enough cpus to run more than one build in parallel. However, as of now, this isn't supported by buildd. What needs to be done is to change the following code to have up to n builds happening in parallel (including of getting the next package if there is a free slot).
    while( 1 )  
        $self->check_restart();
        $self->read_config();
        my ( $dist_config, $pkg_ver) = get_next_REDO($self);
        $self->do_build( $dist_config, $pkg_ver) if $pkg_ver;
        next if $pkg_ver;
        ( $dist_config, $pkg_ver) = get_next_WANNABUILD($self);
        $self->do_build( $dist_config, $pkg_ver) if $pkg_ver;
        next if $pkg_ver;
        # sleep a little bit if there was nothing to do this time
            $self->log("Nothing to do -- sleeping " .
                       $self->get_conf('IDLE_SLEEP_TIME') . " seconds\n");
            my $idle_start_time = time;
            sleep( $self->get_conf('IDLE_SLEEP_TIME') );
            my $idle_end_time = time;
            $self->write_stats("idle-time", $idle_end_time - $idle_start_time);
     
Any takers? The full code is available from git://git.debian.org/buildd-tools/sbuild.git in the file lib/Buildd/Daemon.pm (and I'm happy to try out appropriate patches).

Alexander Reichle-Schmehl: DDs, please feel free to commit to the DPN directly

The svn repository used to draft the Debian Project News just passed the sixth hundreds commit. If my calculations are correct, that's quite more than the number of edits done, back when the DPN was drafted in the wiki. I take it as a hint, that the new work flow works better than the old one. However, playing a bit with statistics (done by horrible shell one liners ;) I noticed, that - while every Debian Developer may commit to the subversion repository - not that many commits from other DDs where made: Of the 602 commits so far, only 70 where done by other DDs:
alex@melusine:~$ svn log --xml svn://svn.debian.org/svn/publicity \
 xmlstarlet sel -t -m "/log/logentry/author" -v "concat(.,' ')"   \
sed -e "s/ /\n/g" sort uniq -c sort -n grep -v guest grep -v tolimar
      1
      1 abe
      1 hertzog
      1 holger
      1 mika
      1 paravoid
      2 pabs
      3 alfie
     15 zobel
     20 gio
     25 spaillard
To give credit where credit is due, here are the guest commits:
     16 tpeteul-guest
     21 jeremiah-guest
     25 gmascellani-guest
     36 madamezou-guest
    122 taffit-guest
Many thanks so far, but I would like to advertise the Debian Project News and invite every DD to help us and to commit directly. It's quite easy: Run svn co svn+ssh://svn.debian.org/svn/publicity/dpn/en/current, edit the index.wml file, and then commit your changes back to the repository. (Well, ideally you would also honor Status flag and won't commit, if it's not open-for-edit.) As for the format: While we indeed use wml, you just need some basic HTML knowledge. To add a paragraph just use the following with proper content:
<a name="X"></a>
<h2>Fancy title</h2>
<p>More details about the topic.</p>
Usually the articles are kind of sorted by importance, so unless you are pretty sure it might be the easiest to just add your article at the end of the regular news, just before the other news sections. Should you not be able to fill an entire paragraph, feel free to just a one or two sentences to the other news section (just add a <p>...</p> with your content to the end of the other news). Don't worry about style, your English or syntax: It won't end up on the web page directly, it's reviewed and checked, so you can't do anything wrong. More details are available in the wiki at http://wiki.debian.org/ProjectNews/HowToContribute. Feel free to ask any questions unanswered on the publicity list at debian-publicity@lists.debian.org. Feel also free to contact us there, if you would like to help by translations or reviews. The more people help, the less work it's for everyone :) If you are not a Debian Developer, but still would like to help us, it's no problem. All you need is an account on our Alioth System (very easy to get, you just need to ask) and request to join the publicity project. See http://wiki.debian.org/ProjectNews/HowToContribute#Becominganeditoryourself for the respective links.

Matt Zimmerman: DebConf 10: Day 3

How We Can Be the Silver Lining of the Cloud (Eben Moglen) Eben s talk was on the same topic as his Internet Society talk in February, which I had downloaded and watched some time ago. He challenges the free software community to develop the software to power the freedom box , a small, efficient and inexpensive personal server. Such a system would put users more in control of their online lives, give them better protection for it under the law, and provide a platform for many new federated services. It sounds like a very interesting project, which I d like to write more about. Statistical Machine Learning Analysis of Debian Mailing Lists (Hanna Wallach) Hanna is bringing together her interests in machine learning and free software by using machine learning techniques to analyze of publicly available data from free software communities. In doing so, she hopes to develop tools for studying the patterns of collaboration, innovation and other behavior in these communities. Her methodology uses statistical topic models, which infer the topic of a document based on the occurrence of topical words, to group Debian mailing list posts by topic. Her example analyzed posts from the debian-project and debian-women mailing lists, inferring a set of topics and categorizing all of the posts according to which topic(s) were represented in them. Using this data, she could plot over time the frequency of discussion of each topic, which revealed interesting patterns. The audience quickly zoned in on practical applications for things like flamewar and troll detection. Debian Derivatives BoF (Matt Zimmerman) I organized this discussion session to share perspectives on Debian derivatives, in particular how we can improve cooperation between derivatives and Debian itself. The room was a bit hard to find, so attendance was relatively small, but this turned out to be a plus. With a smaller group, we were able to get acquainted with each other, and everyone participated. Unsurprisingly, there were many more representatives from Ubuntu than other derivatives, and I was concerned that Ubuntu would dominate the discussion. It did, but I tried to draw out perspectives from other derivatives where possible. On the whole, the tone was positive and constructive. This may be due in part to people self-selecting for the BoF, but I think there is a lot of genuine goodwill between Debian and Ubuntu. Stefano Zacchiroli took notes in Gobby during the session, which I expect he will post somewhere public when he has a chance.

Benjamin Mako Hill: My August

I've got a pretty packed August. I just wrapped the Open and User Innovation Conference at MIT -- the academic conference on user and open innovation connected to my research. I organized the program and was MC for the 120+(!) talks and research updates on the program so it's a huge relief to see it come off successfully. On Thursday, August 5th (at 14:30 UTC) I'll be giving a talk on antifeatures at DebConf (the Annual Debian conference). It was accidentally listed as "Revealing Errors" until a few minutes ago -- sorry about that! It will be streamed live (details on the DC site) for those outside of New York City who might want to follow it. As soon as DebConf is done on August 8th, I'm going to head to Kor ula in Croatia to relax, read, and hopefully get a bit of research done, before I head off to Outlaws and Inlaws in Split on the 19th, a sort of piracy and (vs?) free software summit put on by mi2 connected to the recurring Nothing Will Happen where, from what I hear, quite a lot does. I'm going to have to leave Nothing Will Happen a little early to head to FrOSCon on the 21st where I'll be doing an antifeatures keynote again on the 22nd. I tend not to like to do the same talk too many times, or for more than a year, so this might be one of the last times I present on antifeatures in this form. After that, I'm going to head to Italy where I'll be between the 23rd and the 3rd of September. I'll fly and in and out of Rome and plan to spend some time in Rome, Tuscany, and Florence, but don't have a lot of set plans and might travel to Bologna or elsewhere. My schedule is pretty open. As always, I'm interested in meeting up for coffee or a drink with like-minded hackers, Wikipedians, researchers, activists, etc. If folks are interested in organizing talks or presentations, that sounds fun too. I'm keeping a brief description of my schedule updated alongside a bunch of ways to get in touch with me on my contact page. Don't hesitate to drop me a line!

4 August 2010

Michael Banck: 4 Aug 2010

Science and Math Track at DebConf10 This year's DebConf10 (which is great, by the way) at Columbia University, New York will feature Tracks for the first time. We had a Community Outreach track on Debian Day (to be continued by more awesome talks over the rest of the week), a Java track on Monday and an Enterprise track yesterday. Tomorrow, Thursday afternoon, the S cience and Math track (which I am organizing) will take place in the Interschool lab on level 7 of Schapiro Center. The Track will start at 14:00 with a short welcome from me, followed by presentations of debian-science by Sylvestre Ledru and debian-math by David Bremner. At 15:00, Michael Hanke and Yaroslav Halchenko will present their talk on "Debian as the ultimate platform for neuroimaging research". This will be followed at 16:00 by three mini-talks on "New developments in Science Packaging". Adam C. Powell, IV will talk about MPI, Sylvestre Ledru will present linear algebra implementations in Debian and finally Michael Hanke and Yaroslav Halchenko will discuss the citation/reference infrastructure. At the end of track, the annual debian-science round-table will happen at 17:00, where David Bremner (mathematics), Michael Hanke (neuro-debian), Sylvestre Ledru (debian- science/pkg-scicomp), Adam C. Powell, IV (debian- science/pkg-scicomp) and myself (debichem) will discuss matters about cross-field debian-science and math related topics. If afterwards there are still outstanding matters to be discussed, we can schedule ad-hoc sessions for science or math matters on Friday or Saturday. See you at the science track tomorrow!

Joey Hess: DebConf10 in the neghborhood

I took the train from Charlottesville to New York on Saturday. With this year's DebConf in the USA, I finally have a chance to experience the conference from a local perspective, without jetlag, or a language barrier. I'm still working on that. Part of me thinks I am in a foreign country here. :) Actually, the neighboorhood around Columbia University is one I have ties to. Here is my Mom's old apartment, a few short blocks from the main DebConf venue. And here is family friend Ruth Shereff's memorial garden in the Broadway median. DebConf has been outstanding, and very exhausting. Lars and I have been spending a lot of time here putting the final touches on something to be announced soon.

Enrico Zini: My rule to see if a framework is worth of attention

My rule to see if a framework is worth of attention I came up with a little rule:
In order to be worth of any attention, a framework must be stable enough that I can charge money to train people to use it.
This probably applies to other kinds of software stacks, libraries, development environments and, well, to most software applications. In the context of python web frameworks, this means that: In light of this rule, I regret to notice that I see very few python web frameworks worth of any attention.

Neil Williams: FreedomBox

Eben Moglen's talk at DebConf10 has inspired a flood of responses and activity at DebConf10. The original talk overran the allotted time within the venue and then outside the venue for something like an hour, HackLab discussions migrated to the single theme and brought in more and more people over lunch. When the Skype talk was left without a speaker in the afternoon, it morphed into a FreedomBox BOF with maybe 100 delegates planning and contributing to the ideas and another BOF is planned for Friday. A rough outline is appearing on the wiki page for those who want to follow it and the video of the original talk is available, linked from that Wiki page.

What gets people excited about this project is that we already have all the components to do the work, people have been doing most of this stuff at home, on their own, for some time and it needs to be brought together as a series of packages and meta packages and other middleware glue to get it working. (multistrap is part of that glue - it's already being used commercially in a similar role. Debian Pure Blends is another part of the glue. This can be a Debian solution, it doesn't have to be a separate project.)

The software and glue doesn't have to be dependent on a single hardware platform, this is about getting the software right and deploying it on whatever hardware can run it - albeit that the architecture itself is likely to be ARM-something.

It is time for social networking to be freed from the man-in-the-middle - time to withdraw user data from central servers owned by some third-party and share directly with your friends (whichever network they use). Aggregate all data from all your friends, keep all your data in your own plug and access it from wherever you are using whatever devices you have to hand. What's more, your friends - especially the non-techy friends - can just buy a cheap beige box that hides away in the corner of their home and have all the same access too - everyone gets to choose exactly which pieces of data get shared and to whom and can then use their friends plugs to store encrypted backups of their other data. Or maybe the final solution will be slightly different - it's up to people in Debian to make it happen. The tools are in our own hands, if we want this, we just need to do it.

What is needed now is some direction, there are choices to be made about how the glue is designed and how this is turned into a "self-discoverable" distributed network service. Yes, IPv6 will make this easier but only once ordinary users can deploy the device transparently and without configuration, so an interim solution will be needed.

Who's up for the challenge?

Dirk Eddelbuettel: RQuantLib 0.3.3

A new release (now at version 0.3.3) of RQuantLib is now on CRAN and in Debian. RQuantLib combines (some of) the quantitative analytics of QuantLib with the R statistical computing environment and language. Many of the changes in this new version are internal. The code was re-written using the new Rcpp API throughout, and the build system was further simplified using the LinkingTo: mechanism. The arithmetic average-price asian option pricer was added. A few other code updates were made as well. Full changelog details, examples and more details about this package are at my RQuantLib page.

Martin-&#201;ric Racine: A small death, a big birth.

I dunno why, but hearing bagpipes always gives me the creeps, probably because their melodies sound like a whole country was just brutally slaughtered and is being mourned. Sure enough, this morning, on my way to pick up my new national ID card matching my new citizenship, I ran into a Scottish regiment warming their bagpipes in preparation for the Police marching band festival of Helsinki and instantly had goosebumps, to the point of running a mental list of everyone among my relatives who might have crossed the Styx during the last few hours. While my mother's recent ailments have indeed been a source of concern, I simply couldn't think of any reason for them to degenerate to a fatal extent, so I mentally crossed that possibility out. Still, even after marching a few blocks further downtown towards the central police station, I simply couldn't shake the deep sadness that got onto me upon hearing the bagpipes rehearsal. By the time I reached my destination, an answer came: before I could get my new ID card, I'd have to surrender my old Foreigner's ID card. As odd as it might sound, I felt sad to let go of that pink-tinted little bugger. See, as much as I've hated being a foreigner all these years, that pinkish ID card was my only legal tie to this country and the photo of me it bears represented an important phase of struggles in my life, a phase that I would have loved to document in any possible way. Alas, it was not possible, so I gave one last look at my old card, handed it over and took my new blue-tinted citizen's ID card in exchange. Walking back across downtown towards my home, all I could do was stare at my new ID card in disbelief: the Citizenship line indeed said FIN. I suppose that was the real message: C' tait la fin d'une citoyennet et le d but d'une autre. Une petite mort, une grande (re)naissance. Seh n on hyv vaihtokauppa, miun puolesta.

Gerfried Fuchs: The Good, The Bad and The Ugly

Debian always was known for its communication "style". There were even shirts sold in memory of Espy Klecker with a quote he is known for: Morons. I'm surrounded by morons. Yes, I bought me one of those shirts too in the early days. And there were the talks that promoted Debian as a place to have Good flamewar training. And people considered that to be the fun part. After some years it got tiring. It got stressful. It got annoying. Bad feelings popped up, stirred you into the next flamewar, and it went down the gutter from there. It was almost becoming impossible to not be the target of a flamewar when one was doing more than just basic maintenance. Snide and extreme terse responses became the standard. In the end people are starting to give up and leave. The Ugly thing about this is that human resources are crucial. They aren't endless and can't be replaced as easily as broken hardware, especially when capable people or when people leave who invested an enormous amount of their spare time and effort. And given that a fair amount of people do put their heart into Debian, it feels like a small suicide to them and the public thinking about leaving is meant as a call for help which wasn't and isn't given. The solution to this death swirl? I'm not sure. When one looks over the edge of the plate and ignores for a moment all the bad feelings they one might have built up against Ubuntu because of their success and possibility to find new contributors on a regular basis one is able to find a much friendlier and productive environment there. This might be attributed to the Code of Conduct about which I wrote about last year already and which is an extremely well intended and useful document (the point I raised in there is already solved for a while, so I became a MOTU). And even if it might be hard to follow it at times, Mark Shuttleworth reminds and encourages its contributors to stick to these principles even in tough times. The result? When following the planets, one finds on Planet Ubuntu a very good rate of blog posts on things that had been done, compared to the good rate of blog posts of rants on Planet Debian. And even though people regularly complain about the communication style within Debian, the answers of this year's DPL candidates to the question about a code of conduct for Debian were rather rather disappointing. So it is just well too understandable that people go the path that hurts themself, take a cut and leave the project behind in its mess. For myself? I'm not too far from that point on a regular basis, and I can understand those who did the final step only too well. Regular abuse, especially when doing stuff that others neglect on a regular basis but needs to be done anyway, being belittled on that grounds and not being taken serious and getting disrespectful responses isn't improving the situation. It happens to way too many people, and the only thing that still keeps me on tracks is that I do not want to give in yet, that I don't think that it would improve Debian to leave the grounds to various destructive people. On the other hand, there is only so much abuse one can take...
ObTitle: Ennio Morricone - The Good, The Bad and The Ugly

/debian permanent link Comments: 5

Gerfried Fuchs: The Good, The Bad and The Ugly

Debian always was known for its communication "style". There were even shirts sold in memory of Espy Klecker with a quote he is known for: Morons. I'm surrounded by morons. Yes, I bought me one of those shirts too in the early days. And there were the talks that promoted Debian as a place to have Good flamewar training. And people considered that to be the fun part. After some years it got tiring. It got stressful. It got annoying. Bad feelings popped up, stirred you into the next flamewar, and it went down the gutter from there. It was almost becoming impossible to not be the target of a flamewar when one was doing more than just basic maintenance. Snide and extreme terse responses became the standard. In the end people are starting to give up and leave. The Ugly thing about this is that human resources are crucial. They aren't endless and can't be replaced as easily as broken hardware, especially when capable people or when people leave who invested an enormous amount of their spare time and effort. And given that a fair amount of people do put their heart into Debian, it feels like a small suicide to them and the public thinking about leaving is meant as a call for help which wasn't and isn't given. The solution to this death swirl? I'm not sure. When one looks over the edge of the plate and ignores for a moment all the bad feelings they one might have built up against Ubuntu because of their success and possibility to find new contributors on a regular basis one is able to find a much friendlier and productive environment there. This might be attributed to the Code of Conduct about which I wrote about last year already and which is an extremely well intended and useful document (the point I raised in there is already solved for a while, so I became a MOTU). And even if it might be hard to follow it at times, Mark Shuttleworth reminds and encourages its contributors to stick to these principles even in tough times. The result? When following the planets, one finds on Planet Ubuntu a very good rate of blog posts on things that had been done, compared to the good rate of blog posts of rants on Planet Debian. And even though people regularly complain about the communication style within Debian, the answers of this year's DPL candidates to the question about a code of conduct for Debian were rather rather disappointing. So it is just well too understandable that people go the path that hurts themself, take a cut and leave the project behind in its mess. For myself? I'm not too far from that point on a regular basis, and I can understand those who did the final step only too well. Regular abuse, especially when doing stuff that others neglect on a regular basis but needs to be done anyway, being belittled on that grounds and not being taken serious and getting disrespectful responses isn't improving the situation. It happens to way too many people, and the only thing that still keeps me on tracks is that I do not want to give in yet, that I don't think that it would improve Debian to leave the grounds to various destructive people. On the other hand, there is only so much abuse one can take...
ObTitle: Ennio Morricone - The Good, The Bad and The Ugly

/debian permanent link Comments: 8 Flattr this

Russell Coker: Why Clusters Usually Don t Work

It s widely regarded that to solve reliability problems you can just install a cluster. It s quite obvious that if instead of having one system of a particular type you have multiple systems of that type and a cluster configured such that broken systems aren t used then reliability will increase. Also in the case of routine maintenance a cluster configuration can allow one system to be maintained in a serious way (EG being rebooted for a kernel or BIOS upgrade) without interrupting service (apart from a very brief interruption that may be needed for resource failover). But there are some significant obstacles in the path of getting a good cluster going. Buying Suitable Hardware If you only have a single server that is doing something important and you have some budget for doing things properly then you really must do everything possible to keep it going. You need RAID storage with hot-swap disks, hot-swap redundant PSUs, and redundant ethernet cables bonded together. But if you have redundant servers then the requirement for making one server reliable is slightly reduced. Hardware is getting cheaper all the time, a Dell R300 1RU server configured with redundant hot-plug PSUs, two 250G hot-plug SATA disks in a RAID-1 array, 2G of RAM, and a dual-core Xeon Pro E3113 3.0GHz CPU apparently costs just under $2,800AU (when using Google Chrome I couldn t add some necessary jumper cables to the list so I couldn t determine the exact price). So a cluster of two of them would cost about $5,600 just for the servers. But a Dell R200 1RU server with no redundant PSUs, a single 250G SATA disk, 2G of RAM, and a Core 2 Duo E7400 2.8GHz CPU costs only $1,048.99AU. So if a low end server is required then you could buy two R200 servers that have no redundancy built in which cost less than a single server that has hardware RAID and redundant PSUs. Those two servers have different sets of CPU options and probably other differences in the technical specs, but for many applications they will probably both provide more than adequate performance. Using a server that doesn t even have RAID is a bad idea, a minimal RAID configuration is a software RAID-1 array which only requires an extra disk per server. That takes the price of a Dell R200 to $1,203. So it seems that two low-end 1RU servers from Dell that have minimal redundancy features will be cheaper than a single 1RU server that has the full set of features. If you want to serve static content then that s all you need, and a cluster can save you money on hardware! Of course we can debate whether any cluster node should be missing redundant hot-plug PSUs and disks. But that s not an issue I want to address in this post. Also serving static content is the simplest form of cluster, if you have a cluster for running a database server then you will need a dual-attached RAID array which will make things start to get expensive (or software for replicating the data over the network which is difficult to configure and may be expensive), so while a trivial cluster may not cost any extra money a real-world cluster deployment is likely to add significant expense. My observation is that most people who implement clusters tend to have problems getting budget for decent hardware. When you have redundancy via the cluster you can tolerate slightly less expected uptime from the individual servers. While we can debate about whether a cluster member should have redundant PSUs and other expensive features it does seem that using a cheap desktop system as a cluster node is a bad idea. Unfortunately some managers think that a cluster solves the reliability problem and therefore you can just use recycled desktop systems as cluster nodes, this doesn t give a good result. Even if it is agreed that server class hardware is used for all servers so features such as ECC RAM are used you will still have problems if someone decides to use different hardware specs for each of the cluster nodes. Testing a Cluster Testing a non-clustered server or some servers that use a load-balancing device at the front-end isn t that difficult in concept. Sure you have lots of use cases and exception conditions to test, but they are all mostly straight-through tests. With a cluster you need to test node failover at unexpected times. When a node is regarded as having an inconsistent state (which can mean that one service it runs could not be cleanly shutdown when it was due to be migrated) it will need to be rebooted which is sometimes known as a STONITH. A STONITH event usually involves something like IPMI to cut the power or a command such as reboot -nf , this loses cached data and can cause serious problems for any application which doesn t call fsync() as often as it should. It seems likely that the vast majority of sysadmins run programs which don t call fsync() often enough, but the probability of losing data is low and the probability of losing data in a way that you will notice (IE it doesn t get automatically regenerated) is even lower. The low probability of data loss due to race conditions combined with the fact that a server with a UPS and redundant PSUs doesn t unexpectedly halt that often means that problems don t get found easily. But when clusters have problems and start calling STONITH the probability starts increasing. Getting cluster software to work in a correct manner isn t easy. I filed Debian bug #430958 about dpkg (the Debian package manager) not calling fsync() and thus having the potential to leave systems in an inconsistent or unusable state if a STONITH happened at the wrong time. I was inspired to find this problem after finding the same problem with RPM on a SUSE system. The result of applying a patch to call fsync() on every file was bug report #578635 about the performance of doing so, the eventual solution was to call sync() after each package is installed. Next time I do any cluster work on Debian I will have to test whether the sync() code seems to work as desired. Getting software to work in a cluster requires that not only bugs in system software such as dpkg be fixed, but also bugs in 3rd party applications and in-house code. Please someone write a comment claiming that their favorite OS has no such bugs and the commercial and in-house software they use is also bug-free I could do with a cheap laugh. For the most expensive cluster I have ever installed (worth about 4,000,000 UK pounds back when the pound was worth something) I was not allowed to power-cycle the servers. Apparently the servers were too valuable to be rebooted in that way, so if they did happen to have any defective hardware or buggy software that would do something undesirable after a power problem it would become apparent in production rather than being a basic warranty or patching issue before the system went live. I have heard many people argue that if you install a reasonably common OS on a server from a reputable company and run reasonably common server software then the combination would have been tested before and therefore almost no testing is required. I think that some testing is always required (and I always seem to find some bugs when I do such tests), but I seem to be in a minority on this issue as less testing saves money unless of course something breaks. It seems that the need for testing systems before going live is much greater for clusters, but most managers don t allocate budget and other resources for this. Finally there is the issue of testing issues related to custom code and the user experience. What is the correct thing to do with an interactive application when one of the cluster nodes goes down and how would you implement it at the back-end? Running a Cluster Systems don t just sit there without changing, you have new versions of the OS and applications and requirements for configuration changes. This means that the people who run the cluster will ideally have some specialised cluster skills. If you hire sysadmins without regard to cluster skills then you will probably end up not hiring anyone who has any prior experience with the cluster configuration that you use. Learning to run a cluster is not like learning to run yet another typical Unix daemon, it requires some differences in the way things are done. All changes have to be strictly made to all nodes in the cluster, having a cluster fail-over to a node that wasn t upgraded and can t understand the new data is not fun at all! My observation is that the typical experience of having a team of sysadmins who have no prior cluster experience being hired to run a cluster usually involves learning experiences for everyone. It s probably best to assume that every member of the team will break the cluster and cause down-time on at least one occasion! This can be alleviated by only having one or two people ever work on the cluster and having everyone else delegate cluster work to them. Of course if something goes wrong when the cluster experts aren t available then the result is even more downtime than might otherwise be expected. Hiring sysadmins who have prior experience running a cluster with the software that you use is going to be very difficult. It seems that any organisation that is planning a cluster deployment should plan a training program for sysadmins. Have a set of test machines suitable for running a cluster and have every new hire install the cluster software and get it all working correctly. It s expensive to buy extra systems for such testing, but it s much more expensive to have people who lack necessary skills try and run your most important servers! The trend in recent years has been towards sysadmins not being system programmers. This may be a good thing in other areas but it seems that in the case of clustering it is very useful to have a degree of low level knowledge of the system that you can only gain by having some experience doing system coding in C. It s also a good idea to have a test network which has machines in an almost identical configuration to the production servers. Being able to deploy patches to test machines before applying them in production is a really good thing. Conclusion Running a cluster is something that you should either do properly or not at all. If you do it badly then the result can easily be less uptime than a single well-run system. I am not suggesting that people avoid running clusters. You can take this post as a list of suggestions for what to avoid doing if you want a successful cluster deployment.

Jaldhar Vyas: Sita Sings the Blues

In the evening today (Tuesday), the Debconf organizers arranged for a screening of the Creative Commons licensed film "Sita Sings the Blues" by Nina Paley. I went to see it as I had missed it when it was first released among tales of trials and tribulation at the hands of copyright meanies. So first let me say yay Free Culture!, boo Intellectual Property! etc. etc. Now we have that out of the way... "Sita Sings the Blues" has two main narrative threads: The breakup of Ninas relationship with her boyfriend and the Ramayana particularly a story from the uttarakanda about how after Shri Rama returned from Lanka, Sita had to undergo the agnipravesha to prove her chastity and even after she passed that test was eventually banished to the forest. I had an adverse reaction to this film. In fact I stayed to the end to be polite but I left afterwards as soon as I could. I had almost a physical feeling of nausea and I was surprised by this. Since then I have been pondering why this is so. It's not that its blasphemous. Shri Rama is not just a "perfect man", he is God (an avatar of Vishnu Bhagavan as is briefly touched upon in the film) for millions of people so, well, yes it is blasphemous but its more Rowan Atkinson style rather than Norwegian Black Metal style blasphemy. One never gets the impression that the author hates Hinduism or Indian culture the opposite in fact. And in fact it is not uncommon in our culture to speak of Sita and Rama in this way. They are not remote authority figures but as familiar to us as our own friends and family and the Ramayana is not just about events that happened long ago it is relevant to our lives right now. Even the "good woman done wrong by big dumb male jerk" narrative is not unknown. (See the Uttararamacharita of Bhavabhuti for instance.) though I would suggest reducing the Ramayana this way is akin to treating the Hitchhikers Guide to the Galaxy as a book about towels. Even the milder criticism that it is not accurate is offbase because although the Sanskrit poem by Maharshi Valmiki is the canonical rendition of the Ramayana, there are other well-known versions. Not just in Sanskrit, the Hindi retelling by Sant Tulsidas called Ramacharitamanasa is more popular than Valmiki. Also speaking as someone who has preached Ramakatha "professionally" I can tell you that even the priests do not stick to the letter of the text. It is encouraged and expected that they will riff on the basic stories adding or removing details according to the needs and taste of the audience. And the device of the three narrators made me smile as this is exactly the kind of discussions that go on after hearing the Ramakatha. (I must say Paley picked a notably illiterate trio though.) One place where this fell flat was when they discussed the alleged discrepancy between Sita living simply in the forest and dropping a trail of jewellery on the way to Lanka. Although I can imagine someone at some point saying, "Do not question these stories" not at this point. Even the most retarded westernized Indian would know that removing ones ornaments is a sign of widowhood. A married woman would only do that in extremis certainly not just merely because she was a hermit. In a few cases some extra details might have helped the story. It was not explained that Ravana does not just have rakshasas in his command but he is one himself. (And he certainly wasn't a good guy until one day he got up and kidnapped Sita.) Shri Ramas brother Lakshamana accompanied the couple into the forest to Lanka and back but he only pops up (for no apparent reason) at the end. But look at me, I'm turning into one of those people who rant about Star Trek continuity errors or complain that the Romans in a gladiator movie have the wrong helmets for that time period etc. One has to cut a storyteller some slack and I entered the screening determined to be charitable. It's also not because I don't really see much connection between the two stories. Getting dumped by someone who simply loses interest in you hardly seems equivalent to being trapped by a moral code that doesn't allow for individuals feelings. Paley could have easily made "Princess Leia Sings the Blues" and it wouldn't have changed the significance much. But in India people make far more tenuous connections between the epics and events in their own life so one can hardly complain about that. Aesthetics are not the problem either. The first version of the Ramayana I ever read was the Amar Chitra Katha comicbook version which partly influenced the visual style of this film. (In fact you briefly see it in one scene when different editions of the Ramayana flash by.) Hindu popular culture is very gaudy. (See Ramanand Sagars '80s Ramayana Hindi TV serial, which was very reverential, insanely popular and tacky beyond belief.) The animation in this film is very good and quite artistic. "So Jaldhar", you might be asking if you've read this far, "what exactly is your problem with this film?" I have read the Valmiki Ramayana in Sanskrit with the commentaries from beginning to end. I've read other Ramayanas in Sanskrit, Gujarati, and English. I have read the comicbook, I've read anti-Ramayana polemics. When you learn Sanskrit in the traditional way the first lesson, in the declension of masculine gendered nouns ending in vowels is rAma.h rAmau rAmA.h ... I am well aware there are different emphases, different interpretations in various versions. Why can't I accept this as just another take on a story that has shown its fluidity over and over again? It's just...wrong on a subrational level. How can I put this? Imagine you are flipping through the TV channels and you see your grandparents amongst Lady Gaga's backup dancers. Or you came home one night to find everything that was on the floor meticulously glued to the ceiling. Over the course of my life (to hear my mother tell it my first exposure to the Ramayana was before I could speak.) I have developed a mental bond with this story a certain idea of how it should be. I think so has Nina Paley. And this is the problem. "Sita Sings the Blues" is Nina Paleys Ramayana not mine.

Next.

Previous.