Search Results: "will"

21 June 2013

MJ Ray: Cooperatives Fortnight 2013

[Co-operatives Fortnight image] Cooperatives Fortnight 2013 starts tomorrow. Our co-op is taking part in a few activities this year. Come along and meet us, or find other events on the national website. This Sunday, I ll be riding 50 miles in support of Leonard Cheshire. Some of it is familiar roads, some of it new. I ll also be riding 12 miles just to get to the start line. If you re a UK resident and would like to guess my time and make a small donation, please give it a try on guess2give. Next Thursday 27 June, I ll be at Somerset Cooperative Services AGM in Taunton. See their site for further news. There might be something on Monday 1 July. To be announced later maybe. On Thursday 4July, mjkaye will be at Building Our Co-ops in Liverpool. See the national event listing for details. So what will you be doing for co-ops fortnight?

Lars Wirzenius: How long is a string: how many backups are enough?

You've set up a backup repository, and you have been backing up to it every day for a month now: your backup history is getting long enough to be useful. Can you be happy now? Welcome to the world of threat modelling. Backups are about insurance, of mitigating small and large disasters, but disasters can strike backups as well. When are you so safe you no disaster will harm you? There is always a bigger disaster waiting to happen. If you backup to a USB drive on your work desk, and someone breaks in and steals both your computer and the USB drive, the backups did you no good. You fix that by having two USB drives, and you keep one with your computer and the other in a bank vault. That's pretty safe, unless there's an earth quake that destroys both your home and the bank. You fix that by renting online storage space from another country. That's quite good, except there's a bug in the operating system that you use, which happens to be the same operating system the storage provider uses, and hackers happen to break into both your and their systems, wiping all files. You fix that by hiring a 3D printer that prints slabs of concrete on which your data is encoded using QR codes. You're safe until there's a meteorite hits Earth and destroys the entire civilisation. You fix that by sending out satellites with copies of your data, into stable orbits around all nine planets (Pluto is too a planet!) in the solar system. Your data is safe, even though you yourself are dead from the meteorite, until the Sun goes supernova and destroys everything in the system. There is always a bigger disaster. You have to decide which ones are likely enough that you want to consider them, and also decide what the acceptable costs are for protecting against them. A short list of scenarios for thinking about threats: These questions do not cover everything, but they're a good start. For each one, think about: The threat modelling here is about safety against accidents and natural disasters. Threat modelling against attacks and enemies is similar, but also different, and will be the topic of the next episode in the adventures of Bac-Kup.

Daniel Pocock: Practical challenges for interrupt-free computing

My previous blog on interrupt-free computing has been very well read. I've had a look at some practical implementation possibilities and can share some more details about how to go about it and potential problems. No perfect solution They say that a certain billionaire is so rich that he would not spend 5 seconds bending over to pick up a $100 banknote on the ground because he makes $500 in 5 seconds anyway. A man from Ethiopia may well run a marathon for the same $100. This little anecdote underlines the point that no two people have exactly the same priorities and it is hard to provide a one-size-fits-all solution. A good technical solution needs to let the user have some control over their own priorities as that will go a lot further in respecting their preferences, their objectives and their environment. A practical example: some users may not want to be disturbed by a VoIP call while watching a movie with their PC. Other users may be quite happy to receive these interruptions. Some users may want to only allow this interruption from a subset of possible callers. Building blocks In the original blog on this subject I suggested that Thunderbird and it's popular plugin, Mozilla Lightning calendar would provide one useful framework for integrating events, alerts and to-do items of various grades of importance. I already use these applications myself, from the re-branded icedove and iceowl-extension packages on Debian. They are also available in Fedora and many other distributions. Priority with Lightning An initial look at Lightning reveals a Priority attribute for tasks. It can be set to the values High, Medium or Low. These don't map to the concepts of Urgency and Importance from the Eisenhower method discussed in the original blog. Potential hacks: there are two ideas that come to mind, with no coding changes:
  • One possibility is to set Due dates for any item that has the Urgent characteristic and no date for items without the characteristic. In this case, the Priority attribute can then be used to distinguish Important and non-Important tasks.
  • The other possibility is to use the Categories feature to define four categories: Urgent-Important, non-Urgent-but-Important, Urgent-non-Important and neither-Urgent-Important.
A more elegant solution would be to develop an additional plugin that provides new attributes for the Urgency and Importance characteristics. For server-side storage of these attributes, it would be feasible to use extension properties as defined in RFC 2445, for example, X-urgency and X-importance This approach would make it easier to view a large range of tasks across multiple domains with the most urgent and important tasks at the top. Ideally, the Categories attribute of tasks can be used to distinguish tasks that relate to work (or project/client) from personal tasks (such as planning a holiday) Challenges with Lightning Some of the challenges that I've identified after reviewing this solution:
  • the Categories in Lightning don't appear to be synchronized with the Tags used in Thunderbird for email.
  • Furthermore, the Tags are not synchronized automatically with the list of server-side IMAP keywords and somebody having multiple Thunderbird profiles (e.g. on both a desktop and a laptop) needs to manually synchronize the lists of tags and categories between all their devices.
  • Many people use mobile devices such as smartphones and tablets to access their email, calendar and tasks. The lightweight mail/calendar clients on these devices don't support the full range of features for classification of content. Furthermore, their low-resolution user interfaces may even make it difficult to replicate the same experience on a mobile device. There is some hope, for example, rooting a tablet device and installing Linux would allow it to run a proper installation of Thunderbird with the various plugins described above.
Feeding system events and other data into the pipeline Given the text-based nature of the iCalendar format, it should be easy enough for any application that is capable of generating mail to generate these iCal events with X-Urgency and X-importance properties. These events - such as alerts about security updates - could then be dispatched through the system's mailer. Some additional effort may be needed to avoid duplication. The notification system in the desktop UI may then prioritize events from the mailbox rather than showing all alerts in real-time. Only applications that implicitly need to do so (such as VoIP/chat clients) would display notifications to the user directly. Conclusion: Interrupting the user Finally, after gathering all the events for the user in a mailbox like this, it would be possible for the plugin to be configured to interrupt the user only when some event reaches the user's own urgency/importance threshold. For example:
  • The user may want to be interrupted for Urgent items with a deadline in less than 24 hours, but if the task has a work' category, only interrupt during business hours
  • For all other items, the user only sees them when they open Lightning or some other calendar/task management application to explicitly review things that they need to do.

Dirk Eddelbuettel: Upcoming Rcpp talk in Sydney

The Sydney Users of R Forum (SURF) will be hosting me for a talk on July 10. The focus will be Rcpp for R and C++ integration, and the intent is to have this be really applied with lots of motivating examples. Organizers Louise and Eugene were able to move this to a slightly larger room as the initial capacity of 50 was filled almost immediately. As of right now, the talk page shows a few available slots. If you're in the Sydney are in early July, why not register and swing by? If you're not in the area, the Rcpp Events page lists other upcoming talks too.

20 June 2013

Matt Zimmerman: Scaling Human Systems: Alignment

This is part 1 in a series on organizational design and growth. An important lens for thinking about organizations at this stage of growth is alignment. In an organization which is aligned, the efforts of different people and teams all contribute to forward progress in a shared direction. If two teams are pulling in opposite directions they may make little progress despite great effort, and quickly become frustrated. To take an obvious example, if a marketing team is targeting an audience of large enterprises while the product being developed is only suited for small businesses, the end result of both teams doing a good job will be a failure (i.e. unhappy customers). It s important to note that alignment does not imply sameness. Different teams within an organization can function and behave very differently while still being strongly aligned with each other. When an organization is small, alignment comes naturally. Everyone has some visibility on what everyone else is doing, and when something doesn t line up, the people involved can talk it over and resolve the issue relatively easily. But as the organization grows, the propensity for misalignment increases, and these situations become much more difficult and time-consuming to resolve. The metaphor of the right hand, which doesn t know what the left hand is doing, seems like something that would only happen in larger companies, but it begins much earlier, especially when the company goes through a period of rapid growth. Critical infrastructure, such as communication tools and patterns, lags behind the accelerating needs of the people involved, creating a surprising distance between teams. Organizational alignment is a critical part of scaling successfully. With alignment, growth and momentum are assets. Without it, they are liabilities. Further reading: The Advantage: Why Organizational Health Trumps Everything Else in Business

Matt Zimmerman: Scaling Human Systems: Organizational Design and Growth

This is the beginning of a series of articles about the challenges of growing an organization. I m writing them to share some principles that I ve derived from my own experience, as well as many valuable discussions with friends and colleagues, about helping companies grow from being quite small (say, 1-50 employees) to medium-sized (100-500). There are many different ways to categorize companies by size, and not everyone agrees with me that different organizations tend to face certain similar problems as they grow, based on the number of employees. In any case, hopefully we can all agree that human systems are mind-bogglingly complex entities, and any two organizations will have many important differences such as their culture and market situation which influence their growth and development. For this reason, I believe there are few if any hard and fast rules, and organizational design patterns can be difficult to translate from one organization to another. One organization s solution can be another s problem. Even when there is a perfect fit, the process of organization change is a feat unto itself, one about which many books have been written. Even so, I think there is much to be learned by comparing different organizations, and much inspiration to be found in their successes and failures. Two organizations merit specific mention here, as sources of inspiration for me: Canonical, where I worked as Ubuntu CTO from near inception to when it reached nearly 500 people, and Heroku, where I currently serve as VP Engineering as it grows beyond 100 people.

MJ Ray: What s the current state of Windows Anti-Virus?

One of our co-op s clients asked me what I use for anti-virus at the moment and tips for what they should use on their Windows system. Well, flame me now, but I don t actually use any anti-virus at the moment: I rely on system security, firewalling and intrusion detection. The diversity of GNU/Linux software and I use some pretty odd stuff probably helps too. Even if I did want to run antivirus software, most of what s available for GNU is actually aimed at detecting and preventing transmission of Windows viruses. There are few real-world GNU viruses and fewer attack opportunities left open. Also, I prefer firewalling and fairly paranoid security settings because, like an antibiotic, an antivirus is only effective once the virus is already on your system somehow hopefully held in quarantine by the browser or email client and not actively malignant in the processor. There s quite a list at http://en.wikipedia.org/wiki/Comparison_of_antivirus_software#Microsoft_Windows but I expect most of the purchase-free proprietary ones (labelled as Free or Freemium but you usually pay by watching adverts) will try to sell you upgrades, as that s how their production is funded. If you don t mind doing such things, you can disable the ads in at least one of them The only very free ones I found were Immunet (also funded by upgrades not sure if it s actually Free and Open Source Software) and ClamWin (donation-funded) which both use the same scanning engine. If I had to use Microsoft Windows, I think I d probably use and donate to ClamWin, install the (altruism-funded I think) Clam Sentinel alongside it and be rather cautious about what I downloaded or used online. I m a bit worried that it doesn t do great in reviews, though. What do/would you do? I don t really know about paying for security. The only paid product I ve really seen has been Norton and that seemed no better than the ad-funded ones, still getting in the way and always trying to sell upgrades. It also irks me that there s this huge market just to fix fundamental defects in Microsoft s product. There s a Microsoft Security Essentials add-on listed on Wikipedia, but it does fairly badly in this PC Magazine review and do any of them do intrusion detection? And finally, if you do decide to download something new, I strongly suggest getting it from a trusted source and/or triple-checking the link with wikipedia, a magazine review like CNET and a search engine. Don t just trust a search engine, because fake antivirus software is a big way of getting viruses and worse onto computers: there s even one calling itself Microsoft Security Essentials 2011 !

19 June 2013

Matthew Garrett: Mir, the Canonical CLA and skewing the playing field

Mir is Canonical's equivalent to Wayland - a display server, responsible for getting application pixmaps onto a screen. It's intended to scale from mobile devices to the desktop, and as such is expected to turn up in Ubuntu Phone before too long[1]. There's already plenty of discussion about whether the technical differences between Wayland and Mir are sufficient to justify Canonical going their own way, so I'm not planning on talking about that.

Like many Canonical-led projects, Mir is under GPLv3 - a strong copyleft license. There's a couple of aspects of GPLv3 that are intended to protect users from being unable to make use of the rights that the license grants them. The first is that if GPLv3 code is shipped as part of a user product, it must be possible for the user to replace that GPLv3 code. That's a problem if your device is intended to be locked down enough that it can only run vendor code. The second is that it grants an explicit patent license to downstream recipients, permitting them to make use of those patents in derivative works.

One of the consequences of these obligations is that companies whose business models depend on either selling locked-down devices or licensing patents tend to be fairly reluctant to ship GPLv3 software. In effect, this is GPLv3 acting entirely as intended - unless you're willing to guarantee that a user can exercise the freedoms defined by the free software definition, you don't get to ship GPLv3 material. Some companies have decided that shipping GPLv3 code would be more expensive than either improving existing code under a more liberal license or writing new code from scratch. Android's a pretty great example of this - it contains no GPLv3 code, and even GPLv2 code (outside the kernel) is kept to a minimum.

Which, given Canonical's focus on pushing Ubuntu into GPLv3-hostile markets, makes the choice of GPLv3 an odd one. This isn't a problem as long as they're the sole copyright holder, because the copyright holder is obviously free to ship their code under as many licenses as they want. But Canonical still aim to foster community involvement, and ideally that includes accepting external contributions to their code. If Canonical simply accepted those contributions under GPLv3 then they'd no longer have the right to relicense the entire codebase, so any contributions are only accepted if the contributor has signed a Contributor License Agreement.

Canonical's CLA is pretty simple. In essence, it grants Canonical the right to use, modify and distribute your code, and it grants Canonical a patent license under any patents you own that may cover the code in question. But, most importantly, it grants Canonical the right to relicense your contribution under their choice of license. This means that, despite not being the sole copyright holder, Canonical are free to relicense your code under a proprietary license.

Given Canonical's market goals, this makes sense. They can relicense Mir (and any other GPLv3 projects they own) under licenses that keep their hardware partners happy, and they can ship in the phone market. Everyone's a winner.

Except, if Canonical want to ship proprietary versions, why not just license Mir under a license that permits that in the first place? This is where the asymmetry comes in. The Android userland is released under a permissive license that allows anyone to take Google's code, modify it as they wish and ship it on whatever hardware they want. I could legally start a company that provided customised versions of Android to phone vendors without them having any GPLv3 concerns. I won't be able to do that with Ubuntu Phone.

I'm a fan of GPLv3. I think the provisions it contains to support user freedom are important. I hate the growing trend of using free software to build devices that are, effectively, impossible for the end user to modify. If Canonical were releasing software under GPLv3 because of a commitment to free software then that would be an amazing thing. But it's pretty much impossible to square the CLA's requirement that contributors grant Canonical the right to ship under a proprietary license with a commitment to free software. Instead you end up with a situation that looks awfully like Canonical wanting to squash competition by making it impossible for anyone else to sell modified versions of Canonical's software in the same market.

Canonical aren't doing anything illegal or immoral here. They're free to run their projects in any way they choose. But retaining the right to produce proprietary versions of external contributions without granting equivalent reciprocal rights isn't consistent with caring about free software or contributing to the wider Linux community, especially if it means you get to exclude those external contributors from the market you're selling their code into.

(Edit to add: a friend in the contracting industry points out that it also prevents vendors who won't ship GPLv3 from using external contractors to work on Mir - they have to go to Canonical, because only Canonical can relicense contributions under a proprietary license.)

[1] Right now Ubuntu Phone is using Surfaceflinger, the Android display server, but that's apparently just an interim solution.

comment count unavailable comments

DebConf team: DebConf13 DebCamp confirmation and opening of the reconfirmation period (Posted by Didier Raboud)

Going to DebConf13!
DebConf13 DebCamp confirmed! The DebConf Team is very pleased to announce that DebCamp will be open for almost a full week in August. DebCamp will start on Tuesday 6 August 2013 at the main conference venue, Le Camp, and will be immediately followed by DebConf13, starting Sunday 11th August. Opening of the reconfirmation period As the DebConf Registration Team have announced on debconf-announce (to which you should really be subscribed), now is the time to reconfirm your attendance to DebConf13 (and actually register to DebCamp). This is really important for final calculations of food, room and costs, please read the announcement for all the important details. The reconfirmation period is open until the 30 June 2013. We look forward to seeing everyone in August in Vaumarcus. The DebConf team

Benjamin Mako Hill: Job Market Materials

Last year, I applied for academic, tenure track, jobs at several communication departments, information schools, and in HCI-focused computer science programs with a tradition of hiring social scientists. Being on the market as it is called is both scary and time consuming. Like me, many candidates have never been on the market before. Candidates are asked to produce documents in genres e.g., cover letters, research statements, teaching statements, diversity statements that most candidates have never written, read, or even heard of. Candidates often rely on their supervisors for advice. I did so and my advisors were extremely helpful. The reality, however, is that although candidates advisors may sit on hiring committees, most have not been on candidates side of job market themselves for years or even decades. The Internet is full of websites, like the academic jobs wiki, Academia StackExchange, and the Chronicle of Higher Education forums for people on the market. Confused and insecure candidates ask questions of the form, Does blank matter? and the answer is usually, Doing/having blank may help/hurt, but it is only one factor of many. The result is that candidates worry about everything. Then they worry about what they should be worrying about, but are not. The most helpful thing, for me, was to read and synthesize the material submitted by recent successful job market candidates. For example, Michael Bernstein a friend from MIT, now at Stanford published his research and teaching statements on his website and I found both useful as I prepared mine. That said, I was surprised by how little material like this I could find on the web. For example, I could not find any examples of recent job market cover letters from successful candidates in fields close to mine. So to help fill this gap, I am publishing all of my job market material. I ve posted both the PDFs of the material I submitted as well as the LaTeX templates I used to generate the documents in my packet. My packet included: I hope people going on the market will find these materials useful. Obviously, you should not copy or reuse the text of any of my material. It is your application, after all. That said, please do help yourself to the formatting and structure. Finally, I would encourage anyone who builds on my material to republish their own material to help other candidates. If you do, I d appreciate a link back or comment on this blog post so that my readers can find your improvements.

Daniel Pocock: Australia's war on brains (and immigration)

Some weeks ago, our Prime Minister was slashdotted when she suspended her usual racist attacks on refugees to attack foreign IT workers and the companies that employ them with absurd accusations of "rorts" and "stealing" jobs. Today, she's introduced new laws in the parliament aiming to further bastardization of intelligent, skilled and educated workers and anybody who associates with them, including Australian employers. Unwarranted attention on a minority of IT workers This is not just some random bill before the parliament. There are just two weeks left before the parliament concludes and an election campaign begins. It is clear that what we are seeing now is the Real Julia coming though, choosing to make the small minority of foreign workers in our country at the center of people's thoughts as they go to vote in September. A debt of gratitude to foreign workers These verses from our national anthem, Advance Australia Fair say a lot about how Australia became what it is today:
For those who've come across the seas

We've boundless plains to share;

With courage let us all combine

To Advance Australia Fair.
The last 200 years of Australia's history has been a story of immigration. It is not something to be afraid of: our forefathers celebrated it. Foreign workers: why Australia needs you Australia has had some appalling flops in IT and engineering: There is no doubt in my mind that additional foreign workers would have made a positive contribution to all of these problems or can do so in the future. The sun never sets on IT Every day, I collaborate with dozens of IT specialists all over the world through the virtual workplace that is the Internet, particularly in the free and open source software community. Many of these people, I've never even met and in most cases I don't even know where they are, where they were born or what is the colour of their skin. Those details wouldn't make any difference to the way that we work in IT today. How many IT managers have time to waste dealing with more real world bureaucracy when they've experienced online, global productivity? How many IT workers feel demotivated by having to explain trivial details about their personal life to a Government bureaucrat who doesn't understand their skills and just looks at their colour? If you think about it, any immigration officer who really understands IT wouldn't be an immigration officer. They would be working in IT themselves. Immigration officers, who don't understand IT, are now going to be further empowered to bully companies away from employing some talented workers on the basis of race or nationality. Hiring managers will be intimidated into these prejudiced and biased decisions by delays, processing fees and invasive demands for sensitive documents about business planning and recruitment strategies. Australia's immigration system already has a horrendous reputation. Any visa application seems to take more than a year: no small company can keep a job position vacant that long. Families can't plan their children's schooling. Other life events come and go. There are exhorbitant fees, 1000% higher than in other western countries. Fewer and fewer self-respecting skilled workers are willing to put their spouses and children through the degrading medical examinations. Judging the impact of poor immigration policy While the economic impact of this immigration mess on industry is hard to quantify with an exact figure, we can take some insight from the education system. As the visa system has been hijacked by racists over the last 10 years, there has been a dramatic fall in participation (and revenues) from foreign students. In one year, enrolments (and revenue) fell 30%. This is not just bad for the balance sheets of the universities, it also means that in a future where commerce is global, Australians are more and more isolated and inexperienced culturally. IT workers and their employers have plenty of choices: Australia's close neighbor, Singapore, is one of them. Visas are granted in 2 weeks, no degrading medical exam required, low taxes and tropical sunshine all year round. Many companies that find it impractical to deal with Australia's bureaucracy end up moving their best Australian workers to places like Singapore to be part of a global team. This can't be good for the workforce that is left behind without jobs. The training delusion Government officials continue to rant and rave about companies failing to train Australian workers. The new laws supposedly force companies to "fix" this problem and train Australian workers. This, too, is a delusion: employers are not to blame. Some of the best Australian workers are already long gone to places like Singapore, London and the US. With talented foreign workers denied the opportunity to come and fill the void, there is less opportunity for skills to be acquired by more junior workers in Australian workplaces. It is also extremely difficult for more junior workers to get a foot in the door in the international job market and the primary reason for this is the Australian Government's failure to fund university programs beyond a bachelor's degree. Compare this to Europe and the US where all competent graduates are funded through to a Masters or PhD program. The bottom line is that more junior workers are denied the opportunity to get the best training either at home or abroad and in both cases it's not the foreign workers that can be blamed: it's the Government's own fault. Why do we need skilled foreign workers when Australians can win Nobel Prizes? The Australian press recently went into a frenzy when an Australian won the Nobel Prize for physics. There was a catch though: he's a migrant from the United States (just don't tell the Prime Minister). Dr. Schmidt migrated to Australia 20 years ago when the immigration system was not the same as today. Today, future Nobel Prize winners are being shown a brick wall - maybe we even have one of them rotting away in our death camps or left in the sea for sharks to eat. The BBC recently revealed that Britain's successor to Stephen Hawking may be a young girl who migrated from India - it is chilling to imagine where a child like this may be hidden away under Australia's immigration system. Bureaucracy leads to fraud and exploitation It's been clearly demonstrated that wherever you have elaborate, artificial systems of bureaucracy it leads to inefficiency, it suppresses innovation and in the worst cases it enables fraud and exploitation. The typical examples usually involve police in some third-world African nation setting up road blocks and collecting fees from travellers who want to pass the queues. This type of opportunism has also been found in Australia's immigration system, with one Federal politician already directly implicated and jailed for his role in a visa racket. Gillard's own bullet man A Queensland pensioner made international headlines recently when he was caught sending bullets in the mail to the Prime Minister. His demands were clear: stop immigration. While most world leaders refuse to let nutcases like this dictate their actions, Gillard appears to have been transcribing his racist letters directly into these newest immigration laws. It is a sad reality that Australian politics regularly seeks to appeal to the worst instincts in people like bullet man. The ultimate political failure When politicians stoop to the level of demonizing immigrants it is usually a clue that the politicians themselves are past their use-by dates and out of fresh policy ideas. When former French president Sarkozy tried to play the racist card in his campaign for re-election, it bit him in the bum and he was swept from power by the socialists. As always in politics, there is an element of hypocrisy at work: neither our Head of Government (the Prime Minister) nor our Head of State (the Queen) was born in Australia. Gillard was born in Wales and migrated to Australia as a child. If Australians don't vote for her in September, will she be given 28 days notice to pack her bags and go back home'? From the frying pan and into the fire Tony Abbott, Australia's would-be Prime Minister contemplating his colleague's statements on bestiality The scariest thing is that if Australians see through this racist charade and refuse to vote for it, we could end up with something equally obnoxious: the other major political party is now gaining worldwide attention for their campaign linking gay marriage and homosexuals to bestiality. Which prejudice is the lesser evil: racism or homophobia?

Julien Viard de Galbert: Building a custom debian CD

The French version is available on linuxembedded.fr : Cr e un CD d installation d une debian sp cialis e The goal is to build a debian install CD suitable for the distribution of a complete system including the operating system and applications. Debian already has a tool for that purpose: simple-cdd. Simple-cdd is a set of scripts wrapping debian-cd which is the tool used to build official CDs. In our case, we will include some non-free packages (firmwares for instance) and application specific packages in the system. Using simple-cdd Simple-cdd gets some of its configuration from your host computer, so it s recommended to work on a machine similar to your target (in particular the machine should use the same architecture: i386 or amd64). All the following actions should be done in a working directory that will contain the downloads and configurations for our system. We will start by creting a configuration file my-cdd.conf stating the mirror and components to use:
debian_mirror="http://ftp2.fr.debian.org/debian/"
mirror_components="main contrib non-free"
Then we will launch the build-simple-cdd tool once. This will build a local mirror of the needed packages that we will reused later. So be patient this can be a bit long.
build-simple-cdd --conf ./my-cdd.conf
Once done, you should get a CD iso in the images subfolder. Customizing the image To customize our image we will create a profile my-profile that will define which package to install and which extra package to include on the CD (dev packages for instance). Create a profiles folder:
mkdir profiles
Selecting the packages If you ve got an already installed machine (the one you used to test your application) you can ask dpkg to list the installed packages from that machine and use that as a basis.
dpkg --get-selections >package-list
In this file only the names of packages in the install state are needed:
grep -e '\<install$' package-list   awk ' print $1 ;' >profiles/my-profile.downloads
The file my-profile.downloads is a list of packages to include on the CD-ROM. The list of program to install is my-profile.packages . The tool will resolve dependencies, so we don t need to track down all dependencies add add them to these files. Specific packages We want to add some custom packages that we built ourselves to the CD, those packages not present in the debian archive. Let s create a local_pkg folder and fill it with all of our packages. We now can update the configuration to use all those files and make the profile my-profile the default profile.
debian_mirror="http://ftp2.fr.debian.org/debian/"
mirror_components="main contrib non-free"
simple_cdd_dir=$(pwd)
profiles="my-profile"
auto_profiles="my-profile"
local_packages="$simple_cdd_dir/local_pkg/"
If you now run the tool again, you ll get a full custom install CD!
build-simple-cdd --conf ./my-cdd.conf
However you might have missed some specific packages or their dependencies, in that case they will be missing on the CD. To find this out, check the messages for a line like:
WARNING: missing optional packages from profile my-profile: libevent-2.0-5 [...]
In that case libevent from squeeze-backports was missing. Simple-cdd is not able to download packages from backports, so I just added the packages to local_pkg . Customizing the installer Simple-cdd automatically uses a default profile. This profile might not suite your needs. To override it you just have to place modified file in the profiles folder. To do so, first copy the desired file from /usr/share/simple-cdd/profiles/ and update it. For instance the default.pressed file:
cp /usr/share/simple-cdd/profiles/default.pressed profiles
Then update the wanted options in this file. For example the partitioning options. Also as we enabled contrib and non-free you should uncomment the following lines:
d-i apt-setup/non-free  boolean true
d-i apt-setup/contrib boolean true
Adding extra files to the CD Finally we want to add the documentation, sources and other files to the CD. Simple-cdd can add files to the simple-cdd folder but there is no way to add a complete directory structure. However, as we can override the profiles, we can also override some scripts, especially tools/build/debian-cd that builds the CD.
mkdir -p tools/build
cp /usr/share/simple-cdd/tools/build/debian-cd tools/build
Then add the following lines after the $extras_base_dir cleanup:
if [ -d "$cd_extras" ]; then
  mkdir -p "$extras_base_dir"
  cp -a "$cd_extras/." "$extras_base_dir"
fi
Finally let s add the cd_extras variable to our configuration file :
cd_extras="$simple_cdd_dir/local_extras/"
Be careful to not create files conflicting with the CD content, nothing would warn you about that! Conclusion We now have a way of building a debian install CD including any files we might want to deliver with our system: sources and documentation for example.

18 June 2013

Daniel Pocock: RSA Key Sizes: 2048 or 4096 bits?

Many people are taking a fresh look at IT security strategies in the wake of the NSA revelations. One of the issues that comes up is the need for stronger encryption, using public key cryptography instead of just passwords. This is sometimes referred to as certificate authentication, but certificates are just one of many ways to use public key technology. One of the core decisions in this field is the key size. Most people have heard that 1024 bit RSA keys have been cracked and are not used any more for web sites or PGP. The next most fashionable number after 1024 appears to be 2048, but a lot of people have also been skipping that and moving to 4096 bit keys. This has lead to some confusion as people try to make decisions about which smartcards to use, which type of CA certificate to use, etc. The discussion here is exclusively about RSA key pairs, although the concepts are similar for other algorithms (although key lengths are not equivalent) The case for using 2048 bits instead of 4096 bits
  • Some hardware (many smart cards, some card readers, and some other devices such as Polycom phones) don't support anything bigger than 2048 bits.
  • Uses less CPU than a longer key during encryption and authentication
  • Using less CPU means using less battery power (important for mobile devices)
  • Uses less storage space: while not an issue on disk, this can be an issue in small devices like smart cards that measure their RAM in kilobytes rather than gigabytes
So there are some clear benefits of using 2048 bit keys and not just jumping on the 4096 bit key bandwagon The case for using 4096 bits
  • For some types of attack, security is not just double, it is exponential. 4096 is significantly more secure in this scenario. If an attack is found that allows a 2048 bit key to be hacked in 100 hours, that does not imply that a 4096 bit key can be hacked in 200 hours. The hack that breaks a 2048 bit key in 100 hours may still need many years to crack a single 4096 bit key
  • Some types of key (e.g. an OpenPGP primary key which is signed by many other people) are desirable to keep for an extended period of time, perhaps 10 years or more. In this context, the hassle of replacing all those signatures may be quite high and it is more desirable to have a long-term future-proof key length.
The myth of certificate expiration Many types of public key cryptography, such as X.509, offer an expiry feature. This is not just a scheme to force you to go back to the certificate authority and pay more money every 12 months. It provides a kind of weak safety net in the case where somebody is secretly using an unauthorised copy of the key or a certificate that the CA issued to an imposter. However, the expiry doesn't eliminate future algorithmic compromises. If, in the future, an attacker succeeds in finding a shortcut to break 2048 bit keys, then they would presumably crack the root certificate as easily as they crack the server certificates and then, using their shiny new root key, they would be in a position to issue new server certificates with extended expiry dates. Therefore, the expiry feature alone doesn't protect against abuse of the key in the distant future. It does provide some value though: forcing people to renew certificates periodically allows the industry to bring in new minimum key length standards from time to time. In practical terms, content signed with a 2048 bit key today will not be valid indefinitely. Imagine in the year 2040 you want to try out a copy of some code you released with a digital signature in 2013. In 2040, that signature may not be trustworthy: most software in that era would probably see the key and tell you there is no way you can trust it. The NIST speculates that 2048 bit keys will be valid up to about the year 2030, so that implies that any code you sign with a 2048 bit key today will have to be resigned with a longer key in the year 2029. You would do that re-signing in the 2048 bit twilight period while you still trust the old signature. Fortunately, there are likely to be few projects where such old code will be in demand. 4096 in practice One of the reasons I decided to write this blog is the fact that some organisations have made the 4096 bit keys very prominent (although nobody has made them mandatory as far as I am aware). Debian's guide to key creation currently recommends 4096 bit keys (although it doesn't explicitly mandate their use) Fedora's archive keys are all 4096 bit keys. The CACert.org project has developed a 4096 bit root These developments may leave people feeling a little bit naked if they have to use a shorter 2048 bit key for any of the reasons suggested above (e.g. for wider choice of smart cards and compatibility with readers). It has also resulted in some people spending time looking for 4096 bit smart cards and compatible readers when they may be better off just using 2048 bits and investing their time in other security improvements. In fact, the "risk" of using only 2048 rather than 4096 bits in the smartcard may well be far outweighed by the benefits of hardware security (especially if a smartcard reader with pin-pad is used) My own conclusion is that 2048 is not a dead duck and using this key length remains a valid decision and is very likely to remain so for the next 5 years at least. The US NIST makes a similar recommendation and suggests it will be safe until 2030, although it is the minimum key length they have recommended. My feeling is that the Debian preference for 4096 bit PGP keys is not based solely on security, rather, it is also influenced by the fact that Debian is a project run by volunteers. Given this background, there is a perception that if everybody migrates from 1024 to 2048, then there would be another big migration effort to move all users from 2048 to 4096 and that those two migrations could be combined into a single effort going directly from 1024 to 4096, reducing the future workload of the volunteers who maintain the keyrings. This is a completely rational decision for administrative reasons, but it is not a decision that questions the security of using 2048 bit keys today. Therefore, people should not see Debian's preference to use 4096 bit keys as a hint that 2048 bit keys are fundamentally flawed. Unlike the Debian keys (which are user keys), the CACert.org roots and Fedora archive signing keys are centrally managed keys with a long lifetime and none of the benefits of using 2048 bit keys is a compelling factor in those use cases. Practical issues to consider when choosing key-length Therefore, the choice of using 2048 or 4096 is not pre-determined, and it can be balanced with a range of other decisions:
  • Key lifetime: is it a long life key, such as an X.509 root for an in-house CA or an OpenPGP primary key? Or is it just for a HTTPS web server or some other TLS server that can be replaced every two years?
  • Is it for a dedicated application (e.g. a closed user group all using the same software supporting 4096 bit) or is it for a widespread user base where some users need to use 2048 bit due to old software/hardware?
  • Is it necessary to use the key(s) in a wide variety of smartcard readers?
  • Is it a mobile application (where battery must be conserved) or a server that is likely to experience heavy load?

Tollef Fog Heen: An otter, please (or, a better notification system)

Recently, there's been discussions on IRC and the debian-devel mailing list about how to notify users, typically from a cron script or a system daemon needing to tell the user their hard drive is about to expire. The current way is generally "send email to root" and for some bits "pop up a notification bubble, hoping the user will see it". Emailing me means I get far too many notifications. They're often not actionable (apt-get update failed two days ago) and they're not aggregated. I think we need a system that at its core has level and edge triggers and some way of doing flap detection. Level interrupts means "tell me if a disk is full right now". Edge means "tell me if the checksums have changed, even if they now look ok". Flap detection means "tell me if the nightly apt-get update fails more often than once a week". It would be useful if it could extrapolate some notifications too, so it could tell me "your disk is going to be full in $period unless you add more space". The system needs to be able to take in input in a variety of formats: syslog, unstructured output from cron scripts (including their exit codes), snmp, nagios notifications, sockets and fifos and so on. Based on those inputs and any correlations it can pull out of it, it should try to reason about what's happening on the system. If the conclusion there is "something is broken", it should see if it's something that it can reasonably fix by itself. If so, fix it and record it (so it can be used for notification if appropriate: I want to be told if you restart apache every two minutes). If it can't fix it, notify the admin. It should also group similar messages so a single important message doesn't drown in a million unimportant ones. Ideally, this should be cross-host aggregation. The notifications should be possible to escalate if they're not handled within some time period. I'm not aware of such a tool. Maybe one could be rigged together by careful application of logstash, nagios, munin/ganglia/something and sentry. If anybody knows of such a tool, let me know, or if you're working on one, also please let me know.

17 June 2013

Evgeni Golov: Running Debian without Unity on a machine that is 64 bit capable!

Sorry Bryan,
I can show you plenty of hardware that is perfectly 64 bit capable but probably never will run Ubuntu and/or Unity. First, what is 64 bit for you? Looking at ubuntu.com/download and getting images from there, one gets the impression, that 64 bit is amd64 (also called x86_64). If one digs deeper to cdimage.ubuntu.com, one will find non-Intel images too: PowerPC and amrhf. As the PowerPC images are said to boot on G3 and G4 PowerPCs, these are 32 bit. Armhf is 32 bit too (arm64/aarch64 support in Linux is just evolving). So yes, if 64 bit means amd64, I do have hardware that can run Unity. But you asked if I have hardware that is 64 bit capable and can run Ubuntu/Unity, so may I apply my definiton of 64 bit here? I have an old Sun Netra T1-200 (500MHz UltraSPARC IIe) running Debian s sparc port, which has a 64 bit kernel and 32 bit userland. Unity? No wai. I do not own any ia64 or s390/s390x machines, but I am sure people do. And guess what, no Unity there either :) Sorry for ranting like this, but 64 bit really just means that the CPU can handle 64 bit big addresses etc. End even then, it not always will do so ;)

Lars Wirzenius: If it quacks like a duck: what is a backup?

The purpose of a backup is to allow you to recover from a disaster with reasonable cost and effort. If you delete a file you shouldn't have, or make changes that you shouldn't have, backups are meant to save you from having to re-create the file, or undo a large amount of steps. Speaking very broadly, any copy of your live data is a backup, but this is a uselessly broad definition. For example, if you use an automatic synchronisation system such as Dropbox or git-annex, to keep your live data in sync between two computers, you could pretend they're backups of each other. However, unless the synchronisation also allows you to keep a history of file versions, it's not a very good backup. If you delete your precious file on one computer, and it gets then deleted on the other computer as well, automatically, perhaps in seconds, then the backup is not of much use. Another common assumption is that a RAID array works as a backup. RAID is an excellent technology that allows you to combine several hard disks so that they protect you against loss of data in case of disk failure. If one disk fails, the others have enough data to re-create the data on the failed disk, using either full copies (RAID-1) or error correction codes (RAID-5, RAID-6). This is not a backup. It doesn't protect you against accidental file deletions. There is also no backup history. A version control system is very much like a backup. It stores copies of many of the versions of your project. However, in most version control systems it's fairly easy to make changes that lose history. Ask anyone who has used git reset to change the tip of the master branch to undo a wrong commit or merge, and then accidentally force-pushed that to the server. This is arguably a normal, if uncommon use of the version control system. A good backup system will protect you from you own mistakes, when you do the kinds of things you're expected to do. Version control systems also rarely capture all your data. When you were five, and made some stuff on the family computer, and saved it on a floppy, and then drew a cute little picture of yourself on the floppy to make it clear to everyone it was your floppy, and not anyone else's, certainly not your bully of your brother's, and your mother kept the floppy for decades because of the cute picture, then that is also not a backup. You didn't even know your Mom had kept it. A reasonable backup is one from which you can restore a working copy of your data, when you need to, without too much effort or expense, compared to the disaster you're experiencing. If the disaster is that you deleted a one-page draft outline of the book you want to write someday, the disaster is not very severe. The cost of restoring should be low. If the disaster is that your plans to become the supreme emperor of the world, and make all people your slaves, are in a spreadsheet on your laptop, and your minions accidentally drove a car over your laptop, and you had accidentally not used a Thinkpad as your laptop, the disaster is quite severe. Unless you recover the spreadsheet, you'll never be able to tell apart the buttons to launch the Moon rocket, to self-destruct your HQ, and to switch channels on your TV, and all your work will be in vain, and you'll never, ever, ever convince the pretty girl with red hair living in the house opposite that she should be interested in you. Also, you'll never be able to move away from your parent's house. So, quite severe. It will be acceptable to go to quite some effort and expense to recover that spreadsheet. It's better if you don't need to, but you will, if you have to. Your backup should also be reasonably up to date. Backing up every Christmas is a fine family tradition, but if you don't make a backup also on Easter, Midsummer, and Aunt Agatha's birthday sometime in September was it, or maybe October, you'll risk losing a whole year's worth of work. A year is a long time, and you might never be able to re-do all the work. Personally, I backup my personal laptop every day to a file server at home, and less often to an online backup server. My work laptop gets backed up once an hour to the company file server, which gets backed up to two backup servers about once a day. You need to balance the risk of losing data and work, and the expense and effort to back up your data. How much is a day's work worth to you, or your employer? How much does a backup system cost? In the next episode, I'll ponder on how many backups are enough.

Petter Reinholdtsen: Debian Edu interview: Victor Ni u

The Debian Edu and Skolelinux distribution have users and contributors all around the globe. And a while back, an enterprising young man showed up on our IRC channel #debian-edu and started asking questions about how Debian Edu worked. We answered as good as we could, and even convinced him to help us with translations. And today I managed to get an interview with him, to learn more about him. Who are you, and how do you spend your days? I'm a 25 year old free software enthusiast, living in Romania, which is also my country of origin. Back in 2009, at a New Year's Eve party, I had a very nice beer discussion with a friend, when we realized we have no organised Debian community in our country. A few days later, we put together the infrastructure for such community and even gathered a nice Debian-ish crowd. Since then, I began my quest as a free software hacker and activist and I am constantly trying to cover as much ground as possible on that field. A few years ago I founded a small web development company, which provided me the flexible schedule I needed so much for my activities. For the last 13 months, I have been the Technical Director of Funda ia Ceata, which is a free software activist organisation endorsed by the FSF and the FSFE, and the only one we have in our country. How did you get in contact with the Skolelinux / Debian Edu project? The idea of participating in the Debian Edu project was a surprise even to me, since I never used it before I began getting involved in it. This year I had a great opportunity to deliver a talk on educational software, and I knew immediately where to look. It was a love at first sight, since I was previously involved with some of the technologies the project incorporates, and I rapidly found a lot of ways to contribute. My first contributions consisted in translating the installer and configuration dialogs, then I found some bugs to squash (I still haven't fixed them yet though), and I even got my eyes on some other areas where I can prove myself helpful. Since the appetite for free software in my country is pretty low, I'll be happy to be the first one around here advocating for the project's adoption in educational environments, and maybe even get my hands dirty in creating a flavour for our own needs. I am not used to make very advanced plannings, so from now on, time will tell what I'll be doing next, but I think I have a pretty consistent starting point. What do you see as the advantages of Skolelinux/Debian Edu? Not a long time ago, I was in the position of configuring and maintaining a LDAP server on some Debian derivative, and I must say it took me a while. A long time ago, I was maintaining a bigger Samba-powered infrastructure, and I must say I spent quite a lot of time on it. I have similar stories about many of the services included with Skolelinux, and the main advantage I see about it is the out-of-the box availability of them, making it quite competitive when it comes to managing a school's network, for example. Of course, there is more to say about Skolelinux than the availability of the software included, its flexibility in various scenarios is something I can't wait to experiment "into the wild" (I only played with virtual machines so far). And I am sure there is a lot more I haven't discovered yet about it, being so new within the project. What do you see as the disadvantages of Skolelinux / Debian Edu? As usual, when it comes to Debian Blends, I see as the biggest disadvantage the lack of a numerous team dedicated to the project. Every day I see the same names in the changelogs, and I have a constantly fear of the bus factor in this story. I'd like to see Debian Edu advertised more as an entry point into the Debian ecosystem, especially amongst newcomers and students. IMHO there are a lot low-hanging fruits in terms of bug squashing, and enough opportunities to get the feeling of the Debian Project's dynamics. Not to mention it's a very fun blend to work on! Derived from the previous statement, is the delay in catching up with the main Debian release and documentation. This is common though to all blends and derivatives, but it's an issue we can all work on. Which free software do you use daily? I can hardly imagine myself spending a day without Vim, since my daily routine covers writing code and hacking configuration files. I am a fan of the Awesome window manager (but I also like the Enlightenment project a lot!), Claws Mail due to its ease of use and very configurable behaviour. Recently I fell in love with Redshift, which helps me get through the night without headaches. Of course, there is much more stuff in this bag, but I'll need a blog on my own for doing this! Which strategy do you believe is the right one to use to get schools to use free software? Well, on this field, I cannot do much more than experiment right now. So, being far from having a recipe for success, I can only assume that: I also see some problems in getting Skolelinux into schools; for example, in our country we have a great deal of corruption issues, so it might be hard(er) to fight against proprietary solutions. Also, people who relied on commercial software for all their lives, would be very hard to convert against their will.

Gunnar Wolf: Cultural objects/goods: When a superhero is too famous for his own good

I found the following news item; if you can read Spanish, you will most probably prefer the original version in the Proceso magazine's site. The subject? The federal police (PGR) and army arrest 17 artisans for making money out of Spiderman. The following translation is mine. Done past midnight, and being quite tired, and translated so this news item can reach a broader audience. All errors are mine (except those carried out by the security forces, that is).
June 13, 2013 Cuernavaca, Morelos. Policement from the General Republic Attorney (Procuradur a General de la Rep blica, PGR) and the Army entered and searched the "3 de mayo" neighbourhood, in the municipality of Emiliano Zapata, detaining 17 ceramist artisans that sold candies, dolls and pi atas shaped like Spiderman. This search was done on the evening of last Wednesday, around 16:00. Federal ministerial policement and army soldiers closed a street with several informal stores and detained workers taht were selling this Marvel Comics character, following said company's denounce. As a result for this operation, 17 artisants were detained, although the same day five of them were freed. The policemen also seized 12 bags of candies, pi atas, ceramics and wooden figures of the superhero. PGR closed down 11 stores where ceramics with this same figure was being sold, accusing the detainees of plagiarizing Spiderman's image, protected under the copyright law. The 12 that remained under detention were put at the Federal Justice's disposal, which prompted that this Thursday, around 10AM, hundreds of sellers of "3 de mayo" went out to PGR's building to demand their friends' freedom, who are facing a bail of up to 200,000 pesos (~USD$18,000). Outraged because they said they were treated as if they were part of a drug ring, hundreds of artisans closed intermitently Avenida Cuauhn huac, where the PGR representation in Morelos state is located. The artisans' pressure helped for the amount of the bail to be lowered from MX$200,000 to MX$16,000, and so they were set free. Francisco Fern ndez Flores, president of the Ceramists Association, criticized the operation because, he said, it was as strong as if they were "drug dealers". The artisans explained that they don't even make the Spiderman figures, they are made by the interns of the Centro Estatal de Reinserci n Social de Atlacholoaya (prision), located in the Xochitepec municipality, who offered them to the ceramists so they could be sold. "The Atlacholoaya inmates do them, we buy them to support them, and turns out we are the delinquents now", said Miriam Monroy, sister of one of the detainees. This information was contradicted by Jes s Valencia Valencia, responsible for Morelos' state prision system, who assured that in said prision no ceramics are done. Fern ndez Flores insisted though that from within the prision they are being offered pi atas, candies and "piggy banks" with Spiderman's shape. Jos Luis Pozo, vicepresident of the Ceramists Union, said that to avoid more such federal operations for copyright breaches, they have committed not to produce or commercialize Marvel superhero figures, and any other characters the authority demands. "We do commit to, from now on, those products singled out to us will not be commercialized", he said. Pozo said that the PGR operation caused losses not just to the detained producers and salesmen, but to over 200 ceramists that had to close their stores in solidarity with their friends. Acording to the artisans, the products were a success until the PGR came, seized the products and detained the salesmen.
And yes, the copyright insanity does not stop. Spiderman is by today a clear part of popular culture. Marvel brilliantly succeeded in creating such a popular icon that everybody recognizes, that everybody identifies with And that everybody should be able to recreate. We are not talking about brand protection. Marvel does not, and will never, commercialize pi atas, ceramics or wooden toys. And even if they were plastic-cast While Spiderman is still under the protection of copyright, as the Berne Convention defines it (and of course, as the much stricter Mexican laws agree), that does not mean that any and every product resembling a Spiderman should be protected. Many ceramists and pi ata makers will create unique pieces of art Ok, handicraft. But reading the copyright law more strictly, Spiderman is more treated as a trademark than as a copyright. And it is a trademark that should be declared as having passed on to the public domain.

Andrew Pollock: [life] City2South run report

Yesterday I ran in the City2South. It was a beautiful day for it, and I really enjoyed the run. Thanks to generous support of my donors, I raised $252 for the Heart Foundation. My official time was 1:20:41, which I'm really happy with. I'd only ever run 14km on the preceding Tuesday, and I ran that in 1:32:57, so to do this run 12 minutes faster, on a completely different course felt like quite the accomplishment. I also ran personal best times for 5K and 10K. It's hard to believe that the guy who came first ran it in 44 minutes. the course was really nice, except for running up Highgate Hill at kilometre 12. That was a bit harsh, but I managed to run all the way up it, nonetheless. It only hit me this afternoon when I was replaying the course in my mind just how far I ran. My normal 10K course doesn't feel all that long because it loops back on itself a lot, so it's deceptive how much ground I cover. From a technical perspective, the race was done very well. I liked that they had a Facebook app that in real time posted updates when I crossed the start, 5K, 10K and finish lines, and the official results were online by the time I got home. That said, as I write, the website is throwing all sorts of errors when I try to download my official finishing certificate, or see my photos and finish line video. All things considered, it was a pretty nice way to spend a Sunday morning. I was up at 5am to be on the 5:44am ferry from Hawthorne, and back home again by 10am. I'm very keen to try running a half marathon now, but my next run is the 10km Bridge to Brisbane in September. That one will be more interesting because presumably it involves running up the Gateway Bridge, and I'll be pushing Zoe in a jogging stroller. I'm not expecting any personal best times for that one.

16 June 2013

Daniel Pocock: Monitoring with Ganglia: an O'Reilly community book project

I recently had the opportunity to contribute to an O'Reilly community book project, developing the book Monitoring with Ganglia in collaboration with other members of the Ganglia team

The project itself, as a community book, pays no royalties back to the contributors, as we have chosen to donate all proceeds to charity. People who contributed to the book include
Robert Alexander, Jeff Buchbinder, Frederiko Costa, Alex Dean, Dave Josephsen, Bernard Li, Matt Massie, Brad Nicholes, Peter Phaal and Vladimir Vuksan and we also had generous assistance from various members of the open source community who assisted in the review process. Ganglia itself started at University of California, Berkeley as an initiative of Matt Massie, for monitoring HPC cloud infrastructure My own contact with Ganglia only began in 2008 when I was offered the opportunity to work full-time on the enterprise-wide monitoring systems for a large investment bank. Ganglia had been chosen for this huge project due to it's small footprint, support for many platforms and it's ability to work on a heterogeneous network as well as providing dedicated features for the bank's HPC grid. This brings me to one important point about Ganglia: it's not just about HPC any more. While it is extremely useful for clusters, grids and clouds, it is also quite suitable for a mixed network of web servers, mail servers, databases and all the other applications you may find in a small business, education or ISP environment. Instantly up and running with packages One of the most compelling features, even for small sites with less than 10 nodes, is the ease of installation: install the packages on Debian, Ubuntu, Fedora, OpenCSW and some other platforms, and it just works. Ganglia nodes will find each other over multicast, instantly, no manual configuration changes necessary. On one of the nodes, the web interface must be installed for viewing the statistics. Dare I say it: it is so easy, you hardly even need the book for a small installation. Where the book is really compelling is if you have hundreds or thousands of nodes, if you want custom charts or custom metrics or anything else beyond just installing the package. If monitoring is more than 10% of your job, the book is probably a must-have. Excellent open source architecture Ganglia's simplicity is largely thanks to the way it leverages other open source projects such as Tobi Oetiker's RRDtool and PHP Anybody familiar with these tools will find Ganglia is particularly easy to work with and customise. Custom metrics: IO service times One of my own contributions to the project has been the creation of ganglia-modules-linux, some plugins for Linux-specific metrics and ganglia-modules-solaris providing some similar metrics for Solaris. These projects on github provide an excellent base for people to fork and implement their own custom metrics in C or C++ The book provides a more detailed account of how to work with the various APIs for Python, C/C++, gmetric (command line/shell scripts) and Java. The new web interface For people who had tried earlier versions of Ganglia (and for those people who installed versions < 3.3.0 and still haven't updated), the new web interface is a major improvement and well worth the effort to install. It is available on the most recent packages (for example, it is in Debian 7 (wheezy) but not in Debian 6.) It was originally promoted as a standalone project (code-named gweb2) but was adopted as the official Ganglia web interface around the release of Ganglia 3.3.0. This web page provides a useful overview of what has changed and here is the original release announcement.

Next.

Previous.