Search Results: "xam"

22 June 2013

Benjamin Mako Hill: Iceowl s Awesome New Icon

If you re a Debian user, you are probably already familiar with some of the awesome icons for IceWeasel (rebranded Mozilla Firefox), IceDove (rebranded Mozilla Thunderbird) and IceApe (rebranded Mozilla SeaMonkey).

iceweasel_icon-200pxicedove_icon-200px iceape_icon-200px I was pretty ambivalent about the decision to rebrand Firefox until I saw some of proposed the IceWeasel icons which in my humble opinion were just too cute, and awesome, to pass up.

iceweasel-old Until very recently however, IceOwl (rebranded Mozilla Sundbird) had no such awesome icon. Quite a while ago, I filed bug #658664 in Debian complaining that iceowl does not include awesome icy owl icons. I wrote:

I was extremely disappointed when I installed Iceowl and discovered that it does not ship with an awesome logo or icons showing a picture of an IceOwl. Instead, it seems to be represented by picture of a (boring) paper calendar which is very generic and not awesome at all. IceWeasel, IceDove, and IceApe each include extremely awesome logos/icons that have really cool looking white illustrations of icy weasels, doves, and apes. IceOwl needs a similarly awesome logo to use as its icon. This bug seems particularly egregious because owls actually live in icy climates and come in white versions! For example: https://commons.wikimedia.org/wiki/File:Snowy_Owl_-_Schnee-Eule.jpg While illustrators need to imagine what an ice ape or ice weasel might look like, there is no such need for imagination in the case of an ice owl! As far as I m concerned, this bug should be release critical. Hopefully, someone will upload a patch quickly!
Finally, after many months of all of us suffering in silence, Nick Morrott came along and fixed the bug with the creation of this new, incredibly awesome, icy owl logo!

iceowl_icon-350px

21 June 2013

Daniel Pocock: Practical challenges for interrupt-free computing

My previous blog on interrupt-free computing has been very well read. I've had a look at some practical implementation possibilities and can share some more details about how to go about it and potential problems. No perfect solution They say that a certain billionaire is so rich that he would not spend 5 seconds bending over to pick up a $100 banknote on the ground because he makes $500 in 5 seconds anyway. A man from Ethiopia may well run a marathon for the same $100. This little anecdote underlines the point that no two people have exactly the same priorities and it is hard to provide a one-size-fits-all solution. A good technical solution needs to let the user have some control over their own priorities as that will go a lot further in respecting their preferences, their objectives and their environment. A practical example: some users may not want to be disturbed by a VoIP call while watching a movie with their PC. Other users may be quite happy to receive these interruptions. Some users may want to only allow this interruption from a subset of possible callers. Building blocks In the original blog on this subject I suggested that Thunderbird and it's popular plugin, Mozilla Lightning calendar would provide one useful framework for integrating events, alerts and to-do items of various grades of importance. I already use these applications myself, from the re-branded icedove and iceowl-extension packages on Debian. They are also available in Fedora and many other distributions. Priority with Lightning An initial look at Lightning reveals a Priority attribute for tasks. It can be set to the values High, Medium or Low. These don't map to the concepts of Urgency and Importance from the Eisenhower method discussed in the original blog. Potential hacks: there are two ideas that come to mind, with no coding changes:
  • One possibility is to set Due dates for any item that has the Urgent characteristic and no date for items without the characteristic. In this case, the Priority attribute can then be used to distinguish Important and non-Important tasks.
  • The other possibility is to use the Categories feature to define four categories: Urgent-Important, non-Urgent-but-Important, Urgent-non-Important and neither-Urgent-Important.
A more elegant solution would be to develop an additional plugin that provides new attributes for the Urgency and Importance characteristics. For server-side storage of these attributes, it would be feasible to use extension properties as defined in RFC 2445, for example, X-urgency and X-importance This approach would make it easier to view a large range of tasks across multiple domains with the most urgent and important tasks at the top. Ideally, the Categories attribute of tasks can be used to distinguish tasks that relate to work (or project/client) from personal tasks (such as planning a holiday) Challenges with Lightning Some of the challenges that I've identified after reviewing this solution:
  • the Categories in Lightning don't appear to be synchronized with the Tags used in Thunderbird for email.
  • Furthermore, the Tags are not synchronized automatically with the list of server-side IMAP keywords and somebody having multiple Thunderbird profiles (e.g. on both a desktop and a laptop) needs to manually synchronize the lists of tags and categories between all their devices.
  • Many people use mobile devices such as smartphones and tablets to access their email, calendar and tasks. The lightweight mail/calendar clients on these devices don't support the full range of features for classification of content. Furthermore, their low-resolution user interfaces may even make it difficult to replicate the same experience on a mobile device. There is some hope, for example, rooting a tablet device and installing Linux would allow it to run a proper installation of Thunderbird with the various plugins described above.
Feeding system events and other data into the pipeline Given the text-based nature of the iCalendar format, it should be easy enough for any application that is capable of generating mail to generate these iCal events with X-Urgency and X-importance properties. These events - such as alerts about security updates - could then be dispatched through the system's mailer. Some additional effort may be needed to avoid duplication. The notification system in the desktop UI may then prioritize events from the mailbox rather than showing all alerts in real-time. Only applications that implicitly need to do so (such as VoIP/chat clients) would display notifications to the user directly. Conclusion: Interrupting the user Finally, after gathering all the events for the user in a mailbox like this, it would be possible for the plugin to be configured to interrupt the user only when some event reaches the user's own urgency/importance threshold. For example:
  • The user may want to be interrupted for Urgent items with a deadline in less than 24 hours, but if the task has a work' category, only interrupt during business hours
  • For all other items, the user only sees them when they open Lightning or some other calendar/task management application to explicitly review things that they need to do.

Dirk Eddelbuettel: Upcoming Rcpp talk in Sydney

The Sydney Users of R Forum (SURF) will be hosting me for a talk on July 10. The focus will be Rcpp for R and C++ integration, and the intent is to have this be really applied with lots of motivating examples. Organizers Louise and Eugene were able to move this to a slightly larger room as the initial capacity of 50 was filled almost immediately. As of right now, the talk page shows a few available slots. If you're in the Sydney are in early July, why not register and swing by? If you're not in the area, the Rcpp Events page lists other upcoming talks too.

20 June 2013

Matt Zimmerman: Scaling Human Systems: Alignment

This is part 1 in a series on organizational design and growth. An important lens for thinking about organizations at this stage of growth is alignment. In an organization which is aligned, the efforts of different people and teams all contribute to forward progress in a shared direction. If two teams are pulling in opposite directions they may make little progress despite great effort, and quickly become frustrated. To take an obvious example, if a marketing team is targeting an audience of large enterprises while the product being developed is only suited for small businesses, the end result of both teams doing a good job will be a failure (i.e. unhappy customers). It s important to note that alignment does not imply sameness. Different teams within an organization can function and behave very differently while still being strongly aligned with each other. When an organization is small, alignment comes naturally. Everyone has some visibility on what everyone else is doing, and when something doesn t line up, the people involved can talk it over and resolve the issue relatively easily. But as the organization grows, the propensity for misalignment increases, and these situations become much more difficult and time-consuming to resolve. The metaphor of the right hand, which doesn t know what the left hand is doing, seems like something that would only happen in larger companies, but it begins much earlier, especially when the company goes through a period of rapid growth. Critical infrastructure, such as communication tools and patterns, lags behind the accelerating needs of the people involved, creating a surprising distance between teams. Organizational alignment is a critical part of scaling successfully. With alignment, growth and momentum are assets. Without it, they are liabilities. Further reading: The Advantage: Why Organizational Health Trumps Everything Else in Business

Craig Sanders: grub-list-kernels.pl

Due to boredom and needing something to do to keep my brain from atrophying, I ve decided to polish some of the rough edges off some of my scripts so that they re suitable for publication. here s the first of them, a perl script to list kernels and other grub menu entries, with numeric indexing suitable for use with grub-set-default and grub-reboot output looks like this: 
# grub-list-kernels.pl
   0    Debian GNU/Linux    
   1    Advanced options for Debian GNU/Linux   
 1>0    Debian GNU/Linux, with Linux 3.9-1-amd64    
 1>1    Debian GNU/Linux, with Linux 3.9-1-amd64 (recovery mode)    
 1>2    Debian GNU/Linux, with Linux 3.8-2-amd64    
 1>3    Debian GNU/Linux, with Linux 3.8-2-amd64 (recovery mode)    
 1>4    Debian GNU/Linux, with Linux 3.8-1-amd64    
 1>5    Debian GNU/Linux, with Linux 3.8-1-amd64 (recovery mode)    
 1>6    Debian GNU/Linux, with Linux 3.7-trunk-amd64    
 1>7    Debian GNU/Linux, with Linux 3.7-trunk-amd64 (recovery mode)    
 1>8    Debian GNU/Linux, with Linux 3.2.0-4-amd64  
 1>9    Debian GNU/Linux, with Linux 3.2.0-4-amd64 (recovery mode)  
   2    Network boot (iPXE) 
   3    Bootable floppy: LSI    
   4    Bootable floppy: freedos-bare   
  Default: 0  "Debian GNU/Linux"
(this example tells me i need to uninstall some old kernels) and here s the script: 
#! /usr/bin/perl 
# parse and list the boot entries in the grub boot menu (grub.cfg)
#
# $Id: grub-list-kernels.pl,v 1.8 2013/06/20 10:36:47 cas Exp $
# Copyright (C) 2007, 2008, 2009, 2010, 2011, 2012, 2013 Craig Sanders <cas@taz.net.au>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
# Partial Revision History:
# 2013-06-20 - better handling of non-numeric default saved entries, code still uglyish but better commented.
# 2013-05-13 - first stab at support for submenus...works but uglified the code
# 2010-11-03 - rewrote for grub2 (grub.cfg rather than menu.lst)
use strict;
my $cfg = shift   '/boot/grub/grub.cfg';
my $grubenv='/boot/grub/grubenv';
# hash of kernels (menu entry index -> menu entry description) 
# e.g. '4' => 'Debian GNU/Linux, with Linux 3.8-1-amd64'
my %K = ();
# hash of menuentry_id_option -> menuindex
# e.g. 'gnulinux-3.8-1-amd64-advanced-6bb6d228-0581-49ae-9d49-dd148c273ecc' => 4
my %M = ();
# read in the menu entries.
open(MENU,"<$cfg")   die "couldn't open $cfg for read: $!\n" ;
my $insubmenu = 0;
my $menuitem=0;
my $submenuitem=0;
my $prefix='';
my $menuindex='';
my $kernel='';
my $menuidoption='';
while (<MENU>)  
  #next unless /^\s*menuentry\s/io;
  chomp;
  if ($insubmenu ne 0)  
    next unless /^\s*(menuentry submenu)\s ^ $/io;
    if (m/^ $/)   $insubmenu = 0 ; $prefix='' ; next  ;
    $menuindex = $prefix . $submenuitem++;
    else  
    next unless /^\s*(menuentry submenu)\s/io;
    if (m/submenu/)  
        $insubmenu=1;
        $prefix = $menuitem . '>';
        $submenuitem = 0;
        $menuindex = $submenuitem;
     ;
    $menuindex = $menuitem++;
   
  # sometimes the default entry saved in grubenv is the menu_id_option
  # rather than numeric.  The %M hash links menu_id_option to menuindex
  # numbers.
  # e.g. 'gnulinux-3.8-1-amd64-advanced-6bb6d228-0581-49ae-9d49-dd148c273ecc' => 4
  $menuidoption='';
  my $line = $_;
  if ($line =~ m/menuentry_id_option/)   
      $line =~ s/.*menuentry_id_option '//;
      $line =~ s/'.*//;
      $menuidoption=$line;
      $M $menuidoption  = $menuindex;
   ;
  s/[^'"]*["']([^'"]*)['"].*/$1/;
  $kernel = $_;
  $K $menuindex  = $kernel;
  #printf "%4s\t%s\t%s\n", $menuindex, $_, $menuidoption;
  printf "%4s\t%s\t\n", $menuindex, $_;
  ;
# now get the default kernel and boot-once kernel if any
my $saved_entry='';
my $prev_saved_entry='';
my $defk = 0;
my $once = 0;
open(GRUBENV, "<$grubenv")   die "couldn't open $grubenv for read: $!\n" ;
while (<GRUBENV>)  
  chomp;
  next unless /saved_entry/;
  if (/^saved_entry/)  
    (undef,$saved_entry) = split /=/;
    elsif (/^prev_saved_entry/)  
    (undef,$prev_saved_entry) = split /=/;
   
 
close(GRUBENV);
$saved_entry = 0 if ($saved_entry eq '');
if ($prev_saved_entry ne '')  
  $defk = $prev_saved_entry;
  $once = $saved_entry;
  else  
  $defk = $saved_entry;
 
# if $defk is non-numeric, look up index in %M
if ($defk !~ /^[0-9]+$/)  
  $defk = $M $defk ;
 
# if $once is non-numeric, look up index in %M
if ($once !~ /^[0-9]+$/)  
  $once = $M $once ;
 
print "\n  Default: $defk";
print "  \"", $K $defk , "\"\n";
if ($prev_saved_entry ne '')  
  print "Boot Once: $once";
  print "  \"", $K $once , "\"\n";
grub-list-kernels.pl is a post from: Errata

19 June 2013

Matthew Garrett: Mir, the Canonical CLA and skewing the playing field

Mir is Canonical's equivalent to Wayland - a display server, responsible for getting application pixmaps onto a screen. It's intended to scale from mobile devices to the desktop, and as such is expected to turn up in Ubuntu Phone before too long[1]. There's already plenty of discussion about whether the technical differences between Wayland and Mir are sufficient to justify Canonical going their own way, so I'm not planning on talking about that.

Like many Canonical-led projects, Mir is under GPLv3 - a strong copyleft license. There's a couple of aspects of GPLv3 that are intended to protect users from being unable to make use of the rights that the license grants them. The first is that if GPLv3 code is shipped as part of a user product, it must be possible for the user to replace that GPLv3 code. That's a problem if your device is intended to be locked down enough that it can only run vendor code. The second is that it grants an explicit patent license to downstream recipients, permitting them to make use of those patents in derivative works.

One of the consequences of these obligations is that companies whose business models depend on either selling locked-down devices or licensing patents tend to be fairly reluctant to ship GPLv3 software. In effect, this is GPLv3 acting entirely as intended - unless you're willing to guarantee that a user can exercise the freedoms defined by the free software definition, you don't get to ship GPLv3 material. Some companies have decided that shipping GPLv3 code would be more expensive than either improving existing code under a more liberal license or writing new code from scratch. Android's a pretty great example of this - it contains no GPLv3 code, and even GPLv2 code (outside the kernel) is kept to a minimum.

Which, given Canonical's focus on pushing Ubuntu into GPLv3-hostile markets, makes the choice of GPLv3 an odd one. This isn't a problem as long as they're the sole copyright holder, because the copyright holder is obviously free to ship their code under as many licenses as they want. But Canonical still aim to foster community involvement, and ideally that includes accepting external contributions to their code. If Canonical simply accepted those contributions under GPLv3 then they'd no longer have the right to relicense the entire codebase, so any contributions are only accepted if the contributor has signed a Contributor License Agreement.

Canonical's CLA is pretty simple. In essence, it grants Canonical the right to use, modify and distribute your code, and it grants Canonical a patent license under any patents you own that may cover the code in question. But, most importantly, it grants Canonical the right to relicense your contribution under their choice of license. This means that, despite not being the sole copyright holder, Canonical are free to relicense your code under a proprietary license.

Given Canonical's market goals, this makes sense. They can relicense Mir (and any other GPLv3 projects they own) under licenses that keep their hardware partners happy, and they can ship in the phone market. Everyone's a winner.

Except, if Canonical want to ship proprietary versions, why not just license Mir under a license that permits that in the first place? This is where the asymmetry comes in. The Android userland is released under a permissive license that allows anyone to take Google's code, modify it as they wish and ship it on whatever hardware they want. I could legally start a company that provided customised versions of Android to phone vendors without them having any GPLv3 concerns. I won't be able to do that with Ubuntu Phone.

I'm a fan of GPLv3. I think the provisions it contains to support user freedom are important. I hate the growing trend of using free software to build devices that are, effectively, impossible for the end user to modify. If Canonical were releasing software under GPLv3 because of a commitment to free software then that would be an amazing thing. But it's pretty much impossible to square the CLA's requirement that contributors grant Canonical the right to ship under a proprietary license with a commitment to free software. Instead you end up with a situation that looks awfully like Canonical wanting to squash competition by making it impossible for anyone else to sell modified versions of Canonical's software in the same market.

Canonical aren't doing anything illegal or immoral here. They're free to run their projects in any way they choose. But retaining the right to produce proprietary versions of external contributions without granting equivalent reciprocal rights isn't consistent with caring about free software or contributing to the wider Linux community, especially if it means you get to exclude those external contributors from the market you're selling their code into.

(Edit to add: a friend in the contracting industry points out that it also prevents vendors who won't ship GPLv3 from using external contractors to work on Mir - they have to go to Canonical, because only Canonical can relicense contributions under a proprietary license.)

[1] Right now Ubuntu Phone is using Surfaceflinger, the Android display server, but that's apparently just an interim solution.

comment count unavailable comments

Benjamin Mako Hill: Job Market Materials

Last year, I applied for academic, tenure track, jobs at several communication departments, information schools, and in HCI-focused computer science programs with a tradition of hiring social scientists. Being on the market as it is called is both scary and time consuming. Like me, many candidates have never been on the market before. Candidates are asked to produce documents in genres e.g., cover letters, research statements, teaching statements, diversity statements that most candidates have never written, read, or even heard of. Candidates often rely on their supervisors for advice. I did so and my advisors were extremely helpful. The reality, however, is that although candidates advisors may sit on hiring committees, most have not been on candidates side of job market themselves for years or even decades. The Internet is full of websites, like the academic jobs wiki, Academia StackExchange, and the Chronicle of Higher Education forums for people on the market. Confused and insecure candidates ask questions of the form, Does blank matter? and the answer is usually, Doing/having blank may help/hurt, but it is only one factor of many. The result is that candidates worry about everything. Then they worry about what they should be worrying about, but are not. The most helpful thing, for me, was to read and synthesize the material submitted by recent successful job market candidates. For example, Michael Bernstein a friend from MIT, now at Stanford published his research and teaching statements on his website and I found both useful as I prepared mine. That said, I was surprised by how little material like this I could find on the web. For example, I could not find any examples of recent job market cover letters from successful candidates in fields close to mine. So to help fill this gap, I am publishing all of my job market material. I ve posted both the PDFs of the material I submitted as well as the LaTeX templates I used to generate the documents in my packet. My packet included: I hope people going on the market will find these materials useful. Obviously, you should not copy or reuse the text of any of my material. It is your application, after all. That said, please do help yourself to the formatting and structure. Finally, I would encourage anyone who builds on my material to republish their own material to help other candidates. If you do, I d appreciate a link back or comment on this blog post so that my readers can find your improvements.

Daniel Pocock: Australia's war on brains (and immigration)

Some weeks ago, our Prime Minister was slashdotted when she suspended her usual racist attacks on refugees to attack foreign IT workers and the companies that employ them with absurd accusations of "rorts" and "stealing" jobs. Today, she's introduced new laws in the parliament aiming to further bastardization of intelligent, skilled and educated workers and anybody who associates with them, including Australian employers. Unwarranted attention on a minority of IT workers This is not just some random bill before the parliament. There are just two weeks left before the parliament concludes and an election campaign begins. It is clear that what we are seeing now is the Real Julia coming though, choosing to make the small minority of foreign workers in our country at the center of people's thoughts as they go to vote in September. A debt of gratitude to foreign workers These verses from our national anthem, Advance Australia Fair say a lot about how Australia became what it is today:
For those who've come across the seas

We've boundless plains to share;

With courage let us all combine

To Advance Australia Fair.
The last 200 years of Australia's history has been a story of immigration. It is not something to be afraid of: our forefathers celebrated it. Foreign workers: why Australia needs you Australia has had some appalling flops in IT and engineering: There is no doubt in my mind that additional foreign workers would have made a positive contribution to all of these problems or can do so in the future. The sun never sets on IT Every day, I collaborate with dozens of IT specialists all over the world through the virtual workplace that is the Internet, particularly in the free and open source software community. Many of these people, I've never even met and in most cases I don't even know where they are, where they were born or what is the colour of their skin. Those details wouldn't make any difference to the way that we work in IT today. How many IT managers have time to waste dealing with more real world bureaucracy when they've experienced online, global productivity? How many IT workers feel demotivated by having to explain trivial details about their personal life to a Government bureaucrat who doesn't understand their skills and just looks at their colour? If you think about it, any immigration officer who really understands IT wouldn't be an immigration officer. They would be working in IT themselves. Immigration officers, who don't understand IT, are now going to be further empowered to bully companies away from employing some talented workers on the basis of race or nationality. Hiring managers will be intimidated into these prejudiced and biased decisions by delays, processing fees and invasive demands for sensitive documents about business planning and recruitment strategies. Australia's immigration system already has a horrendous reputation. Any visa application seems to take more than a year: no small company can keep a job position vacant that long. Families can't plan their children's schooling. Other life events come and go. There are exhorbitant fees, 1000% higher than in other western countries. Fewer and fewer self-respecting skilled workers are willing to put their spouses and children through the degrading medical examinations. Judging the impact of poor immigration policy While the economic impact of this immigration mess on industry is hard to quantify with an exact figure, we can take some insight from the education system. As the visa system has been hijacked by racists over the last 10 years, there has been a dramatic fall in participation (and revenues) from foreign students. In one year, enrolments (and revenue) fell 30%. This is not just bad for the balance sheets of the universities, it also means that in a future where commerce is global, Australians are more and more isolated and inexperienced culturally. IT workers and their employers have plenty of choices: Australia's close neighbor, Singapore, is one of them. Visas are granted in 2 weeks, no degrading medical exam required, low taxes and tropical sunshine all year round. Many companies that find it impractical to deal with Australia's bureaucracy end up moving their best Australian workers to places like Singapore to be part of a global team. This can't be good for the workforce that is left behind without jobs. The training delusion Government officials continue to rant and rave about companies failing to train Australian workers. The new laws supposedly force companies to "fix" this problem and train Australian workers. This, too, is a delusion: employers are not to blame. Some of the best Australian workers are already long gone to places like Singapore, London and the US. With talented foreign workers denied the opportunity to come and fill the void, there is less opportunity for skills to be acquired by more junior workers in Australian workplaces. It is also extremely difficult for more junior workers to get a foot in the door in the international job market and the primary reason for this is the Australian Government's failure to fund university programs beyond a bachelor's degree. Compare this to Europe and the US where all competent graduates are funded through to a Masters or PhD program. The bottom line is that more junior workers are denied the opportunity to get the best training either at home or abroad and in both cases it's not the foreign workers that can be blamed: it's the Government's own fault. Why do we need skilled foreign workers when Australians can win Nobel Prizes? The Australian press recently went into a frenzy when an Australian won the Nobel Prize for physics. There was a catch though: he's a migrant from the United States (just don't tell the Prime Minister). Dr. Schmidt migrated to Australia 20 years ago when the immigration system was not the same as today. Today, future Nobel Prize winners are being shown a brick wall - maybe we even have one of them rotting away in our death camps or left in the sea for sharks to eat. The BBC recently revealed that Britain's successor to Stephen Hawking may be a young girl who migrated from India - it is chilling to imagine where a child like this may be hidden away under Australia's immigration system. Bureaucracy leads to fraud and exploitation It's been clearly demonstrated that wherever you have elaborate, artificial systems of bureaucracy it leads to inefficiency, it suppresses innovation and in the worst cases it enables fraud and exploitation. The typical examples usually involve police in some third-world African nation setting up road blocks and collecting fees from travellers who want to pass the queues. This type of opportunism has also been found in Australia's immigration system, with one Federal politician already directly implicated and jailed for his role in a visa racket. Gillard's own bullet man A Queensland pensioner made international headlines recently when he was caught sending bullets in the mail to the Prime Minister. His demands were clear: stop immigration. While most world leaders refuse to let nutcases like this dictate their actions, Gillard appears to have been transcribing his racist letters directly into these newest immigration laws. It is a sad reality that Australian politics regularly seeks to appeal to the worst instincts in people like bullet man. The ultimate political failure When politicians stoop to the level of demonizing immigrants it is usually a clue that the politicians themselves are past their use-by dates and out of fresh policy ideas. When former French president Sarkozy tried to play the racist card in his campaign for re-election, it bit him in the bum and he was swept from power by the socialists. As always in politics, there is an element of hypocrisy at work: neither our Head of Government (the Prime Minister) nor our Head of State (the Queen) was born in Australia. Gillard was born in Wales and migrated to Australia as a child. If Australians don't vote for her in September, will she be given 28 days notice to pack her bags and go back home'? From the frying pan and into the fire Tony Abbott, Australia's would-be Prime Minister contemplating his colleague's statements on bestiality The scariest thing is that if Australians see through this racist charade and refuse to vote for it, we could end up with something equally obnoxious: the other major political party is now gaining worldwide attention for their campaign linking gay marriage and homosexuals to bestiality. Which prejudice is the lesser evil: racism or homophobia?

Julien Viard de Galbert: Building a custom debian CD

The French version is available on linuxembedded.fr : Cr e un CD d installation d une debian sp cialis e The goal is to build a debian install CD suitable for the distribution of a complete system including the operating system and applications. Debian already has a tool for that purpose: simple-cdd. Simple-cdd is a set of scripts wrapping debian-cd which is the tool used to build official CDs. In our case, we will include some non-free packages (firmwares for instance) and application specific packages in the system. Using simple-cdd Simple-cdd gets some of its configuration from your host computer, so it s recommended to work on a machine similar to your target (in particular the machine should use the same architecture: i386 or amd64). All the following actions should be done in a working directory that will contain the downloads and configurations for our system. We will start by creting a configuration file my-cdd.conf stating the mirror and components to use: 
debian_mirror="http://ftp2.fr.debian.org/debian/"
mirror_components="main contrib non-free"
Then we will launch the build-simple-cdd tool once. This will build a local mirror of the needed packages that we will reused later. So be patient this can be a bit long. 
build-simple-cdd --conf ./my-cdd.conf
Once done, you should get a CD iso in the images subfolder. Customizing the image To customize our image we will create a profile my-profile that will define which package to install and which extra package to include on the CD (dev packages for instance). Create a profiles folder: 
mkdir profiles
Selecting the packages If you ve got an already installed machine (the one you used to test your application) you can ask dpkg to list the installed packages from that machine and use that as a basis. 
dpkg --get-selections >package-list
In this file only the names of packages in the install state are needed: 
grep -e '\<install$' package-list   awk ' print $1 ;' >profiles/my-profile.downloads
The file my-profile.downloads is a list of packages to include on the CD-ROM. The list of program to install is my-profile.packages . The tool will resolve dependencies, so we don t need to track down all dependencies add add them to these files. Specific packages We want to add some custom packages that we built ourselves to the CD, those packages not present in the debian archive. Let s create a local_pkg folder and fill it with all of our packages. We now can update the configuration to use all those files and make the profile my-profile the default profile. 
debian_mirror="http://ftp2.fr.debian.org/debian/"
mirror_components="main contrib non-free"
simple_cdd_dir=$(pwd)
profiles="my-profile"
auto_profiles="my-profile"
local_packages="$simple_cdd_dir/local_pkg/"
If you now run the tool again, you ll get a full custom install CD! 
build-simple-cdd --conf ./my-cdd.conf
However you might have missed some specific packages or their dependencies, in that case they will be missing on the CD. To find this out, check the messages for a line like: 
WARNING: missing optional packages from profile my-profile: libevent-2.0-5 [...]
In that case libevent from squeeze-backports was missing. Simple-cdd is not able to download packages from backports, so I just added the packages to local_pkg . Customizing the installer Simple-cdd automatically uses a default profile. This profile might not suite your needs. To override it you just have to place modified file in the profiles folder. To do so, first copy the desired file from /usr/share/simple-cdd/profiles/ and update it. For instance the default.pressed file: 
cp /usr/share/simple-cdd/profiles/default.pressed profiles
Then update the wanted options in this file. For example the partitioning options. Also as we enabled contrib and non-free you should uncomment the following lines: 
d-i apt-setup/non-free  boolean true
d-i apt-setup/contrib boolean true
Adding extra files to the CD Finally we want to add the documentation, sources and other files to the CD. Simple-cdd can add files to the simple-cdd folder but there is no way to add a complete directory structure. However, as we can override the profiles, we can also override some scripts, especially tools/build/debian-cd that builds the CD. 
mkdir -p tools/build
cp /usr/share/simple-cdd/tools/build/debian-cd tools/build
Then add the following lines after the $extras_base_dir cleanup: 
if [ -d "$cd_extras" ]; then
  mkdir -p "$extras_base_dir"
  cp -a "$cd_extras/." "$extras_base_dir"
fi
Finally let s add the cd_extras variable to our configuration file : 
cd_extras="$simple_cdd_dir/local_extras/"
Be careful to not create files conflicting with the CD content, nothing would warn you about that! Conclusion We now have a way of building a debian install CD including any files we might want to deliver with our system: sources and documentation for example.

17 June 2013

Lars Wirzenius: If it quacks like a duck: what is a backup?

The purpose of a backup is to allow you to recover from a disaster with reasonable cost and effort. If you delete a file you shouldn't have, or make changes that you shouldn't have, backups are meant to save you from having to re-create the file, or undo a large amount of steps. Speaking very broadly, any copy of your live data is a backup, but this is a uselessly broad definition. For example, if you use an automatic synchronisation system such as Dropbox or git-annex, to keep your live data in sync between two computers, you could pretend they're backups of each other. However, unless the synchronisation also allows you to keep a history of file versions, it's not a very good backup. If you delete your precious file on one computer, and it gets then deleted on the other computer as well, automatically, perhaps in seconds, then the backup is not of much use. Another common assumption is that a RAID array works as a backup. RAID is an excellent technology that allows you to combine several hard disks so that they protect you against loss of data in case of disk failure. If one disk fails, the others have enough data to re-create the data on the failed disk, using either full copies (RAID-1) or error correction codes (RAID-5, RAID-6). This is not a backup. It doesn't protect you against accidental file deletions. There is also no backup history. A version control system is very much like a backup. It stores copies of many of the versions of your project. However, in most version control systems it's fairly easy to make changes that lose history. Ask anyone who has used git reset to change the tip of the master branch to undo a wrong commit or merge, and then accidentally force-pushed that to the server. This is arguably a normal, if uncommon use of the version control system. A good backup system will protect you from you own mistakes, when you do the kinds of things you're expected to do. Version control systems also rarely capture all your data. When you were five, and made some stuff on the family computer, and saved it on a floppy, and then drew a cute little picture of yourself on the floppy to make it clear to everyone it was your floppy, and not anyone else's, certainly not your bully of your brother's, and your mother kept the floppy for decades because of the cute picture, then that is also not a backup. You didn't even know your Mom had kept it. A reasonable backup is one from which you can restore a working copy of your data, when you need to, without too much effort or expense, compared to the disaster you're experiencing. If the disaster is that you deleted a one-page draft outline of the book you want to write someday, the disaster is not very severe. The cost of restoring should be low. If the disaster is that your plans to become the supreme emperor of the world, and make all people your slaves, are in a spreadsheet on your laptop, and your minions accidentally drove a car over your laptop, and you had accidentally not used a Thinkpad as your laptop, the disaster is quite severe. Unless you recover the spreadsheet, you'll never be able to tell apart the buttons to launch the Moon rocket, to self-destruct your HQ, and to switch channels on your TV, and all your work will be in vain, and you'll never, ever, ever convince the pretty girl with red hair living in the house opposite that she should be interested in you. Also, you'll never be able to move away from your parent's house. So, quite severe. It will be acceptable to go to quite some effort and expense to recover that spreadsheet. It's better if you don't need to, but you will, if you have to. Your backup should also be reasonably up to date. Backing up every Christmas is a fine family tradition, but if you don't make a backup also on Easter, Midsummer, and Aunt Agatha's birthday sometime in September was it, or maybe October, you'll risk losing a whole year's worth of work. A year is a long time, and you might never be able to re-do all the work. Personally, I backup my personal laptop every day to a file server at home, and less often to an online backup server. My work laptop gets backed up once an hour to the company file server, which gets backed up to two backup servers about once a day. You need to balance the risk of losing data and work, and the expense and effort to back up your data. How much is a day's work worth to you, or your employer? How much does a backup system cost? In the next episode, I'll ponder on how many backups are enough.

Tanguy Ortolo: Encryption without a certification layer is (partly) useless

With the PRISM scandal, there has been some talk about encrypted communication systems. For instance, BitMessage is often introduced as an easy and secure message system, that would allow you to communicate with no possible eavesdropping. Apple is also making similar claims about their systems iMessage and FaceTime. This is a good time to remind this: without direct contact or a certification layer, encryption systems are not secure! Or at least, not as secure as you would expect, as they do allow some kind of eavesdropping.Let us take the example of BitMessage:
  1. Alice sends her BitMessage address to Bob by email, an insecure channel;
  2. Mallory catches that email message and changes Alice's BitMessage address with his own;
  3. Bob sends a BitMessage for Alice to the address he received, which he thinks is Alice's, but is in fact Mallory's;
  4. Mallory receives the message, reads it, modifies it if he wants, and sends it to Alice.
This is a man-in-the-middle attack, a kind of active eavesdropping technique that requires read-modify access to a communication line. Without a specific way to ensure that a cryptographic key really belongs to its alleged owner, a cryptographic system is vulnerable to such attacks, even if it does protect against simpler attacks (like passive eavesdropping or introduction of MiTM after the introduction sequence). When you read that a cryptographic system provides end-to-end security and is impervious to eavesdropping, while it provides no mean to make sure you are in contact to the right person, remember to consider this affirmation as what it is: a lie.

Petter Reinholdtsen: Debian Edu interview: Victor Ni u

The Debian Edu and Skolelinux distribution have users and contributors all around the globe. And a while back, an enterprising young man showed up on our IRC channel #debian-edu and started asking questions about how Debian Edu worked. We answered as good as we could, and even convinced him to help us with translations. And today I managed to get an interview with him, to learn more about him. Who are you, and how do you spend your days? I'm a 25 year old free software enthusiast, living in Romania, which is also my country of origin. Back in 2009, at a New Year's Eve party, I had a very nice beer discussion with a friend, when we realized we have no organised Debian community in our country. A few days later, we put together the infrastructure for such community and even gathered a nice Debian-ish crowd. Since then, I began my quest as a free software hacker and activist and I am constantly trying to cover as much ground as possible on that field. A few years ago I founded a small web development company, which provided me the flexible schedule I needed so much for my activities. For the last 13 months, I have been the Technical Director of Funda ia Ceata, which is a free software activist organisation endorsed by the FSF and the FSFE, and the only one we have in our country. How did you get in contact with the Skolelinux / Debian Edu project? The idea of participating in the Debian Edu project was a surprise even to me, since I never used it before I began getting involved in it. This year I had a great opportunity to deliver a talk on educational software, and I knew immediately where to look. It was a love at first sight, since I was previously involved with some of the technologies the project incorporates, and I rapidly found a lot of ways to contribute. My first contributions consisted in translating the installer and configuration dialogs, then I found some bugs to squash (I still haven't fixed them yet though), and I even got my eyes on some other areas where I can prove myself helpful. Since the appetite for free software in my country is pretty low, I'll be happy to be the first one around here advocating for the project's adoption in educational environments, and maybe even get my hands dirty in creating a flavour for our own needs. I am not used to make very advanced plannings, so from now on, time will tell what I'll be doing next, but I think I have a pretty consistent starting point. What do you see as the advantages of Skolelinux/Debian Edu? Not a long time ago, I was in the position of configuring and maintaining a LDAP server on some Debian derivative, and I must say it took me a while. A long time ago, I was maintaining a bigger Samba-powered infrastructure, and I must say I spent quite a lot of time on it. I have similar stories about many of the services included with Skolelinux, and the main advantage I see about it is the out-of-the box availability of them, making it quite competitive when it comes to managing a school's network, for example. Of course, there is more to say about Skolelinux than the availability of the software included, its flexibility in various scenarios is something I can't wait to experiment "into the wild" (I only played with virtual machines so far). And I am sure there is a lot more I haven't discovered yet about it, being so new within the project. What do you see as the disadvantages of Skolelinux / Debian Edu? As usual, when it comes to Debian Blends, I see as the biggest disadvantage the lack of a numerous team dedicated to the project. Every day I see the same names in the changelogs, and I have a constantly fear of the bus factor in this story. I'd like to see Debian Edu advertised more as an entry point into the Debian ecosystem, especially amongst newcomers and students. IMHO there are a lot low-hanging fruits in terms of bug squashing, and enough opportunities to get the feeling of the Debian Project's dynamics. Not to mention it's a very fun blend to work on! Derived from the previous statement, is the delay in catching up with the main Debian release and documentation. This is common though to all blends and derivatives, but it's an issue we can all work on. Which free software do you use daily? I can hardly imagine myself spending a day without Vim, since my daily routine covers writing code and hacking configuration files. I am a fan of the Awesome window manager (but I also like the Enlightenment project a lot!), Claws Mail due to its ease of use and very configurable behaviour. Recently I fell in love with Redshift, which helps me get through the night without headaches. Of course, there is much more stuff in this bag, but I'll need a blog on my own for doing this! Which strategy do you believe is the right one to use to get schools to use free software? Well, on this field, I cannot do much more than experiment right now. So, being far from having a recipe for success, I can only assume that: I also see some problems in getting Skolelinux into schools; for example, in our country we have a great deal of corruption issues, so it might be hard(er) to fight against proprietary solutions. Also, people who relied on commercial software for all their lives, would be very hard to convert against their will.

16 June 2013

Daniel Pocock: Monitoring with Ganglia: an O'Reilly community book project

I recently had the opportunity to contribute to an O'Reilly community book project, developing the book Monitoring with Ganglia in collaboration with other members of the Ganglia team

The project itself, as a community book, pays no royalties back to the contributors, as we have chosen to donate all proceeds to charity. People who contributed to the book include
Robert Alexander, Jeff Buchbinder, Frederiko Costa, Alex Dean, Dave Josephsen, Bernard Li, Matt Massie, Brad Nicholes, Peter Phaal and Vladimir Vuksan and we also had generous assistance from various members of the open source community who assisted in the review process. Ganglia itself started at University of California, Berkeley as an initiative of Matt Massie, for monitoring HPC cloud infrastructure My own contact with Ganglia only began in 2008 when I was offered the opportunity to work full-time on the enterprise-wide monitoring systems for a large investment bank. Ganglia had been chosen for this huge project due to it's small footprint, support for many platforms and it's ability to work on a heterogeneous network as well as providing dedicated features for the bank's HPC grid. This brings me to one important point about Ganglia: it's not just about HPC any more. While it is extremely useful for clusters, grids and clouds, it is also quite suitable for a mixed network of web servers, mail servers, databases and all the other applications you may find in a small business, education or ISP environment. Instantly up and running with packages One of the most compelling features, even for small sites with less than 10 nodes, is the ease of installation: install the packages on Debian, Ubuntu, Fedora, OpenCSW and some other platforms, and it just works. Ganglia nodes will find each other over multicast, instantly, no manual configuration changes necessary. On one of the nodes, the web interface must be installed for viewing the statistics. Dare I say it: it is so easy, you hardly even need the book for a small installation. Where the book is really compelling is if you have hundreds or thousands of nodes, if you want custom charts or custom metrics or anything else beyond just installing the package. If monitoring is more than 10% of your job, the book is probably a must-have. Excellent open source architecture Ganglia's simplicity is largely thanks to the way it leverages other open source projects such as Tobi Oetiker's RRDtool and PHP Anybody familiar with these tools will find Ganglia is particularly easy to work with and customise. Custom metrics: IO service times One of my own contributions to the project has been the creation of ganglia-modules-linux, some plugins for Linux-specific metrics and ganglia-modules-solaris providing some similar metrics for Solaris. These projects on github provide an excellent base for people to fork and implement their own custom metrics in C or C++ The book provides a more detailed account of how to work with the various APIs for Python, C/C++, gmetric (command line/shell scripts) and Java. The new web interface For people who had tried earlier versions of Ganglia (and for those people who installed versions < 3.3.0 and still haven't updated), the new web interface is a major improvement and well worth the effort to install. It is available on the most recent packages (for example, it is in Debian 7 (wheezy) but not in Debian 6.) It was originally promoted as a standalone project (code-named gweb2) but was adopted as the official Ganglia web interface around the release of Ganglia 3.3.0. This web page provides a useful overview of what has changed and here is the original release announcement.

15 June 2013

Lars Wirzenius: "Um, I know I should": talking about backups

... backups? did someone talk about backups? I'm sure I heard someone mention backups here somewhere. Backups! BACKUPS! BACKUPS ARE AWESOME!
That's a direct quote from my recent IRC history. I find backups quite interesting, particularly from an implementation point of view, and I may sometimes obsess about them a little bit. This is why I've written my own backup software. I'm unusual: most people find backups boring at best, and tedious most of the time. When I talk with people about backups, the usual reaction is "um, I know I should". There are a lot of reasons for this. One is that backups are a lot like insurance: you have to spend time, effort, money, up front, to have any use for them. Another is that the whole topic is scary: you have to think about when things go wrong, and that puts people off. A third reason is that while there are lots of backup tools and methods, it's not always easy. After all, backups are about answering the question, "what can I do to keep my data safe, whatever happens in the future?". I've spent a fair bit of the past several years thinking about backups. This is the first in a series of blog posts about backups, where I share my thinking about the topic. Perhaps it can be of some use to others. At least people will poke holes in my delusions. In this post, I'll define some terminology. I am not a backup scholar, and I may have invented some of these terms myself, and they may be different from what real sysadmins use. I'll define the words I use, so we can understand each other. Live data is the data you work with or keep. It's the files on your hard drive: the documents you write, the photos you save, the unfinished novels you wish you'd finish. A backup is a spare copy of your live data. If you lose some or all of your live data, you can get it back ("restore") from your backup. The backup copy is, by practical necessity, older than your live data, but if you made the backup recently enough, you won't lose much. Sometimes it's useful to have more than one old backup copy of your live data. You can have a sequence of backups, made at different times, giving you a backup history. Each copy of your live data in your backup history is a generation. This lets you retrieve a file you deleted a long time ago, but didn't realise you needed until now. If you only keep one backup version, you can't get it back, but if you keep, say, a daily backup for a month, you have a month to realise you need it, before it's lost forever. The place your backups are stored is the backup repository. You can use many kinds of backup media for backup storage: hard drives, tapes, optical disks (DVD-R, DVD-RW, etc), USB flash drives, online storage, etc. Each type of medium has different characteristics: size, speed, convenicence, reliability, price, which you'll need to balance for a backup solution that's reasonable for you. You may need multiple backup repositories or media, with one of them located off-site, away from where your computers normally live. Otherwise, if you house burns down, you'll lose all your backups too. You need to verify that your backups work. It would be awkward to go to the effort and expense of making backups and then not be able to restore your data when you need to. You may even want to test your disaster recovery by pretending that all your computer stuff is gone, except for the backup media. Can you still recover? You'll want to do this periodically, to make sure your backup system keeps working. Most live data is precious in that you'll be upset if you lose it. Some live data is not precious: your web browser cache probably isn't, for example. This distinction can let you limit the amount of data you need to back up, which can significantly reduce your backup costs. There is a very large variety of backup tools. They can be very simple and manual: you can copy files to a USB drive using your file manager, once a blue moon. They can also be very complex: enterprise backup products that cost huge amounts of money and come with a multi-day training package for your sysadmin team, and which require that team to function properly. You'll need to define a backup strategy to tie everything together: what live data to back up, to what medium, using what tools, what kind of backup history to keep, and how to verify that they work. That's the groundwork. In the next episode, I'll blather about what is a backup, and what isn't.

14 June 2013

Steve Kemp: Debian is missing a tool, want to write it?

Seeing this piece in the news, about how Debian-Multimedia.org is now unsafe, I was reminded we don't have a tool to manipulate sources.lists entries. For example: 
$ apt-sources list
..
deb http://ftp.uk.debian.org/debian/ squeeze main non-free contrib
deb-src http://ftp.uk.debian.org/debian/ squeeze main
deb http://security.debian.org/ squeeze/updates main
deb-src http://security.debian.org/ squeeze/updates main
..
How about listing only my repos? 
$ apt-sources list steve.org.uk
deb-src http://packages.steve.org.uk/firefox-wrapper/squeeze/ ./
deb     http://packages.steve.org.uk/firefox-wrapper/squeeze/ ./
deb     http://packages.steve.org.uk/meta/squeeze/ ./
deb-src http://packages.steve.org.uk/meta/squeeze/ ./
deb-src http://packages.steve.org.uk/minidlna/squeeze/ ./
deb     http://packages.steve.org.uk/minidlna/squeeze/ ./
Now add in a command to delete lines matching a given pattern: 
# apt-sources delete debian-multimedia.org
Doesn't that seem like a tool that should exist? I've added this quick hack to this repository which you can submit pull requests against, or use as a base. TODO: Write the "add" handler. Neaten. Ever felt jealous that Ubuntu users can add PPAs? Nows your chance to do something like this: 
# apt-sources add "deb http://packages.steve.org.uk/lumail/wheezy/ ./"

13 June 2013

Daniel Pocock: Skype Scandal and the Jedi Council

Normally Skype isn't mentioned by name on this blog, but today I'm making an exception. When a new scandal broke out in the Australian Army today, one of the first questions asked was how does it compare to the Skype Scandal which had everybody talking (and blushing) in 2011. A concise military reply direct from the General: "I'd say it's worse than the Skype matter" (the original Skype Scandal cadets are going on trial in six weeks) Jedi Council brings cultural change in the Army: from Skype to email Since the original scandal, there have been plenty of PR-orchestrated headlines about cultural change in the Army. Sadly, it seems that several ranking officers believed that was a cue to simply start using email instead of live feeds Operating under the operational code name Jedi Council, they've apparently been swapping annotated images of female colleagues and members of the public service. A more detailed account summarises it: "They were calling themselves the Jedi Council but they were doing the exact opposite of what Jedis are supposed to do" While the Skype scandal involved cadets behaving much like college kids elsewhere, this latest scandal involves the officer corp, the most senior officer implicated is allegedly a Lieutenant Colonel. Senior ranking officers like this have given many years of dedicated service to the country. They are widely respected and the vast majority of officers in these positions are unlikely to ever be implicated in such a scandal. Officers are in leadership roles, it should be no surprise if they have top secret security clearances. After the revelation that Australia has a direct and high-volume data sharing arrangement with the US Government (and in turn Facebook, Google, etc), there are all kinds of questions about whether a couple of rogue officers have access to pursue their private Jedi Council project using tools like PRISM and Boundless Informant. It's already been confirmed that they were keeping their own Jedi database of their victim's names, phone numbers, home addresses and various clothing measurements and if they are already emailing these photos about on Defence computers, does it seem improbable that they would be using PRISM or other sources to cross-reference their data? Behavior described as systematic and predatory It can't be ruled out that some defence computer systems may have direct access to NSA copies of images from those invasive airport body scanners - like these pictures that were not supposed to exist. Just like the British undercover cops stealing the identities of dead children and fully engaging with impressionable young women from environment campaign groups, there are a long list of possible abuses when the security state is over-empowered. While this latest Army example involves private pursuits, it is not hard to imagine many other permutations: perhaps a couple of rogue Government employees making unauthorised use of PRISM data for a commercial objective and gaining the upper hand on private sector competitors who have no such data feed. 100% trust? In every organisation there will always be a few people bending the rules. It's human nature, if it wasn't that way, we'd all be robots (or drones). The Army and the Catholic Church tend to be hammered a lot more in the public eye when these transgressions are discovered although statistically their crime rates are no higher than average. The bottom line is that no organisation is perfect and people have to take some responisbility to object to the excessive growth of the security state and keep our data to ourselves as you just never know who you can trust.

12 June 2013

Dominique Dumont: Improving error messages of Dpkg dependency parser

Hello Config::Model::Dpkg project (a Debian source package model based on Config::Model) is partly based on a ParseRec::Descent grammar. This grammar is used to parse the dependency of a Debian source package. This article will show how such a grammar is written, its limitation regarding error handling and how to improve the situation. Debian package main data is described in debian/control file. This file can feature a list of dependencies, i.e. a list of package that must be installed for the package to work. These dependencies are declared in fields like Build-Depends , or Depends as a list of package. For Dpkg model purpose, I needed only to parse one item of a dependency list at a time. This dependency item can be a simple package name: foo or a package name with a version requirement: foo ( > 1.24 ) or a package name with architectures restrictions: foo [alpha amd64 hurd-arm linux-armeb] or both: foo ( > 1.24 ) [alpha amd64 hurd-arm linux-armeb] or a list of alternate choices combining the possibilities above: foo ( > 1.24 ) bar [ linux-any] baz ( << 3.14 ) [ ! hurd-armel !hurd-armeb ] or a variable that is replaced during package build: $ perl-depends Writing a Parse::RecDescent grammar to parse this is relatively straightforward. The first production handles alternate dependencies separated by and raises an error if some text was not consumed by the dependencies: 
dependency_item: depend(s /\ /) eofile
A dependency as explained above is expressed as: 
depend: pkg_dep   variable
A variable like $ foo or $ bar -1.24~ is parsed with: 
variable: /\$ [\w:\-]+ [\w\.\-~+]*/
This rule handles a package name with optional version or arch restriction: 
pkg_dep: pkg_name dep_version(?) arch_restriction(?) 
pkg_name: /[a-z0-9][a-z0-9\+\-\.]+/
The remaining rules are quite simple: 
dep_version: '(' oper version ')' 
oper: '<<'   '<='   '='   '>='   '>>'
version: variable   /[\w\.\-~:+]+/
arch_restriction: '[' arch(s) ']'
arch:  /!?[\w-]+/
eofile: /^\Z/
The grammar above works well to parse the dependency. You can test it with this small Perl script: 
#!/usr/bin/perl
use strict;
use warnings;
use 5.010 ;
use Parse::RecDescent ;
my $parser = Parse::RecDescent->new(join('',));
my $dep = shift ;
say "parsing '$dep'";
my $ret = $parser->dependency_item($dep) ;
say "result is ", $ret if $ref ;
__DATA__
# insert grammar here !!!
Unfortunately, any error in the optional parts (i.e version requirements and arch restriction) leads to an error message which is not very helpful. The error message only mention that some text could not be parsed: 
parsing 'foo ( != 1.24 )   bar'
       ERROR (line 1): Invalid dependency item: Was expecting /\ / but found
                       "( != 1.24 )   bar" instead
or 
parsing 'foo [ arm & armel]   bar'
       ERROR (line 1): Invalid dependency item: Was expecting /\ / but found
                       "[ arm & armel]   bar" instead
The problem comes from the fact that version requirements or arch restrictions are optional. For instance if a version requirement has a syntax error, Parse::RecDescent will try to parse it as an arch restriction. This arch restriction rule will also fail and the last terminal ( eofile ) will fail. So the error message does not hint at the actual syntax problem. To generate better error messages, I improved the suggestion made in Parse::RecDescent FAQ. Instead of calling a plain subroutine, I use a sub reference that will store the error messages in a closure. This sub ref is declared in a start-up action. Note that the sub ref explicitly returns undef. I ll explain why later. 
 
    my @dep_errors ;
    my $add_error = sub  
        my ($err, $txt) = @_ ;
        push @dep_errors, "$err: '$txt'" ;
        return ;
      ;
The following production always fails while ensuring that the error list is reset. This production is always run at the beginning of the dependency parsing: 
dependency:   @dep_errors = ();
Here s the actual dependency production that is run when dependency method is called on the parser. It will return an array ref containing (1, data) if the dependency is valid or (0, errors) otherwise: 
dependency: depend(s /\ /) eofile
   
    $return = [ 1 , @ $item[1]  ] ;
   
   
   
    push( @dep_errors, "Cannot parse: '$text'" ) unless @dep_errors ;
    $return =  [ 0, @dep_errors ];
The following productions don t change much: 
depend: pkg_dep   variable
variable: /\$ [\w:\-]+ [\w\.\-~+]*/
pkg_dep: pkg_name dep_version(?) arch_restriction(?) 
dep_version: '(' oper version ')'
The first rule of this production parses the package name which must be followed by a space, end of string ( or ['. A positive look-ahead assertion is used so only the package name is consumed. If the first rule fails, the second rule provides a meaningful error message. The second rule will match anything which is not a space and create an error message. Since $add_error returns undef, the second rule returns undef and the production fails. So the text stored in the error message is not consumed: 
pkg_name: /[a-z0-9][a-z0-9\+\-\.]+(?=\s \Z \( \[)/
      /\S+/   $add_error->("bad package name", $item[1]) ;
The same trick is used with these productions: 
oper: '<<'   '='   '>>'
      /\S+/   $add_error->("bad dependency version operator", $item[1]) ; 
version: variable   /[\w\.\-~:+]+(?=\s \) \Z)/
      /\S+/   $add_error->("bad dependency version", $item[1]) ;
The action of this production is a little bit more tricky. The action ensures that '!' are either added before all arch or not at all. Otherwise an error message is generated and added to the list of errors: 
arch_restriction: '[' osarch(s) ']'
     
        my $mismatch = 0;
        # $ref contains ['!',os,arch] or ['',os,arch]
        my $ref = $item[2] ;
        for (my $i = 0; $i < $#$ref -1 ; $i++ )  
            $mismatch  = ($ref->[$i][0] xor $ref->[$i+1][0]) ;
         
        my @a = map   ($_->[0]   '') . ($_->[1]   '') . $_->[2]   @$ref ;
        if ($mismatch)  
            $add_error->("some names are prepended with '!' while others aren't.", "@a") ;
         
        else  
            $return = 1 ;
The check above is possible only if the "osarch" production returns an array ref containing something like ('!','linux','any') for "!linux-any' or ('','linux','any') for "linux-any": 
osarch: not(?) os(?) arch
     
        $return =  [ $item[1][0], $item[2][0], $item[3] ];
     
      /.?(?=\s \] \Z)/   $add_error->("bad arch specification: ", $item[1]) ;  
not: '!'
Here's the remaining of the grammar: 
os: /(any uclibc-linux linux kfreebsd knetbsd etc...)-/
     /\w+/ '-'   $add_error->("bad os in architecture specification", $item[1]) ; 
arch: / (any  alpha amd64  arm\b  arm64  etc... )
        (?=(\]  ))
      /x
        /\w+/   $add_error->("bad arch in architecture specification", $item[1]) ; 
eofile: /^\Z/
That's all for grammar 2.0... Before someone yells: "Show me the message ! ", here are some example of bad dependencies and their error message generated by the parser: 
parse 'foo ( != 1.24 )   bar'
result is: 0 bad dependency version operator: '!='
parsing 'foo [ arm & armel]   bar'
result is: 0 bad arch specification: : '&'
parsing 'foo [ arm armel ]   bar [!moo]'
result is: 0 bad arch specification: : ']' bad arch in architecture specification: 'moo'
The 2 first error messages are spot on the actual error. The second one has a false positive (']' is correct) but correctly highlights the wrong arch name ('moo'). Mission accomplished. In order to keep this post (relatively) simple, I've removed the part that actually store parsed data. They don't really matter for error handling. Nevertheless, you may see the whole grammar in Config::Model::Dpkg::Dependency module. All the best
Tagged: Config::Model, debian, dpkg, package, Parse::RecDescent, Perl

Bernhard R. Link: listing your git repositories on git.debian.org

With the new gitweb version available on alioth after the upgrade to wheezy (thanks to the alioth admins for their work on alioth), there is a new feature available I want to advertise a bit here: listing only a subtree of all repositories. Before now one could only either look at a specific repository or get the list of all repositories and the list of all repositories is quite large and slow. With the new feature you can link to all the repositories in your alioth project. For example in reprepro's case that is http://anonscm.debian.org/gitweb/?pf=mirrorer. Much more I missed what is now possible with the link http://anonscm.debian.org/gitweb/?pf=debian-science: getting a list of all debian-science repositories (still slow enough, but much better than the full list).

Daniel Pocock: The week that everything changed

Last Wednesday, I felt an urge to carefully write out a list of all the possible characteristics that would make communications technology genuinely free. I felt this was important for a number of reasons: for example, to follow up on my earlier claim that free software does not always provide free communications, it is necessary to be able to measure the shortcomings against a perfect (although possibly unachievable) benchmark. Then something happened Just hours after my blog was live, The Guardian, a leading British newspaper (and contributor to open source) started publishing explosive allegations about the extent of US Government monitoring of communications. They kicked off a dramatic four days of coverage of this topic with the story that one of America's largest phone companies, Verizon, is secretly passing details of all customer phone calls and approximate locations (metadata) to the NSA. I've published further blog entries about this subject in the meantime. One thing I want to make clear: real friends tell each other the truth. There is no need to flatter America with gratitude for inventing the Internet when discussing these fundamental privacy failings. Anybody who has tried to generalise any comments about the NSA scandal as anti-American' is themselves failing to respect America's own principles of free speech. A doctor doesn't make up fairy tales for a patient diagnosed with cancer, he puts the facts on the table as that is the first step in making progress. The winds of change Activity on all my free communications web sites, especially Lumicall and OpenTelecoms.org has doubled. Google Play reports that the rate of Lumicall installations have also doubled - hopefully it is getting closer to the point where Metcalfe's Law kicks in and everybody will have federated SIP on their phone. What next One thing is clear: this situation provides a huge opportunity for anybody promoting free software, not just for communications. As I mentioned, web sites on the subject of free and private communications are attracting significant interest and I hope this rubs off on other projects. While I have contended that free software does not always provide free communications, there is a compelling argument for the case that you can't have free communications without having genuinely free software. Some of the developments that are coming up in the very near future: An upcoming release of DruCall, initially packaged for Debian and leveraging the libraries API packaging scheme, is going to make it much, much easier for the vast majority of web sites to offer a secure calling facility, without any third-party browser plugins required. Other CMS vendors such as xWiki are also working on WebRTC support. DebConf13 is aiming to feature a half-day track on Free real-time communications with a focus on the way free operating systems, particularly Debian (and it's derivatives like Ubuntu) are fundamental to rolling out an alternative to the status quo. Federated VoIP is also a confirmed feature of the upcoming Fedora 19 release and will eventually work it's way into EPEL. This is another great way for people who work in an RPM environment to start getting more active deploying SIP as a standard service in their environment. The real-time communication (RTC) quick-start guide is currently being updated and will include a convenient web-checker to help people test their federated connectivity. Can you help? Not sure where to begin? Come and join us on the Free RTC discussion list that has been sponsored by the FSF Europe or join the discussion list of one of your favourite free RTC applications.

Petter Reinholdtsen: Debian Edu interview: Jonathan Carter

There is a certain cross-over between the Debian Edu / Skolelinux project and the Edubuntu project, and for example the LTSP packages in Debian are a joint effort between the projects. One person with a foot in both camps is Jonathan Carter, which I am now happy to present to you. Who are you, and how do you spend your days? I'm a South-African free software geek who lives in Cape Town. My days vary quite a bit since I'm involved in too many things. As I'm getting older I'm learning how to focus a bit more :) I'm also an Edubuntu contributor and I love when there are opportunities for the Edubuntu and Debian Edu projects to benefit from each other. How did you get in contact with the Skolelinux / Debian Edu project? I've been somewhat familiar with the project before, but I think my first direct exposure to the project was when I met Petter [Reinholdtsen] and Knut [Yrvin] at the Edubuntu summit in 2005 in London. They provided great feedback that helped the bootstrapping of Edubuntu. Back then Edubuntu (and even Ubuntu) was still very new and it was great getting input from people who have been around longer. I was also still very excitable and said yes to everything and to this day I have a big todo list backlog that I'm catching up with. I think over the years the relationship between Edubuntu and Debian-Edu has been gradually improving, although I think there's a lot that we could still improve on in terms of working together on packages. I'm sure we'll get there one day. What do you see as the advantages of Skolelinux / Debian Edu? Debian itself already has so many advantages. I could go on about it for pages, but in essence I love that it's a very honest project that puts its users first with no hidden agendas and also produces very high quality work. I think the advantage of Debian Edu is that it makes many common set-up tasks simpler so that administrators can get up and running with a lot less effort and frustration. At the same time I think it helps to standardise installations in schools so that it's easier for community members and commercial suppliers to support. What do you see as the disadvantages of Skolelinux / Debian Edu? I had to re-type this one a few times because I'm trying to separate "disadvantages" from "areas that need improvement" (which is what I originally rambled on about) The biggest disadvantage I can think of is lack of manpower. The project could do so much more if there were more good contributors. I think some of the problems are external too. Free software and free content in education is a no-brainer but it takes some time to catch on. When you've been working with the same proprietary eco-system for years and have gotten used to it, it can be hard to adjust to some concepts in the free software world. It would be nice if there were more Debian Edu consultants across the world. I'd love to be one myself but I'm already so over-committed that it's just not possible currently. I think the best short-term solution to that large-scale problem is for schools to be pro-active and share their experiences and grow their skills in-house. I'm often saddened to see how much money educational institutions spend on 3rd party solutions that they don't have access to after the service has ended and they could've gotten so much more value otherwise by being more self-sustainable and autonomous. Which free software do you use daily? My main laptop dual-boots between Debian and Windows 7. I was Windows free for years but started dual-booting again last year for some games which help me focus and relax (Starcraft II in particular). Gaming support on Linux is improving in leaps and bounds so I suppose I'll soon be able to regain that disk space :) Besides that I rely on Icedove, Chromium, Terminator, Byobu, irssi, git, Tomboy, KVM, VLC and LibreOffice. Recently I've been torn on which desktop environment I like and I'm taking some refuge in Xfce while I figure that out. I like tools that keep things simple. I enjoy Python and shell scripting. I went to an Arduino workshop recently and it was awesome seeing how easy and simple the IDE software was to get up and running in Debian compared to the users running Windows and OS X. I also use mc which some people frown upon slightly. I got used to using Norton Commander in the early 90's and it stuck (I think the people who sneer at it is just jealous that they don't know how to use it :p) Which strategy do you believe is the right one to use to get schools to use free software? I think trying to force it is unproductive. I also think that in many cases it's appropriate for schools to use non-free systems and I don't think that there's any particular moral or ethical problem with that. I do think though that free software can already solve so so many problems in educational institutions and it's just a shame not taking advantage of that. I also think that some curricula need serious review. For example, some areas of the world rely heavily on very specific versions of MS Office, teaching students to parrot menu items instead of learning the general concepts. I think that's very unproductive because firstly, MS Office's interface changes drastically every few years and on top of that it also locks in a generation to a product that might not be the best solution for them. To answer your question, I believe that the right strategy is to educate and inform, giving someone the information they require to make a decision that would work for them.

Next.

Previous.