Hello
Creating a configuration model for
Config::Model can be a boring task: When a configuration system has a lot of parameters, a lot of details need to be specified to have a complete configuration validation tool. So, all tricks are permitted to make this task more bearable.
Let s see how this can be applied to
lcdproc. This project requires a configuration file that supports more than 40 drivers. Each driver has it own set of parameters for a total of about 250 parameters. That s a lot to specify..
Fortunately,
LCDd.conf template file is written in a way which makes it relatively easy to parse and get all required information to build a model.
All drivers are listed in this template file, most parameters have default values and legal values are written in comments in a uniform way. Hence this file (and comments) can be parsed to retrieve information required for the model.
This blog will describe how this task can be implemented in Perl using
Config::Model modules. The code will be slightly trimmed down to keep this blog (relatively) short. All initialisation code and error checks are removed. You can get the whole script on
sourceforge.
The main steps are:
- prepare a parser for LCDd.conf template
- parse LCDd.conf template
- prepare a LCDd model skeleton
- translate LCDd.conf template information in a format suitable to create a model.
- Write the resulting LCDd model
First, here s the necessary incantation to start properly a Perl program:
use strict;
use warnings;
Then load all required libraries. Their use will be described later:
use Config::Model;
use Config::Model::Itself 1.225;
use Config::Model::Backend::IniFile;
use 5.10.0;
use IO::File;
use IO::String;
Perform some initialisation to avoid upsetting the logging system:
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init($WARN);
Let s also initialize a model object that will be very useful later:
my $model = Config::Model->new();
Now we can really start the fun part.
Step 1: Create a parser for LCDd.conf (INI file)
Problem: LCDd.conf comments must also be analysed to get more semantic information. The comments must be retrieved and associated with the INI classes and parameters found in this file.
Fortunately,
Config::Model::Backend::IniFile is able to associate comments with the relevant parameters. But Config::Model::Backend::IniFile must store its values in a configuration tree. So let s create a model suitable for LCDd.conf template that accepts any INI class and any INI parameter
Dummy::Class is used to store any parameter found in an INI class. The
accept keyword with
.* regular expression means that any INI parameter will be accepted in each INI class.
$model->create_config_class(
name => 'Dummy::Class',
accept => [
'.*' =>
type => 'leaf',
value_type => 'uniline'
,
],
);
Now the
Dummy model must provide a configuration class to hold all INI classes:
$model->create_config_class(
name => 'Dummy',
accept => [
'.*' =>
type => 'node',
config_class_name => 'Dummy::Class'
,
],
);
Note that an
INI backend could be specified directly here. But, some useful parameters are commented out in LCD.conf. Some some processing is required to be able to create a model with these commented parameters. See below for this processing.
Step 2: Parse LCDd.conf
Now the dummy configuration class is created. Let s create a configuration tree from this dummy class to store the data from LCDd.conf
my $dummy = $model->instance(
instance_name => 'dummy',
root_class_name => 'Dummy',
)-> config_root;
Now, read LCDd.conf:
my $lcd_file
= IO::File->new('examples/lcdproc/LCDd.conf');
my @lines = $lcd_file->getlines;
And perform the pre-processing on
LCDd.conf mentioned above. Just un-commenting commented parameters is required:
foreach (@lines) s/^#(\w+=)/$1/
Store the pre-processed LCDd.conf in a IO::Handle usable by INI backend
my $ioh = IO::String->new( join( '', @lines ) );
Create an INI backend to read the data from pre-processed LCDd.conf and store it in the configuration tree:
my $ini_backend = Config::Model::Backend::IniFile
->new( node => $dummy );
$ini_backend->read( io_handle => $ioh );
Step 3: Prepare LCDd configuration model
Your attention please. You may remember that we invoked
instance on object to create the dummy configuration tree to hold LCDd.conf data. The
instance method used the
Dummy configuration class.
Now, we must invoke
instance again on object. But this time,
instance will use
Config::Model::Itself. This model is Config::Model s way of eating its own dog food. The configuration tree created there will be able to hold LCDd model. Hence the root of this special configuration tree will be called meta_root to avoid confusion with the other configuration root defined above (for the Dummy model).
Still with me?
Good, let s create this meta tree that will contain LCDd model:
my $meta_root = $model->instance(
root_class_name => 'Itself::Model',
instance_name => 'meta_model',
) -> config_root;
Create the main LCDd configuration class and store the first comment from LCDd.conf as class description:
my $first_comment = $dummy->annotation ;
$meta_root
->grab("class:LCDd class_description")
->store( $first_comment );
Append my own text in this description:
my $extra_description =
"Model information extracted from template /etc/LCDd.conf"
. "\n\n=head1 BUGS\n\nThis model does not support to load "
. "several drivers. Loading several drivers is probably a "
. "marginal case. Please complain to the author if this "
. "assumption is false";
$meta_root->load(
qq!class:LCDd
class_description.="\n\n$extra_description"!
);
Add legal stuff in the model:
$meta_root->load( qq!
class:LCDd
copyright:0="2011, Dominique Dumont"
copyright:1="1999-2011, William Ferrell and others"
license="GPL-2"
!
);
Add INI backend so configuration object created from LCDd model will be able to read user s INI files:
$meta_root->load( qq!
class:LCDd
read_config:0
backend=ini_file
config_dir="/etc"
file="LCDd.conf"
!
);
Note that all the load calls above could be done as one call. They were split for better clarity.
Step 4:Extracting LCDd.conf information
Now that all information from
LCDd.conf template file is stored in
$dummy configuration tree, it s time to use this information to enhance the skeleton of LCDd model created above.
Let s first find all the INI classes defined in
LCDd.conf:
my @ini_classes = $dummy->get_element_name;
Now before actually mining LCDd.conf information, we must prepare several subs to handle them. This is done using a
dispatch table:
my %dispatch;
First create the default entry which will be used for most parameters. This subs is passed: the INI class name, the INI parameter name, the comment attached to the parameter, the INI value, and an optional value type. It will return a string describing a model for simple elements. For instance:
class:"LCDd::MD8800" element:Device type=leaf value_type=uniline default="/dev/ttyS1" description="device to use [default: /dev/ttyS1]"
Here s the subroutine invoked by default. Let s create a an anonymous subroutine and store it in the dispatch table:
$dispatch _default_ = sub
my ( $ini_class, $ini_param, $ini_note,
$ini_v, $value_type ) = @_;
$value_type = 'uniline';
Now, prepare a string that will be used later to create the ini_class model. This string uses Config::Model s
serialisation format described in
Config::Model::Loader:
my $load = qq!class:"$ini_class"
element:$ini_param type=leaf !;
Get semantic information from LCDd.conf template comments (written between square brackets):
my $info = $ini_note =~ /\[(.*)\]/ ? $1 : '';
$info =~ s/\s+//g;
Use this semantic information to better specify the parameter:
$load .=
$info =~ /legal:(\d+)-(\d+)/
? " value_type=integer min=$1 max=$2"
: $info =~ /legal:([\w\,]+)/
? " value_type=enum choice=$1"
: " value_type=$value_type";
if ( $info =~ /default:(\w+)/ )
# specify upstream default value
# if it was found in the comment
$load .= qq! upstream_default="$1"!
if length($1);
else
# or use the value found in INI
# file as default
$ini_v =~ s/^"//g;
$ini_v =~ s/"$//g;
$load .= qq! default="$ini_v"!
if length($ini_v);
The model (in serialised format) is ready, let s return it and exit the subroutine:
return $load;
;
Now let s take care of the special cases. This one deals with Driver parameter found in INI [server] class. This sub will create a string for an enum element. The possible values of the enum are extracted from the comment found in the
LCDd.conf file:
$dispatch "LCDd::server" Driver = sub
my ( $class, $elt, $info, $ini_v ) = @_;
my $load = qq!class:"$class"
element:$elt
type=leaf
value_type=enum
!;
my @drivers = split /\W+/, $info;
while ( @drivers
and ( shift @drivers ) !~ /supported/ )
# nothing, the work is done by shift operator
# in loop test to skip all words in $info until
# 'The following drivers are supported:'.
# ok, that's a hack
$load .= 'choice='
. join( ',', @drivers ) . ' ';
return $load;
;
This sub invokes the default sub and add a small feature to the model to ensure that DriverPath ends with / . This is ensured by the
match="/$" snippet:
$dispatch "LCDd::server" DriverPath = sub
my ( $class, $elt, $info, $ini_v ) = @_;
return $dispatch _default_ ->(@_)
. q! match="/$" !;
;
Likewise, this sub invokes the default sub with the optional 4th parameter that forces the value_type of the element. In this case, we enforce an integer element:
$dispatch "LCDd::server" WaitTime
= $dispatch "LCDd::server" ReportLevel
= $dispatch "LCDd::server" Port
= sub
my ( $class, $elt, $info, $ini_v ) = @_;
return $dispatch _default_ ->( @_, 'integer' );
;
This sub returns a string for a list element with default values:
$dispatch "LCDd::server" GoodBye
= $dispatch "LCDd::server" Hello
= sub
my ( $class, $elt, $info, $ini_v ) = @_;
my $ret = qq( class:"$class"
element:$elt
type=list
) ;
$ret .= 'cargo type=leaf
value_type=uniline - ' ;
$ret .= 'default_with_init:0="\" '.$elt.'\"" ' ;
$ret .= 'default_with_init:1="\" LCDproc!\""';
return $ret ;
;
Now that the dispatch table is ready, let s really mine LCDd.conf information.
First handle all INI classes in the
foreach loop:
foreach my $ini_class (@ini_classes)
Let s retrieve parameters for this INI class from the LCDd.conf template:
my $ini_obj = $dummy->grab($ini_class);
This will be the name of the lcdproc driver configuration class (e.g.
LCDd::imonlcd):
my $config_class = "LCDd::$ini_class";
The configuration class will also be created in a serialised format. Let s create config class in case there s no parameter in INI file:
$meta_root->load(qq!class:"LCDd::$ini_class"!);
Now, loop over all INI parameters and create
LCDd::$ini_class elements:
my @all_ini_params = $ini_obj->get_element_name ;
foreach my $ini_param ( @all_ini_params )
Retrieve the value provided in template LCDd.conf:
my $ini_v = $ini_obj->grab_value($ini_param);
Retrieve the comment attached to the value provided in template LCDd.conf:
my $ini_comment
= $ini_obj->grab($ini_param)->annotation;
# escape embedded quotes
$ini_comment =~ s/"/\\"/g;
Retrieve the correct sub from dispatch table:
my $sub = $dispatch $config_class $ini_param
$dispatch _default_ ;
Run the sub to get the model string:
my $model_spec
= $sub->( $config_class, $ini_param,
$ini_comment, $ini_v );
Attach the comment from LCDd.conf as class description:
$model_spec .= qq! description="$ini_comment"!
if length($ini_comment);
Then, last but not least, load class specification in the model, and end the loop aroung INI parameters:
$meta_root->load($model_spec);
Now create a an $ini_class element in LCDd class (to link LCDd class and LCDd::$ini_class):
my $driver_class_spec = qq!
class:LCDd
element:$ini_class
! ;
Since
server and
menu are not drivers, they have a special treatment:
if ( $ini_class eq 'server'
or $ini_class eq 'menu' )
$driver_class_spec .= qq!
type=node
config_class_name="LCDd::$ini_class"
! ;
Here s the treatment for LCDd drivers. Setup the model with a
warped node so that a driver configuration class is shown only if the driver was selected in the
[server] class:
else
$driver_class_spec .= qq!
type=warped_node
config_class_name="LCDd::$ini_class"
level=hidden
follow:selected="- server Driver"
rules:"\$selected eq '$ini_class'"
level=normal
!;
Now we can load the LCDd driver model in meta root:
$meta_root->load($driver_class_spec);
Step 5: save LCDd model
Still with me ?
Good. Here s the last part, now that the model was created in memory, let s save it in files.
Config::Model::Itself constructor returns an object to read or write the data:
my $rw_obj = Config::Model::Itself
->new( model_object => $meta_root );
say "Writing all models in file (please wait)";
$rw_obj->write_all(
model_dir => 'lib/Config/Model/models/'
);
That s it. Once this script is run, the directory
lib/Config/Model/models/ contains a bunch of files:
You can see the resulting editor in my
first blog on lcdproc.
Conclusion and future steps (with you?)
As mentioned at the end of this blog, the result of this model generation process is far from perfect, because the information retrieved from LCDd.conf template are not complete for this task (which is normal since these comments were targeted for users).
Now, I d really like to discuss with project developers to improve configuration template to generate better models.
For project owner, such an improvement would give them:
- command lines to validate configuration
- graphical configuration editor with integrated help
- Configuration doc generated from the information stored in the template file
Please leave a comment on this blog (or send a message to config-model-users at lists.sourceforge.net) if you are interested in configuration tool generation for a project you contribute to.
All the best
On Friday, June 24, the last day of Open Source Bridge, I won a scarf! It says, "Open Source Citizen."
I wasn't the only one. The attendees nominated people who "made an extra effort to help others and share their knowledge," and the conference committee chose the three people they felt exemplified this. The winners were Sumana Harihareswara, Igal Koshevoy, and me.
In case you don't know Sumana, she's the new Wikimedia Volunteer Development Coordinator, a friend, and a commenter here at Asheeshworld. In the photo of her and her new scarf, you can also see Ward Cunningham. I suggest reading her wrap-up about the conference, and checking out the notes from her talk.
I haven't met Igal, but I've learned he's a fixture of the Portland tech scene. He's user number one on opensourcebridge.org and one of the original contributors to calagator, the most important event calendar in the Portland tech world. That's just a brief summary; follow him on Twitter to keep in touch.
I started the conference a little shy, not wanting to introduce myself to people, so I hung out with Sumana. She talked to everyone she could find, and there I was standing next to her. On the first day, it was only because of Sumana's outgoingness that people knew who I was. On the second and third days, I was involved in two sessions: a panel on open source communities and a talk about the OpenHatch training missions.
I was extremely honored to be chosen. There were so many other spectacular people I met for the first time, like Valerie Aurora,
Jacinta Richardson, Brian Aker, and Alex Linsker. These are all people with three crucial traits: they're extremely knowledgeable, friendly, and opinionated. I'm glad I had a chance to attend and meet them!
Photo credits: Thanks to Noah Swartz for the photo of me. Sumana's photo comes from her post to wikitech-l. Igal's comes from his Open Source Bridge user profile.
So, the "Harmony Project" launched their set of contributor
agreements and tools last week. Colour me unimpressed...
There's a claim on
their 
About a month ago, I submitted a project request to
Somehow, even though I'd know about it for the longest time, I had never get around to reading George 
and I immediately got myself a copy. (Nowadays I mostly do my recreational reading with the Kindle app on my Android smartphone.)
As before, I'm spellbound. I'm already about 25% through it and loving it. What I love about Martin is that he really gets how traditional, "pagan", "feudal" societies work. Take religion for instance. A lot of Sci-Fi/Fantasy authors if they deal with the subject at all usually give a cartoon version of polytheism that owes more to Christian canards and Dungeons & Dragons than any existing human religion. But in Westeros, one can be simultaneously devout, amoral, superstitious, and cynical in a way that makes a lot of sense to me as a Hindu. Or class. Being a peasant in Martins' world is horrible. Your life is completely at the mercy of Lords and Ladies and can end in an instant. But the upper class despite its wealth and power is trapped in its own way too. They must constantly protect their status from jealous competitors and a reversal of political or military fortune can destroy an ancient noble family just as suddenly as a bad harvest can destroy a family of farmers.
Like the other books in the series, this one has a huge cast of characters whose stories weave in and out with each chapter told mainly from one characters viewpoint. This is a tactic which I for one found helpful to keep my attention from flagging but it means you can't really enjoy it unless you've read the previous volumes in order first. I definitely think it's worth the effort though.
Filing bug reports is the most common way for users to contribute. Even if it s not too difficult, I ll give you some advice to improve the quality of your reports. After all, when you go out of your way to report a bug, it is in the hope to see it fixed so let s see how we can make this more likely.
1. Try to reproduce the bug
If you can t reproduce the bug, it s next to impossible to find the root cause and thus to fix it. In that case, I would suggest you to wait until you experienced the bug multiple times. Maybe you ll be able to find something that triggers it (or that makes it more likely to encounter it). If the application has a debug/verbose mode, it might be a good idea to enable it until you experience the bug a second time. The generated log might be helpful for the developer to understand what happens exactly.
So don t file bug reports straight away unless you can reproduce it. The exception to the rule is when the application gives some useful information like a core-dump, a back-trace or an error message.
Obviously if the bug happens during an upgrade, it s difficult to reproduce it (unless you have multiple computers) but you should still report it. Be sure to include all the relevant information (versions of packages before and after the upgrade, logs of the upgrade, etc.).
2. Do your best to identify the faulty package
When you report a bug to Debian, you must assign it to a package. While there are 
5. Describe the problem so that the developer can reproduce it
Ideally your report should include everything required so that the developer can reproduce the problem on his system. If a given document triggers the bug, attach it.
Describe the steps required to reproduce the bug in great details just like you would explain it to your grand-ma. Explain how you expected the program to react and what happened instead.
You can learn much more on how to draft a good bug report in this article: 
Ten years ago, I started to use 
Here is a short snippet used to select a sensible 
Two promising new developments need to be mentioned. First, we were approached
by companies that develop hardware for brain-imaging and psychophysics
research. They were curious to learn about Debian as an integrated platform
that offers free software solutions that an increasing amount of their
customers demands (e.g. 
Do I need to explain what is the Cheese and Wine (C&W) party? Still,
if some of you need more information, please feel free to ask.
The C&W7 wiki page 
Hello this is the third report regarding my project. As usual, quick
2. Do the changes
Execute 
Since some time I ve been thinking about a possible way to delete my Gmail s Trash & Spam folders content automatically without having to bother doing it manually every single time I wanted to check my mail and clean it up. (I love keeping everything in place and having my Trash&Spam folders empty as they should be makes me pretty happy)
A few years ago when Mutt was my main mail client I had the need to filter my mail through IMAP and while googling around for that I found out a great piece of software: 
Image credit: 
Dropbox for Debian
This is not free software but Dropbox is very popular and they did only provide an Ubuntu package that did not work on Debian. So 
), unstable got all the remaining bits allowing us to convert libraries to multiarch (see the 
Misc dpkg work
BTW, I have discounted prices for individuals and non-profits who would like to benefit from my help to create Debian packages.
The Debian Administrator s Handbook
This is the title of the upcoming translation of my book. The project now has a dedicated website: 
