Dominique Dumont: Creating a DEP-5 parser with Config::Model
Hello
Following a discussion on #debian-perl IRC channel, I ve proposed to provide a script to parse DEP-5 files. The goal is to be able to parse *and* validate DEP-5 files. DEP-5 is a
proposal to make debian/copyright machine-interpretable. This proposal is driven by Lars Wirzenius. With Config::Model, any evolution of DEP-5 specification will
be easy to include in the DEP-5 model read by Config::Model. What DEP-5 model ? To keep a long story short, let s say that DEP-5 model is a description of DEP-5 syntax and semantic that can be used by Config::Model to perform validation. For more detail on how to create a model, please read this doc. In other word, DEP-5 model is DEP-5 document translated into a special format. First step was to directly edit the doc and munge it into a YAML document describing the structure of DEP-5. Here s a small extract of this YAML file (slightly edited to remove most long descriptions) :
During this YAML file creation step, the problem raised by the License keyword became obvious because of License properties:
- the parse function to load DEP-5 data in a simple data structure
- the read function to load the simple data structure into Config::Model s configuration tree You can view the code on Config::Model repository. Then the model is divided in 3 configuration classes:
And the full model of Dep-5 can be also read on Config::Model repository:
(The models are biggish because they include help text taken from DEP-5 documentation)
Now some explanation is required on how is performed the License validation.
The trick is that each License used must be listed in Dep5 s license parameter which is specified this way (in YAML syntax):
proposal to make debian/copyright machine-interpretable. This proposal is driven by Lars Wirzenius. With Config::Model, any evolution of DEP-5 specification will
be easy to include in the DEP-5 model read by Config::Model. What DEP-5 model ? To keep a long story short, let s say that DEP-5 model is a description of DEP-5 syntax and semantic that can be used by Config::Model to perform validation. For more detail on how to create a model, please read this doc. In other word, DEP-5 model is DEP-5 document translated into a special format. First step was to directly edit the doc and munge it into a YAML document describing the structure of DEP-5. Here s a small extract of this YAML file (slightly edited to remove most long descriptions) :
---
class:
Debian::Dep5:
class_description: >
Machine-readable debian/copyright
accept:
name_match: X-.*
type: leaf
value_type: string
element:
Format-Specification:
mandatory: 1
type: leaf
value_type: uniline
description: >-
URI of the format specification, such as ...
Name:
type: leaf
value_type: uniline
Files:
type: hash
index_type: string
ordered: 1
cargo:
type: node
config_class_name: Debian::Dep5::Content
During this YAML file creation step, the problem raised by the License keyword became obvious because of License properties:
- Limited number of valid License (no problem, let s use an enum)
- License names are not case sensitive (optimism goes somewhat down)
- License names have version number and an optional + suffix (ok, let s use a regular expression with Config::Model s brand new match specification)
- License can be combined with and or or . (uh oh, the match regexp will not be enough. A grammar would be better.)
- License can specify an abbreviation or the full text of the license.
- the parse function to load DEP-5 data in a simple data structure
- the read function to load the simple data structure into Config::Model s configuration tree You can view the code on Config::Model repository. Then the model is divided in 3 configuration classes:
- Debian::Dep5: the root class
- Debian::Dep5::Content: To represent Files specification
- Debian::Dep5::License
And the full model of Dep-5 can be also read on Config::Model repository:
(The models are biggish because they include help text taken from DEP-5 documentation)
Now some explanation is required on how is performed the License validation.
The trick is that each License used must be listed in Dep5 s license parameter which is specified this way (in YAML syntax):
License:
type: hash
index_type: string
allow_keys_matching: '^(?i:Apache Artistic BSD etc... other)[\d\.\-]*\+?$'
cargo:
type: leaf
value_type: string
So this License element contains the list of licenses with their full text and only Licenses whose name matches the regular expression are accepted.
Now, let s explain how the files are tied to the declaredlicenses. The Debian::Dep5::Content class had a License element that represent the relation between the files and the License(s). This bond is represented by Debian::Dep5::License configuration class. The abbrev and full_license elements are fairly obvious.
The abbrev element is another matter. Here s its declaration (minus the description and help):
abbrev: type: leaf value_type: uniline default: otherSo far, so good. Now the meaty part: the validation requirement based on Parse::RecDescent
grammar: "license (oper license)(s?)
oper: 'and' 'or'
license: /[\\w\\-\\.\\+]+/i\n
# PRD action to check if the license text is provided
$return = $arg[0]->grab('! License')->defined($item[0]);
"
This grammar specifies:
- The syntax of the License line itself (hence something like Perl or GPL )
- An action performed when the grammar is matched. Using Config::Model API, this Perl snippet checks that the License abbreviation has a corresponding License declared in Debian::Dep5 License hash
- Implement exception parsing
- Provided DEP-5 writer
- Provide config-edit-dep5 cli
So I don't forget it, since recent sudo package resets
environment even for $HOME, the working sudoers (5) for me is:
It s finally done. For several 
After a few hours of hacking, I came up with 
[Nothing related to Debian here, move on if you don't care about my recurring obsession with the end of the world, 
Now that 
At some point in the past my bookmarks broke in such a way that I wasn't able anymore to alter in any way the bookmarks, deletions, additions, reorganisation, nothing worked.
Finally (2010.09~rc1-2) I am running Debian's U-Boot on the
Sheevaplug, and booting with this env:
I thought it's about time to write a short report on my RC bug squashing
activities again; nothing spectacular, but maybe it motivats others *hint*
It is not as prestigious as 