I've been maintaining a number of Perl software packages recently.
There's
SReview,
my video review and transcoding system of which I split off
Media::Convert a while
back; and as of about a year ago, I've also added
PtLink, an RSS aggregator
(with future plans for more than just that).
All these come with extensive test suites which can help me ensure that
things continue to work properly when I play with things; and all of
these are hosted on salsa.debian.org, Debian's gitlab instance. Since
we're there anyway, I configured GitLab CI/CD to run a full test suite
of all the software, so that I can't forget, and also so that I know
sooner rather than later when things start breaking.
GitLab has extensive support for various test-related reports, and while
it took a while to be able to enable all of them, I'm happy to report
that today, my perl test suites generate all three possible reports.
They are:
- The
coverage regex, which captures the total reported coverage for
all modules of the software; it will show the test coverage on the
right-hand side of the job page (as in this
example), and
it will show what the delta in that number is in merge request
summaries (as in this
example
- The JUnit report, which tells GitLab in detail which tests were run,
what their result was, and how long the test took (as in this
example)
- The cobertura report, which tells GitLab which lines in the software
were ran in the test suite; it will show up coverage of affected lines
in merge requests, but nothing more. Unfortunately, I can't show an
example here, as the information seems to be no longer available once
the merge request has been merged.
Additionally, I also store the native perl Devel::Cover report as job
artifacts, as they show some information that GitLab does not.
It's important to recognize that not all data is useful. For instance,
the JUnit report allows for a test name and for details of the test.
However, the module that generates the JUnit report from
TAP test suites does not make a
distinction here; both the test name and the test details are reported
as the same. Additionally, the time a test took is measured as the time
between the end of the previous test and the end of the current one;
there is no "start" marker in the TAP protocol.
That being said, it's still useful to see all the available information
in GitLab. And it's not even all that hard to do:
test:
stage: test
image: perl:latest
coverage: '/^Total.* (\d+.\d+)$/'
before_script:
- cpanm ExtUtils::Depends Devel::Cover TAP::Harness::JUnit Devel::Cover::Report::Cobertura
- cpanm --notest --installdeps .
- perl Makefile.PL
script:
- cover -delete
- HARNESS_PERL_SWITCHES='-MDevel::Cover' prove -v -l -s --harness TAP::Harness::JUnit
- cover
- cover -report cobertura
artifacts:
paths:
- cover_db
reports:
junit: junit_output.xml
coverage_report:
path: cover_db/cobertura.xml
coverage_format: cobertura
Let's expand on that a bit.
The first three lines should be clear for anyone who's used GitLab CI/CD
in the past. We create a job called
test; we start it in the
test
stage, and we run it in the
perl:latest docker image. Nothing
spectacular here.
The
coverage line contains a regular expression. This is applied by
GitLab to the output of the job; if it matches, then the first bracket
match is extracted, and whatever that contains is assumed to contain the
code coverage percentage for the code; it will be reported as such in
the GitLab UI for the job that was ran, and graphs may be drawn to show
how the coverage changes over time. Additionally, merge requests will
show the delta in the code coverage, which may help deciding whether to
accept a merge request. This regular expression will match on a line of
that the
cover program will generate on standard output.
The
before_script section installs various perl modules we'll need
later on. First, we intall
ExtUtils::Depends. My code
uses
ExtUtils::MakeMaker,
which ExtUtils::Depends depends on (no pun intended); obviously, if your
perl code doesn't use that, then you don't need to install it. The next
three modules --
Devel::Cover,
TAP::Harness::JUnit and
Devel::Cover::Report::Cobertura
are necessary for the reports, and you should include them if you want
to copy what I'm doing.
Next, we install declared dependencies, which is probably a good idea
for you as well, and then we run
perl Makefile.PL, which will generate
the Makefile. If you don't use ExtUtils::MakeMaker, update that part to
do what your build system uses. That should be fairly straightforward.
You'll notice that we don't actually
use the Makefile. This is because
we only want to run the test suite, which in our case (since these are
PurePerl modules) doesn't require us to build the software first. One
might consider that this makes the call of
perl Makefile.PL useless,
but I think it's a useful test regardless; if that fails, then obviously
we did something wrong and shouldn't even try to go further.
The actual tests are run inside a
script snippet, as is usual for
GitLab. However we do a bit more than you would normally expect; this is
required for the reports that we want to generate. Let's unpack what we
do there:
cover -delete
This deletes any coverage database that might exist (e.g., due to
caching or some such). We don't actually expect any coverage database,
but it doesn't hurt.
HARNESS_PERL_SWITCHES='-MDevel::Cover'
This tells the TAP harness that we want it to load the Devel::Cover
addon, which can generate code coverage statistics. It stores that in
the
cover_db directory, and allows you to generate all kinds of
reports on the code coverage later (but we don't do that here, yet).
prove -v -l -s
Runs the actual test suite, with
verbose output,
shuffling (aka,
randomizing) the test suite, and adding the
lib directory to perl's
include path. This works for us, again, because we don't actually need
to compile anything; if you do, then
-b (for
blib) may be required.
ExtUtils::MakeMaker creates a
test target in its Makefile, and usually
this is how you invoke the test suite. However, it's not the only way to
do so, and indeed if you want to generate a JUnit XML report then you
can't do that. Instead, in that case, you need to use the
prove, so
that you can tell it to load the TAP::Harness::JUnit module by way of
the
--harness option, which will then generate the JUnit XML report.
By default, the JUnit XML report is generated in a file
junit_output.xml. It's possible to customize the filename for this
report, but GitLab doesn't care and neither do I, so I don't. Uploading
the JUnit XML format tells GitLab which tests were run and
Finally, we invoke the
cover script twice to generate two coverage
reports; once we generate the default report (which generates HTML files
with detailed information on all the code that was triggered in your
test suite), and once with the
-report cobertura parameter, which
generates the cobertura XML format.
Once we've generated all our reports, we then need to upload them to
GitLab in the right way. The native perl report, which is in the
cover_db directory, is uploaded as a regular job artifact, which we
can then look at through a web browser, and the two XML reports are
uploaded in the correct way for their respective formats.
All in all, I find that doing this makes it easier to understand how my
code is tested, and why things go wrong when they do.
Back in June 2018, Michael Stapelberg put the Raspberry Pi image building up
for
adoption. He
created the first set of unofficial, experimental Raspberry Pi images for
Debian. I promptly answered to him, and while it took me some time to actually
warp my head around Michael s work, managed to eventually do so. By December, I
started pushing some
updates.
Not only that: I didn t think much about it in the beginning, as the needed
non-free pacakge was called 
So This is basically a call for adoption for the Raspberry Debian images
building service. I do intend to stick around and try to help. It s not only me
(although I m responsible for the build itself) we have a nice and healthy
group of Debian people hanging out in the 
Edited 2023-08-25 01:32 BST to correct a slip.
KDE Mascot
Inspired by the fine Debian Local Groups all over the world, I ve long since wanted to start one in Cape Town. Unfortunately, there s been many obstacles over the years. Shiny distractions, an epidemic, DPL terms these are just some of the things that got in the way.
Fortunately, things are starting to gain traction, and we re well on our way to forming a bona fide local group for South Africa.
We got together at Woodstock Grill, they have both a nice meeting room, and good food and beverage, also reasonably central for most of us.
Welcome to the 41th post in the 
The first time you do this, it will open up a new browser tab where
your Codespace is being instantiated. This first-time instantiation
will take a few minutes (feel free to click View logs to see how
things are progressing) so please be patient. Once built, your Codespace
will deploy almost immediately when you use it again in the future.
After the VS Code editor opens up in your browser, feel free to open
up the 
(Both example screenshots reflect the initial 
](https://jmtd.net/log/zarchscape/carpet_90s.jpg)
