I do not think seasoned Debian contributors quite appreciate how much knowledge we
have picked up and internalized. As an example, when I need to look up
documentation for debhelper, I generally know which manpage to look in. I suspect
most long time contributors would be able to a similar thing (maybe down 2-3
manpages). But new contributors does not have the luxury of years of experience.
This problem is by no means unique to
debhelper.
One thing that
debhelper does very well, is that it is hard for users to tell
where a addon "starts" and
debhelper "ends". It is clear you use addons, but
the transition in and out of third party provided tools is generally smooth. This
is a sign that things "just work(tm)".
Except when it comes to documentation. Here,
debhelper's static documentation
does not include documentation for third party tooling. If you think from a
debhelper maintainer's perspective, this seems obvious. Embedding documentation
for all the third-party code would be very hard work, a layer-violation, etc.. But
from a user perspective, we should not have to care "who" provides "what". As as
user, I want to understand how this works and the more hoops I have to jump through
to get that understanding, the more frustrated I will be with the toolstack.
With this, I came to the conclusion that the best way to help users and solve the
problem of finding the documentation was to provide "online documentation". It
should be possible to ask
debputy, "What attributes can I use in
install-man?" or "What does
path-metadata do?". Additionally, the lookup
should work the same no matter if
debputy provided the feature or some
third-party plugin did. In the future, perhaps also other types of documentation
such as tutorials or how-to guides.
Below, I have some tentative results of my work so far. There are some
improvements to be done. Notably, the commands for these documentation features
are still treated a "plugin" subcommand features and should probably have its own
top level "ask-me-anything" subcommand in the future.
Automatic discard rules
Since the introduction of install rules,
debputy has included an automatic
filter mechanism that prunes out unwanted content. In 0.1.9, these filters
have been named "Automatic discard rules" and you can now ask
debputy to
list them.
$ debputy plugin list automatic-discard-rules
+-----------------------+-------------+
Name Provided By
+-----------------------+-------------+
python-cache-files debputy
la-files debputy
backup-files debputy
version-control-paths debputy
gnu-info-dir-file debputy
debian-dir debputy
doxygen-cruft-files debputy
+-----------------------+-------------+
For these rules, the provider can both provide a description but also an example
of their usage.
$ debputy plugin show automatic-discard-rules la-files
Automatic Discard Rule: la-files
================================
Documentation: Discards any .la files beneath /usr/lib
Example
-------
/usr/lib/libfoo.la << Discarded (directly by the rule)
/usr/lib/libfoo.so.1.0.0
The example is a
live example. That is, the provider will provide
debputy
with a scenario and the expected outcome of that scenario. Here is the concrete
code in
debputy that registers this example:
api.automatic_discard_rule(
"la-files",
_debputy_prune_la_files,
rule_reference_documentation="Discards any .la files beneath /usr/lib",
examples=automatic_discard_rule_example(
"usr/lib/libfoo.la",
("usr/lib/libfoo.so.1.0.0", False),
),
)
When showing the example,
debputy will
validate the example matches what the plugin
provider intended. Lets say I was to introduce a bug in the code, so that the discard rule
no longer worked. Then
debputy would start to show the following:
# Output if the code or example is broken
$ debputy plugin show automatic-discard-rules la-files
[...]
Automatic Discard Rule: la-files
================================
Documentation: Discards any .la files beneath /usr/lib
Example
-------
/usr/lib/libfoo.la !! INCONSISTENT (code: keep, example: discard)
/usr/lib/libfoo.so.1.0.0
debputy: warning: The example was inconsistent. Please file a bug against the plugin debputy
Obviously, it would be better if this validation could be added directly as a plugin test,
so the CI pipeline would catch it. That is one my personal TODO list. :)
One final remark about automatic discard rules before moving on. In 0.1.9,
debputy will also
list any path automatically discarded by one of these rules in the build output to make sure
that the automatic discard rule feature is more discoverable.
Plugable manifest rules like the
install rule
In the manifest, there are several places where rules can be provided by plugins. To make
life easier for users,
debputy can now since 0.1.8 list all provided rules:
$ debputy plugin list plugable-manifest-rules
+-------------------------------+------------------------------+-------------+
Rule Name Rule Type Provided By
+-------------------------------+------------------------------+-------------+
install InstallRule debputy
install-docs InstallRule debputy
install-examples InstallRule debputy
install-doc InstallRule debputy
install-example InstallRule debputy
install-man InstallRule debputy
discard InstallRule debputy
move TransformationRule debputy
remove TransformationRule debputy
[...] [...] [...]
remove DpkgMaintscriptHelperCommand debputy
rename DpkgMaintscriptHelperCommand debputy
cross-compiling ManifestCondition debputy
can-execute-compiled-binaries ManifestCondition debputy
run-build-time-tests ManifestCondition debputy
[...] [...] [...]
+-------------------------------+------------------------------+-------------+
(Output trimmed a bit for space reasons)
And you can then ask
debputy to describe any of these rules:
$ debputy plugin show plugable-manifest-rules install
Generic install ( install )
===========================
The generic install rule can be used to install arbitrary paths into packages
and is *similar* to how dh_install from debhelper works. It is a two "primary" uses.
1) The classic "install into directory" similar to the standard dh_install
2) The "install as" similar to dh-exec 's foo => bar feature.
Attributes:
- source (conditional): string
sources (conditional): List of string
A path match ( source ) or a list of path matches ( sources ) defining the
source path(s) to be installed. [...]
- dest-dir (optional): string
A path defining the destination *directory*. [...]
- into (optional): string or a list of string
A path defining the destination *directory*. [...]
- as (optional): string
A path defining the path to install the source as. [...]
- when (optional): manifest condition (string or mapping of string)
A condition as defined in [Conditional rules](https://salsa.debian.org/debian/debputy/-/blob/main/MANIFEST-FORMAT.md#Conditional rules).
This rule enforces the following restrictions:
- The rule must use exactly one of: source , sources
- The attribute as cannot be used with any of: dest-dir , sources
[...]
(Output trimmed a bit for space reasons)
All the attributes and restrictions are auto-computed by
debputy from information
provided by the plugin. The associated documentation for each attribute is supplied
by the plugin itself, The
debputy API validates that all attributes are covered
and the documentation does not describe non-existing fields. This ensures that you as
a plugin provider never forget to document new attributes when you add them later.
The
debputy API for manifest rules are not quite stable yet. So currently only
debputy provides rules here. However, it is my intention to lift that restriction
in the future.
I got the idea of supporting online validated examples when I was building this feature.
However, sadly, I have not gotten around to supporting it yet.
Manifest variables like
PACKAGE
I also added a similar documentation feature for manifest variables such as
PACKAGE . When I implemented this, I realized listing all manifest variables
by default would probably be counter productive to new users. As an example, if you
list all variables by default it would include
DEB_HOST_MULTIARCH (the most common
case) side-by-side with the the much less used
DEB_BUILD_MULTIARCH and the even
lessor used
DEB_TARGET_MULTIARCH variable. Having them side-by-side implies they
are of equal importance, which they are not. As an example, the ballpark number of
unique packages for which
DEB_TARGET_MULTIARCH is useful can be counted on two
hands (and maybe two feet if you consider
gcc-X distinct from
gcc-Y).
This is one of the cases, where experience makes us blind. Many of us probably have the
"show me everything and I will find what I need" mentality. But that requires experience
to be able to pull that off - especially if all alternatives are presented as equals.
The cross-building terminology has proven to notoriously match poorly to people's
expectation.
Therefore, I took a deliberate choice to reduce the list of shown variables by default
and in the output explicitly list what filters were active. In the current version
of
debputy (0.1.9), the listing of manifest-variables look something like this:
$ debputy plugin list manifest-variables
+----------------------------------+----------------------------------------+------+-------------+
Variable (use via: NAME ) Value Flag Provided by
+----------------------------------+----------------------------------------+------+-------------+
DEB_HOST_ARCH amd64 debputy
[... other DEB_HOST_* vars ...] [...] debputy
DEB_HOST_MULTIARCH x86_64-linux-gnu debputy
DEB_SOURCE debputy debputy
DEB_VERSION 0.1.8 debputy
DEB_VERSION_EPOCH_UPSTREAM 0.1.8 debputy
DEB_VERSION_UPSTREAM 0.1.8 debputy
DEB_VERSION_UPSTREAM_REVISION 0.1.8 debputy
PACKAGE <package-name> debputy
path:BASH_COMPLETION_DIR /usr/share/bash-completion/completions debputy
+----------------------------------+----------------------------------------+------+-------------+
+-----------------------+--------+-------------------------------------------------------+
Variable type Value Option
+-----------------------+--------+-------------------------------------------------------+
Token variables hidden --show-token-variables OR --show-all-variables
Special use variables hidden --show-special-case-variables OR --show-all-variables
+-----------------------+--------+-------------------------------------------------------+
I will probably tweak the concrete listing in the future. Personally, I am considering to provide
short-hands variables for some of the
DEB_HOST_* variables and then hide the
DEB_HOST_*
group from the default view as well. Maybe something like
ARCH and
MULTIARCH, which
would default to their
DEB_HOST_* counter part. This variable could then have extended
documentation that high lights
DEB_HOST_<X> as its source and imply that there are special
cases for cross-building where you might need
DEB_BUILD_<X> or
DEB_TARGET_<X>.
Speaking of variable documentation, you can also lookup the documentation for a given
manifest variable:
$ debputy plugin show manifest-variables path:BASH_COMPLETION_DIR
Variable: path:BASH_COMPLETION_DIR
==================================
Documentation: Directory to install bash completions into
Resolved: /usr/share/bash-completion/completions
Plugin: debputy
This was my update on online reference documentation for
debputy. I hope you found it
useful. :)
Thanks
On a closing note, I would like to thanks Jochen Sprickerhof, Andres Salomon, Paul Gevers
for their recent contributions to debputy. Jochen and Paul provided a number of real
world cases where debputy would crash or not work, which have now been fixed. Andres
and Paul also provided corrections to the documentation.
For documentation, please have a look at:
One of the annoyances I had when I upgraded from Buster to Bullseye (yes, I m talking about an upgrade I did at the end of 2021) is that I ended up moving from Mailman 2 to Mailman 3. Which is fine, I guess, but it meant I could no longer use 
Uhm, salsa is not resolving:
