When I started working on the language server for
debputy,
one of several reasons was about automatic applying a formatting
style. Such that you would not have to remember to manually
reformat the file.
One of the problems with supporting automatic formatting is that
no one agrees on the "one true style". To make this concrete,
Johannes Schauer Marin Rodrigues did the numbers of which
wrap-and-sort option that are most common in
https://bugs.debian.org/895570#46. Unsurprising, we end up
with 14-15 different styles with various degrees of popularity.
To make matters worse,
wrap-and-sort does not provide a way
to declare "this package uses options
-sat".
So that begged the question, how would
debputy know which
style it should use when it was going to reformat file. After
a couple of false-starts, Christian Hofstaedtler mentioned that we
could just have a field in
debian/control for supporting
a "per-package" setting in responds to my concern about adding
a new "per-package" config file.
At first, I was not happy with it, because how would you specify
all of these options in a field (in a decent manner)? But
then I realized that one I do not want all these styles and that
I could start simpler. The Python code formatter
black is
quite successful despite not having a lot of personalized
style options. In fact,
black makes a statement out of
not allowing a lot of different styles.
Combing that, the result was
X-Style: black (to be added
to the
Source stanza of
debian/control), which every
possible reference to the
black tool for how styling
would work. Namely, you outsource the style management to
the tool (
debputy) and then start using your focus on
something else than discussing styles.
As with
black, this packaging formatting style is going
to be opinionated and it will evolve over time. At the starting
point, it is similar to
wrap-and-sort -sat for the deb822
files (
debputy does not reformat other files at the moment).
But as mentioned, it will likely evolve and possible diverge
from
wrap-and-sort over time.
The choice of the starting point was based on the numbers posted
by Johannes #895570. It was not my personal favorite but it seemed
to have a majority and is also close to the one suggested by salsa
pipeline maintainers. The delta being
-kb which I had
originally but removed in
0.1.34 at request of Otto Kek l inen
after reviewing the numbers from Johannes one more time.
To facilitate this new change, I uploaded
debputy/0.1.30
(a while back) to Debian unstable with the following changes:
- Support for the X-Style: black header.
- When a style is defined, the debputy lsp server command
will now automatically reformat deb822 files on save (if
the editor supports it) or on explicit "reformat file" request
from the editor (usually indirectly from the user).
- New subcommand debputy reformat command that will reformat
the files, when a style is defined.
- A new pre-commit hook repo to run debputy lint and
debputy reformat. These hooks are available from
https://salsa.debian.org/debian/debputy-pre-commit-hooks
version v0.1 and can be used with the pre-commit
tool (from the package of same name).
The obvious omission is a salsa-pipeline feature for this. Otto
has put that on to his personal todo list and I am looking forward
to that.
Beyond black
Another thing I dislike about our existing style tooling is that if you
run
wrap-and-sort without any arguments, you have a higher probability
of "trashing" the style of the current package than getting the desired
result. Part of this is because
wrap-and-sort's defaults are out
of sync with the usage (which is basically what
https://bugs.debian.org/895570 is about).
But I see another problem. The
wrap-and-sort tool explicitly defined
options to tweak the style but provided maintainers no way to record their
preference in any machine readable way. The net result is that we have
tons of diverging styles and that you (as a user of
wrap-and-sort)
have to manually tell
wrap-and-sort which style you want every time you
run the tool.
In my opinion that is not playing to the strengths of neither human nor
machine. Rather, it is playing to the weaknesses of the human if anything
at all.
But the salsa-CI pipeline people also ran into this issue and decided to
work around this deficiency. To use
wrap-and-sort in the salsa-CI
pipeline, you have to set a variable to activate the job and another
variable with the actual options you want.
The salsa-CI pipeline is quite machine readable and
wrap-and-sort
is widely used. I had
debputy reformat also check for the salsa-CI
variables as a fallback. This fallback also works for the editor mode
(
debputy lsp server), so you might not even have
to run
debputy reformat. :)
This was a deliberate trade-off. While I do not want all us to have all
these options, I also want Debian packaging to be less painful and have
fewer paper cuts. Having
debputy go extra lengths to meet
wrap-and-sort users where they are came out as the better solution
for me.
A nice side-effect of this trade-off is that
debputy reformat now
a good tool for drive-by contributors. You can safely run
debputy reformat on any package and either it will apply the styling
or it will back out and inform you that no obvious style was detected.
In the latter case, you would have to fallback to manually deducing
the style and applying it.
Differences to
wrap-and-sort
The
debputy reformat has some limitations or known differences
to
wrap-and-sort. Notably,
debputy reformat (nor
debputy lsp server) will not invoke
wrap-and-sort. Instead,
debputy has its own reformatting engine that provides similar
features.
One reason for not running
wrap-and-sort is that I
want
debputy reformat to match the style that
debputy lsp server will give you. That way, you get
consistent style across all
debputy commands.
Another reason is that it is important to me that reformatting is
safe and does not change semantics. This leads to two regrettable
known differences to the
wrap-and-sort behavior due to safety
in addition to one scope limitation in
debputy:
- debputy will ignore requests to sort the stanzas
when the "keep first" option is disabled (-b --no-keep-first).
This combination is unsafe reformatting. I feel it was a mistake for
wrap-and-sort to ever allow this but at least it is no longer
the default (-b is now -bk by default). This will be less
of a problem in debhelper-compat 15, since the concept of
"main package" will disappear and all multi-binary source packages
will be required to use debian/package.install rather than
debian/install.
- debputy will not reorder the contents of debhelper
packaging files such as debian/install. This is also an
(theoretical) unsafe thing to do. While the average package
will not experience issues with this, there are rare corner
cases where the re-ordering can affect the end result. I
happen to know this, because I ran into issues when trying
to optimize dh_install in a way that assumed the order
did not matter. Stuff broke and there is now special-case
code in dh_install to back out of that optimization
when that happens.
- debputy has a limited list of wrap-and-sort options
it understands. Some options may cause debputy to back
out and disable reformatting entirely with a remark that
it cannot apply that style. If you run into a case of this,
feel free to file a feature request to support it. I will
not promise to support everything, but if it is safe and
trivially doable with the engine already, then I probably
will.
As stated, where
debputy cannot implement the
wrap-and-sort styles fully, then it will currently implement
a subset that is safe if that can be identified or back out
entirely of the formatting when it cannot. In all cases,
debputy will not break the formatting if it is correct.
It may just fail at correcting one aspect of the
wrap-and-sort style if you happen to get it wrong.
It is also important to remember that the prerequisite for
debputy applying any
wrap-and-sort style is that
you have set the salsa-CI pipeline variables to trigger
wrap-and-sort with the salsa-CI pipeline. So there is
still a CI check before the merge that will run the
wrap-and-sort in its full glory that provides the final
safety net for you.
Just give me a style
In conclusion, if you, like me, are more interested in getting a consistent
style rather than discussing what that style should be, now you can get that
with
X-Style: black. You can also have your custom
wrap-and-sort
style be picked up automatically for drive-by contributors.
$ apt satisfy 'dh-debputy (>= 0.1.30), python3-lsprotocol'
# Add X-Style: black to debian/control for "just give me a style"
#
# OR, if there is a specific wrap-and-sort style for you then set
# SALSA_CI_DISABLE_WRAP_AND_SORT=no plus set relevant options in
# SALSA_CI_WRAP_AND_SORT_ARGS in debian/salsa-ci.yml (or .gitlab-ci.yml)
$ debputy reformat
It is sadly not yet in the salsa-ci pipeline. Otto is looking into that and
hopefully we will have it soon. :)
And if you find yourself often doing archive-wide contributions and is tired
of having to reverse engineer package formatting styles, consider using
debputy reformat or
debputy lsp server. If you use
debputy in
this way, please consider providing feedback on what would help you.
This is the first article of a 5-episode blog post series written by Guido Berh rster, member of staff at my company Fre(i)e Software GmbH. Thanks, Guido for being on the Polis project.
Enjoy the read on the work Guido has been doing over the past months,
The expected record surpass for the uniform distribution
The expected record surpass for the normal distribution
The expected record surpass for the gamma distribution