The other day, I noted that the
emacs integration with
debputy stopped working.
After debugging for a while, I realized that
emacs no longer sent the
didOpen
notification that is expected of it, which confused
debputy. At this point, I was
already several hours into the debugging and I noted there was some discussions on
debian-devel about
emacs and byte compilation not working. So I figured I would
shelve the
emacs problem for now.
But I needed an LSP capable editor and with my
vi skills leaving much to be desired,
I skipped out on
vim-youcompleteme. Instead, I pulled out
kate, which I had not
been using for years. It had LSP support, so it would fine, right?
Well, no. Turns out that
debputy LSP support had some assumptions that worked for
emacs but not
kate. Plus once you start down the rabbit hole, you stumble on
things you missed previously.
Getting started
First order of business was to tell
kate about
debputy. Conveniently,
kate has
a configuration tab for adding language servers in a JSON format right next to the tab where
you can see its configuration for built-in LSP (also in JSON format9. So a quick bit of
copy-paste magic and that was done.
Yesterday, I opened an MR against upstream to have the configuration added
(
https://invent.kde.org/utilities/kate/-/merge_requests/1748) and they already merged it.
Today, I then filed a wishlist against
kate in Debian to have the Debian maintainers
cherry-pick it, so it works out of the box for Trixie (
https://bugs.debian.org/1099876).
So far so good.
Inlay hint woes
Since July (2024),
debputy has support for
Inlay hints. They are basically small
bits of text that the LSP server can ask the editor to inject into the text to provide
hints to the reader.
Typically, you see them used to provide typing hints, where the editor or the underlying
LSP server has figured out the type of a variable or expression that you did not
explicitly type. Another common use case is to inject the parameter name for positional
arguments when calling a function, so the user do not have to count the position to
figure out which value is passed as which parameter.
In
debputy, I have been using the
Inlay hints to show inherited fields in
debian/control. As an example, if you have a definition like:
Source: foo-src
Section: devel
Priority: optional
Package: foo-bin
Architecture: any
Then
foo-bin inherits the
Section and
Priority field since it does not supply
its own. Previously,
debputy would that by injecting the fields themselves and their
value just below the
Package field as if you had typed them out directly. The editor
always renders
Inlay hints distinctly from regular text, so there was no risk of
confusion and it made the text look like a valid
debian/control file end to end. The
result looked something like:
Source: foo-src
Section: devel
Priority: optional
Package: foo-bin
Section: devel
Priority: optional
Architecture: any
With the second instances of
Section and
Priority being rendered differently than
its surrendering (usually faded or colorlessly).
Unfortunately,
kate did not like injecting
Inlay hints with a newline in them,
which was needed for this trick. Reading into the LSP specs, it says nothing about
multi-line
Inlay hints being a thing and I figured I would see this problem again
with other editors if I left it be.
I ended up changing the
Inlay hints to be placed at the end of the
Package field
and then included surrounding
() for better visuals. So now, it looks like:
Source: foo-src
Section: devel
Priority: optional
Package: foo-bin (Section: devel) (Priority: optional)
Architecture: any
Unfortunately, it is no longer 1:1 with the underlying syntax which I liked about the
previous one. But it works in more editors and is still explicit. I also removed the
Inlay hint for the
Homepage field. It takes too much space and I have yet to
meet someone missing it in the binary stanza.
If you have any better ideas for how to render it, feel free to reach out to me.
Spurious completion and hover
As I was debugging the
Inlay hints, I wanted to do a quick restart of
debputy after
each fix. Then I would trigger a small change to the document to ensure
kate would
request an update from
debputy to render the
Inlay hints with the new code.
The full outgoing payloads are sent via the logs to the client, so it was really about
minimizing which LSP requests are sent to
debputy. Notably, two cases would flood the
log:
- Completion requests. These are triggered by typing anything at all and since I wanted
to a change, I could not avoid this. So here it was about making sure there would be
nothing to complete, so the result was a small as possible.
- Hover doc requests. These are triggered by mouse hovering over field, so this was
mostly about ensuring my mouse movement did not linger over any field on the way
between restarting the LSP server and scrolling the log in kate.
In my infinite wisdom, I chose to make a comment line where I would do the change. I figured
it would neuter the completion requests completely and it should not matter if my cursor
landed on the comment as there would be no hover docs for comments either.
Unfortunately for me,
debputy would ignore the fact that it was on a comment line.
Instead, it would find the next field after the comment line and try to complete based on
that. Normally you do not see this, because the editor correctly identifies that none of
the completion suggestions start with a
\#, so they are all discarded.
But it was pretty annoying for the debugging, so now
debputy has been told to explicitly
stop these requests early on comment lines.
Hover docs for packages
I added a feature in debputy where you can hover over package names in your relationship
fields (such as Depends) and debputy will render a small snippet about it based on
data from your local APT cache.
This doc is then handed to the editor and tagged as markdown provided the editor supports
markdown rendering. Both emacs and kate support markdown. However, not all
markdown renderings are equal. Notably, emacs's rendering does not reformat the text
into paragraphs. In a sense, emacs rendering works a bit like <pre>...</pre> except
it does a bit of fancy rendering inside the <pre>...</pre>.
On the other hand, kate seems to convert the markdown to HTML and then throw the result
into an HTML render engine. Here it is important to remember that not all newlines are equal
in markdown. A Foo<newline>Bar is treated as one "paragraph" (<p>...</p>) and the HTML
render happily renders this as single line Foo Bar provided there is sufficient width to
do so.
A couple of extra newlines made wonders for the kate rendering, but I have a feeling this
is not going to be the last time the hover docs will need some tweaking for prettification.
Feel free to reach out if you spot a weirdly rendered hover doc somewhere.
Making quickfixes available in
kate
Quickfixes are treated as generic code actions in the LSP specs. Each code action has a "type"
(
kind in the LSP lingo), which enables the editor to group the actions accordingly or
filter by certain types of code actions.
The design in the specs leads to the following flow:
- The LSP server provides the editor with diagnostics (there are multiple ways to trigger
this, so we will keep this part simple).
- The editor renders them to the user and the user chooses to interact with one of them.
- The interaction makes the editor asks the LSP server, which code actions are available
at that location (optionally with filter to only see quickfixes).
- The LSP server looks at the provided range and is expected to return the relevant
quickfixes here.
This flow is really annoying from a LSP server writer point of view. When you do the diagnostics
(in step 1), you tend to already know what the possible quickfixes would be. The LSP spec
authors realized this at some point, so there are two features the editor provides to simplify
this.
- In the editor request for code actions, the editor is expected to provide the diagnostics
that they received from the server. Side note: I cannot quite tell if this is optional or
required from the spec.
- The editor can provide support for remembering a data member in each diagnostic. The
server can then store arbitrary information in that member, which they will see again in
the code actions request. Again, provided that the editor supports this optional feature.
All the quickfix logic in
debputy so far has hinged on both of these two features.
As life would have it,
kate provides neither of them.
Which meant I had to teach
debputy to keep track of its diagnostics on its own. The plus side
is that makes it easier to support "pull diagnostics" down the line, since it requires a similar
feature. Additionally, it also means that quickfixes are now available in more editors. For
consistency,
debputy logic is now always used rather than relying on the editor support
when present.
The downside is that I had to spend hours coming up with and debugging a way to find the
diagnostics that overlap with the range provided by the editor. The most difficult part was keeping
the logic straight and getting the runes correct for it.
Making the quickfixes actually work
With all of that, kate would show the quickfixes for diagnostics from debputy and you could
use them too. However, they would always apply twice with suboptimal outcome as a result.
The LSP spec has multiple ways of defining what need to be changed in response to activating a
code action. In debputy, all edits are currently done via the WorkspaceEdit type. It
has two ways of defining the changes. Either via changes or documentChanges with
documentChanges being the preferred one if both parties support this.
I originally read that as I was allowed to provide both and the editor would pick the one it
preferred. However, after seeing kate blindly use both when they are present, I reviewed
the spec and it does say "The edit should either provide changes or documentChanges",
so I think that one is on me.
None of the changes in debputy currently require documentChanges, so I went with just
using changes for now despite it not being preferred. I cannot figure
out the logic of whether an editor supports documentChanges. As I read the notes for this
part of the spec, my understanding is that kate does not announce its support for
documentChanges but it clearly uses them when present. Therefore, I decided to keep it
simple for now until I have time to dig deeper.
Remaining limitations with kate
There is one remaining limitation with kate that I have not yet solved. The kate
program uses KSyntaxHighlighting for its language detection, which in turn is the
basis for which LSP server is assigned to a given document.
This engine does not seem to support as complex detection logic as I hoped from it. Concretely,
it either works on matching on an extension / a basename (same field for both cases) or
mime type. This combined with our habit in Debian to use extension less files like
debian/control vs. debian/tests/control or debian/rules or
debian/upstream/metadata makes things awkward a best.
Concretely, the syntax engine cannot tell debian/control from debian/tests/control as
they use the same basename. Fortunately, the syntax is close enough to work for both and
debputy is set to use filename based lookups, so this case works well enough.
However, for debian/rules and debian/upstream/metadata, my understanding is that if
I assign these in the syntax engine as Debian files, these rules will also trigger for any
file named foo.rules or bar.metadata. That seems a bit too broad for me, so I have
opted out of that for now. The down side is that these files will not work out of the box
with kate for now.
The current LSP configuration in kate does not recognize makefiles or YAML either. Ideally,
we would assign custom languages for the affected Debian files, so we do not steal the ID
from other language servers. Notably, kate has a built-in language server for YAML and
debputy does nothing for a generic YAML document. However, adding YAML as a supported
language for debputy would cause conflict and regressions for users that are already
happy with their generic YAML language server from kate.
So there are certainly still work to be done. If you are good with KSyntaxHighlighting
and know how to solve some of this, I hope you will help me out.
Closing
I am glad I tested with kate to weed out most of these issues in time before
the freeze. The Debian freeze will start within a week from now. Since debputy
is a part of the toolchain packages it will be frozen from there except for
important bug fixes.
A second maintenance release of our still young-ish package tinythemes arrived
on CRAN today. tinythemes
provides the 
The twentysecond release of 
Are you a student aspiring to participate in the 
I am told, by friends who have spent time at Google, about the reason 
A few months ago 
Almost exactly four years after I started with this project, yesterday I
presented my PhD defense.
