tl;dr:
There is a Debian git transition plan. It s going OK so far but we need help, especially with outreach and updating Debian s documentation.
Goals of the Debian git transition project
- Everyone who interacts with Debian source code should be able to do so entirely in git.
That means, more specifically:
- All examination and edits to the source should be performed via normal git operations.
- Source code should be transferred and exchanged as git data, not tarballs. git should be the canonical form everywhere.
- Upstream git histories should be re-published, traceably, as part of formal git releases published by Debian.
- No-one should have to learn about Debian Source Packages, which are bizarre, and have been obsoleted by modern version control.
This is very ambitious, but we have come a long way!
Achievements so far, and current status
We have come a very long way. But, there is still much to do - especially, the git transition team needs your help with adoption, developer outreach, and developer documentation overhaul.
We ve made big strides towards goals 1 and 4. Goal 2 is partially achieved: we currently have dual running. Goal 3 is within our reach but depends on widespread adoption of tag2upload (and/or dgit push).
Downstreams and users can
obtain the source code of any Debian package in git form. (
dgit clone, 2013). They can then work with this source code completely in git, including building binaries, merging new versions, even automatically (eg
Raspbian, 2016), and all without having to deal with source packages at all (eg
Wikimedia 2025).
A Debian maintainer can maintain their own package entirely in git. They can obtain upstream source code from git, and do their packaging work in git (
git-buildpackage, 2006).
Every Debian maintainer can (and should!) release their package
from git reliably and in a standard form (
dgit push, 2013;
tag2upload, 2025). This is not only more principled, but also more convenient, and with better UX, than pre-dgit tooling like
dput.
Indeed a Debian maintainer can now often release their changes to Debian, from git, using
only git branches (so no tarballs). Releasing to Debian can be simply pushing a signed tag (
tag2upload, 2025).
A Debian maintainer can maintain a stack of changes to upstream source code in git (
gbp pq 2009). They can even maintain such a delta series as a rebasing git branch, directly buildable, and use normal
git rebase style operations to edit their changes, (
git-dpm, 2010;
git-debrebase, 2018)
An authorised Debian developer can do a modest update to
any package in Debian, even one maintained by someone else, working entirely in git in a
standard and convenient way (dgit, 2013).
Debian contributors can share their work-in-progress on git forges and collaborate using merge requests, git based code review, and so on. (Alioth, 2003; Salsa, 2018.)
Core engineering principle
The Debian git transition project is based on one core engineering principle:
Every Debian Source Package can be losslessly converted to and from git.
In order to
transition away from Debian Source Packages, we need to
gateway between the old
dsc approach, and the new git approach.
This gateway obviously needs to be bidirectional: source packages uploaded with legacy tooling like
dput need to be imported into a canonical git representation; and of course git branches prepared by developers need to be converted to source packages for the benefit of legacy downstream systems (such as the Debian Archive and
apt source).
This bidirectional gateway is implemented in
src:dgit, and is allowing us to gradually replace dsc-based parts of the Debian system with git-based ones.
Correspondence between dsc and git
A faithful bidirectional gateway must define an invariant:
The canonical git tree, corresponding to a .dsc, is the tree resulting from
dpkg-source -x.
This canonical form is sometimes called the dgit view . It s sometimes not the same as the maintainer s git branch, because many maintainers are still working with patches-unapplied git branches. More on this below.
(For
3.0 (quilt) .dscs, the canonical git tree doesn t include the quilt
.pc directory.)
Patches-applied vs patches-unapplied
The canonical git format is patches applied . That is:
If Debian has modified the upstream source code, a normal git clone of the canonical branch gives the modified source tree, ready for reading and building.
Many Debian maintainers keep their packages in a different git branch format, where the changes made by Debian, to the upstream source code, are in actual
patch files in a
debian/patches/ subdirectory.
Patches-applied has a number of important advantages over patches-unapplied:
- It is familiar to, and doesn t trick, outsiders to Debian. Debian insiders radically underestimate how weird patches-unapplied is. Even expert software developers can get very confused or even accidentally build binaries without security patches!
- Making changes can be done with just normal git commands, eg
git commit. Many Debian insiders working with patches-unapplied are still using quilt(1), a footgun-rich contraption for working with patch files!
- When developing, one can make changes to upstream code, and to Debian packaging, together, without ceremony. There is no need to switch back and forth between patch queue and packaging branches (as with
gbp pq), no need to commit patch files, etc. One can always edit every file and commit it with git commit.
The downside is that, with the (bizarre)
3.0 (quilt) source format, the patch files files in
debian/patches/ must somehow be kept up to date. Nowadays though, tools like
git-debrebase and
git-dpm (and dgit for NMUs) make it very easy to work with patches-applied git branches.
git-debrebase can deal very ergonomically even with
big patch stacks.
(For smaller packages which usually have no patches,
plain git merge with an upstream git branch, and a
much simpler dsc format, sidesteps the problem entirely.)
Prioritising Debian s users (and other outsiders)
We want everyone to be able to share and modify the software that they interact with. That means we should make source code truly accessible, on the user s terms.
Many of Debian s processes assume everyone is an insider. It s okay that there are Debian insiders and that people feel part of something that they worked hard to become involved with. But lack of perspective can lead to software which fails to uphold our values.
Our source code practices in particular, our determination to share properly (and systematically) are a key part of what makes Debian worthwhile at all. Like Debian s installer, we want our source code to be useable by Debian outsiders.
This is why we have chosen to privilege a git branch format which is more familiar to the world at large, even if it s less popular in Debian.
Consequences, some of which are annoying
The requirement that the conversion be
bidirectional,
lossless, and
context-free can be inconvenient.
For example,
we cannot support .gitattributes which modify files during git checkin and checkout.
.gitattributes cause the meaning of a git tree to depend on the context, in possibly arbitrary ways, so the conversion from git to source package wouldn t be stable. And, worse, some source packages might not to be representable in git at all.
Another example: Maintainers often have existing git branches for their packages, generated with pre-dgit tooling which is less careful and less principled than ours. That can result in discrepancies between git and dsc, which need to be resolved before a proper git-based upload can succeed.
That some maintainers use patches-unapplied, and some patches-unapplied, means that there
has to be some kind of conversion to a standard git representation. Choosing the less-popular patches-applied format as the canonical form, means that
many packages need their git representation converted. It also means that user- and outsider-facing branches from
browse,git .dgit.d.o and
dgit clone are not always compatible with maintainer branches on Salsa. User-contributed changes need cherry-picking rather than merging, or conversion back to the maintainer format. The good news is that dgit can automate much of this, and the manual parts are usually easy git operations.
Distributing the source code as git
Our source code management should be normal, modern, and based on git. That means the Debian Archive is obsolete and needs to be replaced with a set of git repositories.
The replacement repository for source code formally released to Debian is
*.dgit.debian.org. This contains all the git objects for every git-based upload since 2013, including the signed tag for each released package version.
The plan is that it will contain a git view of
every uploaded Debian package, by centrally importing all legacy uploads into git.
Tracking the relevant git data, when changes are made in the legacy Archive
Currently, many critical source code management tasks are done by changes to the legacy Debian Archive, which works entirely with dsc files (and the associated tarballs etc). The contents of the Archive are therefore still an important source of truth. But, the Archive s architecture means it cannot sensibly directly contain git data.
To track changes made in the Archive, we added the
Dgit: field to the
.dsc of a git-based upload (2013). This declares which git commit this package was converted from. and where those git objects can be obtained.
Thus, given a Debian Source Package from a git-based upload, it is possible for the new git tooling to obtain the equivalent git objects. If the user is going to work in git, there is no need for any tarballs to be downloaded: the git data could be obtained from the depository using the git protocol.
The
signed tags, available from the git depository, have
standardised metdata which gives traceability back to the uploading Debian contributor.
Why *.dgit.debian.org is not Salsa
We need a git
depository - a formal, reliable and permanent git repository of source code actually released to Debian.
Git forges like Gitlab can be very convenient. But Gitlab is not sufficiently secure,
and too full of bugs, to be the principal and only archive of all our source code. (The open core business model of the Gitlab corporation, and the constant-churn development approach, are
critical underlying problems.)
Our git depository lacks forge features like Merge Requests. But:
- It is dependable, both in terms of reliability and security.
- It is append-only: once something is pushed, it is permanently recorded.
- Its access control is precisely that of the Debian Archive.
- Its ref namespace is standardised and corresponds to Debian releases.
- Pushes are authorised by PGP signatures, not ssh keys, so traceable.
The dgit git depository outlasted Alioth and it may well outlast Salsa.
We need
both a good forge, and the
*.dgit.debian.org formal git depository.
Roadmap
In progress
Right now we are quite focused on
tag2upload.
We are working hard on eliminating the remaining issues that we feel need to be addressed before declaring the service out of beta.
Future Technology
Whole-archive dsc importer
Currently, the git depository only has git data for git-based package updates (tag2upload and dgit push). Legacy dput-based uploads are not currently present there. This means that the git-based and legacy uploads must be resolved client-side, by
dgit clone.
We will want to start importing legacy uploads to git.
Then downstreams and users will be able to get the source code for any package simply with
git clone, even if the maintainer is using legacy upload tools like dput.
Support for git-based uploads to security.debian.org
Security patching is a task which would particularly benefit from better and more formal use of git. git-based approaches to applying and backporting security patches are much more convenient than messing about with actual patch files.
Currently, one can use git to help prepare a security upload, but it often involves starting with a dsc import (which lacks the proper git history) or figuring out a package maintainer s unstandardised git usage conventions on Salsa.
And it is not possible to properly perform the security release
as git.
Internal Debian consumers switch to getting source from git
Buildds, QA work such as lintian checks, and so on, could be simpler if they don t need to deal with source packages.
And since git is actually the canonical form, we want them to use it directly.
Problems for the distant future
For decades, Debian has been built around source packages. Replacing them is a long and complex process. Certainly source packages are going to continue to be supported for the foreseeable future.
There are no doubt going to be unanticipated problems. There are also foreseeable issues: for example, perhaps there are packages that work very badly when represented in git. We think we can rise to these challenges as they come up.
Mindshare and adoption - please help!
We and our users are very pleased with our technology. It is convenient and highly dependable.
dgit in particular is superb, even if we say so ourselves. As technologists, we have been very focused on building good software, but it seems we have fallen short in the marketing department.
A rant about publishing the source code
git is the preferred form for modification.
Our upstreams are overwhelmingly using git. We are overwhelmingly using git. It is a scandal that for many packages, Debian does not properly, formally and officially publish the git history.
Properly publishing the source code as git means publishing it in a way that means that anyone can
automatically and
reliably obtain
and build the
exact source code corresponding to the binaries. The test is: could you use that to build a derivative?
Putting a package
in git on Salsa is often a good idea, but it is not sufficient. No standard branch structure git on Salsa is enforced, nor should it be (so it can t be automatically and reliably obtained), the tree is not in a standard form (so it can t be automatically built), and is not
necessarily identical to the source package. So
Vcs-Git fields, and git from Salsa, will never be sufficient to make a derivative.
Debian is not publishing the source code!
The time has come for proper publication of source code by Debian to no longer be a minority sport. Every maintainer of a package whose upstream is using git (which is nearly all packages nowadays) should be basing their work on upstream git, and properly publishing that via tag2upload or dgit.
And it s not even difficult! The modern git-based tooling provides a far superior upload experience.
A common misunderstanding
dgit push is not an alternative to gbp pq or quilt. Nor is tag2upload. These upload tools
complement your existing git workflow. They replace and improve source package building/signing and the subsequent dput. If you are using one of the usual git layouts on salsa, and your package is in good shape, you can adopt tag2upload and/or dgit push right away.
git-debrebase is distinct and
does provides an alternative way to manage your git packaging, do your upstream rebases, etc.
Documentation
Debian s documentation all needs to be updated, including particularly instructions for packaging, to recommend use of git-first workflows. Debian should not be importing git-using upstreams release tarballs into git. (Debian outsiders who discover this practice are typically horrified.) We should use
only upstream git, work only in git, and properly release (and publish) in git form.
We, the git transition team, are experts in the technology, and can provide good suggestions. But we do not have the bandwidth to also engage in the massive campaigns of education and documentation updates that are necessary especially given that (as with any programme for change) many people will be sceptical or even hostile.
So we would greatly appreciate help with writing and outreach.
Personnel
We consider ourselves the Debian git transition team.
Currently we are:
- Ian Jackson. Author and maintainer of dgit and git-debrebase. Co-creator of tag2upload. Original author of dpkg-source, and inventor in 1996 of Debian Source Packages. Alumnus of the Debian Technical Committee.
- Sean Whitton. Co-creator of the tag2upload system; author and maintainer of git-debpush. Co-maintainer of dgit. Debian Policy co-Editor. Former Chair of the Debian Technical Committee.
We wear the following hats related to the git transition:
You can contact us:
We do most of our heavy-duty development
on Salsa.
Thanks
Particular thanks are due to Joey Hess, who, in the now-famous design session in Vaumarcus in 2013, helped invent dgit. Since then we have had a lot of support: most recently political support to help get tag2upload deployed, but also, over the years, helpful bug reports and kind words from our users, as well as translations and code contributions.
Many other people have contributed more generally to support for working with Debian source code in git. We particularly want to mention Guido G nther (git-buildpackage); and of course Alexander Wirt, Joerg Jaspert, Thomas Goirand and Antonio Terceiro (Salsa administrators); and before them the Alioth administrators.
comments
The Debian LTS Team, funded by Freexian s Debian LTS offering,
is pleased to report its activities for August.
Just over 7 months ago, I
I have been a Puppet user for a couple of years now, first at work, and
eventually for my personal servers and computers. Although it can have a steep
learning curve, I find Puppet both nimble and very powerful. I also prefer it
to Ansible for its speed and the agent-server model it uses.
Sadly, Puppet Labs hasn't been the most supportive upstream and tends to move
pretty fast. Major versions rarely last for a whole Debian Stable release and
the upstream