Matthias Klumpp: PackageKit, AppStream and Listaller A status report
I was asked by some people to write a status report about the whole PK/AS/LI stuff sorry guys that it took so much time to write it
.
PackageKit
(PackageKit is an abstraction layer for package-management systems, allowing applications to access the package-manager using simple DBus calls)
PackageKit is an incredibly successful project. With the 0.8.x series, it received many performance improvements, and has now the same speed on my computer than the distribution s native tools. PackageKit is used in almost all major Linux distributions, except for Ubuntu[1]. But even Ubuntu has written some compatibility layer, so most calls to PackageKit will work.
The only major distro where PackageKit is currently not available, seems to be Gentoo (and I am not sure about the shape of the Gentoo PackageKit backend too).
Debian Wheezy includes PackageKit by default, and in Jessy we are going to replace some distribution-specific tools with PackageKit frontends (mostly the old and unmaintained update-notifier and Software-Updater no worries, we are not going for a Synaptic replacement
(currently this won t be possible with PK anyway))
Unfortunately, some PackageKit backends are still not adjusted for the 0.8.x API and are only running on 0.7.x. This is bad, since 0.8.x is a huge step forward for PackageKit. But the situation is slowly improving, with the latest OpenSUSE release, the Zypper backend is now available on 0.8.x too.
Being able to run a PackageKit from the 0.8.x series is a requirement for both AppStream and Listaller.
AppStream
(AppStream is a cross-distro effort for building Software-Center-like
applications. It contains stuff like a screenshot-service, ratings&reviews etc. The most important component is a Xapian database, storing information about all available applications in the distribution s repositories. The Xapian DB is distro-agnostik, but distributors need to provide data to fill it. AppStream offers an application-centric way to look at a package database)
The AppStream side doesn t look incredibly great, but the situation is improving. As far as I know, OpenSUSE is shipping AppStream XML to generate the database. Ubuntu ships the desktop-files, and I am working on AppStream support in Debian s Archive Kit. On the Fedora side, negotiations with the infrastructure-team are still going on. I haven t heard anything from Mageia and other AppStream participants yet.
Unfortunately, at least for OpenSUSE, the AppStream efforts seem to be stalled, due to people having moved to different tasks. But efforts to add the remaining missing things exist.
On the software side, Apper (KDE PackageKit frontend) has full support for AppStream. Apper just needs to be compiled with some extra flags to make it use the AppStream database.
On the GNOME-side, GNOME-Software is being developed. The tool will make use of the AppStream database, on distributions where it is available.
Also, a Software-Center for Elementary and other GTK+-based desktops is being developed, which is based on AppStream (already quite usable!).
Using the Ubuntu Software Center on not-Ubuntu-based distributions ist still not much fun, but with the AppStream database available and a working PackageKit 0.8.x with a backend which supports parallel transactions, it is possible to use it.
On the infrastructure side: I recently landed some patches in AppStream-Core, which will improve the search function a lot. AppStream-Core contains all tools necessary to generate the AppStream database. It also contains libappstream, a GObject-based library which can be used to access the AppStream database.
Also, we discuss dropping PackageKit s internal desktop-file-cache in favour of using the AppStream database. If we do that, we will also add software descriptions to the AppStream db, to improve search results and to speed up search for applications. Because we have to deprecate API for that, I expect this change to happen with PackageKit 0.9.x.
As soon as the Freedesktop-Wiki is alive again and my account is re-enabled, I will create compatibility-list, showing which distribution implements what of the PK/AS/LI stuff, especially focusing on components needed for AppStream.
Only a few distributions package AppStream-Core so far. Although it is beta-software, creating packages for it and shipping the required data to generate the AppStream database would be a very huge step forward.
Listaller
(Listaller is a cross-distro 3rd-party software installer, which integrates into
PackageKit and AppStream. It allows installing 3rd-party applications, which are not part of the distributor s repositories, using standard tools used also for native-package handling. Everything which uses PackageKit can make use of Listaller packages too. Listaller also allows sandboxing of new applications, and uses an AppDir-like approach for installing software.)
Listaller is currently undergoing it s last transition before a release with stable API and specifications can be made. Dependency solving will be improved a lot during the current release-cycle, making it less powerful, but working on all distributions instead. (Fedora always had an advantage in dependency-solving, due to RPM providing more package metadata for Listaller to use) This change was delayed due to discussing a possible use of ZeroInstall-feeds to provide missing dependencies with the ZeroInstall team. We did not come to a conclusion about extending the XML, so Listaller will contain an own format to define a dependency, which can reference a ZeroInstall feed. That should be a good solution for everyone.
All these changes will result in IPK1.2, a new version of the IPK spec with small changes in the pkoptions file syntax and huge changes in dependency-handling. The new code is slowly stabilizing in a separate branch, and will soon be merged into master.
The next Listaller release will be the last one of the 0.5.x series, we will start 0.6.x then. KDE currently has support for Listaller through Apper, which is enabled on a few distributions. In GNOME, optional Listaller support is being developed and will be available in one of the upcoming releases.
Currently, to my knowledge, only a few distributions package Listaller. This should improve, so it is easier for application developers to deploy IPK packages.
The upcoming changes in KDE and GNOME to build stable developer platforms will help Listaller a lot in finding matching dependencies, and for stuff which only depends on one software frameworks, installations should be a matter of seconds.
As you can see, lots of things are happening, and there is improvement in all components related to installing and presenting software on Linux machines. However, all these projects have a severe lack of manpower, especially AppStream and Listaller have the lowest number of developers working on the tools (at time, only two active developers). This is the main reason for the slow development. But I am confident that we will have something shipped in the next distribution releases. At least AppStream should be ready then.
[1]: I don t blame Ubuntu for that during the time they wrote an own solution, PackageKit did not have all the required features. (This situation has changed now, fortunately.)
NOTE: I might extend this post with feedback from the different distributions, as soon as I get it.
PackageKit is an incredibly successful project. With the 0.8.x series, it received many performance improvements, and has now the same speed on my computer than the distribution s native tools. PackageKit is used in almost all major Linux distributions, except for Ubuntu[1]. But even Ubuntu has written some compatibility layer, so most calls to PackageKit will work.
The only major distro where PackageKit is currently not available, seems to be Gentoo (and I am not sure about the shape of the Gentoo PackageKit backend too).
Debian Wheezy includes PackageKit by default, and in Jessy we are going to replace some distribution-specific tools with PackageKit frontends (mostly the old and unmaintained update-notifier and Software-Updater no worries, we are not going for a Synaptic replacement
applications. It contains stuff like a screenshot-service, ratings&reviews etc. The most important component is a Xapian database, storing information about all available applications in the distribution s repositories. The Xapian DB is distro-agnostik, but distributors need to provide data to fill it. AppStream offers an application-centric way to look at a package database)
The AppStream side doesn t look incredibly great, but the situation is improving. As far as I know, OpenSUSE is shipping AppStream XML to generate the database. Ubuntu ships the desktop-files, and I am working on AppStream support in Debian s Archive Kit. On the Fedora side, negotiations with the infrastructure-team are still going on. I haven t heard anything from Mageia and other AppStream participants yet.
Unfortunately, at least for OpenSUSE, the AppStream efforts seem to be stalled, due to people having moved to different tasks. But efforts to add the remaining missing things exist.
On the software side, Apper (KDE PackageKit frontend) has full support for AppStream. Apper just needs to be compiled with some extra flags to make it use the AppStream database.
On the GNOME-side, GNOME-Software is being developed. The tool will make use of the AppStream database, on distributions where it is available.
Also, a Software-Center for Elementary and other GTK+-based desktops is being developed, which is based on AppStream (already quite usable!).
Using the Ubuntu Software Center on not-Ubuntu-based distributions ist still not much fun, but with the AppStream database available and a working PackageKit 0.8.x with a backend which supports parallel transactions, it is possible to use it.
On the infrastructure side: I recently landed some patches in AppStream-Core, which will improve the search function a lot. AppStream-Core contains all tools necessary to generate the AppStream database. It also contains libappstream, a GObject-based library which can be used to access the AppStream database.
Also, we discuss dropping PackageKit s internal desktop-file-cache in favour of using the AppStream database. If we do that, we will also add software descriptions to the AppStream db, to improve search results and to speed up search for applications. Because we have to deprecate API for that, I expect this change to happen with PackageKit 0.9.x.
As soon as the Freedesktop-Wiki is alive again and my account is re-enabled, I will create compatibility-list, showing which distribution implements what of the PK/AS/LI stuff, especially focusing on components needed for AppStream.
Only a few distributions package AppStream-Core so far. Although it is beta-software, creating packages for it and shipping the required data to generate the AppStream database would be a very huge step forward.
Listaller
(Listaller is a cross-distro 3rd-party software installer, which integrates into
PackageKit and AppStream. It allows installing 3rd-party applications, which are not part of the distributor s repositories, using standard tools used also for native-package handling. Everything which uses PackageKit can make use of Listaller packages too. Listaller also allows sandboxing of new applications, and uses an AppDir-like approach for installing software.)
Listaller is currently undergoing it s last transition before a release with stable API and specifications can be made. Dependency solving will be improved a lot during the current release-cycle, making it less powerful, but working on all distributions instead. (Fedora always had an advantage in dependency-solving, due to RPM providing more package metadata for Listaller to use) This change was delayed due to discussing a possible use of ZeroInstall-feeds to provide missing dependencies with the ZeroInstall team. We did not come to a conclusion about extending the XML, so Listaller will contain an own format to define a dependency, which can reference a ZeroInstall feed. That should be a good solution for everyone.
All these changes will result in IPK1.2, a new version of the IPK spec with small changes in the pkoptions file syntax and huge changes in dependency-handling. The new code is slowly stabilizing in a separate branch, and will soon be merged into master.
The next Listaller release will be the last one of the 0.5.x series, we will start 0.6.x then. KDE currently has support for Listaller through Apper, which is enabled on a few distributions. In GNOME, optional Listaller support is being developed and will be available in one of the upcoming releases.
Currently, to my knowledge, only a few distributions package Listaller. This should improve, so it is easier for application developers to deploy IPK packages.
The upcoming changes in KDE and GNOME to build stable developer platforms will help Listaller a lot in finding matching dependencies, and for stuff which only depends on one software frameworks, installations should be a matter of seconds.
As you can see, lots of things are happening, and there is improvement in all components related to installing and presenting software on Linux machines. However, all these projects have a severe lack of manpower, especially AppStream and Listaller have the lowest number of developers working on the tools (at time, only two active developers). This is the main reason for the slow development. But I am confident that we will have something shipped in the next distribution releases. At least AppStream should be ready then.
[1]: I don t blame Ubuntu for that during the time they wrote an own solution, PackageKit did not have all the required features. (This situation has changed now, fortunately.)
NOTE: I might extend this post with feedback from the different distributions, as soon as I get it.
Lumail <
There was an interesting question by Fool recently on the StackExchange site for Boardgames:
Last week we
One of my recent innocent tweet about Gerrit vs Github triggered much more
reponses and debate that I expected it to. I realize that it might be worth
explaining a bit what I meant, in a text longer than 140 characters.
<figcaption>
A typical case: 3 commits to build a 4 lines long file.
</figcaption>
</figure>
Without Github, the old method that all software used, and that many
software still use (e.g. Linux), is to send a patch set over e-mail (or any
other medium like Gerrit). This method has one positive effect, that it
forces the contributor to acknowledge the list of commits he is going to
publish. So, if the contributor he has fixup commits in his history, they
are going to be seen as first class citizen. And nobody is going to want to
see that, neither your contributor, nor the software maintainers. Therefore,
such a system tend to push contributors to write atomic, logical and
self-contained patchset that can be more easily reviewed.
Technic #2: the History Rewriter
This is actually the good way to build a working and logical patchset using
Git. Rewriting history and amending problematic patches using the famous
git rebase --interactive trick.
The problem is that if your contributor does this and then repush the branch
composing your pull-request to Github, you will both lose the previous
review done, each time. There's no history on the different versions of the
branch that has been pushed.
In the old alternative system like e-mail, no information is lost when
reworked patches are resent, obviously. This is far better because it eases
the following of the iterative discussions that the patch triggered.
Of course, it would be possible for Github to enhance this and fix it, but
currently it doesn't handle this use case correctly..
<figure>
It's not a secret for anyone that I've been contributing to OpenStack as a
daily routine for the last 18 months. The more I contribute, the more I like
the contribution workflow and process. It's already
So while testing


Some weeks ago, I contacted Rosa Mart nez, a tech journalist with some questions regarding what I regarded as
Memo to myself (as I tend to forget it): If you develop gnutls using applications, recompile gnutls with --disable-hardware-acceleration to be able to test them without getting flooded with false-positives.
We ve got all sorts of spiffy changes lined up and another major release! We ve got Hy version
With wheezy released, the floodgates are opened on a lot of debhelper
changes that have been piling up. Most of these should be pretty minor, but
I released one yesterday that will affect all users of