Search Results: "sez"

13 January 2016

Raphaël Hertzog: Freexian s report about Debian Long Term Support, December 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In December, 113.50 work hours have been dispatched among 9 paid contributors. Their reports are available: Evolution of the situation We lost our first silver sponsor (Gandi.net, they prefer to give the same amount of money to Debian directly) and another sponsor reduced his sponsorship level. While this won t show in the hours dispatched in January, we will do a small jump backwards in February (unless we get new sponsors replacing those in the next 3 weeks). This is a bit unfortunate as we are rather looking at reinforcing the amount of sponsorship we get as we approach Wheezy LTS and we will need more support to properly support virtualization related packages and other packages that were formerly excluded from Squeeze LTS. Can you convince your company and help us reach our second goal? In terms of security updates waiting to be handled, the situation is close to last month. It looks like that having about 20 packages needing an update is the normal situation and that we can t really get further down given the time required to process some updates (sometimes we wait until the upstream authors provides a patch, and so on). Thanks to our sponsors We got one new bronze sponsor but he s not listed (he did not fill the form where we request their permission to be listed).

2 comments Liked this article? Click here. My blog is Flattr-enabled.

14 December 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, November 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In November, 114.50 work hours have been dispatched among 8 paid contributors. Their reports are available: Evolution of the situation We lost one hour of funding for December due to a sponsor not renewing, and we don t have any new sponsor lined up right now. There s another sponsor who will reduce his sponsorship starting with 2016. While the situation is relatively healthy right now, we should continue the efforts to find new sponsors, both to ensure we can cover more software in wheezy and to better share the costs: having many small sponsors is more resilient than relying on a few big ones. And we still haven t reached our second goal of funding the equivalent of a full-time position. In terms of security updates waiting to be handled, the situation is close to last month: the dla-needed.txt file lists 19 packages awaiting an update (2 less than last month), the list of open vulnerabilities in Squeeze shows about 22 affected packages in total (1 less than last month). Thanks to our sponsors The new sponsors are in bold.

No comment Liked this article? Click here. My blog is Flattr-enabled.

13 November 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, October 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In September, 85.50 work hours have been dispatched among 8 paid contributors. Their reports are available: Evolution of the situation November crossed a new record with 114.5 hours funded. This is mainly thanks to our first Platinum sponsor: TOSHIBA (through Toshiba Software Development Vietnam). They don t know yet if they can sponsor us in the long term (they hope so), but it s still a nice news as we jumped from 50% to 65% of the objective of the equivalent of a full-time position with a single new sponsor. Currently no change is expected for next month as we don t have any other new sponsor in the process of joining us. We still need more support to be able to support all the packages we could not afford to support during the squeeze cycle. We are currently discussing which package we can or cannot support on the LTS list, see the thread Unsupported packages for Wheezy LTS for the current situation. In terms of security updates waiting to be handled, the situation is close to last month: the dla-needed.txt file lists 21 packages awaiting an update (6 more than last month), the list of open vulnerabilities in Squeeze shows about 23 affected packages in total (exactly like last month). Thanks to our sponsors The new sponsors are in bold.

No comment Liked this article? Click here. My blog is Flattr-enabled.

16 October 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, September 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In September, 71.50 work hours have been dispatched among 7 paid contributors. Their reports are available: Evolution of the situation October is back to the highest level of funding with 85.5 hours funded. The late sponsors have all caught up now. And next month will again rise to a new record with multiple sponsors having joined up. So far we already have two new silver sponsors (Universit Jean Monnet de Saint- tienne and Univention GmbH) and a new bronze sponsor (Entr ouvert). Many thanks to them! With those sponsors we crossed the 50% mark that was our first objective. \o/ But we still need more support to reach our second goal of funding the equivalent of a full time position. That said the increased level of support already allows us to do a better job in some areas that have been neglected : I asked the paid contributors to work towards providing mysql-5.5 in squeeze since version 5.1 is no longer supported by Oracle. We need beta testers to test the upgrade, see this message on the mailling list. In terms of security updates waiting to be handled, the situation is close to last month: the dla-needed.txt file lists 15 packages awaiting an update (3 less than last month), the list of open vulnerabilities in Squeeze shows about 23 affected packages in total (7 less than last month). Thanks to our sponsors The new sponsors are in bold.

No comment Liked this article? Click here. My blog is Flattr-enabled.

17 September 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, August 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In August, 71.50 work hours have been dispatched among 7 paid contributors. Their reports are available: Evolution of the situation September is stable compared to August (71.50 hours per month) and has not caught up back to the level of July as I hoped. Again it s because 2 sponsors were not able to pay their renewal invoice on time (one of last month paid, but another bigger sponsor failed this month). Those sponsors will continue to support us and I would like to be able to say that things will be back to normal next month, but I can t say it since we have also been informed of the (hopefully temporary) defection of another bronze sponsor that will affect us next month. Fortunately there are also good news, we have 3 new sponsors in the pipe (2 silver, 1 platinum) who shall join the project soon. And Blablacar increased their support from Silver to Gold (from 4h/month to 8h/month). But we still need more support in particular since we would like to commit to support virtualization related packages in Wheezy: that s clearly an objective for us. I recently published the summary of the work session held during DebConf 15 in Heidelberg (video recording). It would be really nice if we could get closer to the goal of funding a full-time position. In terms of security updates waiting to be handled, the situation is close to last month: the dla-needed.txt file lists 18 packages awaiting an update (2 less than last month), the list of open vulnerabilities in Squeeze shows about 30 affected packages in total (8 more than last month). Thanks to our sponsors

No comment Liked this article? Click here. My blog is Flattr-enabled.

26 August 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, July 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In July, 79.50 work hours have been dispatched among 7 paid contributors. Their reports are available: Evolution of the situation August has seen a small decrease in terms of sponsored hours (71.50 hours per month) because two sponsors did not pay their renewal invoice on time. That said they reconfirmed their willingness to support us and things should be fixed after the summer. And we should be able to reach our first milestone of funding the equivalent of a half-time position, in particular since a new platinum sponsor might join the project. DebConf 15 happened this month and Debian LTS was featured in a talk and in a work session. Have a look at the video recordings: In terms of security updates waiting to be handled, the situation is better than last month: the dla-needed.txt file lists 20 packages awaiting an update (4 less than last month), the list of open vulnerabilities in Squeeze shows about 22 affected packages in total (11 less than last month). The new LTS frontdesk ensures regular triage of CVE reports and the difference between both counts dropped significantly. That s good! Thanks to our sponsors Thanks to Sig-I/O, a new bronze sponsor, which joins our 35 other sponsors.

One comment Liked this article? Click here. My blog is Flattr-enabled.

16 July 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, June 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In June, 73.50 work hours have been dispatched among 7 paid contributors. Their reports are available: Evolution of the situation July has seen a nice increase in terms of sponsored hours (79.50 hours per month) but the trend is unlikely to continue for the next month, worse it might be negative. While most sponsors who joined us last year in July will renew their support, there are a few where I have no confirmation yet. Many thanks to those who confirmed early: Universit Lille 3, MyTux. Our first milestone of funding the equivalent of a half-time position is unlikely to be reached before DebConf or even this summer. If you want to prove me wrong, it s time to get in touch with your management and convince your company to contribute a small amount. In terms of security updates waiting to be handled, the situation is similar to last month: the dla-needed.txt file lists 24 packages awaiting an update (5 more than last month), the list of open vulnerabilities in Squeeze shows about 33 affected packages in total (3 less than last month). Thanks to our sponsors There are no new sponsors this month. But I decided to include the number of months that the sponsor has been with us. Since we value long-lasting relations, it seemed quite natural to add this.

No comment Liked this article? Click here. My blog is Flattr-enabled.

18 May 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, April 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In April, 81.75 work hours have been dispatched among 5 paid contributors (20.75 hours where unused hours of Ben and Holger that were re-dispatched to other contributors). Their reports are available: Evolution of the situation May has seen a small increase in terms of sponsored hours (66.25 hours per month) and June is going to do even better with at least a new gold sponsor. We will have no problems sustaining the increased workload it implies since three Debian developers joined the team of contributors paid by Freexian (Antoine Beaupr , Santiago Ruano Rinc n, Scott Kitterman). The Jessie release probably shed some light on the Debian LTS project since we announced that Jessie will benefit from 5 years of support. Let s hope that the trend will continue in the following months and that we reach our first milestone of funding the equivalent of a half-time position. In terms of security updates waiting to be handled, the situation is a bit contrasted: the dla-needed.txt file lists 28 packages awaiting an update (12 less than last month), the list of open vulnerabilities in Squeeze shows about 60 affected packages in total (4 more than last month). The extra hours helped to make a good stride in the packages awaiting an update but there are many new vulnerabilities waiting to be triaged. Thanks to our sponsors The new sponsors of the month are in bold.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Raphaël Hertzog: Freexian s report about Debian Long Term Support, April 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In April, 81.75 work hours have been dispatched among 5 paid contributors (20.75 hours where unused hours of Ben and Holger that were re-dispatched to other contributors). Their reports are available: Evolution of the situation May has seen a small increase in terms of sponsored hours (66.25 hours per month) and June is going to do even better with at least a new gold sponsor. We will have no problems sustaining the increased workload it implies since three Debian developers joined the team of contributors paid by Freexian (Antoine Beaupr , Santiago Ruano Rinc n, Scott Kitterman). The Jessie release probably shed some light on the Debian LTS project since we announced that Jessie will benefit from 5 years of support. Let s hope that the trend will continue in the following months and that we reach our first milestone of funding the equivalent of a half-time position. In terms of security updates waiting to be handled, the situation is a bit contrasted: the dla-needed.txt file lists 28 packages awaiting an update (12 less than last month), the list of open vulnerabilities in Squeeze shows about 60 affected packages in total (4 more than last month). The extra hours helped to make a good stride in the packages awaiting an update but there are many new vulnerabilities waiting to be triaged. Thanks to our sponsors The new sponsors of the month are in bold.

No comment Liked this article? Click here. My blog is Flattr-enabled.

4 May 2015

Lunar: Reproducible builds: first week in Stretch cycle

Debian Jessie has been released on April 25th, 2015. This has opened the Stretch development cycle. Reactions to the idea of making Debian build reproducibly have been pretty enthusiastic. As the pace is now likely to be even faster, let's see if we can keep everyone up-to-date on the developments. Before the release of Jessie The story goes back a long way but a formal announcement to the project has only been sent in February 2015. Since then, too much work has happened to make a complete report, but to give some highlights: Lunar did a pretty improvised lightning talk during the Mini-DebConf in Lyon. This past week It seems changes were pilling behind the curtains given the amount of activity that happened in just one week. Toolchain fixes We also rebased the experimental version of debhelper twice to merge the latest set of changes. Lunar submitted a patch to add a -creation-date to genisoimage. Reiner Herrmann opened #783938 to request making -notimestamp the default behavior for javadoc. Juan Picca submitted a patch to add a --use-date flag to texi2html. Packages fixed The following packages became reproducible due to changes of their build dependencies: apport, batctl, cil, commons-math3, devscripts, disruptor, ehcache, ftphs, gtk2hs-buildtools, haskell-abstract-deque, haskell-abstract-par, haskell-acid-state, haskell-adjunctions, haskell-aeson, haskell-aeson-pretty, haskell-alut, haskell-ansi-terminal, haskell-async, haskell-attoparsec, haskell-augeas, haskell-auto-update, haskell-binary-conduit, haskell-hscurses, jsch, ledgersmb, libapache2-mod-auth-mellon, libarchive-tar-wrapper-perl, libbusiness-onlinepayment-payflowpro-perl, libcapture-tiny-perl, libchi-perl, libcommons-codec-java, libconfig-model-itself-perl, libconfig-model-tester-perl, libcpan-perl-releases-perl, libcrypt-unixcrypt-perl, libdatetime-timezone-perl, libdbd-firebird-perl, libdbix-class-resultset-recursiveupdate-perl, libdbix-profile-perl, libdevel-cover-perl, libdevel-ptkdb-perl, libfile-tail-perl, libfinance-quote-perl, libformat-human-bytes-perl, libgtk2-perl, libhibernate-validator-java, libimage-exiftool-perl, libjson-perl, liblinux-prctl-perl, liblog-any-perl, libmail-imapclient-perl, libmocked-perl, libmodule-build-xsutil-perl, libmodule-extractuse-perl, libmodule-signature-perl, libmoosex-simpleconfig-perl, libmoox-handlesvia-perl, libnet-frame-layer-ipv6-perl, libnet-openssh-perl, libnumber-format-perl, libobject-id-perl, libpackage-pkg-perl, libpdf-fdf-simple-perl, libpod-webserver-perl, libpoe-component-pubsub-perl, libregexp-grammars-perl, libreply-perl, libscalar-defer-perl, libsereal-encoder-perl, libspreadsheet-read-perl, libspring-java, libsql-abstract-more-perl, libsvn-class-perl, libtemplate-plugin-gravatar-perl, libterm-progressbar-perl, libterm-shellui-perl, libtest-dir-perl, libtest-log4perl-perl, libtext-context-eitherside-perl, libtime-warp-perl, libtree-simple-perl, libwww-shorten-simple-perl, libwx-perl-processstream-perl, libxml-filter-xslt-perl, libxml-writer-string-perl, libyaml-tiny-perl, mupen64plus-core, nmap, openssl, pkg-perl-tools, quodlibet, r-cran-rjags, r-cran-rjson, r-cran-sn, r-cran-statmod, ruby-nokogiri, sezpoz, skksearch, slurm-llnl, stellarium. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which did not make their way to the archive yet: Improvements to reproducible.debian.net Mattia Rizzolo has been working on compressing logs using gzip to save disk space. The web server would uncompress them on-the-fly for clients which does not accept gzip content. Mattia Rizzolo worked on a new page listing various breakage: missing or bad debbindiff output, missing build logs, unavailable build dependencies. Holger Levsen added a new execution environment to run debbindiff using dependencies from testing. This is required for packages built with GHC as the compiler only understands interfaces built by the same version. debbindiff development Version 17 has been uploaded to unstable. It now supports comparing ISO9660 images, dictzip files and should compare identical files much faster. Documentation update Various small updates and fixes to the pages about PDF produced by LaTeX, DVI produced by LaTeX, static libraries, Javadoc, PE binaries, and Epydoc. Package reviews Known issues have been tagged when known to be deterministic as some might unfortunately not show up on every single build. For example, two new issues have been identified by building with one timezone in April and one in May. RD and help2man add current month and year to the documentation they are producing. 1162 packages have been removed and 774 have been added in the past week. Most of them are the work of proper automated investigation done by Chris West. Summer of code Finally, we learned that both akira and Dhole were accepted for this Google Summer of Code. Let's welcome them! They have until May 25th before coding officialy begins. Now is the good time to help them feel more comfortable by sharing all these little bits of knowledge on how Debian works.

14 April 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, March 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In February, 61 work hours have been equally split among 4 paid contributors. Their reports are available: The remaining hours of Ben and Holger have been redispatched to other contributors for April (during which Mike Gabriel joins the set of paid contributors). BTW, if you want to join the team of paid contributors, read this and apply! Evolution of the situation April has seen no change in terms of sponsored hours but we have two new sponsors in the pipe and May should hopefully have a few more sponsored hours. For the need of a LTS presentation I gave during the Mini-DebConf Lyon I prepared a small graph showing the evolution of the hours sponsored through Freexian:
freexian-hours The growth is rather slow and it will take years to reach our goal of funding the equivalent a full time position (176 hours per month). Even the intermediary goal of funding the equivalent of a half-time position (88h/month) is more than 6 months away given the current growth rate. But the perspective of Wheezy-LTS should help us to convince more organizations and hopefully we will reach that goal sooner. If you want to sponsor the project, check out this page. In terms of security updates waiting to be handled, the situation looks similar to last month: the dla-needed.txt file lists 40 packages awaiting an update (exactly like last month), the list of open vulnerabilities in Squeeze shows about 56 affected packages in total (2 less than last month). Thanks to our sponsors The new sponsors of the month are in bold (none this month).

No comment Liked this article? Click here. My blog is Flattr-enabled.

17 March 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, February 2015

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In February, 58 work hours have been equally split among 4 paid contributors. Their reports are available: Evolution of the situation During the last month, we gained 3 paid work hours: we re now at 61 hours per month sponsored by 28 organizations and we have one supplementary sponsor in the pipe that should bring 4 more hours. The increase is not very quick but seems to be steady. Hopefully at some point, we will have enough resources to do a more exhaustive job. For now, the paid contributors handle in priority the most popular packages used by the sponsors and there are some packages in the end of the queue which have open security issues for months already (example: CVE-2012-6685 on libnokogiri-ruby). So, as usual, we are looking for more sponsors. In terms of security updates waiting to be handled, the situation looks a little bit worse than last month: the dla-needed.txt file lists 40 packages awaiting an update (3 more than last month), the list of open vulnerabilities in Squeeze shows about 58 affected packages in total (5 less than last month). We are getting a bit more effective with CVE triage. A logo for the LTS project? Every time that I write an LTS report, I remember that it would be nice if my LTS related articles could feature a nice picture/logo that reminds people of the LTS team/initiative. Is there anyone up for the challenge of creating that logo? :-) Thanks to our sponsors The new sponsors of the month are in bold.

No comment Liked this article? Click here. My blog is Flattr-enabled.

12 February 2015

Raphaël Hertzog: Freexian s report about Debian Long Term Support, January 2015

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In January, 48 work hours have been equally split among 4 paid contributors. Their reports are available: Evolution of the situation During the last month, the number of paid work hours has made a noticeable jump: we re now at 58 hours per month. At this rate, we would need 3 more months to reach our minimal goal of funding the equivalent of a half-time position. Unfortunately, the number of new sponsors actually in the process is not likely to be enough to have a similar raise next month. So, as usual, we are looking for more sponsors. In terms of security updates waiting to be handled, the situation looks a bit worse than last month: the dla-needed.txt file lists 37 packages awaiting an update (7 more than last month), the list of open vulnerabilities in Squeeze shows about 63 affected packages in total (7 more than last month). The increase is not too worrying, but the waiting time before an issue is dealt with is sometimes more problematic. To be able to deal with all incoming issues in a timely manner, the LTS team needs more resources: some months will have more issues than usual, some issues will be longer to handle than others, etc. Thanks to our sponsors The new sponsors of the month are in bold.

No comment Liked this article? Click here. My blog is Flattr-enabled.

16 January 2015

Raphaël Hertzog: Freexian s fifth report about Debian Long Term Support

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In December 46 work hours have been equally split among 4 paid contributors (note that Thorsten and Rapha l have actually spent more hours because they took over some hours that Holger did not do over the former months). Their reports are available: Evolution of the situation Compared to last month, the number of paid work hours has almost not increased (we are at 48 hours per month). We still have a couple of new sponsors in the pipe but with the new year they did not complete the process yet. Hopefully next month will see a noticeable increase. As usual, we are looking for more sponsors to reach our our minimal goal of funding the equivalent of a half-time position. Those of you who are struggling to spend money in the last quarter due to budget overrun, now is a good time to see if you want to include Debian LTS support in your 2015 budget! In terms of security updates waiting to be handled, the situation looks similar to last month: the dla-needed.txt file lists 30 packages awaiting an update (3 more than last month), the list of open vulnerabilities in Squeeze shows about 56 affected packages in total. We do not manage to clear the backlog but it s not getting significantly worse either. Thanks to our sponsors

No comment Liked this article? Click here. My blog is Flattr-enabled.

16 December 2014

Raphael Geissert: Editing Debian online with sources.debian.net

How cool would it be to fix that one bug you just found without having to download a source package? and without leaving your browser?

Inspired by github's online code editing, during Debconf 14 I worked on integrating an online editor on debsources (the software behind sources.debian.net). Long story short: it is available today, for users of chromium (or anything supporting chrome extensions).

After installing the editor for sources.debian.net extension, go straight to sources.debian.net and enjoy!

Go from simple debsources:


To debsources on steroids:


All in all, it brings:

Clone it or fork it:
git clone https://github.com/rgeissert/ace-sourced.n.git

For example, head to apt's source code, find a typo and correct it online: open apt.cc, click on edit, make the changes, click on email patch. Yes! it can generate a mail template for sending the patch to the BTS: just add a nice message and your patch is ready to be sent.

Didn't find any typo to fix? how sad, head to codesearch and search Debian for a spelling mistake, click on any result, edit, correct, email! you will have contributed to Debian in less than 5 minutes without leaving your browser.

The editor was meant to be integrated into debsources itself, without the need of a browser extension. This is expected to be done when the requirements imposed by debsources maintainers are sorted out.

Kudos to Harlan Lieberman who helped debug some performance issues in the early implementations of the integration and for working on the packaging of the Ace editor.

11 December 2014

Raphaël Hertzog: Freexian s fourth report about Debian Long Term Support

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In November 42.5 work hours have been equally split among 3 paid contributors. Their reports are available: New paid contributors Last month we mentioned the possibility to recruit more paid contributors to better share the work load and this has already happened: Ben Hutchings and Mike Gabriel join the list of paid contributors. Ben, as a kernel maintainer, will obviously take care of releasing Linux security updates. We are glad to have him on board because backporting kernel fixes really need some skills that nobody else had within the team of paid contributors. Evolution of the situation Compared to last month, the number of paid work hours has almost not increased (we are at 45.7 hours per month) but we are in the process of adding a few more sponsors: Roche Diagnostics International AG, Misal-System, Bitfolk LTD. And we are still in contact with a couple of other companies which have announced their willingness to contribute but which are waiting the new fiscal year. But even with those new sponsors, we still have some way to go to reach our minimal goal of funding the equivalent of a half-time position. So consider asking your company representative to join this project! In terms of security updates waiting to be handled, the situation looks better than last month: the dla-needed.txt file lists 27 packages awaiting an update (6 less than last month), the list of open vulnerabilities in Squeeze shows about 58 affected packages in total. Like last month, we re a bit behind in terms of CVE triaging and there are still many packages using SSLv3 where we have no clear plan (in response to the POODLE issues). The good side is that even though the kernel update spent a large chunk of time to Holger and Rapha l, we still managed to further reduce the backlog of security issues. Thanks to our sponsors

No comment Liked this article? Click here. My blog is Flattr-enabled.

12 November 2014

Raphaël Hertzog: Freexian s third report about Debian Long Term Support

Like last month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In October 2014, we affected 13.75h works hours to 3 contributors: Obviously, only the hours done have been paid. Should the backlog grow further, we will seek for more paid contributors (to share the workload) and to make it easier to redispatch work hours once a contributor knows that he won t be able to handle the hours that were affected to him/her. Evolution of the situation Compared to last month, we gained two new sponsors (Daevel and FOSSter, thanks to them!) and we have now 45.5 hours of paid LTS work to spend each month. That s great but we are still far from our minimal goal of funding the equivalent of a half-time position. In terms of security updates waiting to be handled, the situation is a bit worse than last month: while the dla-needed.txt file only lists 33 packages awaiting an update (6 less than last month), the list of open vulnerabilities in Squeeze shows about 60 affected packages in total. This differences has two explanations: CVE triaging for squeeze has not been done in the last days, and the POODLE issue(s) with SSLv3 affects a very large number of packages where it s not always clear what the proper action is. In any case, it s never too late to join the growing list of sponsors and help us do a better job, please check with your company managers. If not possible for this year, consider including it in the budget for next year. Thanks to our sponsors Let me thank our main sponsors:

No comment Liked this article? Click here. My blog is Flattr-enabled.

15 October 2014

Raphaël Hertzog: Freexian s second report about Debian Long Term Support

Like last month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In September 2014, 3 contributors have been paid for 11h each. Here are their individual reports: Evolution of the situation Compared to last month, we have gained 5 new sponsors, that s great. We re now at almost 25% of a full-time position. But we re not done yet. We believe that we would need at least twice as many sponsored hours to do a reasonable work with at least the most used packages, and possibly four times as much to be able to cover the full archive. We re now at 39 packages that need an update in Squeeze (+9 compared to last month), and the contributors paid by Freexian did handle 11 during last month (this gives an approximate rate of 3 hours per update, CVE triage included). Open questions Dear readers, what can we do to convince more companies to join the effort? The list of sponsors contains almost exclusively companies from Europe. It s true that Freexian s offer is in Euro but the economy is world-wide and it s common to have international invoices. When Ivan Kohler asked if having an offer in dollar would help convince other companies, we got zero feedback. What are the main obstacles that you face when you try to convince your managers to get the company to contribute? By the way, we prefer that companies take small sponsorship commitments that they can afford over multiple years over granting lots of money now and then not being able to afford it for another year. Thanks to our sponsors Let me thank our main sponsors:

10 September 2014

Raphaël Hertzog: Freexian s first report about Debian Long Term Support

When we setup Freexian s offer to bring together funding from multiple companies in order to sponsor the work of multiple developers on Debian LTS, one of the rules that I imposed is that all paid contributors must provide a public monthly report of their paid work. While the LTS project officially started in June, the first month where contributors were actually paid has been July. Freexian sponsored Thorsten Alteholz and Holger Levsen for 10.5 hours each in July and for 16.5 hours each in August. Here are their reports: It s worth noting that Freexian sponsored Holger s work to fix the security tracker to support squeeze-lts. It s my belief that using the money of our sponsors to make it easier for everybody to contribute to Debian LTS is money well spent. As evidenced by the progress bar on Freexian s offer page, we have not yet reached our minimal goal of funding the equivalent of a half-time position. And it shows in the results, the dla-needed.txt still shows around 30 open issues. This is slightly better than the state two months ago but we can improve a lot on the average time to push out a security update To have an idea of the relative importance of the contributions of the paid developers, I counted the number of uploads made by Thorsten and Holger since July: of 40 updates, they took care of 19 of them, so about the half. I also looked at the other contributors: Rapha l Geissert stands out with 9 updates (I believe that he is contracted by lectricit de France for doing this) and most of the other contributors look like regular Debian maintainers taking care of their own packages (Paul Gevers with cacti, Christoph Berg with postgresql, Peter Palfrader with tor, Didier Raboud with cups, Kurt Roeckx with openssl, Balint Reczey with wireshark) except Matt Palmer and Luciano Bello who (likely) are benevolent members of the LTS team. There are multiple things to learn here:
  1. Paid contributors already handle almost 70% of the updates. Counting only on volunteers would not have worked.
  2. Quite a few companies that promised help (and got mentioned in the press release) have not delivered the promised help yet (neither through Freexian nor directly).
Last but not least, this project wouldn t exist without the support of multiple companies and organizations. Many thanks to them: Hopefully this list will expand over time! Any help to reach out to new companies and organizations is more than welcome.

One comment Liked this article? Click here. My blog is Flattr-enabled.

27 April 2014

Vincent Bernat: Local corporate APT repositories

Distributing software efficiently accross your platform can be difficult. Every distribution comes with a package manager which is usually suited for this task. APT can be relied upon on when using Debian or a derivative. Unfortunately, the official repositories may not contain everything you need. When you require unpackaged software or more recent versions, it is possible to setup your own local repository. Most of what is presented here was setup for Dailymotion and was greatly inspired by the work done by Rapha l Pinson at Orange.

Setting up your repositories There are three kinds of repositories you may want to setup:
  1. A distribution mirror. Such a mirror will save bandwidth, provide faster downloads and permanent access, even when someone searches Google on Google.
  2. A local repository for your own packages with the ability to have a staging zone to test packages on some servers before putting them in production.
  3. Mirrors for unofficial repositories, like Ubuntu PPA. To avoid unexpected changes, such a repository will also get a staging and a production zone.
Before going further, it is quite important to understand what a repository is. Let s illustrate with the following line from my /etc/apt/sources.list:
deb http://ftp.debian.org/debian/ unstable main contrib non-free
In this example, http://ftp.debian.org/debian/ is the repository and unstable is the distribution. A distribution is subdivided into components. We have three components: main, contrib and non-free. To setup repositories, we will use reprepro. This is not the only solution but it has a good balance between versatility and simplicity. reprepro can only handle one repository. So, the first choice is about how you will split your packages in repositories, distributions and components. Here is what matters:
  • A repository cannot contain two identical packages (same name, same version, same architecture).
  • Inside a component, you can only have one version of a package.
  • Usually, a distribution is a subset of the versions while a component is a subset of the packages. For example, in Debian, with the distribution unstable, you choose to get the most recent versions while with the component main, you choose to get DFSG-free software only.
If you go for several repositories, you will have to handle several reprepro instances and won t be able to easily copy packages from one place to another. At Dailymotion, we put everything in the same repository but it would also be perfectly valid to have three repositories:
  • one to mirror the distribution,
  • one for your local packages, and
  • one to mirror unofficial repositories.
Here is our target setup: Local APT repository

Initial setup First, create a system user to work with the repositories:
$ adduser --system --disabled-password --disabled-login \
>         --home /srv/packages \
>         --group reprepro
All operations should be done with this user only. If you want to setup several repositories, create a directory for each of them. Each repository has those subdirectories:
  • conf/ contains the configuration files,
  • gpg/ contains the GPG stuff to sign the repository1,
  • logs/ contains the logs,
  • www/ contains the repository that should be exported by the web server.
Here is the content of conf/options:
outdir +b/www
logdir +b/logs
gnupghome +b/gpg
Then, you need to create the GPG key to sign the repository:
$ GNUPGHOME=gpg gpg --gen-key
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 10y
Key expires at mer. 08 nov. 2023 22:30:58 CET
Is this correct? (y/N) y
Real name: Dailymotion Archive Automatic Signing Key
Email address: the-it-operations@dailymotion.com
Comment: 
[...]
By setting an empty password, you allow reprepro to run unattended. You will have to distribute the public key of your new repository to let APT check the archive signature. An easy way is to ship it in some package.

Local mirror of an official distribution Let s start by mirroring a distribution. We want a local mirror of Ubuntu Precise. For this, we need to do two things:
  1. Setup a new distribution in conf/distributions.
  2. Configure the update sources in conf/updates.
Let s add this block to conf/distributions:
# Ubuntu Precise
Origin: Ubuntu
Label: Ubuntu
Suite: precise
Version: 12.04
Codename: precise
Architectures: i386 amd64
Components: main restricted universe multiverse
UDebComponents: main restricted universe multiverse
Description: Ubuntu Precise 12.04 (with updates and security)
Contents: .gz .bz2
UDebIndices: Packages Release . .gz
Tracking: minimal
Update: - ubuntu-precise ubuntu-precise-updates ubuntu-precise-security
SignWith: yes
This defines the precise distribution in our repository. It contains four components: main, restricted, universe and multiverse (like the regular distribution in official repositories). The Update line starts with a dash. This means reprepro will mark everything as deleted before updating with the provided sources. Old packages will not be kept when they are removed from Ubuntu. In conf/updates, we define the sources:
# Ubuntu Precise
Name: ubuntu-precise
Method: http://fr.archive.ubuntu.com/ubuntu
Fallback: http://de.archive.ubuntu.com/ubuntu
Suite: precise
Components: main main multiverse restricted universe
UDebComponents: main restricted universe multiverse
Architectures: amd64 i386
VerifyRelease: 437D05B5
GetInRelease: no
# Ubuntu Precise Updates
Name: ubuntu-precise-updates
Method: http://fr.archive.ubuntu.com/ubuntu
Fallback: http://de.archive.ubuntu.com/ubuntu
Suite: precise-updates
Components: main restricted universe multiverse
UDebComponents: main restricted universe multiverse
Architectures: amd64 i386
VerifyRelease: 437D05B5
GetInRelease: no
# Ubuntu Precise Security
Name: ubuntu-precise-security
Method: http://fr.archive.ubuntu.com/ubuntu
Fallback: http://de.archive.ubuntu.com/ubuntu
Suite: precise-security
Components: main restricted universe multiverse
UDebComponents: main restricted universe multiverse
Architectures: amd64 i386
VerifyRelease: 437D05B5
GetInRelease: no
The VerifyRelease lines are GPG key fingerprint to use to check the remote repository. The key needs to be imported in the local keyring:
$ gpg --keyring /usr/share/keyrings/ubuntu-archive-keyring.gpg \
>     --export 437D05B5   GNUPGHOME=gpg gpg --import
Another important point is that we merge three distributions (precise, precise-updates and precise-security) into a single distribution (precise) in our local repository. This may cause some difficulties with tools expecting the three distributions to be available (like the Debian Installer2). Next, you can run reprepro and ask it to update your local mirror:
$ reprepro update
This will take some time on the first run. You can execute this command every night. reprepro is not the fastest mirror solution but it is easy to setup, flexible and reliable.

Repository for local packages Let s configure the repository to accept local packages. For each official distribution (like precise), we will configure two distributions:
  • precise-staging contains packages that have not been fully tested and not ready to go to production.
  • precise-prod contains production packages copied from precise-staging.
In our workflow, packages are introduced in precise-staging where they can be tested and will be copied to precise-prod when we want them to be available for production. You can adopt a more complex workflow if you need. The reprepro part is quite easy. We add the following blocks into conf/distributions:
# Dailymotion Precise packages (staging)
Origin: Dailymotion #  
Label: dm-staging   #  
Suite: precise-staging
Codename: precise-staging
Architectures: i386 amd64 source
Components: main role/dns role/database role/web #  
Description: Dailymotion Precise staging repository
Contents: .gz .bz2
Tracking: keep
SignWith: yes
NotAutomatic: yes #  
Log: packages.dm-precise-staging.log
 --type=dsc email-changes
# Dailymotion Precise packages (prod)
Origin: Dailymotion #  
Label: dm-prod      #  
Suite: precise-prod
Codename: precise-prod
Architectures: i386 amd64 source
Components: main role/dns role/database role/web #  
Description: Dailymotion Precise prod repository
Contents: .gz .bz2
Tracking: keep
SignWith: yes
Log: packages.dm-precise-prod.log
First notice we use several components (in ):
  • main will contain packages that are not specific to a subset of the platform. If you put a package in main, it should work correctly on any host.
  • role/* are components dedicated to a subset of the platform. For example, in role/dns, we ship a custom version of BIND.
The staging distribution has the NotAutomatic flag (in ) which disallows the package manager to install those packages except if the user explicitely requests it. Just below, when a new dsc file is uploaded, the hook email-changes will be executed. It should be in the conf/ directory. The Origin and Label lines (in ) are quite important to be able to define an explicit policy of which packages should be installed. Let s say we use the following /etc/apt/sources.list file:
# Ubuntu packages
deb http://packages.dm.gg/dailymotion precise main restricted universe multiverse
# Dailymotion packages
deb http://packages.dm.gg/dailymotion precise-prod    main role/dns
deb http://packages.dm.gg/dailymotion precise-staging main role/dns
All servers have the precise-staging distribution. We must ensure we won t install those packages by mistake. The NotAutomatic flag is one possible safe-guard. We also use a tailored /etc/apt/preferences:
Explanation: Dailymotion packages of a specific component should be more preferred
Package: *
Pin: release o=Dailymotion, l=dm-prod, c=role/*
Pin-Priority: 950
Explanation: Dailymotion packages should be preferred
Package: *
Pin: release o=Dailymotion, l=dm-prod
Pin-Priority: 900
Explanation: staging should never be preferred
Package: *
Pin: release o=Dailymotion, l=dm-staging
Pin-Priority: -100
By default, packages will have a priority of 500. By setting a priority of -100 to the staging distribution, we ensure the packages cannot be installed at all. This is stronger than NotAutomatic which sets the priority to 1. When a package exists in Ubuntu and in our local repository, we ensure that, if this is a production package, we will use ours by using a priority of 900 (or 950 if we match a specific role component). Have a look at the How APT Interprets Priorities section of apt_preferences(5) manual page for additional information. Keep in mind that version matters only when the priority is the same. To check if everything works as you expect, use apt-cache policy:
$ apt-cache policy php5-memcache
  Installed: 3.0.8-1~precise2~dm1
  Candidate: 3.0.8-1~precise2~dm1
  Version table:
 *** 3.0.8-1~precise2~dm1 0
        950 http://packages.dm.gg/dailymotion/ precise-prod/role/web amd64 Packages
        100 /var/lib/dpkg/status
     3.0.8-1~precise1~dm4 0
        900 http://packages.dm.gg/dailymotion/ precise-prod/main amd64 Packages
       -100 http://packages.dm.gg/dailymotion/ precise-staging/main amd64 Packages
     3.0.6-1 0
        500 http://packages.dm.gg/dailymotion/ precise/universe amd64 Packages
If we want to install a package from the staging distribution, we can use apt-get with the -t precise-staging option to raise the priority of this distribution to 990. Once you have tested your package, you can copy it from the staging distribution to the production distribution:
$ reprepro -C main copysrc precise-prod precise-staging wackadoodle

Local mirror of third-party repositories Sometimes, you want a software published on some third-party repository without to repackage it yourself. A common example is the repositories edited by hardware vendors. Like for an Ubuntu mirror, there are two steps: defining the distribution and defining the source. We chose to put such mirrors into the same distributions as our local packages but with a dedicated component for each mirror. This way, those third-party packages will share the same workflow as our local packages: they will appear in the staging distribution, we validate them and copy them to the production distribution. The first step is to add the components and an appropriate Update line to conf/distributions:
Origin: Dailymotion
Label: dm-staging
Suite: precise-staging
Components: main role/dns role/database role/web vendor/hp
Update: hp
# [...]
Origin: Dailymotion
Label: dm-prod
Suite: precise-prod
Components: main role/dns role/database role/web vendor/hp
# [...]
We added the vendor/hp component to both the staging and the production distributions. However, only the staging distribution gets an Update line (remember, packages will be copied manually into the production distribution). We declare the source in conf/updates:
# HP repository
Name: hp
Method: http://downloads.linux.hp.com/SDR/downloads/ManagementComponentPack/
Suite: precise/current
Components: non-free>vendor/hp
Architectures: i386 amd64
VerifyRelease: 2689B887
GetInRelease: no
Don t forget to add the GPG key to your local keyring. Notice an interesting feature of reprepro: we copy the remote non-free component to our local vendor/hp component. Then, you can synchronize the mirror with reprepro update. Once the packages have been tested, you will have to copy them in the production distribution.

Building Debian packages Our reprepro setup seems complete, but how do we put packages into the staging distribution? You have several options to build Debian packages for your local repository. It really depends on how much time you want to invest in this activity:
  1. Build packages from source by adding a debian/ directory. This is the classic way of building Debian packages. You can start from scratch or use an existing package as a base. In the latest case, the package can be from the official archive but for a more recent distribution or a backport or from an unofficial repository.
  2. Use a tool that will create a binary package from a directory, like fpm. Such a tool will try to guess a lot of things to minimize your work. It can even download everything for you.
There is no universal solution. If you don t have the time budget for building packages from source, have a look at fpm. I would advise you to use the first approach when possible because you will get those perks for free:
  • You keep the sources in your repository. Whenever you need to rebuild something to fix an emergency bug, you won t have to hunt the sources which may be unavailable when you need them the most. Of course, this only works if you build packages that don t download stuff directly from the Internet.
  • You also keep the recipe3 to build the package in your repository. If someone enables some option and rebuild the package, you won t accidently drop this option on the next build. Those changes can be documented in debian/changelog. Moreover, you can use a version control software for the whole debian/ directory.
  • You can propose your package for inclusion into Debian. This will help many people once the package hits the archive.

Builders We chose pbuilder as a builder4. Its setup is quite straightforward. Here is our /etc/pbuilderrc:
DISTRIBUTION=$DIST
NAME="$DIST-$ARCH"
MIRRORSITE=http://packages.dm.gg/dailymotion
COMPONENTS=("main" "restricted" "universe" "multiverse")
OTHERMIRROR="deb http://packages.dm.gg/dailymotion $ DIST -staging main"
HOOKDIR=/etc/pbuilder/hooks.d
BASE=/var/cache/pbuilder/dailymotion
BASETGZ=$BASE/$NAME/base.tgz
BUILDRESULT=$BASE/$NAME/results/
APTCACHE=$BASE/$NAME/aptcache/
DEBBUILDOPTS="-sa"
KEYRING="/usr/share/keyrings/dailymotion-archive.keyring.gpg"
DEBOOTSTRAPOPTS=("--arch" "$ARCH" "--variant=buildd" "$ DEBOOTSTRAPOPTS[@] " "--keyring=$KEYRING")
APTKEYRINGS=("$KEYRING")
EXTRAPACKAGES=("dailymotion-archive-keyring")
pbuilder is expected to be invoked with DIST, ARCH and optionally ROLE environment variables. Building the initial bases can be done like this:
for ARCH in i386 amd64; do
  for DIST in precise; do
    export ARCH
    export DIST
    pbuilder --create
  done
done
We don t create a base for each role. Instead, we use a D hook to add the appropriate source:
#!/bin/bash
[ -z "$ROLE" ]    
  cat >> /etc/apt/sources.list <<EOF
deb http://packages.dm.gg/dailymotion $ DIST -staging role/$ ROLE 
EOF
 
apt-get update
We ensure packages from our staging distribution are preferred over other packages by adding an /etc/apt/preferences file in a E hook:
#!/bin/bash
cat > /etc/apt/preferences <<EOF
Explanation: Dailymotion packages are of higher priority
Package: *
Pin: release o=Dailymotion
Pin-Priority: 900
EOF
We also use a C hook to get a shell in case there is an error. This is convenient to debug a problem:
#!/bin/bash
apt-get install -y --force-yes vim less
cd /tmp/buildd/*/debian/..
/bin/bash < /dev/tty > /dev/tty 2> /dev/tty
A manual build can be run with:
$ ARCH=amd64 DIST=precise ROLE=web pbuilder \
>         --build somepackage.dsc

Version numbering To avoid to apply complex rules to chose a version number for a package, we chose to treat everything as a backport, even in-house software. We use the following scheme: X-Y~preciseZ+dmW.
  • X is the upstream version5.
  • Y is the Debian version. If there is no Debian version, use 0.
  • Z is the Ubuntu backport version. Again, if such a version doesn t exist, use 0.
  • W is our version of the package. We increment it when we make a change to the packaging. This is the only number we are allowed to control. All the others are set by an upstream entity, unless it doesn t exist and in this case, you use 0.
Let s suppose you need to backport wackadoodle. It is available in a more recent version of Ubuntu as 1.4-3. Your first backport will be 1.4-3~precise0+dm1. After a change to the packaging, the version will be 1.4-3~precise0+dm2. A new upstream version 1.5 is available and you need it. You will use 1.5-0~precise0+dm1. Later, this new upstream version will be available in some version of Ubuntu as 1.5-3ubuntu1. You will rebase your changes on this version and get 1.5-3ubuntu1~precise0+dm1. When using Debian instead of Ubuntu, a compatible convention could be : X-Y~bpo70+Z~dm+W.

Uploading To upload a package, a common setup is the following workflow:
  1. Upload the source package to an incoming directory.
  2. reprepro will notice the source package, check its correctness (signature, distribution) and put it in the archive.
  3. The builder will notice a new package needs to be built and build it.
  4. Once the package is built, the builder will upload the result to the incoming directory.
  5. reprepro will notice again the new binary package and integrate it in the archive.
This workflow has the disadvantage to have many moving pieces and to leave the user in the dark while the compilation is in progress. As an alternative, a simple script can be used to execute each step synchronously. The user can follow on their terminal that everything works as expected. Once we have the .changes file, the build script just issues the appropriate command to include the result in the archive:
$ reprepro -C main include precise-staging \
>      wackadoodle_1.4-3~precise0+dm4_amd64.changes
Happy hacking!

  1. The gpg/ directory could be shared by several repositories.
  2. We teached Debian Installer to work with our setup with an appropriate preseed file.
  3. fpm-cookery is a convenient tool to write recipes for fpm, similar to Homebrew or a BSD port tree. It could be used to achieve the same goal.
  4. sbuild is an alternative to pbuilder and is the official builder for both Debian and Ubuntu. Historically, pbuilder was more focused on developers needs.
  5. For a Git snapshot, we use something like 1.4-git20130905+1-ae42dc1 which is a snapshot made after version 1.4 (use 0.0 if no version has ever been released) at the given date. The following 1 is to be able to package different snapshots at the same date while the hash is here in case you need to retrieve the exact snapshot.

Next.

Previous.