Search Results: "Martin"

2 May 2016

Reproducible builds folks: Reproducible builds: week 53 in Stretch cycle

What happened in the Reproducible Builds effort between April 24th and 30th 2016. Media coverage Reproducible builds were mentioned explicitly in two talks at the Mini-DebConf in Vienna: Aspiration together with the OTF CommunityLab released their report about the Reproducible Builds summit in December 2015 in Athens. Toolchain fixes Now that the GCC development window has been opened again, the SOURCE_DATE_EPOCH patch by Dhole and Matthias Klose to address the issue timestamps_from_cpp_macros (__DATE__ / __TIME__) has been applied upstream and will be released with GCC 7. Following that Matthias Klose also has uploaded gcc-5/5.3.1-17 and gcc-6/6.1.1-1 to unstable with a backport of that SOURCE_DATE_EPOCH patch. Emmanuel Bourg uploaded maven/3.3.9-4, which uses SOURCE_DATE_EPOCH for the maven.build.timestamp. (SOURCE_DATE_EPOCH specification) Other upstream changes Alexis Bienven e submitted a patch to Sphinx which extends SOURCE_DATE_EPOCH support for copyright years in generated documentation. Packages fixed The following 12 packages have become reproducible due to changes in their build dependencies: hhvm jcsp libfann libflexdock-java libjcommon-java libswingx1-java mobile-atlas-creator not-yet-commons-ssl plexus-utils squareness svnclientadapter The following packages have became reproducible after being fixed: Some uploads have fixed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Package reviews 95 reviews have been added, 15 have been updated and 129 have been removed in this week. 22 FTBFS bugs have been reported by Chris Lamb and Martin Michlmayr. diffoscope development strip-nondeterminism development tests.reproducible-builds.org Misc. Amongst the 29 interns who will work on Debian through GSoC and Outreachy there are four who will be contributing to Reproducible Builds for Debian and Free Software. We are very glad to welcome ceridwen, Satyam Zode, Scarlett Clark and Valerie Young and look forward to working together with them the coming months (and maybe beyond)! This week's edition was written by Reiner Herrmann and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

20 April 2016

Reproducible builds folks: Reproducible builds: week 51 in Stretch cycle

What happened in the reproducible builds effort between April 10th and April 16th 2016: Toolchain fixes Antoine Beaupr suggested that gitpkg stops recording timestamps when creating upstream archives. Antoine Beaupr also pointed out that git-buildpackage diverges from the default gzip settings which is a problem for reproducibly recreating released tarballs which were made using the defaults. Alexis Bienven e submitted a patch extending sphinx SOURCE_DATE_EPOCH support to copyright year. Packages fixed The following packages have become reproducible due to changes in their build dependencies: atinject-jsr330, avis, brailleutils, charactermanaj, classycle, commons-io, commons-javaflow, commons-jci, gap-radiroot, jebl2, jetty, libcommons-el-java, libcommons-jxpath-java, libjackson-json-java, libjogl2-java, libmicroba-java, libproxool-java, libregexp-java, mobile-atlas-creator, octave-econometrics, octave-linear-algebra, octave-odepkg, octave-optiminterp, rapidsvn, remotetea, ruby-rinku, tachyon, xhtmlrenderer. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet: diffoscope development Zbigniew J drzejewski-Szmek noted in #820631 that diffoscope doesn't work properly when a file contains several cpio archives. Package reviews 21 reviews have been added, 14 updated and 22 removed in this week. New issue found: timestamps_in_htm_by_gap. Chris Lamb reported 10 new FTBFS issues. Misc. The video and the slides from the talk "Reproducible builds ecosystem" at LibrePlanet 2016 have been published now. This week's edition was written by Lunar and Holger Levsen. h01ger automated the maintenance and publishing of this weekly newsletter via git.

4 April 2016

Raphaël Hertzog: My Free Software Activities in February and March 2016

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. I skipped my monthly report last time so this one will cover two months. I will try to list only the most important things to not make it too long. The Debian Handbook I worked with Ryuunosuke Ayanokouzi to prepare a paperback version of the Japanese translation of my book. Thanks to the efforts of everybody, it s now available. Unfortunately, Lulu declined to take it in distribution program so it won t be available on traditional bookstores (like Amazon, etc.). The reason is that they do not support non-latin character sets in the meta-data. I tried to cheat a little bit by inputting the description in English (still explaining that the book was in Japanese) but they rejected it nevertheless because the English title could mislead people. So the paperback is only available on lulu.com. Fortunately, the shipping costs are reasonable if you pick the most economic offer. Following this I invited the Italian, Spanish and Brazilian Portuguese translators to complete the work (they were close will all the strings already translated, mainly missing translated screenshots and some backcover content) so that we can also release paperback versions in those languages. It s getting close to completion for them. Hopefully we will have those available until next month. Distro Tracker In early February, I tweaked the configuration to send (by email) exceptions generated by incoming mails and by routine task. Before this they were logged but I did not take the time to look into them. This quickly brought a few issues into light and I fixed them as they appeared: for instance the bounce handling code was getting confused when the character case was not respected, and it appears that some emails come back to us after having been lowercased. Also the code was broken when the References field used more than one line on incoming control emails. This brought into light a whole class of problems with the database storing twice the same email with only differing case. So I did further work to merge all those duplicate entries behind a single email entry. Later, the experimental Sources files changed and I had to tweak the code to work with the removal of the Files field (relying instead on Checksums-* to find out the various files part of the entry). At some point, I also fixed the login form to not generate an exception when the user submits an empty form. I also decided that I no longer wanted to support Django 1.7 in distro tracker as Django 1.8 is the current LTS version. I asked the Debian system administrators to update the package on tracker.debian.org with the version in jessie-backports. This allowed me to fix a few deprecation warnings that I kept triggering because I wanted the code to work with Django 1.7. One of those warnings was generated by django-jsonfield though and I could not fix it immediately. Instead I prepared a pull request that I submitted to the upstream author. Oh, and a last thing, I tweaked the CSS to densify the layout on the package page. This was one of the most requested changes from the people who were still preferring packages.qa.debian.org over tracker.debian.org. Kali and new pkg-security team As part of my Kali work, I have been fixing RC bugs in Debian packages that we use in Kali. But in many cases, I stumbled upon packages whose maintainers were really missing in action (MIA). Up to now, we were only doing non-maintainers upload (NMU) but I want to be able to maintain those packages more effectively so we created a new pkg-security team (we re only two right now and we have no documentation yet, but if you want to join, you re welcome, in particular if you maintain a package which is useful in the security field). arm64 work. The first 3 packages that we took over (ssldump, sucrack, xprobe) are actually packages that were missing arm64 builds. We just started our arm64 port on Kali and we fixed them for that architecture. Since they were no longer properly maintained, in most cases it was just a matter of using dh_autoreconf to get up-to-date config. sub,guess files. We still miss a few packages on arm64: vboot-utils (that we will likely take over soon since it s offered for adoption), ruby-libv8 and ruby-therubyracer, ntopng (we have to wait a new luajit which is only in experimental right now). We also noticed that dh-make-golang was not available on arm64, after some discussion on #debian-buildd, I filed two bugs for this: #819472 on dh-make-golang and #819473 on dh-golang. RC bug fixing. hdparm was affected by multiple RC bugs and the release managers were trying to get rid of it from testing. This removed multiple packages that were used by Kali and its users. So I investigated the situation of that package, convinced the current maintainers to orphan it, asked for new maintainers on debian-devel, reviewed multiple updates prepared by the new volunteers and sponsored their work. Now hdparm is again RC-bug free and has the latest upstream version. We also updated jsonpickle to 0.9.3-1 to fix RC bug #812114 (that I forwarded upstream first). Systemd presets support in init-system-helpers. I tried to find someone (to hire) to implement the system preset feature I requested in #772555 but I failed. Still Andreas Henriksson was kind enough to give it a try and sent a first patch. I tried it and found some issues so I continued to improve it and simplify it I submitted an updated patch and pinged Martin Pitt. He pointed me to the DEP-8 test failures that my patch was creating. I quickly fixed those afterwards. This patch is in use in Kali and lets us disable network services by default. I would like to see it merged in Debian so that everybody can setup systemd preset file and have their desire respected at installation time. Misc bug reports. I filed #813801 to request a new upstream release of kismet. Same for masscan in #816644 and for wkhtmltopdf in #816714. We packaged (before Debian) a new upstream release of ruby-msgpack and found out that it was not building on armel/armhf so we filed two upstream tickets (with a suggested fix). In #814805, we asked the pyscard maintainer to reinstate python-pyscard that was dropped (keeping only the Python3 version) as we use the Python 2 version in Kali. And there s more: I filed #816553 (segfault) and #816554 against cdebootstrap. I asked for dh-python to have a better behaviour after having being bitten by the fact that dh with python3 was not doing what I expected it to do (see #818175). And I reported #818907 against live-build since it is failing to handle a package whose name contains an upper case character (it s not policy compliant but dpkg supports them). Misc packaging I uploaded Django 1.9.2 to unstable and 1.8.9 to jessie-backports. I provided the supplementary information that Julien Cristau asked me in #807654 but despite this, this jessie update has been ignored for the second point release in a row. It is now outdated until I update it to include the security fixes that have been released in the mean time but I m not yet sure that I will do it the lack of cooperation of the release team for that kind of request is discouraging. I sponsored multiple uploads of dolibarr (on security update notably) and tcpdf (to fix one RC bug). Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

16 March 2016

Dirk Eddelbuettel: Rblpapi 0.3.3

A new release of Rblpapi is now on CRAN. It provides a direct interface between R and the Bloomberg Terminal via the C++ API provided by Bloomberg Labs (but note that a valid Bloomberg license and installation is required). This marks the fourth release since the package first appeared on CRAN last year. Thanks to multiple contributors, it packs a number of changes (including two new functions bsrch() and fieldInfo()), numerous internal as well as build improvements, additional documentation and last but not least use of unit testing (provided a Bloomberg connection is available, ie not at CRAN or on Travis CI). See the NEWS.Rd extract for more details:
Changes in Rblpapi version 0.3.3 (2016-03-14)
  • configure adds a missing method="libcurl" options (PR #109 by Martin Bel).
  • New function bsrch() adds basic search functionality (PR #111 by Morgan Williams fixing ticket #82).
  • The licensing status of the Rblpapi source package was clarified (PR #119).
  • The bds() help page now shows an example using an overrides argument (PR #121).
  • A new function fieldInfo was added, and field information is used in bdh and bdp (PRs #122, #123, #125, and #127).
  • The bdp function now checks the order (PR #126).
  • Unit testing was added (PRs #129 and #131).
  • Data retrieval is now smarter about the types returned from Bloomberg (PR #132 and #133; and #141 and #142).
  • The bdh and bds functions now support an option verbose (PRs #138).
Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the Rblpapi page. Questions, comments etc should go to the issue tickets system at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

10 March 2016

Lunar: Reproducible builds: week 45 in Stretch cycle

What happened in the reproducible builds effort between February 28th and March 5th:

Toolchain fixes
  • Antonio Terceiro uploaded gem2deb/0.27 that forces generated gemspecs to use the date from debian/changelog.
  • Antonio Terceiro uploaded gem2deb/0.28 that forces generated gemspecs to have their contains file lists sorted.
  • Robert Luberda uploaded ispell/3.4.00-5 which make builds of hashes reproducible.
  • C dric Boutillier uploaded ruby-ronn/0.7.3-4 which will make the output locale agnostic. Original patch by Chris Lamb.
  • Markus Koschany uploaded spring/101.0+dfsg-1. Fixed by Alexandre Detiste.
Ximin Luo resubmitted the patch adding the --clamp-mtime option to Tar on Savannah's bug tracker. Lunar rebased our experimental dpkg on top of the current master branch. Changes in the test infrastructure are required before uploading a new version to our experimental repository. Reiner Herrmann rebased our custom texlive-bin against the latest uploaded version.

Packages fixed The following 77 packages have become reproducible due to changes in their build dependencies: asciidoctor, atig, fuel-astute, jekyll, libphone-ui-shr, linkchecker, maven-plugin-testing, node-iscroll, origami-pdf, plexus-digest, pry, python-avro, python-odf, rails, ruby-actionpack-xml-parser, ruby-active-model-serializers, ruby-activerecord-session-store, ruby-api-pagination, ruby-babosa, ruby-carrierwave, ruby-classifier-reborn, ruby-compass, ruby-concurrent, ruby-configurate, ruby-crack, ruby-css-parser, ruby-cucumber-rails, ruby-delorean, ruby-encryptor, ruby-fakeweb, ruby-flexmock, ruby-fog-vsphere, ruby-gemojione, ruby-git, ruby-grack, ruby-htmlentities, ruby-jekyll-feed, ruby-json-schema, ruby-listen, ruby-markerb, ruby-mathml, ruby-mini-magick, ruby-net-telnet, ruby-omniauth-azure-oauth2, ruby-omniauth-saml, ruby-org, ruby-origin, ruby-prawn, ruby-pygments.rb, ruby-raemon, ruby-rails-deprecated-sanitizer, ruby-raindrops, ruby-rbpdf, ruby-rbvmomi, ruby-recaptcha, ruby-ref, ruby-responders, ruby-rjb, ruby-rspec-rails, ruby-rspec, ruby-rufus-scheduler, ruby-sass-rails, ruby-sass, ruby-sentry-raven, ruby-sequel-pg, ruby-sequel, ruby-settingslogic, ruby-shoulda-matchers, ruby-slack-notifier, ruby-symboltable, ruby-timers, ruby-zip, ticgit, tmuxinator, vagrant, wagon, yard. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:
  • #816209 on elog by Reiner Herrmann: use printf instead of echo which is shell-independent.
  • #816214 on python-pip by Reiner Herrmann: removes timestamp from generated Python scripts.
  • #816230 on rows by Reiner Herrmann: tell grep to always treat the input as text.
  • #816232 on eficas by Reiner Herrmann: use printf instead of echo which is shell-independent.
Florent Daigniere and bancfc reported that linux-grsec was currently built with GRKERNSEC_RANDSTRUCT which will prevent reproducible builds with the current packaging.

tests.reproducible-builds.org pbuilder has been updated to the last version to be able to support Build-Depends-Arch and Build-Conflicts-Arch. (Mattia Rizzolo, h01ger) New package sets have been added for Subgraph OS, which is based on Debian Stretch: packages and build dependencies. (h01ger) Two new armhf build nodes have been added (thanks Vagrant Cascadian) and integrated in our Jenkins setup with 8 new armhf builder jobs. (h01ger)

strip-nondeterminism development strip-nondeterminism version 0.016-1 was released on Sunday 28th. It will now normalize the POT-Creation-Date field in GNU Gettext .mo files. (Reiner Herrmann) Several improvements to the packages metadata have also been made. (h01ger, Ben Finney)

Package reviews 185 reviews have been removed, 91 added and 33 updated in the previous week. New issue: fileorder_in_gemspec_files_list. 43 FTBFS bugs were reported by Chris Lamb, Martin Michlmayr, and gregor herrmann.

Misc. After merging the patch from Dhiru Kholia adding support for SOURCE_DATE_EPOCH in rpm, Florian Festi opened a discussion on the rpm-ecosystem mailing list about reproducible builds. On March 4th, Lunar gave an overview of the general reproducible builds effort at the Internet Freedom Festival in Valencia.

5 March 2016

Lunar: Reproducible builds: week 44 in Stretch cycle

What happened in the reproducible builds effort between February 21th and February 27th:

Toolchain fixes Didier Raboud uploaded pyppd/1.0.2-4 which makes PPD generation deterministic. Emmanuel Bourg uploaded plexus-maven-plugin/1.3.8-10 which sorts the components in the components.xml files generated by the plugin. Guillem Jover has implemented stable ordering for members of the control archives in .debs. Chris Lamb submitted another patch to improve reproducibility of files generated by cython.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: dctrl-tools, debian-edu, dvdwizard, dymo-cups-drivers, ekg2, epson-inkjet-printer-escpr, expeyes, fades, foomatic-db, galternatives, gnuradio, gpodder, gutenprint icewm, invesalius, jodconverter-cli latex-mk, libiio, libimobiledevice, libmcrypt, libopendbx, lives, lttnganalyses, m2300w, microdc2, navit, po4a, ptouch-driver, pxljr, tasksel, tilda, vdr-plugin-infosatepg, xaos. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them:

tests.reproducible-builds.org The reproducibly tests for Debian now vary the provider of /bin/sh between bash and dash. (Reiner Herrmann)

diffoscope development diffoscope version 50 was released on February 27th. It adds a new comparator for PostScript files, makes the directory tests pass on slower hardware, and line ordering variations in .deb md5sums files will not be hidden anymore. Version 51 uploaded the next day re-added test data missing from the previous tarball. diffoscope is looking for a new primary maintainer.

Package reviews 87 reviews have been removed, 61 added and 43 updated in the previous week. New issues: captures_shell_variable_in_autofoo_script, varying_ordering_in_data_tar_gz_or_control_tar_gz. 30 new FTBFS have been reported by Chris Lamb, Antonio Terceiro, Aaron M. Ucko, Michael Tautschnig, and Tobias Frost.

Misc. The release team reported on their discussion about the topic of rebuilding all of Stretch to make it self-contained (in respect to reproducibility). Christian Boltz is hoping someone could talk about reproducible builds at the openSUSE conference happening June 22nd-26th in N rnberg, Germany.

4 March 2016

Mike Gabriel: My FLOSS activities in February 2016

February 2016 has been a very active month regarding me contributing to the FLOSS world. Honouring my Sponsors I am happy to share that this month's FLOSS work has been sponsored by various sponsors. Thanks to all people and companies sponsoring my work on FLOSS projects. This month's MATE uploads to Debian With regards to the Beta 1 Freeze date of Ubuntu 16.04 LTS (18th Feb 2016), Martin Wimpress, Vangelis Mouhtsis and I performed quite some work on Debian MATE. Uploads to Debian unstable: The Debian MATE Packaging Team also took over maintenance of the GTK-2+ legacy package libwnck [13]. The first upload introducing some major changes and package clean-ups caused a slight wave [14] because of a missing dependency in libwnck-dev (that fell victim to some clean-ups in debian/control). Those issues have been addressed immediately and have now been settled. The main reason for working on a legacy package like libwnck was the need for having gir1.2-wnck-1.0 (back) in Debian. The new MATE dock applet requires the libwnck GIR package to be present at runtime. One of the novelties in Ubuntu MATE 16.04 LTS will be the option to adapt the look and feel of the MATE desktop to how a Unity-based desktop looks like. Martin Wimpress is giving intense work to providing a dock applet and topmenu support as one alternative among the various Ubuntu MATE desktop experiences provided. The alternative desktop layouts can be configured with the MATE Tweak tool. Work on RDP related packages Work on FreeRDP 1.1 as currently in Debian I finally managed to give some priority (and thus time) to fixing various issues in the freerdp package in Debian [15]. Many people had provided patches and solutions to open issues and I tried to honour as many of those, as possible. Please note that I had to disable the GStreamer support in FreeRDP for the recent uploads, as the currently used Git snapshot of FreeRDP only supports GStreamer 0.10's API whereas the security team is in the process of having gstreamer0.10-* packages removed from the Debian stretch/unstable archives. Work on FreeRDP 2.0, coming to Debian soon Furthermore, Bernhard Miklautz is currently working on a freerdp2 package, which will bring the latest Git snapshot of FreeRDP upstream into Debian (and also re-introduce GStreamer support, based on GStreamer 1.0). Bernhard invested a lot of time on pushing the current HEAD of FreeRDP upstream [16] towards a FreeRDP 2.x version. Starting with FreeRDP 2.x it will be possible to install different FreeRDP versions on one system without file naming conflicts. For March 2016, I have doing the final freerdp2 reviewing on my todo list (possibly together with H ctor Or n Mart nez who is highly interested in the RDP backend support in Wayland/Weston), so that we can provide first uploads to Debian experimental sometime the coming month. The packaging progress is continuously discussed on the #freerdp channel on Freenode and can also be viewed on Github [17]. Review of revised XRDP package Recently, Dominik George from Teckids e.V. [18] contacted me about reviewing their effort of updating the Debian package xrdp, which currently is in ITA state [19]. Feedback has been provided and I am waiting for a ping from his side so that I can take some (ideally) final looks at the package and sponsor the upload. Work on Debian Edu related packages This month, I spent a couple of hours of work on several Debian Edu related tasks, some of them induced by problems at local school sites we support. Work on Debian LTS My 8h-portion of work for the Debian LTS Project, I performed at the very end of February. With the Debian squeeze LTS EOL date on 29th February, I saw to finalizing my personal open todos regarding Debian squeeze LTS, which basically was getting two CVE issues fixed in the lxc package [26]. The rest of the work hours has been spent on helping out the Security Team of Debian with open CVE issues in Debian wheezy packages: The gosa .debdiff has been approved by a member of the Security Team, the upload will happen today. With my LTS frontdesk hat on (during week 9 / 2016) I also spent some time providing help regarding SVN checkout problems and raised a couple of questions on how to coordinate the work phase between the Debian squeeze LTS EOL and the official launch of the Debian wheezy LTS project phase [27]. Work on nx-libs At the end of February, I finally managed to propose a way of dropping the libNX_Xrender.so bundled library from the nx-libs code base. I filed a PR [28] against nx-libs that proposes its removal and provides a patch for using X.Org's libXrender.so version. As a preview for nx-libs work in March 2016... I have started with removing the complete libNX_X11.so library from nx-libs and using X.Org's X11 client library. This will introduce a code removal of around 160.000 lines of code to nx-libs. More to come on this later... light+love,
Mike [1] http://ubuntu-mate.org/
[2] https://www.freexian.com/
[3] http://www.qindel.com/ [4] (caja)
https://lists.debian.org/debian-devel-changes/2016/02/msg00468.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02080.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02086.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02183.html [5] (mate-menu)
https://lists.debian.org/debian-devel-changes/2016/02/msg00469.html [6] (mate-panel)
https://lists.debian.org/debian-devel-changes/2016/02/msg01900.html [7] (mate-dock-applet)
https://lists.debian.org/debian-devel-changes/2016/02/msg01935.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02481.html
https://lists.debian.org/debian-devel-changes/2016/02/msg03097.html [8] (mate-polkit)
https://lists.debian.org/debian-devel-changes/2016/02/msg01936.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02395.html [9] (eom)
https://lists.debian.org/debian-devel-changes/2016/02/msg02073.html [10] (pluma)
https://lists.debian.org/debian-devel-changes/2016/02/msg02128.html [11] (topmenu-gtk)
https://lists.debian.org/debian-devel-changes/2016/02/msg02399.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02501.html [12] (mate-tweak)
https://lists.debian.org/debian-devel-changes/2016/02/msg03086.html [13] (libwnck)
https://lists.debian.org/debian-devel-changes/2016/02/msg01248.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01404.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01825.html [14] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814585
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814588
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814697 [15] (freerdp)
https://lists.debian.org/debian-devel-changes/2016/02/msg02487.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02630.html [16] https://github.com/FreeRDP/FreeRDP
[17] https://github.com/bmiklautz/debian-freerdp2 [18] https://www.teckids.org/ [19] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=719624 [20] (gosa)
https://lists.debian.org/debian-devel-changes/2016/02/msg01554.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01954.html [21] https://sunweavers.net/blog/node/34 [22] (ldap2zone)
https://lists.debian.org/debian-devel-changes/2016/02/msg01966.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01967.html [23] (shutdown-at-night)
https://lists.debian.org/debian-devel-changes/2016/02/msg03605.html [24] (italc)
https://lists.debian.org/debian-devel-changes/2016/02/msg01944.html [25] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=815948 [26] (lxc, Debian squeeze LTS)
https://lists.debian.org/debian-lts-changes/2016/02/msg00037.html [27] https://lists.debian.org/debian-lts/2016/02/msg00155.html
(The thread continues in March 2016) [28] https://github.com/ArcticaProject/nx-libs/pull/93

17 February 2016

Arturo Borrero Gonz lez: An update about the HA stack on Debian


Great news! The HA stack has been finally updated and you can find now both pacemaker & corosync in Debian stretch.

This is thanks to the hard work of some people, specially Christoph Berg, Ferenc W gner, Martin Loschwitz and others.

By the time of this blogpost, in testing (stretch) you have:

Additionally, unstable contains:
which is great news. However, pcs just joined Debian and there seem to be some rough edges to be worked out.
what to do now
Now that the people mentioned above did the hard work developing the packages, please do test them and report bugs.
Having a great stretch stable release (including the HA stack) is in your hands as well.

best regards!

4 February 2016

Martin-Éric Racine: xf86-video-geode 2.11.18

Yesterday, I pushed out version 2.11.18 of the Geode X.Org driver. This is the driver used by the OLPC XO-1 and by a plethora of low-power desktops, micro notebooks and thin clients. This release mostly includes maintenance fixes of all sorts. Of noticeable interest is a fix for the long-standing issue that switching between X and a VT would result in a blank screen (this should probably be cherry-picked for distributions running earlier releases of this driver). Many thanks to Connor Behan for the fix!
Unfortunately, this driver still doesn't work with GNOME. On my testing host, launching GDM produces a blank screen. 'ps' and other tools show that GDM is running but there's no screen content; the screen remains pitch black. This issue doesn't happen with other display managers e.g. LightDM. Bug reports have been filed, additional information was provided, but the issue still hasn't been resolved.
Additionally, X server flat out crashes on Geode hosts running Linux kernels 4.2 or newer. 'xkbcomp' repeatedly fails to launch and X exits with a fatal error. Bug reports have been filed, but not reacted to. However, interestingly enough, X launches fine if my testing host is booted with earliers kernels, which might suggest what the actual cause of this particular bug might be:
Since kernel 4.2 entered Debian, the base level i386 kernel on Debian is now compiled for i686 (without PAE). Until now, the base level was i586. This essentially makes it pointless to build the Geode driver with GX2 support. It also means that older GX1 hardware won't be able to run Debian either, starting with the next stable release.

1 February 2016

Mike Gabriel: My FLOSS activities in January 2016

In January 2016 I was finally able to work on various FLOSS topics again (after two months of heavily focussed local customer work): Upload of MATE 1.12 to Debian testing/unstable At the beginning of the new year, I finalized the bundle upload of MATE 1.12 to Debian unstable [1]. All uploaded packages are available in Debian testing (stretch) and Ubuntu xenial by now. MATE 1.12 will also be the version shipped in Ubuntu MATE 16.04 LTS. Additionally, I finally uploaded caja-dropbox to Debian unstable (non-free), thanks to Vangelis Mouhtsis and Martin Wimpress for doing first steps preparations. The package has already left Debian's NEW queue, but unfortunately has been removed from Debian testing (stretch) again due to build failures in one of its dependencies. Debian LTS work In January 2016 I did my first round of Debian LTS front desk work [2]. Before actually starting with my front desk duty, I worked myself through the documentation and found it difficult to understand the output of the lts-cve-triage.py script. So, I proposed various improvments to the output of that script (all committed by now). During the second week of January then, I triaged the following packages regarding known/open CVE issues: read more

31 January 2016

Martin-Éric Racine: Stupid Design: Facebook's Selling Something feature

Over these past few months, I've been going through my personal belongings to reduce them to the bare essentials more-or-less following the rule "If I cannot remember that I had it, I probably don't need it" with an explicit goal to make it easier for me to relocate in the near future.Part of that iterative process has involved selling surplus items via Facebook groups. While I initially rejoiced at the invention of the new Selling Something feature, the more I use it, the more I end up cursing at the lack of thought that went into designing that feature:For instance, in the feature's most recent implementation, it has become possible to post the same ad in several groups. Great idea, right? Actually, no. What this option does is post individual copies of that exact same ad in each group. Managed to sell a particular item? You'll have to sort through each and every duplicate post in each group to mark the item as sold. Need to edit the ad to lower the price or add requested information about the item? Same thing: go through each individual copy of the ad in each group. Seriously, Facebook? Who the fuck designed that senseless implementation?!Here's a better implementation: centrally manage all items via the user's account preferences, and let the user tick checkboxes besides each item to decide which groups will see that particular item. Ditto whenever editing the ad's content or marking an item as sold; do everything via the user's account preferences. Rather than making the content group-centric, make it user-centric.You're welcome.

24 January 2016

Lunar: Reproducible builds: week 39 in Stretch cycle

What happened in the reproducible builds effort between January 17th and January 23rd:

Toolchain fixes James McCoy uploaded subversion/1.9.3-2 which removes -Wdate-time from CPPFLAGS passed to swig enabling several packages to build again. The switch made in binutils/2.25-6 to use deterministic archives by default had the unfortunate effect of breaking a seldom used feature of make. Manoj Srivastava asked on debian-devel the best way to communicate the changes to Debian users. Lunar quickly came up with a patch that displays a warning when Make encounters deterministic archives. Manoj made it available in make/4.1-2 together with a NEWS file advertising the change. Following Guillem Jover's comment on the latest patch to make mtimes of packaged files deterministic, Daniel Kahn Gillmor updated and extended the patch adding the --clamp-mtime option to GNU Tar. Mattia Rizzolo updated texlive-bin in the reproducible experimental repository.

Packages fixed The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:

reproducible.debian.net Transition from reproducible.debian.net to the more general tests.reproducible-builds.org has started. More visual changes are coming. (h01ger) A plan on how to run tests for F-Droid has been worked out. (hc, mvdan, h01ger) A first step has been made by adding a Jenkins job to setup an F-Droid build environment. (h01ger)

diffoscope development diffoscope 46 has been released on January 19th, followed-up by version 47 made available on January 23rd. Try it online at try.diffoscope.org! The biggest visible change is the improvement to ELF file handling. Comparisons are now done section by section, using the most appropriate tool and options to get meaningful results, thanks to Dhole's work and Mike Hommey's suggestions. Also suggested by Mike, symbols for IP-relative ops are now filtered out to remove clutter. Understanding differences in ELF files belonging to Debian packages should also be much easier as diffoscope will now try to extract debug information from the matching dbgsym package. This means objdump disassembler should output line numbers for packages built with recent debhelper as long as the associated debug package is in the same directory. As diff tends to consume huge amount of memory on large inputs, diffoscope has a limit in place to prevent crashes. diffoscope used to display a difference every time the limit was hit. Because this was confusing in case there were actually no differences, a hash is now internally computed to only report a difference when one exists. Files in archives and other container members are now compared in the original order. This should not matter in most case but overall give more predictable results. Debian .buildinfo files are now supported. Amongst other minor fixes and improvements, diffoscope will now properly compare symlinks in directories. Thanks Tuomas Tynkkynen for reporting the problem.

Package reviews 70 reviews have been removed, 125 added and 33 updated in the previous week, gcc-5 amongst others. 25 FTBFS issues have been filled by Chris Lamb, Daniel Stender, Martin Michlmayr.

Misc. The 16th FOSDEM will happen in Brussels, Belgium on January 30-31st. Several talks will be about reproducible builds: h01ger about the general ecosystem, Fabian Keil about the security oriented ElectroBSD, Baptiste Daroussin about FreeBSD packages, Ludovic Court s about Guix.

Martin Michlmayr: QNAP TS-x09 installer available again

Debian 8.3 came out today. As part of this update, Debian installer images for QNAP TS-109, TS-209 and TS-409 are available again. These devices are pretty old but there are still some users. We dropped installer support several years ago because the installer ramdisk was too large to fit in flash. Since then, users had to install Debian 6.0 (squeeze) and upgrade from there. When squeeze was removed from the Debian mirrors recently, I received mail from a number of users. I investigated a bit and found out that we can bring back the installer thanks to XZ compression and some other changes. The installer is available for jessie and stretch.

17 January 2016

Lunar: Reproducible builds: week 38 in Stretch cycle

What happened in the reproducible builds effort between January 10th and January 16th:

Toolchain fixes Benjamin Drung uploaded mozilla-devscripts/0.43 which sorts the file list in preferences files. Original patch by Reiner Herrmann. Lunar submitted an updated patch series to make timestamps in packages created by dpkg deterministic. To ensure that the mtimes in data.tar are reproducible, with the patches, dpkg-deb uses the --clamp-mtime option added in tar/1.28-1 when available. An updated package has been uploaded to the experimental repository. This removed the need for a modified debhelper as all required changes for reproducibility have been merged or are now covered by dpkg.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: angband-doc, bible-kjv, cgoban, gnugo, pachi, wmpuzzle, wmweather, wmwork, xfaces, xnecview, xscavenger, xtrlock, virt-top. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Untested changes:

reproducible.debian.net Once again, Vagrant Cascadian is providing another armhf build system, allowing to run 6 more armhf builder jobs, right there. (h01ger) Stop requiring a modified debhelper and adapt to the latest dpkg experimental version by providing a predetermined identifier for the .buildinfo filename. (Mattia Rizzolo, h01ger) New X.509 certificates were set up for jenkins.debian.net and reproducible.debian.net using Let's Encrypt!. Thanks to GlobalSign for providing certificates for the last year free of charge. (h01ger)

Package reviews 131 reviews have been removed, 85 added and 32 updated in the previous week. FTBFS issues filled: 29. Thanks to Chris Lamb, Mattia Rizzolo, and Niko Tyni. New issue identified: timestamps_in_manpages_added_by_golang_cobra.

Misc. Most of the minutes from the meetings held in Athens in December 2015 are now available to the public.

11 January 2016

Norbert Preining: 10 years TeX Live in Debian

I recently dug through my history of involvement with TeX (Live), and found out that in January there are a lot of anniversaries I should celebrate: 14 years ago I started building binaries for TeX Live, 11 years ago I proposed the packaging TeX Live for Debian, 10 years ago the TeX Live packages entered Debian. There are other things to celebrate next year (2017), namely the 10 year anniversary of the (not so new anymore) infrastructure in short tlmgr of TeX Live packaging, but this will come later. In this blog post I want to concentrate on my involvement in TeX Live and Debian. TeX Live/Debian Those of you not interested in boring and melancholic look-back onto history can safely skip reading this one. For those a bit interested in the history of TeX in Debian, please read on. Debian releases and TeX systems The TeX system of choice has been for long years teTeX, curated by Thomas Esser. Digging through the Debian Archive and combining it with changelog entries as well as personal experiences since I joined Debian, here is a time line of TeX in Debian, all to my best knowledge.
Date Version Name teTeX/TeX Live Maintainers
1993-96 <1 ? ? Christoph Martin
6/1996 1.1 Buzz ?
12/1996 1.2 Rec ?
6/1997 1.3 Bo teTeX 0.4
7/1998 2.0 Ham teTeX 0.9
3/1999 2.1 Slink teTeX 0.9.9N
8/2000 2.2 Potato teTeX 1.0
7/2002 3.0 Woody teTeX 1.0
6/2005 3.1 Sarge teTeX 2.0 Atsuhito Kohda
4/2007 4.0 Etch teTeX 3.0, TeX Live 2005 Frank K ster, NP
2/2009 5.0 Lenny TeX Live 2007 NP
2/2011 6.0 Squeeze TeX Live 2009
5/2013 7.0 Whezzy TeX Live 2012
4/2015 8.0 Jessie TeX Live 2014
??? ??? Stretch TeX Live 2015
The history of TeX in Debian is thus split more or less in 10 years teTeX, and 10 years TeX Live. While I cannot check back to the origins, my guesses are that already in the very first releases (te)TeX was included. The first release I can confirm (via the Debian archive) shipping teTeX is the release Bo (June 1997). Maintainership during the first 10 years showed some fluctuation: The first years/releases (till about 2002) were dominated by Christoph Martin with Adrian Bunk and few others, who did most packaging work on teTeX version 1. After this Atsuhito Kohda with help from Hilmar Preusse and some people brought teTeX up to version 2, and from 2004 to 2007 Frank K ster, again with help of Hilmar Preusse and some other, took over most of the work on teTeX. Other names appearing throughout the changelog are (incomplete list) Julian Gilbey, Ralf Stubner, LaMont Jones, and C.M Connelly (and many more bug-reporters and fixers). Looking at the above table I have to mention the incredible amount of work that both Atsuhito Kohda and Frank K ster have put into the teTeX packages, and many of their contributions have been carried over into the TeX Live packages. While there haven t been many releases during their maintainership, their work has inspired and supported the packaging of TeX Live to a huge extend. Start of TeX Live I got involved in TeX Live back in 2002 when I started building binaries for the alpha-linux architecture. I can t remember when I first had the idea to package TeX Live for Debian, but here is a time line from my first email to the Debian Developers mailing list concerning TeX Live, to the first accepted upload:
Date Subject/Link Comment
2005-01-11 binaries for different architectures in debian packages The first question concerning packaging TeX Live, about including pre-built binaries
2005-01-25 Debian-TeXlive Proposal II A better proposal, but still including pre-built binaries
2005-05-17 Proposal for a tex-base package Proposal for tex-base, later tex-common, as basis for both teTeX and TeX live packages
2015-06-10 Bug#312897: ITP: texlive ITP bug for TeX Live
2005-09-17 Re: Take over of texinfo/info packages Taking over texinfo which was somehow orphaned started here
2005-11-28 Re: texlive-basic_2005-1_i386.changes REJECTED My answer to the rejection by ftp-master of the first upload. This email sparked a long discussion about packaging and helped improve the naming of packages (but not really the packaging itself).
2006-01-12 Upload of TeX Live 2005-1 to Debian The first successful upload
2006-01-22 Accepted texlive-base 2005-1 (source all) TeX Live packages accepted into Debian/experimental
One can see from the first emails that at that time I didn t have any idea about Debian packaging and proposed to ship the binaries built within the TeX Live system on Debian. What followed was first a long discussion about whether there is any need for just another TeX system. The then maintainer Frank K ster took a clear stance in favor of including TeX Live, and after several rounds of proposals, tests, rejections and improvements, the first successful upload of TeX Live packages to Debian/experimental happened on 12 January 2006, so exactly 10 years ago. Packaging Right from the beginning I used a meta-packaging approach. That is, instead of working directly with the source packages, I wrote (Perl) scripts that generated the source packages from a set of directives. There were several reasons why I choose to introduce this extra layer: Till now I am not 100% sure whether it was the best idea, but the scripts remain in place till now, only adapted to the new packaging paradigm in TeX Live (without xml) and adding new functionality. This allows me to just kick off one script that does do all the work, including building .orig.tar.gz, source packages, binary packages. For those interested to follow the frantic activity during the first years, there is a file CHANGES.packaging which for the years from 2005 to 2011 documents very extensively the changes I made in these years. I don t want to count the hours the went into all this Development over the years TeX Live 2005 was just another TeX system but not the preferred one in Debian Etch and beyond. But then in May 2006, Thomas Esser announced the end of development of teTeX, which cleared the path for TeX Live as main TeX system in Debian (and the world!). The next release of Debian, Lenny (1/2009), already carried only TeX Live. Unfortunately it was only TeX Live 2007 and not 2008, mostly due to me having been involved in rewriting the upstream infrastructure based on Debian package files instead of the notorious xml files. This took quite a lot of attention and time from Debian away to upstream development, but this will be discussed in a different post. Similarly, the release of TeX Live included in Debian Squeeze (released 2/2011) was only TeX Live 2009 (instead of 2010), but since then (Wheezy and Jessie) the releases of TeX Live in Debian were always the latest released ones. Current status Since about 2013 I am trying to keep a regular schedule of new TeX Live packages every month. These helps me to keep up with the changes in upstream packaging and reduces the load of packaging a new release of TeX Live. It also bring to users of unstable and testing a very up-to-date TeX system, where packages at most lack 1 month of behind the TeX Live net updates. Future As most of the readers here know, besides caring for TeX (Live) and related packages in Debian, I am also responsible for the TeX Live Manager (tlmgr) and most of upstream s infrastructure including network distribution. Thus, my (spare, outside work) time needs to be distributed between all these projects (and some others) which leaves less and less time for Debian packaging. Fortunately the packaging is in a state that making regular updates once a month is less of a burden, since most steps are automatized. What is still a bit of a struggle is adapting the binary package (src:texlive-bin) to new releases. But also this has become simpler due to less invasive changes over the years. All in all, I don t have many plans for TeX Live in Debian besides keeping the current system running as it is. Search for and advise to future maintainers and collaborators I would be more than happy if new collaborators appear, with fresh ideas and some spare time. Unfortunately, my experience over these 10 years with people showing up and proposing changes (anyone remembers the guy proposing a complete rewrite in ML or so?) is that nobody really wants to invest time and energy, but search for quick solutions. This is not something that will work with a package like TeX Live, sized of several gigabyte (the biggest in the Debian archive), and complicated inner workings. I advise everyone being interested in helping to package TeX Live for Debian, to first install normal TeX Live from TUG, get used to what actions happen during updates (format rebuilds, hyphenation patters, map file updates). One does not need to have a perfect understanding of what exactly happens down there in the guts (I didn t have in the beginning, either), but if you want to help packaging and never heard about what format dumps or map files are, then this might be a slight obstacle. Conclusion TeX Live is the only TeX system in wide use across lots of architectures and operating systems, and the only comparable system, MikTeX, is Windows specific (also there are traces of ports to Unix). Backed by all the big user groups of TeX, TeX Live will remain the prime choice for the foreseeable future, and thus also TeX Live in Debian.

8 January 2016

Dirk Eddelbuettel: AsioHeaders 1.11.0-1

Making it easier to use C++ in R has been a really nice and rewarding side effect of the work on Rcpp. One really good idea came from something Jay Emerson, Michael Kane and had I kicked about for way too long: shipping Boost headers for easier use by R and CRAN packages (such as their family of packages around bigmemory). The key idea here is headers: Experienced C++ authors such as library writers can organise C++ code in such a way that one can (almost always) get by without any linking. Which makes deployment so much easier in most use cases, and surely also with R which knows how to set an include path. So after years of "yes we really should", Jay deserves the credit for actually sitting down almost three years ago and creating the first version. So at the end of January 2013, we released BH version 1.51.0-0. By now the package is used fifty-four other CRAN packages --- clearly a much stronger uptake then we ever expected. I took over maintenance at some point, and we are currently in-line with the most recent Boost release 1.60.0. But some people were always lusting after (some) parts of Boost which also require linking. For some libraries (such as Boost.Date_Time in my RcppBDT package) you can set a #define to not require linking (and forego some string formatting or parsing we get from R anyway). Some others are trickier; Boost Regex is one example. I do not think you can use it without linking (but then C++11 helps and finally brings regular expression in the default library). Another library which the systems / network programmers at work would frequently rely upon is Boost.Asio, a cross-platform C++ library for network and low-level I/O programming. It was already used on CRAN by the iptools package by Bob Rudis and Oliver Keyes which shied away from building on Windoze (while 'doze and Boost do get along, but the CRAN system makes it a little more involved). A couple of days ago, I somehow learned that the Asio library actually comes in two flavours: the well-known Boost.Asio (requiring linking), and also as a header-only standalone library! Well, well, well -- so I ran that by Bob and Oliver asking if that would be useful. And the response was a resounding and pretty much immediate Hell, Yes!. So I wrapped it in a package, told Bob about it, who tested it and gave the thumbs up. And after a slightly longer-than-usual on-boarding, the package is now on CRAN. As network (and systems) programming is not entirely uncommon even at CRAN, I hope that this package may find a few more use cases. A new version of iptools should be forthcoming "shortly", including for the first time a Windows build thanks to this package. The indefatigable Martin Morgan already spotted our GitHub repo and scored the first fork. Comments and suggestions about AsioHeaders are welcome via the issue tracker at the GitHub GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

4 January 2016

Mike Gabriel: MATE 1.12 landed in Debian unstable

Yesterday, I did a bundle upload of all MATE 1.12 related packages to Debian unstable. Packages are currently building for the 22 architectures supported by Debian, build status can be viewed on the DDPO page of the Debian MATE Packaging Team [1] Again a big thanks to the packaging team. Martin Wimpress amongst others did a fabulous job in bumping all packages towards the 1.12 release series before the Christmas holidays. Over the holidays, I was able to review his work (99% perfect) and upload all binary packages to a staging repository. @Martin Wimpress: It is really time that we make a DM (Debian Maintainer) out of you!!! After testing all MATE 1.12 packages on a Debian unstable system, I decided to do a bundle upload yesterday. Lessons learned about bundling Debian uploads It absolutely makes sense to hold back package uploads of a project like the MATE desktop until all relevant packages are reviewed, pre-built and tested. When releasing MATE packages via the team's packaging Git [2], there are normally two actions to be taken on a package release: When reviewing so many Git projects, it is always problematic that people commit something else during the review phase. Especially, if the review work involves many packages (i.e., Git packaging repos) and requires several days or even weeks to get finished. read more

18 December 2015

Joachim Breitner: A multitude of early Christmas presents

Today was a nice day with a surprising number of early Christmas presents:

Martin Pitt: What s new in autopkgtest: LXD, MaaS, apt pinning, and more

The last two major autopkgtest releases (3.18 from November, and 3.19 fresh from yesterday) bring some new features that are worth spreading. New LXD virtualization backend 3.19 debuts the new adt-virt-lxd virtualization backend. In case you missed it, LXD is an API/CLI layer on top of LXC which introduces proper image management, seamlessly use images and containers on remote locations, intelligently caching them locally, automatically configure performant storage backends like zfs or btrfs, and just generally feels really clean and much simpler to use than the classic LXC. Setting it up is not complicated at all. Install the lxd package (possibly from the backports PPA if you are on 14.04 LTS), and add your user to the lxd group. Then you can add the standard LXD image server with 
  lxc remote add lco https://images.linuxcontainers.org:8443
and use the image to run e. g. the libpng test from the archive: 
  adt-run libpng --- lxd lco:ubuntu/trusty/i386
  adt-run libpng --- lxd lco:debian/sid/amd64
The adt-virt-lxd.1 manpage explains this in more detail, also how to use this to run tests in a container on a remote host (how cool is that!), and how to build local images with the usual autopkgtest customizations/optimizations using adt-build-lxd. I have btrfs running on my laptop, and LXD/autopkgtest automatically use that, so the performance really rocks. Kudos to St phane, Serge, Tycho, and the other LXD authors! The motivation for writing this was to make it possible to move our armhf testing into the cloud (which for $REASONS requires remote containers), but I now have a feeling that soon this will completely replace the existing adt-virt-lxc virt backend, as its much nicer to use. It is covered by the same regression tests as the LXC runner, and from the perspective of package tests that you run in it it should behave very similar to LXC. The one problem I m aware of is that autopkgtest-reboot-prepare is broken, but hardly anything is using that yet. This is a bit complicated to fix, but I expect it will be in the next few weeks. MaaS setup script While most tests are not particularly sensitive about which kind of hardware/platform they run on, low-level software like the Linux kernel, GL libraries, X.org drivers, or Mir very much are. There is a plan for extending our automatic tests to real hardware for these packages, and being able to run autopkgtests on real iron is one important piece of that puzzle. MaaS (Metal as a Service) provides just that it manages a set of machines and provides an API for installing, talking to, and releasing them. The new maas autopkgtest ssh setup script (for the adt-virt-ssh backend) brings together autopkgtest and real hardware. Once you have a MaaS setup, get your API key from the web UI, then you can run a test like this: 
  adt-run libpng --- ssh -s maas -- \
     --acquire "arch=amd64 tags=touchscreen" -r wily \
     http://my.maas.server/MAAS 123DEADBEEF:APIkey
The required arguments are the MaaS URL and the API key. Without any further options you will get any available machine installed with the default release. But usually you want to select a particular one by architecture and/or tags, and install a particular distro release, which you can do with the -r/--release and --acquire options. Note that this is not wired into Ubuntu s production CI environment, but it will be. Selectively using packages from -proposed Up until a few weeks ago, autopkgtest runs in the CI environment were always seeing/using the entirety of -proposed. This often led to lockups where an application foo and one of its dependencies libbar got a new version in -proposed at the same time, and on test regressions it was not clear at all whose fault it was. This often led to perfectly good packages being stuck in -proposed for a long time, and a lot of manual investigation about root causes. . These days we are using a more fine-grained approach: A test run is now specific for a trigger , that is, the new package in -proposed (e. g. a new version of libbar) that caused the test (e. g. for foo ) to run. autopkgtest sets up apt pinning so that only the binary packages for the trigger come from -proposed, the rest from -release. This provides much better isolation between the mush of often hundreds of packages that get synced or uploaded every day. This new behaviour is controlled by an extension of the --apt-pocket option. So you can say 
  adt-run --apt-pocket=proposed=src:foo,libbar1,libbar-data ...
and then only the binaries from the foo source, libbar1, and libbar-data will come from -proposed, everything else from -release. Caveat:Unfortunately apt s pinning is rather limited. As soon as any of the explicitly listed packages depends on a package or version that is only available in -proposed, apt falls over and refuses the installation instead of taking the required dependencies from -proposed as well. In that case, adt-run falls back to the previous behaviour of using no pinning at all. (This unfortunately got worse with apt 1.1, bug report to be done). But it s still helpful in many cases that don t involve library transitions or other package sets that need to land in lockstep. Unified testbed setup script There is a number of changes that need to be made to testbeds so that tests can run with maximum performance (like running dpkg through eatmydata, disabling apt translations, or automatically using the host s apt-cacher-ng), reliable apt sources, and in a minimal environment (to detect missing dependencies and avoid interference from unrelated services these days the standard cloud images have a lot of unnecessary fat). There is also a choice whether to apply these only once (every day) to an autopkgtest specific base image, or on the fly to the current ephemeral testbed for every test run (via --setup-commands). Over time this led to quite a lot of code duplication between adt-setup-vm, adt-build-lxc, the new adt-build-lxd, cloud-vm-setup, and create-nova-image-new-release. I now cleaned this up, and there is now just a single setup-commands/setup-testbed script which works for all kinds of testbeds (LXC, LXD, QEMU images, cloud instances) and both for preparing an image with adt-buildvm-ubuntu-cloud, adt-build-lx[cd] or nova, and with preparing just the current ephemeral testbed via --setup-commands. While this is mostly an internal refactorization, it does impact users who previously used the adt-setup-vm script for e. g. building Debian images with vmdebootstrap. This script is now gone, and the generic setup-testbed entirely replaces it. Misc Aside from the above, every new version has a handful of bug fixes and minor improvements, see the git log for details. As always, if you are interested in helping out or contributing a new feature, don t hesitate to contact me or file a bug report.

17 December 2015

Steinar H. Gunderson: sRGB weirdness: Doing the right thing causes a worse result

A while back, I wrote about how you should always do image calculations in linear light, not gamma space; since then, Tom Forsythe has come out with a much better metaphor than I could cough up myself, namely that you should look at sRGB value as compressed values, not integers. So naturally, when I needed a deinterlacing filter for Movit, my GPU filter library, I wanted to do it in linear light. (In fact, all pixel processing in Movit is in linear light, except in the cases where it's 100% equivalent to do it on the gamma-encoded values and the conversion can be skipped for speed.) After some deliberations, I made an implementation of Martin Weston's three-field deinterlacing filter, known in ffmpeg as w3fdif. I won't discuss deinterlacing in detail here since it's really hard, but I'll note that w3fdif works by way of applying two filters; low-frequency components are estimated from the current field, and high-frequency from the previous and next fields. (This makes intuitive sense; you cannot get the HF information from the current field since you don't have the lines you need for that, but you can hope it hasn't changed too much.) Aha! A filter. Brilliant, that's exactly when linear light means the most, too. But when implementing it, I found that it sometimes looked weird -- and ffmpeg's implementation (which works directly on the sRGB values, which we already established is wrong) didn't. After lots of tweaking back and forth, I decided to set up a synthetic test to settle this once and for all; I took a static test picture (eliminating everything related to video capture, codecs, frame rates, etc.) and compared to ffmpeg. Of course, deinterlacing is all about movement, but this would do to try to nail things down. So after lots of fruitless debugging, I did a last-ditch: What if I turned off the gamma conversions? This gave me a huge surprise; indeed it looked better! I'll provide some upscaled versions; left is the original image, middle is deinterlaced in sRGB space and right is deinterlaced in linear light: Original picture Deinterlaced in sRGB space Deinterlaced in linear light If that's not dramatic enough for you (trust me, you'll notice it when it's animated as you flicker through the two different fields), here's an even more high-contrast example (same ordering): Original picture Deinterlaced in sRGB space Deinterlaced in linear light I guess it's obvious in retrospect what happens; the HF filter picks up residue from its outer edges, and even if the coefficient is just 0.031 (well, times two; it adds that value from both the previous and next field), 3% the photons of a fully lit pixel (which is what you get when working in linear light) is actually quite a bit, whereas a 3% gray is only pixel value 8 or so, which is barely visible. So what am I to make of this? I'm honestly not sure. Maybe it's somehow related to that these filter values were chosen in 1988, where they were relatively unlikely to do this in linear light (although if they did it with analog circuitry, perhaps they could?) and it was tweaked to look good despite doing the wrong thing. Or maybe I need to change my approach here entirely. It always sucks when your fundamental assumptions are challenged, but I think it shows once again that if you notice something funny in your output, you really ought to investigate, because you never know how deep the rabbit hole goes. :-/

Next.

Previous.