Search Results: "Martin"

22 June 2015

Lunar: Reproducible builds: week 8 in Stretch cycle

What happened about the reproducible builds effort this week: Toolchain fixes Andreas Henriksson has improved Johannes Schauer initial patch for pbuilder adding support for build profiles. Packages fixed The following 12 packages became reproducible due to changes in their build dependencies: collabtive, eric, file-rc, form-history-control, freehep-chartableconverter-plugin , jenkins-winstone, junit, librelaxng-datatype-java, libwildmagic, lightbeam, puppet-lint, tabble. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: reproducible.debian.net Bugs with the ftbfs usertag are now visible on the bug graphs. This explain the recent spike. (h01ger) Andreas Beckmann suggested a way to test building packages using the funny paths that one can get when they contain the full Debian package version string. debbindiff development Lunar started an important refactoring introducing abstactions for containers and files in order to make file type identification more flexible, enabling fuzzy matching, and allowing parallel processing. Documentation update Ximin Luo detailed the proposal to standardize environment variables to pass a reference source date to tools that needs one (e.g. documentation generator). Package reviews 41 obsolete reviews have been removed, 168 added and 36 updated this week. Some more issues affecting packages failing to build from source have been identified. Meetings Minutes have been posted for Tuesday June 16th meeting. The next meeting is scheduled Tuesday June 23rd at 17:00 UTC. Presentations Lunar presented the project in French during Pas Sage en Seine in Paris. Video and slides are available.

16 June 2015

Martin Pitt: autopkgtest 3.14 now twice as rebooty

Almost every new autopkgtest release brings some small improvements, but 3.14 got some reboot related changes worth pointing out. First of all, I simplified and unified the implementation of rebooting across all runners that support it (ssh, lxc, and qemu). If you use a custom setup script for adt-virt-ssh you might have to update it: Previously, the setup script needed to respond to a reboot function to trigger a reboot, wait for the testbed to go down, and come back up. This got split into issuing the actual reboot system command directly by adt-run itself on the testbed, and the wait for go down and back up part. The latter now has a sensible default implementation: it simply waits for the ssh port to become unavailable, and then waits for ssh to respond again; most testbeds should be fine with that. You only need to provide the new wait-reboot function in your ssh setup script if you need to do anything else (such as re-enabling ssh after reboot). Please consult the manpage and the updated SKELETON for details. The ssh runner gained a new --reboot option to indicate that the remote testbed can be rebooted. This will automatically declare the reboot testbed capability and thus you can now run rebooting tests without having to use a setup script. This is very useful for running tests on real iron. Finally, in testbeds which support rebooting your tests will now find a new /tmp/autopkgtest-reboot-prepare command. Like /tmp/autopkgtest-reboot it takes an arbitrary marker , saves the current state, restores it after reboot and re-starts your test with the marker; however, it will not trigger the actual reboot but expects the test to do that. This is useful if you want to test a piece of software which does a reboot as part of its operation, such as a system-image upgrade. Another use case is testing kernel crashes, kexec or another nonstandard way of rebooting the testbed. README.package-tests shows an example how this looks like. 3.14 is now available in Debian unstable and Ubuntu wily. As usual, for older releases you can just grab the deb and install it, it works on all supported Debian and Ubuntu releases. Enjoy, and let me know if you run into troubles or have questions!

15 June 2015

Lunar: Reproducible builds: week 7 in Stretch cycle

What happened about the reproducible builds effort for this week: Presentations On June 7th, Reiner Herrmann presented the project at the Gulaschprogrammiernacht 15 in Karlsruhe, Germany. Video and audio recordings in German are available, and so are the slides in English. Toolchain fixes Daniel Kahn Gillmor's report on help2man started a discussion with Brendan O'Dea and Ximin Luo about standardizing a common environment variable that would provide a replacement for an embedded build date. After various proposals and research by Ximin about date handling in several programming languages, the best solution seems to define SOURCE_DATE_EPOCH with a value suitable for gmtime(3).
  1. Martin Borgert wondered if Sphinx could be changed in a way that would avoid having to tweak debian/rules in packages using it to produce HTML documentation.
Daniel Kahn Gillmor opened a new report about icont producing unreproducible binaries. Packages fixed The following 32 packages became reproducible due to changes in their build dependencies: agda, alex, c2hs, clutter-1.0, colorediffs-extension, cpphs, darcs-monitor, dispmua, haskell-curl, haskell-glfw, haskell-glib, haskell-gluraw, haskell-glut, haskell-gnutls, haskell-gsasl, haskell-hfuse, haskell-hledger-interest, haskell-hslua, haskell-hsqml, haskell-hssyck, haskell-libxml-sax, haskell-openglraw, haskell-readline, haskell-terminfo, haskell-x11, jarjar-maven-plugin, kxml2, libcgi-struct-xs-perl, libobject-id-perl, maven-docck-plugin, parboiled, pegdown. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which did not make their way to the archive yet: reproducible.debian.net A new variation to better notice when a package captures the environment has been introduced. (h01ger) The test on Debian packages works by building the package twice in a short time frame. But sometimes, a mirror push can happen between the first and the second build, resulting in a package built in a different build environment. This situation is now properly detected and will run a third build automatically. (h01ger) OpenWrt, the distribution specialized in embedded devices like small routers, is now being tested for reproducibility. The situation looks very good for their packages which seems mostly affected by timestamps in the tarball. System images will require more work on debbindiff to be better understood. (h01ger) debbindiff development Reiner Herrmann added support for decompling Java .class file and .ipk package files (used by OpenWrt). This is now available in version 22 released on 2015-06-14. Documentation update Stephen Kitt documented the new --insert-timestamp available since binutils-mingw-w64 version 6.2 available to insert a ready-made date in PE binaries built with mingw-w64. Package reviews 195 obsolete reviews have been removed, 65 added and 126 updated this week. New identified issues: Misc. Holger Levsen reported an issue with the locales-all package that Provides: locales but is actually missing some of the files provided by locales. Coreboot upstream has been quick to react after the announcement of the tests set up the week before. Patrick Georgi has fixed all issues in a couple of days and all Coreboot images are now reproducible (without a payload). SeaBIOS is one of the most frequently used payload on PC hardware and can now be made reproducible too. Paul Kocialkowski wrote to the mailing list asking for help on getting U-Boot tested for reproducibility. Lunar had a chat with maintainers of Open Build Service to better understand the difference between their system and what we are doing for Debian.

3 June 2015

Raphaël Hertzog: My Free Software Activities in May 2015

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 10.25 hours on Debian LTS. In that time I did the following: Other Debian work Package Tracker. The Debian system administrators upgraded the machine hosting tracker.debian.org to jessie and I dealt with the fallout. Fixing the Apache configuration was easy but DACS also broke and I had to disable it (thus breaking login via sso.debian.org). Fortunately Enrico Zini and Martin Zobel-Helas debugged the problem and restored it. Sponsorship. I sponsored a dolibarr upload and many tryton-modules-* uploads to bring Tryton 3.6 to Debian (and granted DM rights on the newly introduced packages to Matthias Behrle who is maintaining those packages). Misc stuff. I discussed multiple feature requests with Dmitry Smirnov for dh-linktree. Packaging. I uploaded a new upstream version of cpputest. I did that twice actually because the first version had failing tests (see #784674). I also filed #784959 on blhc because I saw what looked like a false positive report for a missing hardening flag. I uploaded Django 1.8 to experimental. This is a major upstream release and shall ideally only be uploaded to sid after having reported problems on reverse dependencies. I doubt we will have the time to do this I started working on Publican 4.3.0 but the test suite fails and it s not even the fault of publican for once. It s a bug in libxml apparently. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

20 May 2015

Martin-Éric Racine: xf86-video-geode 2.11.17

This morning, I pushed out version 2.11.17 of the Geode X.Org driver. This is the driver used by the OLPC XO-1 and by a plethora of low-power desktops, micro notebooks and thin clients. This is a minor release. It merges conditional support for the OpenBSD MSR device (Marc Ballmer, Matthieu Herrb), fixes a condition that prevents compiling on some embedded platforms (Brian A. Lloyd) and upgrades the code for X server 1.17 compatibility (Maarten Lankhorst).
Pending issues:

4 May 2015

Lunar: Reproducible builds: first week in Stretch cycle

Debian Jessie has been released on April 25th, 2015. This has opened the Stretch development cycle. Reactions to the idea of making Debian build reproducibly have been pretty enthusiastic. As the pace is now likely to be even faster, let's see if we can keep everyone up-to-date on the developments. Before the release of Jessie The story goes back a long way but a formal announcement to the project has only been sent in February 2015. Since then, too much work has happened to make a complete report, but to give some highlights: Lunar did a pretty improvised lightning talk during the Mini-DebConf in Lyon. This past week It seems changes were pilling behind the curtains given the amount of activity that happened in just one week. Toolchain fixes We also rebased the experimental version of debhelper twice to merge the latest set of changes. Lunar submitted a patch to add a -creation-date to genisoimage. Reiner Herrmann opened #783938 to request making -notimestamp the default behavior for javadoc. Juan Picca submitted a patch to add a --use-date flag to texi2html. Packages fixed The following packages became reproducible due to changes of their build dependencies: apport, batctl, cil, commons-math3, devscripts, disruptor, ehcache, ftphs, gtk2hs-buildtools, haskell-abstract-deque, haskell-abstract-par, haskell-acid-state, haskell-adjunctions, haskell-aeson, haskell-aeson-pretty, haskell-alut, haskell-ansi-terminal, haskell-async, haskell-attoparsec, haskell-augeas, haskell-auto-update, haskell-binary-conduit, haskell-hscurses, jsch, ledgersmb, libapache2-mod-auth-mellon, libarchive-tar-wrapper-perl, libbusiness-onlinepayment-payflowpro-perl, libcapture-tiny-perl, libchi-perl, libcommons-codec-java, libconfig-model-itself-perl, libconfig-model-tester-perl, libcpan-perl-releases-perl, libcrypt-unixcrypt-perl, libdatetime-timezone-perl, libdbd-firebird-perl, libdbix-class-resultset-recursiveupdate-perl, libdbix-profile-perl, libdevel-cover-perl, libdevel-ptkdb-perl, libfile-tail-perl, libfinance-quote-perl, libformat-human-bytes-perl, libgtk2-perl, libhibernate-validator-java, libimage-exiftool-perl, libjson-perl, liblinux-prctl-perl, liblog-any-perl, libmail-imapclient-perl, libmocked-perl, libmodule-build-xsutil-perl, libmodule-extractuse-perl, libmodule-signature-perl, libmoosex-simpleconfig-perl, libmoox-handlesvia-perl, libnet-frame-layer-ipv6-perl, libnet-openssh-perl, libnumber-format-perl, libobject-id-perl, libpackage-pkg-perl, libpdf-fdf-simple-perl, libpod-webserver-perl, libpoe-component-pubsub-perl, libregexp-grammars-perl, libreply-perl, libscalar-defer-perl, libsereal-encoder-perl, libspreadsheet-read-perl, libspring-java, libsql-abstract-more-perl, libsvn-class-perl, libtemplate-plugin-gravatar-perl, libterm-progressbar-perl, libterm-shellui-perl, libtest-dir-perl, libtest-log4perl-perl, libtext-context-eitherside-perl, libtime-warp-perl, libtree-simple-perl, libwww-shorten-simple-perl, libwx-perl-processstream-perl, libxml-filter-xslt-perl, libxml-writer-string-perl, libyaml-tiny-perl, mupen64plus-core, nmap, openssl, pkg-perl-tools, quodlibet, r-cran-rjags, r-cran-rjson, r-cran-sn, r-cran-statmod, ruby-nokogiri, sezpoz, skksearch, slurm-llnl, stellarium. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which did not make their way to the archive yet: Improvements to reproducible.debian.net Mattia Rizzolo has been working on compressing logs using gzip to save disk space. The web server would uncompress them on-the-fly for clients which does not accept gzip content. Mattia Rizzolo worked on a new page listing various breakage: missing or bad debbindiff output, missing build logs, unavailable build dependencies. Holger Levsen added a new execution environment to run debbindiff using dependencies from testing. This is required for packages built with GHC as the compiler only understands interfaces built by the same version. debbindiff development Version 17 has been uploaded to unstable. It now supports comparing ISO9660 images, dictzip files and should compare identical files much faster. Documentation update Various small updates and fixes to the pages about PDF produced by LaTeX, DVI produced by LaTeX, static libraries, Javadoc, PE binaries, and Epydoc. Package reviews Known issues have been tagged when known to be deterministic as some might unfortunately not show up on every single build. For example, two new issues have been identified by building with one timezone in April and one in May. RD and help2man add current month and year to the documentation they are producing. 1162 packages have been removed and 774 have been added in the past week. Most of them are the work of proper automated investigation done by Chris West. Summer of code Finally, we learned that both akira and Dhole were accepted for this Google Summer of Code. Let's welcome them! They have until May 25th before coding officialy begins. Now is the good time to help them feel more comfortable by sharing all these little bits of knowledge on how Debian works.

1 May 2015

Keith Packard: the machine architecture

The Machine Architecture Here's a brief introduction to some of the hardware concepts within The Machine. As the team at HP that I'm working with are busy working on Linux kernel changes motivated by the hardware, I'm hoping that providing this kind of documentation will help Linux kernel developers outside of HP evaluate that work, and work by others in related areas. Joining HP and learning about The Machine In January, I joined HP to work on Linux for The Machine. I'd watched Martin Fink's video and read other articles on the new hardware coming out of HP labs. I had hints of what they were up to, and the possibilities seemed exciting enough to entice me to go back to HP. When I arrived at HP, one of the first things I got to read was the external reference specification for The Machine. 170 pages detailing a more significant shift in computer architecture than I had been given any hints of, both in my interviews at HP and from what I could see in the press. Since then, I've been eager to tell people about what we're doing, and I'm happy to say that we're finally ready to start the conversation with this brief description. A Short Outline of Storage within The Machine The basic unit of The Machine is a collection of hardware grouped in a Load Store Domain. A Load Store Domain consists of: Here's a diagram of how the various bits of the hardware are hooked together: And, a brief description of the elements within the picture: I've intentionally drawn the shared memory in a large box to emphasize the notion that this machine is more "memory-centric" and less "processor-centric". The shared byte-addressable persistent memory forms the sole persistent storage within The Machine. More to Come I'll continue to publish information about The Machine and our related Linux work as we work on the hardware and software.

5 April 2015

Dominique Dumont: Improving creation of debian copyright file

Hello In my opinion, creating and maintaining Debian copyright file is the most boring task required to create a Debian package. Unfortunately, this file is also one of the most important of a package: it specifies some legal aspect regarding the use of the software. Debian copyright file is scrutinized by ftp masters gatekeepers when accepting a new package in Debian project: this file must accurately describe the copyright and licenses of all files of a source package, preferably using a specific syntax. (Kudos to the ftp-masters team: reading copyright files must be even more boring than writing them). The content of the copyright file must reflect accurately the license of all files. This license is often specified in the comments of a source files. The licencecheck command is able to scan sources files and reports the copyright and licenses declared in there. But it does not summarize this information: a copyright line is generated for each file of a package. licensecheck2dep5 (provided by cdbs package as /usr/lib/cdbs/licensecheck2dep5) does better: the output of licensecheck is consolidated and translated in Debian copyright format. The result is better, but must be heavily edited to be reviewable by ftp-masters team. The new update subcommand of cme (available with libconfig-model-dpkg-perl 2.061 currently in experimental) goes further than licensecheck2deb: For instance, here s the (slightly edited) output of cme run for pan package starting without debian/copyright file:
$ cme update dpkg-copyright -quiet
Adding dummy license text for license public-domain 
for path pan/general/sorted-vector.h
Adding dummy license text for license BSD-2-clause 
for path pan/usenet-utils/MersenneTwister.h
$ cat debian/copyright
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Files: *
Copyright: 1994-2001, Frank Pilhofer. The author may
License: GPL-2+
Files: pan/*
Copyright: 2002-2007, Charles Kerr 
License: GPL-2
Files: pan/data/cert-store.cc
  pan/data/cert-store.h
Copyright: 2011, Heinrich M ller 
  2002-2006, Charles Kerr 
License: GPL-2
Files: pan/general/e-util.cc
  pan/general/e-util.h
Copyright: 2000, 2001, Ximian, Inc
License: LGPL-2
Files: pan/general/locking.h
  pan/general/worker-pool.cc
  pan/general/worker-pool.h
Copyright: 2007, Calin Culianu 
  2002-2007, Charles Kerr 
License: LGPL-2+
Files: pan/general/sorted-vector.h
Copyright: 2002, Martin Holzherr (holzherr@infobrain.com).
License: public-domain
 Please fill license public-domain from header
 of pan/general/sorted-vector.h
[ about 100 more lines including license text for Zlib and several 
  GPL licenses ]
This is a good start, but some modifications must be applied to get a correct license file: These modifications can be done: This post has mentioned creation of Debian copyright file, but does not address the problem of updating an existing copyright file when packaging a new version of a software. This will be the subject of a next post. I hope this new feature of cme will save hours of work for Debian packagers. As usual comments and suggestions are welcome All the best

24 March 2015

DebConf team: Working towards a child-friendly DebConf (Posted by Martin Krafft)

The Debian Project will celebrate its 22nd birthday during DebConf15 in Heidelberg in August 2015. At this age, it s unsurprising that children of Debian contributors have attended our developer conference for several years. Going with the times, we would like to work further towards making DebConf15 a child-friendly (parents-friendly) conference. The conference venue is far away from traffic, self-contained, and there is a dedicated children s play room. There are green areas around, and the Heidelberg Zoo is literally within sight of the venue. We haven t yet discussed deals with them, but we could. In short: if you d like to attend DebConf, but you are yet unsure what to do with your children bring your kids along! The hostel has a number of 3 and 4 bed-rooms with en-suite bathrooms, plus a good supply of cots available for the very little ones. We will allocate such rooms to families exclusively for your privacy (subject to availability, so please register yourself ASAP, and include a note about your kids). We are maintaining answers to commonly-asked questions on the wiki. Please let us know if anything is missing, and feel free to update the page yourself. We would also like to explore additional possibilities to make it easier for parents to participate in the conference. At the moment, we re still scouting for ideas and there are already a number of promising leads. To help us figure out what we d best offer, we need to know about the demand. If you are planning to bring your children, or if you re thinking about it, please drop a short note with number and ages and any other relevant information to kids@debconf.org. Your mail will be read by a few parents involved in the organisation of DebConf15 and we will obviously keep your data private. We also created a (publicly archived) mailing list to discuss options and keep people updated on our plans. Please subscribe yourself to the list, if interested, and feel free to write to debconf-kids@lists.debian.org with any questions or ideas you might have.

23 March 2015

Martin-Éric Racine: This and That

I haven't blogged anything in months and figured that now might be a good time to get around that. Here it goes:
Free Software
While I occasionally upgrade the packaging of the software I maintain at Debian to keep up with best practices, my activity downsizing goes on. Simply put: I never had any ambition to become a Debian Developer. My involvement has always remained pragmatic and mostly from the perspective of packaging software that I found useful. Even then, my motivation for doing that keeps on dwindling into nothingness, because key pieces of software keep on breaking, whenever someone upstream decides to reinvent the wheel.
For instance, GNOME no longer works at all on Geode chipsets and it barely works on Nouveau chipsets. This happened as soon as GNOME 3.14 was uploaded into unstable, right before the freeze started. Then again, I wouldn't jump to a conclusion that GNOME itself might be at fault, since Plymouth also stopped working on the same two video platforms at the same time. For all we know, this could be caused by some changes in the X.Org server code. Bugs were filed, additional information was provided, but no fix has taken place.
Given how Geode and Nouveau represent 80% of my hardware investment (my Intel laptop being the sole exception), it essentially means that the upcoming Debian "stable" is useless for me. Now try and remain motivated, even just as a mere Free Software end-user. At this point, I'm done.
Politics
Finland is holding national elections this April. I still have no idea who I'll vote for this time. The guy I voted for last time has become a career politician with an inflated ego and zero connection to the average Finn's aspirations and worries. Meanwhile, two friends are standing as candidates: one who is a razor-sharp fact finder and who is a proven pragmatic decision-maker, but whose values are slightly off with mine, and one whose actions come straight from the heart but whose concept of today's Finnish reality leaves a lot to be desired.
National Defence
There's been a lot of recent articles about how former hardware and locations of the Finnish defence forces and border guards have been sold, often for peanuts, to Russian interests. In some cases, we're only talking about buildings formerly used for on-site staff accommodations. In other cases, former patrol boats and navy harbours changed hands. Now, to top it all, it appears that our north-western neighbour, Norway, has sold a former submarine base to German investors who, in turn, leased it to you guessed it Russian interests.
Looking at Russian actions in Ukraine, I cannot help but feel great concern that strategic locations are falling into potentially dangerous hands. Just seeing the picture of a former navy harbour with a handful of patrol boats on standby, right on the Finnish coastline, half-way between Helsinki and Turku, was a sobering experience. While the whole idea of shooting at people even invading armies gives me the creeps, at this point, I cannot help but start pondering whether defending this country might in fact be an occupation worth training for.
Employment
It has now been 6 years since I held my last dayjob. Since then, the only thing I've found is an unpaid training in the national bureaucracy. I've also freelanced as an actor and model, but that barely brought me pocket change, if even that. Seeing my face on posters advertising a movie I participated in last year was indeed nice, getting some media attention in connection to that too, but it hasn't lead to additional gigs. As far as I can tell, this was just my Warholian 15 minutes of fame.
However, there's a larger issue at stake. Newspapers recently published an employment statistics map for Nordic countries and the truth couldn't be more bleak: while Norway and Sweden's employment figures are nearly spotless for almost every province, those of Finland are save for a couple of mildly successful provinces outright catastrophic. Given this and despite feeling relatively happy living in Finland and having developed a will to defend this country from an eventual Russian assault, I've come to the conclusion that I would be better off going West, with a strong preference for Norway.
Now, the main question is, doing what? 6 years later, I have strong doubts that I would be remotely considered for any high-tech job. Besides, come to think of it, I wouldn't want any new office job. Off the top of my head, my idea of a cool job that would allow me to stay physically fit would be working as a tourist guide in Lapland. However, if Norway is anything like Finland, someone probably needs a dozen of permits of all sorts (first aid certification, C or even D class driving license, college degree in tourism, etc.) that I cannot afford. What then?

22 February 2015

Zlatan Todori : Loosing yourself

First they came for the Socialists, and I did not speak out Because I was not a Socialist.
Then they came for the Trade Unionists, and I did not speak out
Because I was not a Trade Unionist.
Then they came for the Jews, and I did not speak out
Because I was not a Jew.
Then they came for me and there was no one left to speak for me.
Martin Niem ller

13 February 2015

Jonathan Carter: Debconf 2016 to be hosted in Cape Town

Table Mountain, an icon of Cape Town Long story short, we put in a bid to host Debconf 16 in Cape Town, and we got it! Back at Debconf 12 (Nicaragua), many people asked me when we re hosting a Debconf in South Africa. I just laughed and said Who knows, maybe some day . During the conference I talked to Stefano Rivera (tumbleweed) who said that many people asked him too. We came to the conclusion that we d both really really want to do it but just didn t have enough time at that stage. I wanted to get to a point where I could take 6 months off for it and suggested that we prepare a bid for 2019. Stefano thought that this was quite funny, I think at some point we managed to get that estimate down to 2017-2018. That date crept back even more with great people like Allison Randal and Bernelle Verster joining our team, along with other locals Graham Inggs, Raoul Snyman, Adrianna Pi ska, Nigel Kukard, Simon Cross, Marc Welz, Neill Muller, Jan Groenewald, and our international mentors such as Nattie Mayer-Hutchings, Martin Krafft and Hannes von Haugwitz. Now, we re having that Debconf next year. It s almost hard to believe, not sure how I ll sleep tonight, we ve waited so long for this and we ve got a mountain of work ahead of us, but we ve got a strong team and I think Debconf 2016 attendees are in for a treat! Since I happened to live close to Montr al back in 2012, I supported the idea of a Debconf bid for Montr al first, and then for Cape Town afterwards. Little did I know then that the two cities would be the only two cities bidding against each other 3 years later. I think both cities are superb locations to host a Debconf, and I m supporting Montr al s bid for 2017. Want to get involved? We have a mailing list and IRC channel: #debconf16-capetown on oftc. Thanks again for all the great support from everyone involved so far!

4 February 2015

Vincent Bernat: Directory bookmarks with Zsh

There are numerous projects to implement directory bookmarks in your favorite shell. An inherent limitation of those implementations is they being only an enhanced cd command: you cannot use a bookmark in an arbitrary command. UPDATED: My initial implementation with Zsh was using dynamic named directories. I have been pointed on Twitter that there is a simpler way to implement bookmarks. The article has been updated to reflect that. As a side note, it is also possible to just use shell variables1. Zsh comes with a not well-known feature called static named directories. They are declared with the hash builtin and can be refered by prepending ~ to them:
$ hash -d -- -lldpd=/home/bernat/code/deezer/lldpd
$ echo ~-lldpd/README.md
/home/bernat/code/deezer/lldpd/README.md
$ head -n1 ~-lldpd/README.md
lldpd: implementation of IEEE 802.1ab (LLDP)
Because ~-lldpd is substituted during file name expansion, it is possible to use it in any command like a regular directory, like shown above. The - prefix is only here to avoid collision with home directories. Bookmarks are kept into a dedicated directory, $MARKPATH. Each bookmark is a symbolic link to the target directory: for example, ~-lldpd should be expanded to $MARKPATH/lldpd which points to the appropriate directory. Assuming that you have populated $MARKPATH with some links, here is how bookmarks are registered:
for link ($MARKPATH/*(N@))  
    hash -d -- -$ link:t =$ link:A 
 
You also automatically get completion and prompt expansion:
$ pwd
/home/bernat/code/deezer/lldpd/src/lib
$ echo $ (%):-%~ 
~-lldpd/src/lib
The last step is to manage bookmarks without adding or removing symbolic links manually. The following bookmark() function will display the existing bookmarks when called without arguments, will remove a bookmark when called with -d or add the current directory as a bookmark otherwise.
bookmark()  
    if (( $# == 0 )); then
        # When no arguments are provided, just display existing
        # bookmarks
        for link in $MARKPATH/*(N@); do
            local markname="$fg[green]$ link:t $reset_color"
            local markpath="$fg[blue]$ link:A $reset_color"
            printf "%-30s -> %s\n" $markname $markpath
        done
    else
        # Otherwise, we may want to add a bookmark or delete an
        # existing one.
        local -a delete
        zparseopts -D d=delete
        if (( $+delete[1] )); then
            # With  -d , we delete an existing bookmark
            command rm $MARKPATH/$1
        else
            # Otherwise, add a bookmark to the current
            # directory. The first argument is the bookmark
            # name.  .  is special and means the bookmark should
            # be named after the current directory.
            local name=$1
            [ $name == "." ] && name=$ PWD:t 
            ln -s $PWD $MARKPATH/$name
        fi
    fi
 
Find the complete version on GitHub.

Dynamic named directories Another (more complex) way to achieve the same thing is using dynamic named directories. I was initially using this solution but it is far more complex. This section is only here for historical reason. You can find the complete implementation in my GitHub repository. During file name expansion, a ~ followed by a string in square brackets is provided to the zsh_directory_name() function which will eventually reply with a directory name. This feature can be used to implement directory bookmarks:
$ cd ~[@lldpd]
$ pwd
/home/bernat/code/deezer/lldpd
$ echo ~[@lldpd]/README.md
/home/bernat/code/deezer/lldpd/README.md
$ head -n1 ~[@lldpd]/README.md
lldpd: implementation of IEEE 802.1ab (LLDP)
Like for static named directories, because ~[@lldpd] is substituted during file name expansion, it is possible to use it in any command like a regular directory.

Basic implementation Bookmarks are still kept into a dedicated directory, $MARKPATH and are still symbolic links. Here is how the core feature is implemented:
_bookmark_directory_name()  
    emulate -L zsh #  
    setopt extendedglob
    case $1 in
        n)
            [ [ $2 != ( #b)"@"(?*) ]] && return 1 #  
            typeset -ga reply
            reply=($ $ :-$MARKPATH/$match[1] :A ) #  
            return 0
            ;;
        *)
            return 1
            ;;
    esac
    return 0
 
add-zsh-hook zsh_directory_name _bookmark_directory_name
zsh_directory_name() is a function accepting hooks2: instead of defining it directly, we define another function and register it as a hook with add-zsh-hook. The hook is expected to handle different situations. The first one is to be able to transform a dynamic name into a regular directory name. In this case, the first parameter of the function is n and the second one is the dynamic name. In , the call to emulate will restore the pristine behaviour of Zsh and also ensure that any option set in the scope of the function will not have an impact outside. The function can then be reused safely in another environment. In , we check that the dynamic name starts with @ followed by at least one character. Otherwise, we declare we don t know how to handle it. Another hook will get the chance to do something. (#b) is a globbing flag. It activates backreferences for parenthesised groups. When a match is found, it is stored as an array, $match. In , we build the reply. We could have just returned $MARKPATH/$match[1] but to hide the symbolic link mechanism, we use the A modifier to ask Zsh to resolve symbolic links if possible. Zsh allows nested substitutions. It is therefore possible to use modifiers and flags on anything. $ :-$MARKPATH/$match[1] is a common trick to turn $MARKPATH/$match[1] into a parameter substitution and be able to apply the A modifier on it.

Completion Zsh is also able to ask for completion of a dynamic directory name. In this case, the completion system calls the hook function with c as the first argument.
_bookmark_directory_name()  
    # [...]
    case $1 in
        c)
            # Completion
            local expl
            local -a dirs
            dirs=($MARKPATH/*(N@:t)) #  
            dirs=("@"$ ^dirs ) #  
            _wanted dynamic-dirs expl 'bookmarked directory' compadd -S\] -a dirs
            return
            ;;
        # [...]
    esac
    # [...]
 
First, in , we create a list of possible bookmarks. In *(N@:t), N@ is a glob qualifier. N allows us to not return nothing if there is no match (otherwise, we would get an error) while @ only returns symbolic links. t is a modifier which will remove all leading pathname components. This is equivalent to use basename or $ something##*/ in POSIX shells but it plays nice with glob expressions. In , we just add @ before each bookmark name. If we have b1, b2 and b3 as bookmarks, $ ^dirs expands to b1,b2,b3 and therefore "@"$ ^dirs expands to the (@b1 @b2 @b3) array. The result is then feeded into the completion system.

Prompt expansion Many people put the name of the current directory in their prompt. It would be nice to have the bookmark name instead of the full name when we are below a bookmarked directory. That s also possible!
$ pwd
/home/bernat/code/deezer/lldpd/src/lib
$ echo $ (%):-%~ 
~[@lldpd]/src/lib
The prompt expansion system calls the hook function with d as first argument and the file name to transform.
_bookmark_directory_name()  
    # [...]
    case $1 in
        d)
            local link slink
            local -A links
            for link ($MARKPATH/*(N@))  
                links[$  #link:A $'\0'$ link:A ]=$ link:t  #  
             
            for slink ($ (@On)$ (k)links )  
                link=$ slink#*$'\0'  #  
                if [ [ $2 = ( #b)($ link )( /*) ]]; then
                    typeset -ga reply
                    reply=("@"$ links[$slink]  $(( $  #match[1]  )) )
                    return 0
                fi
             
            return 1
            ;;
        # [...]
    esac
    # [...]
 
OK. This is some black Zsh wizardry. Feel free to skip the explanation. This is a bit complex because we want to substitute the most specific bookmark, hence the need to sort bookmarks by their target lengths. In , the associative array $links is created by iterating on each symbolic link ($link) in the $MARKPATH directory. The goal is to map a target directory with the matching bookmark name. However, we need to iterate on this map from the longest to the shortest key. To achieve that, we prepend each key with its length. Remember, $ link:A is the absolute path with symbolic links resolved. So, $ #link:A is the length of this path. We concatenate the length of the target directory with the target directory name and use $'\0' as a separator because this is the only safe character for this purpose. The result is mapped to the bookmark name. The second loop is an iteration on the keys of the associative array $links (thanks to the use of the k parameter flag in $ (k)links ). Those keys are turned into an array (@ parameter flag) and sorted numerically in descending order (On parameter flag). Since the keys are directory names prefixed by their lengths, the first match will be the longest one. In , we extract the directory name from the key by removing the length and the null character at the beginning. Then, we check if the extracted directory name matches the file name we have been provided. Again, (#b) just activates backreferences. With extended globbing, we can use the or operator, . So, when either the file name matches exactly the directory name or is somewhere deeper, we create the reply which is an array whose first member is the bookmark name and the second member is the untranslated part of the file name.

Easy typing Typing ~[@ is cumbersome. Hopefully, Zsh line editor can be extended with additional bindings. The following snippet will substitute @@ (if typed without a pause) by ~[@:
vbe-insert-bookmark()  
    emulate -L zsh
    LBUFFER=$ LBUFFER "~[@"
 
zle -N vbe-insert-bookmark
bindkey '@@' vbe-insert-bookmark
In combination with the autocd option and completion, it is quite easy to jump to a bookmarked directory.

  1. For example:
    $ lldpd=/home/bernat/code/deezer/lldpd
    $ echo $lldpd/README.md
    /home/bernat/code/deezer/lldpd/README.md
    $ head -n1 $lldpd/README.md
    lldpd: implementation of IEEE 802.1ab (LLDP)
    
    The drawback is that you don t have a separate namespace for your bookmarks. You can still use a special prefix for that. Also, no prompt expansion.
  2. Other functions accepting hooks are chpwd() or precmd().

24 January 2015

Dirk Eddelbuettel: Rcpp 0.11.4

A new release 0.11.4 of Rcpp is now on the CRAN network for GNU R, and an updated Debian package will be uploaded in due course. Rcpp has become the most popular way of enhancing GNU R with C++ code. As of today, 323 packages on CRAN depend on Rcpp for making analyses go faster and further; BioConductor adds another 41 packages, and casual searches on GitHub suggests dozens mores. This release once again adds a large number of small bug fixes, polishes and enhancements. And like the last time, these changes were made by a group of seven different contributors (counting code commits) plus three more providing concrete suggestions. This shows that the Rcpp development and maintenance rests a large number of (broad) shoulders. See below for a detailed list of changes extracted from the NEWS file.
Changes in Rcpp version 0.11.4 (2015-01-20)
  • Changes in Rcpp API:
    • The ListOf<T> class gains the .attr and .names methods common to other Rcpp vectors.
    • The [dpq]nbinom_mu() scalar functions are now available via the R:: namespace when R 3.1.2 or newer is used.
    • Add an additional test for AIX before attempting to include execinfo.h.
    • Rcpp::stop now supports improved printf-like syntax using the small tinyformat header-only library (following a similar implementation in Rcpp11)
    • Pairlist objects are now protected via an additional Shield<> as suggested by Martin Morgan on the rcpp-devel list.
    • Sorting is now prohibited at compile time for objects of type List, RawVector and ExpressionVector.
    • Vectors now have a Vector::const_iterator that is 'const correct' thanks to fix by Romain following a bug report in rcpp-devel by Martyn Plummer.
    • The mean() sugar function now uses a more robust two-pass method, and new unit tests for mean() were added at the same time.
    • The mean() and var() functions now support all core vector types.
    • The setequal() sugar function has been corrected via suggestion by Qiang Kou following a bug report by S ren H jsgaard.
    • The macros major, minor, and makedev no longer leak in from the (Linux) system header sys/sysmacros.h.
    • The push_front() string function was corrected.
  • Changes in Rcpp Attributes:
    • Only look for plugins in the package's namespace (rather than entire search path).
    • Also scan header files for definitions of functions to be considerd by Attributes.
    • Correct the regular expression for source files which are scanned.
  • Changes in Rcpp unit tests
    • Added a new binary test which will load a pre-built package to ensure that the Application Binary Interface (ABI) did not change; this test will (mostly or) only run at Travis where we have reasonable control over the platform running the test and can provide a binary.
    • New unit tests for sugar functions mean, setequal and var were added as noted above.
  • Changes in Rcpp Examples:
    • For the (old) examples ConvolveBenchmarks and OpenMP, the respective Makefile was renamed to GNUmakefile to please R CMD check as well as the CRAN Maintainers.
Thanks to CRANberries, you can also look at a diff to the previous release As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

15 January 2015

Tim Retout: Docker London Meetup - January 2015

Last week, I visited London for the January Docker meetup, which was the first time I'd attended this group. It was a talk-oriented format, with around 200 attendees packed into Shoreditch Village Hall; free pizza and beer was provided thanks to the sponsors, which was awesome (and makes logistics easier when you're travelling there from work). There were three talks. First, Andrew Martin from British Gas spoke about how they use Docker for testing and continuous deployment of their Node.js microservices - buzzword bingo! But it's helpful to see how companies approach these things. Second, Johan Euphrosine from Google gave a short demo of Google Cloud Platform for running Docker containers (mostly around Container Engine, but also briefly App Engine). This was relevant to my interests, but I'd already seen this sort of talk online. Third, Dan Williams presented his holiday photos featuring a journey on a container ship, which wins points from me for liberal interpretation of the meetup topic, and was genuinely very entertaining/interesting - I just regret having to leave to catch a train halfway through. In summary, this was worth attending, but as someone just getting started with containers I'd love some sort of smaller meetings with opportunities for interaction/activity. There's such a variety of people/use cases for Docker that I'm not sure how much everyone had in common with each other; it would be interesting to find out.

22 December 2014

Michael Prokop: Ten years of Grml

* On 22nd of October 2004 an event called OS04 took place in Seifenfabrik Graz/Austria and it marked the first official release of the Grml project. Grml was initially started by myself in 2003 I registered the domain on September 16, 2003 (so technically it would be 11 years already :)). It started with a boot-disk, first created by hand and then based on yard. On 4th of October 2004 we had a first presentation of grml 0.09 Codename Bughunter at Kunstlabor in Graz. I managed to talk a good friend and fellow student Martin Hecher into joining me. Soon after Michael Gebetsroither and Andreas Gredler joined and throughout the upcoming years further team members (Nico Golde, Daniel K. Gebhart, Mario Lang, Gerfried Fuchs, Matthias Kopfermann, Wolfgang Scheicher, Julius Plenz, Tobias Klauser, Marcel Wichern, Alexander Wirt, Timo Boettcher, Ulrich Dangel, Frank Terbeck, Alexander Steinb ck, Christian Hofstaedtler) and contributors (Hermann Thomas, Andreas Krennmair, Sven Guckes, Jogi Hofm ller, Moritz Augsburger, ) joined our efforts. Back in those days most efforts went into hardware detection, loading and setting up the according drivers and configurations, packaging software and fighting bugs with lots of reboots (working on our custom /linuxrc for the initrd wasn t always fun). Throughout the years virtualization became more broadly available, which is especially great for most of the testing you need to do when working on your own (meta) distribution. Once upon a time udev became available and solved most of the hardware detection issues for us. Nowadays X.org doesn t even need a xorg.conf file anymore (at least by default). We have to acknowledge that Linux grew up over the years quite a bit (and I m wondering how we ll look back at the systemd discussions in a few years). By having Debian Developers within the team we managed to push quite some work of us back to Debian (the distribution Grml was and still is based on), years before the Debian Derivatives initiative appeared. We never stopped contributing to Debian though and we also still benefit from the Debian Derivatives initiative, like sharing issues and ideas on DebConf meetings. On 28th of May 2009 I myself became an official Debian Developer. Over the years we moved from private self-hosted infrastructure to company-sponsored systems, migrated from Subversion (brr) to Mercurial (2006) to Git (2008). Our Zsh-related work became widely known as grml-zshrc. jenkins.grml.org managed to become a continuous integration/deployment/delivery home e.g. for the dpkg, fai, initramfs-tools, screen and zsh Debian packages. The underlying software for creating Debian packages in a CI/CD way became its own project known as jenkins-debian-glue in August 2011. In 2006 I started grml-debootstrap, which grew into a reliable method for installing plain Debian (nowadays even supporting installation as VM, and one of my customers does tens of deployments per day with grml-debootstrap in a fully automated fashion). So one of the biggest achievements of Grml is from my point of view that it managed to grow several active and successful sub-projects under its umbrella. Nowadays the Grml team consists of 3 Debian Developers Alexander Wirt (formorer), Evgeni Golov (Zhenech) and myself. We couldn t talk Frank Terbeck (ft) into becoming a DM/DD (yet?), but he s an active part of our Grml team nonetheless and does a terrific job with maintaining grml-zshrc as well as helping out in Debian s Zsh packaging (and being a Zsh upstream committer at the same time makes all of that even better :)). My personal conclusion for 10 years of Grml? Back in the days when I was a student Grml was my main personal pet and hobby. Grml grew into an open source project which wasn t known just in Graz/Austria, but especially throughout the German system administration scene. Since 2008 I m working self-employed and mainly working on open source stuff, so I m kind of living a dream, which I didn t even have when I started with Grml in 2003. Nowadays with running my own business and having my own family it s getting harder for me to consider it still a hobby though, instead it s more integrated and part of my business which I personally consider both good and bad at the same time (for various reasons). Thanks so much to anyone of you, who was (and possibly still is) part of the Grml journey! Let s hope for another 10 successful years! Thanks to Max Amanshauser and Christian Hofstaedtler for reading drafts of this.

15 December 2014

Gustavo Noronha Silva: Web Engines Hackfest 2014

For the 6th year in a row, Igalia has organized a hackfest focused on web engines. The 5 years before this one were actually focused on the GTK+ port of WebKit, but the number of web engines that matter to us as Free Software developers and consultancies has grown, and so has the scope of the hackfest. It was a very productive and exciting event. It has already been covered by Manuel Rego, Philippe Normand, Sebastian Dr ge and Andy Wingo! I am sure more blog posts will pop up. We had Martin Robinson telling us about the new Servo engine that Mozilla has been developing as a proof of concept for both Rust as a language for building big, complex products and for doing layout in parallel. Andy gave us a very good summary of where JS engines are in terms of performance and features. We had talks about CSS grid layouts, TyGL a GL-powered implementation of the 2D painting backend in WebKit, the new Wayland port, announced by Zan Dobersek, and a lot more. With help from my colleague ChangSeok OH, I presented a description of how a team at Collabora led by Marco Barisione made the combination of WebKitGTK+ and GNOME s web browser a pretty good experience for the Raspberry Pi. It took a not so small amount of both pragmatic limitations and hacks to get to a multi-tab browser that can play youtube videos and be quite responsive, but we were very happy with how well WebKitGTK+ worked as a base for that. One of my main goals for the hackfest was to help drive features that were lingering in the bug tracker for WebKitGTK+. I picked up a patch that had gone through a number of iterations and rewrites: the HTML5 notifications support, and with help from Carlos Garcia, managed to finish it and land it at the last day of the hackfest! It provides new signals that can be used to authorize notifications, show and close them. To make notifications work in the best case scenario, the only thing that the API user needs to do is handle the permission request, since we provide a default implementation for the show and close signals that uses libnotify if it is available when building WebKitGTK+. Originally our intention was to use GNotification for the default implementation of those signals in WebKitGTK+, but it turned out to be a pain to use for our purposes. GNotification is tied to GApplication. This allows for some interesting features, like notifications being persistent and able to reactivate the application, but those make no sense in our current use case, although that may change once service workers become a thing. It can also be a bit problematic given we are a library and thus have no GApplication of our own. That was easily overcome by using the default GApplication of the process for notifications, though. The show stopper for us using GNotification was the way GNOME Shell currently deals with notifications sent using this mechanism. It will look for a .desktop file named after the application ID used to initialize the GApplication instance and reject the notification if it cannot find that. Besides making this a pain to test our test browser would need a .desktop file to be installed, that would not work for our main API user! The application ID used for all Web instances is org.gnome.Epiphany at the moment, and that is not the same as any of the desktop files used either by the main browser or by the web apps created with it. For the future we will probably move Epiphany towards this new era, and all users of the WebKitGTK+ API as well, but the strictness of GNOME Shell would hurt the usefulness of our default implementation right now, so we decided to stick to libnotify for the time being. Other than that, I managed to review a bunch of patches during the hackfest, and took part in many interesting discussions regarding the next steps for GNOME Web and the GTK+ and Wayland ports of WebKit, such as the potential introduction of a threaded compositor, which is pretty exciting. We also tried to have Bastien Nocera as a guest participant for one of our sessions, but it turns out that requires more than a notebook on top of a bench hooked up to a TV to work well. We could think of something next time ;D. I d like to thank Igalia for organizing and sponsoring the event, Collabora for sponsoring and sending ChangSeok and myself over to Spain from far away Brazil and South Korea, and Adobe for also sponsoring the event! Hope to see you all next year!
Web Engines Hackfest 2014 sponsors: Adobe, Collabora and Igalia

Web Engines Hackfest 2014 sponsors: Adobe, Collabora and Igalia

2 December 2014

DebConf team: Sponsors for DebConf15 (Posted by Laura Arjona Reina)

The Debian project recently welcomed the first nine sponsors for Debconf15: credativ and sipgate as Gold sponsors, Google, Farsight Security, Inc., Martin Alfke / Buero 2.0 and Ubuntu as silver sponsors, and Logilab, Netways and Hetzner as Bronze-level. But things have kept moving! The Matanel Foundation just committed to support DebConf15 as Gold-level sponsor, and Mirantis as Silver-level. So, at the time of writing this, we have 11 sponsors. We are very thankful for that! The DebConf team aims to organize the Debian Conference as a self-sustaining event, despite its size and complexity (the Final Report for DebConf14 gives a good indication). The financial contributions by individuals, companies and organizations, as well as the support by our infrastructure sponsors, media partners, and volunteers, are pivotal to our success. There are many different possibilities to support DebConf. If you know any company or organization that could be interested, don t hesitate to point them at our sponsorship brochure (also available in German) or tell us about any leads.

Martin-&#201;ric Racine: GNOME is destroying the whole GTK universe

When GNOME 3.14 components were uploaded into Debian (just a few days before the Jessie freeze started), GNOME stopped working on two video platforms: Geode (xf86-video-geode) and NVIDIA (xf86-video-nouveau). On Geode, GDM launches into a black screen. On NVIDIA, GDM launches as expected, but then the GNOME session itself barfs with the dreaded "Oh No! Something went wrong. [Logout]" dialog during session initialization. Basically, components in GNOME have become too tightly dependent upon some video driver features. Thinking out loud, I figured that reverting to a desktop environment that is based upon GTK+3, without GNOME's bells and whistles, would at least restore operation on my NVDIA hosts. Alas, it does not: Cinnamon, too, barfs during session initialization. Great. Now what?

18 November 2014

Christian Perrier: Bug #770000

Martin Pitt reported Debian bug #770000 on Tuesday November 18th, against the release.debian.org pseudo-package. Bug #760000 was reported as of August 30th: so there have been 10,000 bugs reported in 3 months minus 12 days. The bug rate increased quite significantly during the last weeks. We can suspect this is related to the release and the freeze (that triggers many unblock requests) I find it interesting that this bug is directly related to the release, directly related to systemd and originated from one of the systemd packages maintainers, if I'm right. So, I'll take this opportunity to publicly thank all people who have brought the systemd packages to what they are now, whether or not they're still maintaining the package. We've all witnessed that Debian if facing a strong social issue nowadays and I'm very deeply sad about this. I hope we'll be able to go through this without losing too many brilliant contributors, as it happened recently. Please prove me right and do The Right Thing for me to be able to continue this silly "round bug number" contest and still believe that, some day, bug #1000000 will really happen and I'm still there to witness it. Ah, and by the way, systemd bloody works on my system. I can't even remember when I switched to it. It Just Worked.

Next.

Previous.