Search Results: "Mike Gabriel"

6 July 2016

Mike Gabriel: [Arctica Project] Release of nx-libs (version 3.5.99.0)

Introduction NX is a software suite which implements very efficient compression of the X11 protocol. This increases performance when using X applications over a network, especially a slow one. NX (v3) has been originally developed by NoMachine and has been Free Software ever since. Since NoMachine obsoleted NX (v3) some time back in 2013/2014, the maintenance has been continued by a versatile group of developers. The work on NX (v3) is being continued under the project name "nx-libs". History Until January 2015, nx-libs was more mainly a redistribution approach of the original NX (v3) software. We (we as in mainly a group of X2Go developers) kept changes applied to the original NoMachine sources as minimal as possible. We also kept the original files and folders structure. Patches had been maintained via the quilt utility on top of a Git repository, the patches had always been kept separate. That was the 3.5.0.x series of nx-libs. In January 2015, the characteristics of nx-libs as maintained by the X2Go project between 2011 and 2015 changed. A decision was reached: This effort is now about to be released as "nx-libs 3.6.0.0". Contributors Since 2011, the nx-libs code base has to a great extent been maintained in the context of the X2Go Project [1]. Qindel Group joining in... In 2014, developers from the Qindel Group [2] joined the nx-libs maintenance. They found X2Go's work on nx-libs and suggested joining forces as best as possible on hacking nx-libs. The Arctica Project comming up... Starting in January 2015, the development on the 3.6.x branch of the project was moved into a new project called the Arctica Project [3]. Development Funding by Qindel In September 2015, a funding project was set up at Qindel. Since then, the Qindel group greatly supports the development of nx-libs 3.6.x monetarily and with provided man power. The funding project officially is a cooperation between Qindel and DAS-NETZWERKTEAM (business run by Mike Gabriel, long term maintainer of nx-libs). The funding is split into two subprojects and lasts until August 2017: The current nx-libs development effort is coordinated in the context of the Arctica Project (by Mike Gabriel), with use cases in Arctica, X2Go and TheQVD (VDI product worked on at Qindel) in mind. People involved There are various people involved in nx-libs 3.6.x maintenance and development, some of them shall be explicitly named here (in alphabetical order of surnames): Some other people have contributed, but have left the project already. Thanks for your work on nx-libs: A big thanks to everyone involved!!! Special thanks go to Stefan Baur for employing Mihai Moldovan and handling all the bureaucracy, so that Mihai can work on this project and get funded for his work. Achievements of nx-libs 3.5.99.0 We are very close to our self-defined release goal 3.6.0. The below tasks have already been (+/-) completed for version 3.5.99.0: In a previous blog post [4], the code reduction in nx-libs has already been discussed. With this announcement, we want to give a status update about our effort of shrinking the nx-libs code base:
    [mike@minobo nx-libs (3.6.x)]$ cloc --match-f '.*\.(c cpp h)' .
        1896 text files.
        1896 unique files.                                          
        7430 files ignored.
    http://cloc.sourceforge.net v 1.60  T=5.88 s (307.3 files/s, 143310.5 lines/s)
    -------------------------------------------------------------------------------
    Language                     files          blank        comment           code
    -------------------------------------------------------------------------------
    C                              958          68574          74891         419638
    C/C++ Header                   730          25683          33957         130418
    C++                            120          17007          11721          61292
    -------------------------------------------------------------------------------
    SUM:                          1808         111264         120569         611348
    -------------------------------------------------------------------------------
The previous statistics had these sums in the last line. First the nx-libs 3.5.0.x code tree (where we came from):
    -------------------------------------------------------------------------------
    SUM:                          5614         329279         382337        1757743
    -------------------------------------------------------------------------------
Then the nx-libs 3.6.x status as it was on May 9th 2016:
    -------------------------------------------------------------------------------
    SUM:                          1922         118581         126783         662635
    -------------------------------------------------------------------------------
Another 50.000 lines of code have been removed over the past two months. Work pending for the final 3.6.0 release goal Known Issues There are several open issues on the nx-libs Github project space, see https://github.com/ArcticaProject/nx-libs/issues. Testing the nx-libs 3.5.99.0 We are currently working on provisioning release builds and nightly builds of nx-libs for various recent Linux distributions. Please stay tuned and watch Mike Gabriel's blog [5]. We already have nightly builds of nx-libs for Debian and Ubuntu [6], but there are more to come soon. Until now, please use the build recipes provided in the README.md file of the nx-libs source tree [7]. References

27 May 2016

Mike Gabriel: MATE 1.14 landing in Debian unstable...

I just did a bundle upload of all MATE 1.14 related packages to Debian unstable. Packages are currently building for the 23 architectures supported by Debian, build status can be viewed on the DDPO page of the Debian MATE Packaging Team [1] Credits Again a big thanks to the packaging team. Martin Wimpress again did a fabulous job in bumping all packages towards the 1.14 release series during the last weeks. During last week, I reviewed his work and uploaded all binary packages to a staging repository. Also a big thanks to Vangelis Mouhtsis, who recently added more hardening support to all those MATE packages that do some sort of C compilation at build time. After testing all MATE 1.14 packages on a Debian unstable system, I decided to do a bundle upload today. Packages should be falling out of the build daemons within the next couple of hours/days (depending on the architecture being built for). GTK2 -> GTK3 The greatest change for this release of MATE to Debian is the switch over from GTK2 to GTK3. People using the MATE desktop environment on Debian systems are invited to test the new MATE 1.14 packages and give feedback via the Debian bug tracker, esp. on the user experience regarding the switch over to GTK3. Thanks to all who help getting MATE 1.14 in Debian better every day!!! Known issues when running in NXv3 sessions The new GTK3 build of MATE works fine locally (against local X.org server). However, it causes some trouble (i.e. graphical glitches) when running in an NXv3 based remote desktop session. Those issues have to be addressed by me (while wearing my NXv3 upstream hat), I guess (sigh...). light+love,
Mike [1] https://qa.debian.org/developer.php?login=pkg-mate-team@lists.alioth.deb...

24 May 2016

Mike Gabriel: Arctica Project: The Telekinesis Framework, coming soon...

The Arctica Project is a task force of people reinventing the realm of remote desktop computing on Linux. One core component for multimedia experience in remote desktop / application scenarios is the to-be-reloaded / upcoming Telekinesis Framework. Telekinesis provides a framework for developing GUI applications that have a client and server side component. Those applications are visually merged and presented to the end user in such a way that the end user's user experience is the same as if the user was interacting with a strictly server side application. Telekinesis mediates the communication between those server side and client side application parts. As a reference implementation you can imagine a server side media player GUI (TeKi-aware application) and a client side video overlay (corresponding TeKi-aware service). The media player GUI "remote-controls" the client side video overlay. The video overlay receives its video stream from the server. All these interactions are mediated through Telekinesis. A proof of concept has been developed for X2Go in 2012. For the Arctica Server, we are currently doing a (much cleaner!) rewrite of the original prototype [1]. See [2] for the first whitepaper describing how to integrate Telekinesis into existing remote desktop solutions. See [3] for a visual demonstration of the potentials of Telekinesis (still using X2Go underneath and the original Telekinesis prototype). The heavy lifting around Telekinesis development and conceptual design is performed by my project partner Lee from GZ Nianguan FOSS Team [4]. Thanks for continuously giving your time and energy into the co-founding of the Arctica Project. Thanks for always reminding me of doing benchmarks!!! light+love,
Mike [1] http://code.x2go.org/gitweb?p=telekinesis.git;a=summary
[2] https://github.com/ArcticaProject/ArcticaDocs/blob/master/Telekinesis/Te...
[3] https://www.youtube.com/watch?v=57AuYOxXPRU
[4] https://github.com/gznget

17 May 2016

Mike Gabriel: NXv3 Rebase: Build nxagent against X.org 7.0

As already hinted in my previous blog post, here comes a short howto that explains how to test-build nxagent (v3) against a modularized X.org 7.0 source tree. WARNING: Please note that mixing NX code and X.org code partially turns the original X.org code base into GPL-2 code. We are aware of this situation and work on moving all NXv3 related GPL-2 code into the nxagent DDX code (xserver-xorg/hw/nxagent) or--if possible--dropping it completely. The result shall be a range of patches against X.org (licensable under the same license as the respective X.org files) and a GPL-2 licensed DDX (i.e. nxagent). How to build this project For the Brave and Playful
$ git clone https://git.arctica-project.org/nx-X11-rebase/build.git .
$ bash populate.sh sources.lst
$ ./buildit.sh
You can find the built tree in the _install/ sub-directory. Please note that cloning Git repositories over the https protocol can be considerably slow. If you want to speed things up, consider signing up with our GitLab server. For Developers... ... who have registered with our GitLab server.
$ git clone git@git.arctica-project.org:nx-X11-rebase/build.git .
$ bash populate.sh sources-devs.lst
$ ./buildit.sh
You will find the built tree in the _install/ sub-directory. The related git repositories are in the repos/ sub-directory. All repos modified for NX have been cloned from the Arctica Project's GitLab server via SSH. Thus, you as a developer can commit changes on those repos and push back your changes to the GitLab server. Required tools for building Debian/Ubuntu and alike In a one-liner command:
$ sudo apt-get install build-essential automake gawk git pkg-config libtool libz-dev libjpeg-dev libpng-dev
Fedora If someone tries this out in a clean Fedora chroot environment, please let us know about build dependent packages. openSUSE If someone tries this out in a clean openSUSE chroot environment, please let us know about build dependent packages. Testing the built nxagent and nxproxy The tests/ subdir contains some scripts which can be used to test the compile results. Notes on required X.org changes (NX_MODIFICATIONS) For this build workflow to work, we (i.e. mostly Ulrich Sibiller) had to work several NoMachine patches into original X.org 7.0 code. Here is a list of modified X11 components with URLs pointing to the branch containing those changes:
xkbdata                            xorg/data/xkbdata                       rebasenx  1.0.1     https://git.arctica-project.org/nx-X11-rebase/xkbdata.git
libfontenc                         xorg/lib/libfontenc                     rebasenx  1.0.1     https://git.arctica-project.org/nx-X11-rebase/libfontenc.git
libSM                              xorg/lib/libSM                          rebasenx  1.0.0     https://git.arctica-project.org/nx-X11-rebase/libSM.git
libX11                             xorg/lib/libX11                         rebasenx  1.0.0     https://git.arctica-project.org/nx-X11-rebase/libX11.git
libXau                             xorg/lib/libXau                         rebasenx  1.0.0     https://git.arctica-project.org/nx-X11-rebase/libXau.git
libXfont                           xorg/lib/libXfont                       rebasenx  1.3.1     https://git.arctica-project.org/nx-X11-rebase/libXfont.git
libXrender                         xorg/lib/libXrender                     rebasenx  0.9.0.2   https://git.arctica-project.org/nx-X11-rebase/libXrender.git
xtrans                             xorg/lib/libxtrans                      rebasenx  1.0.0     https://git.arctica-project.org/nx-X11-rebase/libxtrans.git
kbproto                            xorg/proto/kbproto                      rebasenx  1.0.2     https://git.arctica-project.org/nx-X11-rebase/kbproto.git
xproto                             xorg/proto/xproto                       rebasenx  7.0.4     https://git.arctica-project.org/nx-X11-rebase/xproto.git
xorg-server                        xorg/xserver                            rebasenx  1.0.1     https://git.arctica-project.org/nx-X11-rebase/xserver.git
mesa                               mesa/mesa                               rebasenx  6.4.1     https://git.arctica-project.org/nx-X11-rebase/mesa.git
Credits Nearly all of this has been achieved by Ulrich Sibiller. Thanks a lot for giving your time and energy to that. As the rebasing of NXv3 is currently a funded project supported by the Qindel Group, we are currently negotiating ways of monetarily appreciating Ulrich's intensive work on this. Thanks a lot, once more!!! Feedback If anyone of you feels like trying out the test build as described above, please consider signing up with the Arctica Project's GitLab server and reporting your issues there directly (against the repository nx-X11-rebase/build). Alternatively, feel free to contact us on IRC (Freenode): #arctica or subscribe to our developers' mailing list. Thank you. light+love
Mike Gabriel

9 May 2016

Mike Gabriel: Recent progress in NXv3 development

This is to give a comprehensive overview on the recent progress made in NXv3 (aka nx-libs) development. The upstream sources of nx-libs can be found at / viewed on / cloned from Github:
https://github.com/ArcticaProject/nx-libs A great portion of the current work is sponsored by the Qindel Group [1] in Spain (QINDEL FORMACI N Y SERVICIOS, S.L.). Thanks for making this possible. Planned release date: 2nd July, 2016 We aim at releasing a completely tidied up nx-libs code tree versioned 3.6.0 on July 2nd, 2016. There is still a whole bunch of work to do for this, but I am positive that we can make this release date. Goals of our Efforts There are basically two major goals for spending a considerable amount of time, money and energy on NXv3 hacking: The efforts undertaken always have the various existing use cases in mind (esp. the framework of the coming-up Arctica Project, TheQVD and X2Go). Overview on Recent Development Progress General Code Cleanups Making this beast maintainable means first of all: identifying code redundancies, unused code passages, etc. and remove them. This is where we came from (NoMachine's NX 3.5.x, including nxcomp, nxcompext, nxcompshad, nx-X11 and nxagent): 1,757,743 lines of C/C++ code.
[mike@minobo nx-libs.35 (3.5.0.x)]$ cloc --match-f '.*\.(c cpp h)$' .
    5624 text files.
    5614 unique files.                                          
    2701 files ignored.
http://cloc.sourceforge.net v 1.60  T=18.59 s (302.0 files/s, 132847.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                             3134         231180         252893        1326393
C/C++ Header                  2274          78062         116132         349743
C++                            206          20037          13312          81607
-------------------------------------------------------------------------------
SUM:                          5614         329279         382337        1757743
-------------------------------------------------------------------------------
On the current 3.6.x branch of nx-libs (at commit 6c6b6b9), this is where we are now: 662,635 lines of C/C++ code, amount of code reduced to a third of the original code lines.
[mike@minobo nx-libs (3.6.x)]$ cloc --match-f '.*\.(c cpp h)' .
    2012 text files.
    2011 unique files.                                          
    1898 files ignored.
http://cloc.sourceforge.net v 1.60  T=5.63 s (341.5 files/s, 161351.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                             1015          74605          81625         463244
C/C++ Header                   785          26992          34354         138063
C++                            122          16984          10804          61328
-------------------------------------------------------------------------------
SUM:                          1922         118581         126783         662635
-------------------------------------------------------------------------------
The latest development branch currently has these statistics: 619,353 lines of C/C++ code, another 40,000 lines could be dropped.
[mike@minobo nx-libs (pr/libnx-xext-drop-unused-extensions)]$ cloc --match-f '.*\.(c cpp h)' .
    1932 text files.
    1931 unique files.                                          
    1898 files ignored.
http://cloc.sourceforge.net v 1.60  T=5.66 s (325.4 files/s, 150598.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                              983          69474          77186         426564
C/C++ Header                   738          25616          33048         131599
C++                            121          16984          10802          61190
-------------------------------------------------------------------------------
SUM:                          1842         112074         121036         619353
-------------------------------------------------------------------------------
Dropping various libNX_X* shared libraries (and using X.org shared client libraries instead) At first, various bundled libraries could be dropped from the nx-X11 code tree. Secondly, several of the bundled X.org libraries could be dropped, because we managed to build against those libraries as provided system-wide. Then, and this undertaking is much trickier, we could drop nearly all Xlib extension libraries that are used by nxagent with its role of being an X11 client. We could sucessfully drop these Xlib extension libraries from nx-X11, because we managed to build nxagent against the matching libraries in X.org: libNX_Xdmcp, libNX_Xfixes, libNX_XComposite, libNX_Xdamage, libNX_Xtst, libNX_Xinerama, libNX_Xfont, libNX_xkbui, and various others. All these droppings happened without a loss of functionality. However, some shared X client libraries are not easy to remove without loss of functionality, or rather not removable at all. Dropping libNX_Xrender We recently dropped libNX_Xrender [2] and now build nxagent against X.org's libXrender. However, this cost us a compression feature in NX. The libNX_Xrender code has passages that do zero padding of the unused memory portions in non-32bit-depth glyphs (the NX_RENDER_CLEANUP feature). However, we have hope for being able to reintroduce that feature again later, but current efforts [3] still fail at run-time. Dropping libNX_Xext is not possible... ...the libNX_Xext / Xserver Xext code has been cleaned up instead. Quite an amount of research and testing has been spent on removing the libNX_Xext library from the build workflow of nxagent. However, it seems that building against X.org's libXext will require us to update the Xext part of the nxagent Xserver at the same time. While taking this deep look into Xext code, we dropped various Xext extensions from the nx-X11 Xserver code. The extensions that got dropped [5] are all extensions that already have been dropped from X.org's Xserver code, as well. Further investigation, however, showed, that actually the only used client side extension code from libNX_Xext is the XShape extension. Thus, all other client side extension got dropped now in a very recent pull request [4]. Dropping libNX_X11 not easy, research summary given here For the sake of dropping the Xlib library bundled with nx-libs, we have attempted at writing a shared library called libNX_Xproxy. This library is supposed to contain a subset of the NXtrans related Xlib code that NoMachine patched into X.org's libX11 (and libxtrans). Results of this undertaking [6] so far: Over the weekend, I thought this all through once more and I am pretty sure right now, that we can actually make libNX_Xproxy work and drop all of libNX_X11 from nx-libs soon. Although we have to port (i.e. copy) various functions related to the NX transport from libNX_X11 into libNX_Xproxy, this change will allow us to drop all Xlib drawing routines and use those provided by X.org's Xlib shared library directly. Composite extension backport in nxagent to version 0.4 Mihai Moldovan looked at what it needs to make the Composite extension functional in nxagent. Unfortunately, the exact answer cannot be given, yet. As a start, Mihai backported latest Composite extension (v0.4) code from X.org's Xserver into the nxagent Xserver [7]. The currently shipped Composite extension version in nxagent is v0.2. Work on the NX Compression shared library (aka nxcomp v3) Fernando Carvajal and Salvador Fandi o from the Qindel Group [1] filed three pull requests against the nxcomp part of nx-libs recently, two of them have been code cleanups (done by Fernando), the third one is a feature enhancement regarding channels in nxcomp (provided by Salva). Protocol clean-up: drop pre-3.5 support With the release of nx-libs 3.6.x, we will drop support for nxcomp versions earlier than 3.5. Thus, if you are still on nxcomp 3.4, be prepared for upgrading at least to version 3.5.x. The code removal had been issued as pull request #111 ("Remove compatibility code for nxcomp before 3.5.0") [8]. The PR has already been reviewed and merged. Fernando filed another code cleanup PR (#119 [9]) against nx-libs that also already got merged into the 3.6.x branch. UNIX Socket Support for Channels The nxcomp library (and thus, nxproxy) provides a feature called "channels". Channels in nxcomp can be used for forwarding traffic between NX client side and the NX server side (along the graphical X11 transport that nxcomp is designed for). Until version 3.5.x, nxcomp was only able to use local TCP sockets for providing / connecting to channel endpoints. We consider local TCP sockets as insecure and aim at adding UNIX file socket support to nxcomp whereever a connection is established. Salva provided a patch against nxcomp that provides UNIX socket support to channel endpoints. The initial use case for this patch is: connect to client side pulseaudio socket file and avoid enabling the TCP listening socket in pulseaudio. The traffic then is channeled to the server side, so that pulse clients can connect to a UNIX socket file rather than to a local TCP port. The channel support patch has already been reviewed and merged into the 3.6.x branch of nx-libs. Rebasing nxagent against latest X.org Ulrich Sibiller spent a considerable amount of time and energy on providing a build chain that allows building nxagent against a modularized X.org 7.0 (rather than against the monolithic build tree of X.org 6.9, like we still do in nx-libs 3.6.x). We plan to adapt and minimize this build workflow for nx-libs 3.7.x (scheduled for summer 2017). A short howto that shows how to build nxagent with that new workflow will be posted on this blog within the next days. So stay tuned. Further work accomplished Quite a lot of code cleanup PRs have been filed by myself against nx-libs. Most of them target at removal of unnecessary code from the nx-X11 Xserver code base and the nxagent DDX: The third one (PR #120) in the list requires some detailled explanation: We discovered that nxagent ships overrides some symbols from the original Xserver code base. These overrides are induced by copies of some files from some Xserver sub-directory placed into the hw/nxagent/ DDX path. All those files' names match the pattern NX*.c. These copies of code are done in a way that the C compiler suppresses throwing its 'symbol "" redefined: first defined in ""; redefined in ""' errors. The approach taken, however, requires to have quite a few 10.000 lines of redundant code in hw/nxagent/NX*.c that also gets shipped in some Xserver sub-directory (mostly dix/ and render/). With pull request #120, we have identified all code passages in hw/nxagent/NX*.c that must be considered as NX'ish. We differentiated the NX'ish functions from functions that never got changed by NoMachine when developing nxagent. I then came up with four different approaches ([13,14,15,16]) of dropping redundant code from those hw/nxagent/NX*.c files. We (Mihai Moldovan and myself) discussed the various approaches and favoured the disable-Xserver-code-and-include-into-NX*.c variant [14] over the others for the following reasons: In the long run, the Xserver portion of the patches provided via this pull request #120 are required to be upstreamed into X.org's Xserver. The discussion around this will be started when we fully dive into rebasing nxagent's Xserver code base against latest X.org Xserver. Tasks ahead before the 3.6.x Release Various tasks we face before 3.6.x can be released. Here is a probably incomplete list: Tasks ahead after the 3.6.x Release (i.e., for 3.7.x) Here is an even rougher and probably highly incomplete list for tasks after the 3.6.x release: Credits Some people have to be named here that give their heart and love to this project. Thank you guys for supporting the development efforts around nx-libs and the Arctica Project: Thanks to Nico Arenas Alonso from and on behalf of the Qindel Group for coordinating the current funding project around nx-libs. Thanks to Ulrich Sibiller for giving a great amount of spare time to working on the nxagent-rebase-against-X.org effort. Thanks to Mihai Moldovan for doing endless code reviews and being available for contracted work via BAUR-ITCS UG [17] on NXv3, as well. Thanks to Mario Becroft for providing a patch that allows us to hook into nxagent X11 sessions with VNC and have the session fully available over VNC while the NX transport is in suspended state. Also Mario is pouring some fancy UDP ideas into the re-invention of remote desktop computing process performed in the Arctica Project. Mario has been an NX supporter for years, I am glad to have him still around after so many years (although he was close to abandoning NX usage at least once). Thanks to Fernando Carvajal from Qindel (until April 2016) for cleaning up nxcomp code. Thanks to Orion Poplawski from the Fedora Project for working on the first bundled libraries removal patches and being a resource on RPM packaging. Thanks to my friend Lee for working behind the scenes on the Arctica Core code and constantly pouring various of his ideas into my head. Thanks for regularly reminding me on benchmarking things.
Folks, thanks to all of you for all your various efforts on this huge beast of software. You are a great resource of expertise and it's a pleasure and honour working with you all.
New Faces Last but not least, I'd like to let everyone know that the Qindel Group sponsors another developer joining in on NXv3 development: Vadim Troshchinskiy (aka vatral on Github). Vadim has worked on NXv3 before and we are looking forward to having him and his skills in the team soon (probably end of May 2016). Welcome on board of the team, Vadim.
[1] http://www.qindel.com
[2] https://github.com/ArcticaProject/nx-libs/pull/93
[3] https://github.com/sunweaver/nx-libs/commit/be41bde7efc46582b442706dfb85...
[4] https://github.com/ArcticaProject/nx-libs/pull/121
[5] https://github.com/ArcticaProject/nx-libs/pull/106
[6] https://github.com/sunweaver/nx-libs/tree/wip/libnx-x11-full-removal
[7] https://github.com/sunweaver/nx-libs/tree/pr/composite-0_4
[8] https://github.com/ArcticaProject/nx-libs/pull/111
[9] https://github.com/ArcticaProject/nx-libs/pull/119
[10] https://github.com/ArcticaProject/nx-libs/pull/102
[11] https://github.com/ArcticaProject/nx-libs/pull/106
[12] https://github.com/ArcticaProject/nx-libs/pull/120
[13] https://github.com/sunweaver/nx-libs/commit/9692e6a7045b3ab5cb0daaed187e... (include NX'ish code into Xserver)
[14] https://github.com/sunweaver/nx-libs/commit/3d359bfc2b6d021c1ae9c6e19e96... (include Xserver code into NX*.c)
[15] https://github.com/sunweaver/nx-libs/commit/af72ee5624a15d21c610528e37b6... (use weak symbols and non-static symbols)
[16] https://github.com/sunweaver/nx-libs/commit/7205bb8848c49ee3e78a82fde906... (override symbols with interceptions)
[17] http://www.baur-itcs.de/20-x2go/20-x2gosupport/

20 April 2016

Reproducible builds folks: Reproducible builds: week 51 in Stretch cycle

What happened in the reproducible builds effort between April 10th and April 16th 2016: Toolchain fixes Antoine Beaupr suggested that gitpkg stops recording timestamps when creating upstream archives. Antoine Beaupr also pointed out that git-buildpackage diverges from the default gzip settings which is a problem for reproducibly recreating released tarballs which were made using the defaults. Alexis Bienven e submitted a patch extending sphinx SOURCE_DATE_EPOCH support to copyright year. Packages fixed The following packages have become reproducible due to changes in their build dependencies: atinject-jsr330, avis, brailleutils, charactermanaj, classycle, commons-io, commons-javaflow, commons-jci, gap-radiroot, jebl2, jetty, libcommons-el-java, libcommons-jxpath-java, libjackson-json-java, libjogl2-java, libmicroba-java, libproxool-java, libregexp-java, mobile-atlas-creator, octave-econometrics, octave-linear-algebra, octave-odepkg, octave-optiminterp, rapidsvn, remotetea, ruby-rinku, tachyon, xhtmlrenderer. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet: diffoscope development Zbigniew J drzejewski-Szmek noted in #820631 that diffoscope doesn't work properly when a file contains several cpio archives. Package reviews 21 reviews have been added, 14 updated and 22 removed in this week. New issue found: timestamps_in_htm_by_gap. Chris Lamb reported 10 new FTBFS issues. Misc. The video and the slides from the talk "Reproducible builds ecosystem" at LibrePlanet 2016 have been published now. This week's edition was written by Lunar and Holger Levsen. h01ger automated the maintenance and publishing of this weekly newsletter via git.

15 April 2016

Raphaël Hertzog: Freexian s report about Debian Long Term Support, March 2016

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In February, 111.75 work hours have been dispatched among 10 paid contributors. Their reports are available: Evolution of the situation The number of sponsored hours started to increase for April (116.75 hours, thanks to Sonus Networks) and should increase even further for May (with a new Gold sponsor currently joining us, Babiel GmbH). Hopefully the trend will continue so that we can reach our objective of funding the equivalent of a full-time position. At the end of the month the LTS team will be fully responsible of all Debian 7 Wheezy updates. For now paid contributors are still helping the security team by fixing packages that were fixed in squeeze already but that are still outstanding in wheezy. They are also looking for ways to ensure that some of the most complicated packages can be supported over the wheezy LTS timeframe. It is likely that we will seek external help (possibly from credativ which is already handling support of PostgreSQL) for the maintenance of Xen and that some other packages (like libav, vlc, maybe qemu?) will be upgraded to newer versions which are still maintained (either upstream or in Debian Jessie by the Debian maintainers). Thanks to our sponsors New sponsors are in bold.

No comment Liked this article? Click here. My blog is Flattr-enabled.

13 April 2016

Mike Gabriel: IPv6: Broken by Design; Digital Ocean - How are we doing?

Recently, Digital Ocean (which I am a customer of) asked me "how they were doing". Well, yet another survey... Let's ignore this one for now... I thought some days ago. And then yesterday, I added IPv6 support to my main mail server (which runs at Hetzner, Germany). All my hosted/rented/whatever systems report back to this main mailserver. Now that that main mail server finally has its AAAA record and its own IPv6 address, all associated systems try to reach this main mail server via IPv6. Of course. Crippling IPv6 support by adding Port Blocks But, then, I see messages like these in my syslog files on droplets hosted at Digital Ocean:
Apr 13 10:10:59 <do-droplet> postfix/smtp[23469]: connect to mail.<mydomain>[<ipv6-address>]:25: Connection timed out
After some more research [1], I realized that the folks out there at DO really apply some port blockings to IPv6 networks, but not to IPv4 networks. Pardon me? From my DO droplets, I can nmap any port on my mail server (25,80,143,443, 465, 587, etc.) via the IPv4 connection, but not over the IPv6 connection. Wait, not fully true: ports 80 and 443 are not blocked, but the other aforementioned ports are definitely blocked. Is Digital Ocean a professional ISP or a WiFi hotspot provider at my nearest coffee place? (This really makes me scratch my head...). Routing only the first 16 addresses of allocated /64 prefixes The above was the second IPv6 brokeness I learned about DO, recently. An earlier issue with DO's IPv6 support, I encountered while I was deploying an IPv6 capable OpenVPN internet gateway via a droplet hosted at DO. Digital Ocean assigns full IPv6 /64 prefixes to each individual droplet (which is great), but only properly routes the first 16 IP addresses of such a /64 prefix [2]. Urgh? I had to work around this flaw by adding an IPv6-over-IPv4 tunnel and attaching an IPv6 /56 prefix obtained from Hurricane Electrics' tunnel broker service [3] to the OpenVPN server. Thanks, Digital Ocean, for reminding me about giving feedback So, today, I luckily received a reminder mail about DO's yet-another-survey survey. My opportunity!!! Here is the feedback, I gave:
DO service is basically good.
BUT: You as a provider SUCK when it comes to IPv6.
(1) http://pixelschatten.net/blocked-ipv6-ports/
-> SMTP/IMAP traffic blocked over IPv6, but not over IPv4... WTF?). I normally have all my systems report back to my main mail server. I expect this to work as it is the default on all Linux hosts nowadays, and that is: prefer IPv6 over IPv4.
(2) https://digitalocean.uservoice.com/forums/136585-digitalocean/suggestion...
-> Droplets get a full /64 prefix assigned, but only the first 16 addresses (or such) get routed properly. WTF?
Please do your homework on IPv6 and don't cripple your service by offering crippled IPv6 support.
I tell people, DO is great, but their IPv6 support is broken-by-design. Let me know, once this is about to change.
Mike Gabriel (aka sunweaver at debian dot org, Debian Developer)
Apology for the tone of the wording Now reading the feedback given, I realize that my tone has been quite impolite. I am sorry about this. However, the experienced IPv6 issues are indeed annoying. So please excuse me for having expressed my annoyance with such harsh words. And... I am still annoyed about myself paying an ISP for such a crippled IPv6 support. (I need to consider migrating the VMs to another hoster, unless there will be some dynamics observable in the near future). @Digital Ocean: Keep up the good work that you do in the realm of VM hosting. Evolve and grow up in the realm of IPv6 networking. Thank you! light+love
Mike [1] http://pixelschatten.net/blocked-ipv6-ports/
[2] https://digitalocean.uservoice.com/forums/136585-digitalocean/suggestion...
[3] https://tunnelbroker.net/

1 April 2016

Mike Gabriel: Force Google Chrome / Chromium Browser to use WPAD proxy detection

At one of your school sites, we encountered an issue where several people could not access the internet via the school's proxy server when using Google Chrome / Chromium Browser By default those two browser variants use the (Linux) system wide proxy settings. This is not always the best choice. For our setups, we prefer configuring proxy settings via WPAD [1]. We did not investigate each user profile for underlying reasons of not being able to connect via the site's proxy server in depth. Instead, we searched for a system-wide solution that easily enforces WPAD protocol usage when browsing the internet with Google Chrome / Chromium Browser. From the command line, forcing Google Chrome / Chromium Browser into WPAD proxy mode is very easy. It can be done with the command line option --proxy-auto-detect:
[user@mymachine ~]$ chromium-browser --proxy-auto-detect
To enforce launching these two browser variants with --proxy-auto-detect whenever users click onto icons provided on the Linux desktop, you can simply follow the below recipe:
  1. Create /usr/local/share/applications/ unless it already exists
  2. Copy /usr/share/applications/google-chrome.deskop to /usr/local/share/applications/.
  3. Copy /usr/share/applications/chromium-browser.deskop to /usr/local/share/applications/.
  4. Set the x-bit on those .desktop files: sudo chmod a+x /usr/local/share/applications/*.desktop
  5. Edit both .desktop files and search for lines starting with Exec=. Add --proxy-auto-detect to each of those occurrences after the actual command (see diff below).
  6. Run update-desktop-database /usr/local/share/applications
When users now click on any of the provided launchers provided by Google Chrome / Chromium Browser, the browser attempts proxy detection via the WPAD protocol. light+love,
Mike [1] http://www.faqs.org/rfcs/rfc3040.html (section 6.4)

PS: Difference between Chromium Browser's default .desktop file and our modified version in /usr/local/share/applications/:
--- usr/share/applications/chromium-browser.desktop	2016-03-16 19:04:18.000000000 +0100
+++ usr/local/share/applications/chromium-browser.desktop	2016-04-01 14:23:27.410775206 +0200
@@ -167,7 +167,7 @@
 Comment[zh_CN]= 
 Comment[zh_HK]= 
 Comment[zh_TW]= 
-Exec=chromium-browser %U
+Exec=chromium-browser --proxy-auto-detect %U
 Terminal=false
 X-MultipleArgs=false
 Type=Application
@@ -216,7 +216,7 @@
 Name[vi]=M  c a s  m i
 Name[zh_CN]= 
 Name[zh_TW]= 
-Exec=chromium-browser
+Exec=chromium-browser --proxy-auto-detect
 
 [Desktop Action Incognito]
 Name=Open a New Window in incognito mode
@@ -254,7 +254,7 @@
 Name[vi]=M  c a s  m i trong ch     n danh
 Name[zh_CN]= 
 Name[zh_TW]= 
-Exec=chromium-browser --incognito
+Exec=chromium-browser --incognito --proxy-auto-detect
 
 [Desktop Action TempProfile]
 Name=Open a New Window with a temporary profile
@@ -292,4 +292,4 @@
 Name[vi]=M  c a s  m i v i h  s  t m
 Name[zh_CN]= 
 Name[zh_TW]= 
-Exec=chromium-browser --temp-profile
+Exec=chromium-browser --temp-profile --proxy-auto-detect


PPS: Similar for Google Chrome:
--- usr/share/applications/google-chrome.desktop	2016-04-01 14:02:29.388283436 +0200
+++ usr/local/share/applications/google-chrome.desktop	2016-04-01 14:13:59.381895899 +0200
@@ -105,7 +105,7 @@
 Comment[zh_CN]= 
 Comment[zh_HK]= 
 Comment[zh_TW]= 
-Exec=/usr/bin/google-chrome-stable %U
+Exec=/usr/bin/google-chrome-stable --proxy-auto-detect %U
 Terminal=false
 Icon=google-chrome
 Type=Application
@@ -165,7 +165,7 @@
 Name[vi]=C a s  M i
 Name[zh_CN]= 
 Name[zh_TW]= 
-Exec=/usr/bin/google-chrome-stable
+Exec=/usr/bin/google-chrome-stable --proxy-auto-detect
 TargetEnvironment=Unity
 
 [NewIncognito Shortcut Group]
@@ -218,5 +218,5 @@
 Name[vi]=C a s   n danh m i
 Name[zh_CN]= 
 Name[zh_TW]= 
-Exec=/usr/bin/google-chrome-stable --incognito
+Exec=/usr/bin/google-chrome-stable --proxy-auto-detect --incognito
 TargetEnvironment=Unity

30 March 2016

Mike Gabriel: Pushing X.org Git repos to Github et al.

TL;DR; If you want to know why cloning several of the X.org repositories to Github or GitLab instances fails and how this can be worked around, you may want to continue reading. Why we stumbled over this issue... As a joint effort of the Arctica Project, TheQVD and X2Go, Ulrich Sibiller and I are currently preparing a build workflow for the nxagent X-server (version 3) [1] that allows building nxagent against the modular X.org 7.0 (using autoconf and automake) rather than the monolithic build workflow of X.org 6.9 (using ancient imake). Our goal is to rewind all X.org components required for building nxagent back to a state where nxagent successfully builds and runs. Then we will go through various, (probably) monthly cycles of Our first hurdle... After Ulrich now has a functioning nxagent-against-X.org-7.0-workflow locally, we want to get everything into the Arctica Project's Github namespace. Which fails...
[mike@minobo xorg.upstream]$ git clone https://anongit.freedesktop.org/git/xorg/lib/libX11.git
Cloning into 'libX11'...
remote: Counting objects: 18520, done.
remote: Compressing objects: 100% (3441/3441), done.
remote: Total 18520 (delta 15094), reused 18325 (delta 14954)
Receiving objects: 100% (18520/18520), 5.98 MiB   549.00 KiB/s, done.
Resolving deltas: 100% (15094/15094), done.
Checking connectivity... done.
[mike@minobo xorg.upstream]$ cd libX11/
[mike@minobo libX11 (master)]$ git remote rename origin upstream 
[mike@minobo libX11 (master)]$ git remote add origin git@github.com:ArcticaProject/libX11.git
[mike@minobo libX11 (master)]$ git push --mirror origin 
Counting objects: 18520, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3301/3301), done.
remote: error: object 70d5e4d45dd7bf1e05b099cb5a4dd529344084f0: missingSpaceBeforeDate: invalid author/committer line - missing space before date
remote: fatal: Error in object
error: pack-objects died of signal 13
error: failed to push some refs to 'git@github.com:ArcticaProject/libX11.git'
This issue should be a known issue at X.org / Mesa upstream [2]. The underlying problem occurs with Git 2.5 and above... If you run git-fsck from Git (>= 2.5) against several of the X.org Git repositories, you will stumble over the above issue. For libX11.git, we see these errors reported when running git-fsck from current Debian unstable:
mike@sid:~/xorg/libX11.git$ git fsck 
Checking object directories: 100% (256/256), done.
error in tag 70d5e4d45dd7bf1e05b099cb5a4dd529344084f0: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag a3bfe698090a5d41f1e9acb1b57a049085d6b04e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag c1fc8c9aec1bc92d03d08d5e986dd40b194a7a3e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag d4c27f7e7d8e2ea32418f341ad85c33f3b76862d: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 79780379aee21bf4d4bcb046a3b54774893ea6b1: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag a638de80ec20dc1de625cd323ed19a6646fecf2e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag f6a6414e4e52a971a35fd118c350567e2383d034: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag b5c6810e21be2e7ccfac8f0539d46bf75dbe50a0: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 86cec9d20428cc190ec7278a0abe481b180288ac: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag ac912988091574790c7cffbbc2c60b8c59f8fcef: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2d01674092951cd3284b3b099978b2436ea468ad: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 922d0534154668918eaa09a36cc5cd53fc6b71b7: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 8ef91b9cc0a4a7b0361f6b40b3f735221c8272cb: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 1516173727005a87aa501e1ad708d72e6ae6e753: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 77c9e63a3abff1593ccbed249b2c3ae7f68e1833: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 58044c7087137ce2b521297dbb31934ccaedc94e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2e0d061c6830891dcc856a04aac900e5bb6d0779: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2f58c7cc4cbf3d30927f6301039b6bf018ed38fd: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 77fc2e1e26019c3ba92275d249fe1979570255d2: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 9b2e06fa262f23dbcbbccd42be6477e08a452268: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 7f99410bb9cc9f7cbbc2d43d8ad044771a6eec0a: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 99b59b83f126b4abb9da89f577a5b8147d543c25: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 97170c33f99761acd932f968e21d6d7664e7ba80: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag b39966d452127c992987d082f69321f950ae4c39: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 05e20c3bd8ac6b7e873607db73894486a4ac0519: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag f478497b5a3e9d6b92237e0f408c9f0f18108f52: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag c5ed936f28ce70c8932e33d67a6e06d140e35ba8: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2b6beba295a6ff997dfe15a732c95ae6577fa735: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 32edbb1e9c5b7d1c6f8639dab85e5368d303df7f: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 490fbe2afb9ffc7057c0eb150f7cdd4d4fa8686c: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag fc605ac48bd7d675d6d04162a0f3bfebe27f2037: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 6289ecc26d1d505f395c59da744a9354d7aad2a2: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 8c9c30dac72aebd1a2c31a55a47c0ac11bc328db: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 3b7aa57994924be0dad693e837559d9ee900299c: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag b02ad9a8ec96e3c1027ca71289aeef5b8748e17e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 4402ab1a86226a9cd0ce84fb8147b1c4958c684d: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag bb02b4fbde316644c771d28d5edeeb9b213a6d6a: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 95b6462c70213b562c9d2b8726eda706edc9c456: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag a99fea43ad53907623f269942c55237c238645e0: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 0c417ce98a6cf927ce7a1359b198b2bb746c707e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag ce3ff6b102f360fc9dcd3a85361f44da054de0b1: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2fb7b16b38a365c5009bd170d4463634c1f3de26: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 95570f484bfde5fc2ed8548cb17b87d8114a2866: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag d6e5895c104cfe0d135494a605013dd2910a93a1: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 49be7144523c4a56afcb389a9aa021d167710d2c: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 020353d1b982ac938d229039ca13bfea5793fcd9: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 991792d4f8477ab65ad4d8a0245c9a3bbb1f0294: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag e8037555efe299a7143370b485f754f6b08ffac7: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 4fdc8e36a5022f3e32d5567305d8e9cde4011f1a: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 9c93a19a76163328e30a7306bb0babe3175fb9a2: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 9f90bf4424250f80340b3e2803d343c74d2adb0b: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2e7a8c7974777e3e869de9013025dc3f61633e1e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 59b7c06da2a64d72668394e9239e3dfe8a5cff59: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 0263c4f684faa9a5126a0db524d5d35be7277e52: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 1514e654c1a9b346cbfca5a81f6d331f106242aa: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 6aa83ec3d1d82ee332cedcf4e99d3cbd56e73edd: missingSpaceBeforeDate: invalid author/committer line - missing space before date
Checking objects: 100% (18523/18523), done.
What actually is going wrong here... When looking at one of those tags (luckily, only tags are affected), we see that something's not ok with the date string of the tag (I am not a Git plumber, so sorry for being superficial here):
mike@sid:~/xorg/libX11.git$ git show 2fb7b16b38a365c5009bd170d4463634c1f3de26
tag XORG-RELEASE-1-TM-MERGE
Tagger: Alan Coopersmith 
Date:   Thu Jan 1 00:00:00 1970 +0000
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
commit 84adc60cae8e944777563501dd633622a28bdb3b
Author: Alan Coopersmith 
Date:   Sat Mar 27 07:08:16 2004 +0000
    Fix typo (IsModiferKey should be IsModifierKey)
[... not quoting the diff for the above commit ...]
How to work around that issue... We then got the idea that we could redo those tags with the date string derived from the commit that this tag points at. This can be scripted (credits go to an unnamed person in Ulrich's surrounding!!!):
mike@sid:~/xorg/libX11$ cat repair.sh
#!/bin/bash
set -x
git fsck 2>&1   tee git-fsck.log   grep "error in tag"   sed -r -e 's/error in tag ([0-9a-f]+):.*/\1/p'   while read broken; do
         echo $broken
         commit=$(git rev-parse $ broken ^ commit )
         tag=$(git cat-file tag $broken   sed -n 's/^tag //p')
         tag_msg=$(git cat-file tag $broken   sed -n '/^$/,$p'   tail -n +2)
         export GIT_COMMITTER_NAME="$(git log -1 --format='%cn' $commit)"
         export GIT_COMMITTER_EMAIL="$(git log -1 --format='%ce' $commit)"
         export GIT_COMMITTER_DATE="$(git log -1 --format='%cD' $commit)"
         git tag -a -f -m "$tag_msg" $tag $commit
done
# drop all old content...
git reflog expire --expire=all --all
git gc --prune=all
# no more errors should be reported...
git fsck
Copy this repair.sh into the Git repo itself and let the script walk over your tags. After this script has been applied you can push the X.org Git repo with broken tags redone to Git hosters like Github or GitLab. light+love,
Mike [1] https://github.com/ArcticaProject/nx-libs
[2] https://lists.freedesktop.org/archives/mesa-dev/2016-February/106268.html

11 March 2016

Rapha&#235;l Hertzog: Freexian s report about Debian Long Term Support, February 2016

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In February, 112.50 work hours have been dispatched among 11 paid contributors. Their reports are available: Evolution of the situation The number of sponsored hours continued to decrease a little bit. It s not worrisome yet but we should try to get back to a positive slope if we want to be able to do an outstanding job for wheezy LTS. On the positive side, TOSHIBA renewed their platinum sponsorship for another 6 months at least and we have some contacts for new sponsors, though they are far from being concluded yet. We are now in transition between squeeze LTS and wheezy LTS. The paid contributors are helping the security team by fixing packages that were fixed in squeeze already but that are still outstanding in wheezy. They are also taking generic measures to prepare wheezy LTS (for example to ensure all packages work with OpenJDK 7.x since support for 6.x will be dropped in the LTS period). Thanks to our sponsors New sponsors are in bold (none this month).

No comment Liked this article? Click here. My blog is Flattr-enabled.

4 March 2016

Mike Gabriel: My FLOSS activities in February 2016

February 2016 has been a very active month regarding me contributing to the FLOSS world. Honouring my Sponsors I am happy to share that this month's FLOSS work has been sponsored by various sponsors. Thanks to all people and companies sponsoring my work on FLOSS projects. This month's MATE uploads to Debian With regards to the Beta 1 Freeze date of Ubuntu 16.04 LTS (18th Feb 2016), Martin Wimpress, Vangelis Mouhtsis and I performed quite some work on Debian MATE. Uploads to Debian unstable: The Debian MATE Packaging Team also took over maintenance of the GTK-2+ legacy package libwnck [13]. The first upload introducing some major changes and package clean-ups caused a slight wave [14] because of a missing dependency in libwnck-dev (that fell victim to some clean-ups in debian/control). Those issues have been addressed immediately and have now been settled. The main reason for working on a legacy package like libwnck was the need for having gir1.2-wnck-1.0 (back) in Debian. The new MATE dock applet requires the libwnck GIR package to be present at runtime. One of the novelties in Ubuntu MATE 16.04 LTS will be the option to adapt the look and feel of the MATE desktop to how a Unity-based desktop looks like. Martin Wimpress is giving intense work to providing a dock applet and topmenu support as one alternative among the various Ubuntu MATE desktop experiences provided. The alternative desktop layouts can be configured with the MATE Tweak tool. Work on RDP related packages Work on FreeRDP 1.1 as currently in Debian I finally managed to give some priority (and thus time) to fixing various issues in the freerdp package in Debian [15]. Many people had provided patches and solutions to open issues and I tried to honour as many of those, as possible. Please note that I had to disable the GStreamer support in FreeRDP for the recent uploads, as the currently used Git snapshot of FreeRDP only supports GStreamer 0.10's API whereas the security team is in the process of having gstreamer0.10-* packages removed from the Debian stretch/unstable archives. Work on FreeRDP 2.0, coming to Debian soon Furthermore, Bernhard Miklautz is currently working on a freerdp2 package, which will bring the latest Git snapshot of FreeRDP upstream into Debian (and also re-introduce GStreamer support, based on GStreamer 1.0). Bernhard invested a lot of time on pushing the current HEAD of FreeRDP upstream [16] towards a FreeRDP 2.x version. Starting with FreeRDP 2.x it will be possible to install different FreeRDP versions on one system without file naming conflicts. For March 2016, I have doing the final freerdp2 reviewing on my todo list (possibly together with H ctor Or n Mart nez who is highly interested in the RDP backend support in Wayland/Weston), so that we can provide first uploads to Debian experimental sometime the coming month. The packaging progress is continuously discussed on the #freerdp channel on Freenode and can also be viewed on Github [17]. Review of revised XRDP package Recently, Dominik George from Teckids e.V. [18] contacted me about reviewing their effort of updating the Debian package xrdp, which currently is in ITA state [19]. Feedback has been provided and I am waiting for a ping from his side so that I can take some (ideally) final looks at the package and sponsor the upload. Work on Debian Edu related packages This month, I spent a couple of hours of work on several Debian Edu related tasks, some of them induced by problems at local school sites we support. Work on Debian LTS My 8h-portion of work for the Debian LTS Project, I performed at the very end of February. With the Debian squeeze LTS EOL date on 29th February, I saw to finalizing my personal open todos regarding Debian squeeze LTS, which basically was getting two CVE issues fixed in the lxc package [26]. The rest of the work hours has been spent on helping out the Security Team of Debian with open CVE issues in Debian wheezy packages: The gosa .debdiff has been approved by a member of the Security Team, the upload will happen today. With my LTS frontdesk hat on (during week 9 / 2016) I also spent some time providing help regarding SVN checkout problems and raised a couple of questions on how to coordinate the work phase between the Debian squeeze LTS EOL and the official launch of the Debian wheezy LTS project phase [27]. Work on nx-libs At the end of February, I finally managed to propose a way of dropping the libNX_Xrender.so bundled library from the nx-libs code base. I filed a PR [28] against nx-libs that proposes its removal and provides a patch for using X.Org's libXrender.so version. As a preview for nx-libs work in March 2016... I have started with removing the complete libNX_X11.so library from nx-libs and using X.Org's X11 client library. This will introduce a code removal of around 160.000 lines of code to nx-libs. More to come on this later... light+love,
Mike [1] http://ubuntu-mate.org/
[2] https://www.freexian.com/
[3] http://www.qindel.com/ [4] (caja)
https://lists.debian.org/debian-devel-changes/2016/02/msg00468.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02080.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02086.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02183.html [5] (mate-menu)
https://lists.debian.org/debian-devel-changes/2016/02/msg00469.html [6] (mate-panel)
https://lists.debian.org/debian-devel-changes/2016/02/msg01900.html [7] (mate-dock-applet)
https://lists.debian.org/debian-devel-changes/2016/02/msg01935.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02481.html
https://lists.debian.org/debian-devel-changes/2016/02/msg03097.html [8] (mate-polkit)
https://lists.debian.org/debian-devel-changes/2016/02/msg01936.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02395.html [9] (eom)
https://lists.debian.org/debian-devel-changes/2016/02/msg02073.html [10] (pluma)
https://lists.debian.org/debian-devel-changes/2016/02/msg02128.html [11] (topmenu-gtk)
https://lists.debian.org/debian-devel-changes/2016/02/msg02399.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02501.html [12] (mate-tweak)
https://lists.debian.org/debian-devel-changes/2016/02/msg03086.html [13] (libwnck)
https://lists.debian.org/debian-devel-changes/2016/02/msg01248.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01404.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01825.html [14] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814585
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814588
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814697 [15] (freerdp)
https://lists.debian.org/debian-devel-changes/2016/02/msg02487.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02630.html [16] https://github.com/FreeRDP/FreeRDP
[17] https://github.com/bmiklautz/debian-freerdp2 [18] https://www.teckids.org/ [19] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=719624 [20] (gosa)
https://lists.debian.org/debian-devel-changes/2016/02/msg01554.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01954.html [21] https://sunweavers.net/blog/node/34 [22] (ldap2zone)
https://lists.debian.org/debian-devel-changes/2016/02/msg01966.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01967.html [23] (shutdown-at-night)
https://lists.debian.org/debian-devel-changes/2016/02/msg03605.html [24] (italc)
https://lists.debian.org/debian-devel-changes/2016/02/msg01944.html [25] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=815948 [26] (lxc, Debian squeeze LTS)
https://lists.debian.org/debian-lts-changes/2016/02/msg00037.html [27] https://lists.debian.org/debian-lts/2016/02/msg00155.html
(The thread continues in March 2016) [28] https://github.com/ArcticaProject/nx-libs/pull/93

19 February 2016

Petter Reinholdtsen: Creating, updating and checking debian/copyright semi-automatically

Making packages for Debian requires quite a lot of attention to details. And one of the details is the content of the debian/copyright file, which should list all relevant licenses used by the code in the package in question, preferably in machine readable DEP5 format. For large packages with lots of contributors it is hard to write and update this file manually, and if you get some detail wrong, the package is normally rejected by the ftpmasters. So getting it right the first time around get the package into Debian faster, and save both you and the ftpmasters some work.. Today, while trying to figure out what was wrong with the zfsonlinux copyright file, I decided to spend some time on figuring out the options for doing this job automatically, or at least semi-automatically. Lucikly, there are at least two tools available for generating the file based on the code in the source package, debmake and cme. I'm not sure which one of them came first, but both seem to be able to create a sensible draft file. As far as I can tell, none of them can be trusted to get the result just right, so the content need to be polished a bit before the file is OK to upload. I found the debmake option in a blog posts from 2014. To generate using debmake, use the -cc option:
debmake -cc > debian/copyright
Note there are some problems with python and non-ASCII names, so this might not be the best option. The cme option is based on a config parsing library, and I found this approach in a blog post from 2015. To generate using cme, use the 'update dpkg-copyright' option:
cme update dpkg-copyright
This will create or update debian/copyright. The cme tool seem to handle UTF-8 names better than debmake. When the copyright file is created, I would also like some help to check if the file is correct. For this I found two good options, debmake -k and license-reconcile. The former seem to focus on license types and file matching, and is able to detect ineffective blocks in the copyright file. The latter reports missing copyright holders and years, but was confused by inconsistent license names (like CDDL vs. CDDL-1.0). I suspect it is good to use both and fix all issues reported by them before uploading. But I do not know if the tools and the ftpmasters agree on what is important to fix in a copyright file, so the package might still be rejected. The devscripts tool licensecheck deserve mentioning. It will read through the source and try to find all copyright statements. It is not comparing the result to the content of debian/copyright, but can be useful when verifying the content of the copyright file. Are you aware of better tools in Debian to create and update debian/copyright file. Please let me know, or blog about it on planet.debian.org. As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b. Update 2016-02-20: I got a tip from Mike Gabriel on how to use licensecheck and cdbs to create a draft copyright file
licensecheck --copyright -r  find * -type f    \
  /usr/lib/cdbs/licensecheck2dep5 > debian/copyright.auto
He mentioned that he normally check the generated file into the version control system to make it easier to discover license and copyright changes in the upstream source. I will try to do the same with my packages in the future. Update 2016-02-21: The cme author recommended against using -quiet for new users, so I removed it from the proposed command line.

14 February 2016

Rapha&#235;l Hertzog: Freexian s report about Debian Long Term Support, January 2016

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In December, 113.50 work hours have been dispatched among 9 paid contributors. Their reports are available: Evolution of the situation As expected, we had a small drop in the amount of hours sponsored. New sponsors (re-)joined but others stopped too (Gree this time) mostly balancing the result. We only lost 2 hours of sponsored work. It would be nice if we could invert that curve and actually start again to get closer to our objective of funding the equivalent of a full time position. Let s hope that the switch to wheezy as the version supported by the LTS team will motivate many companies relying on Debian 7 in their IT system. In terms of security updates waiting to be handled, the situation is close to last month(17 packages in dla-needed.txt, 27 in the list of CVE). It looks like that having about 20 packages needing an update is the normal situation and that we can t really get further down given the time required to process some updates (sometimes we wait until the upstream authors provides a patch, and so on). Thanks to our sponsors New sponsors are in bold.

No comment Liked this article? Click here. My blog is Flattr-enabled.

9 February 2016

Mike Gabriel: Systemd based network setup on Debian Edu jessie workstations

This article describes how to use systemd-networkd on Debian Edu 8.x (aka jessie) notebooks. What we have to deal with? At the schools we support we have several notebooks running Debian Edu 8.x (aka jessie) in the field. For school notebooks (classroom sets) we install the Debian Edu Workstation Profile. Those machines are mostly used over wireless network. We know that Debian Edu also offers a Roaming Workstation Profile at installation time, but with that profile chosen, user logins create local user accounts and local home directories on the notebooks (package: libpam-mklocaluser). For our customers, we do not want that. People using the school notebooks shall always work on their NFS home directories. School notebooks shall not be usable outside of the school network. Our woes... The default setup on Debian Edu jessie workstations regarding networking is this: We have observed various problems with that setup: read more

Mike Gabriel: R sum of our Edu Workshop in Kiel (26th - 29th January)

In the last week of January, the project IT-Zukunft Schule (Logo EDV-Systeme GmbH and DAS-NETZWERKTEAM) had visitors from Norway: Klaus Ade Johnstad and Linnea Skogtvedt from LinuxAvdelingen [1] came by for exchanging insights, knowledge, technology, stories regarding IT services at school in Norway and Nothern Germany. This was our schedule... Tuesday Wednesday Thursday read more

1 February 2016

Mike Gabriel: My FLOSS activities in January 2016

In January 2016 I was finally able to work on various FLOSS topics again (after two months of heavily focussed local customer work): Upload of MATE 1.12 to Debian testing/unstable At the beginning of the new year, I finalized the bundle upload of MATE 1.12 to Debian unstable [1]. All uploaded packages are available in Debian testing (stretch) and Ubuntu xenial by now. MATE 1.12 will also be the version shipped in Ubuntu MATE 16.04 LTS. Additionally, I finally uploaded caja-dropbox to Debian unstable (non-free), thanks to Vangelis Mouhtsis and Martin Wimpress for doing first steps preparations. The package has already left Debian's NEW queue, but unfortunately has been removed from Debian testing (stretch) again due to build failures in one of its dependencies. Debian LTS work In January 2016 I did my first round of Debian LTS front desk work [2]. Before actually starting with my front desk duty, I worked myself through the documentation and found it difficult to understand the output of the lts-cve-triage.py script. So, I proposed various improvments to the output of that script (all committed by now). During the second week of January then, I triaged the following packages regarding known/open CVE issues: read more

14 January 2016

Lunar: Reproducible builds: week 37 in Stretch cycle

What happened in the reproducible builds effort between January 3rd and January 9th 2016:

Toolchain fixes David Bremner uploaded dh-elpa/0.0.18 which adds a --fix-autoload-date option (on by default) to take autoload dates from changelog. Lunar updated and sent the patch adding the generation of .buildinfo to dpkg.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: aggressive-indent-mode, circe, company-mode, db4o, dh-elpa, editorconfig-emacs, expand-region-el, f-el, geiser, hyena, js2-mode, markdown-mode, mono-fuse, mysql-connector-net, openbve, regina-normal, sml-mode, vala-mode-el. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:
  • #809780 on flask-restful by Chris Lamb: implement support for SOURCE_DATE_EPOCH in the build system.
  • #810259 on avfs by Chris Lamb: implement support for SOURCE_DATE_EPOCH in the build system.
  • #810509 on apt by Mattia Rizzolo: ensure a stable file order is given to the linker.

reproducible.debian.net Add 2 more armhf build nodes provided by Vagrant Cascadian. This added 7 more armhf builder jobs. We now run around 900 tests of armhf packages each day. (h01ger) The footer of each page now indicates by which Jenkins jobs build it. (h01ger)

diffoscope development diffoscope 45 has been released on January 4th. It features huge memory improvements when comparing large files, several fixes of squashfs related issues that prevented comparing two Tails images, and improve the file list of tar and cpio archive to be more precise and consistent over time. It also fixes a typo that prevented the Mach-O to work (Rainer M ller), improves comparisons of ELF files when specified on the command line, and solves a few more encoding issues.

Package reviews 134 reviews have been removed, 30 added and 37 updated in the previous week. 20 new fail to build from source issues were reported by Chris Lamb and Chris West. prebuilder will now skip installing diffoscope to save time if the build results are identical. (Reiner Herrmann)

13 January 2016

Rapha&#235;l Hertzog: Freexian s report about Debian Long Term Support, December 2015

A Debian LTS logoLike each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In December, 113.50 work hours have been dispatched among 9 paid contributors. Their reports are available: Evolution of the situation We lost our first silver sponsor (Gandi.net, they prefer to give the same amount of money to Debian directly) and another sponsor reduced his sponsorship level. While this won t show in the hours dispatched in January, we will do a small jump backwards in February (unless we get new sponsors replacing those in the next 3 weeks). This is a bit unfortunate as we are rather looking at reinforcing the amount of sponsorship we get as we approach Wheezy LTS and we will need more support to properly support virtualization related packages and other packages that were formerly excluded from Squeeze LTS. Can you convince your company and help us reach our second goal? In terms of security updates waiting to be handled, the situation is close to last month. It looks like that having about 20 packages needing an update is the normal situation and that we can t really get further down given the time required to process some updates (sometimes we wait until the upstream authors provides a patch, and so on). Thanks to our sponsors We got one new bronze sponsor but he s not listed (he did not fill the form where we request their permission to be listed).

2 comments Liked this article? Click here. My blog is Flattr-enabled.

4 January 2016

Mike Gabriel: My FLOSS activities in December 2015

December 2015 was a month mainly dedicated to work for local contractors (local schools mainly) and my employer (University of Kiel, Git server migration). At the end of the month I had the privilege of attending the 32c3 ([1]) where we had a little sprint for the Arctica Project. Thanks to my family and esp. to my wonderful wife for letting me attend this always fascinating event at the end of each year. Horde Hacking One of my local customers is really interested in using a non-gated-community mail provider, so he asked me to host his company's mail addresses on my mail company's server. Something I regularly don't offer (anymore) except for dear friends and very patient customers. This customer sponsored several more work hours on hacking on the Kolab_Storage code in Horde and proposing bug fixes upstream [2,3,4,5,6,7,8]. Thanks for supporting my work on the Horde Groupware Framework. Thanks to Horde upstream maintainers (esp. Michael Rubinsky) for reacting on my bug submissions so promptly. Debian and Debian LTS Locally, I did a lot of work for our Debian Edu / Skolelinux customers again this months. read more

Next.

Previous.