Search Results: "Sylvestre Ledru"

2 June 2020

Sylvestre Ledru: Debian rebuild with clang 10 + some patches

Because of the lock-down in France and thanks to Lucas, I have been able to make some progress rebuilding Debian with clang instead of gcc.

TLDR Instead of patching clang itself, I used a different approach this time: patching Debian tools or implementing some workaround to mitigate an issue.
The percentage of packages failing drop from 4.5% to 3.6% (1400 packages to 1110 - on a total of 31014). I focused on two classes of issues:

Qmake As I have no intention to merge the patch upstream, I used a very dirty workaround. I overwrote the g++ qmake file by clang's:
https://salsa.debian.org/lucas/collab-qa-tools/-/blob/master/modes/clang10#L44-47 I dropped the number of this failure to 0, making some packages build flawlessly (example: qtcreator, chessx, fwbuilder, etc). However, some packages are still failing later and therefore increasing the number of failures in some other categories like link error. For example, qtads fails because of ordered comparison between pointer and zero or oscar fails on a -Werror,-Wdeprecated-copy error. Breaking the build later also highlighted some new classes of issues which didn't occur with clang < 10.
For example, warnings related to C++ range loop or implicit int float conversion (I fixed a bunch of them in Firefox) .

Symbol differences Historically, symbol management for C++ in Debian has been a pain. Russ Allbery wrote a blog post in 2012 explaining the situation. AFAIK, it hasn't changed much.
Once more, I took the dirty approach: if there new or missing symbols, don't fail the build.
The rational is the following: Packages in the Debian archive are supposed to build without any issue. If there is new or missing symbols, it is probably clang generating a different library but this library is very likely working as expected (and usable by a program compiled with g++ or clang). It is purely a different approach taken by the compiler developer. In order to mitigate this issue, before the build starts, I am modifying dpkg-gensymbols to transform the error into a warning.
So, the typical Debian error some new symbols appeared in the symbols file or some symbols or patterns disappeared in the symbols file will NOT fail the build. Unsurprisingly, all but one package (libktorrent) build. Even if I am pessimistic, I reported a bug on dpkg-dev to evaluate if we could improve dpkg-gensymbol not to fail on these cases.

Next steps The next offender is Imake.tmpl:2243:10: fatal error: ' X11 .rules' file not found with more than an hundred occurrences, reported upstream quite sometime ago. Then, the big issues are going to be much harder to fix as they are real issues/warnings (with -Werror) in the code of the packages. Example: -Wc++11-narrowing & Wreserved-user-defined-literal... The list is long.
I will probably work on that when llvm/clang 11 are in RC phase.

For maintainers & upstream Maintainer of Debian/Ubuntu packages? I am providing a list of failing packages per maintainer: https://clang.debian.net/maintainers.php
For upstream, it is also easy to test with clang. Usually, apt install clang && CC=clang CXX=clang++ <build step> is good enough.

Conclusion With these two changes, I have been able to fix about 290 packages. I think I will be able to get that down a bit more but we will soon reach a plateau as many warnings/issues will have to fix in the C/C++ code itself.

22 March 2020

Sylvestre Ledru: Some clang rebuild results (8.0.1, 9.0.1 & 10rc2)

As part of the LLVM release cycle, I am continuing rebuilding the Debian archive with clang instead of gcc to evaluate potential regressions. Processed results are available on the website: https://clang.debian.net/status.php - Now includes some fancy graphs to show the evolution
Raw logs are published on github: https://github.com/opencollab/clang.debian.net/tree/master/logs Since my last blog post on the subject (August 2017), Clang is more and more present in the tech ecosystem. It is now the compiler used to build Firefox and Chrome upstream binaries on all the supported architectures/operating systems. More architectures are supported, it has a new linker (lld), a new hybrid IR (MLIR), a lot of checkers in clang-tidy, cross-language linking with Rust, etc.

Results
Now, about Debian results, we rebuilt using 8.0.1, 9.0.1 and 10.0rc2. Results are pretty similar to what we had with previous versions: between 4 to 5% of packages are failing when gcc is replaced by clang.
Some clang rebuild results (8.0.1, 9.0.1 &amp; 10rc2)

Even if most of the software are still using gcc as compiler, we can see that clang has a positive effect on code quality. With many different kinds of errors and warnings found clang over the years, we noticed a steady decline of the number of errors. For example, the number of incorrect C/C++ main declarations has been decreasing years after years:
Some clang rebuild results (8.0.1, 9.0.1 &amp; 10rc2)

Errors found
The biggest offender is still the qmake changes which doesn't allow the used workaround (replacing /usr/bin/gcc by /usr/bin/clang) - about 250 errors. Most of these packages would probably compile fine with clang. More on the Qt bug tracker. The workaround proposed in the bug isn't applicable for us as we use the dropped-in replacement of the
The second error is still some differences in symbol generation. Unlike gcc, it seems that clang doesn't generate some symbols (or adds some). As a bunch of Debian packages are checking the list of symbols in the library (for ABI management), the build fails on purpose. For example, with libcec, the symbol _ZN10P8PLATFORM14CConditionImplD1Ev@Base 3.1.0 isn't generated anymore. I am not expecting this to be a big deal: the generated libraries probably works most of the time. More on C++ symbol management in Debian.

Current status
As previously said in a blog post, I don't think there is a strong intensive to go away from gcc for most of the Linux distributions. The big reason for BSD was the license (even if the move to the Apache 2 license wasn't received positively by some of them).
While the LLVM/clang ecosystem clearly won the tooling battle, as a C/C++ compiler, gcc is still an excellent compiler which supports more architecture and more languages.
In term of new warnings and checks, as the clang community moved the efforts in clang-tidy (which requires more complex tooling), out of the box, gcc provides a better experience (as example, see the Firefox meta bug to build with -Werror with the default warnings using gcc 9, gcc 10 and clang trunk for example).

Next steps
I see some potential next steps to decrease the number of failure:
  • Workaround the Qt/Qmake issue
  • Fix the objective-c header include issues (echo "#include <objc/objc.h>" > foo.m && clang -c foo.m is currently failing)
  • Identify why clang generates more/less symbols that gcc in the library and try to fix that
  • Rebuild the archive with clang-7 - Seems that I have some data problem

Many thanks to Lucas Nussbaum for the rebuilds.

Sylvestre Ledru: Some clang rebuild results (8.0.1, 9.0.1 & 10rc2)

As part of the LLVM release cycle, I am continuing rebuilding the Debian archive with clang instead of gcc to evaluate potential regressions. Processed results are available on the website: https://clang.debian.net/status.php - Now includes some fancy graphs to show the evolution
Raw logs are published on github: https://github.com/opencollab/clang.debian.net/tree/master/logs Since my last blog post on the subject (August 2017), Clang is more and more present in the tech ecosystem. It is now the compiler used to build Firefox and Chrome upstream binaries on all the supported architectures/operating systems. More architectures are supported, it has a new linker (lld), a new hybrid IR (MLIR), a lot of checkers in clang-tidy, cross-language linking with Rust, etc.

Results
Now, about Debian results, we rebuilt using 8.0.1, 9.0.1 and 10.0rc2. Results are pretty similar to what we had with previous versions: between 4 to 5% of packages are failing when gcc is replaced by clang.
Some clang rebuild results (8.0.1, 9.0.1 &amp; 10rc2)

Even if most of the software are still using gcc as compiler, we can see that clang has a positive effect on code quality. With many different kinds of errors and warnings found clang over the years, we noticed a steady decline of the number of errors. For example, the number of incorrect C/C++ main declarations has been decreasing years after years:
Some clang rebuild results (8.0.1, 9.0.1 &amp; 10rc2)

Errors found
The biggest offender is still the qmake changes which doesn't allow the used workaround (replacing /usr/bin/gcc by /usr/bin/clang) - about 250 errors. Most of these packages would probably compile fine with clang. More on the Qt bug tracker. The workaround proposed in the bug isn't applicable for us as we use the dropped-in replacement of the compiler.
The second error is still some differences in symbol generation. Unlike gcc, it seems that clang doesn't generate some symbols (or adds some). As a bunch of Debian packages are checking the list of symbols in the library (for ABI management), the build fails on purpose. For example, with libcec, the symbol _ZN10P8PLATFORM14CConditionImplD1Ev@Base 3.1.0 isn't generated anymore. I am not expecting this to be a big deal: the generated libraries probably works most of the time. More on C++ symbol management in Debian.
I reported this bug upstream a while back: https://bugs.llvm.org/show_bug.cgi?id=30441

Current status
As previously said in a blog post, I don't think there is a strong intensive to go away from gcc for most of the Linux distributions. The big reason for BSD was the license (even if the move to the Apache 2 license wasn't received positively by some of them).
While the LLVM/clang ecosystem clearly won the tooling battle, as a C/C++ compiler, gcc is still an excellent compiler which supports more architecture and more languages.
In term of new warnings and checks, as the clang community moved the efforts in clang-tidy (which requires more complex tooling), out of the box, gcc provides a better experience (as example, see the Firefox meta bug to build with -Werror with the default warnings using gcc 9, gcc 10 and clang trunk for example).

Next steps
I see some potential next steps to decrease the number of failure:
  • Workaround the Qt/Qmake issue
  • Fix the objective-c header include issues (echo "#include <objc/objc.h>" > foo.m && clang -c foo.m is currently failing)
  • Identify why clang generates more/less symbols that gcc in the library and try to fix that
  • Rebuild the archive with clang-7 - Seems that I have some data problem

Many thanks to Lucas Nussbaum for the rebuilds.

23 August 2017

Sylvestre Ledru: Rebuild of Debian using Clang 3.9, 4.0 and 5.0

tldr: The percentage of failure is decreasing, Clang support is improving but there is a long way to go. The goal of this initiative is to rebuild Debian using Clang as a compiler instead of gcc. I have been doing this analysis for the last 6 years. Recently, we rebuilt the archive of the Debian archive with Clang 3.9.1 (July 6th), 4.0.1 (July 6th) and 5.0 rc2 (August 20th). For various reasons, we didn't perform a rebuild since June 2016 with version 3.8. Therefor, we took the opportunity to do three over the last month. Now, the 3.9 & 4.0 results are impacted by a build failure when building all haskell packages (the -no-pie option in Clang doesn't exist - I introduced it in clang 5.0). Fixing this issue with 5.0 removed more than 860 failures. Also, for the same versions, a Qt compiler detection is considering that Clang is not a C++11 compiler because clang++, by default, defines __cplusplus as 199711L (-std=c++11 has to be added to define a correct __cplusplus). See https://bugreports.qt.io/browse/QTBUG-62535 for more information. Some discussions happened on the upstream mailing list about changing the default C++ dialect.
For example, with 4.0, this is causing 132 errors. With 5.0, probably thanks to a new Qt version, roughly the same number of packages are failing but because gcc just triggers a warning with the "nodiscard" attribute being incorrectly used when clang triggers an error. In parallel, ignoring the haskell build failures, the numbers sightly increased since last year even if the overall percentage decreased (new packages being uploaded in the archive).
VersionBuild failuresIgnoring haskell pkgs
3.81367 / 5.6%
3.92274 / 8.1%1618 / 5.8%
4.02311 / 8.3%1655 / 5.9%
5.01445 / 5.1%
In parallel, new warnings and errors showed up in Clang.
This is causing a new set of build failures (especially with the usage of -Werror). As few examples:
* Starting with 4.0, clang triggers an error ordered comparison between pointer and zero ('char *' and 'int').
* Similarly, with this version, -Wmain introduces a new warning which will trigger a warning when a bool literal is returned from main.
* clang also introduced a new warning called -Waddress-of-packed-member causing 5 new errors.
* With the same version, clang can trigger a new error when auto is used in function return type. Now, as a conclusion, having Debian being built with clang by default is still a long shot.
First, when Clang became usable for a general audience, gcc was lagging in term of warning and error detections. Now, gcc is in a much better position than it was, decreasing the interest to have clang replacing gcc. In parallel, most of the efforts in term of warnings
and mistake detections are currently done under the clang tidy umbrella, making them less intrusive as part of this initiative (but harder to use and to deploy).
As an example, the gcc warning -Wmisleading-indentation has been implemented under a clang-tidy checker.
Second, the very permissive license of clang has been a key factor for some operating systems to switch like the PS4, Mac OS X or FreeBSD. With Debian, the community is generally happy with the GPL.
Third, the performances are similar enough that it is not worth the work, except for some projects with very special needs. Last, despite that it is much easier to contribute to llvm/clang than gcc (not copyright assignment or actual review system for example), this isn't a big differentiator for most of the projects. Of course, I will continue to run and analysis these rebuilds as this is a great source of information for clang upstream developers to improve the compatibility with gcc and understand some impacts. However, until there is a big game changer, I will stop pursuing the goal of having Debian switching to clang instead of gcc. I will stop effort on the debile project (which was aiming to rebuild in the background packages).

10 March 2016

Mike Hommey: RIP Iceweasel, 13 Nov 2006 10 Mar 2016

This took longer than it should have, but a page is now officially turned. I uploaded Firefox and Firefox ESR to Debian unstable. They will have to go through the Debian NEW queue because they are new source packages, so won t be immediately available, but they should arrive soon enough. People using Iceweasel from Debian unstable will be upgraded to Firefox ESR. Debian stable will receive Firefox ESR after Iceweasel/Firefox ESR38 is end-of-lifed, in about 3 months. Thanks go to Sylvestre Ledru, Mike Connor (the same who filed bug 354622) and Stefano Zacchiroli.

11 September 2014

Sylvestre Ledru: Rebuild of Debian using Clang 3.5.0

Clang 3.5.0 has just been released. A new rebuild has been done highlight the progress to get Debian built with clang. tl;dr: Great progress. We decreased from 9.5% to 5.7% of failures. Full results are available on http://clang.debian.net At time of the rebuild with 3.4.2, we had 2040 packages failing to build with clang. With 3.5.0, this dropped to 1261 packages. Fixes With Arthur Marble and Alexander Ovchinnikov, both GSoC students, we worked on various ways to decrease the number of errors. Upstream fixes First, the most obvious way, we fixed programming bugs/mistakes in upstream sources. Basically, we took categories of failure and fixed issues one after the other. We started with simple bugs like 'Wrong main declaration', 'non-void function should return a value' or 'Void function should not return a value'.

They are trivial to fix. We continued with harder fixes like ' Undefined reference' or 'Variable length array for a non POD (plain old data) element'.

So, besides these one, we worked on:
In total, we reported 295 bugs with patches. 85 of them have been fixed (meaning that the Debian maintainer uploaded a new version with the fix).

In parallel, I think that the switch by FreeBSD and Mac OS X to Clang also helped to fix various issues by upstreams. Hacking in clang As a parallel approach, we started to implement a suggestion from Linus Torvalds and a few others. Instead of trying to fix all upstream, where we can, we tried to update clang to improve the gcc compatibility.

gcc has many flags to disable or enable optimizations. Some of them are legacy, others have no sense in clang, etc. Instead of failing in clang with an error, we create a new category of warnings (showing optimization flag '%0' is not supported) and moved all relevant flags into it. Some examples, r212805, r213365, r214906 or r214907

We also updated clang to silent some useless arguments like -finput_charset=UTF-8 (r212110), clang being UTF-8 compliant.

Finally, we worked on the forwarding of linker flags. Clang and gcc have a very different behavior: when gcc does not know an argument, it is going to forward the argument to the linker. Clang, in this case, is going to reject the argument and fail with an error. In clang, we have to explicitly declare which arguments are going to be transfer to the linker. Of course, the correct way to pass arguments to the linker is to use -Xlinker or -Wl but the Debian rebuild proved that these shortcuts are used. Two of these arguments are now forwarded: New errors Just like in other releases, new warnings are added in clang. With (bad) usage of -Werror by upstream software, this causes new build failures: I also took the opportunity to add some further categorizations in the list of errors. Some examples: Next steps The Debile project being close to ready with Cl ment Schreiner's GSoC, we will now have an automatic and transparent way to rebuild packages using clang. Conclusion As stated, we can see a huge drop in term of number of failures over time:
Hopefully, Clang getting better and better, more and more projects adopting it as the default compiler or as a base for plugin/extension developments, this percentage will continue to decrease.
Having some kind of release goal with clang for Jessie+1 can now be considered as potentially reachable. Want to help? There are several things which can be done to help: Acknowledgments Thanks to David Suarez for the rebuilds of the archive, Arthur Marble and Alexander Ovchinnikov for their GSoC works and Nicolas S velin-Radiguet for the few fixes.

11 August 2014

Sylvestre Ledru: clang 3.4, 3.5 and 3.6 are now coinstallable in Debian

Clang is finally co installable on Debian. 3.4, 3.5 and the current trunk (snapshot) can be installed together. So, just like gcc, the different version can be called with clang-3.4, clang-3.5 or clang-3.6. /usr/bin/clang, /usr/bin/clang++, /usr/bin/scan-build and /usr/bin/scan-view are now handled through the llvm-defaults package. llvm-defaults is also now managing clang-check, clang-tblgen, c-index-test, clang-apply-replacements, clang-tidy, pp-trace and clang-query. Changes are also available on llvm.org/apt/.
The next step will be to manage also llvm-defaults on llvm.org/apt to simplify the transition for people using these packages. So, with:
# /etc/apt/sources.list
deb http://llvm.org/apt/unstable/ llvm-toolchain main
deb http://llvm.org/apt/unstable/ llvm-toolchain-3.4 main
deb http://llvm.org/apt/unstable/ llvm-toolchain-3.5 main
$ apt-get install clang-3.4 clang-3.5 clang-3.6
$ clang-3.4 --version
Debian clang version 3.4.2 (branches/release_34) (based on LLVM 3.4.2)
Target: x86_64-pc-linux-gnu
Thread model: posix
$ clang-3.5 --version
Debian clang version 3.5.0-+rc2-1~exp1 (tags/RELEASE_350/rc2) (based on LLVM 3.5.0)
Target: x86_64-pc-linux-gnu
Thread model: posix
$ clang-3.6 --version
Debian clang version 3.6.0-svn214990-1~exp1 (trunk) (based on LLVM 3.6.0)
Target: x86_64-pc-linux-gnu
Thread model: posix

7 August 2014

Sylvestre Ledru: Debian Twitter accounts are back

After some downtime due to the identi.ca changes, the Debian Twitter accounts are now back. New Twitter feed ideas are welcome.

18 June 2014

Matthias Klumpp: Tanglu 2 (Bartholomea annulata) status update #1

bartholomea-annulata

Bartholomea annulata (c) Kevin Bryant

It is time for a new Tanglu update, which has been overdue for a long time now! Many things happened in Tanglu development, so here is just a short overview of what was done in the past months. Infrastructure Debile The whole Tanglu distribution is now built with Debile, replacing Jenkins, which was difficult to use for package building purposes (although Jenkins is great for other things). You can see the Tanglu builders in action at buildd.tg.o. The migration to Debile took a lot of time (a lot more than expected), and blocked the Bartholomea development at the beginning, but now it is working smoothly. Many thanks to all people who have been involved with making Debile work for Tanglu, especially Jon Severinsson. And of course many thanks to the Debile developers for helping with the integration, Sylvestre Ledru and of course Paul Tagliamonte. Archive Server Migration Those who read the tanglu-announce mailinglist know this already: We moved the main archive server stuff at archive.tg.o to to a new location, and to a very powerful machine. We also added some additional security measures to it, to prevent attacks. The previous machine is now being used for the bugtracker at bugs.tg.o and for some other things, including an archive mirror and the new Tanglu User Forums. See more about that below :-) Transitions There is huge ongoing work on package transitions. Take a look at our transition tracker and the staging migration log to get a taste of it. Merging with Debian Unstable is also going on right now, and we are working on merging some of the Tanglu changes which are useful for Debian as well (or which just reduce the diff to Tanglu) back to their upstream packages. Installer Work on the Tanglu Live-Installer, although badly needed, has not yet been started (it s a task ready for taking by anyone who likes to do it!) however, some awesome progress has been made in making the Debian-Installer work for Tanglu, which allows us to perform minimal installations of the Tanglu base systems and allows easier support of alternative Tanglu falvours. The work on d-i also uncovered a bug which appeared with the latest version of findutils, which has been reported upstream before Debian could run into it. This awesome progress was possible thanks to the work of Philip Mu kovac and Thomas Funk (in really hard debug sessions).
Tanglu ForumsTanglu Users We finally have the long-awaited Tanglu user forums ready! As discussed in the last meeting, a popular demand on IRC and our mailing lists was a forum or Stackexchange-like service for users to communicate, since many people can work better with that than with mailinglists. Therefore, the new English TangluUsers forum is now ready at TangluUsers.org. The forum software is in an alpha version though, so we might experience some bugs which haven t been uncovered in the testing period. We will watch how the software performs and then decide if we stick to it or maybe switch to another one. But so far, we are really happy with the Misago Forums, and our usage of it already led to the inclusion of some patches against Misago. It also is actively maintained and has an active community. Misc Thingstanglu logo pure KDE We will ship with at least KDE Applications 4.13, maybe some 4.14 things as well (if we are lucky, since Tanglu will likely be in feature-freeze when this stuff is released). The other KDE parts will remain on their latest version from the 4.x series. For Tanglu 3, we might update KDE SC 4.x to KDE Frameworks 5 and use Plasma 5 though. GNOME Due to the lack manpower on the GNOME flavor, GNOME will ship in the same version available in Debian Sid maybe with some stuff pulled from Experimental, where it makes sense. A GNOME flavor is planned to be available. Common infrastructure We currently run with systemd 208, but a switch to 210 is planned. Tanglu 2 also targets the X.org server in version 1.16. For more changes, stay tuned. The kernel release for Bartholomea is also not yet decided. Artwork Work on the default Tanglu 2 design has started as well any artwork submissions are most welcome! Tanglu joins the OIN The Tanglu project is now a proud member (licensee) of the Open Invention Network (OIN), which build a pool of defensive patents to protect the Linux ecosystem from companies who are trying to use patents against Linux. Although the Tanglu community does not fully support the generally positive stance the OIN has about software patents, the OIN effort is very useful and we agree with it s goal. Therefore, Tanglu joined the OIN as licensee.
And that s the stuff for now! If you have further questions, just join us on #tanglu or #tanglu-devel on Freenode, or write to our newly created forum! You can, as always, also subscribe to our mailinglists to get in touch.

2 June 2014

Rapha&#235;l Hertzog: My Free Software Activities since January 2014

If you follow my blog closely, you noticed that I skipped all my usual monthly summaries in 2014. It s not that I stopped doing free software work, instead I was just too busy to be able to report about what I did. As an excuse, let me tell you that we just moved into a new house which was in construction since may last year. The lack of visible activity on my blog resulted in a steady decrease of the amount of donations received (January: 70.72 , February: 71.75 , March: 51.25 , April: 39.9 , May: 40.33 ). Special thanks to all the people who kept supporting my work even though I stopped reporting about it. So let s fix this. This report will be a bit less detailed since it covers the whole period since the start of the year. Debian France Preparations related to general assemblies. The year started with lots of work related to Debian France. First I took care of setting up limesurvey with Alexandre Delano to handle the vote to pick our new logo:
The new logo of Debian France I also helped Sylvestre Ledru to finalize and close the accounting books for 2013 in preparation for the general assembly that was due later in the month. I wrote the moral report of the president to be presented to the assembly. And last step, I collected vote mandates to ensure that we were going to meet the quorum for the extraordinary assembly that was planned just after the usual yearly assembly. The assemblies took place during a two days mini-debconf in Paris (January 17-18) where I was obviously present even though I gave no talk besides announcing the logo contest winner and thanking people for their participation.
Assembl e g n rale 2014 de Debian France

The Debian France members during the general assembly

It s worth noting that the extraordinary assembly was meant primarily to enshrine in our bylaws the possibility to act as a trusted organization for Debian. This status should be officialized by the Debian project leader (Lucas Nussbaum) in the upcoming weeks since we answered satisfactorily to all questions. Our paypal donation form and the accounting tools behind it are ready. Galette packaging and members map. I managed to hand over the package maintenance of galette to Fran ois-R gis Vuillemin. I sponsored all his uploads and we packaged a new plugin that allows to create a map with all the members who accept to share their location. The idea was to let people meet each other when they don t live far away with the long term goal to have Debian France organized activities not only in Paris but everywhere in France. New contributor game. Last but not least, I organized a game to encourage people to do their first contribution to Debian by offering them a copy of my book if they managed to complete a small Debian project. We got many interesting projects but the result so far seem to be very mixed. Many people did not complete their project (yet) that said for the few that did substantial work, it was rather good and they seem to be interested to continue to contribute. Debian France booth at Solutions Linux in Paris. Like each year, I spent two days in Paris to help man the Debian France booth at Solutions Linux. We had lots of goodies on sale and we made more than 2000 EUR in earnings during the two days. I also used this opportunity to try to convince companies to support the new Debian LTS effort.
Debian France booth at Solutions Linux

Tanguy Ortolo and Fernando Lagrange behind the Debian France booth

The Debian Administrator s Handbook In the last days of 2013, we released the wheezy update of the book. Then I quickly organized everything needed so that the various translation teams can now focus their efforts on the latest release of the book. Later (in February) I announced the availability of the French and Spanish translations. Debian Squeeze LTS When the security team called for help to try to put in place long term support for Squeeze, I replied positively because I m convinced that it s very important if Debian wants to stay an acceptable choice in big deployments and because I knew that some of my customers would be interested Thus I followed all the discussions (on a semi-private list first and then on debian-lts@lists.debian.org) and contributed my own experience. I have also taken up the responsibility to coordinate with the Debian contributors who can be hired to work on Squeeze LTS so that we have a clear common offer for all the companies who have offered financial support towards Squeeze LTS. Expect further news on this front in the upcoming days/weeks. Tryton I have been a long time user of SQL-Ledger to manage the accounting of my company Freexian. But while the license is free software, the project is not. It s the work of a single developer who doesn t really accept help. I have thus been considering to move to something else for a long time but never did anything. This year, after some rough evaluation, I decided to switch to Tryton for my company. It s probably not a wise choice from a business perspective because that migration took me many hours of unpaid labor but from a free software perspective it s definitely better than everything else I saw. I contributed a lot of bug reports and a few patches already (#3596, #3631, #3633, #3665, #3667, #3694, #3695, #3696, #3697) mainly about problems found in the French chart of accounts but also about missing features for my use case. I also accepted to sponsor Matthias Berhle, who is maintaining the official Debian packages of Tryton. He s already a Debian maintainer so it s mainly a matter of reviewing new source packages and granting him the required rights. Misc Debian work Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

21 March 2014

Sylvestre Ledru: Rebuild of Debian using Clang 3.4

Using the AWS donation, David Suarez and myself have been able to rebuild the whole archive with Clang 3.4.
The rebuild has been done January 10th but, with my new job, I did not find the time to publish the result.
Releases after releases, the results are getting better and better. Currently, 2193 packages in the archive are failing to build from source.
That is roughly the same number of build failures as with the precedent rebuild with Clang 3.3.
However, this is good news for two reasons:
* the number of new packages in Debian increased (18854 at time of the 3.3 release, 21204 for the 3.4)
* clang 3.4 has more checks and error detections. I also started to update clang to make it closer to gcc. For example, I transformed wrong usage of -O (> 6) error to be treated as regular warning. However, a critical bug has emerged during this release. When using -D_FORTIFY_SOURCE=2 (which is the case of many Debian packages), the binaries produced freeze (infinity loop in the ASM). This has been reported upstream as bug 16821 and concerns about 150 packages (including Firefox, gcc, LLVM, etc). Hopefully, this will be fixed in 3.5 (this is unlikely for 3.4.1). About the new build failures, now, Clang triggers an error (or warning + -Werror) on: * Wrong usage of the default argument (should be done in the definition)
16 occurrences * Usage of C++11 feature without the appropriate argument
7 occurrences * Unused static const declaration
5 occurrences * Recursive template instantiation exceeded
4 occurrences * Defitinion of a builtin function
3 occurrences * Read-only variable is not assignable
2 occurrences By the way, I proposed a Google Summer of Code Project to work faster on a support of Debian built by Clang. As requirements, I asked students to fix some bugs, they already did a great job.

22 February 2014

Sylvestre Ledru: Some updates on llvm.org/apt/

I made some changes on http://llvm.org/apt/ for the last 2 months.

11 February 2014

Sylvestre Ledru: Hello Planet Mozilla

Hello Planet Mozilla I started as a new employee a few weeks ago. I am working as Release Manager for the next releases of Firefox (both on desktop + android).
As a start, Lukas and I will be working on Firefox 29 (no pressure at all to start with Australis + Firefox account).
I grew up with the first versions of Netscape, followed the Phoenix/Firebird adventures and I have been a supporter of Firefox since then.
I am glad to join such an important project, especially in the perspective of the recent privacy scandals. I think that Mozilla is in the best position to communicate to the public, to denounce certain activities and provides technologies to bypass them. Besides that, let me introduce myself. I have been in free software for almost a decade.
My previous work as working on Scilab (from Inria to Scilab Enterprises) where I stayed for almost 8 years. Over the years, I worked on various development (C, C++, Java, Python, Scilab, etc), user experience, roadmap definition, release management, R and D / customer project management, management, board member, etc. During this period, I learnt tons of things from coding, release and product management, QA, etc. In parallel, I worked part time at IRILL, a free software research center (hosted by Inria). I was involved as community management ie organizing event like GNU Hacker meeting, FOSDEM, Euro LLVM, etc and recording them. I had the chance to meet plenty of people from the FLOSS world (including folks from Mozilla who suggested me to join the adventure). I am also involved in two communities: Debian and LLVM. The first one is Debian. I started to contribute in Debian to get Scilab properly packaged. 7 years
after, I am now maintaining
important packages
like LLVM, Clang, libedit, emscripten (!) or BLAS/LAPACK, treasurer of the French representative association and organizing the French mini Debian Conference (Debconf) / Meetup. For the last couple of years, I contributed to LLVM, Clang and LLDB. I participated to the release 3.3 and 3.4. My technical contributions were mainly driven by the packaging, the nightly Debian/Ubuntu packages and also by the rebuild of the Debian archive using Clang (instead of gcc). And when I am not behind my laptop, I am teaching scuba diving with my girlfriend or hiking, mountaineering, ski touring, climbing, etc. Yeh, I am a bit hyperactive.

17 January 2014

Sylvestre Ledru: Debian & LLVM events

Being a bit hyperactive, I have been involved in the organization of two events. I am the main organizer with Alexandre Delano of the Mini Debconf 2014 in Paris, January 18 & 19th. The (great) planning is available here:
https://france.debian.net/events/minidebconf2014/
Saturday morning presentations will be general public, the beginning of Saturday afternoon will be used by the Debian France association to vote the new status (1901 law and Debian Trusted Organization).
Sunday will be more focused on Debian itself.
During the week end, I will be talking about the Debile project, the finance of Debian France and be part of the round table on compiler selection for Debian.
The (mandatory) registration should be done on the Wiki or meetup.com In parallel, with Tobias Grosser, we organized the LLVM devroom track at FOSDEM (Bruxelles), February 2nd (Sunday).
The schedule is a mix between core developers, third party software using LLVM / Clang and academic users.
https://fosdem.org/2014/schedule/track/llvm/
I will be talking on how to become a LLVM contributor. Both events should be recorded.

15 January 2014

Sylvestre Ledru: Debian France shop

During the month of December, we have been working with folks from En Vente Libre to create a shop with Debian products: http://www.enventelibre.org/debian We have now a shop with plenty of Debian products. The design has been done by Ahmet DEMIR. Most of them are already known by the community (they were sold during FOSDEM or various Debconf) like the Buff, Swiss Army knife or the classical polos. But, t-shirts based on Wheezy theme are brand new:
They are available in three colors:
* Red
* Blue
* Grey For now, the website is only French, an English version might come in the future.

Sylvestre Ledru: Some thoughts concerning LLVM & Clang and their evolutions: Release of LLVM & Clang 3.4

We just released LLVM & Clang 3.4 (already available in Debian Jessie and Ubuntu Trusty). The clang new release introduces some more warning and error detections. The point of the blog post is not to detail the changes (LLVM / Clang) but more to take a step back on the LLVM and Clang toolchains, usages and dissemination. For the last couple years, I have been presenting our work on the Debian side to make Clang part of the Debian infrastructure. I add the chance to speak at many events (Debconf 12 & 13, FOSDEM 2013, Linux Plumbers 2013, Distro recipes, etc). I even had Linus Torvalds attending to one of these talks (and proposing new approach). I had many feedback from the Debian/Ubuntu communities but also from other communities.
More and more projects are using Clang as part of their workflows, in term of development, production and QA. For example, some of them are running continuous integration using clang instead of gcc, others are running scan-build (Wireshark or LLVM itself) or Address Sanitizer (asan) (Firefox for example), etc.
More and more actors are moving from gcc to clang for their developments (Sony with their PS4, Chrome under Mac OS X, Apple, the Linux Kernel with the LLVMLinux project, FreeBSD, etc).
I stopped counting the number of times when people answered me "Yeh, I am now using Clang" when I said I am involved upstream. In parallel, more and more projects are using LLVM as a backend or computing engine and, in the meantime, getting more and more attraction (not because they are based on LLVM but just because they are great projects). For example, Mesa (llvmpipe), Julia, Rust, Emscripten or Native Client (NaCL) are excellent examples of this. The two last are excellent proofs that LLVM based technologies have an impact which goes further than geek-tools and will be used on a daily basis by millions of users. I also saw an interesting and growing number of new projects using libclang and libTooling. My feeling is that the lack of plugin capabilities in gcc for a long time and, now, its current complexity limited the innovation in term of compilation capabilities, analysis and fun tools. The features and the quality of this library are enabling new developments and I am sure we will see more and more excellent tools based on this library during 2014.
For example, the LLVM toolchain ships Clang modernize and clang format and some cool projects like include-what-you-use, DXR, OCLint or creduce are emerging. Thanks to this great competition, gcc is also improving (ASAN, JIT, better warnings, etc), remaining still relevant as before. No doubt that 2014 will be a great year for compilers.

6 January 2014

Rapha&#235;l Hertzog: My Free Software Activities in December 2013

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (147.56 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. The Debian Administrator s Handbook Cover of the Debian Administrator's Handbook (Wheezy edition)I spent a good chunk of December on the book. First finalizing the English version and getting it out (BTW, just for the launch, there s a 10% discount on the paperback that lasts only until January 9th!). Then working on updating the French translation. Eyrolles will publish a new edition of the French book based on this translation. Expect some further news about this during January! Debian France I contributed to many discussions within Debian France. Starting with a complaint that most events are organized in Paris, I proposed to map the location of Debian France members. We added new fields in the membership management page so that members can add their GPS coordinates and Fr d ric Decou made some experiments with Openstreetmap. Someone else (Kiriarat) volunteered to write the required glue code. A manual map is currently maintained on the website. Sample logo receivedIn the discussions about the setup of the Debian France shop, I suggested to update our logo with a nicer looking one. We got a few suggestions and after further discussions with Alexandre Delano and Sylvestre Ledru, we organized a small contest to entice designers to submit a logo proposal to us (the winner earns a set of Debian goodies). We got 46 proposals (see my favorite on the right)! The board is currently pre-selecting the logos and setting up the final vote for our members. The winner shall be announced at the end of the upcoming mini-debconf in Paris. I also continued the work to finalize new bylaws and new internal rules. They shall be adopted during the next general assembly which will happen during the mini-debconf. Misc Debian Work WordPress maintenance. I mentored Pablo Vasquez to do his first small contribution to the WordPress packaging. I really appreciate this but he s not yet ready to assume maintenance of a big package like WordPress on his own. I got multiple other offers of help and pinged them all while filing #733726 to coordinate the work on the new upstream version. But I got no reply :-( Handing over packages to new maintainers is hard Init system discussion. The technical committee has the hard task of picking the default init system that will replace the traditional System V init (see #727708). I followed this huge discussion closely and contributed a bit where I add something meaningful to say. Final decision is expected sometimes in January. FWIW, I share entirely Russ Allbery s point of view in those discussions. I have been running systemd on some of my computers for a few months already. Fixing lxc in stable. The lxc package in stable has a non-working debian template. I really dislike documenting that things are broken so instead of doing that in the Debian Administrator s Handbook, I opted to do something about it. I prepared a non maintainer upload for stable (see #680469 for the problem and #732358 for the stable update request). Misc stuff. I sponsored a tcpdf upload. I filed an enhancement request on Publican to have it keep processing instructions present in translations. I uploaded new versions of publican-librement and debian-handbook. I filed #732678 against git-buildpackage because it failed to properly call lintian when given the -A dpkg-buildpackage argument. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

1 December 2013

Rapha&#235;l Hertzog: My Free Software Activities in November 2013

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (44.52 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. The Debian Administrator s Handbook Wheezy update completed. Roland and I completed the update of the Debian Administrator s Handbook for Debian 7 Wheezy. We still have some proofreading work to do but you can already enjoy the result here: http://debian-handbook.info/browse/wheezy/ Feel free to report back any problem that you discover. You can also submit us patches ready to apply if you want to go one step further. Publican contributions. The book is generated with publican and I maintain its Debian package. This month I got a release critical bug because it stopped working it turns out that the problem lied in libxml-treebuilder-perl and I thus reassigned #728885 while providing a tentative patch to the upstream author. After a few days without action from the pkg-perl team, and after having received a FTBFS bug on debian-handbook (of course publican was broken in unstable!), I prepared a fixed package myself and I uploaded it (I m still part of the pkg-perl team although I m inactive). Since I used publican heavily this month, I filed two tickets in its bugzilla. I requested a new feature in #1034836 (the possibility to keep around the former string for fuzzy strings to update), and I reported a problem with the handling of \n in PO files in #1036150. Debian France Galette update. I updated the galette package and its paypal plugin, and I deployed those on france.debian.net. It had some fixes for the reminder mails sent to members. Bylaws update. I also resumed my work on preparing new bylaws for Debian France. Sylvestre Ledru came up with a draft (with the help of a lawyer) a few months ago and I m reviewing/improving them now. The main goal is to clarify that Debian France is meant to be a Trusted Organization for the Debian project. Debian France Shop. We had the idea since a few months already but Sylvestre did the leg work to open a Debian France shop with the help of EnVenteLibre. I asked our members to prepare some CSS that better match the Debian colors and this should be fixed in a few days. The first goodies will also start to appear shortly, just in time for Christmas! Misc Debian work Distro Tracker. In the continuation of the Google Summer of Code, I asked the DSA team to setup a new virtual machine to host tracker.debian.org, an instance of Distro Tracker, the rewritten Package Tracking System. They have done their part of the job (except the mail setup), it s now waiting on me to find some time to complete some cleanups and deploy the thing. WordPress. I packaged wordpress 3.7.1 and sent a call for help on debian-mentors. I got 3 replies, I gave them some initial direction but I haven t heard back anything since. WordPress 3.8 is expected in a few days, hopefully one of the new volunteers will take care of preparing the next update. Dpkg regressions. I haven t done anything for multiple months but at least I keep running the git version of dpkg and I detected two regressions. Good to have them squashed before the upcoming 1.17.2 upload to unstable. PTS fix. I fixed some warnings that the PTS code started generating since the upgrade of its host to wheezy. They were generating some annoying backscatter mails to users of the pts@qa.debian.org bot. Ruby security update. I helped the ruby team to prepare the required security updates of ruby1.8 and ruby1.9.1 (see #730178 and #730189). This work was sponsored by Kali/Offensive Security. Smartcard setup. I bought 2 OpenPGP smartcards with a reader and I moved all my private keys on those devices (one card with the master key for signature/certification to be kept at home, one card for daily/mobile usage with the subkeys for encryption/signature/authentication). My laptop s harddrive doesn t contain any private key anymore. I have kept the required offline backup in a safe place, but in the end, my private keys are much harder to steal. I should write down my findings in another article Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

7 September 2013

Sylvestre Ledru: Include What You Use - Yet a new cool tool in Debian.

iwyu (Include What You Use) has just been accepted in Debian. This tool, based on LLVM/Clang, checks the headers of a C/C++ file. It will detect which include directives are useless, what could be improve and add comments to explain why #include are there. Let's take this C file:

#include <stdio.h>
#include <malloc.h>
#include <limits.h>

int foo(void)
fprintf(stderr, "an error\n");
return INT_MAX;

$ iwyu foo.c will return:

foo.c should remove these lines:
- #include <malloc.h> // lines 2-2

The full include-list for foo.c:
#include <limits.h> // for INT_MAX
#include <stdio.h> // for fprintf, stderr
---
Package on Debian

6 September 2013

Sylvestre Ledru: About optimization flags in software

Following a thread on the Clang mailing list, I did a quick analysis of the optimization flags used to build C or C++ code. I used a rebuild result from last mid July of all Debian native packages (meaning that I don't have the log of full Java, Python or other packages without native code) and wrote a small script to count all their occurrences.
By default, deb packages uses the command "dpkg-buildflags --get CFLAGS" which will set the optimization level to -O2. However, many packages and upstream build systems do not respect such flags. On 10320 binary-only packages (in total, Debian has around 19000), the results are the following:
Optimization levelNumber of occurrencesNumber of packages
-O35597269
-Os30849145
-O013993294
-O125494809
-O29958037685
-O3106048531
-O4140216
-O500
-O6693549
Since a graphic talks more than numbers (note that the Y axis is logarithmic):
Disclaimer:
Some packages are using several optimization flags during the same build.
Some packages are not showing the full command line (see jessie release goal: verbose build logs)
The results might be approximate since I am basically grepping "-OX " on the log files.

Next.