Search Results: "thibaut"

14 February 2016

Lunar: Reproducible builds: week 42 in Stretch cycle

What happened in the reproducible builds effort between February 7th and February 13th 2016:

Toolchain fixes
  • James McCoy uploaded devscripts/2.16.1 which makes dcmd supports .buildinfo files. Original patch by josch.
  • Lisandro Dami n Nicanor P rez Meyer uploaded qt4-x11/4:4.8.7+dfsg-6 which make files created by qch reproducible by using a fixed date instead of the current time. Original patch by Dhole.
Norbert Preining rejected the patch submitted by Reiner Herrmann to make the CreationDate not appear in comments of DVI / PS files produced by TeX. He also mentioned that some timestamps can be replaced by using the -output-comment option and that the next version of pdftex will have patches inspired by reproducible build to mitigate the effects (see SOURCE_DATE_EPOCH patches) .

Packages fixed The following packages have become reproducible due to changes in their build dependencies: abntex, apt-dpkg-ref, arduino, c++-annotations, cfi, chaksem, clif, cppreference-doc, dejagnu, derivations, ecasound, fdutils, gnash, gnu-standards, gnuift, gsequencer, gss, gstreamer0.10, gstreamer1.0, harden-doc, haskell98-report, iproute2, java-policy, libbluray, libmodbus, lizardfs, mclibs, moon-buggy, nurpawiki, php-sasl, shishi, stealth, xmltex, xsom. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:
  • #813944 on cvm by Reiner Herrmann: remove gzip headers, fix permissions of some directories and the order of the md5sums.
  • #814019 on latexdiff by Reiner Herrmann: remove the current build date from documentation.
  • #814214 on rocksdb by Chris Lamb: add support for SOURCE_DATE_EPOCH.

reproducible.debian.net A new armhf build node has been added (thanks to Vagrant Cascadian) and integrated into the Jenkins setup for 4 new armhf builder jobs. (h01ger) All packages for Debian testing (Stretch) have been tested on armhf in just 42 days. It took 114 days to get the same point for unstable back when the armhf test infrastructure was much smaller. Package sets have been enabled for testing on armhf. (h01ger) Packages producing architecture-independent ( Arch:all ) binary packages together with architecture dependent packages targeted for specific architectures will now only be tested on matching architectures. (Steven Chamberlain, h01ger) As the Jenkins setup is now made of 252 different jobs, the overview has been split into 11 different smalller views. (h01ger)

Package reviews 222 reviews have been removed, 110 added and 50 updated in the previous week. 35 FTBFS reports were made by Chris Lamb, Danny Edel, and Niko Tyni.

Misc. The recordings of Ludovic Court s' talk at FOSDEM 16 about reproducible builds and GNU Guix is now available. One can also have a look at slides from Fabian Keil's talk about ElecrtroBSD and Baptiste Daroussin's talk about FreeBSD packages.

20 November 2015

Jonathan Dowland: smartmontools

It's been at least a year since I last did any work on Debian, but this week I finally uploaded a new version of squishyball, an audio sample comparison tool, incorporating a patch from Thibaut Girka which fixes the X/X/Y test method. Shamefully Thibaut's patch is nearly a year old too. Better late than never... I've also uploaded a new version of smartmontools which updates the package to the new upstream version. I'm not the regular maintainer for this package, but it is in the set of packages covered by the collab-maint team. To be polite I uploaded it to DELAYED-7, so it will take a week to hit unstable. I've temporarily put a copy of the package here in the meantime.

20 June 2015

Lunar: Reproducible builds: week 4 in Stretch cycle

What happened about the reproducible builds effort for this week: Toolchain fixes Lunar rebased our custom dpkg on the new release, removing a now unneeded patch identified by Guillem Jover. An extra sort in the buildinfo generator prevented a stable order and was quickly fixed once identified. Mattia Rizzolo also rebased our custom debhelper on the latest release. Packages fixed The following 30 packages became reproducible due to changes in their build dependencies: animal-sniffer, asciidoctor, autodock-vina, camping, cookie-monster, downthemall, flashblock, gamera, httpcomponents-core, https-finder, icedove-l10n, istack-commons, jdeb, libmodule-build-perl, libur-perl, livehttpheaders, maven-dependency-plugin, maven-ejb-plugin, mozilla-noscript, nosquint, requestpolicy, ruby-benchmark-ips, ruby-benchmark-suite, ruby-expression-parser, ruby-github-markup, ruby-http-connection, ruby-settingslogic, ruby-uuidtools, webkit2gtk, wot. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which did not make their way to the archive yet: Also, the following bugs have been reported: reproducible.debian.net Holger Levsen made several small bug fixes and a few more visible changes: strip-nondeterminism Version 0.007-1 of strip-nondeterminism the tool to post-process various file formats to normalize them has been uploaded by Holger Levsen. Version 0.006-1 was already in the reproducible repository, the new version mainly improve the detection of Maven's pom.properties files. debbindiff development At the request of Emmanuel Bourg, Reiner Herrmann added a comparator for Java .class files. Documentation update Christoph Berg created a new page for the timestamps in manpages created by Doxygen. Package reviews 93 obsolete reviews have been removed, 76 added and 43 updated this week. New identified issues: timestamps in manpages generated by Doxygen, modification time differences in files extracted by unzip, tstamp task used in Ant build.xml, timestamps in documentation generated by ASDocGen. The description for build id related issues has been clarified. Meetings Holger Levsen announced a first meeting on Wednesday, June 3rd, 2015, 19:00 UTC. The agenda is amendable on the wiki. Misc. Lunar worked on a proof-of-concept script to import the build environment found in .buildinfo files to UDD. Lucas Nussbaum has positively reviewed the proposed schema. Holger Levsen cleaned up various experimental toolchain repositories, marking merged brances as such.

12 September 2012

Thibaut Girka: State of Multiarch Cross-toolchains three weeks after GSoC

Hi, It's been three weeks already since the end of the Google Summer of Code 2012, and I've been told I still haven't blogged about the state of multiarch cross-toolchains. So, here it is! Building multiarch cross-toolchains This section is mostly a rehash of my previous post. Cross-toolchains build properly for most architectures in Debian, with the exception of arches having clashing ld.so paths. The following instructions assume you are using Wheezy or unstable, and have applied my patches. Adding a foreign architecture You are going to use some packages meant for a different architecture than your native one. This means you'll have to tell dpkg about a foreign architecture, for instance, armel. 
# dpkg --add-architecture armel
Then, you need to update apt's database. 
# apt-get update
If one of the repository you use only supports a subset of your configured architecture (for instance, when working with an unofficial port), you will need to use arch restrictions in your sources.list (see the manpage). Cross-binutils Absolutely nothing has changed there, it works exactly the same way it did before the GSoC. That is, grab binutils' source package, set $TARGET to a GNU triplet or a Debian arch, and run dpkg-buildpackage. 
$ apt-get source binutils
$ cd binutils-*/
$ TARGET=arm-linux-gnueabihf dpkg-buildpackage -uc -us -b
Cross-gcc Building cross-gcc is done the same way it was before the GSoC, except it uses cross-arch dependencies instead of mangled packages for foreign packages. This means you shouldn't use dpkg-cross at all, but use multiarch instead. Then, building the cross-compiler is as easy as running (with $target being a Debian arch name): 
$ export DEB_CROSS_NO_BIARCH=yes
$ echo $target > debian/arch
$ debian/rules control
$ dpkg-buildpackage -uc -us -b
Biarch/multilib and multiarch do not really play well. Using some trickery, you might be able to build multilib cross-compilers, but I would recommend building two cross-compilers instead. You will of course need to install the required build-dependencies, some of which are packages for the target architecture, but that is no different from building any other Debian package, provided that multiarch is correctly configured. This will produce cross-compilers packages using multiarch paths and cross-arch deps, along with some binary packages for the target architecture. Those binary packages are cross-built versions of the runtime libs. They are not needed, as they should be identical to the natively-built ones present in the archive, but using them shouldn't be a problem. Installing the cross-toolchains Installing cross-binutils is straightforward, installing cross-gcc itself is, but installing cross-g++ is not. Indeed there is bug #678623: g++-4.7 depends on libstdc++6-4.7-dev, which is multiarch-unaware and depends on g++-4.7. Assuming you've compiled your cross-gcc using my patches, the cross-compiled libstdc++6-4.7-dev will not have these problems, but still won't be co-installable with your native libstdc++6-4.7-dev, since it's not M-A: same. The easiest way to work around this issue is probably to download the binary package, modify its control file to include Multi-Arch: same , and install the modified package. Using the cross-toolchains Once installed, using the cross-toolchains should be as easy as running $triplet-gcc-4.7 file.c . Note the -4.7 part. Indeed, the cross-compilers packages do not provide the $triplet-gcc symlink. This matches what is done with the native compiler pase, where the gcc symlink is not provided by the compiler's package, but by the gcc package. However, there is no cross-gcc package or anything like that yet, which means you have to use the full name, or provide the symlink yourself. Another issue is pkg-config. To know where to search for libraries, pkg-config has to be told which architecture you are compiling for. Autotools does that by calling $triplet-pkg-config instead of pkg-config if it exists. The pkg-config binary packages comes with a wrapper you can symlink to: /usr/share/pkg-config-crosswrapper . Conclusion Despite a few shortcomings (bug #678623, missing symlinks, multilib issues and clashing ld.so paths), multiarch cross-toolchains are already usable on Wheezy. There is still quite a lot of work left (like fixing #678623, splitting gcc-4.7 in two, or changing wanna-build, britney etc. to handle cross-arch deps) but hopefully this will be done for Jessie.

2 July 2012

Thibaut Girka: apt-get install gcc-4.7-arm-linux-gnueabihf

Hi! As you may know, I'm working on multiarch cross-toolchains as part of the Google summer of Code (see my previous post for details), and although there is still a lot to do, the project is doing fairly well, and a cross-gcc is already installable and working. I'll proceed with detailed instructions on how to install the cross-compiler, along with some explanations on multiarch itself, and how to build cross-toolchains from source. Prerequisites First, you need specific versions apt (>= 0.9.6) and dpkg (>= 1.16.5).
This is because the cross-toolchains packages I'm working on depend on specific cross-arch dependencies, such as libc6:armhf. Support for such dependencies has been added in the aforementioned versions of apt and dpkg. This means you should be running unstable, or at least use dpkg from unstable. Adding a foreign architecture Adding a foreign architecture is straightforward, but I've seen users do overcomplicated, or just plain wrong things. So, let's do this step by step (well, there are only two of them), with some explanations.
  1. Tell dpkg to handle an additional architecture by typing dpkg --add-architecture armhf. There isn't much to say. It will only add an architecture to the list of architectures dpkg would accept to handle.
  2. Update APT's database. You need to run apt-get update after adding or removing an architecture to get package lists for all wanted architectures and rebuild APT's database, including the implicit cross-arch dependencies/conflicts brought by multiarch.
And that's it for the common case. Now, a word on architecture restrictions: a common misconception about multiarch seems to be that you have to tell APT which architectures you want from the mirrors by modifying your sources.list file(s). This is not true, as, by default, APT gets the list of accepted architectures from dpkg, and gets the correct package lists from the mirrors configured in the sources.list. You only need to use arch restrictions ( deb [arch=arch1,arch2,...] ... ) when you want to get packages from a mirror only for a subset of your accepted architectures. This is very unlikely, and in the general you do not need to modify your sources.list, at all. Getting the cross-compilers from my repository I provide compiled versions of gcc-4.7-arm-linux-gnueabihf for i386 and amd64 in my repository. If you have followed the previous paragraphs, nothing should be easier than adding a deb http://emdebian.org/~thibg/repo/ sid main line to your sources.list and doing the following: 
# apt-get update
[...]
# apt-get --no-install-recommends install gcc-4.7-arm-linux-gnueabihf
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  binutils-arm-linux-gnueabihf cpp-4.7-arm-linux-gnueabihf gcc-4.7-arm-linux-gnueabihf-base gcc-4.7-base:armhf libc6:armhf libgcc1:armhf
  libgmp10 libgomp1:armhf libmpc2 libmpfr4
Suggested packages:
  binutils-doc gcc-4.7-locales libmudflap0-4.7-dev:armhf gcc-4.7-doc libgcc1-dbg:armhf libgomp1-dbg:armhf libitm1-dbg:armhf
  libquadmath-dbg:armhf libmudflap0-dbg:armhf binutils-gold glibc-doc:armhf locales:armhf
Recommended packages:
  libc6-dev:armhf
The following NEW packages will be installed:
  binutils-arm-linux-gnueabihf cpp-4.7-arm-linux-gnueabihf gcc-4.7-arm-linux-gnueabihf gcc-4.7-arm-linux-gnueabihf-base gcc-4.7-base:armhf
  libc6:armhf libgcc1:armhf libgmp10 libgomp1:armhf libmpc2 libmpfr4
0 upgraded, 11 newly installed, 0 to remove and 0 not upgraded.
Need to get 23.5 MB of archives.
After this operation, 57.2 MB of additional disk space will be used.
Do you want to continue [Y/n]?
I've disabled the recommends since, libc6-dev:armhf would recommend gcc:armhf itself, which is not co-installable with your native gcc if you have it installed. Building the cross-toolchain yourselves If you don't trust the repository above, or simply want to rebuild the toolchain yourself, it is rather simple: get my gcc-4.7 patches and: 
$ apt-get source binutils
$ cd binutils-*/
$ TARGET=arm-linux-gnueabihf dpkg-buildpackage -uc -us -b
$ cd ../
# dpkg -i binutils*arm-linux-gnueabihf*.deb
$ apt-get source gcc-4.7
$ cd gcc-4.7/
$ for patch in path/to/the/patches/*.patch; do patch -p1 < $patch ; done
$ GCC_TARGET=armhf debian/rules control
$ GCC_TARGET=armhf dpkg-buildpackage -uc -us -b
Using the cross-compiler It's easy, but there is a trick, and dpkg-buildpackage -aarmhf won't work out of the box yet, as gcc-4.7-arm-linux-gnueabihf provides the arm-linux-gnueabihf-gcc-4.7 executable, but no arm-linux-gnueabihf-gcc symlink. So, for the moment: 
$ cat > test.c << __EOF__
#include <stdio.h>
int main(void)
 
  printf("Hello world\\n");
  return 0;
 
__EOF__
$ arm-linux-gnueabihf-gcc-4.7 test.c
$ qemu-arm ./a.out 
Hello world
Conclusion The toolchain is not complete yet (there is some more work to do on binutils and libstdc++, and equivalents to the gcc , cpp , and build-essential packages to provide), but it should still be useful to cross-compile C programs. Multi-arch cross-toolchains for wheezy can definitely be hosted at emdebian, and I hope to clean up the build process so that it could get included in the archive for wheezy+1.

3 June 2012

Johannes Schauer: cross-compilable and bootstrappable Debian

When packaging software for Debian, there exist two important assumptions:
  1. Compilation is done natively
  2. Potentially all of Debian is available at compile time
Both assumptions make the life of a package maintainer much easier and they do not create any problem unless you are one of the unlucky few who want to run Debian on an architecture that it does not yet exist for. You will then have to use other distributions like OpenEmbedded or Gentoo which you compiled (or retrieved otherwise) for that new architecture to hack a core of Debian source packages until they build a minimal Debian system that you can chroot into and continue natively building the rest of it. But even if you manage to get that far you will continue to be plagued by cyclic build and runtime dependencies. So you start to hack source packages so that they drop some dependencies and you can break enough cycles to advance step by step. The Debian ports page lists 24 ports of Debian, so despite its unpleasant nature, porting it is something that is not done seldom. The process as laid out above has a number of drawbacks: If Debian would provide a set of core packages that are cross-compilable and which suffice for a minimal foreign build system, and if it would also have enough source packages that provide a reduced build dependency set so that all dependency cycles can be broken, building Debian for a yet unknown architecture could be mostly automated. The benefits would be: With three of this year's GSoC projects, this dream seems to come into reach. There is the "Multiarch Cross-Toolchains" project by Thibaut Girka and mentored by Hector Oron and Marcin Juszkiewicz. Cross-compiling toolchains need packages from the foreign architecture to be installed alongside the native libraries. Cross-compiler packages have been available through the emdebian repositories but always were more of a hack. With multiarch, it is now possible to install packages from multiple architectures at once, so that cross-compilation toolchains can be realized in a proper manner and therefor can also enter the main archives. Besides creating multiarch enabled toolchains, he will also be responsible for making them build on the Debian builld system as cross-architecture dependencies are not yet supported. There is also the "Bootstrappable Debian" project by Patrick "P. J." McDermott and mentored by Wookey and Jonathan Austin. He will make a small set of source packages multiarch cross-compilable (using cross-compilers provided by Thibaut Girka) and add a Build-Depends-StageN header to critical packages so that they can be built with reduced build dependencies for breaking dependency cycles. He will also patch tools as necessary to recognize the new control header. And then there is my project: "Port bootstrap build-ordering tool" (Application). It is mentored by Wookey and Pietro Abate. In contrast to the other two, my output will be more on the meta-level as I will not modify any actual Debian package or patch Debian tools with more functionality. Instead the goal of this project is threefold:
  1. find the minimal set of source packages that have to be cross compiled
  2. help the user to find packages that are good candidates for breaking build dependency cycles through added staged build dependencies or by making them cross-compilable
  3. develop a tool that takes the information about packages that can be cross compiled or have staged build dependencies to output an ordering with which packages must be built to go from nothing to a full archive
More on that project in my follow-up post.

28 May 2012

Thibaut Girka: Multiarch Cross-toolchains

Hi! I have been selected to work on multiarch cross-toolchains as part of the Google summer of Code. Although coding officially started last week, I have been busy with exams and other uni stuff at the time. Anyway, let's talk about the project itself, that is, multiarch cross-toolchains. What is multiarch? In order to understand what the project is about, one need to know what multiarch is. I'll quickly describe it, and I invite anyone interested to read the spec (really, it's interesting, and not that long). Basically, multiarch lets you install binary packages for multiple architectures on one system, and let package managers resolve dependencies as one would expect. This works by identifying four sets of packages: Anyway, multiarch is an amazing thing that may be of use in various cases, such as running i386-only software on an amd64 system, running armel software on an armhf system, running i386 software on an armhf system using qemu-user-static and binfmt, cross-grading (switching, for instance, from i386 to amd64 on a running system), and, in our case, easing cross-compilation. State of cross-toolchains in Debian There aren't really cross-toolchains available in Debian (although some can be built from source packages), but they are available in emdebian's repositories. However, those packages predate multiarch, and don't make use of it. Instead, they depend on special packages converted using dpkg-cross. Such packages are merely arch: all versions of foreign packages, with files moved to arch-qualified paths to prevent conflicts with normal packages. Such packages thus have to be generated from real packages, either manually or using some sort of automation. In addition, this additional step brings several issues, like keeping in sync converted packages with real packages from the archive. With multiarch, such packages are rendered useless, and may in fact bring additional problems, since converted and multiarch packages may share the same files, and thus may not be co-installable. Switching from dpkg-cross to cross-arch deps Cross-toolchains packages currently depend on a few packages converted using dpkg-cross (ex: libc6-$ARCH-cross and libc6-dev-$ARCH-cross for some packages). They need to depend on the real package with the right arch (ex: libc6:$ARCH, libc6-dev:$ARCH). The obvious way to do so is by annotating the dependency with a specific arch qualifier (ex: Depends: libc6:armhf ). This is, however, not covered by the multiarch spec. Note that some lib*-$ARCH-cross packages will remain (for instance, libgcc1-armhf-cross). Indeed, those packages are different from the packages mentioned above in that they are of the architecture of the build system and actually part of the toolchain itself.
The above isn't actually true. Indeed, those packages are runtime library cross-built during gcc's build. Likewise, cross-toolchains may build-depend on foreign packages (ex: libc6-dev). Thus, specific arch qualifiers must be handled in the Build-Depends source package field too. Changing dpkg and apt to support that should be fairly straightforward. In fact, I already have a few patches, but I need to refresh them, maybe clean them up a little, and push them a little harder to get them accepted!

27 April 2012

Ana Beatriz Guerrero Lopez: Debian in the Google Summer of Code 2012

This year our efforts have paid off and despite there being more mentoring organizations than there were in 2011 (175 in 2011 and 180 in 2012), this year in Debian we got 81 submissions versus 43 submissions in 2011.
You can see here the graphs of applications against time from this year: 2012 The result is this year we ll have 15 students in Debian versus 9 students last year! Without further ado, here is the list of projects and student who will be working with us this summer: If you want to know more about these projects, follow the links and ask the students (and mentors)!

26 February 2012

Gregor Herrmann: RC bugs 2012/08

here comes my usual report about my work on RC bugs. this time a bit more "added a comment" or "sent a patch" but also some uploads.

4 March 2011

Cyril Brulebois: Debian XSF News #7

Time for a seventh Debian XSF News issue!

19 September 2010

Obey Arthur Liu: Google Summer of Code 2010 Debian Report

Hello fellow developers, The summer is over :( but I m happy to announce that this year s Summer of Code at Debian has been better than ever! :) This is indeed the 4th time we had the privilege of participating in the Google Summer of Code and each year has been a little different. This year, 8 of our 10 students succeeded in our (very strict!) final evaluations, but we have reasons to believe that they will translate into more long-term developers than ever, all thank to you. The highlight this year has been getting almost all of our students at DebConf10. Thanks again this year to generous Travel Grants from the Google Open Source Team, we managed to fly in 7 of our students (up from 3!). You certainly saw them, presenting during DebianDay, hacking on the grass of Columbia, hacking^Wcheering our Debian Project Leader throwing the inaugural pitch of a professional baseball game or hacking^Wsun-tanning on the tr s kitsch Coney Island beach. Before I give the keyboard to our Students, I d like to tell you that it will be the pleasure and honor of Obey Arthur Liu (yours truly, as Administrator) and Bastian Venthur (as Mentor) to represent Debian at the Summer of Code 2010 Mentors Summit on 23-24 October 2010, at the Google Headquarters in Mountain View. Like last year, we expect many other DDs to be present under other hats. We will be having 2 days of unconference on GSoC and free software related topics. We all look forward to reporting from California on Planet and soc-coordination@l.a.d.o! All of our students had a wonderful experience, even if they couldn t come to DebConf, that is best shared in their own voice, so without further ado, our successful projects: Multi-Arch support in APT by David Kalnischkies, mentored by Michael Vogt apt-get install MultiArch does mostly work now as most code is already merged in squeeze, but if not complain about us at deity@l.d.o! Still, a lot left on the todo list not only in APT so let us all add MultiArch again to the Release Goals and work hard on squeezing it into wheezy. :) Debbugs Bug Reporting and Manipulation API by David Wendt Jr., mentored by Bastian Venthur Hello, I m David Wendt, and I went to Debconf10 to learn more about the development side of Debian. Having used it since the 9th grade, I ve been intimately familiar with many of Debian s internals. However, I wanted to see the developers and other Debian users. At DebConf, I was able to see a variety of talks from Debian and Ubuntu developers. I also got to meet with my mentor as well as the maintainer of Debbugs. Content-aware Config Files Upgrading by Krzysztof Tyszecki, mentored by Dominique Dumont Config::Model is now capable of manipulating files using shorter and easier to write models. Thanks to that, packagers may start experiment with creating upgrade models. Further work is needed to support more complicated config files Dominique Dumont is working on DEP-5 parser, I ll shortly start working on a cupsd config file parser.
The best thing about DebConf10 is that every person I talked with knew what I was doing. I had a mission to get some feedback on my project. Everybody liked the idea of making upgrades less cumbersome. On the other side, it was my first visit to United States, so I decided to go on a daytrip on my own (instead of staying inside the building, despite heat warnings). I had a chance to visit many interesting places like Ground Zero, the UN headquarters, Grand Central Terminal, Times square and Rockefeller Center that was a great experience. Hurd port and de-Linux-ization of Debian-Installer by J r mie Koenig, mentored by Samuel Thibault Debconf10 was great! Among other people working on the installer, I met Aur lien Jarno from the Debian/kFreeBSD team and we worked together on a cross-platform busybox package. Besides, the talks were very interesting and I ve filled my TODO-list for the year.
For instance I learned about the Jigsaw project of OpenJDK, and how Debian would be the ideal platform to experiment with it. More generally, some people think Debian could push Java 7 forward and I d like to see this happen. Smart Upload Server for FTP Master by Petr Jasek, mentored by Joerg Jaspert I must say that it was great time for me in NY, I ve met and talked and coded with people from ftp-master team like Torsten Werner who helped me to push the project a bit further and with some other people who were looking forward to release of the tool which I hope they will use quite soon. Everybody interested, everybody excited, really cool place and time. And I can t forget the Coney Island beach and stuff, lot of fun, lot of sun;) Aptitude Qt by Piotr Galiszewski, mentored by Sune Vuorela Currently, development branches support full features searching, viewing extended package s informations, performing cache and packages operations. Code and GUI still require a lot of work which will be continued. Informations about further progress could be found on aptitude mailing list and repository rss channel. Debian-Installer on Neo FreeRunner and Handheld Devices by Thibaut Girka, mentored by Gaudenz Steinlin For me, DebConf 10 started at the airport, where Sylvestre Ledru (whom I didn t know of before) was wearing a GSoC 2007 t-shirt, that is, given the circumstances, almost equivalent to say I m a hacker, I m going to DebConf 10 .
I ve spent my time at the conference attending various talks, hacking, meeting DDs and other hackers (amongst others, my co-mentor Per Andersson, Paul Wise, Julien Cristau, Christian Perrier, Cyril Brulebois, Martin Michlmayr, Colin Watson and Otavio Salvadores who I have to thank for his patience while dealing with my questions), chatting, cross-signing keys, rushing to finish eating before 7pm, getting sunburnt, sightseeing (thanks, Arthur, for the lightning-fast tour of Manhattan!), and so on. Debian Developers and community, we count on you. See you next year! (cross-posted to debian-devel-announce@l.d.o and soc-coordination@l.a.d.o)

16 August 2010

Thibaut Girka: End of GSoC

Hey! This year's GSoC is coming to an end, and it's great time for a quick update on the situation: Using my modified packages, you can install Debian, configure u-boot, and have a bootable and usable system. You can even use the graphical installer to do that! However, you can't really use the provided kernel for a smartphone use: no GPS, no GSM, no audio, no bluetooth... only SD and screen. Of course, I'll be working on upstreaming the missing parts in mainline kernel, but it'll probably take some time, and any help is welcome! Since almost nothing has been upstreamed yet, installing Debian is still a bit tricky, with the missing debs (you have to make a repository to store them). But hopefully, this will be sorted out shortly after the GSoC. Anyway, here are instructions to try out my work!

1 August 2010

Thibaut Girka: Sixth GSoC report

Hi! First, sorry not to have posted anything in the last two weeks! I was on vacation, and so, did less work on the project. But in no means it is dead, and development continues! First week of my vacation was (apart from resting, and vacation stuff) almost entirely dedicated to make my build environment work without internet access. Here, I was lucky to have made a complete debian/main mirror using reprepro before leaving! However, I had to fight a bit with debian-installer: first, for an unknown reason (if someone knows why, speak up!) it didn't search for the debian-installer section and I had to specify 'main/debian-installer' instead of 'main' in sources.list.udeb.local. Second, because I'm lazy, my reprepo repo isn't signed, and so, I had to pass some apt options to make when building d-i. After this first week, I've been able to work on uboot-installer, that you can find in pkg-fso/d-i.git. Originally written by my co-mentor Per, this package/script is meant to update the device's U-Boot env so it can boot Debian. It installs the uboot-envtools package, tries to read the U-Boot environment to make sure everything is ok, determines the new boot command, and shows/write it in the uboot env. When working on uboot env modification, I ran in a strange issue: I can read the env without problem using fwprintenv, modify it, and read it back, but U-Boot then fails, saying the environment is corrupted. In fact, this is because I've enabled CONFIGMTDNANDS3C2410_HWECC in config.s3c24xx, and it appears to cause invalid ECC values to be written on the flash. Switching it off solved the issue. Apart from that, I've made some minor modifications in the installation script, and I've (re)installed Debian countless times on my FR, and everything seems to work fine so far (although my kernel is non-installable at the moment because of version mismatch with what's in my repo). Building d-i for the FR So, you're welcome to test! Everything is available in the three (pkg-fso/d-i.git, pkg-fso/linux-2.6.git, and usbboot (hg)) repositories. You can rebuild the d-i image and packages the following way: 
mkdir ~/incoming/
cd d-i.git/packages/
cd flash-kernel && debuild -uc -us && cd .. && mv *.*deb ~/incoming
cd base-installer && debuild -uc -us && cd .. && mv *.*deb ~/incoming
cd hw-detect && debuild -uc -us && cd .. && mv *.*deb ~/incoming
cd libdebian-installer && autoreconf -i -v && debuild -uc -us && cd .. && mv *.*deb ~/incoming
cd partman/partman-auto && debuild -uc -us && cd .. && mv *.*deb ~/incoming && cd ..
cd arch/armel/uboot-installer && debuild -uc -us && cd .. && mv *.*deb ~/incoming && cd ../..
# For some reason, I had to rebuild installation-locale too
cd installation-locale && debuild -uc -us && cd .. && mv *.*deb ~/incoming
Then, copy everything that is in ~/incoming to your repo. Alternatively, you can use localudebs and add the following lines to pkg-lists/netboot/network-console/armel/s3c24xx.cfg: 
mmc-modules-$ kernel:Version 
mmc-core-modules-$ kernel:Version 
crc-modules-$ kernel:Version 
md-modules-$ kernel:Version 
ext2-modules-$ kernel:Version 
ext3-modules-$ kernel:Version 
flash-kernel-installer
base-installer
hw-detect
libdebian-installer4-udeb
partman-auto
uboot-installer
installation-locale
# For some reason, flash-kernel-installer brings live-installer in,
# disable it explicitly as we don't want it
live-installer -
Now that you have built every needed package, you can build the d-i image: 
make clean_s3c24xx_network-console
# The SECOPTS thing is only needed if you are using an unsigned repository
make build_s3c24xx_network-console SECOPTS="--allow-unauthenticated"
Now, you can find the ready-to-use u-boot image in dest/s3c24xx/network-console/uImage-multi! There are a few differences between the pre-built image and the one you have generated. Those differences are: the kernel in use (2.6.32 for the pre-built image, 2.6.34 if you follow the steps above), the network config (192.168.42.202 for the pre-built image, 192.168.0.202 for your own), security (unauthenticated repositories are allowed in the pre-built image!), and uboot-installer (some changes are still staging on my local working copy). Testing d-i on your FR Beware, those images are only for testing (especially the pre-built one, which accepts unauthenticated repositories). It should not harm your device, but use with care anyway. Now that you have built your u-boot image multi (if you don't want to, you can use the pre-built one), you are welcome to follow the steps described in the How to Boot article. Now, follow the instructions that appears on the FR screen and you should be able to install Debian like you would on any machine. The equivalent of grub-installer is divided into two steps (the two are needed, but each provides bootable-system, yes, that's a bug): flash-kernel ("Make the system bootable") and uboot-installer ("Modify U-Boot environment on flash"). (Note: SSH password is "install") What's coming next/has to be done? I've just arrived at DebConf 10, and some intensive testing should be done here. Apart from that, there are quite a few things left to do (right now, or in a distant future):

11 July 2010

Thibaut Girka: Fifth GSoC report

Ok, here is the (rather short) fifth report! First, and most important (but not so interesting, as I haven't had any review yet): I've submitted the glamo drivers usptream! There is still a long way to go before having a working Debian kernel on the FR, but I think it's still an important step (well, even more important would be its acceptance in mainline). Another thing is the uboot-envtools-udeb/uboot-installer thing. I haven't started it yet, but I have a rough idea of how to do that. Last but not least, the installation "script". It is meant to run on the host, and takes care of pushing necessary files to the FR and boot it, working around any issue ("slow typing" u-boot for instance). I've designed it in an extensible way, but now, I wonder if I've overdone it. However, it's here, and it's easy to strip down if we want something simpler.

1 July 2010

Thibaut Girka: Fourth GSoC report

This week's report is unfortunately even lesser interesting than the previous week, but it doesn't mean nothing has been done. The main work of this week have been to fix an outstanding bug with the glamo-mci driver. What I'll do next week will depend on the acceptance of my last patches in the OpenMoko kernel tree, but I'll probably submit the glamo drivers upstream. Other than that, I plan to work on flash-installer and uboot-envtools-udeb/uboot-installer, which has already been worked on by Per Andersson, to provide automatic u-boot configuration at the end of the installation process, in a way similar to what is done for GRUB on a regular PC.

25 June 2010

Thibaut Girka: Third GSoC report

Hi again! Sorry for the non-existent post of last week, but I didn't really have anything exciting to say. The project is still alive, and I'm still making (rather slow) progress. Since the last report, I've been reading, cleaning, and fixing the drivers for the Smedia Glamo (the multi-function device that does SD card access, display handling, and coffee), in order to submit them upstream and eventually backport them in Debian. Hopefully it'll soon be suitable for upstream and I'll be back to d-i itself! By the way, the git repositories are updated with a cleaner patchset and with framebuffer support!

9 June 2010

Thibaut Girka: Second week of GSoC

Hi, already two weeks (and a half)! Now, this time, I haven't made as much progress I would have wanted, but here is what have been done during this week (and a half): What I've been doing As a result, the Debian Installer works on the FreeRunner, you can install Debian on the FR using it, but it won't setup anything for the bootloader. Oh, and another important thing is that the GIT repos are now up and running! You can try the prebuilt image or build it yourself. Note that you'll probably want to use a custom repo with the modified udebs. An alternative is to modify pkg-lists/netboot/network-console/armel/s3c24xx.cfg to include them in the image. What's coming next Well, nothing is sure, but I may:

29 May 2010

Thibaut Girka: First week of GSoC

It's almost been a week since the GSoC started! Before describing what I've been doing this week I would like to write a word on the project and thank the ones I'm thankful to. The project Currently, Debian can be installed on the Neo Freerunner (FR) by running a shell script called install.sh on an already running Linux installation on the device (more information here). This works. However, it doesn't make use of the Debian Installer, and it is specific to the FR. The plan is to port Debian Installer to the FR and other similar devices. In order to succeed in this project, I'm mentored by Gaudenz Steinlin and Per Andersson, who I want to thank for their guidance (and the patience they'll probably need to have :p). I have to thank Obey Arhur Liu for all his organizing work, especially for the DebConf 10 sponsoring. While I'm at thanking, I want to thank Google, whithout which this program wouldn't exist at all, and especially Carol Smith, who makes an amazing job at organizing every aspect of the GSoC! What I've been doing Now, what have I been doing this week? As we did not set up git repositories yet, I'll not share the full patches, but only a few files for now: If you want to try the uImage, here is how to boot it. What's coming next First, I hope everything will be in place to share the full sources by then. The first thing I'll do next week is finishing identifying and including the needed drivers. Then, depending on how good goes the installation process, I may:

26 April 2010

Obey Arthur Liu: Welcome to our 2010 Debian Google Summer of Code students!

I d like to extend a warm welcome to our selected students for the 2010 Debian Google Summer of Code! They should pop up on Debian Planet soon and you re welcome to come talk to them on #debian-soc on irc.debian.org Aptitude Qt by Piotr Galiszewski, mentored by Sune Vuorela Qt GUI for aptitude. Currently, KDE users need to use Aptitude via the console interface, or install the newly developed GTK frontend, which does not fit well into KDE desktop. Making Qt frontend to Aptitude would solve this problem and bring an advanced and fully Debian-compliant graphical package manager to KDE. Content-aware Config Files Upgrading by Krzysztof Tyszecki, mentored by Dominique Dumont When a package deliver configuration files, the problem of merging user data with new configuration instructions will arise during package upgrades on users systems. Sometimes merging can be done with 3 way merge, but this process does not insure that the resulting file is correct or even legal. This project intends to create standards, tools an heuristics to make the scary config file conflict resolution debconf prompt a thing of the past. Debbugs Bug Reporting and Manipulation API by David Wendt Jr., mentored by Bastian Venthur Currently debbugs supports a SOAP interface for querying Debian s Bug Tracking System. Unfortunately this operation is read-only. This project would create an API for debbugs which supports sending and manipulating bug reports, without having to resort to email. This project does not intend to replace email as mean to manipulate the BTS but rather to enhance the BTS to allow other means of bug creation and manipulation. Debian High Performance Computing on Clouds by Dominique Belhachemi, mentored by Steffen Moeller The project paves a way to combine the demands in high performance computing with the dynamics of compute clouds with Debian. Combining the Eucalyptus cloud computing infrastructure with the TORQUE resource manager and preparing the components for dynamically added and removed instances provides the user with a attractive high performance computing environment. Such a system allows users to share resources with large compute centers with minimal changes in their workflow and scripts. Debian-Installer on Neo FreeRunner and Handheld Devices by Thibaut Girka, mentored by Gaudenz Steinlin This project aims to improve the installation experience of Debian on handheld devices by replacing ad-hoc install scripts by a full-blown and adapted Debian-Installer. The Neo FreeRunner is used as it is the most convenient and open device from a development standpoint, but other devices will also be explored. Hurd port and de-Linux-ization of Debian-Installer by J r mie Koenig, mentored by Samuel Thibault The primary means of distributing the Hurd is through Debian GNU/Hurd. However, the installation CDs presently use an ancient, non-native installer. The goal of this project is to port the missing parts of Debian-Installer to Hurd. To achieve this, all problematic Linux-specific code in Debian-Installer will be replaced by less or non-kernel dependent code, paving the way for better support of other non-Linux ports of Debian. Multi-Arch support in APT by David Kalnischkies, mentored by Michael Vogt Hardware like 64bit processors are perfectly able to execute 32bit opcode but until now this potentiality is disregard as the infrastructure tools like dpkg and APT are not able to install and/or solve dependencies across multiple architectures. The project therefore focuses on enabling APT to work out good solutions in a MultiArch aware environments without the need of hacky and partly working biarch packages currently in use. Package Repository Analysis and Migration Automation by Ricardo O Donell, mentored by Neil Williams Emdebian uses a filter to select packages from the main Debian repositories that are considered useful to embedded devices, excluding the majority of packages. The results of processing the filter are automated but maintaining the filter list is manual. This project seeks to automate certain elements of the filtering process to cope with specific conditions. This project will also generalize to more elaborate and intelligent algorithms to improve the transitions of the main Debian archives. Smart Upload Server for FTP Master by Petr Jasek, mentored by Joerg Jaspert Making packages upload smarter, more interactive and painless for uploaders by switching from anonymous FTP and Cron jobs to a robust protocol and modern package checking and processing daemon. This daemon would test early and report early, saving developers time. More details coming soon on http://wiki.debian.org/gsoc Congratulations everyone and have a fruitful summer!

23 December 2008

Emilio Pozuelo Monfort: Collaborative maintenance

The Debian Python Modules Team is discussing which DVCS to switch to from SVN. Ondrej Certik asked how to generate a list of commiters to the team s repository, so I looked at it and got this:
emilio@saturno:~/deb/python-modules$ svn log egrep "^r[0-9]+ cut -f2 -d sed s/-guest// sort uniq -c sort -n -r
865 piotr
609 morph
598 kov
532 bzed
388 pox
302 arnau
253 certik
216 shlomme
212 malex
175 hertzog
140 nslater
130 kobold
123 nijel
121 kitterma
106 bernat
99 kibi
87 varun
83 stratus
81 nobse
81 netzwurm
78 azatoth
76 mca
73 dottedmag
70 jluebbe
68 zack
68 cgalisteo
61 speijnik
61 odd_bloke
60 rganesan
55 kumanna
52 werner
50 haas
48 mejo
45 ucko
43 pabs
42 stew
42 luciano
41 mithrandi
40 wardi
36 gudjon
35 jandd
34 smcv
34 brettp
32 jenner
31 davidvilla
31 aurel32
30 rousseau
30 mtaylor
28 thomasbl
26 lool
25 gaspa
25 ffm
24 adn
22 jmalonzo
21 santiago
21 appaji
18 goedson
17 toadstool
17 sto
17 awen
16 mlizaur
16 akumar
15 nacho
14 smr
14 hanska
13 tviehmann
13 norsetto
13 mbaldessari
12 stone
12 sharky
11 rainct
11 fabrizio
10 lash
9 rodrigogc
9 pcc
9 miriam
9 madduck
9 ftlerror
8 pere
8 crschmidt
7 ncommander
7 myon
7 abuss
6 jwilk
6 bdrung
6 atehwa
5 kcoyner
5 catlee
5 andyp
4 vt
4 ross
4 osrevolution
4 lamby
4 baby
3 sez
3 joss
3 geole
2 rustybear
2 edmonds
2 astraw
2 ana
1 twerner
1 tincho
1 pochu
1 danderson
 As it s likely that the Python Applications Packaging Team will switch too to the same DVCS at the same time, here are the numbers for its repo:

emilio@saturno:~/deb/python-apps$ svn log egrep "^r[0-9]+ cut -f2 -d sed s/-guest// sort uniq -c sort -n -r
401 nijel
288 piotr
235 gothicx
159 pochu
76 nslater
69 kumanna
68 rainct
66 gilir
63 certik
52 vdanjean
52 bzed
46 dottedmag
41 stani
39 varun
37 kitterma
36 morph
35 odd_bloke
29 pcc
29 gudjon
28 appaji
25 thomasbl
24 arnau
20 sc
20 andyp
18 jalet
15 gerardo
14 eike
14 ana
13 dfiloni
11 tklauser
10 ryanakca
10 nxvl
10 akumar
8 sez
8 baby
6 catlee
4 osrevolution
4 cody-somerville
2 mithrandi
2 cjsmo
1 nenolod
1 ffm
 Here I m the 4th most committer :D And while I was on it, I thought I could do the same for the GNOME and GStreamer teams:
emilio@saturno:~/deb/pkg-gnome$ svn log egrep "^r[0-9]+ cut -f2 -d sed s/-guest// sort uniq -c sort -n -r
5357 lool
2701 joss
1633 slomo
1164 kov
825 seb128
622 jordi
621 jdassen
574 manphiz
335 sjoerd
298 mlang
296 netsnipe
291 grm
255 ross
236 ari
203 pochu
198 ondrej
190 he
180 kilian
176 alanbach
170 ftlerror
148 nobse
112 marco
87 jak
84 samm
78 rfrancoise
75 oysteigi
73 jsogo
65 svena
65 otavio
55 duck
54 jcurbo
53 zorglub
53 rtp
49 wasabi
49 giskard
42 tagoh
42 kartikm
40 gpastore
34 brad
32 robtaylor
31 xaiki
30 stratus
30 daf
26 johannes
24 sander-m
21 kk
19 bubulle
16 arnau
15 dodji
12 mbanck
11 ruoso
11 fpeters
11 dedu
11 christine
10 cpm
7 ember
7 drew
7 debotux
6 tico
6 emil
6 bradsmith
5 robster
5 carlosliu
4 rotty
4 diegoe
3 biebl
2 thibaut
2 ejad
1 naoliv
1 huats
1 gilir
emilio@saturno:~/deb/pkg-gstreamer$ svn log egrep "^r[0-9]+ cut -f2 -d sed s/-guest// sort uniq -c sort -n -r
891 lool
840 slomo
99 pnormand
69 sjoerd
27 seb128
21 manphiz
8 he
7 aquette
4 elmarco
1 fabian
 Conclusions:
- Why do I have the full python-modules and pkg-gstreamer trees, if I have just one commit to DPMT, and don t even have commit access to the GStreamer team?
- If you don t want to seem like you have done less commits than you have actually done, don t change your alioth name when you become a DD ;) (hint: pox-guest and piotr in python-modules are the same person)
- If the switch to a new VCS was based on a vote where you have one vote per commit, the top 3 commiters in pkg-gnome could win the vote if they chosed the same! For python-apps it s the 4 top commiters, and the 7 ones for python-modules. pkg-gstreamer is a bit special :)

Next.