Search Results: "Pino Toscano"

7 September 2016

Reproducible builds folks: Reproducible Builds: week 71 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday August 28 and Saturday September 3 2016: Media coverage Antonio Terceiro blogged about testing build reprodubility with debrepro . GSoC and Outreachy updates The next round is being planned now: see their page with a timeline and participating organizations listing. Maybe you want to participate this time? Then please reach out to us as soon as possible! Packages reviewed and fixed, and bugs filed The following packages have addressed reproducibility issues in other packages: The following updated packages have become reproducible in our current test setup after being fixed: The following updated packages appear to be reproducible now, for reasons we were not able to figure out yet. (Relevant changelogs did not mention reproducible builds.) The following 4 packages were not changed, but have become reproducible due to changes in their build-dependencies: Some uploads have addressed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Reviews of unreproducible packages 706 package reviews have been added, 22 have been updated and 16 have been removed in this week, adding to our knowledge about identified issues. 5 issue types have been added: 1 issue type has been updated: Weekly QA work FTBFS bugs have been reported by: diffoscope development diffoscope development on the next version (60) continued in git, taking in contributions from: strip-nondeterminism development Mattia Rizzolo uploaded strip-nondeterminism 0.023-2~bpo8+1 to jessie-backports. A new version of strip-nondeterminism 0.024-1 was uploaded to unstable by Chris Lamb. It included contributions from: Holger added jobs on jenkins.debian.net to run testsuites on every commit. There is one job for the master branch and one for the other branches. disorderfs development Holger added jobs on jenkins.debian.net to run testsuites on every commit. There is one job for the master branch and one for the other branches. tests.reproducible-builds.org Debian: We now vary the GECOS records of the two build users. Thanks to Paul Wise for providing the patch. Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

20 December 2015

Lunar: Reproducible builds: week 34 in Stretch cycle

What happened in the reproducible builds effort between December 13th to December 19th: Infrastructure Niels Thykier started implementing support for .buildinfo files in dak. A very preliminary commit was made by Ansgar Burchardt to prevent .buildinfo files from being removed from the upload queue. Toolchain fixes Mattia Rizzolo rebased our experimental debhelper with the changes from the latest upload. New fixes have been merged by OCaml upstream. Packages fixed The following 39 packages have become reproducible due to changes in their build dependencies: apache-mime4j, avahi-sharp, blam, bless, cecil-flowanalysis, cecil, coco-cs, cowbell, cppformat, dbus-sharp-glib, dbus-sharp, gdcm, gnome-keyring-sharp, gudev-sharp-1.0, jackson-annotations, jackson-core, jboss-classfilewriter, jboss-jdeparser2, jetty8, json-spirit, lat, leveldb-sharp, libdecentxml-java, libjavaewah-java, libkarma, mono.reflection, monobristol, nuget, pinta, snakeyaml, taglib-sharp, tangerine, themonospot, tomboy-latex, widemargin, wordpress, xsddiagram, xsp, zeitgeist-sharp. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet: reproducible.debian.net Packages in experimental are now tested on armhf. (h01ger) Arch Linux packages in the multilib and community repositories (4,000 more source packages) are also being tested. All of these test results are better analyzed and nicely displayed together with each package. (h01ger) For Fedora, build jobs can now run in parallel. Two are currently running, now testing reproducibility of 785 source packages from Fedora 23. mock/1.2.3-1.1 has been uploaded to experimental to better build RPMs. (h01ger) Work has started on having automatic build node pools to maximize use of armhf build nodes. (Vagrant Cascadian) diffoscope development Version 43 has been released on December 15th. It has been dubbed as epic! as it contains many contributions that were written around the summit in Athens. Baptiste Daroussin found that running diffoscope on some Tar archives could overwrite arbitrary files. This has been fixed by using libarchive instead of Python internal Tar library and adding a sanity check for destination paths. In any cases, until proper sandboxing is implemented, don't run diffosope on unstrusted inputs outside an isolated, throw-away system. Mike Hommey identified that the CBFS comparator would needlessly waste time scanning big files. It will now not consider any files bigger than 24 MiB 8 MiB more than the largest ROM created by coreboot at this time. An encoding issue related to Zip files has also been fixed. (Lunar) New comparators have been added: Android dex files (Reiner Herrmann), filesystem images using libguestfs (Reiner Herrmann), icons and JPEG images using libcaca (Chris Lamb), and OS X binaries (Clemens Lang). The comparator for Free Pascal Compilation Unit will now only be used when the unit version matches the compiler one. (Levente Polyak) A new multi-file HTML output with on-demand loading of long diffs is available through the --html-dir option. On-demand loading requires jQuery which path can be specified through the --jquery option. The diffs can also be simply browsed for non-JavaScript users or when jQuery is not available. (Joachim Breitner) Example of on-demand loading in diffosope Portability toward other systems has been improved: old versions of GNU diff are now supported (Mike McQuaid), suggestion of the appropriate locale is now the more generic en_US.UTF-8 (Ed Maste), the --list-tools option can now support multiple systems (Mattia Rizzolo, Levente Polyak, Lunar). Many internal changes and code clean-ups have been made, paving the way for parallel processing. (Lunar) Version 44 was released on December 18th fixing an issue affecting .deb lacking a md5sums file introduced in a previous refactoring (Lunar). Support has been added for Mozilla optimized Zip files. (Mike Hommey). The HTML output has been optimized in size (Mike Hommey, Esa Peuha, Lunar), speed (Lunar), and will now properly number lines (Mike Hommey). A message will always be displayed when lines are ignored at the end of a diff (Lunar). For portability and consistency, Python os.walk() function is now used instead of find to perform directory listing. (Lunar) Documentation update Package reviews 143 reviews have been removed, 69 added and 22 updated in the previous week. Chris Lamb reported 12 new FTBFS issues. News issues identified this week: random_order_in_init_py_generated_by_python-genpy, timestamps_in_copyright_added_by_perl_dist_zilla, random_contents_in_dat_files_generated_by_chasen-dictutils_makemat, timestamps_in_documentation_generated_by_pandoc. Chris West did some improvements on the scripts used to manage notes in the misc repository. Misc. Accounts of the reproducible builds summit in Athens were written by Thomas Klausner from NetBSD and Hans-Christoph Steiner from The Guardian Project. Some openSUSE developers are working on a hackweek on reproducible builds which was discussed on the opensuse-packaging mailing-list.

3 August 2015

Lunar: Reproducible builds: week 14 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes akira submitted a patch to make cdbs export SOURCE_DATE_EPOCH. She uploded a package with the enhancement to the experimental reproducible repository. Packages fixed The following 15 packages became reproducible due to changes in their build dependencies: dracut, editorconfig-core, elasticsearch, fish, libftdi1, liblouisxml, mk-configure, nanoc, octave-bim, octave-data-smoothing, octave-financial, octave-ga, octave-missing-functions, octave-secs1d, octave-splines, valgrind. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: In contrib, Dmitry Smirnov improved libdvd-pkg with 1.3.99-1-1. Patches submitted which have not made their way to the archive yet: reproducible.debian.net Four armhf build hosts were provided by Vagrant Cascadian and have been configured to be used by jenkins.debian.net. Work on including armhf builds in the reproducible.debian.net webpages has begun. So far the repository comparison page just shows us which armhf binary packages are currently missing in our repo. (h01ger) The scheduler has been changed to re-schedule more packages from stretch than sid, as the gcc5 transition has started This mostly affects build log age. (h01ger) A new depwait status has been introduced for packages which can't be built because of missing build dependencies. (Mattia Rizzolo) debbindiff development Finally, on August 31st, Lunar released debbindiff 27 containing a complete overhaul of the code for the comparison stage. The new architecture is more versatile and extensible while minimizing code duplication. libarchive is now used to handle cpio archives and iso9660 images through the newly packaged python-libarchive-c. This should also help support a couple other archive formats in the future. Symlinks and devices are now properly compared. Text files are compared as Unicode after being decoded, and encoding differences are reported. Support for Sqlite3 and Mono/.NET executables has been added. Thanks to Valentin Lorentz, the test suite should now run on more systems. A small defiency in unquashfs has been identified in the process. A long standing optimization is now performed on Debian package: based on the content of the md5sums control file, we skip comparing files with matching hashes. This makes debbindiff usable on packages with many files. Fuzzy-matching is now performed for files in the same container (like a tarball) to handle renames. Also, for Debian .changes, listed files are now compared without looking the embedded version number. This makes debbindiff a lot more useful when comparing different versions of the same package. Based on the rearchitecturing work has been done to allow parallel processing. The branch now seems to work most of the time. More test needs to be done before it can be merged. The current fuzzy-matching algorithm, ssdeep, has showed disappointing results. One important use case is being able to properly compare debug symbols. Their path is made using the Build ID. As this identifier is made with a checksum of the binary content, finding things like CPP macros is much easier when a diff of the debug symbols is available. Good news is that TLSH, another fuzzy-matching algorithm, has been tested with much better results. A package is waiting in NEW and the code is ready for it to become available. A follow-up release 28 was made on August 2nd fixing content label used for gzip2, bzip2 and xz files and an error on text files only differing in their encoding. It also contains a small code improvement on how comments on Difference object are handled. This is the last release name debbindiff. A new name has been chosen to better reflect that it is not a Debian specific tool. Stay tuned! Documentation update Valentin Lorentz updated the patch submission template to suggest to write the kind of issue in the bug subject. Small progress have been made on the Reproducible Builds HOWTO while preparing the related CCCamp15 talk. Package reviews 235 obsolete reviews have been removed, 47 added and 113 updated this week. 42 reports for packages failing to build from source have been made by Chris West (Faux). New issue added this week: haskell_devscripts_locale_substvars. Misc. Valentin Lorentz wrote a script to report packages tested as unreproducible installed on a system. We encourage everyone to run it on their systems and give feedback!

10 September 2014

Ian Donnelly: New Release: Elektra 0.8.8

Hi Everybody! Great news! I am very happy to announce that we have reached a new milestone for Elektra and released a new version, 0.8.8! This release comes right on the tail of the 0.8.7 release and it might just be our biggest release yet! We already have a great article covering all the changes from the previous release on our News documentation on GitHub. I just wanted to focus on a few of those changes on this blog, especially the ones that pertain to my Google Summer of Code Project. First of all, Felix has worked to greatly improve the ini plug-in. This is the plug-in I used in my technical demo for mounting Samba s smb.conf file. It now works even better with complex ini files such as smb.conf which means the automatic merging of files like smb.conf is even better now! That really goes to show one of the greatest strengths of the design of Elektra. Just by improving plug-ins, all the functions of Elektra can improve as well. The merge code was not changed in this release, yet because of an updated plug-in, the merge has improved. Secondly, there have been some good improvements to the kdb command-line tool. Many of these improvements were used in my technical demo, but now they are actually a part of release (and a little more refined from then). We added a new command called kdb remount which allows a user (or script) to mount a file to the Elektra Key Database using an existing backend. An example of this command is:
kdb remount [new filename] [new path] [existing mountpoint]

This command mounts the new file to the new path in the Key Database using an existing backend. This works with the conffile merging by allowing us to mount the various versions of the conffile without having to specify which backend to use (it will use the same backend as the currently used conffile). Additionally, the umount command was updated to allow users to umount using the current mountpath (allowing commands such as kdb umount system/smb.conf) as opposed to backends. Moreover, we added an option to the kdb import command to specify a merge strategy using thing -s option. Now you can import a file into the Key Database and merge the content of that file with the current Keys in the Database. Thirdly, we added some new scripts to Elektra to help with the ucf integration. These scripts were used in my technical demo, but now they are part of the release. elektra-mount and elektra-umount are wrappers for the commands kdb mount and kdb umount respectively. They are designed to be used in debian package scripts and are adapted for easier use than the generic commands. For instance, running elektra-mount will check to see if a file is already mounted at that location in the Key Database. Similarly, elektra-umount will not produce an error if the file was already unmounted. This is because maintainer scripts can be run multiple times in a row and producing an error will stop dpkg even when it shouldn t. Additionally, we added a script called elektra-merge which can be used as a method for ucf to merge configuration files. This script acts as a liaison between ucf and elektra allowing automatic merges to be done by ucf using Elektra s merge features in a seamless manner. For information of how these scripts work, check out my tutorial on integrating elektra-merge into a debian package. The last bit of news I would like to share is the great progress of the Debian package. Thanks to Pino Toscano, version 0.8.7-4 of Elektra is now available in the Debian testing repo! This is great news as we are now that much closer to replacing the outdated Elektra 0.7 versions that are currently the latest versions of Elektra in the stable repo. Once the 0.8.X versions of Elektra make it to stable it will be much easier for us to keep the latest versions of Elektra in Debian, and that s key to allowing Elektra to help improve users lives. You can download the release from:
http://www.markus-raab.org/ftp/elektra/releases/elektra-0.8.8.tar.gz size: 1644441
md5sum: fe11c6704b0032bdde2d0c8fa5e1c7e3
sha1: 16e43c63cd6d62b9fce82cb0a33288c390e39d12
sha256: ae75873966f4b5b5300ef5e5de5816542af50f35809f602847136a8cb21104e2 And the API-Documentation can be found here: http://doc.libelektra.org/api/0.8.8/html/ Hope you enjoy the new release! Sincerely,
Ian S. Donnelly

22 August 2014

Ian Donnelly: Wrapping Up

Hi Everybody! I have been keeping very busy on this blog the past few days with some very exciting updates! Everything is now in place for my Google Summer of Code project! Elektra now has the ability to merge KeySets, ucf has been patched to allow custom merge commands, we included a new elektra-merge script to use in conjunction with the new ucf command, and we wrote a great tutorial on how to use Elektra to merge configuration files in any package using ucf. There are few things I have missed updating you all on or are still wrapping up. First of all, I would like to address the Debian packages. While Elektra includes everything needed for my Google Summer of Code Project, it must be built from source right now. Unfortunately, with the rapid development Elektra has seen the past few months, we did not pay enough attention to the Debian packages and they became dusty and riddles with bugs. Fortunately, we have a solution and his name is Pino Toscano. Pino has, very graciously, agreed to help us fix our Debian packages back into shape. If you wish to see the current progress of the packages, you can check his repo. Pino has already made fantastic progress creating the Debian packages. I will post here when packages are all fixed up and the latest versions of Elektra are into the Debian repo. Some great news that we just received is that Elektra 0.8.7 has just been accepted into Debian unstable! This is huge progress for our team and means that users can now download our packages directly from Debian s unstable repo and test out these new features. Obviously, the normal caveats apply for any packages in unstable but this is still an amazing piece of news and I would like to thank Pino again for all the support he has provided. Another worthy piece of news that I unfortunately haven t had time to cover is that thanks to Felix Berlakovich Elektra has a new plug-in called ini. In case you couldn t guess, this new plug-in is used to mount ini files and the good news is that it is a vast improvement on our older simpleini plug-in. While it is still a work in progress, this new plug-in is much more powerful than simpleini. The new plug-in supports section and comments and works with many more ini files than the old plug-in. You may have noticed that I used this new plug-in to mount smb.conf in my technical demo of Samba using Elektra from configuration merging. Since smb.conf follows the same syntax as an ini file this new plug-in works great for mounting this file into the Elektra Key Database. I hope my blogs so far have been very informative and helpful. Sincerely,
Ian S. Donnelly

4 August 2014

Ian Donnelly: New Release: Elektra 0.8.7

Hi Everybody! I am very proud to inform you all that Elektra has just shipped a new release, version 0.8.7, with many great features and fixes! First of all, I want to let you all know that a lot of work from my Google Summer of Code Project has made its way into this release. Elektra now includes support for a three way merge of KeySets! A special and sincere thanks goes out to Felix Berlakovich for helping me test this new merge feature and adding some great features to allow for different merge strategies and dealing with meta keys. You can try out the new merge features using the kdb merge command or by using the Elektra API. There is still work to be done with Merging and improving documentation (also look for some posts on this blog soon about the feature). Additionally, thanks to Felix, we have technical previews for some new plug-ins. The new plug-ins are keytometa and ini. In short the keytometa plugin allows to convert normal keys to meta keys during the get operation and reverting this conversion during the set operation. The ini plugin is basically a rewrite of the simpleini plugin and makes use of the inih library. Also there have been many improvements made to the glob plug-in. He even found some time to add a new script for bash tab completion which is located under scripts/kdb-bash-completion. To use it on debian just copy it to /etc/bash_completion.d/ and make sure it is executable. Moreover, we fixed a lot of things with this newest release. Pino Toscano has been working on fixing up the Debian packages for Elektra but he has also fixed many other things along the way including fixing a lot of spelling errors, simplifying the RPATH setting, improvements to respecting $HOME and $TMPDIR, and improvements to some test cases. The kdb tool now does a better job of checking for subfolders that aren t allow and it now makes sure to output warnings before errors so errors can more easily be seen. We have also improved some tests for kdb tool and some plugins as well as fixed compiler warnings on clang and gcc 4.9. We also made some fixes to kdb import and export for some storage plugins and fixed some bugs so that kdb run_all now works flawlessly. There have also been a few tweaks to the API for this release, specifically in the C++ bindings. There is now a delMeta() function for C++. The reason for this is that contrary to the C API, calling

key.setMeta("metaname", NULL)

does not delete the metadata, but stores the value 0 . Additionally, we changed the arguments for isBelow, isDirectBelow, and isBelowSame for the C++ binding to be easier to understand and be more natural to use. Before this change, the C++ binding closely mirrored the C API which lead to an unintuitive behaviour. Before the change the API did the following:

Key ( user/config/key/below ).isBelow (Key ( user/config )) == false
Key ( user/config/key/below ).isBelow (Key ( user/config/key/below/deeper )) == true

That is because the first argument in the C API is the object itself in the C++ API.
The attribute of being below the key in question (the object) refers to the second key in the C API.
While this makes some sense for the C API, it definitely does not for the C++ API. Now the API behaves as follows (as intuitively expected):

Key ( user/config/key/below ).isBelow (Key ( user/config )) == true
Key ( user/config/key/below ).isBelow (Key ( user/config/key/below/deeper )) == false

We even had time for a bunch of documentation changes. We now have a tutorial for contextual values to GitHub so developers can start using contextual values with Elektra. We also included a specification for metadata and a better specification for contracts. There is even a little bit of extra news to share. We now use GitHub for active development of Elektra. We have adopted its issue tracker for issues. Also, now pull requests automatically get built by the server to see if the merge would brake the build and whether it passes all the tests. We are also in the process of updating a lot of our documentation and READMEs to use Markdown so they can be viewed easily on GitHub. Also, Raffael Pancheri has been making really great progress on a qt-gui for Elektra. There is still work to be done but it looks great and is coming along nicely. You can download the release now from Markus site: http://www.markus-raab.org/ftp/elektra/releases/elektra-0.8.7.tar.gz size: 1566800
md5sum: 4996df62942791373b192c793d912b4c
sha1: 00887cc8edb3dea1bc110f69ea64f6b700c29402
sha256: 698ebd41d540eb0c6427c17c13a6a0f03eef94655fbd40655c9b42d612ea1c9b Also there are packages already ready for some distributions: There is a lot of ongoing work to fix the Debian packages and I will post about it on this blog when they are good to go! Enjoy the new release!
-Ian S. Donnelly

28 June 2013

Justus Winter: Getting into the mood - my second week

tl; dr version: shutdown works, Debian GNU/Hurd booting using sysvinit:
Loading GNU Mach ...
Loading the Hurd ...
GNU Mach 1.3.99-486
AT386 boot: physical memory map from 0x0 to 0x9f400
AT386 boot: physical memory map from 0x100000 to 0x1fffe000
AT386 boot: physical memory from 0x0 to 0x1fffe000
Enabling FXSR
pcibios_init : BIOS32 Service Directory structure at 0xfcff0
pcibios_init : BIOS32 Service Directory entry at 0xfc7ba
pcibios_init : PCI BIOS revision 2.10 entry at 0xfc78c
Probing PCI hardware.
ide: Intel 82371 PIIX3 (dual FIFO) DMA Bus Mastering IDE
    Controller on PCI bus 0 function 9
ide: BM-DMA feature is not enabled (BIOS), enabling
    ide0: BM-DMA at 0xc100-0xc107
    ide1: BM-DMA at 0xc108-0xc10f
hd0: got CHS=762/128/63 CTL=c8 from BIOS
hd0: QEMU HARDDISK, 3001MB w/256kB Cache, CHS=762/128/63, DMA
hd2: QEMU DVD-ROM, ATAPI CDROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
Floppy drive(s): fd0 is 1.44M
FDC 0 is a S82078B
probing scsi 5/22: Adapteco1542FInvalidiaddresseforishpnt.with 1542.
Invalid address for shpnt with 1542.
probing scsi 18/22:AWesternxDigitalAWD-7000nFailed9initialization of WD-7000 SCSI card!
probing eata on/1f0EATA0:Daddress 0x1f04in3use, skipping probe.
probing eata on 170EATA0: address 0x170 in use, skipping probe.
probing scsi 21/22: Iomega7parport ZIP drive ppa: Version 1.42
ppa: Probing port 03bc
ppa: Probing port 0378
ppa:         SPP port present
ppa:         PS/2 bidirectional port present
ppa: Probing port 0278
done
scsi : 0 hosts.
scsi : detected total.
Partition check (DOS partitions):
 hd0: hd0s1 hd0s2 < hd0s5 >
lpr0: at atbus0, port = 378, spl = 6, pic = 7.
2omultibootxmodulesd/execl$(exec-task=task-create)ne=$ kernel-command-line  --host-priv-port=$ host-port  --device-master-port=$ device-port  --exec-server-task=$ exec-task  -T typed $ root  $(task-create) $(task-resume)               task loaded: ext2fs --readonly --multiboot-command-line=root=device:hd0s1 console=com0 --host-priv-port=1 --device-master-port=2 --exec-server-task=3 -T typed device:hd0s1
task loaded: exec /hurd/exec
start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec init proc auth
INIT: version 2.88 booting
Using makefile-style concurrent boot in runlevel S.
Activating swap...done.
Checking root file system...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
ext2fs_check_if_mount: Can't check if filesystem is mounted due to missing mtab file while determining whether /dev/hd0s1 is mounted.
/dev/hd0s1: clean, 43930/181056 files, 281248/723200 blocks
done.
Cleaning up temporary files... /tmp.
/etc/rcS.d/S06mtab.sh: 48: /etc/rcS.d/S06mtab.sh: cannot open /proc/mounts: No such file
Activating lvm and md swap...(default pager): Already paging to partition hd0s5!
done.
Checking file systems...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
done.
Mounting local filesystems...mount: invalid option -- 'O'
Try  mount --help' or  mount --usage' for more information.
failed.
Activating swapfile swap...(default pager): Already paging to partition hd0s5!
done.
df: Warning: cannot read table of mounted file systems: No such file or directory
Cleaning up temporary files....
Configuring network interfaces...inetutils-ifconfig: invalid arguments
ifup: failed to open pid file /run/network/ifup-/dev/eth0.pid: No such file or directory
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
DHCPDISCOVER on /dev/eth0 to 255.255.255.255 port 67 interval 6
DHCPREQUEST on /dev/eth0 to 255.255.255.255 port 67
DHCPOFFER from 10.0.2.2
DHCPACK from 10.0.2.2
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
bound to 10.0.2.15 -- renewal in 38544 seconds.
done.
Cleaning up temporary files....
Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
INIT: Entering runlevel: 2
Using makefile-style concurrent boot in runlevel 2.
Starting enhanced syslogd: rsyslogd.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting system message bus: dbusFailed to set socket option"/var/run/dbus/system_bus_socket": Protocol not available.
Starting OpenBSD Secure Shell server: sshd.
GNU 0.3 (debian) (console)
login:
This is the result of: This yields a surprisingly functional Debian/Hurd system with sysvinit. There are still lot's of loose ends, I populated my status page with them. I spent quite some time in #hurd on freenode and Pino Toscano offered to send me some old notes of his about the sysvinit issue. This turned out to be most helpful as his notes were about plugging sysvinits init into the boot process. This was the next logical thing to do for me, so first thing Monday morning I just put his runsystem.sysv file with some very minor tweaks (my version) into my overlay and had a working system in front of my eyes. That certainly was a nice way to start a week of work :), so many thanks again Pino! For your convenience I've created an overlay with all necessary changes and recompiled binaries. Be careful though, you might want to backup /sbin/reboot first or you won't be able to reboot your system! Of course my tool does that for you, and I've managed to create a reasonably small (253 mb) image thanks to the zerofree tool, the process of compacting images is implemented in hurdtest as the compact subcommand. I also added sysvinit specific tests. So if you want to try hurdtest, download the image and unxz it, my tool expects the image to be in the current working directory and be named image.qcow2. Download and untar an overlay and execute:
% hurdtest -logfile log -sysvinit overlay path/to/second-week-overlay
If you specify -logfile you can execute tail -f path/to/log to see what the VM is printing to the console in real time. The sysvinit specific tests demonstrate that indeed sysvinit is managing the system and that e. g. init 0 or shutdown -r now works. There are at least three major problems that I will need to tackle for this project: I think the last issue is what I'm going to look at next week. There's information about this in the Hurd wiki and I've spoken with Richard Braun about it and he drilled me to think about the hurdish way to solve this. I think I've got a pretty good picture of how to implement this in my head and am eager to give it a go. This will involve adding a function to Hurds filesystem interface that allows one to query an active translator about his mount options and "source" and about all active and passive translators that are attached to one of its nodes. This is where the fun actually begins. This is changing an integral part of an operating system, but this is okay since it is just another program written in c, operating in userspace. But hey I know that, that's just c and if something goes wrong (and it always does) I fire up gdb and have a look. And mind you, I'm running all this as an unprivileged user, no need to jeopardize the entire system for that. Besides implementing /proc/mounts I will also add -O, --test-opts to mount, and -v, --verbose to swapo n,ff . PS: My blog is still not properly registered to planet.d.o. I asked planet@ about that, but got no response. Any help? I never imagined blogging being so stressful, I care about losing potential readers :/

16 February 2013

Luca Falavigna: The DPL Game

Playing the DPL Game: here are my nominations for the Fantastic Four: P.S. Sorry Joss, I m not your man ;)

19 April 2012

Rapha&#235;l Hertzog: People behind Debian: Samuel Thibault, working on accessibility and the Hurd

Samuel Thibault is a French guy like me, but it took years until we met. He tends to keep a low profile, even though he s doing lots of good work that deserves to be mentioned. He focuses on improving Debian s accessibility and contributes to the Hurd. Who said he s a dreamer? :-) Checkout his interview to have some news of Wheezy s status on those topics. Raphael: Who are you? Samuel: I am 30 years old, and live in Bordeaux, France. During the workday, I teach Computer Science (Architecture, Networking, Operating Systems, and Parallel Programming, roughly) at the University of Bordeaux, and conduct researches in heterogeneous parallel computing. During the evening, I play the drums and the trombone in various orchestra (harmonic/symphonic/banda/brass). During the night, I hack on whatever fun things I can find, mainly accessibility and the Hurd at the moment, but also miscellaneous bits such as the Linux console support. I am also involved in the development of Aquilenet, an associative ISP around Bordeaux, and getting involved in the development of the network infrastructure in Bordeaux. I am not practicing Judo any more, but I roller-skate to work, and I like hiking in the mountains. I also read quite a few mangas. Saturday mornings do not exist in my schedule (Sunday mornings do, it s Brass Band rehearsal :) ). Raphael: How did you start contributing to Debian? Samuel: Bit by bit. I have been hacking around GNU/Linux since around 1998. I installed my first Debian system around 2000, as a replacement for my old Mandrake installation (which after all my tinkering was actually no longer looking like a Mandrake system any more!). That was Potato at the time, which somebody offered me through a set of CDs (downloading packages over the Internet was unthinkable at the time with the old modems). I have been happily reading and hacking around documentation, source code, etc. provided on them. Contribution things really started to take off when I went to the ENS Lyon high school in 2001: broadband Internet access in one s own student room! Since sending a mail was then really free, I started submitting bugs against various packages I was using. Right after that I started submitting patches along them, and then patches to other bugs. I did that for a long time actually. I had very little knowledge of all packaging details at the time, I was just a happy hacker submitting reports and patches against the upstream source code. At ENS Lyon, I met a blind colleague with very similar hacking tastes (of course we got friends) and he proposed me, for our student project, to work on a brlnet project (now called brlapi), a client/server protocol that lets applications render text on braille devices themselves. Along the way, I got to learn in details how a blind person can use a Unix system and the principles that should be followed when developing Accessibility. That is how I got involved in it. We presented our project at JDLL, and the Hurd booth happened to be next to our table, so I discussed with the Hurd people there about how the Hurd console could be used through braille. That is how I got into the Hurd too. From then on, I progressively contributed more and more to the upstream parts of both accessibility software and the Hurd. And then to the packaging part of them. Through patches in bug reports first, as usual, as well as through discussions on the mailing lists. But quickly enough people gave me commit access so I could just throw the code in. I was also given control over the Hurd buildds to keep them running. It was all good at that stage: I could contribute in all the parts I was caring about. People however started telling me that I should just apply for being a Debian Developer; both from accessibility and Hurd sides. I had also seen a bunch of my friends going through the process. I was however a bit scared (or probably it was just an excuse) by having to manage a gpg key, it seemed like a quite dangerous tool to me (even if I already had commit access to glibc at the time anyway ). I eventually applied for DM in 2008 so as to at least be able to upload some packages to help the little manpower of the Accessibility and Hurd teams. Henceforth I had already a gpg key, thus no excuse any more. And having it in the DM keyring was not enough for e.g. signing the hurd-i386 buildd packages. So I ended up going through NM in 2009, which went very fast, since I had already been contributing to Debian and learning all the needed stuff for almost 10 years! I now have around 50 packages in my QA page, and being a DD is actually useful for my work, to easily push our software to the masses :) So to sum it up, the Debian project is very easy to contribute to and open to new people. It was used during discussions at the GNU Hackers Meeting 2011 as an example of a very open community with public mailing lists and discussions. The mere fact that anybody can take the initiative of manipulating the BTS (if not scared by the commands) without having to ask anybody is an excellent thing to welcome contributions; it is notable tha the GNU project migrated to the Debbugs BTS. More generally, I don t really see the DD status as a must, especially now that we have the DM status (which is still a very good way to drag people into becoming DDs). For instance, I gave a talk at FOSDEM 2008 about the state of accessibility in Debian. People did not care whom I was, they cared that there was important stuff going on and somebody talking about it. More generally, decisions that are made through a vote are actually very rare. Most of the time, things just happen on the mailing lists or IRC channels where anybody can join the discussion. So I would recommend beginners to first use the software, then start reporting bugs, then start digging in the software to try fix the bugs by oneself, eventually propose patches, get them reviewed. At some point the submitted patches will be correct already most of the time. That s when the maintainers will start getting bored of just applying the patches, and simply provide with commit access, and voil , one has become a main contributor. Raphael: You re one of the main contributors to the Debian GNU/Hurd port. What motivates you in this project? Samuel: As I mentioned above, I first got real contact with the Hurd from the accessibility point of view. That initially brought me into the Hurd console, which uses a flexible design and nice interfaces to interact with it. The Hurd driver for console accessibility is actually very straightforward, way simpler than the Windows or Linux drivers. That is what caught me initially. I have continued working on it for several reasons. First, the design is really interesting for users. There are many things that are natural in the Hurd while Linux is still struggling to achieve them, such as UID isolation, recently mentioned in LWN. What I really like in the Hurd is that it excels at providing users with the same features as the administrator s. For instance, I find it annoying that I still can not mount an ISO image that I build on e.g. ries.debian.org. Linux now has FUSE which is supposed to permit that, but I have never seen it enabled on an ssh-accessible machine, only on desktop machines, and usually just because the administrator happens to be the user of the machine (who could as well just have used sudo ) For me, it is actually Freedom #0 of Free Software: let the user run programs for any purpose, that is, combining things together all the possible ways, and not being prevented from doing some things just because the design does not permit to achieve them securely. I had the chance to give a Hurd talk to explain that at GHM 2011, whose main topic was extensibility , I called it GNU/Hurd AKA Extensibility from the Ground, because the design of the Hurd is basically meant for extensibility, and does not care whether it is done by root or a mere user. All the tools that root uses to build a GNU/Hurd system can be used by the user to build its own GNU/Hurd environment. That is guaranteed by the design itself: the libc asks for things not to the kernel, but to servers (called translators), which can be provided by root, or by the user. It is interesting to see that it is actually also tried with varying success in GNU/Linux, through gvfs or Plash. An example of things I love being able to do is: $ zgrep foo ~/ftp://cdn.debian.net/debian/dists/sid/main/Contents-*.gz On my Hurd box, the ~/ftp: directory is indeed actually served by an ftpfs translator, run under my user uid, which is thus completely harmless to the system. Secondly and not the least, the Hurd provides me with interesting yet not too hard challenges. LWN confirmed several times that the Linux kernel has become very difficult to significantly contribute to, so it is no real hacking fun any more. I have notably implemented TLS support in the Hurd and the Xen and 64bit support in the GNU Mach kernel used by the Hurd. All three were very interesting to do, but were already done for Linux (at least for all the architectures which I actually know a bit and own). It happens that both TLS and Xen hacking experience became actually useful later on: I implemented TLS in the threading library of our research team, and the Xen port was a quite interesting line on my CV for getting a postdoc position at XenSource :) Lastly, I would say that I am used to lost causes :) My work on accessibility is sometimes a real struggle, so the Hurd is almost a kind of relief. It is famous for his vapourware reputation anyway, and so it is fun to just try to contribute to it nevertheless. An interesting thing is that the opinion of people on the Hurd is often quite extreme, and only rarely neutral. Some will say it is pure vapourware, while others will say that it is the hope of humanity (yes we do see those coming to #hurd, and they are not always just trolls!). When I published a 0.401 version on 2011 April 1st, the comments of people were very diverse, and some even went as far as saying that it was horrible of us to make a joke about the promised software :) Raphael: The FTPmasters want to demote the Hurd port to the debian-ports.org archive if it doesn t manage a stable release with wheezy. We re now at 2 months of the freeze. How far are you from being releasable ? Samuel: Of course, I can not speak for the Debian Release team. The current progress is however encouraging. During Debconf11, Michael Banck and I discussed with a few Debian Release team members about the kind of goals that should be achieved, and we are near completion of that part. The Debian GNU/Hurd port can almost completely be installed from the official mirrors, using the standard Debian Installer. Some patches need some polishing, but others are just waiting for being uploaded Debian GNU/Hurd can start a graphical desktop and run office tools such as gnumeric, as well as the iceweasel graphical web browser, KDE applications thanks to Pino Toscano s care, and GNOME application thanks to Emilio Pozuelo Monfort s care. Of course, general textmode hacking with gcc/make/gdb/etc. just works smoothly. Thanks to recent work on ghc and ada by Svante Signell, the archive coverage has passed 76%. There was a concern about network board driver support: until recently, the GNU Mach kernel was indeed still using a glue layer to embed the Linux 2.2 or even 2.0 drivers (!). Finding a network board supported by such drivers had of course become a real challenge. Thanks to the GSoC work of Zheng Da, the DDE layer can now be used to embed Linux 2.6.32 drivers in userland translators, which was recently ACCEPTed into the archive, and thus brings way larger support for network boards. It also pushes yet more toward the Hurd design: network drivers as userland process rather than kernel modules. That said, the freeze itself is not the final deadline. Actually, freeze periods are rests for porters, because maintainers stop bringing newer upstream versions which of course break on peculiar architectures. That will probably be helpful to continue improving the archive coverage. Raphael: The kfreebsd port brought into light all the packages which were not portable between different kernels. Did that help the Hurd port or are the problems too different to expect any mutual benefit? Samuel: The two ports have clearly helped each other in many aspects. The hurd-i386 port is the only non-Linux one that has been kept working (at least basically) for the past decade. That helped to make sure that all tools (dpkg, apt, toolchain, etc.) were able to cope with non-Linux ports, and keep that odd-but-why-not goal around, and evidently-enough achievable. In return, the kFreeBSD port managed to show that it was actually releasable, at least as a technological preview, thus making an example. In the daily work, we have sometimes worked hand in hand. The recent porting efforts of the Debian Installer happened roughly at the same time. When fixing some piece of code for one, the switch-case would be left for the other. When some code could be reused by the other, a mail would be sent to advise doing so, etc. In the packaging effort, it also made a lot of difference that a non-Linux port is exposed as released architecture: people attempted by themselves to fix code that is Linuxish for no real reason. The presence of the kFreeBSD is however also sometimes a difficulty for the Hurd: in the discussions, it sometimes tends to become a target to be reached, even if the systems are not really comparable. I do not need to detail the long history of the FreeBSD kernel and the amount of people hacking on it, some of them full-time, while the Hurd has only a small handful of free-time hackers. The FreeBSD kernel stability has already seen long-term polishing, and a fair amount of the Debian software was actually already ported to the FreeBSD kernel, thanks to the big existing pure-FreeBSD hackerbase. These do not hold for the GNU/Hurd port, so the expectations should go along. Raphael: You re also very much involved in the Debian Accessibility team. What are the responsibilities of this team and what are you doing there? Samuel: As you would expect it, the Debian Accessibility team works on packaging accessibility-related packages, and helping users with them; I thus do both. But the goal is way beyond just that. Actual accessibility requires integration. Ideally enough, a blind user should be able to just come to a Debian desktop system, plug his braille device, or press a shortcut to enable speech synthesis, and just use the damn computer, without having to ask the administrator to install some oddly-named package and whatnot. Just like any sighted user would do. He should be able to diagnose why his system does not boot, and at worse be able to reinstall his computer all by himself (typically at 2am ). And that is hard to achieve, because it means discussing about integration by default of accessibility features. For instance, the Debian CD images now beep during at the boot menu. That is a precious feature that has been discussed between debian-boot and debian-accessibility for a few weeks before agreeing on how to do it without too much disturbance. Similarly, my proposition of installing the desktop accessibility engines has been discussed for some time before being commited. What was however surprisingly great is that when somebody brought the topic back for discussion, non-debian-accessibility people answered themselves. This is reassuring, because it means things can be done durably in Debian. On the installation side, our current status is that the stable Debian installer has a high contrast color theme, and several years ago, I have pushed toward making standard CD images automatically detect braille devices, which permits standalone installation. I have added to the Wheezy installer some software speech synthesis (which again brought discussion about size increase vs versatility etc.) for blind people who do not have a braille device. I find it interesting to work on such topic in Debian rather than another distribution, because Debian is an upstream for a lot of distributions. Hopefully they just inherit our accessibility work. It at least worked for the text installer of Ubuntu. Of course, the Accessibility team is looking for help, to maintain our current packages, but also introduce new packages from the TODO list or create some backports. One does not need to be an expert in accessibility: tools can usually be tested, at least basically, by anybody, without particular hardware (I do not own any, I contributed virtual ones to qemu). For new developments and ideas, it is strongly recommended to come and discuss on debian-accessibility, because it is easy to get on a wrong track that does not bring actual accessibility. We still have several goals to achieve: the closest one is to just fix the transition to gnome3, which has been quite bad for accessibility so far :/ On the longer run, we should ideally reach the scenario I have detailed above: desktop accessibility available and ready to be enabled easily by default. Raphael: What s the biggest problem of Debian? Samuel: Debian is famous for its heated debian-devel discussions. And some people eventually say this no fun any more . That is exemplified in a less extreme way in the debian-boot/accessibility discussions that I have mentioned above. Sometimes, one needs to have a real stubborn thick head to continue the discussion until finding a compromise that will be accepted for commit. That is a problem because people do not necessarily have so much patience, and will thus prefer to contribute to a project with easier acceptance. But it is also a quality: as I explained above, once it is there, it is apparently for good. The Ubuntu support of accessibility in its installer has been very diverse, in part due to quite changing codebase. The Debian Installer codebase is more in a convergence process. Its base will have almost not changed between squeeze and wheezy. That allowed the Debian Accessibility team to continue improving its accessibility support, and not have to re-do it. A wiki page explains how to test its accessibility features, and some non-debian-accessibility people do go through it. A problem I am much more frightened by is the manpower in some core teams. The Debian Installer, grub, glibc, Xorg, gcc, mozilla derivatives, When reading the changelogs of these, we essentially keep seeing the same very few names over and over. And when one core developer leaves, it is very often still the same names which appear again to do the work. It is hard to believe that there are a thousand DDs working on Debian. I fear that Debian does not manage to get people to work on core things. I often hear people saying that they do not even dare thinking about putting their hands inside Xorg, for instance. Xorg is complex, but it seems to me that it tends to be overrated, and a lot of people could actually help there, as well as all the teams mentioned above. And if nobody does it, who will? Raphael: Do you have wishes for Debian Wheezy? Samuel: That is an easy one :) Of course I wish that we manage to release the hurd-i386 port. I also wish that accessibility of gnome3 gets fixed enough to become usable again. The current state is worrying: so much has changed that the transition will be difficult for users already, the current bugs will clearly not help. I also hope to find the time to fix the qt-at-spi bridge, which should (at last!) bring complete KDE accessibility. Raphael: Is there someone in Debian that you admire for their contributions? Samuel: Given the concerns I expressed above, I admire all the people who do spend time on core packages, even when that is really not fun everyday. Just to alphabetically name a few people I have seen so often here and there in the areas I have touched in the last few years: Aur lien Jarno, Bastian Blank, Christian Perrier, Colin Watson, Cyril Brulebois, Frans Pop, J rg Jaspert, Joey Hess, Josselin Mouette, Julien Cristau, Matthias Klose, Mike Hommey, Otavio Salvador, Petr Salinger, Robert Millan, Steve Langasek. Man, so many things that each of them works on! Of course this list is biased towards the parts that I touched, but people working in others core areas also deserve the same admiration.
Thank you to Samuel for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Note that older interviews are indexed on wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook.

3 comments Liked this article? Click here. My blog is Flattr-enabled.

30 September 2011

Axel Beckert: Fun facts from the UDD

After spotting an upload of mira, who in turn spotted an upload of abe (the package, not an upload by me aka abe@d.o), mira (mirabilos aka tg@d.o) noticed that there are Debian packages which have same name as some Debian Developers have as login name. Of course I noticed a long time ago that there is a Debian package with my login name abe . Another well-known Debian login and former package name is amaya. But since someone else came up with that thought, too, it was time for finding the definite answer to the question which are the DD login names which also exist as Debian package names. My first try was based on the list of trusted GnuPG keys:
$ apt-cache policy $(gpg --keyring /etc/apt/trusted.gpg --list-keys 2>/dev/null   \
                     grep @debian.org   \
        	     awk -F'[<@]' ' print $2 '   \
                     sort -u) 2>/dev/null   \
                   egrep -o '^[^ :]*'
alex
tor
ed
bam
ng
But this was not satisfying as my own name didn t show up and gpg also threw quite a lot of block reading errors (which is also the reason for redirecting STDERR). mira then had the idea of using the Ultimate Debian Database to answer this question more properly:
udd=> SELECT login, name FROM carnivore_login, carnivore_names
      WHERE carnivore_login.id=carnivore_names.id AND login IN
      (SELECT package AS login FROM packages, active_dds
       WHERE packages.package=active_dds.login UNION
       SELECT source AS name FROM sources, active_dds
       WHERE sources.source=active_dds.login)
      ORDER BY login;
 login                   name
-------+---------------------------------------
 abe     Axel Beckert
 alex    Alexander List
 alex    Alexander M. List  4402020774 9332554
 and     Andrea Veri
 ash     Albert Huang
 bam     Brian May
 ed      Ed Boraas
 ed      Ed G. Boraas [RSA Compatibility Key]
 ed      Ed G. Boraas [RSA]
 eric    Eric Dorland
 gq      Alexander GQ Gerasiov
 iml     Ian Maclaine-cross
 lunar   J r my Bobbio
 mako    Benjamin Hill
 mako    Benjamin Mako Hill
 mbr     Markus Braun
 mlt     Marcela Tiznado
 nas     Neil A. Schemenauer
 nas     Neil Schemenauer
 opal    Ola Lundkvist
 opal    Ola Lundqvist
 paco    Francisco Moya
 paul    Paul Slootman
 pino    Pino Toscano
 pyro    Brian Nelson
 stone   Fredrik Steen
(26 rows)
Interestingly tor (Tor Slettnes) is missing in this list, so it s not complete either At least I m quite sure that nobody maintains a package with his own login name as package name. :-) We also have no packages ending in -guest , so there s no chance that a package name matches an Alioth guest account either

17 February 2011

Cyril Brulebois: Debian XSF News #5

Time for a fifth Debian XSF News issue!