(continuation of Linksys NSLU2 adventures into the NetBSD land passed through JTAG highlands - part 1; meanwhile, my article was mentioned briefly in BSDNow Episode 89 - Exclusive Disjunction around minute 36:25)

Choosing to call RedBoot from a hacked Apex
As I was saying in my previous post, in order to be able to automate the booting of the NetBSD image via TFTP, I opted for using a 2nd stage bootloader (planning to flash it in the NSLU2 instead of a Linux kernel), and since Debian was already using Apex, I chose Apex, too.

The first problem I found was that the networking support in Apex was relying on an old version of the Intel NPE library which I couldn't find on Intel's site. The new version was incompatible/not building with the old build wrapper in Apex, so I was faced with 3 options:

1. Fight with the availabel Intel code and try to force it to compile in Apex
2. Incorporate the NPE driver from NetBSD into a rump kernel to be included in Apex instead of the original Intel code, since the NetBSD driver only needed an easily compilable binary blob
3. Hack together an Apex version that simulates the typing necessary RedBoot commands to load via TFTP the netbsd image and execute it.

After taking a look at the NPE driver buildsystem, I concluded there were very few options less attractive that option 1, among which was hammering nails through my forehead as a improvement measure against the severe brain damage which I would probably be likely to be inflicted with after dealing with the NPE "build system".

Option 2 looked like the best option I could have, given the situation, but my NetBSD foo was too close to 0 to even dream to endeavor on such a task. In my opinion, this still remains the technically superior solution to the problem since is very portable and a flexible way to ensure networking works in spite of the proprietary NPE code.

But, in practice, the best option I could implement at the time was option 3. I initially planned to pre-fill from Apex my desired commands into the RedBoot buffer that stored the keyboard strokes typed by the user:

load -r -b 0x200000 -h 192.168.0.2 netbsd-nfs.bin
g

Since this was the first time ever for me I was going to do less than trivial reverse engineering in order to find the addresses and signatures of interesting functions in the RedBoot code, it wasn't bad at all that I had a version of the RedBoot source code.

When stuck with reverse engineering, apply JTAG
The bad thing was that the code Linksys published as the source of the RedBoot running inside the NSLU2 was, in fact, a different code which had some significant changes around the code pieces I was mostly interested in. That in spite of the GPL terms.

But I thought that I could manage. After all, how hard could it be to identify the 2-3 functions I was interested in and 1 buffer? Even if I only had the disassembled code from the slug, it shouldn't be that hard.

I struggled with this for about 2-3 weeks on the few occasions I had during that time, but the excitement of leaning something new kept me going. Until I got stuck somewhere between the misalignment between the published RedBoot code and the disassembled code, the state of the system at the time of dumping the contents from RAM (for purposes of disassemby), the assembly code generated by GCC for some specific C code I didn't have at all, and the particularities of ARM assembly.

What was most likely to unblock me was to actually see the code in action, so I decided attaching a JTAG dongle to the slug and do a session of in-circuit-debugging was in order.

Luckily, the pinout of the JTAG interface was already identified in the NSLU2 Linux project, so I only had to solder some wires to the specified places and a 2x20 header to be able to connect through JTAG to the board.

JTAG connections on Kinder (the NSLU2 targeting NetBSD)

After this was done I tried immediately to see if when using a JTAG debugger I could break the execution of the code on the system. The answer was sadly, no.

The chip was identified, but breaking the execution was not happening. I tried this in OpenOCD and in another proprietary debugger application I had access to, and the result was the same, breaking was not happening.
Looking into the documentation I found a bit of information on the XScale processors[X] which suggested that XScale processors might necessarily need the (otherwise optional) SRST signal on the JTAG interface to be able to single step the chip.

This confused me a lot since I was sure other people had already used JTAG on the NSLU2.

The options I saw at the time were:

1. my NSLU2 did have a fully working JTAG interface (either due to the missing SRST signal on the interface or maybe due to a JTAG lock on later generation NSLU-s, as was my second slug)
2. nobody ever single stepped the slug using OpenOCD or other JTAG debugger, they only reflashed, and I was on totally new ground

I even contacted Rod Whitby, the project leader of the NSLU2 project to try to confirm single stepping was done before. Rod told me he never did that and he only reflashed the device.

This confused me even further because, from what I encountered on other platforms, in order to flash some device, the code responsible for programming the flash is loaded in the RAM of the target microcontroller and that code is executed on the target after a RAM buffer with the to be flashed data is preloaded via JTAG, then the operation is repeated for all flash blocks to be reprogrammed.

I was aware it was possible to program a flash chip situated on the board, outside the chip, by only playing with the chip's pads, strictly via JTAG, but I was still hoping single stepping the execution of the code in RedBoot was possible.

Guided by that hope and the possibility the newer versions of the device to be locked, I decided to add a JTAG interface to my older NSLU2, too. But this time I decided I would also add the TRST and SRST signals to the JTAG interface, just in case single stepping would work.

This mod involved even more extensive changes than the ones done on the other NSLU, but I was so frustrated by the fact I was stuck that I didn't mind poking a few holes through the case and the prospect of a connector always sticking out from the other NSLU2, which was doing some small, yet useful work in my home LAN.

It turns out NOBODY single stepped the NSLU2 After biting the bullet and soldering JTAG interface with also the TRST and the SRST signals connected as the pinout page from the NSLU2 Linux wiki suggested, I was disappointed to observe that I was not able to single step the older NSLU2 either, in spite of the presence of the extra signals.

I even tinkered with the reset configurations of OpenOCD, but had not success. After obtaining the same result on the proprietary debugger, digging through a presentation made by Rod back in the hay day of the project and the conversations on the NSLU2 Linux Yahoo mailing list, I finally concluded:

Actually nobody single stepped the NSLU2, no matter the version of the NSLU2 or connections available on the JTAG interface!

So I was back to square 1, I had to either struggle with disassembly, reevaluate my initial options, find another option or even drop entirely the idea. At that point I was already committed to the project, so dropping entirely the idea didn't seem like the reasonable thing to do.

Since I was feeling I was really close to finish on the route I had chosen a while ago, I was not any significantly more knowledgeable in the NetBSD code, and looking at the NPE code made me feel like washing my hands, the only option which seemed reasonable was to go on.

Digging a lot more through the internet, I was finally able to find another version of the RedBoot source which was modified for Intel ixp42x systems. A few checks here and there revealed this newly found code was actually almost identical to the code I had disassembled from the slug I was aiming to run NetBSD on. This was a huge step forward.

Long story short, a couple of days later I had a hacked Apex that could go through the RedBoot data structures, search for available commands in RedBoot and successfully call any of the built-in RedBoot commands!

Testing with loading this modified Apex by hand in RAM via TFTP then jumping into it to see if things woked as expected revealed a few small issues which I corrected right away.

Flashing a modified RedBoot?! But why? Wasn't Apex supposed to avoid exactly that risky operation?
Since the tests when executing from RAM were successful, my custom second stage Apex bootloader for NetBSD net booting was ready to be flashed into the NSLU2.

I added two more targets in the Makefile in the code on the dedicated netbsd branch of my Apex repository to generate the images ready for flashing into the NSLU2 flash (RedBoot needs to find a Sercomm header in flash, otherwise it will crash) and the exact commands to be executed in RedBoot are also print out after generation. This way, if the command is copy-pasted, there is no risk the NSLU2 is bricked by mistake.

After some flashing and reflashing of the apex_nslu2.flash image into the NSLU2 flash, some manual testing, tweaking and modifying the default built in APEX commands, checking that the sequence of commands 'move', 'go 0x01d00000' would jump into Apex, which, in turn, would call RedBoot to transfer the netbsd-nfs.bin image from a TFTP to RAM and then execute it successfully, it was high time to check NetBSD would boot automatically after the NSLU was powered on.

It didn't. Contrary to my previous tests, no call made from Apex to the RedBoot code would return back to Apex, not even the execution of a basic command such as the 'version' command.

It turns out the default commands hardcoded into RedBoot were 'boot; exec 0x01d00000', but I had tested 'boot; go 0x01d0000', which is not the same thing.

While 'go' does a plain jump at the specified address, the 'exec' command also does some preparations so it allows a jump into the Linux kernel and those preparations break some environment the RedBoot commands expect. I don't know which those are and didn't had the mood or motivation to find out.

So the easiest solution was to change the RedBoot's built-in command and turn that 'exec' into a 'go'. But that meant this time I was actually risking to brick the NSLU, unless I
was able to reflash via JTAG the NSLU2.


(to be continued - next, changing RedBoot and bisecting through the NetBSD history)

[X] Linksys NSLU2 has an XScale IXP420 processor which is compatible at ASM level with the ARMv5TEJ instruction set

I manage a few servers for myself, friends and family as well as for the Libravatar project. Here is how I customize recent releases of Debian on those servers.

Hardware tests

apt-get install memtest86+ smartmontools e2fsprogs

Prior to spending any time configuring a new physical server, I like to ensure that the hardware is fine. To check memory, I boot into memtest86+ from the grub menu and let it run overnight. Then I check the hard drives using:

smartctl -t long /dev/sdX
badblocks -swo badblocks.out /dev/sdX

Configuration

apt-get install etckeepr git sudo vim

To keep track of the configuration changes I make in /etc/, I use etckeeper to keep that directory in a git repository and make the following changes to the default /etc/etckeeper/etckeeper.conf:

• turn off daily auto-commits
• turn off auto-commits before package installs

To get more control over the various packages I install, I change the default debconf level to medium:

dpkg-reconfigure debconf

Since I use vim for all of my configuration file editing, I make it the default editor:

update-alternatives --config editor

ssh

apt-get install openssh-server mosh fail2ban

Since most of my servers are set to UTC time, I like to use my local timezone when sshing into them. Looking at file timestamps is much less confusing that way. I also ensure that the locale I use is available on the server by adding it the list of generated locales:

dpkg-reconfigure locales

Other than that, I harden the ssh configuration and end up with the following settings in /etc/ssh/sshd_config (jessie):

HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256
Ciphers chacha20-poly1305@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com
UsePrivilegeSeparation sandbox
AuthenticationMethods publickey
PasswordAuthentication no
PermitRootLogin no
AcceptEnv LANG LC_* TZ
LogLevel VERBOSE
AllowGroups sshuser

or the following for wheezy servers:

HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
KexAlgorithms ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256
Ciphers aes256-ctr,aes192-ctr,aes128-ctr
MACs hmac-sha2-512,hmac-sha2-256

On those servers where I need duplicity/paramiko to work, I also add the following:

KexAlgorithms ...,diffie-hellman-group-exchange-sha1
MACs ...,hmac-sha1

Then I remove the "Accepted" filter in /etc/logcheck/ignore.d.server/ssh (first line) to get a notification whenever anybody successfully logs into my server. I also create a new group and add the users that need ssh access to it:

addgroup sshuser
adduser francois sshuser

and add a timeout for root sessions by putting this in /root/.bash_profile:

TMOUT=600

Security checks

apt-get install logcheck logcheck-database fcheck tiger debsums corekeeper
apt-get remove john john-data rpcbind tripwire

Logcheck is the main tool I use to keep an eye on log files, which is why I add a few additional log files to the default list in /etc/logcheck/logcheck.logfiles:

/var/log/apache2/error.log
/var/log/mail.err
/var/log/mail.warn
/var/log/mail.info
/var/log/fail2ban.log

while ensuring that the apache logfiles are readable by logcheck:

chmod a+rx /var/log/apache2
chmod a+r /var/log/apache2/*

and fixing the log rotation configuration by adding the following to /etc/logrotate.d/apache2:

create 644 root adm

I also modify the main logcheck configuration file (/etc/logcheck/logcheck.conf):

INTRO=0
FQDN=0

Other than that, I enable daily checks in /etc/default/debsums and customize a few tiger settings in /etc/tiger/tigerrc:

Tiger_Check_RUNPROC=Y
Tiger_Check_DELETED=Y
Tiger_Check_APACHE=Y
Tiger_FSScan_WDIR=Y
Tiger_SSH_Protocol='2'
Tiger_Passwd_Hashes='sha512'
Tiger_Running_Procs='rsyslogd cron atd /usr/sbin/apache2 postgres'
Tiger_Listening_ValidProcs='sshd mosh-server ntpd'

General hardening

apt-get install harden-clients harden-environment harden-servers apparmor apparmor-profiles apparmor-profiles-extra

While the harden packages are configuration-free, AppArmor must be manually enabled:

perl -pi -e 's,GRUB_CMDLINE_LINUX="(.*)"$,GRUB_CMDLINE_LINUX="$1 apparmor=1 security=apparmor",' /etc/default/grub
update-grub

Entropy and timekeeping

apt-get install haveged rng-tools ntp

To keep the system clock accurate and increase the amount of entropy available to the server, I install the above packages and add the tpm_rng module to /etc/modules.

Preventing mistakes

apt-get install molly-guard safe-rm sl

The above packages are all about catching mistakes (such as accidental deletions). However, in order to extend the molly-guard protection to mosh sessions, one needs to manually apply a patch.

Package updates

apt-get install apticron unattended-upgrades deborphan debfoster apt-listchanges update-notifier-common aptitude popularity-contest

These tools help me keep packages up to date and remove unnecessary or obsolete packages from servers. On Rackspace servers, a small configuration change is needed to automatically update the monitoring tools. In addition to this, I use the update-notifier-common package along with the following cronjob in /etc/cron.daily/reboot-required:

#!/bin/sh
cat /var/run/reboot-required 2> /dev/null   true

to send me a notification whenever a kernel update requires a reboot to take effect.

Handy utilities

apt-get install renameutils atool iotop sysstat lsof mtr-tiny

Most of these tools are configure-free, except for sysstat, which requires enabling data collection in /etc/default/sysstat to be useful.

Apache configuration

apt-get install apache2-mpm-event

While configuring apache is often specific to each server and the services that will be running on it, there are a few common changes I make. I enable these in /etc/apache2/conf.d/security:

<Directory />
    AllowOverride None
    Order Deny,Allow
    Deny from all
</Directory>
ServerTokens Prod
ServerSignature Off

and remove cgi-bin directives from /etc/apache2/sites-enabled/000-default. I also create a new /etc/apache2/conf.d/servername which contains:

ServerName machine_hostname

Mail

apt-get install postfix

Configuring mail properly is tricky but the following has worked for me. In /etc/hostname, put the bare hostname (no domain), but in /etc/mailname put the fully qualified hostname. Change the following in /etc/postfix/main.cf:

inet_interfaces = loopback-only
myhostname = (fully qualified hostname)
smtp_tls_security_level = may
smtp_tls_protocols = !SSLv2, !SSLv3

Set the following aliases in /etc/aliases:

• set francois as the destination of root emails
• set an external email address for francois
• set root as the destination for www-data emails

before running newaliases to update the aliases database. Create a new cronjob (/etc/cron.hourly/checkmail):

#!/bin/sh
ls /var/mail

to ensure that email doesn't accumulate unmonitored on this box. Finally, set reverse DNS for the server's IPv4 and IPv6 addresses and then test the whole setup using mail root.

Network tuning To reduce the server's contribution to bufferbloat I change the default kernel queueing discipline (jessie or later) by putting the following in /etc/sysctl.conf:

net.core.default_qdisc=fq_codel

Few days ago, I've started writing Odorik module to manipulate with API of one Czech mobile network operator. As usual, the code comes with documentation written in English. Given that vast majority of users are Czech, it sounds useful to have in Czech language as well. The documentation itself is written in Sphinx and built using Read the Docs. Using those to translate the documentation is quite easy. First step is to add necessary configuration to the Sphinx project as described in their Internationalization Quick Guide. It's matter of few configuration directives and invoking of sphinx-intl and the result can be like this commit. Once the code in repository is ready, you can start building translated documentation on the Read the docs. There is nice guide for that as well. All you need to do is to create another project, set it's language and link it from master project as translation. The last step is to find some translators to actually translate the document. For me the obvious choice was using Weblate, so the translation is now on Hosted Weblate. The mass import of several po files can be done by import_project management command. And thanks to all these you can now read Czech documentation for python Odorik module.

Filed under: Coding English Odorik Weblate 0 comments

Categories:

• Debian-Blog
• Computing
• Technology

Keywords:

• Lenovo
• Yoga
• Yoga 2 13
• debian

I was first elected to the Software in the Public Interest board back in 2009. I was re-elected in 2012. This July I am up for re-election again. For a variety of reasons I ve decided not to stand; mostly a combination of the fact that I think 2 terms (6 years) is enough in a single stretch and an inability to devote as much time to the organization as I d like. I mentioned this at the May board meeting. I m planning to stay involved where I can. My main reason for posting this here is to cause people to think about whether they might want to stand for the board. Nominations open on July 1st and run until July 13th. The main thing you need to absolutely commit to is being able to attend the monthly board meeting, which is held on IRC at 20:30 UTC on the second Thursday of the month. They tend to last at most 30 minutes. Of course there s a variety of tasks that happen in the background, such as answering queries from prospective associated projects or discussing ongoing matters on the membership or board lists depending on circumstances. It s my firm belief that SPI do some very important work for the Free software community. Few people realise the wide variety of associated projects. SPI offload the boring admin bits around accepting donations and managing project assets (be those machines, domains, trademarks or whatever), leaving those projects able to concentrate on the actual technical side of things. Most project members don t realise the involvement of SPI, and that s largely a good thing as it indicates the system is working. However it also means that there can sometimes be a lack of people wanting to stand at election time, and an absence of diversity amongst the candidates. I m happy to answer questions of anyone who might consider standing for the board; #spi on irc.oftc.net is a good place to ask them - I am there as Noodles.

What happened about the reproducible builds effort for this week: Media coverage Debian's effort on reproducible builds has been covered in the June 2015 issue of Linux Magazin in Germany. Toolchain fixes

• gregor herrmann uploaded libextutils-depends-perl/0.404-1 which makes its output deterministic.
• Christian Hofstaedtler uploaded yard/0.8.7.4-2 which will not write timestamps in the generated documentation. Original patch by Chris Lamb, does not write timestamps in the generated documentation anymore.
• Emmanuel Bourg uploaded maven-plugin-tools/3.3-2 which removes the date from the plugin descriptor. Patch by Reiner Herrmann.
• Emmanuel Bourg uploaded maven-archiver/2.6-1 which now uses the date set in the DEB_CHANGELOG_DATETIME environment variable for the timestamp in the pom.properties file embedded in the jar files. Original patch by Chris West.
• Nicolas Boulenguez uploaded dh-ada-library/6.4 which will warn against non deterministic ALI for sources newer than changelog.

josch rebased the experimental version of debhelper on 9.20150507. Packages fixed The following 515 packages became reproducible due to changes of their build dependencies: airport-utils, airspy-host, all-in-one-sidebar, ampache, aptfs, arpack, asciio, aspell-kk, asused, balance, batmand, binutils-avr, bioperl, bpm-tools, c2050, cakephp-instaweb, carton, cbp2make, checkbot, checksecurity, chemeq, chronicle, cube2-data, cucumber, darkstat, debci, desktop-file-utils, dh-linktree, django-pagination, dosbox, eekboek, emboss-explorer, encfs, exabgp, fbasics, fife, fonts-lexi-saebom, gdnsd, glances, gnome-clocks, gunicorn, haproxy, haskell-aws, haskell-base-unicode-symbols, haskell-base64-bytestring, haskell-basic-prelude, haskell-binary-shared, haskell-binary, haskell-bitarray, haskell-bool-extras, haskell-boolean, haskell-boomerang, haskell-bytestring-lexing, haskell-bytestring-mmap, haskell-config-value, haskell-mueval, haskell-tasty-kat, itk3, jnr-constants, jshon, kalternatives, kdepim-runtime, kdevplatform, kwalletcli, lemonldap-ng, libalgorithm-combinatorics-perl, libalgorithm-diff-xs-perl, libany-uri-escape-perl, libanyevent-http-scopedclient-perl, libanyevent-perl, libanyevent-processor-perl, libapache-session-wrapper-perl, libapache-sessionx-perl, libapp-options-perl, libarch-perl, libarchive-peek-perl, libaudio-flac-header-perl, libaudio-wav-perl, libaudio-wma-perl, libauth-yubikey-decrypter-perl, libauthen-krb5-simple-perl, libauthen-simple-perl, libautobox-dump-perl, libb-keywords-perl, libbarcode-code128-perl, libbio-das-lite-perl, libbio-mage-perl, libbrowser-open-perl, libbusiness-creditcard-perl, libbusiness-edifact-interchange-perl, libbusiness-isbn-data-perl, libbusiness-tax-vat-validation-perl, libcache-historical-perl, libcache-memcached-perl, libcairo-gobject-perl, libcarp-always-perl, libcarp-fix-1-25-perl, libcatalyst-action-serialize-data-serializer-perl, libcatalyst-controller-formbuilder-perl, libcatalyst-dispatchtype-regex-perl, libcatalyst-plugin-authentication-perl, libcatalyst-plugin-authorization-acl-perl, libcatalyst-plugin-session-store-cache-perl, libcatalyst-plugin-session-store-fastmmap-perl, libcatalyst-plugin-static-simple-perl, libcatalyst-view-gd-perl, libcgi-application-dispatch-perl, libcgi-application-plugin-authentication-perl, libcgi-application-plugin-logdispatch-perl, libcgi-application-plugin-session-perl, libcgi-application-server-perl, libcgi-compile-perl, libcgi-xmlform-perl, libclass-accessor-classy-perl, libclass-accessor-lvalue-perl, libclass-accessor-perl, libclass-c3-adopt-next-perl, libclass-dbi-plugin-type-perl, libclass-field-perl, libclass-handle-perl, libclass-load-perl, libclass-ooorno-perl, libclass-prototyped-perl, libclass-returnvalue-perl, libclass-singleton-perl, libclass-std-fast-perl, libclone-perl, libconfig-auto-perl, libconfig-jfdi-perl, libconfig-simple-perl, libconvert-basen-perl, libconvert-ber-perl, libcpan-checksums-perl, libcpanplus-dist-build-perl, libcriticism-perl, libcrypt-cracklib-perl, libcrypt-dh-gmp-perl, libcrypt-mysql-perl, libcrypt-passwdmd5-perl, libcrypt-simple-perl, libcss-packer-perl, libcss-tiny-perl, libcurses-widgets-perl, libdaemon-control-perl, libdancer-plugin-database-perl, libdancer-session-cookie-perl, libdancer2-plugin-database-perl, libdata-format-html-perl, libdata-uuid-libuuid-perl, libdata-validate-domain-perl, libdate-jd-perl, libdate-simple-perl, libdatetime-astro-sunrise-perl, libdatetime-event-cron-perl, libdatetime-format-dbi-perl, libdatetime-format-epoch-perl, libdatetime-format-mail-perl, libdatetime-tiny-perl, libdatrie, libdb-file-lock-perl, libdbd-firebird-perl, libdbix-abstract-perl, libdbix-class-datetime-epoch-perl, libdbix-class-dynamicdefault-perl, libdbix-class-introspectablem2m-perl, libdbix-class-timestamp-perl, libdbix-connector-perl, libdbix-oo-perl, libdbix-searchbuilder-perl, libdbix-xml-rdb-perl, libdevel-stacktrace-ashtml-perl, libdigest-hmac-perl, libdist-zilla-plugin-emailnotify-perl, libemail-date-format-perl, libemail-mime-perl, libemail-received-perl, libemail-sender-perl, libemail-simple-perl, libencode-detect-perl, libexporter-tidy-perl, libextutils-cchecker-perl, libextutils-installpaths-perl, libextutils-libbuilder-perl, libextutils-makemaker-cpanfile-perl, libextutils-typemap-perl, libfile-counterfile-perl, libfile-pushd-perl, libfile-read-perl, libfile-touch-perl, libfile-type-perl, libfinance-bank-ie-permanenttsb-perl, libfont-freetype-perl, libfrontier-rpc-perl, libgd-securityimage-perl, libgeo-coordinates-utm-perl, libgit-pureperl-perl, libgnome2-canvas-perl, libgnome2-wnck-perl, libgraph-readwrite-perl, libgraphics-colornames-www-perl, libgssapi-perl, libgtk2-appindicator-perl, libgtk2-gladexml-simple-perl, libgtk2-notify-perl, libhash-asobject-perl, libhash-moreutils-perl, libhtml-calendarmonthsimple-perl, libhtml-display-perl, libhtml-fillinform-perl, libhtml-form-perl, libhtml-formhandler-model-dbic-perl, libhtml-html5-entities-perl, libhtml-linkextractor-perl, libhtml-tableextract-perl, libhtml-widget-perl, libhtml-widgets-selectlayers-perl, libhtml-wikiconverter-mediawiki-perl, libhttp-async-perl, libhttp-body-perl, libhttp-date-perl, libimage-imlib2-perl, libimdb-film-perl, libimport-into-perl, libindirect-perl, libio-bufferedselect-perl, libio-compress-lzma-perl, libio-compress-perl, libio-handle-util-perl, libio-interface-perl, libio-multiplex-perl, libio-socket-inet6-perl, libipc-system-simple-perl, libiptables-chainmgr-perl, libjoda-time-java, libjsr305-java, libkiokudb-perl, liblemonldap-ng-cli-perl, liblexical-var-perl, liblingua-en-fathom-perl, liblinux-dvb-perl, liblocales-perl, liblog-dispatch-configurator-any-perl, liblog-log4perl-perl, liblog-report-lexicon-perl, liblwp-mediatypes-perl, liblwp-protocol-https-perl, liblwpx-paranoidagent-perl, libmail-sendeasy-perl, libmarc-xml-perl, libmason-plugin-routersimple-perl, libmasonx-processdir-perl, libmath-base85-perl, libmath-basecalc-perl, libmath-basecnv-perl, libmath-bigint-perl, libmath-convexhull-perl, libmath-gmp-perl, libmath-gradient-perl, libmath-random-isaac-perl, libmath-random-oo-perl, libmath-random-tt800-perl, libmath-tamuanova-perl, libmemoize-expirelru-perl, libmemoize-memcached-perl, libmime-base32-perl, libmime-lite-tt-perl, libmixin-extrafields-param-perl, libmock-quick-perl, libmodule-cpanfile-perl, libmodule-load-conditional-perl, libmodule-starter-pbp-perl, libmodule-util-perl, libmodule-versions-report-perl, libmongodbx-class-perl, libmoo-perl, libmoosex-app-cmd-perl, libmoosex-attributehelpers-perl, libmoosex-blessed-reconstruct-perl, libmoosex-insideout-perl, libmoosex-relatedclassroles-perl, libmoosex-role-timer-perl, libmoosex-role-withoverloading-perl, libmoosex-storage-perl, libmoosex-types-common-perl, libmoosex-types-uri-perl, libmoox-singleton-perl, libmoox-types-mooselike-numeric-perl, libmousex-foreign-perl, libmp3-tag-perl, libmysql-diff-perl, libnamespace-clean-perl, libnet-bonjour-perl, libnet-cli-interact-perl, libnet-daap-dmap-perl, libnet-dbus-glib-perl, libnet-dns-perl, libnet-frame-perl, libnet-google-authsub-perl, libnet-https-any-perl, libnet-https-nb-perl, libnet-idn-encode-perl, libnet-idn-nameprep-perl, libnet-imap-client-perl, libnet-irc-perl, libnet-mac-vendor-perl, libnet-openid-server-perl, libnet-smtp-ssl-perl, libnet-smtp-tls-perl, libnet-smtpauth-perl, libnet-snpp-perl, libnet-sslglue-perl, libnet-telnet-perl, libnhgri-blastall-perl, libnumber-range-perl, libobject-signature-perl, libogg-vorbis-header-pureperl-perl, libopenoffice-oodoc-perl, libparse-cpan-packages-perl, libparse-debian-packages-perl, libparse-fixedlength-perl, libparse-syslog-perl, libparse-win32registry-perl, libpdf-create-perl, libpdf-report-perl, libperl-destruct-level-perl, libperl-metrics-simple-perl, libperl-minimumversion-perl, libperl6-slurp-perl, libpgobject-simple-perl, libplack-middleware-fixmissingbodyinredirect-perl, libplack-test-externalserver-perl, libplucene-perl, libpod-tests-perl, libpoe-component-client-ping-perl, libpoe-component-jabber-perl, libpoe-component-resolver-perl, libpoe-component-server-soap-perl, libpoe-component-syndicator-perl, libposix-strftime-compiler-perl, libposix-strptime-perl, libpostscript-simple-perl, libproc-processtable-perl, libprotocol-osc-perl, librcs-perl, libreadonly-xs-perl, libreturn-multilevel-perl, librivescript-perl, librouter-simple-perl, librrd-simple-perl, libsafe-isa-perl, libscope-guard-perl, libsemver-perl, libset-tiny-perl, libsharyanto-file-util-perl, libshell-command-perl, libsnmp-info-perl, libsoap-lite-perl, libstat-lsmode-perl, libstatistics-online-perl, libstring-compare-constanttime-perl, libstring-format-perl, libstring-toidentifier-en-perl, libstring-tt-perl, libsub-recursive-perl, libsvg-tt-graph-perl, libsvn-notify-perl, libswish-api-common-perl, libtap-formatter-junit-perl, libtap-harness-archive-perl, libtemplate-plugin-number-format-perl, libtemplate-plugin-yaml-perl, libtemplate-tiny-perl, libtenjin-perl, libterm-visual-perl, libtest-block-perl, libtest-carp-perl, libtest-classapi-perl, libtest-cmd-perl, libtest-consistentversion-perl, libtest-data-perl, libtest-databaserow-perl, libtest-differences-perl, libtest-file-sharedir-perl, libtest-hasversion-perl, libtest-kwalitee-perl, libtest-lectrotest-perl, libtest-module-used-perl, libtest-object-perl, libtest-perl-critic-perl, libtest-pod-coverage-perl, libtest-script-perl, libtest-script-run-perl, libtest-spelling-perl, libtest-strict-perl, libtest-synopsis-perl, libtest-trap-perl, libtest-unit-perl, libtest-utf8-perl, libtest-without-module-perl, libtest-www-selenium-perl, libtest-xml-simple-perl, libtest-yaml-perl, libtex-encode-perl, libtext-bibtex-perl, libtext-csv-encoded-perl, libtext-csv-perl, libtext-dhcpleases-perl, libtext-diff-perl, libtext-quoted-perl, libtext-trac-perl, libtext-vfile-asdata-perl, libthai, libthread-conveyor-perl, libthread-sigmask-perl, libtie-cphash-perl, libtie-ical-perl, libtime-stopwatch-perl, libtk-dirselect-perl, libtk-pod-perl, libtorrent, libturpial, libunicode-japanese-perl, libunicode-maputf8-perl, libunicode-stringprep-perl, libuniversal-isa-perl, libuniversal-moniker-perl, liburi-encode-perl, libvi-quickfix-perl, libvideo-capture-v4l-perl, libvideo-fourcc-info-perl, libwiki-toolkit-plugin-rss-reader-perl, libwww-mechanize-formfiller-perl, libwww-mechanize-gzip-perl, libwww-mechanize-perl, libwww-opensearch-perl, libx11-freedesktop-desktopentry-perl, libxc, libxml-dtdparser-perl, libxml-easy-perl, libxml-handler-trees-perl, libxml-libxml-iterator-perl, libxml-libxslt-perl, libxml-rss-perl, libxml-validator-schema-perl, libxml-xpathengine-perl, libxml-xql-perl, llvm-py, madbomber, makefs, mdpress, media-player-info, meta-kde-telepathy, metamonger, mmm-mode, mupen64plus-audio-sdl, mupen64plus-rsp-hle, mupen64plus-ui-console, mupen64plus-video-z64, mussort, newpid, node-formidable, node-github-url-from-git, node-transformers, nsnake, odin, otcl, parsley, pax, pcsc-perl, pd-purepd, pen, prank, proj, proot, puppet-module-puppetlabs-postgresql, python-async, python-pysnmp4, qrencode, r-bioc-graph, r-bioc-hypergraph, r-bioc-iranges, r-bioc-xvector, r-cran-pscl, rbenv, rlinetd, rs, ruby-ascii85, ruby-cutest, ruby-ejs, ruby-factory-girl, ruby-hdfeos5, ruby-kpeg, ruby-libxml, ruby-password, ruby-zip-zip, sdl-sound1.2, stterm, systemd, taktuk, tcc, tryton-modules-account-invoice, ttf-summersby, tupi, tuxpuck, unknown-horizons, unsafe-mock, vcheck, versiontools, vim-addon-manager, vlfeat, vsearch, xacobeo, xen-tools, yubikey-personalization-gui, yubikey-personalization. The following packages became reproducible after getting fixed:

• cwirc/2.0.0-8 uploaded by Colin Tuckley, original patch by Reiner Herrmann.
• darkplaces/0~20140513+svn12208-1 by Simon McVittie.
• exactimage/0.9.1-4 by Sven Eckelmann.
• gnupg/1.4.19-1 by Daniel Kahn Gillmor.
• httpunit/1.7+dfsg-11 by Emmanuel Bourg.
• hy/0.10.1-2 uploaded by Tianon Gravi, original patch by Reiner Herrmann.
• ioquake3/1.36+u20150412+dfsg1-2 by Simon McVittie, original patch by Reiner Herrmann.
• kiwi/1.9.22-3 by Jelmer Vernooij.
• lava-server/2015.05-1 uploaded by Neil Williams, original patch by Reiner Herrmann.
• libelixirfm-perl/1.1.976-4 uploaded by gregor herrmann, original patch by Chris Lamb.
• littler/0.2.3-2 by Dirk Eddelbuettel.
• mednafen/0.9.38.1-1 by Stephen Kitt.
• nftables/0.4-4 by Arturo Borrero Gonzalez.
• ntdb/1.0-7 by Jelmer Vernooij.
• onioncat/0.2.2+svn566-1 by intrigeri.
• openarena/0.8.8-13 by Simon McVittie.
• openarena-085-data/0.8.5split-6 by Simon McVittie.
• openarena-088-data/0.8.8-3 by Simon McVittie.
• openarena-data/0.8.5split-6 by Simon McVittie.
• openarena-maps/0.8.5split-6 by Simon McVittie.
• openarena-players/0.8.5split-6 by Simon McVittie.
• openarena-players-mature/0.8.5split-6 by Simon McVittie.
• openarena-textures/0.8.5split-6 by Simon McVittie.
• pybik/2.0-1 by B. Clausius.
• python-xmp-toolkit/2.0.1+git20140309.5437b0a-1 by Daniel Stender.
• quakespasm/0.90.0-3 by Stephen Kitt.
• traceroute/1:2.0.21-1 uploaded by Laszlo Boszormenyi, original patch by Lunar.
• unar/1.8.1-4 uploaded by Matt Kraai, original patch by Lunar.
• websvn/2.3.3-1.3 uploaded by Thijs Kinkhorst, original patch by Chris Lamb.
• xd/3.23.01-2 uploaded by Frank B. Brokken, original patch by Chris Lamb.

Some uploads fixed some reproducibility issues but not all of them:

• ada-reference-manual/1:2012.2-5 by Nicolas Boulenguez.
• apparmor/2.9.2-2 by intrigeri.
• argyll/1.7.0+repack-1 by J rg Frings-F rst.
• lava-dispatcher/2015.05-1 by Neil Williams.
• libaunit/3.7.1-2 by Nicolas Boulenguez.
• libflorist/2014-2 by Nicolas Boulenguez.
• mailcrypt/3.5.9-8 uploaded by Barak A. Pearlmutter, original patch by Chris Lamb.
• openchange/1:2.2-7 by Jelmer Vernooij.
• sane-backends/1.0.24-11 by J rg Frings-F rst timestamps in .dvi and .ps
• tomcat6/6.0.41-4 by Emmanuel Bourg.
• tomcat7/7.0.61-1 by Emmanuel Bourg; currently FTBFS.
• tomcat8/8.0.22-2 by Emmanuel Bourg.

Patches submitted which did not make their way to the archive yet:

• #784541 on yasm by Lunar: remove build date from version strings.
• #784694 on smcroute by Micha Lenk: remove build date from version string.
• #784672 on gnumeric by Daniel Kahn Gillmor: remove timestamps in embedded gzip'ed data in shared library.
• #774347 on sed by Lunar: fix permissions before creating the package.
• #784352 on icebreaker by Reiner Herrmann: use UTC timezone when calculating version date.
• #784325 on kde-workspace by Lunar: make the output of kdm confproc.pl stable.
• #784602 on monkeysign by Daniel Kahn Gillmor: use time of debian/changelog entry when generating documentation.
• #784723 on alot by Juan Picca: pass time of debian/changelog entry to Sphinx.
• #784538 on file-rc by Lunar: use sed instead of grep+mv to keep correct file permissions.
• #784335 on libapache2-mod-perl2 by Lunar: set PERL_HASH_SEED=0 during configure to make the generated .c and .h files stable.
• #784267 on mpv by Lunar: pass --disable-build-date to ./configure.
• #784793 on bugs-everywhere by Daniel Kahn Gillmor: use time of debian/changelog entry as build date.
• #784318 on gnome-desktop3 by Lunar: use time of debian/chanelog entry as build date.
• #774504 on debianutils by Lunar: fix file permissions.

reproducible.debian.net Alioth now hosts a script that can be used to redo builds and test for a package. This was preliminary done manually through requests over the IRC channel. This should reduce the number of interruptions for jenkins' maintainers The graph of the oldest build per day has been fixed. Maintainance scripts will not error out when they are no files to remove. Holger Levsen started work on being able to test variations of CPU features and build date (as in build in another month of 1984) by using virtual machines. debbindiff development Version 18 has been released. It will uses proper comparators for pk3 and info files. Tar member names are now assumed to be UTF-8 encoded. The limit for the maximum number of different lines has been removed. Let's see on reproducible.debian.net how it goes for pathological cases. It's now possible to specify both --html and --text output. When neither of them is specified, the default will be to print a text report on the standard output (thanks to Paul Wise for the suggestion). Documentation update Nicolas Boulenguez investigated Ada libraries. Package reviews 451 obsolete reviews have been removed and 156 added this week. New identified issues: running kernel version getting captured, random filenames in GHC debug symbols, and timestamps in headers generated by qdbusxml2cpp. Misc. Holger Levsen went to re:publica and talked about reproducible builds to developers and users there. Holger also had a chance to meet FreeBSD developers and discuss the status of FreeBSD. Investigations have started on how it could be made part of our current test system. Laurent Guerby gave Lunar access to systems in the GCC Compile Farm. Hopefully access to these powerful machines will help to fix packages for GCC, Iceweasel, and similar packages requiring long build times.

Categories:

• Debian-Blog

Keywords:

• drupal
• drush
• sky
• adaptive themes

Images:

I've recently been contacted by Johannes Hubertz, who is writing a new book about Python in German called "Softwaretests mit Python" which will be published by Open Source Press, Munich this summer. His book will feature some interviews, and he was kind enough to let me write a bit about software testing. This is the interview that I gave for his book. Johannes translated to German and it will be included in Johannes' book, and I decided to publish it on my blog today. Following is the original version. How did you come to Python? I don't recall exactly, but around ten years ago, I saw more and more people using it and decided to take a look. Back then, I was more used to Perl. I didn't really like Perl and was not getting a good grip on its object system. As soon as I found an idea to work on if I remember correctly that was rebuildd I started to code in Python, learning the language at the same time. I liked how Python worked, and how fast I was to able to develop and learn it, so I decided to keep using it for my next projects. I ended up diving into Python core for some reasons, even doing things like briefly hacking on projects like Cython at some point, and finally ended up working on OpenStack. OpenStack is a cloud computing platform entirely written in Python. So I've been writing Python every day since working on it. That's what pushed me to write The Hacker's Guide to Python in 2013 and then self-publish it a year later in 2014, a book where I talk about doing smart and efficient Python. It had a great success, has even been translated in Chinese and Korean, so I'm currently working on a second edition of the book. It has been an amazing adventure! Zen of Python: Which line is the most important for you and why? I like the "There should be one and preferably only one obvious way to do it". The opposite is probably something that scared me in languages like Perl. But having one obvious way to do it is and something I tend to like in functional languages like Lisp, which are in my humble opinion, even better at that. For a python newbie, what are the most difficult subjects in Python? I haven't been a newbie since a while, so it's hard for me to say. I don't think the language is hard to learn. There are some subtlety in the language itself when you deeply dive into the internals, but for beginners most of the concept are pretty straight-forward. If I had to pick, in the language basics, the most difficult thing would be around the generator objects (yield). Nowadays I think the most difficult subject for new comers is what version of Python to use, which libraries to rely on, and how to package and distribute projects. Though things get better, fortunately. When did you start using Test Driven Development and why? I learned unit testing and TDD at school where teachers forced me to learn Java, and I hated it. The frameworks looked complicated, and I had the impression I was losing my time. Which I actually was, since I was writing disposable programs that's the only thing you do at school. Years later, when I started to write real and bigger programs (e.g. rebuildd), I quickly ended up fixing bugs I already fixed. That recalled me about unit tests and that it may be a good idea to start using them to stop fixing the same things over and over again. For a few years, I wrote less Python and more C code and Lua (for the awesome window manager), and I didn't use any testing. I probably lost hundreds of hours testing manually and fixing regressions that was a good lesson. Though I had good excuses at that time it is/was way harder to do testing in C/Lua than in Python. Since that period, I have never stopped writing "tests". When I started to hack on OpenStack, the project was adopting a "no test? no merge!" policy due to the high number of regressions it had during the first releases. I honestly don't think I could work on any project that does not have at least a minimal test coverage. It's impossible to hack efficiently on a code base that you're not able to test in just a simple command. It's also a real problem for new comers in the open source world. When there are no test, you can hack something and send a patch, and get a "you broke this" in response. Nowadays, this kind of response sounds unacceptable to me: if there is no test, then I didn't break anything! In the end, it's just too much frustration to work on non tested projects as I demonstrated in my study of whisper source code. What do you think to be the most often seen pitfalls of TDD and how to avoid them best? The biggest problems are when and at what rate writing tests. On one hand, some people starts to write too precise tests way too soon. Doing that slows you down, especially when you are prototyping some idea or concept you just had. That does not mean that you should not do test at all, but you should probably start with a light coverage, until you are pretty sure that you're not going to rip every thing and start over. On the other hand, some people postpone writing tests for ever, and end up with no test all or a too thin layer of test. Which makes the project with a pretty low coverage. Basically, your test coverage should reflect the state of your project. If it's just starting, you should build a thin layer of test so you can hack it on it easily and remodel it if needed. The more your project grow, the more you should make it sold and lay more tests. Having too detailed tests is painful to make the project evolve at the start. Having not enough in a big project makes it painful to maintain it. Do you think, TDD fits and scales well for the big projects like OpenStack? Not only I think it fits and scales well, but I also think it's just impossible to not use TDD in such big projects. When unit and functional tests coverage was weak in OpenStack at its beginning it was just impossible to fix a bug or write a new feature without breaking a lot of things without even noticing. We would release version N, and a ton of old bugs present in N-2 but fixed in N-1 were reopened. For big projects, with a lot of different use cases, configuration options, etc, you need belt and braces. You cannot throw code in a repository thinking it's going to work ever, and you can't afford to test everything manually at each commit. That's just insane.

This is the first blog post of a series I will be writing about my experiences contributing to Debian for Google Summer of Code 2015.

A bit about myself I m a Spanish student doing a master s in Computer Science in Barcelona. I graduated on Electrical Engineering (we call it Telecommunications here). I ve always been interested in computing and programming and I have worked on several projects on my own using C, python and go. My main interests in the field are programming languages, security, cryptography, distributed and decentralized systems and embedded systems. I m an advocate of free software, I try to use it as much as I m able to in my devices and also try to convince my friends and family of its benefits. I have been using GNU/Linux for nearly ten years as my main operating system and I have tried several *BSD s recently. One of my latest personal projects is a gameboy emulator written in C (still work in progress) which already plays many games (without sound though) . You can find other minor projects in my github page (I try to publish all the code I write online, under free software licence) After so many years of using free software and benefiting from it, I thought it was about time to contribute back! That s why I gave GSoC a try and applied to work on the Reproducible Builds project for Debian :) And I got accepted!

Reproducible Builds The idea behind this project is that currently many packages aren t built in a reproducible manner; that is, they contain timestamps, building machine name, unique IDs, and results from other processes that happen differently between machines, like file ordering in compressed files. The project aims to patch all the Debian packages / the building scripts in order to generate the same binary (bit by bit) independently of the machine, timezone, etc where it is built. This way, a cryptographic hash of the built package can be distributed and many people can rebuild the package to verify that the binary in the repositories indeed corresponds to the right source code by means of comparing the hash.

Motivation One of the main advantages of the free software is that source code is available for peer review. This makes it easier for users to trust their software, as they can check the source to verify that the program is not doing anything bad. Even if the user doesn t do that, they can trust the wider community with that task. But many distributions serve packages in binary form, so how do we know that the binary comes from the publicly available source code? The current solution is that the developers who build the packages sign them cryptographically; but this lands all the trust to the developer and the machines used for building. I became interested in this topic with a very nice talk given at 31c3 by Mike Perry from Tor and Seth Schoen from the EFF. They focused on reproducible builds applied to the tor browser bundle, showing a small demo of how a building machine could be compromised to add hidden functionalities when compiling code (so that the developer could be signing a compromised package without their knowledge).

Benefits There are two main groups who benefit with reproducible builds:

For users The user can be more secure when installing packages in binary form since they don t need to trust a specific developer or building machine. Even if they don t rebuild the package by themselves to verify it, there would be others doing so, who will easily alert the community when the binary doesn t match the source code.

For developers The developer no longer has the responsibility of using his identity to sign the package for wide distribution, nor is that much responsible of the damage to users if their machine is compromised to alter the building process, since the community will easily detect it and alert them. This later point is specially useful with secure and privacy aware software. The reason is that there are many powerful organizations around the world with interest on having backdoors in widely used software, be it to spy on users or to target specific groups of people. Considering the amount of money these organizations have for such purposes, it s not hard to imagine that they could try to blackmail developers into adding such backdoors on the built packages. Or they could try to compromise the building machine. With reproducible builds the developer is safer, as such attack is no longer useful.

Reproducible Builds in Debian The project kicked-off at Debian at mid 2013 , leaded by Lunar and soon followed by many other developers (h01ger, deki, mapreri, ). Right now about 80% of the packages in the unstable branch of Debian can be built reproducibly. The project is very active, with many developers sending patches every week. A machine running Jenkins (which was set up at the end of 2012 for other purposes) is being used since late 2014 to continuously build packages in different settings to check if they are built reproducibly or not. In order to analyze why packages fail to build reproducibly, a tool called debbindiff has been developed, which is able to output in text or html form a smart diff of two builds. Another tool called strip-nondeterminism has been developed to remove non-determinism from files during the building process. For this GSoC I plan on helping improving these tools (mainly debbindiff), write many patches to achieve reproducibility in more packages and write documentation about it. Some of the packages fail to build reproducibly due to specifics of their building processes, whereas others fail due to the usage of toolchains that add non-determinism. I ll focus more on the later ones in order to improve the state more packages. akira will also be working on this project for this GSoC. Finally, I just want to add that I m looking forward to contribute to Debian, meet the community and learn more about the internals of this awesome distribution!

In my recent post (I give up on Google), a lot of people suggested using OwnCloud as a replacement for several Google services. I ve been playing around with it for a few days, and it is something of a mix of awesome and disappointing, in my opinion. Files OwnCloud started as a file-sync tool, somewhat akin to Google Drive and Dropbox. It has clients for every platform, and it is also a client for every platform: you can have subfolders of your OwnCloud installation stored on WebDav, *FTP*, Google Drive, Dropbox, you name it. It is a pretty nice integrator of other storage services, and provides the only way to use some of them on Linux (*cough* Google Drive *cough*) One particularly interesting feature is the live editing in the browser of ODT, DOCX, and TXT files. This is somewhat similar to Google Docs and the only such thing I ve seen in Open Source software. It writes changes directly back to the documents and, in my limited testing, seems to work well. A very nice feature! I ve tested the syncing only on Linux so far, but it looks solid. There are two surprising issues, however: there is no deduplication and no delta-uploads. Add 10 bytes to the end of a 1GB file, and you re-upload the 1GB file. Thankfully the OwnCloud GUI client is smart enough to use inotify to notice an mv, but my guess is and I haven t tested this, but apparently OwnCloud doesn t use hashes at all that the CLI client would require a reupload after any mv, because it doesn t run continuously. In some situations, Syncany may be a useful work-around for this, as it does chunk-based dedup and client-side encryption. However, you would lose a lot of the sharing features inside OwnCloud by doing this, and the integration with the OwnCloud apps for photos, videos, and music. The Android/mobile apps support all the usual auto-upload options. Calendar A lot of people report using OwnCloud as a calendar server, and it does indeed use CalDAV. With a program like DAVDroid or Mozilla Lightning, this makes, in theory, a full-functioning calendar syncing tool. There is, of course, also a web interface to the calendar. It, sadly, is limited. Or shall we say, VERY limited. Even something like sending an invite is missing and in fact, the GUI for sharing an event is baffling. You can share it with someone, they get no say in whether or not it shows up, and it shows up on their calendar on the web only (not on synced copies) and they have no way to remove it! Sharing calendars is similar; you can hide the display of any one of your calendars on the web interface, but not of any calendars shared with you. Baffling. Address Book I haven t tested this yet, but there s not much to test, I suspect. It can be shared with others, which I could see as a nice feature. Bookmarks An interesting bookmarks manager, though mysteriously not with Firefox sync support. There is Chrome sync support, and a separate Mozilla Sync support, but it doesn t provide cross-browser syncing, apparently. Music It is designed to present an interface to music that is stored in Files. It provides an Ampache-compatible API, so there are a lot of clients that can stream music. It has very few options, not even for transcoding, so I don t see how it would be useful for my FLAC collection. Pictures Sort of a gallery view of photos synced up with Files. Very basic. Has a sharing button to share a link to an entire folder, but no option to embed photos in blog posts at a lower resolution or shortcut to sharing individual photos. Notes, Tasks, etc. I haven t had the chance to look at this much. Some of them sync to various clients. The Notes are saved as HTML files that get synced down. Clients overall There is a very helpful page that lists all the sync clients for OwnCloud not just for files, but also for calendars, contacts, etc. The list is extensive! Other options The two other Open Source options mentioned on my blog post were Kolab and Sogo, and there is also Zimbra which also has a community edition. The Debian Groupware page lists a number of other groupware options as well. Citadel caught my eye (wow, it s still around!). Sogo has ActiveSync support, which might make phone integration a lot easier. It is not dead-simple to set up like OwnCloud is, though, so I haven t tried it out, but I will probably be looking at it and Citadel next.

When paultag recently announced a project to try to move debian infrastructure to python3, my first thought was how large that undertaking would likely be. It seems like a classic engineering task, full of work and nit-picky details to get right, useful/necessary in the long-term, painful in the short-term, and if you manage to pull it off successfully, the best you can usually hope for is that no one will notice that it was done at all. I always find that kind of task a little off-putting and difficult to tackle, but I was happy to see someone driving the project, since it does need to get done. Debian is potentially also in a position to help the upstream python community, because we have a pretty good view of what things are being used, at least within our own ecosystem. I'm happy to say that i also missed one of the other great benefits of paultag's audacious proposal, which is how it has engaged people who already knew about debian but who aren't yet involved. Evidence of this engagement is already visible on the py3porters-devel mailing list. But if that wasn't enough, I ran into a friend recently who told me, "Hey, I found a way to contribute to debian finally!" and pointed me to the py3-porters project. People want to contribute to the project, and are looking for ways in. So cheers to the people who propose audacious projects and make them inviting to everyone, newcomers included. And cheers to the people who step up to potentially daunting work, stake out a task, roll up their sleeves, and pitch in. Even if the py3porters project doesn't move all of debian's python infrastructure to pyt3 as fast as paultag wants it to, i think it's already a win for the project as a whole. I am looking forward to seeing what comes out of it (and it's reminding me i need to port some of my own python work, too!) The next time you stumble over something big that needs doing in debian, even something that might seem impossible, please make it inviting, and dive in. The rest of the project will grow and improve from the attempt.Tags: py3-porters

Is the locked books room at McMahon Hall at UW a metaphor for DRM in the academy? Could it be, like so many things in Seattle, sponsored by Amazon? Mika noticed the room several weeks ago but felt that today s International Day Against DRM was a opportune time to raise the questions in front of a wider audience.

For the 2015 International Day Against DRM, I wrote a short essay on DRM for streaming services posted on the Defective by Design website. I m republishing it here. Between 2003 and 2009, most music purchased through Apple s iTunes store was locked using Apple s FairPlay digital restrictions management (DRM) software, which is designed to prevent users from copying music they purchased. Apple did not seem particularly concerned by the fact that FairPlay was never effective at stopping unauthorized distribution and was easily removed with publicly available tools. After all, FairPlay was effective at preventing most users from playing their purchased music on devices that were not made by Apple. No user ever requested FairPlay. Apple did not build the system because music buyers complained that CDs purchased from Sony would play on Panasonic players or that discs could be played on an unlimited number of devices (FairPlay allowed five). Like all DRM systems, FairPlay was forced on users by a recording industry paranoid about file sharing and, perhaps more importantly, by technology companies like Apple, who were eager to control the digital infrastructure of music distribution and consumption. In 2007, Apple began charging users 30 percent extra for music files not processed with FairPlay. In 2009, after lawsuits were filed in Europe and the US, and after several years of protests, Apple capitulated to their customers complaints and removed DRM from the vast majority of the iTunes music catalog. Fundamentally, DRM for downloaded music failed because it is what I ve called an antifeature. Like features, antifeatures are functionality created at enormous cost to technology developers. That said, unlike features which users clamor to pay extra for, users pay to have antifeatures removed. You can think of antifeatures as a technological mob protection racket. Apple charges more for music without DRM and independent music distributors often use DRM-free as a primary selling point for their products. Unfortunately, after being defeated a half-decade ago, DRM for digital music is becoming the norm again through the growth of music streaming services like Pandora and Spotify, which nearly all use DRM. Impressed by the convenience of these services, many people have forgotten the lessons we learned in the fight against FairPlay. Once again, the justification for DRM is both familiar and similarly disingenuous. Although the stated goal is still to prevent unauthorized copying, tools for stripping DRM from services continue to be widely available. Of course, the very need for DRM on these services is reduced because users don t normally store copies of music and because the same music is now available for download without DRM on services like iTunes. We should remember that, like ten years ago, the real effect of DRM is to allow technology companies to capture value by creating dependence in their customers and by blocking innovation and competition. For example, DRM in streaming services blocks third-party apps from playing music from services, just as FairPlay ensured that iTunes music would only play on Apple devices. DRM in streaming services means that listening to music requires one to use special proprietary clients. For example, even with a premium account, a subscriber cannot listen to music from their catalog using an alternative or modified music player. It means that their television, car, or mobile device manufacturer must cut deals with their service to allow each paying customer to play the catalog they have subscribed to. Although streaming services are able to capture and control value more effectively, this comes at the cost of reduced freedom, choice, and flexibility for users and at higher prices paid by subscribers. A decade ago, arguments against DRM for downloaded music focused on the claim that users should have control over the music they purchase. Although these arguments may not seem to apply to subscription services, it is worth remembering that DRM is fundamentally a problem because it means that we do not have control of the technology we use to play our music, and because the firms aiming to control us are using DRM to push antifeatures, raise prices, and block innovation. In all of these senses, DRM in streaming services is exactly as bad as FairPlay, and we should continue to demand better.

Debian published Jessie last month. Big congratulations to the release team and all the contributors; quality is again at the rendez-vous! This time, I could not contribute much, being busy with parental activities (hello my son, if you read me), apart from making sure that my packages are ready for the D Day. This work was made easy by the automated removals of buggy packages set up by the release team. Many thanks for that development. I hope to use Jessie for a long time without the need of upgrading to Testing. At the moment, it provides everything I need, including the input of Japanese characters together with the possibility to switch between Japanese (or American) and multilingual Canadian keyboard layouts, which was not easy to do anymore in Wheezy. Many thanks again.

I approached the SuperTuxKart community fearing some backslash due to last week s discussion about their release 0.9, to find instead a nice, friendly and welcoming community. I have already had some very nice talks with them since then, and they have patiently explained to me the sequence of events that led to the situation that I mentioned and that, for the sake of fairness, I consider that I have to share here too. You can read the log of the first conversation I had with them (the log has been edited and cleared up for clarity and readability). I seriously recommend reading it, it s a honest friendly conversation, and it s first hand. For those who don t already know the game:

All this story seems to start with the complain of a 6 yo girl, close relative of one of the developers and STK user, who explained that she always felt that Mario Kart was better because there was a princess in it. I m not particularly happy with princesses as role models for girls, but one thing I have always said is that we have to listen to kids and take their opinions into accounts, and I know that if I had such a request from one of the kids closer to me, I probably would have fulfilled it too. In any case, Free Software projects based on volunteer work are essentially a do-ocracy and it is assumed that whoever does the work, gets to decide about it.

So that is how Princess Sara was added to the game. While developing it, I was assured that they took extra care that her proportions were somehow realistic, and not as distorted as we re used to see in Barbie or many Disney films. Sara is inspired on an OpenGameArt s wizard and is not supposed to be a weak damsel in distress, but in fact a powerful character in the world s universe.

Sara is not the only female character playable. There are a few others: Suzanne (a monkey, Blender s mascot), Xue (XFCE s mouse) and Amanda (a panda, the mascot of windows maker). Sara happens to be the only human character playable, male or female. While it has been argued that by adding that character, a player might have the impression that the rest of the characters would be male by default, I have been told that the intention is exactly the opposite,and that the fact that the only human playable character in the game is female should make it more attractive to girls. To some, at least. Here are some images of Sara:

So the fact is that they have invested a lot of time in developing Sara s model. I m not an artist myself, so I don t know first hand how much time and effort it takes to make such a model, but in any case it seems that quite a lot. When they designed the beach track Gran Paradiso, they wanted to add people to the beach. That track is, in fact, inspired on a real existing place: Princess Juliana Airport. Time was over and they wanted to publish a version with what they already had, so they used Sara s model in a bikini on the beach, with the intention of adding more people, male and female, later. The overall view of the beach would be:

This is how that track shows when the players are driving in it:

Now, about the poster of version 0.9, it is supposed to be inspired in the previous poster of version 0.8.1, only this time inspired in Carnival (which is, in fact, a celebration in which sexualization of both genders is a core part). I know that there are accusations of cultural apropriation, but I couldn t know, as my white privilege probably shields me from seeing that. Up to now, no one has said anything about that, only Gunnar explaining his point of view as a non-native mexican: While the poster does not strike as the most cautious possible, I do not see it as culturally offensive. It does not attempt to set a scene portraiting what were the cultures really like; the portrait it paints is similar to so many fantasy recreations . In my opinion, even when the model is done in good taste, with no superbig breasts and no unrealistic waist, it s still depicting a girl without much clothes as the main element of the scene, with an attire, a posture and an attitude that clearly resembles carnival and, thus, inevitably conveys a message of sexualization. Even though I can t deny that it s a cute poster, it s one I wouldn t be happy to see for example in a school, if someone wanted to promote the game there. The author of the poster, anyway, tells me that he had a totally different intention when doing it, and he wanted to depict a powerful princess, in the center of SuperTuxKart s universe, celebrating the new engine.

About the panties showing every now and then, I ve been told that it s something so hard to see that in fact you would really have to open the model itself to view them. I m not saying that I like them though, I think it would have been better if Sara would have had short pants under the skirt, if she was going to drive the snowmobile with a dress, but I m not sure if that s something important enough to condemn the game. The original girl mentioned at the beginning of this post seems to have found the animation funny, started laughing, and said that Sara is very silly, and that was all. It s probably something more silly than naughty, I guess. Even though, as I said, it s something I don t like too much. I don t have to agree with STK developers in everything. I guess.

There s one thing I would like to highlight about my conversations with the developers of SuperTuxKart, though. I like them. They seem to be as concerned about the wellbeing of kids as I am, they have their own ethic norms of what s acceptable and what s not, and they want to do something to be proud of. Sometimes, many of these conflicts arise from a lack of trust. When I first saw the screenshots with the girl in bikini and the panties showing, I was honestly concerned about the direction the project was taking. After having talked with the developers, I am more calmed about it, because they seem to have their heart in the right place, they care, they are motivated and they work hard. I don t know if a princess would be my first choice for a main female character, but at least their intention seems to be to give some girls a sensible role model in the game with who they can identify.

Today at 18:00 UTC (this evening for me) there will be a Debian Publicity Team IRC meeting (open meeting, everybody invited), and I m very happy because it will be the first meeting that I know of, since I joined the team (years!). Being part of the Publicity team There are many tasks handled by Publicity, and when I joined, I supposed that I was going to be part of a team with many members and well structured. And it was true but not as I imagined. Publicity is a great team, in the sense that it accepts contributions from many people, and the few core members do an amazing work: on one side, get things done; on the other side, integrate all those occasional contributions from the wider community. But there are fewer core contributors than what one would expect by the output of the team. I would say we are maximum 10 people (out of 353 voters, 1033 Debian Developers, and 1197 contributors in the Debian Community in 2015). And as far as I know, everybody is member of some other teams too (I m a translator, others in website team, sysadmins, packaging teams and now we have a member sharing Publicity membership with DPL-ship!). Organisation around the tasks Publicity regular tasks (announcements, the newsletter Debian Project News , posting in social networks and in bits.debian.org, and other ) are all well defined and documented, in order to allow anybody jump in and help, and this is great, because it ensures a way for contributions to arrive the wide audience from the very first day: you pick something, you follow the instructions, and you re done. I love this approach, because I tend to prefer to follow instructions than to create something, and my Debian time is made of small chunks at random days/times. But sometimes I feel that we all work alone , in something like a cold, robotic do-ocracy, and I also wonder how many people don t contribute or become regular contributors just because they don t understand the procedures, or they don t like them, or other reasons IRC meeting IRC is something that I use only for contributing to free software, not in other parts of my life. I like IRC, it s productive and fun, but I m not always there, and I don t save logs when I am idle, and I usually prefer email for communication. However, I try to be more present in the Debian IRC channels of the teams where I contribute, because I ve learned that it plays a big role in feeling at home in Debian . Currently you can find me in #debconf-team #debconf15-germany #debian-i18n #debian-l10n-spanish #debian-publicity #debian-women and #debian-www . I ve attended some meetings in IRC (MediaGoblin monthly meetings, and DebConf15 meetings) and I ve learned about MeetBot and more or less how to chair an IRC meeting. Today it will be my first time chairing, it s hard to emulate so great chairs as Marga or Chris Webber, bu I hope I do it decently, and we all have a nice time knowing each other and sharing ideas for the Publicity team. Want to attend? All the details (when, where, agenda ) in the wiki page of the meeting. See you in a few hours!
Filed under: My experiences and opinion Tagged: Communities, Contributing to libre software, Debian, English, Free Software, IRC, libre software, Project Management

Gnome Shell, besides providing the main user interface for GNOME 3, is a Javascript shell with bindings to many native interfaces that allow e.g. Window manipulation, graphics rendering and animations, compositing, etc. It also allows developers to write extensions changing Gnome Shell s behavior. Less known is that it is possible to replace the entire Javascript code base that Gnome Shell uses. It can be useful to hack on Gnome Shell itself (no need to fiddle with system files, or, since 3.12, no need to rebuild libgnome-shell.so), but it can also be used to implement a completely new User Interface in Javascript. I m starting to experiment with the latter, because I want to try building a window manager that fits my needs, while keeping away the boring details of EWMH, xinerama, and other X11 things. And because it s fun. But baby steps, first: let s bootstrap a Hello world with Gnome shell.

• Create a directory that will hold your code.
• In that directory, create a ui subdirectory.
• In that ui directory, create a environment.js file, with the following contents:
  

  const Shell = imports.gi.Shell;
  function init()  
    window.global = Shell.Global.get();
   
  

• In the same directory, create a main.js file, with the following contents:
  

  const St = imports.gi.St;
  function start()  
    let text = new St.Label(  text: "Hello, world!"  );
    global.stage.add_actor(text);
    global.stage.show();
   
  

• Run Gnome Shell with your code:
  

  $ GNOME_SHELL_JS=/path/to/parent/of/ui gnome-shell
  

  You may want to run this in a separate X server (I use Xephyr)

I tested this with Gnome Shell 3.14. Trying various older versions, I got different results for reasons I don t know. 3.4 doesn t display anything unless, paradoxically, global.stage.show() is removed, and 3.8 doesn t display anything no matter what. I guess the next step is to go through some Clutter tutorials and transpose them to Javascript. Update: On the other hand, a lot of the window managing is still done by mutter under the hood, which doesn t leave a lot of space for something really different.

This weekend has been all about migrations:

Host Migrations

I've migrated several more systems to the Jessie release of Debian GNU/Linux. No major surprises, and now I'm in a good state. I have 18 hosts, and now 16 of them are running Jessie. One of them I won't touch for a while, and the other is a KVM-host which runs about 8 guests - so I won't upgraded that for a while (because I want to schedule the shutdown of the guests for the host-reboot).

Password Migrations

I've started migrating my passwords to pass, which is a simple shell wrapper around GPG. I generated a new password-managing key, and started migrating the passwords. I dislike that account-names are stored in plaintext, but that seems known and unlikely to be fixed. I've "solved" the problem by dividing all my accounts into "Those that I wish to disclose post-death" (i.e. "banking", "amazon", "facebook", etc, etc), and those that are "never to be shared". The former are migrating, the latter are not. (Yeah I'm thinking about estates at the moment, near-death things have that effect!)

Common Lisp users are very happy to use Quicklisp when it comes to downloading and maintaining dependencies between their own code and the librairies it is using. Sometimes I am pointed that when compared to other programming languages Common Lisp is lacking a lot in the batteries included area. After having had to package about 50 common lisp librairies for debian I can tell you that I politely disagree with that. And this post is about the tool and process I use to maintain all those librairies. Quicklisp is good at ensuring a proper distribution of all those libs it supports and actually tests that they all compile and load together, so I've been using it as my upstream for debian packaging purposes. Using Quicklisp here makes my life much simpler as I can grovel through its metadata and automate most of the maintenance of my cl related packages. It's all automated in the ql-to-deb software which, unsurprisingly, has been written in Common Lisp itself. It's a kind of a Quicklisp client that will fetch Quicklisp current list of releases with version numbers and compare to the list of managed packages for debian in order to then build new version automatically. The current workflow I'm using begins with using ql-to-deb is to check for the work to be done today:

$ /vagrant/build/bin/ql-to-deb check
Fetching "http://beta.quicklisp.org/dist/quicklisp.txt"
Fetching "http://beta.quicklisp.org/dist/quicklisp/2015-04-07/releases.txt"
update: cl+ssl cl-csv cl-db3 drakma esrap graph hunchentoot local-time lparallel nibbles qmynd trivial-backtrace
upload: hunchentoot

After careful manual review of the automatic decision, let's just update all what check decided would have to be:

$ /vagrant/build/bin/ql-to-deb update
Fetching "http://beta.quicklisp.org/dist/quicklisp.txt"
Fetching "http://beta.quicklisp.org/dist/quicklisp/2015-04-07/releases.txt"
Updating package cl-plus-ssl from 20140826 to 20150302.
     see logs in "//tmp/ql-to-deb/logs//cl-plus-ssl.log"
Fetching "http://beta.quicklisp.org/archive/cl+ssl/2015-03-02/cl+ssl-20150302-git.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/cl+ssl-20150302-git.tgz"
      md5: 61d9d164d37ab5c91048827dfccd6835
Building package cl-plus-ssl
Updating package cl-csv from 20140826 to 20150302.
     see logs in "//tmp/ql-to-deb/logs//cl-csv.log"
Fetching "http://beta.quicklisp.org/archive/cl-csv/2015-03-02/cl-csv-20150302-git.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/cl-csv-20150302-git.tgz"
      md5: 32f6484a899fdc5b690f01c244cd9f55
Building package cl-csv
Updating package cl-db3 from 20131111 to 20150302.
     see logs in "//tmp/ql-to-deb/logs//cl-db3.log"
Fetching "http://beta.quicklisp.org/archive/cl-db3/2015-03-02/cl-db3-20150302-git.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/cl-db3-20150302-git.tgz"
      md5: 578896a3f60f474742f240b703f8c5f5
Building package cl-db3
Updating package cl-drakma from 1.3.11 to 1.3.13.
     see logs in "//tmp/ql-to-deb/logs//cl-drakma.log"
Fetching "http://beta.quicklisp.org/archive/drakma/2015-04-07/drakma-1.3.13.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/drakma-1.3.13.tgz"
      md5: 3b548bce10728c7a058f19444c8477c3
Building package cl-drakma
Updating package cl-esrap from 20150113 to 20150302.
     see logs in "//tmp/ql-to-deb/logs//cl-esrap.log"
Fetching "http://beta.quicklisp.org/archive/esrap/2015-03-02/esrap-20150302-git.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/esrap-20150302-git.tgz"
      md5: 8b198d26c27afcd1e9ce320820b0e569
Building package cl-esrap
Updating package cl-graph from 20141106 to 20150407.
     see logs in "//tmp/ql-to-deb/logs//cl-graph.log"
Fetching "http://beta.quicklisp.org/archive/graph/2015-04-07/graph-20150407-git.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/graph-20150407-git.tgz"
      md5: 3894ef9262c0912378aa3b6e8861de79
Building package cl-graph
Updating package hunchentoot from 1.2.29 to 1.2.31.
     see logs in "//tmp/ql-to-deb/logs//hunchentoot.log"
Fetching "http://beta.quicklisp.org/archive/hunchentoot/2015-04-07/hunchentoot-1.2.31.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/hunchentoot-1.2.31.tgz"
      md5: 973eccfef87e81f1922424cb19884d63
Building package hunchentoot
Updating package cl-local-time from 20150113 to 20150407.
     see logs in "//tmp/ql-to-deb/logs//cl-local-time.log"
Fetching "http://beta.quicklisp.org/archive/local-time/2015-04-07/local-time-20150407-git.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/local-time-20150407-git.tgz"
      md5: 7be4a31d692f5862014426a53eb1e48e
Building package cl-local-time
Updating package cl-lparallel from 20141106 to 20150302.
     see logs in "//tmp/ql-to-deb/logs//cl-lparallel.log"
Fetching "http://beta.quicklisp.org/archive/lparallel/2015-03-02/lparallel-20150302-git.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/lparallel-20150302-git.tgz"
      md5: dbda879d0e3abb02a09b326e14fa665d
Building package cl-lparallel
Updating package cl-nibbles from 20141106 to 20150407.
     see logs in "//tmp/ql-to-deb/logs//cl-nibbles.log"
Fetching "http://beta.quicklisp.org/archive/nibbles/2015-04-07/nibbles-20150407-git.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/nibbles-20150407-git.tgz"
      md5: 2ffb26241a1b3f49d48d28e7a61b1ab1
Building package cl-nibbles
Updating package cl-qmynd from 20141217 to 20150302.
     see logs in "//tmp/ql-to-deb/logs//cl-qmynd.log"
Fetching "http://beta.quicklisp.org/archive/qmynd/2015-03-02/qmynd-20150302-git.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/qmynd-20150302-git.tgz"
      md5: b1cc35f90b0daeb9ba507fd4e1518882
Building package cl-qmynd
Updating package cl-trivial-backtrace from 20120909 to 20150407.
     see logs in "//tmp/ql-to-deb/logs//cl-trivial-backtrace.log"
Fetching "http://beta.quicklisp.org/archive/trivial-backtrace/2015-04-07/trivial-backtrace-20150407-git.tgz"
Checksum test passed.
     File: "/tmp/ql-to-deb/archives/trivial-backtrace-20150407-git.tgz"
      md5: 762b0acf757dc8a2a6812d2f0f2614d9
Building package cl-trivial-backtrace

Quite simple. To be totally honnest, I first had a problem with the parser generator library esrap wherein the README documentation changed to be a README.org file, and I had to tell my debian packaging about that. See the 0ef669579cf7c07280eae7fe6f61f1bd664d337e commit to ql-to-deb for details. What about trying to install those packages locally? That's usually a very good test. Sometimes some dependencies are missing at the dpkg command line, so another apt-get install -f is needed:

$ /vagrant/build/bin/ql-to-deb install
sudo dpkg -i /tmp/ql-to-deb/cl-plus-ssl_20150302-1_all.deb /tmp/ql-to-deb/cl-csv_20150302-1_all.deb /tmp/ql-to-deb/cl-csv-clsql_20150302-1_all.deb /tmp/ql-to-deb/cl-csv-data-table_20150302-1_all.deb /tmp/ql-to-deb/cl-db3_20150302-1_all.deb /tmp/ql-to-deb/cl-drakma_1.3.13-1_all.deb /tmp/ql-to-deb/cl-esrap_20150302-1_all.deb /tmp/ql-to-deb/cl-graph_20150407-1_all.deb /tmp/ql-to-deb/cl-hunchentoot_1.2.31-1_all.deb /tmp/ql-to-deb/cl-local-time_20150407-1_all.deb /tmp/ql-to-deb/cl-lparallel_20150302-1_all.deb /tmp/ql-to-deb/cl-nibbles_20150407-1_all.deb /tmp/ql-to-deb/cl-qmynd_20150302-1_all.deb /tmp/ql-to-deb/cl-trivial-backtrace_20150407-1_all.deb
(Reading database ... 79689 files and directories currently installed.)
Preparing to unpack .../cl-plus-ssl_20150302-1_all.deb ...
Unpacking cl-plus-ssl (20150302-1) over (20140826-1) ...
Selecting previously unselected package cl-csv.
Preparing to unpack .../cl-csv_20150302-1_all.deb ...
Unpacking cl-csv (20150302-1) ...
Selecting previously unselected package cl-csv-clsql.
Preparing to unpack .../cl-csv-clsql_20150302-1_all.deb ...
Unpacking cl-csv-clsql (20150302-1) ...
Selecting previously unselected package cl-csv-data-table.
Preparing to unpack .../cl-csv-data-table_20150302-1_all.deb ...
Unpacking cl-csv-data-table (20150302-1) ...
Selecting previously unselected package cl-db3.
Preparing to unpack .../cl-db3_20150302-1_all.deb ...
Unpacking cl-db3 (20150302-1) ...
Preparing to unpack .../cl-drakma_1.3.13-1_all.deb ...
Unpacking cl-drakma (1.3.13-1) over (1.3.11-1) ...
Preparing to unpack .../cl-esrap_20150302-1_all.deb ...
Unpacking cl-esrap (20150302-1) over (20150113-1) ...
Preparing to unpack .../cl-graph_20150407-1_all.deb ...
Unpacking cl-graph (20150407-1) over (20141106-1) ...
Preparing to unpack .../cl-hunchentoot_1.2.31-1_all.deb ...
Unpacking cl-hunchentoot (1.2.31-1) over (1.2.29-1) ...
Preparing to unpack .../cl-local-time_20150407-1_all.deb ...
Unpacking cl-local-time (20150407-1) over (20150113-1) ...
Preparing to unpack .../cl-lparallel_20150302-1_all.deb ...
Unpacking cl-lparallel (20150302-1) over (20141106-1) ...
Preparing to unpack .../cl-nibbles_20150407-1_all.deb ...
Unpacking cl-nibbles (20150407-1) over (20141106-1) ...
Preparing to unpack .../cl-qmynd_20150302-1_all.deb ...
Unpacking cl-qmynd (20150302-1) over (20141217-1) ...
Preparing to unpack .../cl-trivial-backtrace_20150407-1_all.deb ...
Unpacking cl-trivial-backtrace (20150407-1) over (20120909-2) ...
Setting up cl-plus-ssl (20150302-1) ...
dpkg: dependency problems prevent configuration of cl-csv:
 cl-csv depends on cl-interpol; however:
  Package cl-interpol is not installed.
dpkg: error processing package cl-csv (--install):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of cl-csv-clsql:
 cl-csv-clsql depends on cl-csv; however:
  Package cl-csv is not configured yet.
dpkg: error processing package cl-csv-clsql (--install):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of cl-csv-data-table:
 cl-csv-data-table depends on cl-csv; however:
  Package cl-csv is not configured yet.
dpkg: error processing package cl-csv-data-table (--install):
 dependency problems - leaving unconfigured
Setting up cl-db3 (20150302-1) ...
Setting up cl-drakma (1.3.13-1) ...
Setting up cl-esrap (20150302-1) ...
Setting up cl-graph (20150407-1) ...
Setting up cl-local-time (20150407-1) ...
Setting up cl-lparallel (20150302-1) ...
Setting up cl-nibbles (20150407-1) ...
Setting up cl-qmynd (20150302-1) ...
Setting up cl-trivial-backtrace (20150407-1) ...
Setting up cl-hunchentoot (1.2.31-1) ...
Errors were encountered while processing:
 cl-csv
 cl-csv-clsql
 cl-csv-data-table

Let's make sure that our sid users will be happy with the update here:

$ sudo apt-get install -f
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  g++-4.7 git git-man html2text libaugeas-ruby1.8 libbind9-80
  libclass-isa-perl libcurl3-gnutls libdns88 libdrm-nouveau1a
  libegl1-mesa-drivers libffi5 libgraphite3 libgssglue1 libisc84 libisccc80
  libisccfg82 liblcms1 liblwres80 libmpc2 libopenjpeg2 libopenvg1-mesa
  libpoppler19 librtmp0 libswitch-perl libtiff4 libwayland-egl1-mesa luatex
  openssh-blacklist openssh-blacklist-extra python-chardet python-debian
  python-magic python-pkg-resources python-six ttf-dejavu-core ttf-marvosym
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
  cl-interpol
The following NEW packages will be installed:
  cl-interpol
0 upgraded, 1 newly installed, 0 to remove and 51 not upgraded.
3 not fully installed or removed.
Need to get 20.7 kB of archives.
After this operation, 135 kB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://ftp.fr.debian.org/debian/ sid/main cl-interpol all 0.2.1-2 [20.7 kB]
Fetched 20.7 kB in 0s (84.5 kB/s)
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Selecting previously unselected package cl-interpol.
(Reading database ... 79725 files and directories currently installed.)
Preparing to unpack .../cl-interpol_0.2.1-2_all.deb ...
Unpacking cl-interpol (0.2.1-2) ...
Setting up cl-interpol (0.2.1-2) ...
Setting up cl-csv (20150302-1) ...
Setting up cl-csv-clsql (20150302-1) ...
Setting up cl-csv-data-table (20150302-1) ...

All looks fine, time to sign those packages. There's a trick here, where you want to be sure you're using a GnuPG setup that allows you to enter your passphrase only once, see ql-to-deb vm setup for details, and the usual documentations about all that if you're interested into the details.

$ /vagrant/build/bin/ql-to-deb sign
 signfile /tmp/ql-to-deb/cl-plus-ssl_20150302-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-plus-ssl_20150302-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/cl-csv_20150302-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-csv_20150302-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/cl-db3_20150302-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-db3_20150302-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/cl-drakma_1.3.13-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-drakma_1.3.13-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/cl-esrap_20150302-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-esrap_20150302-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/cl-graph_20150407-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-graph_20150407-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/hunchentoot_1.2.31-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/hunchentoot_1.2.31-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/cl-local-time_20150407-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-local-time_20150407-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/cl-lparallel_20150302-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-lparallel_20150302-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/cl-nibbles_20150407-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-nibbles_20150407-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/cl-qmynd_20150302-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-qmynd_20150302-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files
 signfile /tmp/ql-to-deb/cl-trivial-backtrace_20150407-1.dsc 60B1CB4E
 signfile /tmp/ql-to-deb/cl-trivial-backtrace_20150407-1_amd64.changes 60B1CB4E
Successfully signed dsc and changes files

Ok, with all tested and signed, it's time we upload our packages on debian servers for our dear debian users to be able to use newer and better versions of their beloved Common Lisp librairies:

$ /vagrant/build/bin/ql-to-deb upload
Trying to upload package to ftp-master (ftp.upload.debian.org)
Checking signature on .changes
gpg: Signature made Sat 02 May 2015 05:06:48 PM MSK using RSA key ID 60B1CB4E
gpg: Good signature from "Dimitri Fontaine <dim@tapoueh.org>"
Good signature on /tmp/ql-to-deb/cl-plus-ssl_20150302-1_amd64.changes.
Checking signature on .dsc
gpg: Signature made Sat 02 May 2015 05:06:46 PM MSK using RSA key ID 60B1CB4E
gpg: Good signature from "Dimitri Fontaine <dim@tapoueh.org>"
Good signature on /tmp/ql-to-deb/cl-plus-ssl_20150302-1.dsc.
Uploading to ftp-master (via ftp to ftp.upload.debian.org):
  Uploading cl-plus-ssl_20150302-1.dsc: done.
  Uploading cl-plus-ssl_20150302.orig.tar.gz: done.
  Uploading cl-plus-ssl_20150302-1.debian.tar.xz: done.
  Uploading cl-plus-ssl_20150302-1_all.deb: done.
  Uploading cl-plus-ssl_20150302-1_amd64.changes: done.
Successfully uploaded packages.

Of course the same text or abouts is then repeated for all the other packages. Enjoy using Common Lisp in debian! Oh and remember, the only reason I've written ql-to-deb and signed myself up to maintain those upteens Common Lisp librairies as debian package is to be able to properly package pgloader in debian, as you can see at https://packages.debian.org/sid/pgloader and in particular in the Other Packages Related to pgloader section of the debian source package for pgloader at https://packages.debian.org/source/sid/pgloader. That level of effort is done to ensure that we respect the Debian Social Contract wherein debian ensures its users that it's possible to rebuild anything from sources as found in the debian repositories.