This post is almost a carbon copy of the Kallithea 0.2 release notes. Kallithea project have just released Kallithea 0.2. Kallithea is a Python-based GPLv3 source code management software for web-based hosting of Mercurial and Git repositories. This release brings many changes since 0.1. Notably, pull requests system have been improved, making contributing changes more robust. The visual appearance has also been refined: modern font-based symbolic icons from FontAwesome and GitHub Octicons have replaced the previously used bitmap icons, and revision graphs are now drawn with HiDPI display support. Kallithea now supports Mercurial 3.3 and Dulwich 0.9.9. Several fixes in the database code boosted performance significantly. We have also updated our Javascript libraries: jQuery, CodeMirror and Mergely. Javascript and CSS code have been cleaned up, with less and less code depending on Yahoo UI library. Since 0.1 we have discovered two security issues, so all users are strongly recommended to upgrade. For more details on these issues, please see our Security Notices page For more information, see https://kallithea-scm.org/ or http://docs.kallithea-scm.org/. The summary of the changes since 0.1 release is below. Bug fixes:

• forms: add CSRF protection to all forms CVE-2015-0276
• api: don t send internal data unless asked for it CVE-2015-0260
• middleware: fix force_tls typo in force_https backward compatibility code (Issue #44)
• rebranddb: update user extern_type and _name to internal instead of kallithea (Issue #38)
• git: do not fail if git is not installed and has been removed from __init__.py backends
• git: fix 'name' must be bytestring, not unicode error on browsing changesets
• git: close SubprocessIOChunker inputstream used for git (Issue #32)
• git: introduce hack for handling git failure on --depth cloning (Issue #33)
• git: fix version detection with unexpected version string (Issue #71)
• git: preserve line endings when calling out to git
• hg: fix clone from svn+http urls using hg-svn (Issue #72)
• migrate: add missing import (Issue #29)
• pullrequests: handle pull requests to empty repos (Issue #27)
• pullrequests: when creating PRs, fix handling of new ajax requests while other ajax requests are pending
• pullrequests: fix updates for PRs between different repositories
• pullrequests: preserve query parameters in pull request overview paging links
• compare: workaround unexpected Mercurial behaviour when finding ancestor of null rev
• javascript: remove trailing comma to please IE8 (Issue #39)
• download: fix for zip file downloads not being valid zips (Issue #35)
• helpers: fix crash on new users without email (Issue #28)
• middleware: change middleware ordering so we don t buffer hgweb output in redirect
• bin: give kallithea_config.py #!/usr/bin/env python so it can be executed directly
• bin: reintroduce cleanup-repos paster command
• bin: fix update_repoinfo command
• urlify: don t include trailing punctuation in markup
• urlify: markup of hashes in ()
• styling: don t loop on trying to load kallithea-logo.png after kallithea-logo.svg failed
• error: don t crash on /error/document urls
• error: don t crash on response without status
• admin: prevent deletion of users that are owners of a repository/user group (Issue #64)
• admin: make settings hook name field smaller so it is less likely to overlap with the value field
• admin: when scanning for repos to add, ignore removed groups
• auth_crowd: fix Admin Group Membership when using the Atlassian Crowd plugin
• auth: enable selecting PAM authentication module
• auth_ldap: fix user automatically activated with LDAP authentication (Issue #78)
• db: fix beaker cache key for user groups don t collide with users
• db: fix handling of (invalid) unicode email addresses
• db: don t use sql in on empty sets in compare avoid warning of bad performance
• db: to the extent it makes sense to have a one byte size limit, make it 255 instead of 256 to please MySQL (Issue #96)
• files: use current revision as default for show at (Issue #31)
• files: fix HTML injection via file names
• files: use forward slash as path separator for consistency
• files: don t use HTML encoding where not needed (Issue #74).
• feed: urlify and escape the commit description
• diff: fix diff of renamed files with whitespace in their names
• diff: don t split lines on bare CR as python splitlines do
• mergely: update Mergely to v3.3.9 (Issue #83)
• jquery: upgrade to 1.11.1
• codemirror: cleanup of integration and update to version 4.7
• setup: constrain dulwich to version 0.9.9, fixes CVE-2015-0838
• gists: add missing formencode.htmlfill import
• ini: Make celeryd.log.level default values uppercase lowercase might not be recognized and cause failure
• notification: handle unicode translations (Issue #95)
• repos: fix redirect after repo creation (and user/group add focus) (Issue #98)
• changeset: don t crash on malformed whitespace parameter return 400 Bad Request
• stats: fix display when no data ready yet
• stats: fix show more link

Features and improvements:

• ui: use font based icons FontAwesome and Git Hub Octicons from fontello
• ui: HiDPI / Retina support in graphs and other places
• paster: add install-iis command to automate IIS handler generation and update documentation
• email templates: send text/plain part as well
• docs: lots of improvements of structure and content and language
• translations: updated using WebLate
• admin: show links to added repos after repo scan
• admin: show links to created users and groups
• admin: when scanning for repos without remove , report the repos that could/would be removed
• setup: support Mercurial 3.3
• db: drop most lazy joins significant performance improvement
• db: add missing indices significant performance improvement
• db: don t commit session if cache invalidation didn t change anything improves performance
• comment: show full username in emails for PR/changeset comments
• changelog: added Settings to repo context menu
• utils: better display of ages >1 year when in short form
• remotes: add support to clone from Mercurial repositories over ssh
• hg: reimplement branch head listings more efficiently
• pullrequests: use same code and table ui for Repo Pull Requests and My Pull Requests
• pullrequests: add delete button to My Pull Requests overview
• pullrequests: on my pullrequests show the user s own vote as second column next to latest vote
• pullrequests: add option for adding repo owner as reviewer if none assigned
• pullrequests: make top pane one big form with meta data and one save button and preview changes in pane below more like when creating the PR
• pullrequests: make update functionality a separate submit button in the big form, automatically enabled when an update changeset is chosen
• pullrequests: show graph next to the list of changesets that are available for update
• pullrequests: for available changesets, also show changesets that not are descendants and thus not directly available but will be merged in
• pullrequests: show unrelated branch heads in the graph too show the problem instead of trying to explain it
• pullrequests: show pending reviewer changes with (not saved)
• pullrequests: show comment count in top of summary box
• diff: improved parsing, be more strict and fail with good error messages
• diff: handle GIT delta binary patches
• diff: show renamed files as that
• diff: make trailing spaces more visible make vertical line less transparent
• diff: show CR (\r) in diffs
• comment: warn before leaving page with open comments
• comment: clarify shown comment count (inline vs general)
• comment: extend next/previous links to global comments (Issue #91)

Changes (might influence workflows):

• i18n: en translation has been removed make sure to update your .ini files to have lang = (nothing)
• ini: turn show_revision_number off by default
• pullrequests: don t give changeset unreviewed status just because it is added to a PR
• pullrequests: tweak readable titles just append /_/ and branch name
• pullrequests: only author and admin can close
• pullrequests: don t add repo owner as reviewer in all PRs
• pullrequests: PRs between different repos should default to same destination/source branch
• pullrequests: use -- as separator in the automatic text when updating pull requests and is it to put latest updates first
• diff: don t group diffs based on the kind of change
• admin: set current user as owner when running repo scan
• changelog: change default view to 100 changesets, max to 2000
• date representation: use ISO8601 rather than a specific locale
• user groups: drop add/remove all buttons for member selection they are too dangerous and multi-select can do the same
• hg: drop support for hg web.hidden
• template: link the last revision in the file browser to the changeset

Cleanup:

• code: Fix typos and language in comments and code
• ui: Correct capitalization and improved English text in the UI
• javascript: use strict mode
• tags: rename old non-Kallithea tags and prefix them with 0.0 to show that Kallithea is superior
• javascript: more jQuery, less YUI, less clutter
• templates: better structure, less cut n paste coding
• html: various improvements
• css: various improvements
• exceptions: avoid silently catching exceptions especially avoid catch all
• tests: stability fixes
• tests: remove large unused test fixture
• whitespacecleanup.sh: run it regularly to ensure consistent whitespace and avoid spurious changes
• controllers: consistently use formfill.render with force_defaults=False
• gists: use allow_extra_fields and filter_extra_fields in gist forms like in most other forms
• json: always use kallithea.lib.compat for json and nothing else
• json: drop simplejson fallback and just use stdlib
• db: check for None instead of boolean false if that is what we mean

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 15.25 hours on Debian LTS. In that time I did the following:

• CVE triage: I pushed 37 commits to the security tracker and contacted 20 maintainers about security issues affecting their packages.
• I started a small helper script based on the new JSON output of the security tracker (see #761859 for details). It s not ready yet but will make it easier to detect issues where the LTS team lags behind the security team, and other divergences like this and will speed up future CVE triage work (once done).
• I sent DLA-174-1 (tcpdump update fixing 3 CVE) after having received a debdiff from the Romain Fran oise.
• I prepared DLA-175-1 on gnupg, fixing 3 CVE.
• I prepared DLA-180-1 on gnutls26, fixing 3 CVE.

That s it for the paid work. But still about LTS, I proposed two events for Debconf 15:

• Inner workings of an unusual team in Debian: the Long Term Support team: a generic presentation of the team and the project;
• Preparing for Wheezy LTS: a work session between the security team and the LTS team.

In my last Freexian LTS report, I mentioned briefly that it would be nice to have a logo for the LTS project. Shortly after I got a first logo prepared by Damien Escoffier and a few more followed: they are available on a wiki page (and the logo you see above is from him!). Following a suggestion of Paul Wise, I registered the logo request on another wiki page dedicated to artwork requests. That kind of collaboration is awesome! Thanks to all the artists involved in Debian. Debian packaging Django. This month has seen no less than 3 upstream point releases packaged for Debian (1.7.5, 1.7.6 and 1.7.7) and they have been accepted by the release team into Jessie. I m pleased with this tolerance as I have argued the case for it multiple times in the past given the sane upstream release policy (bugfix only in a given released branch). Python code analysis. I discovered a few months ago a tool combining the power of multiple Python code analysis tools: it s prospector. I just filed a Request for Package for it (see #781165) and someone already volunteered to package it, yay \o/ update-rc.d and systemd. While working on a Kali version based on Jessie, I got hit by what boils down to a poor interaction between systemd and update-rc.d (see #746580) and after some exchanges with other affected users I raised the severity to serious as we really ought to do something about it before release. I also opened #781155 on openbsd-inetd as its usage of inetd.service instead of openbsd-inetd.service (which is only provided as a symlink to the former) leads to multiple small issues. Misc Debian France. The general assembly is over and the new board elected its new president: it s now official, I m no longer Debian France s president. Good luck to Nicolas Dandrimont who took on this responsibility. Salt s openssh formula. I improved salt s openssh formula to make it possible to manage the /etc/ssh/ssh_known_hosts file referencing the public SSH keys of other managed minions. Tendenci.com. I was looking for a free software solution to handle membership management of a large NPO and I discovered Tendenci. It looked very interesting feature wise and written with a language/framework that I enjoy (Python/Django). But while it s free software, there s no community at all. The company that wrote it released it under a free software license and it really looks like that they did intend to build a community but they failed at it. When I looked their development forums were web-based and mostly empty with only initial discussion of the current developers and no reply from anybody there s also no mention of an IRC channel or a mailing list. I sent them a mail to see what kind of collaboration we could expect if we opted for their software and got no reply. A pity, really. What free software membership management solution would you use when you have more than 10000 members to handle and when you want to use the underlying database to offer SSO authentication to multiple external services? Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

Several improvements have been made to UDD s Bug Search and Maintainer Dashboard recently. On the Maintainer Dashboard side, the main new feature is a QA checks table that provides an overview of results from lintian, reproducible builds, piuparts, and ci.debian.net. Check the dashboard for the Ruby team for an example. Also, thanks to Daniel Pocock, the TODO items can now be exported as iCalendar tasks. Bugs Search now has much better JSON and YAML outputs. It s probably a good start if you want to do some data-mining on bugs. Packages can now be selected using the same form as the Maintainer Dashboard s one, which makes it easy to build your own personal bug list, and will suppress the need for some of the team-specific listings. Many bugs have been fixed too. More generally, thanks to the work of Christophe Siraut, the code is much better now, with a clean separation of the data analysis logic and the rendering sides that will make future improvements easier. As the reminder, it s quite easy to hack on UDD (even if you are not a DD). Please report bugs, including about additional features you would like to see!

For many the PHP JSON extension license might look like a storm in a teacup, but for many Linux distributions the bits of the free software licenses are very important. That was the case when Debian decided (#692613) to remove the non-free JSON extension as part of the transition to PHP 5.5 in May 2013 (after the Debian 7 release). This change was done with the help of Remi Collet (from Fedora / Red Hat) who wrote an alternative extension based on JSON-C implementation. A similar change was done by other Linux distributions, and this became the defacto standard for most Linux users. The situation has recently changed with the acceptance of the Jakub Zelenka s jsond RFC to replace the current non-free implementation with a free one. The change was committed to the code base on early February (Closing #63520) and expected to be released later this year as part of PHP7.
Filed under: Free software licenses, PHP

My LTS February In February I didn't get much LTS work done, as other projects grabbed my attention. I started working on #761859 but that took a lot more time than initially thought, as the database design has been bended several times to make it fit for some new usecases. For example, "end-of-life" information is saved in the urgency field, making this one of several reasons why a bug can have different urgencies in different releases (despite that a CVE always only has one urgency). There are more oddities in the code base which was written with the (now long gone) assumption to be a tracker for security issues in testing. Constantly adding more requirements and use cases didn't really help to make it more maintainable. I'm glad that at least since yesterday https://security-tracker.debian.org/tracker/data/json provides valid JSON, though probably still a bit of finetuning is needed before I'll consider the output stable and close the bug. In related news I've filed #780201: "new codename needed for oldstable (due to squeeze-lts) when stable becomes oldstable" today as else the security tracker will cease to work for squeeze LTS once we release jessie. And that would spoil that day a bit...

I was writing in the previous post about Occasional Rsnapshot and how I ended up writing it.

Just before releasing v1.2.1 I realized it would make sense Semantic Versioning which, in just a few words means:

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards-compatible manner, and
PATCH version when you make backwards-compatible bug fixes.

I just released Occasional Rsnapshot v1.3.1 which mainly fixed issue 4:

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 14.5 hours on Debian LTS. I worked mostly on CVE triage (41 commits in the security tracker) and organizational issues. One maintainer complained that he had not been kept in the loop for an LTS update of his package. After some discussion, I decided to change the way I did CVE triage so that any time that I add a package to our list of packages needing an update, I also send a mail to the maintainer, thus offering him the opportunity to step in. To make this sustainable, I wrote a small helper script that will generate a mail out of a template. And to kickstart the process I mailed all maintainers of packages that were already listed in our queue of packages to update. To improve the email generated, I requested a JSON export of the security tracker data (see discussions in #761859). In the mean time, Holger worked on this already and after a few iterations we did converge on an output format that will be really useful both for my needs in terms of CVE triage but also for the Package Tracker to be able to display the list of security vulnerabilities affecting each release (see #761730). Last but not least, I don t want to be the only one doing CVE triage for our LTS release so I documented the process in our wiki page. As a side note, I sponsored an e2fsprogs update prepared by Nguyen Cong and I sent the DLA for the embargoed samba update that had been prepared by Ivo de Decker (thanks to both of them!). Tryton Like last month, I invested again a copious amount of time on Tryton, fixing some bugs that were affecting me and improving the French chart of accounts to properly manage purchases and sales within the European Union. Here are some links for more details:

• #4523: Change account 58 into a real non-view account
• Preserve the user-supplied date on account moves when it s valid
• #4532: Rename some tax codes hosting base amounts to avoid having two codes with the same name
• #4568: Fix the credit note base tax code and tax code for the sales in France (took quite some discussion to double check this with an accountant)
• #4569: Track the amount invoiced in sales outside Europe
• #4570: Properly handle EU-sales and EU-purchases in the VAT report (this was by far the most complicated issue I dealt with, it took several days and multiple tries to get it right).

Debian I did some work on Distro Tracker, I fixed #777453 (password reset not working because the generated email was using an invalid From email) and #779247 (obsolete build reproducibility action items were not dropped). I also started to work on restructuring the mail handling in distro-tracker (cf #754913) but it s not public yet. While I have no plans to stop contributing to Debian (it s part of my day job!), I reduced my non-work related involvement by officially recognizing that I was no longer properly assuming some of my responsibilities and that I was following too many mailing lists and RSS feeds. The most notable changes are that I removed myself from the maintenance of dpkg, developers-reference, quilt, sql-ledger, and a few perl/python modules. Misc Voting software. Part of the reason why I m reducing my involvement in Debian is that I got more involved in Nouvelle Donne (a French political party) and in particular in the handling of its digital infrastructure (currently running on Ubuntu, doh!). As part of this, I was looking for free software to handle secure votes and elections (and if possible adhering to the principles of liquid democracy). There s no perfect solution and no clear winner. That said I started following the evolution of AgoraVoting because it seems to have a good momentum and has some interesting features (it already supports votes with ranked choices, supports good crypto, has been used for elections involving large numbers of voters in the context of Podemos in Spain). But it still has some ways to go to establish itself as a truly international and community-backed project. GDM bug. Due to my work on Kali, I filed a bug against GDM (this one has been quickly fixed upstream, it s still open in Debian) and another one against accountsservice to request the possibility to define the default graphical session. Dirvish formula for Salt. I contributed another formula to manage backups with dirvish. Thanks See you next month for a new summary of my activities.

One comment Liked this article? Click here. My blog is Flattr-enabled.

Weblate 2.2 has been released today. It comes with improved search, user interface cleanup and various other fixes. Full list of changes for 2.2:

• Performance improvements.
• Fulltext search on location and comments fields.
• New SVG/javascript based activity charts.
• Support for Django 1.8.
• Support for deleting comments.
• Added own SVG badge.
• Added support for Google Analytics.
• Improved handling of translation file names.
• Added support for monolingual JSON translations.
• Record component locking in a history.
• Support for editing source (template) language for monolingual translations.
• Added basic support for Gerrit.

You can find more information about Weblate on http://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used https://hosted.weblate.org/ as official translating service for phpMyAdmin, Gammu, Weblate itself and other projects. If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you. Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! PS: The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

Filed under: English phpMyAdmin SUSE Weblate 0 comments Flattr this!

I haven't blogged for a long time, but I've decided that I'm going to try to write again, at least about technical stuff. My plan was to blog about the projects I've been working on lately, the main one being the setup of the latest version of Kolab with the systems we already have at work, but I'll do that on the next days. Today I'm just going to make a list of the tools I use on a daily basis and my plans to start using additional ones in the near future. Shells, Terminals and Text Editors I do almost all my work on Z Shell sessions running inside tmux; for terminal emulation I use gnome-terminal on X, VX ConnectBot on Android systems and iTerm2 on Mac OS X. For text editing I've been using Vim for a long time (even on Mobile devices) and while I'm aware I don't know half of the things it can do, what I know is good enough for my day to day needs. In the past I also used Emacs as a programming editor and my main tool to write HTML, SGML and XML, but since I haven't really needed an IDE for a long time and I mainly use Lightweight Markup Languages I haven't used it for a long time (I briefly tried to use Org mode, but for some reason I ended up leaving it). Documentation formats and tools Since a long time ago I've been an advocate of Lightweight Markup Languages; I started to use LaTeX and Lout, then moved to SGML/XML formats (LinuxDoc and DocBook) and finally moved to plain text based formats. I started using Wiki formats (parsewiki) and soon moved to reStructuredText; I also use other markup languages like Markdown (for this blog, aka ikiwiki) and tried MultiMarkdown to replace reStructuredText for general use, but as I never liked Markdown syntax I didn't liked an extended version of it. While I've been using ReStructuredText for a long time, I recently found Asciidoctor and the Asciidoc format and I guess I'll be using it instead of rst whenever I can (I still need to try the slide backends and conversions to ODT, but if that works I guess I'll write all my new documents using Asciidoc). Programming languages I'm not a developer, but I read and patch a lot of free software code written on a lot of different programming languages (I wouldn't be able to write whole programs on most of them, but thanks to Stack Overflow I'm usually able to fix what I need). Anyway, I'm able to program in some languages; I write a lot of shell scripts and I go for Python and C when I need something more complicated. On the near future I plan to read about javascript programming and nodejs (I'll probably need it at work) and I already started looking at Haskell (I guess it was time to learn about functional programming and after reading about it, it looks like haskell is the way to go for me). Version Control For a long time I've been a Subversion user, at least for my own projects, but seems that everything has moved to git now and I finally started to use it (I even opened a github account) and plan to move all my personal subversion repositories at home and at work to git, including the move of all my debian packages from svn-buildpackage to git-buildpackage. Further Reading With the previous plans in mind, I've started reading a couple of interesting books:

• Learn You a Haskell by Miran Lipova a (http://learnyouahaskell.com/)
• Pro Git written by Scott Chacon and Ben Straub (http://git-scm.com/book/en/v2)

Now I just need to get enough time to finish reading them ... ;)

I ve been working on setting up a Docker container environment allowing to test the RuneStone Interactive server. RuneStone Interactive allows the publication of courses containing interactive Python examples, and while most of the content is static (the Python examples are run innside a Python interpreter implemented in JavaScript, hence locally in the JS VM of the Web browser), the tool also offers an environment allowing to monitor the progress of learners in a course, which is dynamic and is queried by the browser over AJAX APIs. That s the part which I wanted to be able to operate for test purposes. As it is a web2py application, it s not exactly obvious to gather all dependencies and run locally. Well, in fact it is, but I want to understand the architecture of the tool to be able to understand the deployment constraints, so making a docker image will help in this purpose. The result is the following :

• a container image which allows to test web2py over Apache (with mod_wsgi) on Debian (see it on github : olberger/docker-web2py-debian)
• a container image (reusing the previous one) which allows to test runestone (see it on github : olberger/runestone-web2py)

Now, it s easier to test the writing of a new course (yet another container above the latter one), and directly test for real.

TLPL; j'ai chang de logiciel pour la gestion de mon blog. TLDR; I have changed my blog from Drupal to Ikiwiki. Note: since this post uses ikiwiki syntax (i just copied it over here), you may want to read the original version instead of this one. will continue operating for a while to
give a chance to feed aggregators to catch that article. It will also
give time to the Internet archive to catchup with the static
stylesheets (it turns out it doesn't like Drupal's CSS compression at
all!) An archive will therefore continue being available on the
internet archive for people that miss the old stylesheet. Eventually, I will simply redirect the anarcat.koumbit.org URL to
the new blog location, . This will likely be my
last blog post written on Drupal, and all new content will be
available on the new URL. RSS feed URLs should not change.

Why I am migrating away from Drupal because it is basically impossible to
upgrade my blog from Drupal 6 to Drupal 7. Or if it is, I'll have to
redo the whole freaking thing again when Drupal 8 comes along. And frankly, I don't really need Drupal to run a blog. A blog was
originally a really simple thing: a web blog. A set of articles
written on the corner of a table. Now with Drupal, I can add
ecommerce, a photo gallery and whatnot to my blog, but why would I do
that? and why does it need to be a dynamic CMS at all, if I get so
little comments? So I'm switching to ikiwiki, for the following reason:

• no upgrades necessary: well, not exactly true, i still need to
  upgrade ikiwiki, but that's covered by the Debian package
  maintenance and I only have one patch to it, and there's no data migration! (the last such migration in ikiwiki was in 2009 and was fully supported)
• offline editing: this is a a big thing for me: i can just note
  things down and push them when I get back online
• one place for everything: this blog is where I keep my notes, it's
  getting annoying to have to keep track of two places for that stuff
• future-proof: extracting content from ikiwiki is amazingly
  simple. every page is a single markdown-formatted file. that's it.

Migrating will mean abandoning the
barlow theme, which was
seeing a declining usage anyways.

What So what should be exported exactly. There's a bunch of crap in the old
blog that i don't want: users, caches, logs, "modules", and the list
goes on. Maybe it's better to create a list of what I need to extract:

• nodes
  • title ([[ikiwiki/directive/meta]] title and guid tags, guid to avoid flooding aggregators)
  • body (need to check for "break comments")
  • nid (for future reference?)
  • tags (should be added as \[[!tag foo bar baz]] at the bottom)
  • URL (to keep old addresses)
  • published date ([[ikiwiki/directive/meta]] date directive)
  • modification date ([[ikiwiki/directive/meta]] updated directive)
  • revisions?
  • attached files
• menus
  • RSS feed
  • contact
  • search
• comments
  • author name
  • date
  • title
  • content
• attached files
  • thumbnails
  • links
• tags
  • each tag should have its own RSS feed and latest posts displayed

When Some time before summer 2015.

Who Well me, who else. You probably really don't care about that, so let'S
get to the meat of it.

How How to perform this migration... There are multiple paths:

• MySQL commandline: extracting data using the commandline mysql tool (drush sqlq ...)
• Views export: extracting "standard format" dumps from Drupal and
  parse it (JSON, XML, CSV?)

Both approaches had issues, and I found a third way: talk directly to
mysql and generate the files directly, in a Python script. But first,
here are the two previous approaches I know of.

MySQL commandline LeLutin switched using MySQL requests,
although he doesn't specify how content itself was migrated. Comments
importing is done with that script:

echo "select n.title, concat('  [[!comment  format=mdwn  username=\"', c.name, '\"  ip=\"', c.hostname, '\"  subject=\"', c.subject, '\"  date=\"', FROM_UNIXTIME(c.created), '\"  content=\"\"\" ', b.comment_body_value, ' \"\"\"]]') from node n, comment c, field_data_comment_body b where n.nid=c.nid and c.cid=b.entity_id;"   drush sqlc   tail -n +2   while read line; do if [ -z "$i" ]; then i=0; fi; title=$(echo "$line"   sed -e 's/[    ]\+ .*//' -e 's/ /_/g' -e 's/[:(),?/+]//g'); body=$(echo "$line"   sed 's/[^ ]*  //'); mkdir -p ~/comments/$title; echo -e "$body" > ~/comments/$title/comment_$i._comment; i=$((i+1)); done

Kind of ugly, but beats what i had before (which was "nothing"). I do think it is the good direction to take, to simply talk to the
MySQL database, maybe with a native Python script. I know the Drupal
database schema pretty well (still! this is D6 after all) and it's
simple enough that this should just work.

Views export [[!img 2015-02-03-233846_1440x900_scrot.png class="align-right" size="300x" align="center" alt="screenshot of views 2.x"]] mvc recommended views data export on Lelutin's
blog. Unfortunately, my experience with the views export interface has
been somewhat mediocre so far. Yet another reason why I don't like
using Drupal anymore is this kind of obtuse dialogs: I clicked through those for about an hour to get JSON output that
turned out to be provided by views bonus instead of
views_data_export. And confusingly enough, the path and
format_name fields are null in the JSON output
(whyyy!?). views_data_export unfortunately only supports XML,
which seems hardly better than SQL for structured data, especially
considering I am going to write a script for the conversion anyways. Basically, it doesn't seem like any amount of views mangling will
provide me with what i need. Nevertheless, here's the [[failed-export-view.txt]] that I was able to
come up with, may it be useful for future freedom fighters.

Python script I ended up making a fairly simple Python script to talk directly to
the MySQL database. The script exports only nodes and comments, and nothing else. It makes
a bunch of assumptions about the structure of the site, and is
probably only going to work if your site is a simple blog like mine,
but could probably be improved significantly to encompass larger and
more complex datasets. History is not preserved so no interaction is
performed with git.

Generating dump First, I imported the MySQL dump file on my local mysql server for easier
development. It is 13.9MiO!!

mysql -e 'CREATE DATABASE anarcatblogbak;'
ssh aegir.koumbit.net "cd anarcat.koumbit.org ; drush sql-dump"   pv   mysql anarcatblogbak

I decided to not import revisions. The majority (70%) of the content has
1 or 2 revisions, and those with two revisions are likely just when
the node was actually published, with minor changes. ~80% have 3
revisions or less, 90% have 5 or less, 95% 8 or less, and 98% 10 or
less. Only 5 articles have more than 10 revisions, with two having the
maximum of 15 revisions. Those stats were generated with:

SELECT title,count(vid) FROM anarcatblogbak.node_revisions group
by nid;

Then throwing the output in a CSV spreadsheet (thanks to
mysql-workbench for the easy export), adding a column numbering the
rows (B1=1,B2=B1+1), another for generating percentages
(C1=B1/count(B$2:B$218)) and generating a simple graph with
that. There were probably ways of doing that more cleanly with R,
and I broke my promise to never use a spreadsheet again, but then
again it was Gnumeric and it's just to get a rough idea. There are 196 articles to import, with 251 comments, which means an
average of 1.15 comment per article (not much!). Unpublished articles
(5!) are completely ignored. Summaries are also not imported as such (break comments are
ignored) because ikiwiki doesn't support post summaries.

Calling the conversion script The script is in [[drupal2ikiwiki.py]]. It is called with:

./drupal2ikiwiki.py -u anarcatblogbak -d anarcatblogbak blog -vv

The -n and -l1 have been used for first tests as well. Use this
command to generate HTML from the result without having to commit and
push all:

ikiwiki --plugin meta --plugin tag --plugin comments --plugin inline  . ../anarc.at.html

More plugins are of course enabled in the blog, see the setup file for
more information, or just enable plugin as you want to unbreak
things. Use the --rebuild flag on subsequent runs. The actual
invocation I use is more something like:

ikiwiki --rebuild --no-usedirs --plugin inline --plugin calendar --plugin postsparkline --plugin meta --plugin tag --plugin comments --plugin sidebar  . ../anarc.at.html

I had problems with dates, but it turns out that I wasn't setting
dates in redirects... Instead of doing that, I started adding a
"redirection" tag that gets ignored by the main page.

Files and old URLs The script should keep the same URLs, as long as pathauto is enabled
on the site. Otherwise, some logic should be easy to add to point to
node/N. To redirect to the new blog, rewrite rules, on original blog, should
be as simple as:

Redirect / http://anarc.at/blog/

When we're sure:

Redirect permanent / http://anarc.at/blog/

Now, on the new blog, some magic needs to happen for files. Both
/files and /sites/anarcat.koumbit.org/files need to resolve
properly. We can't use symlinks because
ikiwiki drops symlinks on generation. So I'll just drop the files in /blog/files directly, the actual
migration is:

cp $DRUPAL/sites/anarcat.koumbit.org/files $IKIWIKI/blog/files
rm -r .htaccess css/ js/ tmp/ languages/
rm foo/bar # wtf was that.
rmdir *
sed -i 's#/sites/anarcat.koumbit.org/files/#/blog/files/#g' blog/*.mdwn
sed -i 's#http://anarcat.koumbit.org/blog/files/#/blog/files/#g' blog/*.mdwn
chmod -R -x blog/files
sudo chmod -R +X blog/files

A few pages to test images:

• http://anarcat.koumbit.org/node/157
• http://anarcat.koumbit.org/node/203

There are some pretty big files in there, 10-30MB MP3s - but those are
already in this wiki! so do not import them! Running fdupes on the result helps find oddities. The meta guid directive is used to keep the aggregators from finding
duplicate feed entries. I tested it with Liferea, but it may freak out
some other sites.

Remaining issues

• postsparkline and calendar archive disrespect meta(date)
• merge the files in /communication with the ones in /blog/files
  before import
• import non-published nodes
• check nodes with a format different than markdown (only a few 3=Full
HTML found so far)
• replace links to this wiki in blog posts with internal links

More progress information in [[the script drupal2ikiwiki.py]] itself.

November 2014:
Sunday 2nd:
Travel from Moscow to Paris Monday 3rd to Sunday 8th:
Summit in Paris Monday 10th:
Uploaded python-rudolf to Sid (needed by Fuel)
Uploaded python-invoke and python-invocations (needed to run fabric s unit tests)
Uploaded python-requests-kerberos/0.5-2 fixing CVE-2014-8650: failure to handle mutual authentication. Asked the release team for unblock.
Uploaded openstack-pkg-tools version 19 fixing startup with systemd in Jessie (added RuntimeDirectory directive). Asked the release team for unblock.
Opened ticket to remove TripleO, Tuskar and Ironic packages from Jessie. I don t consider them ready for a Debian stable release, and there s no long term support from upstream.
Fixed Designate Juno dbsync process which prevented it from being installed.
Fixed Ironic Juno unowned files after purge (policy 6.8, 10.8): /var/lib/ironic/ cache, ironicdb (eg: purging these folders on purge) Thuesday 11:
Fixed nova-api CVE-2014-3708: Nova network DoS through API filtering in both the Juno and Icehouse release. Asked the release team to unblock the Icehouse version for Jessie. See: https://bugs.debian.org/769163
Uploaded Cinder with Duch debconf translation fix and pt.po
Uploaded python-django-pyscss with upstream patch for Django 1.7 support instead of the Debian one that I wrote 2 months ago. Asked the release team to unblock which they did. Wednesday 12:
Uploaded fix for horizon (see #769101) unowned files after purge (policy 6.8, 10.8). Now purging /usr/share/openstack-dashboard/openstack_dashboard on purge.
Uploaded Ironic with Duch translations of debconf
Uploaded Designate with Duch translations of Debconf screens
Uploaded openstack-trove with Duch translations of Debconf screens
Uploaded Tuskar with Duch translations of Debconf screens
Updated python-oslotest in Experimental to version 1.2.0 Thursday 13:
Uploaded new packages: python-oslo.middleware and python-oslo.concurrency.
Opened a new packaging branch for Nova Kilo, and updated (build-)depends.
Uploaded fix for Icehouse Cinder: delete volume failed due to unicode problems , and asked for unblock.
Uploaded new package: python-pygit2 and python-xmlbuilder, needed for fuel-agent-ci.
Uploaded sheepdog with Duch debconf translation.
Uploaded python-daemonize to Sid (in FTP master NEW queue).
Re-uploaded python-invoke after FTP master rejection (missing copyright information) Friday 14:
Uploaded liberasurecode & python-pyeclib to Sid, now in the FTP masters NEW queue waiting for approval. This will soon be needed by Swift. Monday 17:
Worked on the Cobbler packaging (all day long ) Tuesday 18:
Worked on backporting all of Fuel packages to Wheezy. Done with fuelclient already.
Uploaded ruby-cstruct and ruby-rethtool to Sid (needed by nailgun-agent) Wednesday 19:
Uploaded pyeclib again, with fixes for the build-depends. Package is still in the NEW queue anyway.
Built a Debian-based bootstrap hardware discovery image for Fuel, and it seems that it works already (to be checked )! \o/
To be added as packages in the ISO:
* nailgun-mcagents
* nailgun-net-check
* fuel-agent
* python-tasklib Thursday 20:
Uploaded python-tasklib to Sid (now in NEW queue )
Continued working on the discovery bootstrap ISO Friday 21:
Documented Sahara procedure in Debian in the official install-guide: https://review.openstack.org/136237
Fixed oslo.messaging so it doesn t use PROTOCOL_SSLv3 because its support has been removed from Debian (due to possible protocol downgrade attacks): https://review.openstack.org/136278 and uploaded fixed packages for Sid and Experimental.
Uploaded fixed Neutron packages for CVE-2014-7821 in both Sid and Experimental (eg: Icehouse and Juno) Monday 24:
Uploaded new package: python-os-client-config (in NEW queue)
Installed new Xen server to be used as my new Jenkins build machine
Moved the juno-wheezy VM to it
Finished to package python-pymysql and uploaded to Sid. It s now running all unit tests successfully! \o/ Tuesday 25:
Uploaded fix for openstack-debian-images to add the -o compat=1.0 option when building an image with Qemu > 1.0. Opened bug to the release team to have it unblocked.
Continued working on unit tests for fuel-nailgun. Wednesday 26:
Uploaded python-os-net-config to Sid (new package)
Worked briefly on python-cassandra-driver. It needs cassandra to be in, which is a LOT of work.
Found a (not useable) hack to run nailgun unit tests. It works, however, it doesn t seem like fuel-nailgun is designed to be able to use unix socket for the postgres connection in its unit tests.
Uploaded python-pykmip to Sid (new package)
Updated the Debian wheezy backport repository for libvirt to version 1.2.9 from official wheezy-backports. Removed policykit-1 and libusb from there too, as it broke stuff to use a backported version (X and usb were not useable on my Wheezy laptop when using it ). Thursday 27 & Friday 28:
Uploaded new Javascript packages or dependencies for Fuel: libjs-autonumeric, libjs-backbone-deep-model, libjs-backbone.stickit, libjs-cocktail, libjs-i18next, libjs-require-css, libjs-requirejs, libjs-requirejs-text Sunday 30:
Uploaded debian/copyright fixes for libjs-backbone-deep-model, libjs-backbone.stickit and libjs-cocktail after the packages were accepted by the FTP masters and they gave remarks about copyright. DECEMBER 2014 Monday 01:
Uploaded new Debian image to MOX, after I unerstood the issue was about the architecture field that I was wrongly filling. I ll be able to use that for Tempest checking on my dev account. Tuesday 02:
Uploaded python-q-text-as-data to Sid (new awesome package!)
Uploaded Horizon with some triggers mechanisms to start the compress when one of its JS depends is updated. That s very important for security!
Uploaded a fixed version of heat-cfntools to Sid (it was missing the /usr/lib/python* folder). Asked the release team for an unblock so it can reach Jessie.
Fixed unit tests in fuel-nailgun, thanks to a patch from Sebastian Kalinowski. Now all unit tests are passing but one (for which I opened a launchpad bug: tests are trying to write in /var/log/nailgun, which is impossible at package build time). Wednesday 03:
Uploaded fixed version of ruby-rethtool after FTP master s rejection and upstream correction of licensing files.
Uploaded fixed version of libjs-require-css after FTP master s rejection
Fixed (in Git only) python-sysv-ipc missing build-depends on dh-python as per bug opened by James Page (this is not so important, but I did it still).
Continued working on the tempest-ci scripts.
Added to the image-guide docs about openstack-debian-images: https://review.openstack.org/#/c/138743/ Thursday 04:
Uploaded new package: python-proliantutils. Send patch to upstream about an issue in indentation (mix-up with space and tabs) which made the package uninstallable with Python 3.4. Friday 05:
Worked on the package CI. Monday 07:
Worked on the package CI. All works now, up to all of the Tempest tests for Keystone. Now need to fix the neutron config. Thuesday 08:
Continued working on the CI. Wednesday 09:
Uploaded fix for FTBFS of python-tasklib (Closes: #772606)
Uploaded fix for libjerasure-deb missing dependency on libgf-complete-dev, package already unblocked and will migrate to Jessie.
Uploaded fix for Designate Juno fail to upgrade from Icehouse: this was due to the database_connection directive renamed to connection =.
Uploaded fix for Designate purge in Sid (Icehouse release of Designate).
Commited to git updates of the German debconf translation in both Icehouse and Juno.
Updated nova to use libvirtd as init script dependency instead of libvirt-bin (this was renamed in the libvirt-daemon-system package).
Do not touch the db connection directive if user didn t ask for db handling by the package. Thursday 10 to Saturday 13:
Finally understood the issues with systemd service files not being activated by default. Fixed openstack-pkg-tools, and uploaded version 20 to Sid, after the release team accepted the changes. Sunday 14:
Uploaded Juno 2014.2.1 to Experimental: ceilometer, cinder, glance, python-glance-store, heat, horizon, keystone Monday 15:
Finished uploading Juno 2014.2.1 to Experimental: Nova, Neutron, Sahara Tuesday 16:
Added crontab to flush tokens in Icehouse Keystone
Some more CI work Wednesday 17:
Uploaded keystone with systemd fix and crontab to flush the token table in Sid (eg: Icehouse).
Uploaded nova Icehouse with a bunch of fixes in Sid. Thursday 18:
Updated some issues in Nova Icehouse (Sid/Jessie) Friday 19:
Started building a new Jenkins instance for building Kilo packages Monday 22:
Finished building the new Jenkins instance for building Kilo packages, and rebuilt every packages there, using Jessie as a base. Tuesday 23:
Updated version for the following packages: oslo.utils, oslo.middleware, stevedore, oslo.concurency, pecan, oslo.concurrency, python-oslo.vmware, python-glance-store
Built so far: Ceilometer, Keystone, python-glanceclient, cinder, glance Wednesday 24:
Continued packaging Kilo beta 1. Updated: nova, designate, neutron
Uploaded python-tempest-lib to Debian Unstable (new package) Wednesday 31:
Continued packaging Kilo beta 1. Updated: heat JANUARY 2015 Thursday 01:
Continued packaging Kilo beta 1. Updated: ironic, openstack-trove, openstack-doc-tools, ceilometer Friday 02:
Finished packaging Kilo beta 1. Updated: Sahara, Murano, Murano-dashboard, Murano-agent Sunday 04:
Started testing Kilo beta 1. Fixed a few issues on default configuration for Ceilometer and Glance. Monday 05:
Fixed openstack-pkg-tools which failed to create PID files at boot time, Uploaded to Sid, asked the release team for unblock.
Uploaded ceilometer & cinder to Sid, rebuilt against openstack-pkg-tools 21.
Did more testing of Kilo beta 1, fixed a few more minor issues. Tuesday 06:
Uploaded glance, neutron, nova, designate, keystone, heat, trove to Sid, so that all sysv-rc init scripts are fixed with the new openstack-pkg-tools 21. Designate, heat, keystone and trove contains other minor fixes reported to the Debian BTS. Wednesday 07:
Asked the Debian release team (open bugs with debdiff as attachment) for unblocks of glance, neutron, nova, designate, keystone, heat, trove so they migrate to Jessie.
Fixed a few minor issues tracked in the Debian BTS on various packages. Thesday 08:
James Page from Canonical informed me that they are now using openstack-pkg-tools for maintaining their daemons in Nova, Cinder and Keystone in Ubuntu. That s an awesome news : more QA for both platforms.
James Page found out that dh_installinit *must* be called *after* the call of dh_systemd_enable, otherwise, daemons aren t started automatically at the first install of packages, as the unmask of systemd happens after the invoke-rc.d. Friday 09:
Did some QA checks on the latest upload. Fixed Heat which broke because using the wrong template name (glance instead of heat). Monday 12:
Started re-running the automated openstack-deploy scrip in Icehouse, Juno and Kilo. Found out the issue in Keystone wasn t fixed in Juno (but was fixed in other releases), and fixed it.
Removed the use of ssl.PROTOCOL_SSLv3 from heat (removed form Debian). Uploaded the fixed package to Sid.
All of openstack-deploy (debian/kilo branch) now works and succesfully installs OpenStack again. If dh_installinit is called before, we have:

# Automatically added by dh_installinit
if [ -x "/etc/init.d/keystone" ]; then
update-rc.d keystone defaults >/dev/null
fi
if [ -x "/etc/init.d/keystone" ]   [ -e "/etc/init/keystone.conf" ]; then
invoke-rc.d keystone start   true
fi
# End automatically added section
# Automatically added by dh_systemd_enable
# This will only remove masks created by d-s-h on package removal.
deb-systemd-helper unmask keystone.service >/dev/null   true
# was-enabled defaults to true, so new installations run enable.
if deb-systemd-helper --quiet was-enabled keystone.service; then
# Enables the unit on first installation, creates new
# symlinks on upgrades if the unit file has changed.
deb-systemd-helper enable keystone.service >/dev/null   true
else
# Update the statefile to add new symlinks (if any), which need to be
# cleaned up on purge. Also remove old symlinks.
deb-systemd-helper update-state keystone.service >/dev/null   true
fi
# End automatically added section

If it s called after dh_systemd_enable, we have:

# Automatically added by dh_systemd_enable
# This will only remove masks created by d-s-h on package removal.
deb-systemd-helper unmask keystone.service >/dev/null   true
# was-enabled defaults to true, so new installations run enable.
if deb-systemd-helper --quiet was-enabled keystone.service; then
# Enables the unit on first installation, creates new
# symlinks on upgrades if the unit file has changed.
deb-systemd-helper enable keystone.service >/dev/null   true
else
# Update the statefile to add new symlinks (if any), which need to be
# cleaned up on purge. Also remove old symlinks.
deb-systemd-helper update-state keystone.service >/dev/null   true
fi
# End automatically added section
# Automatically added by dh_installinit
if [ -x "/etc/init.d/keystone" ]; then
update-rc.d keystone defaults >/dev/null
fi
if [ -x "/etc/init.d/keystone" ]   [ -e "/etc/init/keystone.conf" ]; then
invoke-rc.d keystone start   true
fi
# End automatically added section

As a consequence, I have to re-upload version 22 of openstack-pkg-tools and also re-upload all OpenStack core packages to Debian Sid. Fixed a number of issues like:
* dbc_upgrade = true check which shouldn t have been there in postinst.
* <project>/configure_db default value is now always false
* db_sync and pkgos_dbc_postinst are now only done if <project>/configure_db is set to true.
Rebuilt all packages in Juno and Kilo with the above changes. Tuesday 13:
Opened unblock bugs for the release team to unblock all fixed packages.
Made more tests in Juno and Kilo to make sure the fixed bugs in Icehouse are fixed there too.
Fixed numerous issues in Trove (missing trove-conductor.conf, wrong trove-api init file, etc.). More work will be needed for it for both Icehouse and newer releases. Wednesday 14:
Did a doc meeting about debconf. Some doc contributors still want to kill the debconf / debian manual, and I have to not agree.
Made a new patch to better document the keystone install procedure:
Did some bug triaging in the doc about Debian.
Uploaded new versions of core packages to Experimental (eg: Juno) built against openstack-pkg-tools >= 22~, and some fixes forward ported from Icehouse: Keystone, Ceilometer, Cinder, Glance, Heat, Ironic, Murano, Neutron, Nova, Saraha and Murano-agent. All where rebuilt in Juno (Wheezy + Trusty) and Kilo (Jessie only) on my Jenkins. Thuesday 15:
Succesfully booted a live-build Debian live image containing mcollective and nailgun-agent as a Debian replacement for the hardware discovery / boostrap image of Fuel. Now, I need to find a way to use just a kernel + initramfs Friday 16 to Tuesday 20:
Worked on the packaging CI. Wednesday 21:
Fixed https://bugs.debian.org/775636 (Horizon failed to build due to a Moscow timezone change and wrong test). Uploaded to Sid, asked for unblock.
Fixed https://bugs.debian.org/775926: CVE-2015-1195: Glance still allows users to download and delete any file in glance-api server (applied upstream patch). Uploaded to Sid, asked for unblock. Uploaded Juno version to Experimental.
Uploaded openstack-trove with the remaining fixes, asked release team for unblock.
Uploaded python-glanceclient 0.15.0 (Juno) to Experimental because it fixes an issue with HTTPS. Added to it a patch from James Page not yet merged, which fixes unit test with Python 2.7.9 (7 failures otherwise).
Uploaded python-xstatic-d3 as it can t be installed anymore in Sid after a new version of d3 was uploaded. Thursday 22:
Uploaded python-xstatic-smart-table and libjs-angularjs-smart-table to Sid (new packages, now in NEW queue). Friday 23:
Ask for the removal of the below list of packages from Jessie:
python-xstatic
python-xstatic-angular
python-xstatic-angular-cookies
python-xstatic-angular-mock
python-xstatic-bootstrap-datepicker
python-xstatic-bootstrap-scss
python-xstatic-d3
python-xstatic-font-awesome
python-xstatic-hogan
python-xstatic-jasmine
python-xstatic-jquery
python-xstatic-jquery-migrate
python-xstatic-jquery-ui
python-xstatic-jquery.bootstrap.wizard
python-xstatic-jquery.quicksearch
python-xstatic-jquery.tablesorter
python-xstatic-jsencrypt
python-xstatic-qunit
python-xstatic-rickshaw
python-xstatic-spin
libjs-jsencrypt
libjs-spin.js
libjs-twitter-bootstrap-datepicker
libjs-twitter-bootstrap-wizard They are used only in OpenStack Horizon starting on 2014.2 (aka Juno), and Jessie is shipped with Icehouse, so it s IMO best to not carry the burden of maintaining these packages for the life of Jessie. Monday 26:
Enhanced and review requested changes for https://review.openstack.org/147296 (ie: Keystone install with more details about what the package does).
Finished testing network on the CI install. Now need to automate all. Tuesday 27:
Closed all bugs on the rabbitmq-server package (2 correction, one bug triage).
Uploaded a fix for the missing conntrack dependency in neutron-l3-agent.
Restarted working on CI setup of Juno after success with manual install in a Xen domU.
Uploaded fix to make sheepdog build reproducible (patch from the Debian BTS). Thursday 28:
Fixed and uploaded to Sid openstack-debian-images 2 bugs reported by Steve McIntire. Official Debian images for OpenStack are now available at:
http://cdimage.debian.org/cdimage/openstack/ \o/
Note that this is the weekly build of testing. We wont get Debian Stable images before Jessie is out.
Documented the new image thing here: http://docs.openstack.org/image-guide/content/ch_obtaining_images.html#debian-images as a new patch: https://review.openstack.org/#/c/151015/
Fixed my patch for keystone debconf doc at: https://review.openstack.org/#/c/147296/ Wednesday 29:
Continued working on packaging CI Thursday 30:
Fixed CVE on Neutron (Juno): L3 agent denial of service with radvd 2.0+
Fixed CVE on Glance (Icehouse + Juno): Glance user storage quota bypass. Asked release team for unblock.
Fixed the image-guide patch after review (ie: https://review.openstack.org/151015)

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 12 hours on Debian LTS. I did the following tasks:

• CVE triage. I pushed 24 commits to the securitry tracker. I spent more time on this task than usually (see details below).
• I released DLA-143-1 on python-django (fixing 3 CVE). While I expected the update to be quick, my testing revealed that even though the patches applied mostly fine, they did not work as expected. I ended up spending almost 4 hours to properly backport the fixes and the corresponding tests (to ensure that the fixes are working properly).

I want to expand on two cases that I stumbled upon in my CVE triage work and that took quite long to investigate each. While my after-the-fact description is rather straightforward, the real process involved more iterations and data gathering that I do not mention here. First I was investigating CVE-2012-6685 on libnokogiri-ruby and the upstream bug discussion revealed that libxml2 could also be part of the problem. Using the tests cases submitted there, I confirmed that libxml2 was also affected by an issue of its own then I started to analyze the history of CVE of libxml2 to find out whether that issue got a CVE assigned: yes, that was CVE-2014-0191 (although the CVE description is unrelated). But this CVE was marked as fixed in all releases. Why? It turns out that the upstream fix for this CVE is just the complement of another commit that was merged way earlier (and that was used as a basis for the commit as the copy/paste of the comment shows). When the security teams integrated the upstream patch in wheezy/squeeze, they were probably not aware that a full fix required to also include something else. In the end, I thus reopened CVE-2014-0191 on our tracker (commit here). The second problematic case was pound. Thijs Kinkhorst added pound related data on the multiple (high profile) SSL related issues. So it appeared on my radar of new vulnerable package in Squeeze because it was marked that CVE-2009-3555 was fixed in version 2.6-2 while Squeeze has 2.5-1. There was no bug reference in the security tracker and the Debian changelog for that version only mentioned an anti_beast patch which is yet another issue (CVE-2011-3389). I had to dig a bit deeper in the end I discovered that the above patch also has provisions for the CVE that was of interest to me, except that Brian May recently reported in #765649 that the package was still vulnerable to this issue I tried to understand where the above patch was failing and thus submitted my findings to the bug. And I updated the tracker data with my newly gained knowledge (commit 31751 and 31752). Tryton For me, January is always the month where I try to close the accounting books of Freexian. This year is no exception except that it s the first year where I do this with Tryton. I first upgraded to Tryton 3.4 to have the latest version. Despite this I discovered multiple problems while doing so since I don t want to have those problems next year, I reported them and prepared fixes for those related to the French chart of accounts:

• #4464: CSV export on tree views is unusable
• #4466: add missing deferral properties on accounts
• #4468: drop abusive reconcile properties on some accounts
• #4469: convert account 6354 into a real non-view account
• #4479: balance non-deferral accounts is broken with non-view parent accounts

Saltstack I mentioned this idea last month setting up and maintaining a lot of sbuild chroots can be tiresome so I wanted to automate this as much as possible. To achieve this I created three Salt formulas and got them added to the official Saltstack repository:

• debootstrap-formula
• schroot-formula
• sbuild-formula

Each one builds on top of the former. debootstrap-formula creates chroots with debootstrap or cdebootstrap. schroot-formula does the same and registers those chroots in schroot. And sbuild-formula does the same as schroot-formula but with different defaults that are more suited to sbuild chroots (and obviously ensures that sbuild is installed and that generated chroots are buildd chroots). With the sbuild formula I can put this in pillar data:

sbuild:
  chroots:
    wheezy:
      architectures: [amd64, i386]
      extra_dists:
        - wheezy-backports
        - wheezy-security
      extra_aliases:
        - wheezy-backports
        - stable-security
        - wheezy-security
    jessie:
    [...]

And then a simple salt-call state.highstate (I m running in standalone mode) will ensure that I have all the chroots properly setup. Misc packaging I packaged new upstream releases of Django in experimental and opened a pre-approval request to get the latest 1.7.x in jessie (#775892). It seems to be a difficult sell for the release team, which is a pity because we have active Debian developers, active upstream developers, and everybody is well aware of the no-new features rule to avoid regressions. Where is the risk? I also filed an unblock request for Dolibarr (on the request of the security team which wants to see the CVE fix reach Jessie). I did small contributions to two bugs that were of special interest to some of my donators (#751339 and #774811), they were not under my responsibility but I tried to get them moving by pinging the relevant people. I prepared a security upload for Django in Wheezy (python-django_1.4.5-1+deb7u9) and sent it to the security team. While doing this I discovered a small problem in their backported patch that I reported upstream in Django s ticket #24239. Debian France With the new year, it s again time to organize a general assembly with the election of a third of its board. So we solicited candidacies among the members and I m pleased to see that we got 6 candidacies for the 3 seats. It s a good sign that we still have enough persons caring about the association. One of them is even speaking of Debconf 17 in France great plans! On my side, I announced that I would not candidate to be president for the next year. I will stay on the board though to ensure we have a smooth transition. Thanks See you next month for a new summary of my activities.

One comment Liked this article? Click here. My blog is Flattr-enabled.

A new release 0.11.4 of Rcpp is now on the CRAN network for GNU R, and an updated Debian package will be uploaded in due course. Rcpp has become the most popular way of enhancing GNU R with C++ code. As of today, 323 packages on CRAN depend on Rcpp for making analyses go faster and further; BioConductor adds another 41 packages, and casual searches on GitHub suggests dozens mores. This release once again adds a large number of small bug fixes, polishes and enhancements. And like the last time, these changes were made by a group of seven different contributors (counting code commits) plus three more providing concrete suggestions. This shows that the Rcpp development and maintenance rests a large number of (broad) shoulders. See below for a detailed list of changes extracted from the NEWS file.

Changes in Rcpp version 0.11.4 (2015-01-20)

• Changes in Rcpp API:
  • The ListOf<T> class gains the .attr and .names methods common to other Rcpp vectors.
  • The [dpq]nbinom_mu() scalar functions are now available via the R:: namespace when R 3.1.2 or newer is used.
  • Add an additional test for AIX before attempting to include execinfo.h.
  • Rcpp::stop now supports improved printf-like syntax using the small tinyformat header-only library (following a similar implementation in Rcpp11)
  • Pairlist objects are now protected via an additional Shield<> as suggested by Martin Morgan on the rcpp-devel list.
  • Sorting is now prohibited at compile time for objects of type List, RawVector and ExpressionVector.
  • Vectors now have a Vector::const_iterator that is 'const correct' thanks to fix by Romain following a bug report in rcpp-devel by Martyn Plummer.
  • The mean() sugar function now uses a more robust two-pass method, and new unit tests for mean() were added at the same time.
  • The mean() and var() functions now support all core vector types.
  • The setequal() sugar function has been corrected via suggestion by Qiang Kou following a bug report by S ren H jsgaard.
  • The macros major, minor, and makedev no longer leak in from the (Linux) system header sys/sysmacros.h.
  • The push_front() string function was corrected.
• Changes in Rcpp Attributes:
  • Only look for plugins in the package's namespace (rather than entire search path).
  • Also scan header files for definitions of functions to be considerd by Attributes.
  • Correct the regular expression for source files which are scanned.
• Changes in Rcpp unit tests
  • Added a new binary test which will load a pre-built package to ensure that the Application Binary Interface (ABI) did not change; this test will (mostly or) only run at Travis where we have reasonable control over the platform running the test and can provide a binary.
  • New unit tests for sugar functions mean, setequal and var were added as noted above.
• Changes in Rcpp Examples:
  • For the (old) examples ConvolveBenchmarks and OpenMP, the respective Makefile was renamed to GNUmakefile to please R CMD check as well as the CRAN Maintainers.

Thanks to CRANberries, you can also look at a diff to the previous release As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

After releasing Weblate 2.0 with Bootstrap based UI, there was still lot of things to improve. Weblate 2.1 brought more consistency in using buttons with colors and icons. Weblate 2.2 will bring some improvements in other graphics elements. One of thing which was for quite long in our issue tracker is to provide own renderer for SVG status badge. So far Weblate has offered either PNG badge or external SVG rendered by shields.io. Relying on external service was not good in a long term and also caused requests to third party server on many pages, what could be considered bad privacy wise. Since this week, Weblate can render SVG badge on it's own and they are also matching current style used by other services (eg. Travis CI): One last thing which really did not fit into new UI were activity charts. In past they were rendered as PNG on server side, but for upcoming releases we have switched to use Chartist javascript library and render them as SVG on client side. This way we can nicely style them to fit into page, they scale properly and also reduce server load. You can see them in action on Hosted Weblate server:

Filed under: English phpMyAdmin SUSE Weblate 0 comments Flattr this!

Last week, I visited London for the January Docker meetup, which was the first time I'd attended this group. It was a talk-oriented format, with around 200 attendees packed into Shoreditch Village Hall; free pizza and beer was provided thanks to the sponsors, which was awesome (and makes logistics easier when you're travelling there from work). There were three talks. First, Andrew Martin from British Gas spoke about how they use Docker for testing and continuous deployment of their Node.js microservices - buzzword bingo! But it's helpful to see how companies approach these things. Second, Johan Euphrosine from Google gave a short demo of Google Cloud Platform for running Docker containers (mostly around Container Engine, but also briefly App Engine). This was relevant to my interests, but I'd already seen this sort of talk online. Third, Dan Williams presented his holiday photos featuring a journey on a container ship, which wins points from me for liberal interpretation of the meetup topic, and was genuinely very entertaining/interesting - I just regret having to leave to catch a train halfway through. In summary, this was worth attending, but as someone just getting started with containers I'd love some sort of smaller meetings with opportunities for interaction/activity. There's such a variety of people/use cases for Docker that I'm not sure how much everyone had in common with each other; it would be interesting to find out.

apt install nodejs-legacy if you want npm install to work.