Midsummer for me is also known as
Noite do Lume Novo (literally New Fire Night ), one of the big calendar
events of the year, marking the end of the school year and the beginning of
summer.
On this day, there are
celebrations
not very unlike the bonfires in the
Guy Fawkes
Night in England or Britain
[1]. It is a bit different in that
it is not a single event for the masses, more of a friends and neighbours thing,
and that it lasts for a big chunk of the night (sometimes until morning).
Perhaps for some people, or outside bigger towns or cities, Guy Fawkes Night is
also celebrated in that way and that's why during the first days of November
there are fireworks rocketing and cracking in the neighbourhoods all around.
Like many other
celebrations around the world involving
bonfires, many of
them also happening around the summer solstice, it is supposed to be a time of
renewal of cycles, purification and keeping the evil spirits away; with rituals
to that effect like jumping over the fire when the flames are not high and it
is safe enough.
So it was fitting that, in the middle of June (almost Midsummer in the northern
hemisphere), I learnt that I was about to leave my now-previous job, which is a
pretty big signal and precursor for renewal (and it might have something to
do with purifying and keeping the evil away as well
;-) ).
Whatever... But what does all of this have to do with
aptitude or Debian, anyway?
For one, it was a question of timing.
While looking for a new job (and I am still at it), I had more spare time than
usual.
DebConf 15 @ Heidelberg was within
sight, and for the first time circumstances allowed me to attend this event.
It also coincided with the time when I re-gained access to commit to
aptitude on the 19th of June.
Which means Renewal.
End of June was also the time of the announcement of the colossal
GCC-5/C++11
ABI transition in
Debian, that was scheduled to start on the 1st of August, just before the
DebConf. Between 2 and 3 thousand source packages in Debian were affected by
this transition, which a few months later is not yet finished (although the most
important parts were completed by mid-end September).
aptitude itself is written in C++, and depends on several libraries written in
C++, like
Boost,
Xapian and
SigC++. All of them had to be compiled with
the new C++11 ABI of GCC-5, in unison and in a particular order, for
aptitude
to continue to work (and for minimal breakage).
aptitude and some
dependencies did not even compile straight away, so this transition meant that
aptitude needed attention just to keep working.
Having recently being awarded again with the Aptitude Hat, attending DebConf for
the first time and sailing towards the Transition Maelstrom, it was a clear sign
that Something Had to Be Done (to avoid the sideways looks and consequent shame
at DebConf, if nothing else).
Happily (or a bit
unhappily for me, but let's pretend...), with the unexpected
free time in my hands, I changed the plans that I had before re-gaining the
Aptitude Hat (some of them involving Debian, but in other ways maybe I will
post about that soon).
In July I worked to fix the problems before the transition started, so
aptitude would be (mostly) ready, or in the worst case broken only for a few
days, while the chain of dependencies was rebuilt. But apart from the changes
needed for the new
GCC-5, it was decided at the last minute that
Boost 1.55
would not be rebuilt with the new ABI, and that the only version with the new
ABI would be 1.58 (which caused further breakage in
aptitude, was added to
experimental only a few days before, and was moved to
unstable after the
transition had started). Later, in the first days of the transition,
aptitude
was affected for a few days by breakage in the dependencies, due to not being
compiled in sequence according to the transition levels (so with a mix of old
and new ABI).
With the critical intervention of
Axel
Beckert (abe / XTaran), things were not so
bad as they could have been. He was busy testing and uploading in the critical
days when I was enjoying a small holiday on my way to DebConf, with minimal
internet access and communicating almost exclusively with him; and he promptly
tended the complaints arriving in the Bug Tracking System and asked for rebuilds
of the dependencies with the new ABI. He also brought the packaging up to
shape, which had decayed a bit in the last few years.
Gruesome Challenges
But not all was solved yet, more storms were brewing and started to appear in
the horizon, in the form of clouds of fire coming from
nearby
realms.
The APT Deities, which had long ago spilled out their
secret, inner
challenge
(just the initial paragraphs), were relentless. Moreover, they were present at
Heidelberg in full force, in or close to their home grounds, and they were
Marching Decidedly towards Victory:
In the talk @ DebConf
This APT has Super Cow
Powers
(video available), by David Kalnischkies, they told us about the niceties of
apt 1.1 (still in
experimental but hopefully coming to
unstable soon), and
they boasted about getting the lead in our
arms race (should I say
bugs
race?) by a few open bug reports.
This act of provocation further escalated the tensions. The fierce competition
which had been going on for some time gained new heights. So much so that APT
Deities and our team had to sit together in the outdoor areas of the venue and
have many a
weissbier together, while discussing and fixing bugs.
But beneath the calm on the surface, and while pretending to keep good
diplomatic relations, I knew that Something Had to Be Done, again. So I could
only do one thing jump over the bonfire and Keep the Evil away, be that
Keep Evil bugs Away or Keep Evil APT Deities Away from winning the challenge, or
both.
After returning from DebConf I continued to dedicate time to the project, more
than a full time job in some weeks, and this is what happened in the last few
months, summarised in another graph, showing the evolution of the BTS for
aptitude:
The numbers for
apt right now (15th November 2015) are:
- 629 open (731 if counting all merged bugs independently)
- 0 Release Critical
- 275 (318 unmerged) with severity Important or Normal
- 354 (413 unmerged) with severity Minor or Wishlist
- 0 marked as Forwarded or Pending
The numbers for
aptitude right now are:
- 488 (573 if counting all merged bugs independently)
- 1 Release Critical (but it is an artificial bug to keep it from migrating to
testing)
- 197 (239 unmerged) with severity Important or Normal
- 271 (313 unmerged) with severity Minor or Wishlist
- 19 (20 unmerged) marked as Forwarded or Pending
The Aftermath
As we can see, for the time being I could keep the Evil at bay, both in terms of
bugs themselves and re-gaining the lead in the
bugs race the Evil APT
Deities were thwarted again in their efforts.
... More seriously, as most of you suspected, the graph above is not the whole
truth, so I don't want to boast
too much. A big part of the reduction in the
number of bugs is because of merging duplicates, closing obsolete bugs, applying
translations coming from multiple contributors, or simple fixes like typos and
useful suggestions needing minor changes. Many of remaining problems are
comparatively more difficult or time consuming that the ones addressed so far
(except perhaps avoiding the immediate breakage of the transition, that took
weeks to solve), and there are many important problems still there, chief among
those is
aptitude offering
very poor solutions to resolve conflicts.
Still, even the simplest of the changes takes effort, and triaging hundreds of
bugs is not fun at all and mostly a thankless effort althought there is the
occasionally kind soul that thanks you for handling a decade-old bug.
If being subjected to the rigours of the BTS and reading and solving hundreds of
bug reports is not Purification, I don't know what it is.
Apart from the triaging, there were 118 bugs closed (or pending) due to
changes made in the upstream part or the packaging in the last few months, and
there are many changes that are not reflected in bugs closed (like most of the
changes needed due to the C++11 ABI transition, bugs and problems fixed that had
no report, and general rejuvenation or improvement of some parts of the code).
How long this will last, I cannot know. I hope to find a job at some point,
which obviously will reduce the time available to work on this.
But in the meantime, for all
aptitude users: Enjoy the fixes and new features!
Notes
[1]
^ Some visitors of the
recent
mini-DebConf @
Cambridge
perhaps thought that the fireworks and throngs gathered were in honour of our
mighty Universal Operating System, but sadly they were not. They might be, some
day. In any case, the reports say that the visitors enjoyed the fireworks.
Get involved
After this happened, my next step was to get re-involved in Debian Live to help it carry on after the loss of Daniel. Here s a quick update on some team progress, notes that could help people building Stretch images right now, and what to expect next.
Team progress
Last Friday, I 
Pros
Pros
Suggested by 
Pros
Pros
Pros
Long over due post
It s been a long time I haven t written here. And lots of things happened in the OpenStack planet. As a full time employee with the mission to package OpenStack in Debian, it feels like it is kind of my duty to tell everyone about what s going on.
Liberty is out, uploaded to Debian
Since my last post, OpenStack Liberty, the 12th release of OpenStack, was released. In late August, Debian was the first platform which included Liberty, as I proudly outran both RDO and Canonical. So I was the first to make the announcement that Liberty passed most of the Tempest tests with the beta 3 release of Liberty (the Beta 3 is always kind of the first pre-release, as this is when feature freeze happens). Though I never made the announcement that Liberty final was uploaded to Debian, it was done just a single day after the official release.
Before the release, all of Liberty was living in Debian Experimental. Following the upload of the final packages in Experimental, I uploaded all of it to Sid. This represented 102 packages, so it took me about 3 days to do it all.
Tokyo summit
I had the pleasure to be in Tokyo for the Mitaka summit. I was very pleased with the cross-project sessions during the first day. Lots of these sessions were very interesting for me. In fact, I wish I could have attended them all, but of course, I can t split myself in 3 to follow all of the 3 tracks.
Then there was the 2 sessions about Debian packaging on upstream OpenStack infra. The goal is to setup the OpenStack upstream infrastructure to allow packaging using Gerrit, and gating each git commit using the usual tools: building the package and checking there s no FTBFS, running checks like lintian, piuparts and such. I knew already the overview of what was needed to make it happen. What I didn t know was the implementation details, which I hoped we could figure out during the 1:30 slot. Unfortunately, this didn t happen as I expected, and we discussed more general things than I wished. I was told that just reading the docs from the infra team was enough, but in reality, it was not. What currently needs to happen is building a Debian based image, using disk-image-builder, which would include the usual tools to build packages: git-buildpackage, sbuild, and so on. I m still stuck at this stage, which would be trivial if I knew a bit more about how upstream infra works, since I already know how to setup all of that on a local machine.
I ve been told by Monty Tailor that he would help. Though he s always a very busy man, and to date, he still didn t find enough time to give me a hand. Nobody replied to my request for help in the openstack-dev list either. Hopefully, with a bit of insistence, someone will help.
Keystone migration to Testing (aka: Debian Stretch) blocked by python-repoze.who
Absolutely all of OpenStack Liberty, as of today, has migrated to Stretch. All? No. Keystone is blocked by a chain of dependency. Keystone depends on python-pysaml2, itself blocked by python-repoze.who. The later, I upgraded it to version 2.2. Though python-repoze.what depends on version <= 1.9, which is blocking the migration. Since python-repoze.who-plugins, python-repoze.what and python-repoze.what-plugins aren t used by any package anymore, I asked for them to be removed from Debian (see #805407). Until this request is processed by the FTP masters, Keystone, which is the most important piece of OpenStack (it does the authentication) will be blocked for migration to Stretch.
New OpenStack server packages available
On my presentation at Debconf 15, I quickly introduced new services which were released upstream. I have since packaged them all:
I have a 
Pros
Pros
Pros
Pros
Pros
Pros
There is a companion logrotate script, to avoid filling the file system:
FOSDEM 2016 is going to be great (again!) and you still have the chance to be one of the stars.
Have you submitted your talk to the Desktops DevRoom yet?
No?
Remember: we will only accept proposals until December 6th. After that, the Organization Team will get busy and vote and choose the talks.
Here is the full Call for Participation, in case you need to check the details on how to submit:
I've previously written about Canonical's 
Now you should see all the video sequence editor controls:
Setup the properties for your project
Click the context menu under the strip editor panel and change the panel to a Properties panel:
The video file we are playing with is 720p, so it seems reasonable to use 720p for the output too. Change that here:
The input file is 25fps so we need to use exactly the same frame rate for the output, otherwise you will either observe the video going at the wrong speed or there will be a conversion that is CPU intensive and degrades the quality. Also check that the resolution_percentage setting under the picture dimensions is 100%:
Now specify output to PNG files. Later we will combine them into a WebM file with a script. Specify the directory where the files will be placed and use the # placeholder to specify the number of digits to use to embed the frame number in the filename:
Now your basic rendering properties are set. When you want to generate the output file, come back to this panel and use the Animation button at the top.
Editing the video
Use the context menu to change the properties panel back to the strip view panel:
Add the video file:
and then right click the video strip (the lower strip) to highlight it and then add a transform strip:
Audio waveform
Right click the audio strip to highlight it and then go to the properties on the right hand side and click to show the waveform:
Rendering length
By default, Blender assumes you want to render 250 frames of output. Looking in the properties to the right of the audio or video strip you can see the actual number of frames. Put that value in the box at the bottom of the window where it says 250:
Enable AV-sync
Also at the bottom of the window is a control to enable AV-sync. If your audio and video are not in sync when you preview, you need to set this AV-sync option and also make sure you set the frame rate correctly in the properties:
Add the other sound strip
Now add the other sound file that was recorded using the lapel microphone:
Enable the waveform display for that sound strip too, this will allow you to align the sound strips precisely:
You will need to listen to the strips to make an estimate of the time difference. Use this estimate to set the "start frame" in the properties for your audio strip, it will be a negative value if the audio strip starts before the video. You can then zoom the strip panel to show about 3 to 5 seconds of sound and try to align the peaks. An easy way to do this is to look for applause at the end of the audio strips, the applause generates a large peak that is easily visible.
Once you have synced the audio, you can play the track and you should not be able to hear any echo. You can then silence the audio track from the camera by right clicking it, look in the properties to the right and change volume to 0.
Make any transforms you require
For example, to zoom in on the speaker, right click the transform strip (3rd from the bottom) and then in the panel on the right, click to enable "Uniform Scale" and then set the scale factor as required:
Render the video output to PNG
Click the context menu under the Curves panel and choose Properties again.
Click the Animation button to generate a sequence of PNG files for each frame.
Render the audio output
On the Properties panel, click the Audio button near the top. Choose a filename for the generated audio file.
Look on the bottom left-hand side of the window for the audio file settings, change it to the ogg container and Vorbis codec:
Ensure the filename has a .ogg extension
Now look at the top right-hand corner of the window for the Mixdown button. Click it and wait for Blender to generate the audio file.
Combine the PNG files and audio file into a WebM video file
You will need to have a few command line tools installed for manipulating the files from scripts. Install them using the package manager, for example, on a Debian or Ubuntu system:
Writing programs is fun, but making them fast can be a pain. Python programs
are no exception to that, but the basic profiling toolchain is actually not
that complicated to use. Here, I would like to show you how you can quickly
profile and analyze your Python code to find what part of the code you should
optimize.
What's profiling?
Profiling a Python program is doing a dynamic analysis that measures the
execution time of the program and everything that compose it. That means
measuring the time spent in each of its functions. This will give you data
about where your program is spending time, and what area might be worth
optimizing.
It's a very interesting exercise. Many people focus on local optimizations,
such as determining e.g. which of the Python functions 
Over the rainy weekend we watched two movies: 
My personal verdict: 3 rotten eggs (because 
What happened in the 
The variation in accuracy displayed is a result of experimentation with different dataset sizes and algorithm variations. More information and details can be found in the 