As I've
blogged
before,
I've been on and off working on
SReview, a video review system.
Development over the past year has been mostly driven by the need to
have something up and running for first FOSDEM 2017, and then DebConf17,
and so I've cut corners left and right which made the system, while
functional, not quite entirely perfect everywhere. For instance, the
backend scripts were done in ad-hoc perl, each reinventing their own
wheel. Doing so made it easier for me to experiment with things and
figure out where I want them to go, without immediately creating a lot
of baggage that is not necessarily something I want to be stuck to. This
flexibility has already paid off, in that I've redone the state machine
between FOSDEM and DebConf17 and all it needed was to update a few
SQL statements here and there. Well, and add a few of them, too.
It was always the intent to replace most of the ad-hoc perl with something
better, however, once the time was ripe. One place where historical baggage is
not so much of a problem, and where in fact abstracting away the complexity
would now be an asset, is in the area of FFmpeg command lines. Currently,
these are built by simple string expansion. For example, we do something like
this (shortened for brevity):
system("ffmpeg -y -i $outputdir/$slug.ts -pass 1 -passlogfile ...");
inside an environment where the
$outputdir and
$slug variables are
set in a perl environment. That works, but it has some downsides; e.g.,
adding or removing options based on which codecs we're using is not so
easy. It would be much more flexible if the command lines were generated
dynamically based on requested output bandwidth and codecs, rather than
that they be hardcoded in the file. Case in point: currently there are
multiple versions of some of the backend scripts, that only differ in
details mostly the chosen codec on the ffmpeg command line.
Obviously this is suboptimal.
Instead, we want a way where video file formats can be autodetected, so
that I can just say "create a file that uses encoder etc settings of
this other file here". In addition, we
also want a way where we can
say "create a file that uses encoder etc settings of this other file
here,
except for these one or two options that I want to fine-tune
manually". When I first thought about doing that about a year ago, that
seemed complicated and not worth it or at least not to that
extent.
Enter
Moose.
The Moose OO system for Perl 5 is an interesting way to do object
orientation in Perl. I knew Perl supports OO, and I had heard about
Moose, but never had looked into it, mostly because the standard perl OO
features were "good enough". Until now.
Moose has a concept of adding 'attributes' to objects. Attributes can be
set at object construction time, or can be accessed later on by way of
getter/setter functions, or even simply functions named after the
attribute itself (the default). For more complicated attributes, where
the value may not be known until some time after the object has been
created, Moose borrows the concept of "lazy" variables from Perl 6:
package Object;
use Moose;
has 'time' => (
is => 'rw',
builder => 'read_time',
lazy => 1,
);
sub read_time
return localtime();
The above object has an attribute 'time', which will not have a value
initially. However, upon first read, the 'localtime()' function will be
called, the result is cached, and then (and on all further calls of the
same function), the cached result will be returned. In addition, since
the attribute is read/write, the time can also be written to. In that
case, any cached value that may exist will be overwritten, and if no
cached value exists yet, the read_time function will never be called.
(it is also possible to clear values if needs be, so that the function
would be called again).
We use this with the following pattern:
package SReview::Video;
use Moose;
has 'url' => (
is => 'rw',
)
has 'video_codec' => (
is => 'rw',
builder => '_probe_videocodec',
lazy => 1,
);
has 'videodata' => (
is => 'bare',
reader => '_get_videodata',
builder => '_probe_videodata',
lazy => 1,
);
has 'probedata' => (
is => 'bare',
reader => '_get_probedata',
builder => '_probe',
lazy => 1,
);
sub _probe_videocodec
my $self = shift;
return $self->_get_videodata-> codec_name ;
sub _probe_videodata
my $self = shift;
if(!exists($self->_get_probedata-> streams ))
return ;
foreach my $stream(@ $self->_get_probedata-> streams )
if($stream-> codec_type eq "video")
return $stream;
return ;
sub _probe
my $self = shift;
open JSON, "ffprobe -print_format json -show_format -show_streams " . $self->url . " "
my $json = "";
while(<JSON>)
$json .= $_;
close JSON;
return decode_json($json);
The
videodata and
probedata attributes are internal-use only
attributes, and are therefore of the 'bare' type that is, they
cannot be read nor written to. However, we do add 'reader' functions
that can be used from inside the object, so that the object
itself can
access them. These reader functions are generated, so they're not part
of the object source. The
probedata attribute's builder simply calls
ffprobe with the right command-line arguments to retrieve data in JSON
format, and then decodes that JSON file.
Since the passed JSON file contains an array with (at least) two
streams one for video and one for audio and since the
ordering of those streams depends on the file and is therefore not
guaranteed, we have to loop over them. Since doing so in each and every
attribute of the file we might be interested in would be tedious, we add
a
videodata attribute that just returns the data for the first found
video stream (the actual source also contains a similar one for audio
streams).
So, if you create an
SReview::Video object and you pass it a filename
in the
url attribute, and then immediately run
print
$object->video_codec, then the object will
- call
ffprobe, and cache the (decoded) output for further use
- from that, extract the video stream data, and cache that for further use
- from that, extract the name of the used codec, cache it, and then
return that name to the caller.
However, if the caller
first calls
$object->video_codec('h264'),
then the
ffprobe and most of the caching will be skipped, and instead
the
h265 data will be returned as video codec name.
Okay, so with a reasonably small amount of code, we now have a bunch of
attributes that have defaults based on actual files but can be
overwritten when necessary. Useful, right? Well, you might also want to
care about the fact that sometimes you want to generate a video file
that uses the same codec settings of this
other file here. That's
easy. First, we add another attribute:
has 'reference' => (
is => 'ro',
isa => 'SReview::Video',
predicate => 'has_reference'
);
which we then use in the
_probe method like so:
sub _probe
my $self = shift;
if($self->has_reference)
return $self->reference->_get_probedata;
# original code remains here
With that, we can create an object like so:
my $video = SReview::Video->new(url => 'file.ts');
my $generated = SReview::Video->new(url => 'file2.ts', reference => $video);
now if we ask the
$generated object what the value of its
video_codec setting is without telling it ourselves first, it will use
the
$video object for its probed data, and use that.
That only misses generating the ffmpeg command line, but that's all
fairly straightforward and therefore left as an exercise to the reader.
Or you can cheat, and
look it
up.
On Saturday 29 August 2020, the annual Debian Developers
and Contributors Conference came to a close.
DebConf20 has been held online for the first time, due to the coronavirus
(COVID-19) disease pandemic.
All of the sessions have been streamed, with a variety of ways of participating:
via IRC messaging, online collaborative text documents,
and video conferencing meeting rooms.
With more than 850 attendees from 80 different countries and a
total of over 100 event talks, discussion sessions,
Birds of a Feather (BoF) gatherings and other activities,
DebConf20 was a large success.
When it became clear that DebConf20 was going to be an online-only
event, the DebConf video team spent much time over the next months to
adapt, improve, and in some cases write from scratch, technology that
would be required to make an online DebConf possible. After lessons
learned from the MiniDebConfOnline in late May, some adjustments were
made, and then eventually we came up with a setup involving Jitsi, OBS,
Voctomix, SReview, nginx, Etherpad, and a newly written web-based
frontend for voctomix as the various elements of the stack.
All components of the video infrastructure are free software, and the
whole setup is configured through their public
ansible repository.
The DebConf20 schedule included
two tracks in other languages than English: the Spanish language MiniConf,
with eight talks in two days,
and the Malayalam language MiniConf, with nine talks in three days.
Ad-hoc activities, introduced by attendees over the course of the entire conference,
have been possible too, streamed and recorded. There have also been several
team gatherings to sprint on certain Debian development areas.
Between talks, the video stream has been showing the usual sponsors on the loop, but also
some additional clips including photos from previous DebConfs, fun facts about Debian
and short shout-out videos sent by attendees to communicate with their Debian friends.
For those who were not able to participate, most of the talks and sessions are already
available through the
Debian meetings archive website,
and the remaining ones will appear in the following days.
The DebConf20 website
will remain active for archival purposes and will continue to offer
links to the presentations and videos of talks and events.
Next year, DebConf21 is planned to be held
in Haifa, Israel, in August or September.
DebConf is committed to a safe and welcome environment for all participants.
During the conference, several teams (Front Desk, Welcome team and Community team)
have been available to help so participants get their best experience
in the conference, and find solutions to any issue that may arise.
See the web page about the Code of Conduct in DebConf20 website
for more details on this.
Debian thanks the commitment of numerous sponsors
to support DebConf20, particularly our Platinum Sponsors:
Lenovo,
Infomaniak,
Google
and
Amazon Web Services (AWS).
About Debian
The Debian Project was founded in 1993 by Ian Murdock to be a truly
free community project. Since then the project has grown to be one of
the largest and most influential open source projects. Thousands of
volunteers from all over the world work together to create and
maintain Debian software. Available in 70 languages, and
supporting a huge range of computer types, Debian calls itself the
universal operating system.
About DebConf
DebConf is the Debian Project's developer conference. In addition to a
full schedule of technical, social and policy talks, DebConf provides an
opportunity for developers, contributors and other interested people to
meet in person and work together more closely. It has taken place
annually since 2000 in locations as varied as Scotland, Argentina, and
Bosnia and Herzegovina. More information about DebConf is available from
https://debconf.org/.
About Lenovo
As a global technology leader manufacturing a wide portfolio of connected products,
including smartphones, tablets, PCs and workstations as well as AR/VR devices,
smart home/office and data center solutions, Lenovo
understands how critical open systems and platforms are to a connected world.
About Infomaniak
Infomaniak is Switzerland's largest web-hosting company,
also offering backup and storage services, solutions for event organizers,
live-streaming and video on demand services.
It wholly owns its datacenters and all elements critical
to the functioning of the services and products provided by the company
(both software and hardware).
About Google
Google is one of the largest technology companies in the
world, providing a wide range of Internet-related services and products such
as online advertising technologies, search, cloud computing, software, and hardware.
Google has been supporting Debian by sponsoring DebConf for more than
ten years, and is also a Debian partner sponsoring parts
of Salsa's continuous integration infrastructure
within Google Cloud Platform.
About Amazon Web Services (AWS)
Amazon Web Services (AWS) is one of the world's
most comprehensive and broadly adopted cloud platforms,
offering over 175 fully featured services from data centers globally
(in 77 Availability Zones within 24 geographic regions).
AWS customers include the fastest-growing startups, largest enterprises
and leading government agencies.
Contact Information
For further information, please visit the DebConf20 web page at
https://debconf20.debconf.org/
or send mail to press@debian.org.
Many people labour under the assumption that pop culture is trivial and useless while only 'high' art can grant us genuine and eternal knowledge about the world. Given that we have a finite time on this planet, we are all permitted to enjoy pop culture up to a certain point, but we should always minimise our interaction with it, and consume more moral and intellectual instruction wherever possible.
Or so the theory goes. What these people do not realise is that pop and 
Float/String Conversion in Picolibc: Enter Ry
I 
Something I ve found myself doing as the pandemic rolls on is picking
out and (re-)reading through sections of the 
Halloween is here, time to upload a new set of scary packages of TeX Live. About a month has passed, so there is the usual big stream up updates. There was actually an intermediate release to get out some urgent fixes, but I never reported the news here. So here are the accumulated changes and updates.
My favorite this time is 
"Qubes OS from the POV of a Debian developer" and "Qubes OS user meetup at Bornhack"
I wrote the following while on my way home from 
At DebConf17 I gave a talk titled "Qubes OS from the POV of a Debian developer" and while the