So, DebConf8 is over (currently we have three machines left in the network, the video storage server, the video encoding server and my laptop...) and IMO it rocked! I really enjoyed being here, meeting many many known faces and getting to know quite some new ones and learning a bit here and there, even though I was mostly doing videoteam work :-)

And this should be the main message of this blog post: many many thanks to all the different members of the videoteam. You have been awesome! The camera operators, the sound and video mixers, the moderators (which strictly speaking are not part of the videoteam but nonetheless helped us very much), those of you who reviewed the videos, the debconf network admins, the localteam members who prepared the venue fantastically, those who prepaid for hardware (and Debian for paying in the end) and took it here, those who helped with the set up here, the hotel staff who generally were very helpful (for example they agreed to dig holes in their walls), the absent team members who helped via irc and very much Dami n Viano for his awesome work on pentabarf and Ben Hutchings for dvswitch and most of the debconf-video package.

Also I like to thank Amaya very much for bearing with me and for all the love and joy you give me! Te quiero!

Thank you all! Very very much.

(And now I'm curious who I forgot...)

I'm also really happy we got videoteam t-shirts this year and at the same time this is probably the biggest regret I have regarding the videoteam: We didn't have enough t-shirts for everyone who deserved one. But things can definitly be worse :-)

Recordings, as much as we have them, are being uploaded at the usual place. You can help us with post processing by adding comments about problems (non-obvious ones only, please. If a event is split into two files, thats an obvious problem), to our todo wikipage.

See you! Have fun!

Since 2005, the talks at Debconf have been recorded on video, to be
published online. Beginning in 2006, talks have also been streamed
live. One of the goals is to get recordings in high quality online
immediately, with little or no post-production work. The video
switcher dvswitch, realized in software, and a disk-based recording
workflow is bringing us closer to that goal. Tape is still used as
backup in case of failures, but we expect those to become rarer as
the system matures. A large part of the post production work is
restoring failed recordings.

Dvswitch was developed by Ben Hutchings just in time for the Debconf7
in Edinburgh. It enabled a multi-camera live production using cheap
cameras and modest laptops. The laptops relays DV streams over
TCP/IP. This avoids any degradation due to long cable stretches, and
makes cabling cheap and convenient. The output is DV straight from
the cameras, producing a very crisp master video.

Dvswitch works with sources and sinks. The sources send video from a
file or a firewire port to a listening instance of dvswitch, which can
run on a remote machine.
dvswitch listens to several sources and displays a scaled down black
and white preview image of each one. One of the sources is displayed
at full resolution in colour; this is the outgoing stream, which is
sent to the listening sinks. Each sink may send the stream to disk or
to an encoder for live streaming. It is common to do both; store the
raw master to disk, and forward the master to an encoder, and
subsequently to a streaming server.

While dvswitch saves us a bundle in hardware, the video team is still
heavy on manpower. Remotely operated cameras would help, and we are
still looking for affordable solutions that work well on Linux.


The talk will cover the hardware, software and manpower required for
a typical dvswitch usecase. Some caveats and challenges regarding
sound, lighting and rigging will be discussed. A live demonstration
of dvswitch in use will be given. We would also like to offer
rehearsal sessions with the demo rig for those who are interested.Full event details

Since 2005, the talks at Debconf have been recorded on video, to be
published online. Beginning in 2006, talks have also been streamed
live. One of the goals is to get recordings in high quality online
immediately, with little or no post-production work. The video
switcher dvswitch, realized in software, and a disk-based recording
workflow is bringing us closer to that goal. Tape is still used as
backup in case of failures, but we expect those to become rarer as
the system matures. A large part of the post production work is
restoring failed recordings.

Dvswitch was developed by Ben Hutchings just in time for the Debconf7
in Edinburgh. It enabled a multi-camera live production using cheap
cameras and modest laptops. The laptops relays DV streams over
TCP/IP. This avoids any degradation due to long cable stretches, and
makes cabling cheap and convenient. The output is DV straight from
the cameras, producing a very crisp master video.

Dvswitch works with sources and sinks. The sources send video from a
file or a firewire port to a listening instance of dvswitch, which can
run on a remote machine.
dvswitch listens to several sources and displays a scaled down black
and white preview image of each one. One of the sources is displayed
at full resolution in colour; this is the outgoing stream, which is
sent to the listening sinks. Each sink may send the stream to disk or
to an encoder for live streaming. It is common to do both; store the
raw master to disk, and forward the master to an encoder, and
subsequently to a streaming server.

While dvswitch saves us a bundle in hardware, the video team is still
heavy on manpower. Remotely operated cameras would help, and we are
still looking for affordable solutions that work well on Linux.


The talk will cover the hardware, software and manpower required for
a typical dvswitch usecase. Some caveats and challenges regarding
sound, lighting and rigging will be discussed. A live demonstration
of dvswitch in use will be given. We would also like to offer
rehearsal sessions with the demo rig for those who are interested.Full event details

In addition to live streams, recordings of each event should be published on meetings-archive.debian.net. There is an RSS feed of these, which should work nicely in Miro: http://meetings-archive.debian.net/pub/debian-meetings/2008/debconf8/index.rss. That feed is also included on Planet Debian. This is the first time I've tried generating RSS so apologies in advance if I make a mess of the Planet.

Sorry, I missed a few last time.

Dependency-based boot sequence	low	high
Data-mining Popcon	low	high
Debian-installer - an update	low	high
Wacky ideas II	low	high

A few of the recordings from DebConf 7 had to be recovered in whole or in part from tape. For personal and technical reasons, a few of these were not done last year, and I have finally dealt with them now. The last few sessions are:

The secure Debian Desktop	low	high
Debian - The Universal Operating System?	low	high
Wacky ideas BoF	low	high
lintian BoF	low	high
Debian travels around the world	low	high
Leading a Free Software project	low	high
Method diffusion in large volunteer projects	low	high
Debtags is ready	low	high
OpenStreetMap	low	high
Maintaining packages with Git	low	high
netconf	low	high
Time for a better init system	low	high

Hopefully these are useful to someone. Sorry it took so long!

Miscellaneous bash tips A few useful configuration bits:

# Don't put duplicate lines in the history
export HISTCONTROL=ignoredups
# When you use history expansion (the !something), allows to edit the
# expanded line before executing it
shopt -s histverify
# Correct spelling mistakes when using 'cd'
shopt -s cdspell
# Do not attempt completion on an empty line
shopt -s no_empty_cmd_completion

A few useful keystrokes:

• Alt + dot: insert the last argument of the previous command. Equivalent to inserting the expansion of !$ at the cursor position.
• Alt + >: go to the end of the history. Sometimes it happens, when using Ctrl + r, that I mistakenly land before the command that I was looking for, and I need to hold arrow-down for a while to go back at the end. This does the trick. Of course, Alt + < goes at the beginning. Also, Ctrl
  • s does forward search.
• If you need to execute a bunch of consecutive lines from the history, try Ctrl + o: it runs the current line and then takes you to the next one in the history.
• Unsure of expansion, or you would like to act on *.jpg exept maybe a few of them? Ctrl + Alt + e does full expansion on the current line, and Alt + ^ does history expansion only. If the expansion is too big, use Ctrl+x then Ctrl+e, which loads the current command in an editor and runs when you exit the editor.
• If you want to postpone the current command, Alt + # will comment it out and and enter it into the history, so that you can look it up later. (via Ben Hutchings)

For more useful keystrokes, run man bash, look for "Commands for Manipulating the History" and read down from there.

After a fair amount of clean-up, a lot of waiting for review, and some ruthless pruning, I and my colleagues at Solarflare finally got our net driver accepted for Linux 2.6.26. Meanwhile a larger version of the driver made it into Xen's Linux tree, along with dependent drivers to provide a secure fast path to the hardware from domU. Somewhere down the road we're going to have to resolve these two versions, but now I'm just pleased to have a working driver in-tree. There are some other changes I've worked on over the past few months:

• I maintain diagnostic scripts for our NICs and drivers. Among other things they include Vital Product Data (VPD). There's a standard way to read VPD in PCI (actually there are two, in different versions of the spec) but in Linux this is only accessible to root, and even then there are race conditions. We provide an MTD driver for flash and/or EEPROM attached to the NIC, and I know where the VPD appears in there, but most users won't be loading that driver. So I wrote this: Expose PCI VPD through sysfs
• Along the way I found a bug in sysfs, which I fixed: Disallow truncation of files in sysfs
• One day I accidentally bridged one of our network interfaces, which has Large Receive Offload (LRO) enabled by default, with one of the motherboard's network interfaces, and the results were not pretty. LRO isn't really compatible with packet forwarding, and currently causes the GSO code to crash or warn. I wanted to solve this by disabling forwarding of LRO skbs, but it turns out to be trickier than I thought.
• We wanted to use topology information to allocate one IRQ per CPU package, not per core (or per thread, with HT enabled) for Receive-Side Scaling. Unfortunately this information is only available in some kernel configurations and not for all architectures. I had a go at providing reasonable defaults, but it didn't work too well!

However, this is still only part of my work. My main task at the moment is to write more unit tests for our next NIC.

I'm employed at Solarflare Communications in the test group, supporting the work of testers and developers by developing test frameworks. (We actually have very few pure testers and a whole lot of automation.). In a major break from my previous jobs, I've been working mostly in Python on applications for use in-house. More recently I've become involved in maintenance of our network driver for Linux, starting with updating it to use current kernel coding conventions, and submitting for inclusion in the kernel. This has become a bit frustrating because while there are a lot of people prepared to run checkpatch and pick nits, there are not so many who will do a thorough review of a driver that's 20,000-odd lines long. And without someone doing that, it won't go in. Meanwhile some of my colleagues have done some great work to accelerate Linux networking on Xen by having drivers in dom0 and domU cooperate to expose hardware resources directly to domU. Our hardware architecture supports many virtual NICs with their own packet queues and set of buffers, so this does not compromise the security of the physical machine and dom0. The results are very impressive. The original motivation for the virtual NIC architecture was as a basis for a user-level networking stack, which can provide very low latency and reduced CPU usage. This software was originally proprietary but has now been released under GPLv2 as OpenOnload. Google invited two of my senior colleagues to talk about this - here's a video of their talk about user-level networking.

Most of the Debian Miniconf 7 sessions at LCA 2008 in Melbourne were recorded by the LCA video team. The recordings are now mirrored on meetings-archive.debian.net.

To guard against late-night administrative mistakes: pam_breathalyser.

L: Bill Gates; R: Matthew Garrett

It's noon now and half the attendees or so the day hasn't really begun. Sure, some people here at the 24C3 in Berlin are having lunch already, but most are having breakfast or not even that yet. As the Debian booth is set up (mugs are sold out, bags almost...) and there are no interesting talks for me at the moment, I'm having a second breakfast in the smokers lounge is the cellar. Officially, this is the third non smoking congress and inofficially it's the third time there is an official smoker place ;-) This time, it's a lounge with comfortable sofas, ambient light and loud music...

About four weeks ago I was contacted by Martin Michlmayr and asked if the video team could make use of some hardware donation a company wanted to sponsor. I said yes, and so last week two boxes arrived at my house: a small audio mixer, a Behringer XENYX 1202 with 12-channels and XLR and chinch inputs, which is supposed to help us improve the audio quality in small venues. The second box contained a TwinPact100, which is a video converter box. It has an SVGA-input (supporting up to 1600x1200), an SVGA-output (sending the same signal out) and DV (firewire) output (and more features we don't really care about). So for us it's basically a SVGA-passthrough box which also outputs a video stream (of a speakers laptop in our use case) over firewire. So no more capturing of slides with a normal camera, yay! Yesterday Ben Hutchings and me tested this box and found it working, then Ben also fixed an issue with dvswitch, which we use to mix different firewire video streams together, probably even as soon as FOSDEM again! Many thanks to hitflip.de for this nice donation!

When I arrived here this morning I hoped to see the BabelBox I set up last night to be still running, and it was, yay! BabelBox is a d-i setup which loops through an installation of Debian etch in 36 languages, mostly using the graphical installer. And this will be running here for approx 72 hours non-stop resulting in 289 installations without a single keystroke :) IMO it shows real nicely two of the three major strengths of d-i: internationalisation and automatisation.

I guess it's pretty unlikely that I'll find some time during the last two days of the congress to take Andre Salomons packages for the OLPC laptop and demonstrate d-i's third major feature: it's modular design. It's more likely to be an excercise for next year though (I'll arrive home on the 31st..), here is too much stuff happening. Installing babelbox was just following a good howto. In any case, there is a OLPC.de meeting planned tomorrow at 12.30 noon at the Debian booth or in the Embedded Corner. Just come along and look out for the green cluster of XOs :)

And by now it's 8 PM - time flies when you're having fun. I just noticed that my wireless connection is up since seven hours uninterrupted! Wow.

As threatened promised, I have produced a renamed version of Ion3 that should avoid upstream trademark claims and thus be DFSG-compliant. Some translations are out-of-date, but I'll release a first version now and try to get those updated later. The new name is ParticleMan. When I put the name to a vote some time back the winner was Iceparticle, but there's already an icewm and I couldn't resist the TMBG reference. (Whoever suggested this name to me: I had already thought of it but forgot to include it in the poll.) The packages are in my repository for now.

Since ries (ftp-master.debian.org) is still down, I've uploaded packages for the latest version of Ion3 to my own server. That's:

deb http://womble.decadent.org.uk/debian/ sid/
deb-src http://womble.decadent.org.uk/debian/ sid/

The repository is signed with my personal key.

The DebConf 7 videos needed some post-processing, mostly to recover missing pieces from tape. I've done a large part of that, but it's not yet complete. The video team to-do list has details of the recordings that still need work, and the sessions that we believe were not recorded. Please tell us if we did record some of these, and we will have another look through our tapes for them. The conversion to "publication" formats (i.e. small enough to offer for download) was started at the conference and I have continued this at home over the past 2 weeks. Unfortunately this has been interrupted by some hardware failures. It looks like the CPUs in two of my PCs were running hot enough to be damaged (slowly) but not hot enough to trigger an alarm or automatic shutdown. The remaining conversions will be running mostly on my laptop, which is pretty fast and should be able to finish conversion of the recordings that are in a good state in about 2 days. However, some of the converted files may be broken and need some manual attention. My upload bandwidth is only 448 kbit/s so the high-quality files will take about a week to upload. As ever, the video files are being published on meetings-archive.debian.net.

Not my title, but that of Peter Gutmann's presentation about PKI, specifically X.509 as used in SSL. It's informative, entertaining, and I suspect not as well known as it should be.

Ion3 supports docked applications and is compatible with the docking protocols used by WindowMaker and KDE. Normally you would use:

dopath("mod_dock")

to load mod_dock, which provides a standard corner dock, toggled using MOD1+D. However, recent versions also support docking applications in the status bar, in "system tray" style:

-- In cfg_ion.lua:
defwinprop  
   class = "foo-window-class",
   statusbar = "foo"
 
-- In cfg_statusbar.lua:
mod_statusbar.create  
   ...
   template="... %systray_foo"
 

GNOME unfortunately uses a different dock protocol. However, the docker program can adapt from this to the WindowMaker protocol. So you can include a GNOME system tray in your status bar by including docker in your X session and this in your Ion3 configuration:

-- In cfg_ion.lua:
defwinprop  
   class = "Docker",
   statusbar = "dock"
 
-- In cfg_statusbar.lua:
mod_statusbar.create  
   ...
   template="... %systray_dock"
 

It's day 2 of DebConf and dvswitch is being used by many members of the video team. So I have a pretty good idea of what it does well and what I need to fix in it. The most serious problem is that there's no way to start and stop recording from the GUI. That's actually controlled by starting and stopping the file sink, which runs on a remote machine. Here I have arranged to start it by running a command with ssh in the background, which is stopped by quitting dvswitch or with pkill! The result is that we can record a lot of junk between events, which has to be checked and then discarded, or alternately stop recording and risk missing the next event. Also, we like to number our cameras: 1 for the speaker, 2 for the audience, 3 for slides. But dvswitch assigns numbers to sources in order of connection. I fudged around this by again using ssh to start the sources in the proper order after dvswitch. Despite these and a number of other flaws, it seems that several people have been impressed by the ability to cover talks from multiple angles. I hope this will attract more people to contribute to dvswitch so that it can serve DebConf and other conferences better.

The DebConf video team is using a large number of programs, some common and some of our own, spread across 20-30 computers. Some may be interested to know what this is.

• FAI for package installation and configuration on DebConf machines.
• dvgrab for video grabbing. We are using a variety of DV cameras, but dvgrab works well with almost all of them.
• dvswitch for switching between multiple cameras and files.
• ffmpeg2theora for transcoding to Ogg Theora/Vorbis, both for live streams and for the downloadable files slowly appearing on meetings-archive.
• A small number of scripts for starting the above three programs and recording to files.
• Icecast 2 for streaming. We are currently using the sarge package rebuilt for etch because the server in the etch package is unstable in our double-relayed configuration. (To limit bandwidth use between DebConf and the Internet, we have an internal server for internal use and an external relay that serves as master for all other relays.)
• GeoDNS and a monitoring script to resolve streams.video.debconf.org to the nearest working relay server.
• A newly-developed extension to the Pentabarf conference management system that presents recordings for review. Reviewers match up recordings to events, give the start and end time of the event in the recording so we can cut out extra material, and rate the quality of the recordings and transcoded files.
• A script to import the details of new recordings into the database ready for review, run as a cron job.
• A script to transcode recordings into multiple versions using the timing and naming information provided by reviewers. This spreads the work across multiple servers controlled using ssh (again!).

Most of the custom software and configurations should be included in the debconf-video project on Alioth.