Search Results: "tar"

24 January 2011

Julien Danjou: Emacs snapshot Ubuntu packages

Emacs snapshot packages for Ubuntu are now available, thanks to Dimitri Fontaine.

23 January 2011

Wouter Verhelst: PMW vs Lilypond

A few days ago, I discovered From Punched Cards To Flat Screens, the "technical memoirs" of Philip Hazel (of exim and PCRE fame). This document, by itself a very good read, mentioned something about him having written a piece of software for the Acorn RISC OS platform, starting in the early 90s, to typeset music based on a text file; very much the way in which lilypond operates. The document also mentioned that after the demise of Acorn's platform, Philip had ported that application, PMW, to the unix platform. Being interested in music, this had me curious, so I did a bit of investigation. Lilypond is an interesting piece of software, but as I've written before, I do have a few gripes with it. Once you get beyond the basics, the lilypond syntaxis is somewhat counter-intuitive in my opinion, and there are a number of things that are just wrong with it. Upon reading through the first few examples, I quickly learned that most of the major issues that I have with lilypond's syntaxis simply do not exist with PMW, and that it also has quite a number of features which lilypond in its (in the mean time) fairly long history, still does not support. And while lilypond does have a slightly higher amount of output options, that simply does not weigh against the much higher processing speed and much more sensible input syntaxis of PMW. I've made a short comparison of the two, which shows some of their differences, and contains some notes on my opinion of the two. This table may grow in the future... I've only been playing with PMW for a few days now, but I've already decided that I'm going to ditch Lilypond.

Guido G nther: git-buildpackage tip #3: disabling desktop notifications

Since waiting for long running builds is boring and having to check if the build already finished distracts one from doing other stuff, git-buildpackage 0.5.16 reports the build status of a package via desktop notifications if python-notify is installed: notification with gnome-shell

To disable this feature use:

[git-buildpackage]
notify = off

in your ~/.gbp.conf. This blog is flattr enabled.

Julien Danjou: OrgCamp Paris 2011 review

Yesterday afternoon, I was at the first OrgCamp in Paris. It was my first attendance to a BarCamp, and I really liked it. It's basically the first geek event I do not find boring nor useless. There was about 18-20 persons participating, which was quite high, since we all initially though we would have been only 5. We had several presentations of various features and personal usages of Org-mode. For my part, I've quickly presented the agenda, and my BBDB replacement named org-contacts (I'll probably talk about it on this blog in another post later). The only downside was that Bastien (the new Org-mode maintainer) was not able to come and join us. On the other side, there were so much to tell for a first time, I did not have so much time to code. I only have been able to fix one bug reported during my agenda presentation. In the end, the overall atmosphere was very enthusiastic and friendly, which was extremely pleasant. The #org-mode-fr IRC channel has been created on Freenode, following this event. Feel free to join us. Since people liked it so badly, it seems there should be another barcamp in the next months. Stay tuned.

Joachim Breitner: A Mistake in Church s Paper

For a course at the IIT Bombay on functional programing, I was preparing a presentation on Alonzo Church s theorem, based on his original 1936 paper An Unsolvable Problem of Elementary Number Theory where he first showed that the Entscheidungsproblem is not solvable. In that paper, he states a theorem (Theorem II) If a formula has a normal form [ ] any sequence of reductions of the formula must (if continued) terminate in the normal form . This seems to be wrong. Consider the lambda expression KI , where K=( xy.x), I=( x.x) and =( x.xx)( x.xx). This has a normal form I. But because reduces to , there is a non-terminating sequence KI KI , in contradiction to Church s claim. The same example contradicts his Theorem III: If a formula has a normal form, every well-formed part of it has a normal form. , which is used in the very proof of his main result, Theorem XVIII. There is no proof for Theorem II in this paper. He cites it from the then forthcoming paper Some properties of Conversion by him and J. B. Rosser. There, proofs are given (see Theorem 2 and its Corollary) but they are quite impenetrable to me. So I was searching for some correction paper, or any other discussion of this issue, but could not find any. Does any reader of this blog know more? What happened in that times when a paper had an error in some proof? What would happen now? Or is this not an error after all, but just a subtle difference between the definitions of -calculus as we know and as they introduced it? Unrelated to this question: The professor asked me to write down a summary of (my interpretation of) the importance and impact of Church s paper with regard to G del and Hilbert s program, and how that relates to B hm s theorem. In the spirit of sharing, I have uploaded my thoughts on Church and B hm, comments are welcome. Update: Christian von Essen solved the riddle in a comment to this post: Church only considers lambda abstractions as well-formed when the bound variable actually occurs (freely) in the abstracted term. This does not allow for the K combinator and thus there is no problem.

Russell Coker: Autism, Food, etc

James Purser wrote Stop Using Autism to Push Your Own Brand of Lunacy about the organisations that use Autism for their own political agenda [1]. He references the ongoing vaccine issue (which didn t even stop when Andrew Wakefield was proven to have engaged in fraudulent research) and a campaign of PETA.PETA s ClaimsPETA claims that many autistic kids improve dramatically when put on a diet free of dairy foods [2]. They also speculate about the possibility of Autism being caused by dairy.One thing that confuses the issue is the impact of stress on people with Autism Spectrum Disorders (ASDs). Whatever you are trying to do you will probably do it less effectively if under an excessive amount of stress, and the background stress levels of people on the Autism Spectrum tend to be quite high at the best of times. So add in some extra stress from a food allergy and the ability to emulate an NT can decrease a lot.There are people who make credible claims about significant behavior improvements in Autistic children after changing their diet. Eliminate foods that trigger a minor allergic response and the behavior of NT kids will improve too.I m sure that there are some Autistic kids who have a minor intolerance to lactose or casein who s behavior would improve if they had no dairy products. There are probably just as many Autistic kids who have a minor allergy to peanuts who would get the same benefit from avoiding peanut butter. It s not inconceivable that some Autistic people may be allergic to soy or wheat and find that they can cope better if their consume more milk, cheese, and steak and less vegetables!Also some people on the Autism Spectrum have a better sense of smell than NTs, and it s fairly common to be more fussy about the appearance of food. Force children to eat food that smells bad and their behavior might deteriorate.The issues of diet and Autism are more difficult than those of vaccines because while people just imagine the vaccines to have a bad affect, there are people who are proven to get a bad affect from certain foods.Some Ideas for ResearchI think that the biggest problem with PETA s research in this regard is citing a study of only 20 individuals and only testing the removal of milk. A proper study would involve more subjects (maybe 100) and separate tests of removing milk, eggs, wheat, rice, seeds, nuts, soy, fish, spices, and food colorings and additives.It would also test various methods of reducing stress, having a more quiet environment etc.PETA do deserve some credit for leaving this URL in the comments [3], it s a study that disproves the link between milk and Autism. But it s still a small sample size of 14.Why They Target AutismIn spite of psychologists moving to define Asperger Syndrome and Autism as the same thing, the people who make such claims aren t talking about people like me. They want to make claims about people who can t refute them, such as people who fit the Rain Man stereotype. The Low Functioning Autistic (LFA) people have less ability to communicate and are easier to ignore, so if you want to make some stuff up about a group of people then they are an easy target.Parents of LFA kids are often desperate and will latch on to any idea that seems to help. They can imagine that things have improved in spite of a lack of evidence and also fail to realise that children can improve rapidly just due to their natural development, 1 month is not much time for an adult but for a 4yo it s 2% of their life!Also any supposed research that includes testimony of parents should be regarded as rubbish. If you talk to any child psychologist they will tell you about how difficult it is to convince parents that their child is on the Autism Spectrum, a Google search for autism + denial also brings up plenty of hits. Parental denial doesn t stop at refusing to accept a diagnosis, it includes wanting to reject it at any later time based on any shred of evidence that may be available. So when there is a diet change or some other supposed cause followed by some apparently better behavior they want to claim that Autism is cured.I ve written a document titled Some Suggestions for Parents of Aspies [4] which is largely based on the childhood memories and adult experiences of various Aspies. My document covers many issues and few kids on the Spectrum will benefit from all of them, but I think that there is something there that can benefit every child on the Spectrum and probably a lot of geeky NT kids too. It seems to me that when dealing with Autistic kids it s a lot better to seek advice from adults on the Autism Spectrum than from random Neuro-Typical people (NTs), particularly if those NTs happen to have an agenda to push. The sensible parents are the ones who seek advice from psychologists and people like me rather than from random unqualified NTs.

22 January 2011

Matthew Palmer: Debugging Debian Installer

I've been doing a bit of work recently on the netcfg portion of Debian Installer, with the largest work so far being adding IPv6 support. Unlike most of the rest of d-i, netcfg is written in C, and as such suffers from some difficulties in the realm of debugging. Worse, the whole d-i environment is rather bereft of the usual debugging tools (strace, lsof, etc) -- but most importantly, in this case, gdb. Thanks to some hints from Julien Cristau, I was reminded of the existence of gdbserver, a small part of gdb that allows you to debug a program running on a remote machine, using a copy of gdb running on your local machine. It's designed for debugging embedded systems, I think, because all of the heavy lifting (symbol lookups, etc) happens on your local machine, not where gdbserver is running. Here are the steps I took to get a useful gdbserver session inside d-i:

Get the gdb sources (apt-get source gdb) and drop into the gdbserver tree (cd gdb-x.y.z/gdb/gdbserver)
Configure gdbserver to build a static binary (LDFLAGS=-static ./configure). This is necessary because you don't want to have to faff around with library dependencies inside the d-i environment. Then build gdbserver (make).
Copy gdbserver over to your d-i machine/VM (I like using netcat; nc -l -p 31310 >/bin/gdbserver on the d-i machine, and nc IPADDR 31310 <gdbserver where you built gdbserver), then make it executable (chmod /bin/gdbserver)
Fire up gdbserver in d-i, attaching it to the process you wish to debug: gdbserver host:31337 --attach PID
Now things get a little trickier... on your local machine, you need to have an unstripped copy of the binary you're debugging. For me, this is easy, because I'm building netcfg myself (to keep the symbols, I just run DEB_BUILD_OPTIONS=noopt dpkg-buildpackage -b -rfakeroot) but I'm not sure what you'd do if you were just tracing around for fun.
Once you've got your binary, just run gdb BINARY on your host machine to start gdb running.
For the "even trickier" part of the show, your local machine also needs copies of all the libraries your binary is linked against. If you don't have this, when you try to debug you'll get all sorts of "wrong library or version mismatch?" errors, and your backtraces will likely break (amongst other things). Again, for me, this is easy, because I'm only running the d-i initrd, but if you're debugging something later in the installation process you might need to unpack some udebs somewhere. All I have to do is keep the initrd build tree around (which lives in the installer tree, under build/tmp/netboot/tree). Once you've got your tree of libraries, you can continue.
You need to tell gdb about that tree of libraries, by running the following in gdb: set solib-absolute-prefix /path/to/library/tree
Now, tell gdb to connect to your gdbserver: target remote IPADDR:31337 -- gdb should tell you "Remote debugging using IPADDR:31337", and the gdbserver should say "Remote debugging from host LOCALIP:31337".
Now, you should be able to interact with the program being debugged just like any other. Enjoy!

Mark Brown: Tracing ASoC with trace events

Kernel 2.6.38 will add support for tracing ASoC using trace points. Previously all logging for ASoC had been done using printk(), meaning that changing the active logging required a kernel rebuild and that when trace was enabled the volume of trace could easily become very disruptive to other logging within the system. Trace points solve these problems by providing a framework for enabling and disabling individual traces dynamically and separate logging infrastructure designed for easy filtering and post-processing. When using trace points it s reasonable to leave them enabled all the time, making life much easier especially when debugging intermittent problems. There s a bunch of existing documentation out there for the trace subsystem, including some in the kernel under Documentation/trace, which I won t repeat here. The ASoC events are grouped together under asoc . The raw log can be viewed by looking at the file tracing/trace in debugfs. Here s a sample output from starting playback of an MP3 on a Samsung SMDK6410 reference system using WM8580, a high performance six channel CODEC with a very simple software interface:

mplayer-1673  [000]   183.920000: snd_soc_dapm_start: card=SMDK-I2S
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Rear val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Center+Sub val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Front val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT3R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT3L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT2R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT2L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT1R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT1L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC3 val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC2 val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC1 val=1
mplayer-1673  [000]   183.920000: snd_soc_bias_level_start: card=SMDK-I2S val=2
mplayer-1673  [000]   183.920000: snd_soc_bias_level_done: card=SMDK-I2S val=2
mplayer-1673  [000]   183.920000: snd_soc_reg_read: codec=wm8580-codec.0-001b.27 reg=32 val=1e
mplayer-1673  [000]   183.920000: snd_soc_reg_write: codec=wm8580-codec.0-001b.27 reg=32 val=2
mplayer-1673  [000]   183.920000: snd_soc_bias_level_start: card=SMDK-I2S val=3
mplayer-1673  [000]   183.920000: snd_soc_bias_level_done: card=SMDK-I2S val=3
mplayer-1673  [000]   183.920000: snd_soc_dapm_done: card=SMDK-I2S
mplayer-1673  [000]   183.920000: snd_soc_reg_read: codec=wm8580-codec.0-001b.27 reg=13 val=10
mplayer-1673  [000]   183.920000: snd_soc_reg_write: codec=wm8580-codec.0-001b.27 reg=13 val=0

Each line represents a trace event, showing the process that initiated the event, the CPU it ran on and the time it was recorded at together with event-specific details. On this system the time is reported with 10ms accuracy and the CODEC is very simple so the timing information is not terribly informative, but more accurate timers can be really helpful in analysing what s going on with more complex CODECs. The set of events available will change over time, the key ones currently present in the kernel are:

snd_soc_dapm_start and snd_soc_dapm_done: These events show the start and stop of the DAPM algorithms, showing the time spent on power management decisions.
snd_soc_dapm_widget_power events are generated for each of the widgets which DAPM has determined needs to change state. The val=1 means that all these widgets are being powered up, val=0 would show power down.
snd_soc_reg_write events show values written to registers.
snd_soc_reg_read events show values read from registers these reads may be done from the register cache, or may be from the hardware.
snd_soc_dapm_widget_event_start and snd_soc_dapm_widget_event_done show the time spent in per-widget event callbacks.

The set of events traced will change as the subsystem develops and as people gain more experience with using the trace API.

Joey Hess: command composition

Traditionally, Unix has operated by piping between commands ("find grep sort uniq"). I wonder if we are experiencing a shift to command composition being a more common model? I started thinking about this when I considered the following command line:

chronic mr run git annex get

Unpacking, that uses chronic, which arranges for quiet running of mr (unless it fails, then it shows the full output), which in turn runs git in every repository it knows about, and git is asked to run the git-annex subcommand, which in turn has a "get" subcommand of its own. So, in other words, this is a cron job that syncs large files in a set of git repos, quietly. You might want to make it run niced also, which adds 3 more commands:

chronic nice ionice -c3 trickle -d100 mr run git annex get

This is the most extreme example I know of (unless you tack in a "sudo"), but there does seem to have been a gradual shift to using more composition, and more subcommands. In git in particular, two-levels of subcommands is not unknown, for example "git remote add". Subcommands, especially as implemented in git (or earlier, in arch), are kind of a special case of command composition that adds a namespace. I'm sure that a count of commands with subcommands would show it exploded in the past 10-12 years. I'll bet a count of composable commands would also show a significant increase. Anyone have other examples of long compositions of commands?

Ian Wienand: Adding a javascript tracker to your Docbook document

If you convert a docbook document in a chunked HTML form, it would presumably be very useful for you to place one of the various available javascript based trackers on each of your pages to see what is most popular, linked to etc. I'm assuming you've already got to the point where you have your DSSSL stylesheet going (see something like my prior entry). It doesn't appear you can get arbitrary output into the <head> tag -- the %html-header-tags% variable is really just designed for META tags and there doesn't appear to be anything else in the standard stylesheets to override. So the trick is to use $html-body-start$ . But you have to be a little bit tricky to actually get your javascript to slip past the SGML parser. After several attempts and a bit of googling I finally ended up with the following for a Google Analytics tracker, using string-append to get the javascript comment filtering in:

(define ($html-body-start$)
  (make element gi: "script"
  attributes: '(("language" "javascript")
  	            ("type" "text/javascript"))
		    (make formatting-instruction 
		      data: (string-append "<" "!--
 var _gaq = _gaq   [];
  _gaq.push(['_setAccount', 'UA-XXXXXXXXXX-1']);
  _gaq.push(['_trackPageview']);
  (function()  
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
   )();
// --" ">"))))

Julien Viard de Galbert: Triage X Bugs of 2011 first Weeks (TXBW9-10-11)

I ve been lasy about the reporting lately, so this is a 3 weeks update at once ! The bug count according to udd is down to 818. I guess this is due to Kibi s triaging on input related bugs, but I helped to close a few one too ! I kept my week numbering as before, so now I know the first bugs I pinged are more that 11 weeks old. I guess if some submitter did not answer in that delay, they will probably never do it. I think the bug count will go down again !

#451708 ping & close xserver-xorg-input-synaptics: Option GuestMouseOff true not working.
#424743 ping & forwarded X11 pen driver doesn t work on tabletPC Fujitsu Stylistic 2300 (fix included).
#499664 ping, try nouveau X fails to start with xserver-xorg-video-nv 1:2.1.12-1.
#512614 confirmed & upstream xset dpms rejects reasonable values as illegal.
#512711 ping and later closed All keypresses repeated roughly three times each, makes login impossible.
#513128 ping X server crashes every time when gnome-screensaver starts.
#486356 merge with #482592 numlock led is inversed (using numlockx) .
#515737 ping & closed After upgrading from Etch to Lenny, mouse pointer is very slow and keys don t repeat.
#515840 ping Desktop is not using full screen after upgrade from etch to lenny.
#516860 closed after looking at the git history Xorg with -sharevts use almost 100% of the CPU.
#362434 ping & closed, suggested to try nouveau nv has problems with xcursor-themes.
#502131 ping & closed System gets into a state and fails to process modifier keys.
#453296 closed (old ping unanswered) MacBook: cannot assign right mouse click to lower Enter key.
#504537 ping X server eats CPU on sparc.
#502313 ping Applications are getting ghost Alt keypress events, but I press no keys.
#527118 ping autoadded input devices kill X server.
#270887 forwarded startx and xinit do not preserve client arguments.
#542542 closed after looking at the git history non-VGA compatible graphics devices won t run X.
#492888 ping xorg ignores keystrokes.
#543210 merge with #492783 Alt-F7 to return to X, passes F7 key to top focus d window.
#524413 closed xauth fails with xorg 7.4, this was actually an xinit bug and the git log lead to #549377 being closed

The X Strike Force (still) needs you ! You can have a look at the X Strike Force Bug Closing Procedure and check XSF unstable bugs sorted by date.

Enrico Zini: Match package names across distributions

Match package names across distributions What would happen if we had a quick and reliable way to match package names across distributions? These ideas came up at the appinstaller2011 meeting:

it would be easy to lookup screenshots in the local distro, and if there are none then fall back on other distributions;
it would be easy to port Debtags to other distributions, and possibly get changes back;
it would be trivial to add a [patches in $DISTRO] link to the PTS
it would be easy to point to other BTSes

We thought they were good ideas, so we started hacking. To try it, you need to get the code and build the index first:

git clone git://git.debian.org/users/enrico/distromatch.git
cd distromatch
# Careful: 90Mb
wget http://people.debian.org/~enrico/dist-info.tar.gz
tar zxf dist-info.tar.gz
# Takes a long time to do the indexing
./distromatch --reindex --verbose

Then you can query it this way:

./distromatch $DISTRO $PKGNAME [$PKGNAME1 ...]

This would give you, for the package $PKGNAME in $DISTRO, the corresponding package names in all other distros for which we have data. If you do not provide package names, it automatically shows output for all packages in $DISTRO. For example:

$ time ./distromatch debian libdigest-sha1-perl
debian:libdigest-sha1-perl fedora:perl-Digest-SHA1
debian:libdigest-sha1-perl mandriva:perl-Digest-SHA1
debian:libdigest-sha1-perl suse:perl-Digest-SHA1
real    0m0.073s
user    0m0.056s
sys 0m0.016s

Yes it's quick. It builds a Xapian index with the information it needs, and then it reuses it. As soon as I find a moment, I intend to deploy an instance of it on DDE. It is using a range of different heuristics:

match packages by name;
match packages by desktop files contained within;
match packages by pkg-config metadata files contained within;
match packages by [/usr]/bin/* files contained within;
match packages by shared library files contained within;
match packages by devel library files contained within;
match packages by man pages contained within;
match stemmed form of development library package names;
match stemmed form of shared library package names;
match stemmed form of perl library package names;
match stemmed form of python library package names.

This list may get obsolete soon as more heuristics get implemented. Euristics will never cover all corner cases we surely have, but the idea is that if we can match a sizable amout of packages, the rest can be somehow fixed by hand as needed. The data it requires for a distribution should be rather straightforward to generate:

a file which maps binary package names to source package names
a file with the list of files in all the packages

For example:

$ ls -l dist-debian/
total 39688
-rw-r--r--  1 enrico enrico  1688249 Jan 20 17:37 binsrc
drwxr-xr-x  2 enrico enrico     4096 Jan 21 19:12 db
-rw-r--r--  1 enrico enrico 29960406 Jan 21 10:02 files.gz
-rw-r--r--  1 enrico enrico  8914771 Jan 21 18:39 interesting-files
$ head dist-debian/binsrc 
openoffice.org-dev openoffice.org
ext4-modules-2.6.32-5-4kc-malta-di linux-kernel-di-mipsel-2.6
linux-headers-2.6.30-2-common linux-2.6
libnspr4 nspr
ipfm ipfm
libforks-perl libforks-perl
med-physics debian-med
libntfs-3g-dev ntfs-3g
libguppi16 guppi
selinux selinux
$ zcat dist-debian/files.gz   head
memstat etc/memstat.conf
memstat usr/bin/memstat
memstat usr/share/doc/memstat/changelog.gz
memstat usr/share/doc/memstat/copyright
memstat usr/share/doc/memstat/memstat-tutorial.txt.gz
memstat usr/share/man/man1/memstat.1.gz
libdirectfb-dev usr/bin/directfb-config
libdirectfb-dev usr/bin/directfb-csource
libdirectfb-dev usr/include/directfb-internal/core/clipboard.h
libdirectfb-dev usr/include/directfb-internal/core/colorhash.h

interesting-files and db are generated when indexing. To prove the usefulness of the idea (but does it need proving?), you can find in the same git repo a little example app (it took me 10 minutes to write it), that uses the distromatch engine to export Debtags tags to other distributions:

$ ./exportdebtags fedora   head
memstat: admin::benchmarking, interface::commandline, role::program, use::monitor
libdirectfb-dev: devel::lang:c, devel::library, implemented-in::c, interface::framebuffer, role::devel-lib
libkonqsidebarplugin4a: implemented-in::c++, role::shared-lib, suite::kde, uitoolkit::qt
libemail-simple-perl: devel::lang:perl, devel::library, implemented-in::perl, role::devel-lib, role::shared-lib, works-with::mail
libpoe-component-pluggable-perl: devel::lang:perl, devel::library, implemented-in::perl, role::shared-lib
manpages-ja: culture::japanese, made-of::man, role::documentation
libhippocanvas-dev: devel::library, qa::low-popcon, role::devel-lib
libexpat-ocaml-dev: devel::lang:ocaml, devel::library, implemented-in::c, implemented-in::ocaml, role::devel-lib, works-with-format::xml
libgnutls-dev: devel::library, role::devel-lib, suite::gnu

Just in case this made you itch to play with Debtags in a non-Debian distribution, I've generated the full datasets for Fedora, Mandriva and OpenSUSE. Others have been working on the same matching problem. After we started writing code we started to become aware of existing work:

whohas
PackageMap
Equivalent-Packages, statistically generated from package contents, more info in this post

I'd like to make use of those efforts, maybe to cross-validate results, maybe even better as yet another heuristics.

21 January 2011

Adrian von Bidder: Wasted developer resources?

I had some hope when I read Girish Ramakrishnan's blog post that starts with To my knowledge, there are 3 Qt based JSON parsers out there . But I was really disappointed: instead of trying to get a consolidation going or at least highlighting the need for three different parsers, he announces yet another implementation. And, to make matters worse, it is intended to be statically linked whenever it is used. A friendly wave to all security conscious engineers who will now have to hunt down and kill security issues in various places wherever this json parser was used, in various different versions, possibly with local modifications. Girish, please do not take this as a personal attack, but what you're doing is just bad engineering practice. I don't claim qjsonparser is buggy. I haven't even looked at the code. But let's face it: bugs happen, and json is often passed over the net, so any parser is attack surface. So it should be as easy as possible to get fixed versions of the code out to the users. The means to do this is by allowing distribution builders to be aware of where the code in question was used, and to get fixed versions of it distributed easily. In other words: such code should always be in a shared library. Take, for example, the history of xpdf/poppler: many people spent countless hours chasing copies of xpdf code in many applications before they finally had enough, forked xpdf (if I have the history correctly) and created the poppler library which is now widely used. Now security issues with the PDF parser require one security fix, not 10.

Raphaël Hertzog: People behind Debian: Michael Vogt, synaptic and APT developer

Michael and his daughter Marie

Michael has been around for more than 10 years and has always contributed to the APT software family. He s the author of the first real graphical interface to APT synaptic. Since then he created software-center as part of his work for Ubuntu. Being the most experienced APT developer, he s naturally the coordinator of the APT team. Check out what he has to say about APT s possible evolutions. My questions are in bold, the rest is by Michael. Who are you? My name is Michael Vogt, I m married and have two little daughters. We live in Germany (near to Trier) and I work for Canonical as a software developer. I joined Debian as a developer in early 2000 and started to contribute to Ubuntu in 2004. What s your biggest achievement within Debian or Ubuntu? I can not decide on a single one so I will just be a bit verbose. From the very beginning I was interested in improving the package manager experience and the UI on top for our users. I m proud of the work I did with synaptic. It was one of the earliest UIs on top of apt. Because of my work on synaptic I got into apt development as well and fixed bugs there and added new features. I still do most of the uploads here, but nowadays David Kalnischkies is the most active developer. I also wrote a bunch of tools like gdebi, update-notifier, update-manager, unattended-upgrade and software-properties to make the update/install situation for the user easier to deal with. Most of the tools are written in python so I added a lot of improvements to python-apt along the way, including the initial high level apt interface and a bunch of missing low-level apt_pkg features. Julian Andres Klode made a big push in this area recently and thanks to his effort the bindings are fully complete now and have good documentation. My most recent project is software-center. Its aim is to provide a UI strongly targeted for end-users. The goal of this project is to make finding and installing software easy and beautiful. We have a fantastic collection of software to offer and software-center tries to present it well (including screenshots, instant search results and soon ratings&reviews). This builds on great foundations like aptdaemon by Sebastian Heinlein, screenshots.debian.net by Christoph Haas, ddtp.debian.org by Michael Bramer, apt-xapian-index by Enrico Zini and many others (this is what I love about free software, it usually adds , rarely takes away ). What are your plans for Debian Wheezy? For apt I would love to see a more plugable architecture for the acquire system. It would be nice to be able to make apt-get update (and the frontends that use this from libapt) be able to download additional data (like debtags or additional index file that contains more end-user targeted information). I also want to add some scripts so that apt (optionally) creates btrfs snapshots on upgrade and provide some easy way to rollback in case of problems. There is also some interesting work going on around making the apt problem resolver a more plugable part. This way we should be able to do much faster development. software-center will get ratings&reviews in the upstream branch, I really hope we can get that into Wheezy. If you could spend all your time on Debian, what would you work on? In that case I would start with a refactor of apt to make it more robust about ABI breaks. It would be possible to move much faster once this problem is solved (its not even hard, it just need to be done). Then I would add a more complete testsuite. Another important problem to tackle is to make maintainer scripts more declarative. I triaged a lot of upgrade bug reports (mostly in ubuntu though) and a lot of them are caused by maintainer script failures. Worse is that depending on the error its really hard for the user to solve the problem. There is also a lot of code duplication. Having a central place that contains well tested code to do these jobs would be more robust. Triggers help us a lot here already, but I think there is still more room for improvement. What s the biggest problem of Debian? That s a hard question

I mostly like Debian the way it is. What frustrated me in the past were flamewars that could have been avoided. To me being respectful to each other is important, I don t like flames and insults because I like solving problems and fighting like this rarely helps that. The other attitude I don t like is to blame people and complain instead of trying to help and be positive (the difference between it sucks because it does not support $foo instead of it would be so helpful if we had $foo because it enables me to let me do $bar ). For a long time, I had the feeling you were mostly alone working on APT and were just ensuring that it keeps working. Did you also had this feeling and are things better nowadays ? I felt a bit alone sometimes

That being said, there were great people like Eugene V. Lyubimkin and Otavio Salvador during my time who did do a lot of good work (especially at release crunch times) and helped me with the maintenance (but got interested in other area than apt later). And now we have the unstoppable David Kalnischkies and Julian Andres Klode. Apt is too big for a single person, so I m very happy that especially David is doing superb work on the day-to-day tasks and fixes (plus big project like multiarch and the important but not very thankful testsuite work). We talk about apt stuff almost daily, doing code reviews and discuss bugs. This makes the development process much more fun and healthy. Julian Andres Klode is doing interesting work around making the resolver more plugable and Christian Perrier is as tireless as always when it comes to the translations merging. I did a quick grep over the bzr log output (including all branch merges) and count around ~4300 total commits (including all revisions of branches merged). Of that there ~950 commits from me plus an additional ~500 merges. It was more than just ensuring that it keeps working but I can see where this feeling comes from as I was never very verbose. Apt also was never my only project, I am involved in other upstream work like synaptic or update-manager or python-apt etc). This naturally reduced the time available to hack on apt and spend time doing the important day-to-day bug triage, response to mailing list messages etc. One the python-apt side Julian Andres Klode did great work to improve the code and the documentation. It s a really nice interface and if you need to do anything related to packages and love python I encourage you to try it. Its as simple as:

import apt
cache = apt.Cache()
cache["update-manager"].mark_install()
cache.commit()

Of course you can do much more with it (update-manager, software-center and lots of more tools use it). With pydoc apt you can get a good overview. The apt team always welcomes contributors. We have a mailing list and a irc channel and it s a great opportunity to solve real world problems. It does not matter if you want to help triage bugs or write documentation or write code, we welcome all contributors. You re also an Ubuntu developer employed by Canonical. Are you satisfied with the level of cooperation between both projects? What can we do to get Ubuntu to package new applications developed by Canonical directly in Debian? Again a tricky question

When it comes to cooperation there is always room for improvement. I think (with my Canonical hat on) we do a lot better than we did in the past. And it s great to see the current DPL coming to Ubuntu events and talking about ways to improve the collaboration. One area that I feel that Debian would benefit is to be more positive about NMUs and shared source repositories (collab-maint and LowThresholdNmu are good steps here). The lower the cost is to push a patch/fix (e.g. via direct commit or upload) the more there will be. When it comes to getting packages into Debian I think the best solution is to have a person in Debian as a point of contact to help with that. Usually the amount of work is pretty small as the software will have a debian/* dir already with useful stuff in it. But it helps me a lot to have someone doing the Debian uploads, responding to the bugmail etc (even if the bugmail is just forwarded as upstream bugreports

IMO it is a great opportunity especially for new packagers as they will not have to do a lot of packaging work to get those apps into Debian. This model works very well for me for e.g. gdebi (where Luca Falavigna is really helpful on the Debian side). Is there someone in Debian that you admire for his contributions? There are many people I admire. Probably too many to mention them all. I always find it hard to single out individual people because the project as a whole can be so proud of their achievements. The first name that comes to my mind is Jason Gunthorpe (the original apt author) who I ve never met. The next is Daniel Burrows who I met and was inspired by. David Kalnischkies is doing great work on apt. From contributing his first (small) patch to being able to virtually fix any problem and adding big features like multiarch support in about a year. Sebastian Heinlein for aptdaemon. Christian Perrier has always be one of my heroes because he cares so much about i18n. Christoph Haas for screenshots.debian.net, Michael Bramer for his work on debian translated package descriptions.

Thank you to Michael for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

4 comments Liked this article? Click here. My blog is Flattr-enabled.

Richard Darst: DebConf organization/volunteer meeting

As I just announced, the DebConf team will have an "organizational meeting". Every year, many teams need new people, since it is such a intense job for a few months. This year, I'm trying to get people who have not previously been involved in DebConf organization to help with these jobs.The meeting will be held on #debconf-team on irc.debian.org at 20:00 UTC on 25th January 2011. The main point of the meeting is to go through and fill out the teams list. This doesn't mean talking about how jobs should be done this year, just figuring out who is interested, sign a team up, and let the team start thinking about the details. The team can discuss details among themselves, and with the larger DebConf team on the lists and in future meetings.In past years, this meeting has helped to kickstart the teams and really began the DebConf cycle. This year, I am hoping to also make it a volunteer recruitment meeting. The DebConf team works hard, often to the point of burn-out, and thus would love new people. We need new people.In the next few days, I'll try to make more posts in my DebConf series to help make people familiar with the way DebConf works. You could consider this meeting to be the ultimate goal of the DebConf article series: helping to recruit new people at this meeting.Of course, if you want to join the DebConf team any other time, just drop by and offer a hand.

Cyril Brulebois: Debian XSF News #2

Time for a second Debian XSF News issue! I ll skip stuff related to the new katamari since I wrote about that already.

I reassigned a big memory leak to the kernel. It happens with a Radeon HD 4200 card and the xserver-xorg-video-ati driver.
I mentioned on debian-boot@ how to reassign bug to the xorg meta package. We regularly receive bug reports about either X-related issues in the graphical installer, or about the just-installed X stack.
I started a xsf-docs repository to hold some documentation, trying to keep up-to-date information about the X stack. The wiki is unfortunately a bit of a mess, and contains some very old stuff like an ancient FAQ. This has historical value, but doesn t really help. ;) The new documentation lives at http://pkg-xorg.alioth.debian.org/, and it should be packaged at some point (maybe merged into an existing package).
Julien Cristau uploaded xorg-server 2:1.7.7-10 to unstable, and it was unblocked for squeeze.
Triage happened on the xorg meta package, which should be our frontdesk : people should report or reassign bugs against this package when they re not sure which X component is at fault.
The intel driver was tweaked to fail properly if KMS is disabled, so that the X server autodetection code could fall back to vesa. Unfortunately, it turned out to be insufficient in some cases. That was fixed in the we wish you a working driver release. The latter migrated to squeeze.
On a related note, we discovered (through bug reports), that the intel driver in experimental somehow needs libdrm-intel1 from experimental. Not that symbols would be missing, but some nasty bugs happen when the version from unstable is used instead. We still need to address this issue.
Julien pushed more patches to the server-1.7-branch upstream, which I merged, uploaded as 2:1.7.7-11, and got unblocked for squeeze. A crasher was reported right after that, but it s probably going to wait for squeeze r1 (see next entry).
Bernhard R. Link reported an issue with old XF86Config-4 configuration files. We cloned that bug report and fixed both in git. I ll probably upload xorg-server 2:1.7.7-12 to unstable soon with those fixes. Then we ll probably upload 2:1.7.7-12~squeeze1 to stable-proposed-updates once squeeze is released.
To keep track of this kind of bugs, I ve started using the squeeze-candidate usertag. That s documented in the how to triage bugs page, added to xsf-docs.
I updated the cirrus driver to improve support in QEMU and FAUmachine, it s been unblocked for squeeze.
I fixed partial upgrades for the libx11-6 and libx11-data packages.
Julien Viard de Galbert has been very active in the bug triaging area! See the reports: TXBW3, TXBW4+5, TXBW6, TXBW7, TXBW8.
He also blogged about how easy it is to triage X bugs. Join the fun!
I also wrote a tiny doc about configuring input, and poked a bunch of bugs for the following input drivers: evdev, kbd, mouse, and synaptics.
I digged into the infamous gdm3 bug, which causes xkb options to be lost when one logs in. A summary is available in the upstream bug report.

20 January 2011

John Goerzen: Research on deduplicating disk-based and cloud backups

Yesterday, I wrote about backing up to the cloud. I specifically was looking at cloud backup services. I ve been looking into various options there, but also various options for disk-based backups. I d like to have both onsite and offsite backups, so both types of backup are needed. Also, it is useful to think about how the two types of backups can be combined with minimal overhead. For the onsite backups, I d want to see:

Preservation of ownership, permissions, etc.
Preservation of symlinks and hardlinks
Space-efficient representation of changes ideally binary deltas or block-level deduplication
Ease of restoring
Support for backing up Linux and Windows machines

Deduplicating Filesystems for Local Storage Although I initially thought of block-level deduplicating file systems as something to use for offsite backups, they could also make an excellent choice for onsite disk-based backups. rsync-based dedup backups One way to use them would be to simply rsync data to them each night. Since copies are essentially free, we could do (or use some optimized version of) cp -r current snapshot/2011-01-20 or some such to save off historic backups. Moreover, we d get dedup both across and within machines. And, many of these can use filesystem-level compression. The real upshot of this is that the entire history of the backups can be browsed as a mounted filesystem. It would be fast and easy to find files, especially when users call about that file that they deleted at some point in the past but they don t remember when, exactly what it was called, or exactly where it was stored. We can do a lot more with find and grep to locate these things than we could do with tools in Bacula (or any other backup program) restore console. Since it is a real mounted filesystem, we could also do fun things like make tarballs of it at will, zip parts up, scp them back to the file server, whatever. We could potentially even give users direct access to their files to restore things they need for themselves. The downside of this approach is that rsync can t store all the permissions unless it s running as root on the system. Wrappers such as rdup around rsync could help with that. Another downside is that there isn t a central scheduling/statistics service. We wouldn t want the backup system to be hammered by 20 servers trying to send it data at once. So there d be an element of rolling our own scripts, though not too bad. I d have preferred not to authorize a backup server with root-level access to dozens of machines, but may be inescapable in this instance. Bacula and dedup The other alternative I thought of system such as Bacula with disk-based volumes . A Bacula volume is normally a tape, but Bacula can just write them to disk files. This lets us use the powerful Bacula scheduling engine, logging service, pre-backup and post-backup jobs, etc. Normally this would be an egregious waste of disk space. Bacula, like most tape-heritage programs, will write out an entire new copy of a file if even one byte changes. I had thought that I could let block-level dedupe reduce the storage size of Bacula volumes, but after looking at the Bacula block format spec, this won t be possible as each block will have timestamps and such in it. The good things about this setup revolve around using the central Bacula director. We need only install bacula-fd on each server to be backed up, and it has a fairly limited set of things it can do. Bacula already has built-in support for defining simple or complicated retention policies. Its director will email us if there is a problem with anything. And its logs and catalog are already extensive and enable us to easily find out things such as how long backups take, how much space they consume, etc. And it backs up Windows machines intelligently and comprehensively in addition to POSIX ones. The downsides are, of course, that we don t have all the features we d get from having the entire history on the filesystem all at once, and far less efficient use of space. Not only that, but recovering from a disaster would require a more extensive bootstrapping process. A hybrid option may be possible: automatically unpacking bacula backups after they ve run onto the local filesystem. Dedupe should ensure this doesn t take additional space if the Bacula blocksize aligns with the filesystem blocksize. This is certainly not a given however. It may also make sense to use Bacula for Windows and rsync/rdup for Linux systems. This seems, however, rather wasteful and useless. Evaluation of deduplicating filesystems I set up and tested three deduplicating filesystems available for Linux: S3QL, SDFS, and zfs-fuse. I did not examine lessfs. I ran a similar set of tests for each:

Copy /usr/bin into the fs with tar -cpf - /usr/bin tar -xvpf - -C /mnt/testfs
Run commands to sync/flush the disk cache. Evaluate time and disk used at this point.
Rerun the tar command, putting the contents into a slightly different path in the test filesystem. This should consume very little additional space since the files will have already been there. This will validate that dedupe works as expected, and provide a hint about its efficiency.
Make a tarball of both directories from the dedup filesystem, writing it to /dev/zero (to test read performance)

I did not attempt to flush read caches during this, but I did flush write caches. The test system has 8GB RAM, 5GB of which was free or in use by a cache. The CPU is a Core2 6420 at 2.13GHz. The filesystems which created files atop an existing filesystem had ext4 mounted noatime beneath them. ZFS was mounted on an LVM LV. I also benchmarked native performance on ext4 as a baseline. The data set consists of 3232 files and 516MB. It contains hardlinks and symlinks. Here are my results. Please note the comments below as SDFS could not accurately complete the test.

Test	ext4	S3QL	SDFS	zfs-fuse
First copy	1.59s	6m20s	2m2s	0m25s
Sync/Flush	8.0s	1m1s	0s	0s
Second copy+sync	N/A	0m48s	1m48s	0m24s
Disk usage after 1st copy	516MB	156MB	791MB	201MB
Disk usage after 2nd copy	N/A	157MB	823MB	208MB
Make tarball	0.2s	1m1s	2m22s	0m54s
Max RAM usage	N/A	150MB	350MB	153MB
Compression	none	lzma	none	gzip-2

It should be mentioned that these tests pretty much ruled out SDFS. SDFS doesn t appear to support local compression, and it severely bloated the data store, which was much larger than the original data. Moreover, it permitted any user to create and modify files, even if the permissions bits said that the user couldn t. tar gave many errors unpacking symlinks onto the SDFS filesystem, and du -s on the result threw up errors as well. Besides that, I noted that find found 10 fewer files than in my source data. Between the huge memory consumption, the data integrity concerns, and inefficient disk storage, SDFS is out of the running for this project. S3QL is optimized for storage to S3, though it can also store its files locally or on an sftp server a nice touch. I suspect part of its performance problem stems from being designed for network backends, and using slow compression algorithms. S3QL worked fine, however, and produced no problems. Creating a checkpoint using s3qlcp (faster than cp since it doesn t have to read the data from the store) took 16s. zfs-fuse appears to be the most-used ZFS implementation on Linux at the moment. I set up a 2GB ZFS pool for this test, and set dedupe=on and compress=gzip-2. When I evaluated compression in the past, I hadn t looked at lzjb. I found a blog post comparing lzjb to the gzip options supported by zfs and wound up using gzip-2 for this test. ZFS really shone here. Compared to S3QL, it took 25s instead of over 6 minutes to copy the data over and took only 28% more space. I suspect that if I selected gzip -9 compression it would have been closer both in time and space to S3QL. But creating a ZFS snapshot was nearly instantaneous. Although ZFS-fuse probably doesn t have as many users as ZFS on Solaris, still it is available in Debian, and has a good backing behind it. I feel safer using it than I do using S3QL. So I think ZFS wins this comparison. I spent quite some time testing ZFS snapshots, which are instantaneous. (Incidentally, ZFS-fuse can t mount them directly as documented, so you create a clone of the snapshot and mount that.) They worked out as well as could be hoped. Due to dedupe, even deleting and recreating the entire content of the original filesystem resulted in less than 1MB additional storage used. I also tested creating multiple filesystems in the zpool, and confirmed that dedupe even works between filesystems. Incidentally wow, ZFS has a ton of awesome features. I see why you OpenSolaris people kept looking at us Linux folks with a sneer now. Only our project hasn t been killed by a new corporate overlord, so guess that maybe didn t work out so well for you <grin>. The Cloud Tie-In This discussion leaves another discussion: what to do about offsite backups? Assuming for the moment that I want to back them up over the Internet to some sort of cloud storage facility, there are about 3 options:

Get an Amazon EC2 instance with EBS storage and rsync files to it. Perhaps run ZFS on that thing.
Use a filesystem that can efficiently store data in S3 or Cloud Files (S3QL is the only contender here)
Use a third-party backup product (JungleDisk appears to be the leading option)

There is something to be said for using a different tool for offsite backups if there is some tool-level issue, that could be helpful. One of the nice things about JungleDisk is that bandwidth is free, and disk is the same $0.15/GB-mo that RackSpace normally charges. JungleDisk also does block-level dedup, and has a central management interface. This all spells nice for us. The only remaining question would be whether to just use JungleDisk to back up the backup server, or to put it on each individual machine as well. If it just backs up the backup server, then administrative burdens are lower; we can back everything there up by default and just not worry about it. On the other hand, if there is a problem with our main backups, we could be really stuck. So I d say I m leaning towards ZFS plus some sort of rsync solution and JungleDisk for offsite. I had two people suggest CrashPlan Pro on my blog. It looks interesting, but is a very closed product which makes me nervous. I like using standard tools and formats gives me more peace of mind, control, and recovery options. CrashPlan Pro supports multiple destinations and says that they do cloud hosting, but don t list pricing anywhere. So I ll probably not mess with it. I m still very interested in what comments people may have on all this. Let me know!

Alexander Reichle-Schmehl: Myths and Facts about Firmwares and their non-removal from Debian

Debian's announcement to release Squeeze with a completely free Linux kernel caused quite some attention, which is actually a good thing. However, it also seems to have caused quite some uncertainty and was often partially misunderstood and miss quoted. I'll try to summarize and answer some of them in this blog post:

Myth: Debian removed all firmware files from its kernels!
Fact: No, it's just about the kernels, which will be shipped with the upcoming release Debian 6.0 Squeeze. The kernels in the current stable release Debian 5.0 Lenny remain as they are... Well, of course we will release security updates for them, but they will still contain the same firmware files as the present kernels.
Myth: Debian is ripping stuff out of it's kernels
Fact: Debian moved some firmware files from its main archive to the non-free part of the archive. They are still there, just in the part of archive for stuff not satisfying our Free Software Guidelines.
Myth: Debian will be uninstallable for many users
Fact: The non-free firmware files are still available over our infrastructure. Those, who are needed during installation (e.g. for network or storage controllers) can also be loaded during the installation (be it from CD or USB-Stick). We offer tarballs with these files (just unpack them on a usb-stick and plug it in when asked) as well as netinstall iso images already containing these non-free files. Of course these will stay there, after Squeeze has been released, too.
Myth: Firmware files are needed, ripping them out doesn't accomplish anything and isn't good for our users
Fact: Firmware files are needed by some drivers for some specific hardware, yes. But not all users want them. And, as we are now able to load all these firmware files when needed (in stead of compiling them into the driver itself), we are now able so ship them separately, why not do so? That allows those, who need non-free firmware to use them, while those, who don't want them, to not install them at all.
Myth: Ah, those Debian freedom zealots again...
Fact: It's not only us, actually, without the cooperation of many Linux Kernel developers, we couldn't have achieved that goal again. And it's not only us, who's interested in creating a free Linux Kernel, other major distributions see the problems, too. See for example the recent comment by a Fedora developer about changes in such a non-free firmware file. It just seems, that Debian was one of the first to realise the problem of non-free firmware files.
Myth: Debian is going down on it's knees before Stallman
Fact: I haven't talked with RMS about that, but I think Debian is still not free enough for him; as far as I know, he would like to see the non-free archive vanish completely, or at least not mentioned anywhere at all.

So, one question remains: What is to bad about non-free firmware files? Aren't they just some tiny programs executed in the CPU of the device? Why care about them Good question! Let's take possible legal issues aside, and just look the practical side. The core problem is: Without source (and tool chain to use the source) firmware files are just some random numbers for us. We don't know what they are doing, we can't analyse and improve them. We can't change them, we can't support them. Maybe you already followed the link above to comment of the Fedora developer. I cite him here again, because I think he summarized the problem so well:

Updated qlogic 2400 and 2500 firmware to 5.03.13. What does 5.03.13 do? No one knows, except for QLogic, and they're not telling. I asked, and they told me that information was only available under NDA. So, I encourage you to imagine what this firmware does, and the bugs it fixes. While you're at it, imagine a world where vendors release source code for their firmware.

So, now that we established the fact, that we can't support firmware files, one could wonder, if we actually need to do so. What harm can a simple, tiny program in a peripheral device do to your computer? Well, scientist already managed to created trojaned firmwares for some network cards. So it is a problem, and can even under some circumstanced lead to security problems! So, to summarise: Yes, Debian changed something in the Kernels. No, it will continue to work as well as usual. Some users might need to enable the non-free repository in their sources list, but those who won't, don't have to. Firmware files needed during installation are also available, and can be loaded by the installation system. So, what's all the fuss about? BTW: Those of you, who can't remember links very well, but fear, they still might need the links to the non-free images and tarballs. just remember two things: wiki and Firmware, as you'll find all you need on the Firmware page of Debian's wiki.

Julien Danjou: Code fontification with Gnus and Org-mode

I've added code fontification using Org src blocks inside Gnus. This interprets the block as an Org buffer and fontify it accordingly if org-src-fontify-natively it set to t. Thanks to S bastien Vauban for the original idea and implementation. Now it works out of the box without any customization.

Richard Darst: The DebConf talk selection and scheduling process

The DebConf talk selection process is in charge of selecting the talks at DebConf. Some of the reasons for having a talks team are to help ensure a high quality of talks, maintain a somewhat academic feeling for the conference, and to let attendees know what talks can be expected to be good.Even though there is a big talk selection process in advance, DebConf has always allowed attendees to schedule new talks during the conference - at DebConf10, these were called "ad-hoc" talks. At DC10, attendees could sign up for a room+time slot on a wiki page, and each night they would be added to the main schedule. In past years, this distinction had been called "official" (pre-selected) vs "non-official" (new ideas during DebConf) talks, which was extremely confusing to people. "Ad-hoc" talks are a very important part of DebConf: not all development meetings are pre-planned.Since we pretty much always surplus of talk rooms and times and we let people schedule talks during DebConf, people have asked: what is the purpose of talk selection? Are we going to let all talks be given? Are we going to reject talks just to say we selected them? When talks were "rejected" in the past, the speakers were told to just ask for a room during DebConf if they still wanted to give it. This question was raised on our mailing list. Even if there is a surplus of talk rooms, there are reasons for talk selection: it allows attendees to know what is most prepared, helps the video team to capture the best talks, lets talks be scheduled in the best room. You could also say it helps to organize talks into tracks and to collect abstracts for proceedings.For DebConf10, we introduced a "conference track" concept, a set of related talks which would be scheduled in the same room in sequence. This allowed a more structured conference for attendees, and perhaps recruiting speakers to fill in requested talks for the tracks. Example tracks from DebConf10 are Science, Enterprise, and Community Outreach. I would have liked to see some tracks more Debian related, such as: project infrastructure, project communication, project social issues, ..., but what we had was a good start that can be improved on, if the new team desires to.As for scheduling, there are two main parts: one phase before DebConf, assigning the selected talks to rooms and making the advance schedule. The other phase happens during DebConf and handles assigning rooms to people who want to do "ad-hoc" talks. The two phases aren't always done by the same person. The schedulers need to balance the expected audience and room capacity, need for video coverage, BOF vs auditorium style seating, not making talks overlap that are likely to appeal to the same audience, ... . The scheduling team works with the talk team, but doesn't necessarily have to be the same people.I'm very happy that there was good discussion about the selection and scheduling process for DebConf10. In the end, a talk selection and scheduling wiki page was created. This page had two main purposes. First, it discussed and documented the selection and scheduling process, so that it was transparent to people that year and provides a reference for future teams. Secondly, other people have added their proposed talks workflows, that they can be considered in future years.
Talk selection has many competing demands - because of the completeness of the wiki page, I'll just refer you there. The rest of this post mainly frames the types of questions the talks team has to face. If you have thoughts on the DebConf talk selection process, feel free to add your ideas to the talk selection process wiki page, or better yet, help out with the team this year.

Next.

Previous.