Search Results: "cas"

22 April 2009

Neil Williams: package lists without fluff

(Sometimes a blog is just somewhere to put stuff you'll otherwise forget.)

dpkg -l colrm 90 colrm 1 2 grep '^ ' column -t

(There are two spaces in that grep, just in case it doesn't come out right in the final blog rendering.)

For those times when you just want the package list without the fluff - or, as a script that can accept a parameter:

#!/bin/sh
dpkg -l "$*" colrm 90 colrm 1 2 grep '^ ' sort column -t


$ ~/scripts/versiontable.sh apt* dpkg*

Could add grep ii as an extra pipe in that (after the initial dpkg -l call) to only list installed versions etc.

$ ~/scripts/versiontable.sh dpkg* apt*
apt              0.7.21
apt-arm-cross 0.7.11
apt-cross 0.13.0
apt-doc 0.7.21
aptitude 0.4.11.11-1+b1
apt-show-source 0.10
apt-utils 0.7.21
dpkg 1.14.26
dpkg-cross 2.5.1
dpkg-dev 1.14.26
dpkg-www 2.54+nmu1


Interesting side-effects though - *foo* is different to "*foo*" or \*foo\*. Might be better to use $@ but that means always needing quote marks for multiple package queries.

21 April 2009

Romain Beauxis: Liquidsoap now supports AAC+ encoding.

The plot Following many user requests , we were considering adding this format to the list of supported encoders. As for mp3 encoding, this support was subject to some legal issues. These issues were indeed the initial motivation for implementing icecast2 using the patent-free ogg codecs and container for streaming data. However, it seems that many users and net-radios are still asking for these formats. There can be several motivations for that, among which I can list the following major ones:
  • These formats are supported by more players and, in particular, embedded devices, which are the main target for net-radios
  • For the case of AAC+, the codec supports a very good audio quality for low bandwidths, such as 64 and 32kb/s.

The philosophy There is a constant complain from the open source communities about the use of patent-encumbered formats. In particular, those formats are not allowed in the Wikimedia commons and the corresponding encoders, in particular mp3 encoders, are not supported by for instance the Debian distribution. In a similar intent to push for a wider usage of patent-free codecs, the icecast project reimplemented its streaming server in order to use the ogg container and codecs. All these actions are very important. In particular, the support for ogg/theora video files in firefox, and hopefully the major HTML 5 browser is mainly the result of wikipedia refusing other video formats. Indeed, wikipedia is now a major website, and it would be insane to not support the videos there. However, I for myself do not like when these actions become too restrictive to the user. For this matter, I personally prefer freedom of choice among other considerations. Also, as a matter of fact, when trying to change the situation about these issues, one has to take into account the relative importance of each actor. It is because wikipedia is already huge that it has managed to change the things. Hence, in the case of net-radios, a majority of potential users are asking for AAC+ and other patent-encumbered formats. Although this is not the ideal situation we would like to see, there comes a point were either the free-software projects have something to propose, or the users will just go for a proprietary software, which is even worse to the situation. For this purpose, I believed it was important to add such support into our software. However, as you will see later, we do not implement ourself the encoding algorithm, but only added support for external encoders. That way, we simply let the user choose what he wants to use, but remain a patent-free and open-source software.

The implementation Since mp3 was, and still is, the major target for net-radios, icecast2 still has the possibility to stream this format. In fact, this format, as well as AAC+, does not have headers, such that it is possible to decode a stream at any point in it. Hence, in this case, icecast only acts as a proxy, and the only constraint is to send it the encoded data at real rate. Hence, the main library for streaming to icecast, libshout, parses the audio data that it receives, and schedules the streaming according to real rate. Additionally, it also supports a raw mechanism, for which there would be not such mangling. In the case of liquidsoap, we did not need this feature of the libshout since the software is already working in real time. Furthermore, the source connection protocol provides a content-type header field, which allows icecast to display the correct mime for the data format that is streamed. All in all, in order to send AAC+ data to icecast, one needs to send the correct mime-type in the source connection headers, and then send audio data at the correct rate. However, the libshout did not allow to set an arbitrary content-type header. Hence, I proposed a patch. It was proposed for an inclusion into main code, since I believe it is interesting not only for our specific need, but also for any other generic data format support. Although the patch was accepted, it was never committed. So, we decided to go on: since liquidsoap need not the real rate mangling feature of the libhsout, and since the source protocol is mainly a basic HTTP, we decided it was worth a custom implementation. This implementation was quite easy, thanks to the powerful capabilities of the OCaml language, and resulted in a custom implementation, ocaml-cry, that does the same as libshout, except that it does not control the rate of the data flow sent to the icecast server. Then, we also needed to support encoding data to the AAC format. After looking at the only available open-source encoder, aacplusenc, I tried to propose some changes in order to export a library and be able to use it in liquidsoap. However, these propositions did not make it into upstream's code, so I decided to also move on and use directly the encoder binary. Hence, using only the good old unix standard input/output mechanism, it is possible to start an encoder as a child process, send it the raw PCM data to its standard input and read the encoded data from its standard output. This lead to the development of the external output operators, which include output.icecast.external. Although this implementation was not easy, it now appears to work quite well in the current SVN version. In particular, the encoder process exits correctly for the FLAC encoder, which is needed in order to stream correctly, since the ogg container needs a restart for every track. Hence, we also gained support for encoding to the FLAC audio format, which was not binded for OCaml (yet ?) because its callback-driven API makes it tricky to do. Additionally, the support for external processes as encoders allows to use the lame binary for encoding mp3 data. Hence, liquidsoap can now encode mp3 data without the need of a built-in support for it. And, as always, the idea of using external processes spread over several other operators, including the support for external file and stream decoders, enabling support for many audio format (in fact almost all supported in linux, using mplayer). In conclusion, liquidsoap now has a fairly good support for AAC+ audio streaming, as well as a many new formats for decoding files and streams. It also support arbitrary formats and encoders using a custom variant of the output.icecast.external. Hence, provided your format can be decoded from any point in the stream, you should be able to use liquidsoap to stream data to icecast as soon as you have a binary which is able to receive RAW or WAV PCM data to its standard input and send encoded data to its standard output. As an illustration, here is an implementation of the output.icecast.aacplusenc:
def output.icecast.aacplusenc(
~id="output.icecast.aacplusenc",
~start=true,
~restart=true,
~restart_delay=3,
~host="localhost",
~port=8000,
~user="source",
~password="hackme",
~genre="Misc",
~url="http://savonet.sf.net/",
~description="OCaml Radio!",
~public=true,
~dumpfile="",
~mount="Use [name]",
~name="Use [mount]",
~protocol="http",
~aacplusenc="aacplusenc",
~bitrate=64,
~restart_on_crash=false,
~restart_on_new_track=false,
~restart_encoder_delay=3600,
~headers=[],
s)
# Metadata update is set by ICY with icecast
def aacplusenc_p(m)
"# aacplusenc - - # bitrate "
end
output.icecast.external(id=id,
process=aacplusenc_p,
bitrate=bitrate,
start=start,
restart=restart,
restart_delay=restart_delay,
host=host,
port=port,
user=user,
password=password,
genre=genre,
url=url,
description=description,
public=public,
dumpfile=dumpfile,
name=name,
mount=mount,
protocol=protocol,
header=true,
restart_on_crash=restart_on_crash,
restart_on_new_track=restart_on_new_track,
headers=headers,
restart_encoder_delay=restart_encoder_delay,
format="audio/aacp",
s)
end

Lucas Nussbaum: I hate thunderbird (and its HTML-only emails)

Apparently, since Thunderbird 2.0, the user now has an easy way to choose to send messages as HTML-only emails, instead of the sane defaults of doing text only or text + HTML . As a result, lots of Thunderbird users now send HTML-only emails, which are a PITA to read with mutt. Does someone know if there s an open bug about reverting that change? Does someone know of a good strategy to convert HTML-only emails to text emails, preferably pluggable into procmail, so it happens at delivery time? It s how course possible to read HTML emails using an external HTML viewer, but I can t find a way to reply to a stripped version of the messages.

Russell Coker: Amusing Thanks.txt Entry

My SE Linux Play Machine [1] has a file named thanks.txt for users to send messages to me [2]. On a number of occasions people have offered to give me things in exchange for the password for the bofh account (the one with sysadm_r privileges). I ve been offered stolen credit cards, a ponzi scheme of root access to servers on the net, and various other stuff. Today I received an amusing joke entry: Hello Kind Sir, I am Dr. Adamu Salaam, the the bank manager of bank of africa (BOA) Burkina Faso West I am sending you this message about the $3.14159 million dollars in bank account number 2718281828450945. I will give you this money in exchange for the password to the bofh account. The amount of money is based on the value of Pi. The account number is based on the mathematical constant e [3]. It s a pity that the author of that one didn t sign their real name. Whoever created that should have claimed credit for their work.

20 April 2009

Russell Coker: The Future of Electric Cars

TED published an interesting interview with Shai Agassi about electric cars [1]. One idea that I hadn t heard before is that of moving car batteries between regions as they lose capacity. An old battery for an electric car that can only handle short journeys may be useful in a region where journeys are typically short. On a similar note I expect that in a few decades the less prosperous countries will import old electric vehicles and fit them with 4 or more batteries. Last time I checked the Prius battery pack weighed about 120Kg, so the car would be usable with 4 battery packs if driven at low speeds. Shai Agassi also gave a TED talk on this topic [2]. The real solution for the problem of providing convenient and affordable electric vehicles is to start by recharging the batteries whenever the vehicle is parked (at the office, shopping center, home, etc). Then on the rare occasions when the car is being driven for longer distances and the battery gets flat it can be swapped for a charged battery. They have apparently designed a robot for changing car batteries, so changing the battery would be like driving through a car-wash. He describes this as an economic model that decouples the expensive battery from the car, so you pay for the use of the battery not the ownership - just as with a petrol car you pay for the petrol you use not for a portion of the ownership of an oil well. He also pointed out that cars produce 25% of the world s CO2 emissions, so his plan for all electric cars everywhere seems to be an essential part of solving the environmental problems. He then compared this to the UK parliamentary discussion on ending slavery, at the time slaves provided 25% of the energy used by the UK. After a month of discussion the decision was made to make the moral choice and end slavery regardless of the cost.

Bdale Garbee: TeleMetrum First Flight

Today was the season opener for Tripoli Colorado at their launch site on the buffalo ranch near Hartsel, Colorado. After huge snowfalls along the front range of the mountains in the last few days, we were a little tentative about going, but it turned out to be nearly perfect flying conditions! There was apparently much less snow this week on the high plains west of the front range, and by this morning the snow had almost entirely disappeared, the skies were clear and blue, and the winds were calm except for a few gusty bursts in the afternoon. This launch site is really something special, at 8800 feet above mean sea level, in the middle of a huge area of wide-open short grass prairie. We love flying there, and today the drive to and from the launch site through the snow-covered Colorado Rockies was just beautiful! Son Robert and I flew three rockets today, including his LOC/Precision Lil Nuke with added payload section on an Aerotech G54W-M reload, and our Polecat Aerospace 5.5 inch Goblin kit on one of the relatively new Aerotech I245G-M "Mojave Green" green-flame reloads. But by far the highlight of the day was flying my Giant Leap Vertical Assault on a Cesaroni J335 red-flame reload... with serial number 1 of TeleMetrum on board collecting our first-ever flight data! From the ground, it looked like a textbook perfect dual-deploy flight, with a small drogue chute out at apogee around 4000 feet above ground, and the main chute deploying at 700 feet above ground for a beautiful, soft landing only a couple minutes walk from the launch rail. The ejection events were controlled by a PerfectFlite MiniAlt/WD. The data recovered from it shows a big negative spike in the altitude right at apogee, coincident with firing of the apogee deployment charge. I have to assume this means the aft bulkhead on the avionics bay in the reconstructed coupler section isn't sealing well, and some pressure from the apogee deployment charge leaked in to the avionics bay "fooling" the altimeter into thinking the altitude was lower for a sample or two. Clearly, that needs to get fixed before that airframe flies again. Further evidence that we had a mighty kick from the apogee ejection charge was discovered when we went to clean the motor casing. The Cesaroni reloads are packaged in a plastic liner tube that slides into the reusable aluminum case. When we pulled the spent reload out of the case, it was significantly shorter than when it was loaded, suggesting that the ejection charge forced the forward closure on the reload backwards compressing the heat-softened plastic. This could be evidence that the reconstructed coupler was slow to separate from the booster airframe due to excessive friction? The flight data recovered from the TeleMetrum board looks great until apogee, when the data collection stopped. Since I flew firmware that was compiled and flashed on the flight line from Keith's latest git commit as of this morning, it's entirely possible that there was a software bug that caused data collection to terminate at apogee. We'll investigate that. But I personally think what actually happened is that we experienced a temporary short in the power supply at the time the apogee ejection charge fired. On extraction of the electronics sled from the avionics bay this evening, I noticed that one of the mounting screws has gone missing. If it wasn't snug enough, and vibrated loose during flight, it could have been torn loose at the time of the ejection event and shorted something as it rattled around in the avionics bay. The screw is now just missing, but may have fallen out when we were extracting data on the flight line just after the flight without being noticed at the time. So I'm not inclined to worry much about this, at least until we can get some more flight data! Keith post-processed the raw flight data and presented me with a plot showing two traces, acceleration and barometric altitude. The data from the accelerometer closely matches the published data for the motor we flew, which is a really cool result, and my 10-year-old son enjoyed figuring out why the rocket showed negative acceleration after the motor burn out but was still climbing. (See, there really is some science education hidden in the fun!) All in all, we had a great time, and it's totally cool to have data from a first flight of TeleMetrum! Can't wait to fly it again!

19 April 2009

Martin F. Krafft: Extending the X keyboard map with xkb

xmodmap has long been the only way to modify the keyboard map of the X server, short of the complex configuration daemon approaches used by the large desktop managers, like KDE and GNOME. But it has always been a hack: it modifies the X keyboard map and thus requires a baseline to work from, kind of like a patch needs the correct context to be applicable. Worse yet, xmodmap weirdness required me to invoke it twice to get the effect I wanted. When the recent upgrade to X.org 7.4 broke larger parts of my elaborate xmodmap configuration, I took the time to finally ditch xmodmap and implement my modifications as proper xkb configuration.

Background information I had tried before to use per-user xkb configuration, but could not find the answers I want. It was somewhat by chance that I found Doug Palmer s Unreliable Guide to XKB configuration at the same time that Julien Cristau and Matthew W. S. Bell provided me the necessary hints on the #xorg/irc.freenode.org IRC channel to get me started. The other resource worth mentioning is Ivan Pascal s collection of XKB documents, which were instrumental in my gaining an understanding of xkb. And just as I am writing this document, Debian s X Strike Force have published their Input Hotplug Guide, which is a nice complement to this very document you are reading right now, since it focuses on auto-configuration of xkb with HAL. The default xkb configuration comes with a lot of flexibility, and often you don t need anything else. But when you do, then this is how to do it:

Installing a new keyboard map The most basic way to install a new keyboard map is using xkbcomp, which can also be used to dump the currently installed map into a file. So, to get a bit of an idea of what we ll be dealing with, please run the following commands:
xkbcomp $DISPLAY xkb.dump
editor xkb.dump
xkbcomp xkb.dump $DISPLAY

The file is complex and large, and it completely went against my aesthetics to simply edit it to have xkb work according to my needs. I sought a way in which I could use as much as possible of the default configuration, and only place self-contained additional snippets in place to do the things I wanted done differently. setxkbmap and rule files Thus began my voyage into the domain of rule files. But before we dive into those, let s take a look at setxkbmap. Despite the trivial invocation of e.g. setxkbmap us to install a standard US-American keyboard map, the command also takes arguments. More specifically, it allows you to specify the following high-level parameters, which determine the sequence of events between key press and an application receiving a KeyPress event:
  • Model: the keyboard model, which defines which keys are where
  • Layout: the keyboard layout, which defines what the keys actually are
  • Variant: slight variantions in the layout
  • Options: configurable aspects of keyboard features and possibilities
Thus, with the following command line, I would select a US layout with international (dead) keys for my Thinkpad keyboard, and switch to an alternate symbol group with the windows keys (more on that later):
setxkbmap -model thinkpad -layout us -variant intl -option grp:win_switch

In many cases, between all combinations of the aforementioned parameters, this is all you ever need. But I wanted more. If you append -print to the above command, it will print the keymap it would install, rather than installing it:
% setxkbmap -model thinkpad -layout us -variant intl -option grp:win_switch -print
xkb_keymap  
  xkb_keycodes    include "evdev+aliases(qwerty)"        ;
  xkb_types       include "complete"     ;
  xkb_compat      include "complete"     ;
  xkb_symbols     include "pc+us(intl)+inet(evdev)+group(win_switch)"    ;
  xkb_geometry    include "thinkpad(us)"         ;
 ;

There are two things to note:
  1. The -option grp:win_switch argument has been turned into an additional include group(win_switch) on the xkb_symbols line, just like the model, layout, and variant are responsible for other aspects in the output.
  2. The output seems related to what xkbcomp dumped into the xkb.dump file we created earlier. Upon closer inspection, it turns out that the dump file is simply a pre-processed version of the keyboard map, with include instructions exploded.
At this point, it became clear to me that this was the correct way forward, and I started to investigate those points in order. The translation from parameters to an xkb_keymap stanza by setxkbmap is actually governed by a rule file. A rule is nothing more than a set of criteria, and what setxkbmap should do in case they all match. On a Debian system, you can find this file in /usr/share/X11/xkb/rules/evdev, and /usr/share/X11/xkb/rules/evdev.lst is a listing of all available parameter values. The xkb_symbols include line in the above xkb_keymap output is the result of the following rules in the first file, which setxkbmap had matched (from top to bottom) and processed:
! model         layout              =       symbols
  [...]
  *             *                   =       pc+%l(%v)
! model                             =       symbols
  *                                 =       +inet(evdev)
! option                            =       symbols
  [...]
  grp:win_switch                    =       +group(win_switch)

It should now not be hard to deduce the xkb_symbols include line quoted above, starting from the setxkbmap command line. I ll reproduce both for you for convenience:
setxkbmap -model thinkpad -layout us -variant intl -option grp:win_switch
xkb_symbols     include "pc+us(intl)+inet(evdev)+group(win_switch)"    ;

A short note about the syntax here: group(win_switch) in the symbols column simply references the xkb_symbols stanza named win_switch in the symbols file group (/usr/share/X11/xkb/symbols/group). Thus, the rules file maps parameters to sets of snippets to include, and the output of setxkbmap applies those rules to create the xkb_keymap output, to be processed by xkbcomp (which setxkbmap invokes implicitly, unless the -print argument was given on invocation). It seems that for a criteria (option, model, layout, ) to be honoured, it has to appear in the corresponding listing file, evdev.lst in this case. There is also evdev.xml, but I couldn t figure out its role.

Attaching symbols to keys I ended up creating a symbols file of reasonable size, which I won t discuss here. Instead, let s solve the following two tasks for the purpose of this document:
  1. Make the Win-Hyphen key combination generate an en dash ( ), and Win-Shift-Hyphen an em dash ( ).
  2. Let the Caps Lock key generate Mod4, which can be used e.g. to control the window manager.
To approach these two tasks, let s create a symbols file in ~/.xkb/symbols/xkbtest and add two stanzas to it:
partial alphanumeric_keys
xkb_symbols "dashes"  
  key <AE11>  
    symbols[Group2] = [ endash, emdash ]
   ;
 ;
partial modifier_keys
xkb_symbols "caps_mod4"  
  replace key <CAPS>  
    [ VoidSymbol, VoidSymbol ]
   ;
  modifier_map Mod4   <CAPS>  ;
 ;

Now let me explain these in turn:
  1. We used the option grp:win_switch earlier, which told xkb that we would like to use the windows keys to switch to group 2. In the custom symbols file, we now simply define the symbols to be generated for each key, when the second group has been selected. Key <AE11> is the hyphen key. To find out the names of all the other keys on your keyboard, you can use the following command:
    xkbprint -label name $DISPLAY -   gv -orientation=seascape -
    
    
    I had to declare the stanza partial because it is not a complete keyboard map, but can only be used to augment/modify other maps. I also declared it alphanumeric_keys to tell xkb that I would be modifying alphanumeric keys inside it. If I also wanted to change modifier keys, I would also specify modifier_keys. The rest should be straight-forward. You can get the names of available symbols from keysymdef.h (/usr/include/X11/keysymdef.h on a Debian system, package x11proto-core-dev), stripping the XK_ prefix.
  2. The second stanza replaces the Caps Lock key definition and prevents it from generating symbols (VoidSymbol). The important aspect of the second stanza is the modifier_map instruction, which causes the key to generate the Mod4 modifier event, which I can later use to bind key combinations for my window manager (awesome).
The easiest way to verify those changes is to put the setxkbmap -print output of the keyboard map you would like to use as a baseline into ~/.xkb/keymap/xkbtest, and append snippets to be included to the xkb_symbols line, e.g.:
"pc+us(intl)+inet(evdev)+group(win_switch)+xkbtest(dashes)+xkbtest(caps_mod4)"

When you try to load this keyboard map with xkbcomp, it will fail because it cannot find the xkbtest symbol definition file. You have to let the tool know where to look, by appending a path to its search list (note the use of $HOME instead of ~, which the shell would not expand):
xkbcomp -I$HOME/.xkb ~/.xkb/keymap/xkbtest $DISPLAY

You can use xev to verify the results, or just type Win-Hyphen into a terminal; does it produce ? By the way, I found xev much more useful for such purposes when invoked as follows (thanks to Penny for the idea):
xev   sed -ne '/^KeyPress/,/^$/p'

Unfortunately, xev does not give any indication of which modifier symbols are generated. I have found no other way to verify the outcome, other than to tell my window manager to do something in response to e.g. Mod4-Enter, reloaded it, and then tried it out.

Rules again, and why I did not use them in the end Once I got this far, I proceeded to add option-to-symbol-snippet mappings to the rules file, and added each option to the listing file too. A few bugs [[!debbugs 524512 desc=later]], I finally had setxkbmap spit out the right xkb_keymap and could install the new keyboard map with xkbcomp, like so:
setxkbmap -I$HOME/.xkb [...] -print   xkbcomp -I$HOME/xkb - :0

I wrote a small script to automatically do that at the start of the X session and could have gone to play outside, if it hadn t been for the itch I felt due to the entire rule file stored in my configuration. I certainly did not like that, but I could also not find a way to extend a rule file with additional rules. When I looked at the aforementioned script again, it suddenly became obvious that I was going a far longer path than I had to. Even though the rule system is powerful and allows me to e.g. automatically include symbol maps to remap keys on my Thinkpad, based on the keyboard model I configured, the benefit (if any) did not justify the additional complexity. In the end, I simplified the script that loads the keyboard map, and defined a default xkb_keymap, as well as one for the Thinkpad, wich I identify by its fully-qualified hostname. If a specific file is available for a given host, it is used. Otherwise, the script uses the default.

18 April 2009

Wouter Verhelst: Using NBD to get rid of an LVM installation.

LVM can be a virtue. However, it can also be a pain at times; for instance, there's loss of diskspace due to LVM's metadata that needs to be stored as well. This amounts to approximately 5% of the diskspace (if I'm not mistaken), which can be quite a bit. Also, this is an extra abstraction layer, which requires memory and CPU time to be processed. This will be negligable on most modern computers, but sometimes it isn't. I had a system that had quite a few things on LVM, and wanted to get rid of them. Unfortunately, to remove the LVM physical volumes would mean that I'd have to get rid of the data, which would mean storing it somewhere else, which in turn would probably mean that I wouldn't be able to use the system for quite a long time. That wasn't acceptable. So, instead, what I did was to export a whole hard disk (one with enough space to hold the logical volumes on that machine) using NBD, and migrate stuff that way: This final step might not be as simple as it sounds; e.g., if your /home filesystem is on LVM currently and you want to move that out of LVM, you'll have to make sure only root is logged on; If /var is on LVM, too, you may have to go to single-user; etc. Figuring out the details for this one is left as an exercise to the reader. However, I will note that since somewhere between etch and lenny, doing root-on-NBD is possible; and if you're running unstable, then doing root-on-LVM-on-NBD should be possible, too, although I would suggest a little caution before trying that. In any case, using NBD in this way just saved me a whole lot of no-computer-time here. Which is great.

16 April 2009

Erich Schubert: Finding an web editor widget

I've previously mentioned my plans on redoing my blog. Well, I've settled down on some design issues already (posts will be stored as mini Atom feeds, which makes the generation of Atom feeds for the blog and categories trivial, and gives me maximum flexibility. I already have a working converter for my existing blog to Atom posts.)Generating static HTML pages from that will be easily possible using an XSTL transformation (for example), and I got the feedback that I could just use Google AppEngine for blog comments, so my actual blog could remain static-only (and thus much more secure and reliable). Any attack/spammer/bot can then only kill the comment functionality, not my own site.Which brings me to another design consideration: the editing widget. Either for the blog comment application, for writing my own blog entries (via a https protected script or whatever) or maybe for a small CMS I've been thinking about - having a reliable HTML in-browser editing widget is something I could use every now and then (well, I'm not doing much Web stuff anymore these days).Geniisoft has a good overview over in-browser (aka: Through The Web, TTW) editors. The top candidates seem to be: I've heard before of FCKeditor and TinyMCE; I think I've been on the Xinha page before, too. However, comments on them have not always been good.To some extend they all seem to have (to some extend) feature creep which usually is a bad sign - most often this means that there are security issues in one or another module or plugin.TinyMCE for example has been described as "a bit of a pain" and "a tad clumsy" on the GSoC mailing list. I have had less fights with it than with the Debian Wikis (MoinMoin) markup language though.I'm not looking for anything as big as these - I just need an editor that allows for some basic formatting (bold etc., links) and that produces reasonable XHTML output. I'll be feeding the output through some custom cleanup script anyway, which will kill disallowed code. So I don't want any editor which allows the user to create code that will then be killed afterwards.Any personal experiences with any of these, or an important alternative that I might have missed (no PHP involved, please!) - email me (no comments on blog) at erich AT debian org.Update: I've received a couple of pointers. Please don't send me links to projects that are not actively maintained anymore - I don't want to care about having to fix bugs in the editor widget myself.One link I've received twice is actually quite impressive: WYMeditor. It doesn't try to look like a word processor, but actually is more of a semantic editor. Much more what I'm looking for than any of the others. I've also received a link to the Yahoo! UI Library Editor, which is quite clutter-free, but in the default setup at least very text-formatting oriented, not very semantic (that doesn't mean you couldn't change it that way, I guess you can). I was also pointed to Dojo, but that framework is totally feature creep (which also explains why it loads so slowly I guess), and the last time I looked at it's source code, I had some WTF moments - code quality at least outside of core doesn't seem to be very high (Yes, that code implements the "mod 7" operation using list shifts instead of a simple arithmetic operation).Looks like I'll give the first try to WYMeditor. Update #2: the code seems to be rather ... complex. I'm looking for something neat and clean; it doesn't need to bring along yet another XHTML schema and validator ... Maybe I should try one of the others first?

Stefano Zacchiroli: happy term Steve

DPL candidacy woes Well, time to face it publicly: I lost and Steve/Luk won, it is as simple as that :-) This post is to: All in all, standing for DPL has been worth and interesting no matter the result. Time to get back to my usual Debian duties now ...

Martin F. Krafft: Government exaction

Do you listen to the radio? I don t, and there are multiple reasons for that: First and foremost, I don t like the music they play these days. Even though there are stations which will occasionally play a more eclectic tune, the Swiss government does not, to my knowledge, dictate any quota, like New Zealand actually requires stations to broadcast New Zealand music at least 20% of the time. The result is music made by producers, who stand pop icons with little clothing in front of microphones, instruct them to move their bodies in obscene ways, and coach them to croak songs someone else wrote en masse in some basement somewhere. All those songs sound alike. To ensure a constant supply of pop dolls, they hype television shows with titles such as Germany s seeks the next superstar , when most of Germany actually couldn t care less, and only those with vacuous brain capacity follow with excitement, because they wouldn t know what else to do. Second, I don t have the time or energy to listen to discussion rounds on the radio, especially not when topic are about society , which noone can define in the first place. Wittgenstein always had it right. The same goes for most commentaries. If I don t have anything to do, I prefer to listen to birds in the trees, rather than someone s polished opinion. The scariest aspect is that those people actually believe that they have something worthwhile to say, and so do their producers and the other staff, when in most cases, they don t. At least not worthwhile to people who have other things occupying their minds. I also don t like someone to make a pre-selection of news and squash that into 5-15 minute time slots, to be read out by dull-sounding speakers. I d rather skim over articles, and the Web ensures that I get the information I want much quicker than the radio stations can transmit them. And sports over the radio waves makes me come close to aural epileptic seizures. I am painting the sky black a bit. I am fully aware that there exist people with thought-provoking arguments, that some stations play interesting music I would otherwise never discover, and that the radio is a source of entertainment in the lives of thousands of other people who are less privileged than me. But when the Billag rang my doorbell today and wanted CHF 14,10 per month for posession of any device capable of turning radio waves into sound, including those radio waves broadcast over the Internet these days, I got a bit furious. In the past, they concentrated only on actual radio and TV receivers. When I moved into my previous flat in Zurich in 2002, they wanted money because my stereo had a tuner button. I refused to pay, opened the device, unsoldered the radio coil, and sent it to them with an angry letter of refusal; they let me off the hook. Since then, they changed the law in 2007 (but haven t actually updated the forms they force you to sign) to include audio players, cellular phones, and computers, which are all radio-capable. I d really prefer if they used their funds to revert that trend, or at least remove the loudspeakers from all mobile phones. But instead, this morning, I skid right into their claws. As I said before, the radio is also a social support system, and as such I have less of a problem to subsidise it. However, given that I strongly disagree with the quality level broadcast, I would prefer to direct my money towards causes I consider more useful, and with which I could better identify. Unfortunately, the whole system is governmentalised to the point where this choice is taken away from me. The Billag website helpfully offers answers to the common questions, including one along the lines of quality. They suggest that you get in touch directly with the programme directors to complain about their content, and offer suggestions. Shortly after I moved to Zurich, Radio Lora offered me a radio show. Radio Lora is a private station, but they also receive subsidies from the monies exacted by the government, so I am using them as an example here. I suggested that I concentrated only on progressive and psychedelic rock from the sixties and the seventies. They refused the show in the end, claiming that this would not be what their listeners want. So instead I now get to pay for content that is deemed more appropriate. I can only imagine the pains I d have to go through to be able to host a radio show concentrating on eclectic tunes on one of the public radio stations. And even if I succeeded (without becoming a full-time public radio broadcaster), then I could only contribute a few hours a week to that content. If you ask me, the Swiss government is overstepping its competencies a bit. NP: Mono: Hymn to the Immortal Wind

15 April 2009

Julian Andres Klode: python-apt to become first Debian package to support Python 3


Python 3.1 is still in experimental, but python-apt already has a fully working Python 3 version. With the patch being available in the jak branch, python-apt Bug#523645 can be closed soon. And this is not the only change. Memory usage has been decreased by 10MB by creating the Package() objects on the fly instead of pre-creating all 25000 ones. All classes which previously supported the has_key() method now support __contains__, which allows you to write key in mapping . But that s not all. The Package class now gained support for setting the candidate version of a package, in the same way you get it, using the property: mypkg.candidate = max(mypkg.versions). And you can now pass file descriptors to functions previously only working with file() objects. But we re not done yet. The complete API change is pending, renaming everything to PEP-8 compliant names and deprecating the old ones, which will be removed at a later time (after the release of Squeeze). The module apt.debfile is not affected by this change, because it already used the correct names. But for Python 3, we will not keep backward compatibility, in order to make coding easier and to make it clear that applications should be ported to the new API. And if you want to get your feature into python-apt now is your time. If you want something merged, provide the code, the documentation (with example) and a list of use cases; and report a wishlist bug in the Debian BTS. And if you want to write a tutorial about python-apt, write it in reStructuredText and I will add it to the documentation. The documentation is currently basically an API reference and there could be much more text. Python-apt 0.7.90 is currently uploading. The whole development will result in python-apt 0.8. Posted in Debian

Stefano Zacchiroli: xmonad + gnome on Debian

... finally I'm tiling too! Short story: I've just switched to xmonad as my window manager of choice within a Gnome desktop environment; I give some tips on how to achieve that quickly on a current Debian/unstable machine. /me and tiling window managers I switched to Gnome a long time ago. The benefit of that over good old window managers is enjoying an integrated environment and a workplace that you can find on other machines (e.g., the netbook you installed for your mother, the office desktop, the student lab machines, ...). Still, I've never been particularly happy about Gnome's default window manager: metacity. The feature I was missing the most is the ability to configure windows programmatically so that they appear where and how I want by default; sadly enough, the Gnome session mechanism is not yet able to do that properly. Hence, I've tried various time switching to a tiling window manager, imposing the extra requirement that it should integrate with Gnome. The coolest guy on the block of that area seems to be awesome and I gave it a try some months ago. I liked most of its (sometime revolutionary) principles, like window tagging, but I equally hated various aspects of it: People nowadays tell me that awesome works fine with Gnome, so maybe I've just been unlucky back then, but sometime the first impression does count. I've never considered seriously xmonad, thinking (with no sound reason, apparently) that it was a project of the series: "I'm a fan of programming language $LANG, all my apps should be written in $LANG, my window manager is no exception", with LANG=haskell. Then two things happened:
  1. a couple of weeks ago I stumbled upon the xmonad wiki and documentation, and I was impressed by the average quality of the information shipped there. It is not something you usually find in "language showcase" projects.
  2. Patrick blogged about its switch to xmonad + Gnome, with enthusiastic feedback. If that's not a sign I don't know what it can be :-)
Getting started with xmonad + Gnome on Debian There is a page explaining how to use xmonad with Gnome in the xmonad wiki, but a handful of additional tips will help you get started in Debian, even if you know (almost) nothing about Haskell (yet). The (crazy) way in which xmonad works is that you create your config file as ~/.xmonad/xmonad.hs. That file is a full fledged Haskell program, which gets compiled to obtain your own xmonad executable which you run as your own window manager. /usr/bin/xmonad is the executable you initially run, but is just a wrapper around the compilation + run mechanism The modMask setting means that ALT will get delivered to applications, while all xmonad commands are delivered via the infamous windows key: no more conflicts between apps and WM \o/. The default Gnome configuration is quite nice: you can retain your Nautilus desktop (in spite of what is written on the wiki), mod-p will get you a gnome-run dialog box, and appropriate padding for the (upper!) Gnome panel. The only glitch is that you will get 9 desktops which can clutter your workspace switched. My solution is to have the appler show workspaces on 2 rows but beware of bug 524065, which is really annoying. The above example will make the pidgin buddy list and run application dialog box float by default. The commented line is an example of how to deliver specific windows to arbitrary workspaces. To find out window properties on which match in your xmonad.hs, my magic command is xprop egrep "CLASS NAME". Note that you don't need to care about panel placement because it is already taken care of by the default gnome config. Still beware of not using myManageHook as your only management hook, you should update the default one using the <+> monad as I did above. xmonad rant So, what's wrong with xmonad + Gnome in Debian? Well, the fact that it will take about 200 Mb of disk space on amd64 by default. o_O Why so? Because xmonad is affected by the static linking hell which seems to be shared by modern statically typed functional languages, including in the category both Haskell and my beloved OCaml. In short, applications written for those language link statically code written in the implementation language in most cases; they rely on legacy shared libraries only for bindings to C libraries. For xmonad that means that if you are fine with the default configuration you will get a cheap 2 Mb binary package. If you are not, you will need to write your conffile ... which is a Haskell program ... using several contrib libraries ... and needing ghc6 to be compiled and run as your own window manager executable. This is plainly dumb and induces all the usual problems of static linking, like disk consumption due to non-shared (Haskell) code, and worries about staying up to date with the latest library release in case of fixes (including security fixes) need to be propagated to user executables. Truth to be said, /usr/bin/xmonad does a wonderful job in hiding all this to the final user, but still the design choice is an unpleasant one.

Uwe Hermann: Resolving mysterious kernel/firmware problems via apt-get install firmware-linux

If you recently upgraded your kernel to the 2.6.29 Debian package, you might have noticed some (e.g. graphics) drivers stopped working or are working slower. In my case, this was the radeon driver, which inexplicably seemed to cause lots of slowdowns in some applications and games. A quick look into dmesg revealed the reason:
  [drm] Initialized radeon 1.29.0 20080528 on minor 0
  agpgart-intel 0000:00:00.0: AGP 2.0 bridge
  agpgart-intel 0000:00:00.0: putting AGP V2 device into 4x mode
  pci 0000:01:00.0: putting AGP V2 device into 4x mode
  [drm] Setting GART location based on new memory map
  [drm] Loading R200 Microcode
  platform radeon_cp.0: firmware: requesting radeon/R200_cp.bin
  radeon_cp: Failed to load firmware "radeon/R200_cp.bin"
  [drm:radeon_do_init_cp] *ERROR* Failed to load firmware!
As noted in the changelog file, the radeon firmware R200_cp.bin has been removed from the kernel, and is now available in the separate firmware-linux Debian package. So the simple fix for this issues is:
  $ apt-get install firmware-linux
  $ dpkg -L firmware-linux   grep R200_cp.bin
  /lib/firmware/radeon/R200_cp.bin
After restarting X, the dmesg output looks more sane again:
  agpgart-intel 0000:00:00.0: AGP 2.0 bridge
  agpgart-intel 0000:00:00.0: putting AGP V2 device into 4x mode
  pci 0000:01:00.0: putting AGP V2 device into 4x mode
  [drm] Setting GART location based on new memory map
  [drm] Loading R200 Microcode
  platform radeon_cp.0: firmware: requesting radeon/R200_cp.bin
  [drm] writeback test succeeded in 2 usecs

14 April 2009

Mike Hommey: Efficient use of mod_rewrite, part 1

Consider the following mod_rewrite configuration :
RewriteCond % REQUEST_URI !^/FR/fr
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ /FR/fr$1 [PT,L]
The need here, is to get anything that wouldn t begin with /FR/fr from $DOCROOT/FR/fr (through, possibly, any other module rules, which was important in the setup where these rules were), except if they begin with /common/ or is /favicon.ico. Depending on the requests pattern, reordering the RewriteConds can be better. Anyways, the case at hand was to massively duplicate this setup over different countries and languages. Basically, domain.fr/foo/bar would need to come from $DOCROOT/FR/fr/foo/bar, domain.co.uk/foo/bar from $DOCROOT/GB/en/foo/bar, while anydomain/common/foo/bar would come from $DOCROOT/common/foo/bar. In such a case, you have a lot of possible solutions, each of which has its pros and cons. The main problem with mod_rewrite is that there is no way to use variables or back references in the test patterns. In our case, we d like to be able to do something like this:
RewriteMap l10n txt:l10n.map # switch to dbm when necessary
RewriteCond % REQUEST_URI !^$ l10n:% SERVER_NAME
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ $ l10n:% SERVER_NAME $1 [PT,L]
Or
RewriteMap l10n txt:l10n.map # switch to dbm when necessary
RewriteCond $ l10n:% SERVER_NAME ^(.+)$
RewriteCond % REQUEST_URI !^%1
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ %1$1 [PT,L]
The map file would have on each line a domain name followed by the corresponding url start (/FR/fr for domain.fr, etc.). Unfortunately, the above setup is not possible. A way around this lack of functionnality is to use some nice perl regexp trick.
RewriteMap l10n txt:l10n.map # switch to dbm when necessary
RewriteCond $ l10n:% SERVER_NAME ^(.+)$
RewriteCond %1% REQUEST_URI !^(.+)\1
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ %1$1 [PT,L]
As you can see, only the second RewriteCond differs. What we really want to test is whether the % REQUEST_URI begins with the proper url start or not. Let s say we re considering domain.fr and the map gave us /FR/fr. Following the first RewriteCond, %1 contains /FR/fr. At the second RewriteCond, if % REQUEST_URI begins with /FR/fr then %1% REQUEST_URI begins with /FR/fr/FR/fr. Otherwise, it will only begin with /FR/fr. What we can test, then, is whether this %1% REQUEST_URI aggregate contains a repeating pattern at its beginning. This is exactly what the perl regexp does: it captures at least one character at the beginning of the tested string (^(.+)), and wants to find this captured string again (\1). It is worth mentioning that this obviously falls flat when the url start itself contains a repeating pattern (e.g. /fr/fr instead of /FR/fr). As we want the RewriteRule to work only when our % REQUEST_URI does not begin with /FR/fr, we negate the regexp, which means nothing will actually be captured, such that %1 in the RewriteRule will still be the last captured text, from the very first RewriteCond. Note that if all the tested url starts are the same length and/or pattern, it may be worth changing the regexp to be more precise (and faster to match or not). Such as ^(/../..)\1 in our present case.

Michael Schutte: Chain of unfortunate events

I did some work on kbd today. I basically took an old Debian patch, the only significant divergence from the original tarball, and altered it in a way that I can read and understand it more easily (so I can send it upstream in good conscience). This process naturally involved a lot of errors. As I m talking about loadkeys here, they became unpleasantly obvious whenever I tested my code: First I messed up everything but plain printable ASCII characters (so the return key was mapped to Unicode U+0201 instead of the correct K(0x02, 0x01)), then I managed to completely freeze everything. In both cases, Alt+F7 was affected by the damage. Just like Ctrl+Alt+Del. Time for the good old boring elephants. I learn from mistakes when I make them twice, so a while sleep 60; do chvt 7; done loop saved me from further reboots. I nuked /dev/tty1 once more (in the really cool, irreversible way that disables all keypresses), but I could move on to the next VT. After a final adjustment, my modified patch finally worked. I sent it upstream, forgot about the troublesome debugging and went on to something else. Later in the evening, I decided to end the last school holidays in my life with a bit of instant messaging. BitlBee, a Jabber IRC gateway, is running on my virtual private server for this purpose. Being the occasional chatter that I am, I haven t touched it since the recent upgrade to Lenny. The format of its account configuration files has apparently changed, so I started to set up BitlBee anew. It kept telling me that there is an Authentication failure ; given the peculiar setup of the XMPP server I use, this didn t really surprise me. But still, this thing had worked before. Perhaps I really got my password wrong? I haven t typed it for ages, BitlBee stored it for me. As the administrator of my Jabber server, I could easily reset my password to what I thought it was. BitlBee still refused to get past the identification stage. I decided to install Gajim, entered my account information and told it to connect. Gnah. One or the other Python exception caused an eternal stream of error dialogue boxes to appear simultaneously and steal my focus. I don t have a keyboard shortcut for xkill and I couldn t type anything in a Urxvt (error message #54321 would receive the keystrokes). Ctrl+Alt+F1, get me out of here! Oh, right, I broke that one! At that point, I burst into a laugh. Somewhere in this post, there are a few bugs to report.

Christian Perrier: Lost tapping in X.org?

In case you last tapping with your touchpad with the latest X.org update in sid, you might want to read bug #497523 or just add the following to your xorg.conf file:
        Option          "TapButton1"            "1"
        Option          "TapButton2"            "2"
        Option          "TapButton3"            "3"
in the InputDevice section that's relevant for your touchpad. ...and good luck exploring the joy of X.org updates (I am currently back in time fighting again with a jerky and unreliable Suspend-to-RAM which sometimes freezes my display...or just exits the KDE session...or sometimes just works....).

12 April 2009

Biella Coleman: Memefactory, the video

I have found myself, at least on a few occasions, talking to a person whose relationship to the Internets is much thinner than my own, trying to explain the nature of certain Internet memes. I find this a pretty tough task to pull off as I tend to make memes sound really infantile (and perhaps that is just what they are). To explain this world to the uninitiated, I either: 1) very quickly give up and move onto some other less obscure 2) go into semi-professorial lecture mode and give a mini-low down on memes, some of the early examples, a small theory of self-referential irony, 4chan, rickrolls, encyclopedia dramatic etc etc and then show a bunch of memes as really you can t do it justice without the audio visul component. Now, I can just point people to the Meme Factory to three youngish guys from Brooklyn who put together a **fantastic show**** a few weeks ago on, yes, Internet Memes. It was a dazzling ride into the (sometimes very obnoxious) territory that is memeology, addressing and displaying both the happy-cheery-cute side of memes as well as its darker side. The video they have on their site documents their presentation and the word is out that they will soon craft something more like a stand-alone video-essay about Internet memes. As we wait for this next step in their project, this video is an amusing watch (though the interesting stuff starts about 3 minutes in).

Biella Coleman: Memefactory, the video

I have found myself, at least on a few occasions, talking to a person whose relationship to the Internets is much thinner than my own, trying to explain the nature of certain Internet memes. I find this a pretty tough task to pull off as I tend to make memes sound really infantile (and perhaps that is just what they are). To explain this world to the uninitiated, I either: 1) very quickly give up and move onto some other less obscure 2) go into semi-professorial lecture mode and give a mini-low down on memes, some of the early examples, a small theory of self-referential irony, 4chan, rickrolls, encyclopedia dramatic etc etc and then show a bunch of memes as really you can t do it justice without the audio visul component. Now, I can just point people to the Meme Factory to three youngish guys from Brooklyn who put together a **fantastic show**** a few weeks ago on, yes, Internet Memes. It was a dazzling ride into the (sometimes very obnoxious) territory that is memeology, addressing and displaying both the happy-cheery-cute side of memes as well as its darker side. The video they have on their site documents their presentation and the word is out that they will soon craft something more like a stand-alone video-essay about Internet memes. As we wait for this next step in their project, this video is an amusing watch (though the interesting stuff starts about 3 minutes in).

Sean Finney: a productive weekend for debian stuff

(note that i've just tried enabling the comments feature of ikiwiki, so feel free to use it :) ) (update: okay so i apparently failed to enable comments earlier... but it should be working now :) ) compiz(-fusion) 0.8.2 uploaded to unstable for all you lovers of desktop bling, the latest stable release of compiz-fusion has been uploaded to unstable. new features of note: new one-off script for building compiz packages the biggest problem with trying to maintain compiz packages is that they're for some reason distributed as almost a dozen different source projects upstream, but extremely interdependant wrt their API. therefore updating the packages to a new upstream version is a Big Pain. i've numbed that pain a bit with a new helper script, which should amongst other things make it easier to manage backports to lenny, which i've already seen people requesting. compiz 0.8.2 lenny packages available for testing ..which brings us to our next topic. before i upload them to somewhere a little more official, i figure i could put a call out to teh lazywebs to do some QA work and make sure that it doesn't totally explode on your system (unless you're using the explode animation plugin, in which case it's functioning properly) sources.list entry to try it out on an amd64 lenny system:
deb http://people.debian.org/~seanius/compiz/lenny-backports/amd64 ./

sources.list entry to try it out on an i386 lenny system:
deb http://people.debian.org/~seanius/compiz/lenny-backports/i386 ./

and then apt-get install compiz compiz-fusion-plugins-main (etc) so give it a try and let me know how it goes! pkg-php moving to git with some initial legwork from Mark Hershberger we've migrated to a git repository for the php packaging vcs. due to the long and convoluted history of our svn repository, we decided it wasn't worth the effort of trying to get every single svn commit mapped into the new repo. as a compromise to having to start from scratch, we used the existing branches/tags on top of a series of upstream tarball imports to get an accurate representation of the release history. new git<->bts integration hook for debian packaging i've been dabbling with a new hook to make for more useful integration from a git-buildpackage oriented workflow. in this workflow the changelog is often the last thing prepared, possibly days or even weeks after a fix has been committed for a bug. therefore the standard tagpending and typical changelog-scanning hooks aren't incredibly useful. instead, this hook scans for "Closes:" meta-info in the commits, similar to how "git dch" does, and then sends a notification and/or control commands to the bts. here's an example. note that it also mentions the branch that recieved the fix in the notification, so it's easier to see which branches have a fix at a quick glance. how to use this for your own git repo:
REPO_PATH=/path/to/your/repo.git
HOOK_REPO_PATH=/somewhere/you/want/to/put/it
git clone git://git.debian.org/users/seanius/vcs-hooks/git-hooks.git $HOOK_REPO_PATH
ln -sf $HOOK_REPO_PATH/git-post-receive-url-notifications.py $REPO_PATH/hooks/post-receive

there's a number of configurable options (take a look at the top of the file for some fairly verbose comments), but it should work with some fairly reasonable defaults out of the box. it can also be configured to do the more traditional changelog-scanning, but i'm finding it to be a better workflow to avoid combining changelog entries with the actual fixes (makes them harder to cherry-pick later), and a notification with a link to the changelog entry really isn't that useful beyond what "bts tag nnn pending" can already do. plus i don't like the typical duplication involved in manually managing the changelog. i find it better to generate the changelog via git-dch and automatically get all the "closes:" tags, and then just do a bit of editorial touchups before preparing/releasing the upload. so give it a try if you like, feel free to send feedback/fixes/etc. for example i think the utf-8 support might still be a bit dodgy, as well as possibly the automatic gitweb url detection and changelog scanning.

Next.

Previous.