Biella Coleman: A name
An oldie but goodie. I certainly enjoy watching students squirm their way into naming me.
So in three weeks time, two once large Unix vendors are gone SGI on april 1st, and now Sun. SGI as a Unix vendor had been dead for years anyway, but still.
Raphael Bauduin emailed me to say that he s organising an event for Profoss: (28 april in Brussels) will take a look at legal aspects of the use of FOSS in professional environments. http://www.profoss.eu/events
It will bring interesting information for most ICT professionals, even for those specialised in FOSS, but if this is not for you, it might be of interest to people around you (your boss or colleagues), so don t hesitate to spread the information around you.
There is problem with the current website: it requires javascript. An updated site without that problem will appear very soon .
If you go to the event, I d love to hear how it goes - please leave a comment on this article!
*g*
xmodmap has long
been the only way to modify the keyboard map of the X server, short
of the complex configuration daemon approaches used by the large
desktop managers, like KDE and GNOME. But it has always been a
hack: it modifies the X keyboard map and thus requires a baseline
to work from, kind of like a patch needs the correct context to be
applicable.
Worse yet, xmodmap weirdness required me to
invoke it twice to get the effect I wanted.
When the recent upgrade to X.org 7.4 broke larger
parts of
my elaborate xmodmap configuration, I took the time to finally
ditch xmodmap and implement my modifications as proper
xkb configuration.
#xorg/irc.freenode.org IRC channel to get
me started.
The other resource worth mentioning is Ivan Pascal s collection of XKB
documents, which were instrumental in my gaining an
understanding of xkb.
And just as I am writing this document, Debian s X Strike Force
have published their Input
Hotplug Guide, which is a nice complement to this very document
you are reading right now, since it focuses on auto-configuration
of xkb with HAL. The default
xkb configuration comes with a lot of flexibility, and
often you don t need anything else.
But when you do, then this is how to do it:
xkbcomp, which can also be used to dump the currently
installed map into a file. So, to get a bit of an idea of what
we ll be dealing with, please run the following commands:
xkbcomp $DISPLAY xkb.dump
editor xkb.dump
xkbcomp xkb.dump $DISPLAY
The file is complex and large, and it completely went against my
aesthetics to simply edit it to have xkb work
according to my needs. I sought a way in which I could use as much
as possible of the default configuration, and only place
self-contained additional snippets in place to do the things I
wanted done differently.
setxkbmap and rule files
Thus began my voyage into the domain of rule files. But before
we dive into those, let s take a look at setxkbmap.
Despite the trivial invocation of e.g. setxkbmap us to
install a standard US-American keyboard map, the command also takes
arguments. More specifically, it allows you to specify the
following high-level parameters, which determine the sequence of
events between key press and an application receiving a KeyPress
event:
setxkbmap -model thinkpad -layout us -variant intl -option grp:win_switch
In many cases, between all combinations of the aforementioned
parameters, this is all you ever need.
But I wanted more.
If you append -print to the above command, it will
print the keymap it would install, rather than installing it:
% setxkbmap -model thinkpad -layout us -variant intl -option grp:win_switch -print
xkb_keymap
xkb_keycodes include "evdev+aliases(qwerty)" ;
xkb_types include "complete" ;
xkb_compat include "complete" ;
xkb_symbols include "pc+us(intl)+inet(evdev)+group(win_switch)" ;
xkb_geometry include "thinkpad(us)" ;
;
There are two things to note:
-option grp:win_switch argument has been turned
into an additional include group(win_switch) on the
xkb_symbols line, just like the model, layout, and
variant are responsible for other aspects in the output.
xkbcomp dumped
into the xkb.dump file we created earlier. Upon closer
inspection, it turns out that the dump file is simply a
pre-processed version of the keyboard map, with
include instructions exploded.
xkb_keymap
stanza by setxkbmap is actually governed by a rule
file. A rule is nothing more than a set of criteria, and what
setxkbmap should do in case they all match. On a
Debian system, you can find this file in
/usr/share/X11/xkb/rules/evdev, and
/usr/share/X11/xkb/rules/evdev.lst is a listing of all
available parameter values.
The xkb_symbols include line in the above
xkb_keymap output is the result of the following rules
in the first file, which setxkbmap had matched (from
top to bottom) and processed:
! model layout = symbols
[...]
* * = pc+%l(%v)
! model = symbols
* = +inet(evdev)
! option = symbols
[...]
grp:win_switch = +group(win_switch)
It should now not be hard to deduce the xkb_symbols
include line quoted above, starting from the setxkbmap
command line. I ll reproduce both for you for convenience:
setxkbmap -model thinkpad -layout us -variant intl -option grp:win_switch
xkb_symbols include "pc+us(intl)+inet(evdev)+group(win_switch)" ;
A short note about the syntax here:
group(win_switch) in the symbols column simply
references the xkb_symbols stanza named
win_switch in the symbols file group
(/usr/share/X11/xkb/symbols/group).
Thus, the rules file maps parameters to sets of snippets to
include, and the output of setxkbmap applies those
rules to create the xkb_keymap output, to be processed
by xkbcomp (which setxkbmap invokes
implicitly, unless the -print argument was given on
invocation).
It seems that for a criteria (option, model, layout, ) to be
honoured, it has to appear in the corresponding listing file,
evdev.lst in this case. There is also
evdev.xml, but I couldn t figure out its role.
Mod4, which can be
used e.g. to control the window manager.
~/.xkb/symbols/xkbtest and add two stanzas to it:
partial alphanumeric_keys
xkb_symbols "dashes"
key <AE11>
symbols[Group2] = [ endash, emdash ]
;
;
partial modifier_keys
xkb_symbols "caps_mod4"
replace key <CAPS>
[ VoidSymbol, VoidSymbol ]
;
modifier_map Mod4 <CAPS> ;
;
Now let me explain these in turn:
grp:win_switch earlier, which
told xkb that we would like to use the windows keys to
switch to group 2. In the custom symbols file, we now simply define
the symbols to be generated for each key, when the second group has
been selected.
Key <AE11> is the hyphen key. To find out the
names of all the other keys on your keyboard, you can use the
following command:
xkbprint -label name $DISPLAY - gv -orientation=seascape -
I had to declare the stanza partial because it is
not a complete keyboard map, but can only be used to augment/modify
other maps. I also declared it alphanumeric_keys to
tell xkb that I would be modifying alphanumeric keys
inside it. If I also wanted to change modifier keys, I would
also specify modifier_keys.
The rest should be straight-forward. You can get the names of
available symbols from keysymdef.h
(/usr/include/X11/keysymdef.h on a Debian system,
package x11proto-core-dev),
stripping the XK_ prefix.
VoidSymbol).
The important aspect of the second stanza is the
modifier_map instruction, which causes the key to
generate the Mod4 modifier event, which I can later
use to bind key combinations for my window manager (awesome).
setxkbmap -print output of the keyboard map you would
like to use as a baseline into ~/.xkb/keymap/xkbtest,
and append snippets to be included to the xkb_symbols
line, e.g.:
"pc+us(intl)+inet(evdev)+group(win_switch)+xkbtest(dashes)+xkbtest(caps_mod4)"
When you try to load this keyboard map with
xkbcomp, it will fail because it cannot find the
xkbtest symbol definition file. You have to let the
tool know where to look, by appending a path to its search list
(note the use of $HOME instead of ~,
which the shell would not
expand):
xkbcomp -I$HOME/.xkb ~/.xkb/keymap/xkbtest $DISPLAY
You can use xev to verify the results, or just type
Win-Hyphen into a terminal; does it produce ?
By the way, I found xev much more useful for such
purposes when invoked as follows (thanks to Penny for the idea):
xev sed -ne '/^KeyPress/,/^$/p'
Unfortunately, xev does not give any indication of
which modifier symbols are generated. I have found no other way to
verify the outcome, other than to tell my window manager to do
something in response to e.g. Mod4-Enter, reloaded it, and then
tried it out.
setxkbmap spit out the
right xkb_keymap and could install the new keyboard
map with xkbcomp, like so:
setxkbmap -I$HOME/.xkb [...] -print xkbcomp -I$HOME/xkb - :0
I wrote
a small script to automatically do that at the start of the X
session and could have gone to play outside, if it hadn t been
for the itch I felt due to the entire rule file stored in my
configuration. I certainly did not like that, but I could also not
find a way to extend a rule file with additional rules.
When I looked at the aforementioned script again, it suddenly
became obvious that I was going a far longer path than I had to.
Even though the rule system is powerful and allows me to e.g.
automatically include symbol maps to remap keys on my Thinkpad,
based on the keyboard model I configured, the benefit (if any) did
not justify the additional complexity.
In the end, I simplified
the script that loads the keyboard map, and defined
a default xkb_keymap, as well as
one for the Thinkpad, wich I identify by its fully-qualified
hostname. If a specific file is available for a given host, it is
used. Otherwise, the script uses the default.
This is a continuation from
before.
Before I go plunging into writing code for a generic
vcs-pkg implementation, I wanted to take a close look
at my current, working, non-generic implementation: making sure
that the generic implementation can support at least this one
concrete work-flow will keep me grounded.
One of the features of my home grown porcelain for building
package has been that I use a fixed layout for all the packages I
maintain. There is a top level directory for all working trees.
Each package gets a sub-directory under this working area. And in
each package sub-directory, are the upstream versions, the checked
out VCS working directory, and anything else package related. With
this layout, knowing the package name is enough to locate the
working directory. This enable me to, for example, hack away at a
package in Emacs, and when done, go to any open terminal window,
and say stage_release kernel-package or
tag_releases ucf without needing to know what the
current directory is (usually, the packages working directory is
several levels deep
/usr/local/git/debian/make-dfsg/make-dfsg-3.91, for
instance.
However, this is less palatable for a generic tool imposing a
directory structure layout is pretty heavy. And I guess I can
always create a function called cdwd, or something, to
take away the tedium of typing out long cd
commands.
Anyway, looking at my code, there is the information that the
scripts seem to need in order to do their work.
rpm based sources, look for the
spec filedebian/rulesspec or
debian/rules in the current directory, and parse
either the spec file or
debian/changelog.tla tree-rootbzr infogit rev-parse --show cduphg rootdebian directory, and
changelog and rules files existspec file or
debian/rules in the base directory
spec or changelog files.pristine-tar is in use, given two trees
(branches, commits. etc), namely:
tar archive.dpkg-buildpackage imposes this too).pristine-tar or have the upstream
tar archive in the parent directory of the working
directorygitpkg has a different
concept: it works purely on the git objects, you feed it upto three
tree objects, the first being the tree with sources to build, and
the second and third trees being looked at only if the upstream tar
archive can not be located, and passes the trees to pristine tar to
re-construct the upstram tar. The package name and version are
constructed after
the source-tar archive is extracted to the staging area. I like the
minimality of this.
This is continued
here.
For a research project, I'm looking for some real-world time series data.
Time-series are an interesting thing to study, however it is hard to get access
to interesting non-trivial real-world data.I was wondering if some people could contribute me some summarized
web access data; no URLs or IP addresses.The data I'd like to get can best be explained by the preprocessing step:
... perl -ne '/\[(\d+\/\w+\/\d 4 :\d\d):\d\d/ && print $1."\n";' sort uniq -c(Sorry if you aren't fluent in regexp - it extracts the date and hour out of an Apache default log file, nothing else. These lines are then summarized by counting their unique appearances.)That should produce 24 lines per day (one per hour), looking like this:
count day-of-month/month/year:hourIt would be cool if you could send me some series for a couple of sites, if you happen to be in the position to provide this data. The data should cover at least a few weeks, the longer the better even up to a few years.Too small sites are however not very useful but might be too noisy (so probably not the personal home page of your mom). If you are providing a larger number of series, you are of course free to include them.I don't care much about what the site actually contains, I'd just ask you to give a tiny amount of meta information:
I've previously mentioned my plans on redoing my blog. Well, I've settled down on some design issues already (posts will be stored as mini Atom feeds, which makes the generation of Atom feeds for the blog and categories trivial, and gives me maximum flexibility. I already have a working converter for my existing blog to Atom posts.)Generating static HTML pages from that will be easily possible using an XSTL
transformation (for example), and I got the feedback that I could just use
Google AppEngine for
blog comments, so my actual blog could remain static-only (and thus much
more secure and reliable). Any attack/spammer/bot can then only kill the
comment functionality, not my own site.Which brings me to another design consideration: the editing widget. Either for
the blog comment application, for writing my own blog entries (via a https
protected script or whatever) or maybe for a small CMS I've been thinking about
- having a reliable HTML in-browser editing widget is something I could use
every now and then (well, I'm not doing much Web stuff anymore these days).Geniisoft has a good
overview over in-browser (aka: Through The Web, TTW) editors. The top
candidates seem to be:
I've heard before of FCKeditor and TinyMCE; I think I've been on the Xinha
page before, too. However, comments on them have not always been good.To some extend they all seem to have (to some extend)
feature creep
which usually is a bad sign - most often this means that there are security
issues in one or another module or plugin.TinyMCE for example has been described as "a bit of a pain" and "a tad
clumsy" on the GSoC mailing list. I have had less fights with it than
with the Debian Wikis (MoinMoin) markup language though.I'm not looking for anything as big as these - I just need an editor that
allows for some basic formatting (bold etc., links) and that produces
reasonable XHTML output. I'll be feeding the output through some custom
cleanup script anyway, which will kill disallowed code. So I don't want
any editor which allows the user to create code that will then be killed
afterwards.Any personal experiences with any of these, or an important alternative that
I might have missed (no PHP involved, please!) - email me (no comments on
blog) at erich AT debian org.Update: I've received a couple of pointers. Please don't send me links to
projects that are not actively maintained anymore - I don't want to care about
having to fix bugs in the editor widget myself.One link I've received twice is actually quite impressive:
WYMeditor. It doesn't try to look like a
word processor, but actually is more of a semantic editor. Much more what I'm
looking for than any of the others. I've also received a link to the
Yahoo! UI Library Editor,
which is quite clutter-free, but in the default setup at least very
text-formatting oriented, not very semantic (that doesn't mean you couldn't
change it that way, I guess you can). I was also pointed to Dojo, but that
framework is totally feature creep (which also explains why it loads so slowly
I guess), and the last time I looked at it's source code, I
had
some WTF moments - code quality at least outside of core doesn't seem to
be very high (Yes, that code implements the "mod 7" operation using list shifts
instead of a simple arithmetic operation).Looks like I'll give the first try to WYMeditor. Update #2: the code seems to
be rather ... complex. I'm looking for something neat and clean; it doesn't
need to bring along yet another XHTML schema and validator ... Maybe I should
try one of the others first?
There are a lot of little git scripts and tools being written by
a lot of people. Including a lot of tools written by people I have
a lot of respect for. And yet, they are mostly useless for me. Take
git-pkg. Can t use it. Does not work with git submodules. Then
there is our nice, new, shiny, incredibly bodacious 3.0 (git)
source format. Again, useless: does not cater to submodules.
I like submodules. They are nice. They allow for projects to
take upstream sources, add Debian packaging instructions, and put
them into git. They allow you to stitch together disparate
projects, with different authors, and different release schedules
and goals, into a coherent, integrated, software project.
Yes, I use git submodules for my Debian packaging. I think it is
conceptually and practically the correct solution. Why submodules?
Well, one of the first things I discovered was that most of the
packaging for my packages was very similar but not identical.
Unfortunately, the previous incarnation of my packages with a
monolithic rules file in each ./debian/ directory, it
was easy for the rules files in packages to get out of sync and
there was no easy way to merge changes in the common portions an
any sane automated fashion. The ./debian/ directories
for all my packages package that they are instrumental in
packaging. So, since I make the ./debian/ directories
branches of the same project, it is far easier to package a new
package, or to roll out a new feature when policy changes the
same commit can be applied across all the branches, and thus all my
source packages, easily. With a separate debian-dir
project, I can separate the management of the packaging rules from
the package code itself.
Also, I have abstracted out the really common bits across all my
packages into a ./debian.common directory, which is
yet another project, and included in as a submodule in all the
packages so there is a central place to change the common bits,
without having to duplicate my efforts 30-odd times.
Now people are complaining since they have no idea how to clone
my package repositories, since apparently no one actually pays
attention to a file called .gitmodules, and even when
they do, they, and the tools they use, have no clue what to do with
it. I am tired of sending emails with one off-cluebats, and I am
building my own porcelain around something I hope to present as a
generic vcs-pkg implementation soon. The firs step is
a wrapper around git-clone, that understands git
submodules.
So, here is the
browsable code (there is a link in there to the downloadable
sources too). Complete with a built in man page. Takes the same
arguments as git-clone, but with fewer options. Have
fun.
django-dumpslow is a reusable Django application that logs requests that take a long time to execute and provides an tool to summarise the resulting data.
Modelled on mysqldumpslow, django-dumpslow is intended as a tool to determine which parts of a project might benefit most from optimisation and to provide valuable empirical data.
The tool groups requests that are similar by exploiting Django's ability to reverse URLs - requests that use the same view are not counted distinctly. Views can then be grouped by the total accumulated time spent by that view or simply by their raw frequency in the logs.
An example output is:
$ ./manage.py dumpslow View Accumulated time ============================================================= example.views.slow 92.88 /unknown (unreversible url) 16.84If the log entries contain timestamps then django-dumpslow can optionally limit queries by a user-defined interval. This can be useful for generating regular reports of slow pages, perhaps by regularly emailing the "top 10 worst pages" to a development mailing list. Source, tarballs, etc. Patches welcome.
The monthly board meeting of Software in the Public Interest will take place on irc.oftc.net #spi tonight (Wed 15 April) at 20:00 UTC. The meeting announcement was posted, but a bit late, so there s another discussion about that going on behind the scenes. The frustrating thing is that fixing the announcements doesn t really need selling out to Google or anything complicated - simply using at on a server to schedule the email would work I know, because that s pretty much what I do at TTLLP for routine member emails.
Only the secretary has posted any report at the time of writing and there are no discussion items. There are two sets of minutes up for approval, but one contains at least one error which I ve already described by email. Should be a short meeting, though.
Once again (aren t Wednesday evenings awful for me?), I may be without a stable network connection during the meeting. Even so, watch the comments below this article for a link to the summary when posted.
RewriteCond % REQUEST_URI !^/FR/fr
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ /FR/fr$1 [PT,L]
The need here, is to get anything that wouldn t begin with /FR/fr from $DOCROOT/FR/fr (through, possibly, any other module rules, which was important in the setup where these rules were), except if they begin with /common/ or is /favicon.ico.
Depending on the requests pattern, reordering the RewriteConds can be better. Anyways, the case at hand was to massively duplicate this setup over different countries and languages.
Basically, domain.fr/foo/bar would need to come from $DOCROOT/FR/fr/foo/bar, domain.co.uk/foo/bar from $DOCROOT/GB/en/foo/bar, while anydomain/common/foo/bar would come from $DOCROOT/common/foo/bar.
In such a case, you have a lot of possible solutions, each of which has its pros and cons.
Pros: quite efficient for execution time. Cons: Doesn t scale very well for humans : a lot of virtual hosts and rewrite rules to maintain ; if the list of urls that don t need rewrite increases, all set of rules need to be updated.<VirtualHost *:80>
ServerName domain.fr
RewriteCond % REQUEST_URI !^/FR/fr
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ /FR/fr$1 [PT,L]
</VirtualHost><VirtualHost *:80>
ServerName domain.co.uk
RewriteCond % REQUEST_URI !^/GB/en
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ /GB/en$1 [PT,L]
</VirtualHost>
( )
SERVER_NAME:Pros: are there ?. Cons: Doesn t scale very well : a lot of rewrite rules to maintain ; rewrite rules being executed sequencially, the more domains there are, the more checks are done for the last domains ; if the list of urls that don t need rewrite increases, all set of rules need to be updated.<VirtualHost *:80>
ServerName domain.fr
ServerAlias domain.co.ukRewriteCond % SERVER_NAME =domain.fr
RewriteCond % REQUEST_URI !^/FR/fr
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ /FR/fr$1 [PT,L]RewriteCond % SERVER_NAME =domain.co.uk
RewriteCond % REQUEST_URI !^/GB/en
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ /GB/en$1 [PT,L](...)
</VirtualHost>
Pros: gives a canonical place for urls that don t need rewrite. Cons: Doesn t scale very well : a lot of rewrite rules to maintain ; rewrite rules being executed sequencially, the more domains there are, the more checks are done for the last domains.<VirtualHost *:80>
ServerName domain.fr
ServerAlias domain.co.ukRewriteCond % REQUEST_URI ^/common/ [OR]
RewriteCond % REQUEST_URI ^/favicon.ico$
RewriteRule .* - [L]RewriteCond % SERVER_NAME =domain.fr
RewriteCond % REQUEST_URI !^/FR/fr
RewriteRule ^(.*)$ /FR/fr$1 [PT,L]RewriteCond % SERVER_NAME =domain.co.uk
RewriteCond % REQUEST_URI !^/GB/en
RewriteRule ^(.*)$ /GB/en$1 [PT,L](...)
</VirtualHost>
RewriteMap.RewriteMap l10n txt:l10n.map # switch to dbm when necessary
RewriteCond % REQUEST_URI !^$ l10n:% SERVER_NAME
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ $ l10n:% SERVER_NAME $1 [PT,L]
Or
RewriteMap l10n txt:l10n.map # switch to dbm when necessary
RewriteCond $ l10n:% SERVER_NAME ^(.+)$
RewriteCond % REQUEST_URI !^%1
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ %1$1 [PT,L]
The map file would have on each line a domain name followed by the corresponding url start (/FR/fr for domain.fr, etc.).
Unfortunately, the above setup is not possible. A way around this lack of functionnality is to use some nice perl regexp trick.
RewriteMap l10n txt:l10n.map # switch to dbm when necessary
RewriteCond $ l10n:% SERVER_NAME ^(.+)$
RewriteCond %1% REQUEST_URI !^(.+)\1
RewriteCond % REQUEST_URI !^/common/
RewriteCond % REQUEST_URI !^/favicon.ico$
RewriteRule ^(.*)$ %1$1 [PT,L]
As you can see, only the second RewriteCond differs. What we really want to test is whether the % REQUEST_URI begins with the proper url start or not. Let s say we re considering domain.fr and the map gave us /FR/fr. Following the first RewriteCond, %1 contains /FR/fr.
At the second RewriteCond, if % REQUEST_URI begins with /FR/fr then %1% REQUEST_URI begins with /FR/fr/FR/fr. Otherwise, it will only begin with /FR/fr.
What we can test, then, is whether this %1% REQUEST_URI aggregate contains a repeating pattern at its beginning. This is exactly what the perl regexp does: it captures at least one character at the beginning of the tested string (^(.+)), and wants to find this captured string again (\1).
It is worth mentioning that this obviously falls flat when the url start itself contains a repeating pattern (e.g. /fr/fr instead of /FR/fr).
As we want the RewriteRule to work only when our % REQUEST_URI does not begin with /FR/fr, we negate the regexp, which means nothing will actually be captured, such that %1 in the RewriteRule will still be the last captured text, from the very first RewriteCond.
Note that if all the tested url starts are the same length and/or pattern, it may be worth changing the regexp to be more precise (and faster to match or not). Such as ^(/../..)\1 in our present case.
Next.