autofs
(no GUI popups, no need to manually unmount and so on).
But of course, you can't very well just mount the drive and store your
keys on it in the clear. Well, actually you can (I did it for way too
long), but let's assume you want to do without. Because USB keys get
stolen or lost, too.
autofs
just handles the mounting of the drives, not their unlocking,
and there's no hook in there specifically for LUKS. There is,
however, a more generic way of running arbitrary commands on mounting:
the program maps. It's all described in the documentation, so I'll
just paste the relevant parts of my /etc/auto.removable
, with a few
comments. Remember to make it executable and reference it in
/etc/auto.master
.
The astute reader will have noticed that the keys volume is mounted either read-only or read-write with the same mechanism. This volume contains SSH keys that I use quite regularly, that filesystem is going to be mounted and unmounted very often. USB keys, and flash memory in general, don't really like repeated writes. Fortunately, the SSH keys are mostly read, and very rarely written to, so I can afford to mount the partition read-only and save the write cycles for when I generate new keys (the mere action of mounting a partition read-write changes it, even if the files in it are never modified). The backups volume is mostly accessed in write mode, and much less frequently anyway, so there's no need for a distinction there. This supposes that the partitions on the USB key appear as#! /bin/sh volume=$1 autoluks () dev=/dev/$1 cryptname=$ 1 _crypt cryptdev=/dev/mapper/$cryptname keyfile=/etc/private/$cryptname.key options=$2 [ ! -e $dev ] && exit 0 if [ -b $cryptdev ] && ! cryptsetup status $cryptname grep -q device:.*$dev ; then cryptsetup remove $cryptname logger fi cryptsetup --key-file $keyfile luksOpen $dev $cryptname logger [ -b $cryptdev ] && echo $options :$cryptdev true case "$volume" in # LUKS-encrypted volumes lacie-keys) autoluks lacie-keys -fstype=ext4,ro,noatime,nodev,noexec,nosuid ;; lacie-keys-rw) autoluks lacie-keys -fstype=ext4 ;; lacie-backups) autoluks lacie-backups -fstype=ext4 ;; # Non-encrypted volumes cd) echo -fstype=iso9660,ro,nodev,nosuid :/dev/cdrom ;; floppy) echo -fstype=auto,sync,nodev,nosuid :/dev/fd0 ;; esac
/dev/lacie-keys
and /dev/lacie-backups
. Due to the dynamic naming
of devices, you may like to use rules similar to the following in
/etc/udev/rules.d/local
.
You'll also need to initialize LUKS on the partitions, save the key, make the filesystems, and so on. If you're still with me, you probably know what I'm talking about, then I don't need to explain about that. This setup works for me, but there's no guarantee and so on. Based on a tutorial found on the Debian Administration website (great resource, by the way). Update: I'm told a large part of the script could be replaced withKERNEL=="sd[a-z]1", ATTRS idProduct =="1027", ATTRS idVendor =="059f", ATTRS manufacturer =="LaCie", ATTRS product =="LaCie iamaKey", SYMLINK+="lacie-keys" KERNEL=="sd[a-z]2", ATTRS idProduct =="1027", ATTRS idVendor =="059f", ATTRS manufacturer =="LaCie", ATTRS product =="LaCie iamaKey", SYMLINK+="lacie-backups"
pmount
. If you like short scripts, you may want to investigate that
too.
Update 2: Josselin points out
a different solution using
Gnome, udisks
, gnome-disk-utility
and Nautilus. It's worth
checking out, for those who like to run all that and click around. I
can't test it right now (Gnome/Nautilus currently refuses to mount any
USB key, for some presumably transitional reason), but I have no
reason to believe it's inherently broken. I just don't like having to
do anything when I plug my key or before I unplug it, and I'm not
always running a graphical session with a desktop.
Weaving a Semantic Web Across OSS Repositories: Unleashing a New Potential for Academia and PracticeHere s the abstract:
(DOI:10.4018/jossp.2010040103
)
Authors: Olivier Berger (Institut Telecom, France); Valentin Vlasceanu (Institut Telecom, France); Christian Bac (Institut Telecom, France); Quang Vu Dang (Institut Telecom, France); St phane Lauriere (Mandriva, France)
in International Journal of Open Source Software & Processes (IJOSSP) Volume: 2, Issue: 2 (2010), Pages: 29-40 pp.
Several public repositories and archives of facts about libre software projects, maintained either by open source communities or by research communities, have been flourishing over the Web in recent years. These have enabled new analysis and support for new quality assurance tasks.Keywords : RDF, forge, archive, bug, semantic, semantic Web, ontology, database, repository of repositories,
This paper presents some complementary existing tools, projects and models proposed both by OSS actors or research initiatives that are likely to lead to useful future developments in terms of study of the FLOSS phenomenon, and also to the very practitioners in the FLOSS development projects.
A goal of the research conducted within the HELIOS project is to address bugs traceability issues. In this regard, the authors investigate the potential of using Semantic Web technologies in navigating between many different bugtracker systems scattered all over the open source ecosystem.
By using Semantic Web techniques, it is possible to interconnect the databases containing data about open-source software projects development, which enables OSS partakers to identify resources, annotate them, and further interlink those using dedicated properties and collectively designing a distributed semantic graph.
Two years ago, while working on awesome, I joined the Freedesktop initiative to work on XCB. I had to learn the arcane of the X11 protocol and all the mysterious and old world that goes with it. Now that I've swum all this months in this mud, I just feel like I need to share my thoughts about what become a mess over the decades. When I was unborn
the Toto band were releasing their song "Africa" and some smart guys were working on a windowing system: the X Window System (this is its full name) which therefore has a (too) long history. The latest version of its protocol, the 11th one, has been designed in the 80's. You can learn more about the history in the Wikipedia article about X. In 2010, we still listen disco music and we still use various protocols designed in the 80's and even before X. Music have evolved, protocols have evolved, and so did X11. The problem is that X11 did not evolve that well. The guys at MIT-and-some other-places-with-very-smart-people-in-it created X version 1 in 1984, and updated it until X version 11 (the one we're still using) in 1987. Eleven version in 3 years, that was following the "release early, release often" model. But I don't know why, it just stopped to happen for the last 23 years1. I don't know what changes have been made in the first 11 major versions of the X protocol, but I'm rather sure we should have deserve a couple of major version updates this last 2 decades. In my humble opinion, X11 was not designed to live 23 years. But hey, I'm not blaming anyone here: I was 4 years old and playing Lego when they released this latest version of the X protocol, so there is little chance I'd have done something better.
1. That's not totally true: they added (and then deprecated) many extensions. We won't fix. We'll work-around.
That is probably one of the guideline of the X protocol for the last years. And don't misread me: I'm not bashing anyone thereafter. Since the X11 protocol was aging, the X guys started to add extensions. They added tons of them over the years. This, in application of one of the early principles of X:
All of them with no exception were added because, bad luck, the X11 protocol did not anticipated the things that happened in the last 23 years, like video, OpenGL, multiple monitors, or the pleasure to draw oval windows. Some of this extensions are still in use, while some of them have been dropped. While this is not a bad thing to extends the protocol, it seems like a bad thing to try to fix the protocol with for example the XFixes extension, even with all the good intentions Keith Packard might have in his greatness. Actually it's even worst than you thinkIt is as important to decide what a system is not as to decide what it is. Do not serve all the world's needs; rather, make the system extensible so that additional needs can be met in an upwardly compatible fashion.
The X11 protocol (without extensions) defines about 120 types of requests: create a window, move a window, etc. Nowadays, there's at least 25 % of them which are useless: usage of server-side font, or the drawing of squares and polygon, are unused by any modern application or toolkit. All of this is superseded by requests from extensions, like the XRender one. The handling of multiple monitors displays has totally been screwed up. X11 has been designed to work in Zaphod mode (independent monitors). But Xinerama, and nowadays XRandR have replaced it up: recent X servers (released after ~2007) does not support Zaphod mode anymore, even if it's a core piece of the X11 protocol. Worst: on many requests, there's limitation or design flaws, like described in this document: Why X Is Not Our Ideal Window System by DEC researchers. We'll add more broken standard on top of that
Following its early principle, X does not define policies but only mechanisms, which seems like a good thing, Consequently, people started writing specifications to determine a number of stuff and dogmas: ICCCM. That was 22 years ago in 1988. It's useless to add that many things in this specification are now obsolete, useless, or that it misses many modern stuff. I was not the only one to think that. The people from what will be the major desktop environments, KDE and GNOME, saw that too in the 90's while I was learning to count. So they wrote EWMH, another standard that comes on top of ICCCM and extends it with nifty features like maximization, full screen mode, etc. The problem is that this standard has also been written by narrow-minded people who at that time, were working on GNOME or KDE (and maybe others). This desktop environments were having and still have some strong concepts of how should work a desktop: "it should have work-spaces", "a window is only on one workspace", "we only see a workspace at a time", "you do not have multiple screens", etc. Dude, we don't care: we have toolkits!
This vision of how the desktop should work have now been written in marble in all applications and libraries implementing EWMH, like GTK+ or Qt. Nowadays, everybody forgot about all of this standards. Toolkits have implemented this, circumvented the X11 protocol limitation and flaws, and nobody wants to look back. Like all standards, obviously some people implemented them badly. This had some side effects, like OpenOffice acting like a pager. We don't look back? Worst, we forgot where we came from!
With all these layers of bad designed standards, the desktop continued to evolve for more than a decade. They continued to add more standard, the more recent ones being based on D-Bus like the Desktop Notification Specification or the latest Status Notifier Specification developed by KDE. The Status Notifier is a new implementation of the good old system tray based on XEmbed, using D-Bus instead of the X11 mechanisms, and adding the possibility to show the system tray with something else than icons. This specification draft saw an important issue and design flaw raised by Wolfgang Draxinger in this thread on the XDG mailing-list. What Wolfgang points out, is that X is network-oriented, and D-Bus is not. Therefore, making the Status Notifier specification to use D-Bus to pass system tray messages around is a bad idea, since running a X application from host A on host B will draw the system tray on the wrong host! Apparently, reading the thread, this does not fear some of the KDE people:
What Oswald describes as a corner case is an actual common use case for many of us. Of course, YMMV. From my point of view, this is a step back in the wrong direction. But we can conclude that the network part of X is now worthless, to at least KDE. I used to believe in XCBof course this is a bizarre corner case not worth much thought. at least that's what you'll think until you actually run into it yourself (be it because you are testing something or because you are setting up some weird kiosk environment).
When I joined Freedesktop, it was to work on XCB, the X C Binding. XCB is a nice, clean, 21st century technology based API to play with the X11 protocol. Its code is auto generated based on XML file describing the protocol. In comparison, Xlib is 80's obfuscated code with almost no comments and hard-coded things. Only a few people can understand some of its corner like its i18n or XKB implementations. And all its code is synchronous. For people not knowing it yet, X is a network protocol where you send request (like a GET in HTTP) and then get a response. Xlib forces the application to wait for the reply to its request, so the application is blocked until the X server sends the reply to the request. XCB on the other hand does not block and allows the application to send a batch of requests, do some other stuff in the mean time, and then gets the replies. It's like your Web browser would send one request at a time to a Web server, and would wait until you downloaded all the images one by one to display the page. In cases where X and all its clients are on the same host, the latency is small and not really visible, therefore the gain for XCB to be asynchronous is small. On slow network however, the gain can be huge, as proved in the rewrite of xlsclients with XCB by Peter Harris. One of the long standing goal of the XCB folks is to kick-out Xlib, to increase speed and hides latency in X11 applications. That requires to port many libraries, because almost none of them (Cairo being an exception) supports XCB. From where I stand, I don't really see if the work is worth it now. The desktop world is trusted by GNOME and KDE, meaning GTK+ and Qt. It seems none of this toolkits are interested to work on XCB, neither on the X protocol. They probably put hard effort in bypassing X limitation and flaws, and they now sit on top of crap of workarounds and broken-by-design-standard implementation. It seems to me they don't want to go back in the layers and improves things. They're too high to go back down and they don't see what the gain would be. Enlightenment with its EFL was the first toolkit to have an XCB back-end with the work of Vincent Torri. Unfortunately, the back-end is not maintained and nobody cares about it. Last time I tried it, it did not compile at all. X12 ?
There's a page called X12 on the Freedesktop wiki, listing all the things that should be fixed some days. Unfortunately, the list continues to grow up an no one talks about working on X12. On the other hand, there's a handset of people trying to work when they will have time on XKB2, the second version of the "let's-try-to-fix-up-the keyboard-part-of-the-protocol-we-wrote-23-years-ago-a-second-time" extension. To me, it does not seem X12 will happen in the next decade neither. Alternative ?
Do we got alternative to X ? There's Wayland, but it's far from being usable. There's DirectFB, but that's not very portable. None seems candidate to replace X some days to me. Anyhow, none of the main toolkits around support this alternative. GTK+ once supported DirectFB, but as far as I know, it is not supported nor works nowadays, as stated by Josselin Mouette. This is why recent versions of the Debian installer have migrated to X for the graphic part, thanks to Cyril Brulebois work. Conclusion
XCB has been around for more than half-a-decade, and very few people showed interested in it. As far as I can see, nobody is interested to use the X protocol and everybody tries to encapsulate it in some higher-level API as soon as possible to stop seeing it. This leads to poorly written application and toolkits, with a lot of ugly hack. All of that also means that starting to write applications and graphical toolkits based on XCB would be a very interesting project, but that would lead to spend too much time learning to circumvent the X protocol flaws, things that have been done in years by predecessors like Qt and GTK+. Major toolkits implementations have almost nothing to win in going back in the dark water of X. I guess most of their folks prefer to work on shiny 3D effects based on your GPS location, rather than redefining better basis for everyone. The manpower available in the X world is very small. Debian lacking of X maintainers is just the summit of that. There is very smart and very competent and skilled guys in the X world, as you can see by simply reading blog posts on Planet Freedesktop for example (me excluded). Unfortunately, there's not enough of them to cover all the things involved in X: input devices, graphics devices, new protocol extension specification and so on. The X server is really late, and it seems most of the developers prefers to work on the server itself than on the protocol behalf. Which is understandable. I'm curious to see where all of that will lead in the upcoming years. I've been walking in the X world hallways for about 3 years now, and I feel desktop alternatives to KDE and GNOME will all die sooner or later. The time were you could choose between a dozen "modern" window managers has passed away. After all, maybe that is simply Darwinism applied to computer software.
Do you like Debian? Do you like GNOME? Do you want to help Debian and GNOME?
If you have answered yes to all those questions, join this weekend on #debian-gnome on irc.debian.org / irc.oftc.net and participate in the Debian/GNOME bug session! More information at Joss s blog.
Gtk
actually works
Since the udebs
are quite fat already, I decided to put the contents
of the gtk2.0-examples
package into a new revision of the new
libgtk-x11-udeb
package. gtk-demo
was running fine (if one forgets
that this demo application is really easier to deal with if one uses a
window manager), although image handling was broken. Julien and I
spent quite some time double-checking that the
libpixbufloader-png.so
plugin was present; that both previous
DirectFB
-based udeb
and new x11
-based udeb
were very similar;
that configuration was generated properly; we went so far as to include
strace
in the netboot image, and to check the configuration file
was open()
d, but the plugin was never to be even looked for. All
this resulting in a Unrecognized file format message every time.
Josselin, our Messiah, enlightened us: although Gtk was previously
using some configuration files to determine which pixbufloader
plugin to use, it still continues to access those files but relies on
a MIME database in the end. Installing the host s
/usr/share/mime/mime.cache
database did the trick (we have a proper
fix pending), and all of a sudden, we got image support (which is
needed for the logo_installer.png
displayed on top of every screen,
as well as for some icons during the installation, like the final
one).
Tweaking d-i itself
Julien took care of rootskel-gtk
. It was mostly about:
DirectFB
start-up scripts.Xorg -retro -noreset &
export DISPLAY=:0
cdebconf
. Mostly: search and destroy on the
few bits using DirectFB
. An interesting issue we had was that even
though a proper cursor theme was installed and made the default, we
only had a black cross as X
pointer. That s where we learnt it s
needed to manually set a cursor for the root window, which can be done
this way:
GdkCursor *cursor = gdk_cursor_new(GDK_LEFT_PTR);
gdk_window_set_cursor(gdk_get_default_root_window(), cursor);
gdk_cursor_unref(cursor);
Tweaking gtk2-engines
was trivial, just a matter of building against
the x11
flavour of Gtk
instead of the directfb
one. Some
substitutions in debian/control
and debian/rules
, and that was
done.
Julien also took care of tweaking console-setup
so that
keyboard-configuration
s and console-setup-udeb
s postinst scripts
call setxkbmap
when $DISPLAY
is set. Meaning we were able to
replace kbd-chooser
with console-setup-udeb
, leading to a correct
setting within X
right after having selected the layout!
Reverting d-i
s r61645 (which disabled Gtk-based d-i
images) was
trivial (although while looking back, it was probably unnecessary),
and after having messed around with the packages lists (there s a lot
of local udebs
at this point), especially to add some fonts, we
managed to get an image which seems to work fine (at least in my
VirtualBox environment). I tried Chinese, Japanese, Arabic, and French
installations, and all of them seem to work fine (installed systems
were bootable after an installation in the selected language) while
issues were limited to some rendering or translation glitches, which
didn t seem critical at all.
Next steps
Since there were some iterations, and since the trial-and-error
approach was chosen, the first goal is to generate clean patches
against all modified packages, and make sure a single walk through all
packages in a clean chroot is sufficient to generate a working d-i
image again.
Once that done, which should probably considered (at least, I hope!)
quite a good base already, the idea is to look into the various
udebs
to see whether there are some parts that could be disabled
(maybe introducing a trimmed-down flavour especially for the udeb
in
some non-flavoured-yet libraries), so as to reduce the size of the
resulting image.
In the meanwhile, one can fetch the current hand-built netboot image
by browsing http://people.debian.org/~kibi/di-x11/. It s only
available for amd64
, and weights 23MB
. There s a sha256sum
as
well as a detached GPG signature in the same directory.
libfooN
entry in debian/control
.-udeb
suffix to the Package
line.Package-Type: udeb
line.Section
is debian-installer
.Priority
is only optional
.pkg-x11
packages are quite sane, using debhelper
s
dh_install
. So duplicating debian/libfooN.install
into
debian/libfooN-udeb.install
is usually sufficient. It s a bit
trickier when dh_movefiles
is used, but calling install
and cp
-r
after the dh_movefiles
call should be sufficient.
Next step, tweak shlibs
for libfooN
. Without that step, a
libbarM
package built against libfooN
would get libfooN
in
Depends, which is not appropriate. The idea is to get libfooN-udeb
instead. Trivial: add --add-udeb=libfooN-udeb
to the dh_makeshlibs
call.
Possible issues at that step:
debian/shlibs
or debian/libfooN.shlibs
is used instead of
dh_makeshlibs
. Trivial fix: feed its contents to dh_makeshlibs
through its -V
argument, and delete that file. Caveat: when it
comes to applying a patch removing that file, make sure it s
actually removed, and not just empty. I m going to file a bug about
that, the default behaviour with empty files seems a bit buggy.--add-udeb
arguments. Solution: use several
dh_makeshlibs -p$pkg -V$depends --add-udeb=$pkg-udeb
call. (I
thought it was needed for libx11
but I can t read, actually:
libxcb1
libx11-xcb1
.)debian/libfooN/DEBIAN/shlibs
is correct: it should have
a udeb
line. Example:
libfoo N libfooN (>= some-version)
udeb: libfoo N libfooN (>= some-version)
Depends
. Since I m still at the proof-of-concept stage, I ve
iterated when I encountered such additional packages, without
checking what was actually needed. The idea is to get rid of extra
stuff once the final needs figured out. Of course, one has to track
down the dependencies to get the packages incrementally built and
installed (for the updated shlibs
files).[n]
: package patched at step N
.*
: warning when patching, shlibs
file to be removed.libx11-xcb1
is probably not needed.d-i
image
Basic d-i
image
General guidelines (see d-i
documentation for detailed installation,
needed packages, etc.):
svn co svn://svn.debian.org/svn/d-i/trunk/ debian-installer-trunk
cd debian-installer-trunk/installer/build
make build_netboot
If that breaks (e.g. due to missing packages), make sure there s a
similar entry in sources.list.udeb
:
deb http://yourmirror/debian squeeze main/debian-installer
If everything goes well, the generated image should be available as
dest/netboot/mini.iso
, and tmp/netboot/udeb.list
should contain
the list of embedded udebs.
To give it a try:
qemu-system-x86_64 -cdrom dest/netboot/mini.iso
Actually adding udebs
Drop all additional udebs under localudebs/
. Needed Packages
and
Packages.gz
should be generated on the fly (but beware, secure apt
is then disabled, even for non-local udebs).
Now, tell d-i
we want some more packages. If all of them are wanted,
one can do that:
for i in localudebs/*.udeb; do \
dpkg-deb --showformat='$ Package \n' -W $i; \
done > pkg-lists/local
(Update: Thanks to Guillem for pointing out a better and cleaner way
than dpkg --info $i grep Package: awk ' print $2 '
)
Generate a new image (note the rebuild
part of the target):
make rebuild_netboot
Make sure they got added by looking in tmp/netboot/udeb.list
and
profit!
What s next
Future steps include:
mini.iso
(using d-i
s trunk) wasn t
that difficult, so hopefully once I m done with a few udebs, it should
be possible to add them to this image and make sure those are usable,
at least when loaded manually.
Once that finished, gluing everything together is probably going to be
another story, but hopefully that won t be impossible.
Unrelated stories
kfreebsd-*
now have DSA-admin d
experimental
buildds. Aur lien took care of setting them up; they
managed to go through all experimental
packages, and bugs were
filed accordingly. Some were just marked as failed
, usually with
Needs porting. , especially when there were no regressions with
respect to what was previously built on those architectures (as a
reminder: out-of-date
means serious
, uncompiled
means
important
).lenny
environment, but maybe people would like it to be directly
available once the lenny-backports
repository enabled?
Mail/wishlist bug welcome if you want it.;)
). Key rollover being thought of.Next.