Earlier today I watched the presentation
Jonathan Oxer's
gave during LCA about
Package caching solutions.
Although it was certainly an interesting presentation and although I very much
agree that my current local mirror is wasting a lot of diskspace and bandwidth,
I'm still not going to switch from
debmirror to any of the available
caching solutions, because (unless I'm really missing something) none of them
scratches my itch.
My local mirror currently consists of five architectures (i386, amd64, hppa,
sparc and s390) and only has unstable and testing. I use it for:
- (fast and convenient) updating of my systems
- doing Debian Installer builds and installation tests
- test builds of installation CDs (using
debian-cd)
Now, the last one is somewhat hard (
debian-cd uses hardlinks to packages on
a local mirror instead of retrieving packages), so let's concentrate on the
first two.
Caching is great if you have a large number of machines – of the same
architecture and that are all likely to need roughly the same packages –
sitting behind the proxy: the first one triggers the download of the package
and the rest gets it almost instantaneously.
It is a lot less great if you have only one, maybe two systems per architecture:
most of the time you'll still end up going down that (relatively) slow ADSL
connection.
An important reason why I have my mirror is so that when I do my daily updates
for sid or run an installation test, the packages
are already available locally.
I really don't want to double or even treble the time needed for installation
tests just because some required packages aren't yet available locally and need
to be downloaded over that slow line.
So, I have my partial mirror. Somewhat tuned (I exclude some ridiculously large
debug packages for example, saving about 10GB), but still with
a lot of
junk^Wpackages on it I'll never ever use, especially for hppa, sparc and s390
as those systems only have fairly basic installations.
Getting rid of that would significantly reduce my daily sync and allow me,
for example, to also have a mirror of stable and oldstable, keep old versions
of packages and probably
still save a lot of diskspace.
Wishlist
What we should have is a hybrid solution: a program that will present itself
as a proxy to clients, but is smart enough to pre-fetch new packages that are
likely to be needed in a sync run, based on usage date from the the proxy and
configuration settings.
Some ideas for features/configuration options it could have:
- should support both source and binaries (debs and udebs)
- transparently retrieve packages that are requested but not available locally
(just like a regular proxy)
- options to always sync (per architecture):
- all required and standard packages
- packages listed in a certain config file
- packages in a certain section (e.g. udebs in the debian-installer section)
- for packages that have been used (requested by clients) within the last X
days: pre-fetch any new version (per architecture)
- options to include security/volatile updates in that
- be smart about ABI changes: when a package name changes because the ABI
version changes, also pre-fetch the new package; an alternative solution for
that could be:
- for recently used packages: ensure that their dependencies are also pre-fetched;
this would also provide support when packages are renamed (dependencies in the
transition packages would ensure the new packages are pre-fetched)
- expire packages from the mirror that have not been used for X days
- allow faster expiration of certain types of packages (-dev or -dbg for example)
- options to keep X previous versions that are no longer referenced in the
Packages files for Y days
Possibly such an implementation could even be used on some of the lower tier
Debian mirrors.
Unfortunately, unlike some of my esteemed colleagues, I'm not able to just
whip up something like this, so I'm condemned to wait and see if there's
someone else who'd like to pick up this idea. I am of course more than
willing to help develop this idea further and to test it.
Now, if I've totally failed in my research and something like this already
exists, a pointer in the right direction would be much appreciated.
Some time ago Yubico were kind enough to send me an evaluation copy of their Yubikey device. I ve finally got around to reviewing it and making deployment plans for buying some more. Above is a picture of my Yubikey on the keyboard of my Thinkpad T61 for scale. The newer keys apparently have a different color in the center of the circular press area and also can be purchased in white plastic.
The Yubikey is a USB security token from Yubico [1]. It is a use-based token that connects via the USB keyboard interface (see my previous post for a description of the various types of token [2]). The Yubikey is the only device I know of which uses the USB keyboard interface, it seems that this is an innovation that they invented. You can see in the above picture that the Yubikey skips the metal that is used to surround most USB devices, this probably fails to meet some part of the USB specification but does allow them to make the key less than half as thick as it might otherwise be. Mechanically it seems quite solid.
The Yubikey is affordable, unlike some token vendors who don t even advertise prices (if you need to ask then you can t afford it) and they have an online sales site. $US25 for a single key and discounts start when you buy 10. As it seems quite likely that someone who wants such a token will want at least two of them for different authentication domains, different users in one home, or as a backup in case one is lost or broken (although my experiments have shown that Yubikeys are very hardy and will not break easily). The discount rate of $20 will apply if you can find four friends who want to use them (assuming two each), or if you support several relatives (as I do). The next discount rate of $15 applies when you order 100 units, and they advise that customers contact their sales department directly if purchasing more than 500 units so it seems likely that a further discount could be arranged when buying more than 500 units. They accept payment via Paypal as well as credit cards. It seems to me that any Linux Users Group could easily arrange an order for 100 units (that would be 10 people with similar needs to me) and a larger LUG could possibly arrange an order of more than 500 units for a better discount. If an order of 500 can t be arranged then an order of 200 would be a good thing to get half black keys and half white ones you can only buy a pack of 100 in a single color.
There is a Wordpress plugin to use Yubikey authentication [3]. It works, but I would be happier if it had an option to accept a Yubikey OR a password (currently it demands both a Yubikey AND a password). I know that this is less secure, but I believe that it s adequate for an account that doesn t have administrative rights.
To operate the Yubikey you just insert it into a USB slot and press the button to have it enter the pass code via the USB keyboard interface. The pass code has a prefix that can be used to identify the user so it can replace both the user-name and password fields of course it is technically possible to use one Yubikey for authentication with multiple accounts in which case a user-name would be required. Pressing the Yubikey button causes the pass code to be inserted along with the ENTER key, this can take a little getting used to as a slow web site combined with a habit of pressing ENTER can result in a failed login (at least this has happened to me with Konqueror).
As the Yubikey is use-based, it needs a server to track the usage count of each key. Yubico provides source to the server software as well as having their own server available on the net obviously it might be a bad idea to use the Yubico server for remote root access to a server, but for blog posting that is a viable option and saves some effort.
If you have multiple sites that may be disconnected then you will either need multiple Yubikeys (at a cost of $20 or $15 each) or you will need to have one Yubikey work with multiple servers. Supporting a single key with multiple authentication servers means that MITM attacks become possible.
The full source to the Yubikey utilities is available under the new BSD license. In Debian the base functionality of talking to the Yubikey is packaged as libyubikey0 and libyubikey-dev, the server (for validating Yubi requests via HTTP) is packaged as yubikey-server-c, and the utility for changing the AES key to use your own authentication server is packaged as yubikey-personalization thanks Tollef Fog Heen for packaging all this!
The YubiPAM project (a PAM module for Yubikey) is licensed under the GPL [4]. It would be good if this could be packaged for Debian (unfortunately I don t have time to adopt more packages at the moment).
There is a new model of Yubikey that has RFID support. They suggest using it for public transport systems where RFID could be used for boarding and the core Yubikey OTP functionality could be used for purchasing tickets. I don t think it s very interesting for typical hobbyist and sysadmin work, but RFID experts such as Jonathan Oxer might disagree with me on this issue.
. As usual grab the video 
. Patents are GPL incompatible, but fine with BSD/MIT. The Tom Tom is the first direct attack using MS s patent portfolio. This undermines all the outreach work done by the MS Open Source team which Jeremy tells us are true believers in open source, trying to change MS from the inside. Look for MS pushing RAND patented standards: such things lock us out.
Netbooks are identified as a key point for MS to fight on lose that and the desktop position is massively weakened.
We should:
