Tag clouds [en.wikipedia.org] are a current must-have for web 2.0 applications. Examples can be found for bookmarks, blog entries, photos, music or books. Tag clouds are hip, because they're a dynamic feature and show the "Zeitgeist" [wikipedia]. They given an overview on the users ("California" in flickr) or on current hot topics ("Israel", "Lebanon" in technorati). However, tag clouds also have severe limitations. First of all, they're arbitrarily ordered. Usually alphabetic, so there is no content relationship among the entires. Secondly, they only show an excerpt, since there are usually much more tags than fit on the screen. Thirdly, they're atomic information, whereas relations as used e.g. in RDF [wikipedia] can convey much more complex information. I'm trying to push tag clouds to a next level. They're a gimmick right now, but maybe we can make them to a powerful navigation tool? Together with Enrico Zini I've just created my first tag cloud (I've skipped making a tag cloud for my blog...). Well, it quickly evolved beyond a tag cloud. You could maybe call it a tag sky. Or tag forest. I'm not using my blog or something like this for the tag cloud. That would be quite boring, I'm not doing real tagging on it. Instead I'm using software tags. The Debtags project, led by Enrico and I, has been working on software tags for some years now during our spare time. We have about 600 tags in a dozen of facets, and 15000 software packages (I don't have the number ready how many of that are somewhat tagged already). Well, the tagging efforts are still far from complete, thats why we're currently working on an AI to assist tagging efforts, too. We generated two different renderings of the tag clouds for you: one separated cloud per facet, and all folded into one big cloud. Oh, and actually click on one of the tags, it will take you to a more complex tag-based navigation tool and a tagger. So what makes these different from the usual tag clouds you see everywhere (apart from the sheer size, sorry about that. Maybe we'll add buttons next to hide/show tags with low occurrence numbers)? Well, the tags next to each other aren't completely unrelated any more, since they are (in both renderings) grouped by their facet. This makes it easier to locate something - go through the red facets first, then look at the tags in the group. I'm thinking about a second step, which would involve dynamic expanding details in the tag cloud, or hiding them, finally transforming the tag cloud into a true navigation utility beyond a "single click filter". In my final diploma thesis, one of the topics to work on suggested by my professor is doing "tag clouds" (i.e. weighted lists) for relations. The prototype will likely be integrated with the IkeWiki semantic wiki. I don't have a clear vision of how the "relation cloud" will work or look like, but I havn't started with my tesis yet anyway. I currently imagine up to three clouds (corresponding to the empty places in the relation) that will dynamically adopt to the choices already made by the user. Some zooming will probably be needed, too. Another use of tag clouds would be a visualization of the AI - the weights could be chosen by how sure the AI is about this tag; the cloud would then describe the AIs rating of a software package description. If you have some ideas, good links, relevant papers or other feedback, just send me an email to erich@debian.org. Thank you.

Hm, the concert by Massive Attack was a bit disappointing. Parts were really good, for example "safe from harm" in a ten minute version with long solos; however a lot of it was just drawing out every last bit from each theme, reaching its "climax" at the end where it was essentially four chords repeated over and over for ten minutes, which ended up getting boring at some point.

I was added in the tasksel group some weeks ago, and started my own branch containing stuff that hopefully will be considered for the tasksel that will be used in post-beta3 d-i.<br /><br />The changes i made are:<br /><ul><li>Add security-audit task.</li><li>Add backup-server task.</li><li>Add scanner support in desktop task using xsane.</li><li>Add video playback and cd/dvd burn support to gnome-desktop using totem-mozilla and bonfire respectively.</li><li>Replace qpopper with courier-pop in mail-server task.</li><li>Replace uw-imapd with courier-imap in mail-server task.</li><li>Add sqwebmail and courier-webadmin in mail-server task.<br /></li></ul>Suggestions are welcome, but keep in mind that the 'cd/dvd burn support' thing should be changed. I don't think bonfire is the best choice by default, but added it to avoid push some old dependencies that for example gnomebaker (if added) would bring in.<br /><br />The stratus' tasksel branch is browsable at:<br /><a href="http://svn.debian.org/wsvn/tasksel/branches/stratus/?rev=0&sc=0">http://svn.debian.org/wsvn/tasksel/branches/stratus/?rev=0&amp;sc=0</a>

I like the Caps Lock key. My fingers automatically turn it on whenever I need to type more than three consecutive upper case characters. For example, when I type C code, and need to define a new FILE variable, hey, there was Caps Lock. Or a NULL constant. Or, heck, EWOULDBLOCK. I touch type. I spent a lot of time and energy in the 80s to learn to touch type. Using Caps Lock is fully automatic, to me, and it is quite a helpful key. (This log entry is part of the "Save Caps Lock" campaign brought to you by the International Society of Caps Lock Fanciers. No upper case characters were hurt in the production of this entry.)

Dear Lazyweb, I was recently thinking about SMS encryption (you know, the short messages sent from your cell phone). Or MMS encryption for that matter. Are there any Free Software solutions to implement such a thing, based on well-known crypto primitives and proven implementations thereof? From a quick glance I could not find anything usable, only lots of commercial, closed-source "solutions" which cost money and are basically crap. I was thinking about hacking together a small Java application (so that most modern phones can run it) which basically asks for a password and pipes your text through AES before sending the SMS. On the receiver's side, you enter the same password and get to see the plaintext. That's pretty much it. If you want something more elaborate you can use public-key crypto (basically embed GnuPG or similar into the application), but the above should be fine for most uses. Anyways, I do not want to start implementing something like this if there are other, more mature projects out there which do the same... Why you'd want to do this (other than simple paranoia, or the fact that governments and other institutions are spying on everyone these days, whether they're allowed to or not)? Well, one good reason for SMS encryption is when you're using some kind of SMS gateways, e.g. a website which gives you 2-3 free SMS per month if you sign up with them and give them tons of personal data. That alone is crappy enough, but you don't want their admins to be able to read and store all your SMSes in addition (which consist of simple plain-text, transmitted though HTTP in this case!). Neither should any local or remote scriptkiddy who knows how to use a sniffer be able to read your private SMSes. Solution: Write the text in your favorite text editor, pipe it through aespipe (for example), cut'n'paste the result in the web form of the SMS service. The receiver does everthing backwards and you're done.

No thanks. Especially when distributing kernel images without the corresponding source is surely against this whole GPL thing. Great example to set to vendors: 'I'd give you the source to our kernel modifications to get Linux up and running on this wireless router, but it'll cost you'.

I'm giving up on backporting SELinux to sarge. Too many changes are needed in too many apps to make them really compatible with SELinux. Nothing serious, but just lots and lots of small things. For example, the amavisd-new package will need a cronjob modification. This has already been resolved (somewhat) for unstable, but it means I would need to provide a modified amavisd-new package or a backport. I hope that when etch gets released end of the year (and I actually believe this will happen), many issues will already be resolved. But it depends of course on many people using SELinux in different settings. My most-annoying-issue with SELinux on Debian: cron bug #333837, open for some 300 days now. The debian cron package will backup e.g. /etc/shadow, which actually sounds like a quite inappropriate place for this task. And of course it's all in one file named /etc/cron.d/standard, instead of e.g. /etc/cron.d/backup-key-system-files or so, which I could then label backup_exec_t or something else to assign the special privilege of reading shadow files... It's bugs like these, unhandled for 300 days, together with having the impression of being the only one trying to get SELinux running and receiving basically no support by the SELinux upstream "community" (which is almost exclusively "enterprise", it seems). It's pretty much like everybody wants you to not use SELinux. Or in my case, not enable people to use SELinux on Debian, since I'm not just "joe average user", but actually trying to add SELinux support to the Debian distribution (which would help Ubuntu to get SELinux, too. The Ubuntu people seem to have given up on SELinux already).

Kernel being dumb by being smart Memory cache is a good thing to speed up a computer, but it’s a PITA when it comes to stopping playing a DVD or a big video file. The kernel usually keeps in memory cache data that has been read from the hard drive so that it can get it faster next time the same data is accessed. I don’t know exactly on what grounds fragments are decided to be kept or not, but observation shows that playing a DVD or a Xvid file makes the cache grow more than significantly. Moreover, while playing the video, the other software from the desktop (nautilus, for example), is not used, thus, the kernel decides it can swap it to disk virtual memory. Then, when you stop reading your video and want to come back to normal activity, your desktop is damn slow for some long seconds. That makes the experience a real pain. Overuse of network bandwidth I noticed that when I switched from a samba network share to an apache based DAV one: reading a video file from the DAV share with totem-gstreamer filled my apache logs very quickly. It seems gstreamer/gnomevfs seeks a lot in the file to play the video. Instead of making one HTTP request, it does several thousands, each time requesting again data it already received. As a result of that, it usually takes up to 1 mega byte per second to play a video that has a bitrate of 1 mega bit per second, with a nice effect of network congestion when the bit rate grows for detailed sequences. Do you find it normal that it takes 5 minutes to download a 45 minutes file at 2~3 MB/s, and 1MB/s for 45 minutes to play it directly from the network ? Well, now, gstreamer 0.10 doesn’t even support reading from HTTP/DAV, but the problem remains the same when reading via sftp with gnomevfs.

If one really wants to make international flying perfectly safe (at least safe from terrorists masquerading as passengers), it is in fact very simple and quite cheap: * No carry-on or check-in luggage at all (all kinds of chemical, bacteriological or radioactive materials could be hidden there, for example it can be trivial to make a laptop burn up in flames mid-flight and make in in non-detectable way). * No personal items of clothes (clothes can be soaked in certain chemicals to make them explosive). * Secure passengers during the flight ( so that they can not actually do any terrorist activity). So, the optimal way of securing air transportation would be to have people come in without any luggage at all, remove all items and clothing at check-in, dress in uniform one-time paper clothing where one can not hide anything, sedate them, feed them trough introspective inspection to detect in-body foreign objects and check the chemistry of skin and hair, pack them into the airplane on secured padded shelves, fly them to the destination, reverse the sedative and give them a standard set of clothes and essential personal items. In addition to perfect security, this plan will also reduce the overall cost of air travel by allowing to pack more people into an airplane of the same size - the padded shelves can go from the bottom of the airplane to the very top with minimal spacing (all passengers are sedated - they will not walk around), no separate cargo hold will be needed (no luggage) and no in-flight entertainment or food will be required. If the current security craze continues, we might reach this perfection by 2020. Congratulations! Note: All software by that time will be Web-based, so there will be no sense in taking your “personal” laptop with you.

One big benefit of free software is that there's lots of helpful advice on the web. One big problem of free software is that there's lots of dodgy advice on the web. Examples:

1. Solving a connection problem with ssh by switching to "default permit".
2. "Web resources mention that I may need to recompile Apache especially to support PHP".

Got more? My advice - try to learn the implications of a change before running untrusted commands from the web.

Tons of videos on YouTube use background music. Usually its some well-known song, in full length. So the authors are violating the song artists copyright, aren't they? I wonder when RIAA will target youtube users because of the background music used in their videos. For example the Israel attacks, Lebanon burns pathetic "video". Isn't that U2 in the background and thus most likely a copyright violation (apart from all the photos which most likely are just copied off some web site). And there a tons of examples. I bet you'll find people dancing to all kind of copyrighted music on YouTube...

try Today was a quite exciting day, first of all I had to stand up early in the morning, fortunately i reached the train on time. At 7.30 the school began and we had 3 lessons “ABU”, this was the most interesting part because later we had 2 hours of english. I mean, I don’t speak really good english yet, but I would like to learn it better, and the only thing we learned today was that “must not” doesnt mean “not have to” as most native german speakers would think. But yeah, I already learned it in the second month in the 8. schoolyear which is now nearly 3 years ago. The afternoon was also a little bit a horror, first we had programming, the topic was “Exceptionhandling in Java”. I think it’s a quite interesting subject, but the teacher was so boring, I mean, the examples didn’t “fix” a problem during runtime, they only printed out “there was an error!” or whatever useless. Then we had theory about servers and a little bit networking, I mean, most IT-students administrate some servers in their office and we learned today “A server provides a service in a network”, very interesting! Yes, so I will better forget this day! catch(ExceptionMarioDemotivated md) There are also reasons to celebrate! Today are 2 birthdays which are quite important to me. First Kerim, one of my best friends, became 17 and has now only to wait one year anymore until he’s “adult” here in Switzerland, ok, “to be adult” is relative anyway. Kerim, I wish you a lot of luck and love and whatever for your future! Stay as you are and I’m already glad to go to Zurich with you again (I’ll pay the next Frappu) The other important birthday belongs to the Debian project which is now 13 years young. Thanks to the whole community. For example I’m writing now this blog (and serve it) with Debian, so it’s time to say all “members” THANK YOU to make it possible! I’m thinking again about the future, what will be actual in 13 years in the IT? Will we win “the war” against software patents and all the other problems? At least our success with the ODF is motivating for the next 13 years! Ok, I’m quite busy now with baking some Debian-bread, I will offer a little brunch tomorrow in the office. Details, pictures and recipe will follow tomorrow.

It is funny that people say that no one has written software that does Beysian filtering in order to reject Spam at the SMTP level, just when I felt like writing up my mimedefang setup. I am of the school of thought that the only acceptable method of dealing with spam is to reject it at the SMTP protocol level. To accomplish this, I run crm114 and spamassassin from my mimedefang-filter, which runs as a milter while my sendmail is processing mail. I also do conditional grey listing, more on that below. The reason for running two different Spam detection mechanisms is that while one or the other is often fooled and mis-categorizes mail, the score or confidence level of such mis-categorization is low, and the mechanism that is not fooled enables the combined system to do the right thing in most cases. I tend to quarantine and keep even the messages I reject for about a month, and do spot checks, so out of roughly a 800 mails a day volume, I get about one or two spam messages a day, and did not ever reject a mail I manually would classify as ham in the 6 months n which I double checked every decision taken by my system, and in the spot checks I continue to perform. While this is not as good a record as the author of crm114 reports, it is good enough for me. The grey listing: since I try to be conservative, and opt to lean towards not discarding any email that is legitimate at the expense of not trapping all spam, and since I do not discard any mail I accept unread based on an automated decision, I was getting more Spam than I cared for still. So, I added the grey listing layer – but only for mail that were not ambiguously spam or viruses (which are rejected), or ham, which are accepted for delivery. Most of the Spam that was slipping through came from the grey area. So, I implemented grey listing only for these mails. Grey listing happens on a triplet of the class C domain of the senders IP, the sender, and recipient. A grey listed email encounters processing problems, and is temporarily rejected with a 503 for 30 minutes, and then retries are allowed through for 4 hours, in which case the triplet is white-listed for 35 days (catches monthly announcement messages). Known Spam resets the white-listing, and retrying past the window does too. Doing it this way, known good correspondents suffer no delay, unambiguous mail from strangers flows right through, and I improve my filters based on the results all the time, so my discriminator improves. My filter is available for examination. See also the older posting, like the time honoured tradition of /., this posting is a duplicate.

Russell, you are unhappy that you cannot send mail to some addresses because the recipients are using callbacks, trying to verify your address. Actually that looks almost like a feature, the system is preventing you from sending mail to somebody who would not be able to reply to you. ;-) If you wanted to break this nice symmetry while continueing the usage of blacklists to reject all mail from the blacklisted hosts but allowing smtp callbacks to succeed you could reject these on pre-DATA stage instead after RCPT, i.e.

 -> RCPT TO:<foo@example.com>
<-  250 Accepted
 -> DATA
<** 550 go away

Martin, I know about TDD, thanks. It is great for preventing all the kinds of bugs you expect to make (or make again), and can write test cases for. Been using it for years. It's part of what I meant when I said "every now and then a bug slips through, and then I'm screwed". It doesn't, however, help, when something really unexpected happens. For example, who would have expected beforehand to write a test case for the Pentium floating point bug? Not just a test case of whether floating point arithmetic works at all, but a test case specifically for the small number of special cases in which the original Pentium failed. It's for the unexpected cases where you really need your debugging skills. That's what the book is good for.

My plea has been answered by a DVFS project specification. For some strange reason, my edits do not show up in the freedesktop.org wiki page and the blog of its creator is not reachable, so I post it here. I am very interested to help develop this tool and then use it in Simple Backup Suit (sbackup). An additional feature that I would love to see is automatic splitting of a file into several chunks. In an ideal work it would be possible for me to get (in Python) a filehandle-like object that would compress the data (gzip/bzip/..), encrypt it (with simple encr. or gpg style), transfer it over network (ssh/ftp/dav/smb/…) and then split into 100 Mb pieces on the remote server (to overcome possible max filesize problems with 10,50 or 100 Gb backups). And have the freedom of flexibility of activation of each element of this chain when creating the file handle. Also the interface must react in exactly the same manner regardless or the backed being user (for example opening non-existing files must always rise the same error).

Today was a strange christian day here in Kanton Solothurn and I didn’t have to work, so I had a lot of time to prepare my presentation for next Friday about OpenLDAP. First of all I prepared the stuff for the school which begins tomorrow, so my holidays are now definitely past . I was excited that I didn’t receive my timetable by snailmail so I had to google and found it finally. Poorwise i have now to stand up very early in the morning, my train goes at 07.07 in the morning. Now to OpenLDAP, fortunately Mathias Weyland, the president from the LUGS, gave me a book about OpenLDAP from O’reilly, I really like it: It’s quite informative and sometimes the author is making some jokes. Priska Rubischon already bothered that i have to tell something about netgroups and autofs, this was completely new because I newer worked with NIS (and netgroups etc.) and neither with autofs (in most cases I use samba and libpam-mount). It’s not so difficult as i thaught. OpenLDAP in Debian comes with nice schemas which help to migrate from NIS to LDAP, so I had a look at those schemes and the chapter about netgroups in the book. Also autofs is not complicated, for example, if you want to share /home to the clients you have to prepare a NFS-Server which shares this directory and configure /etc/auto.master and /etc/auto.home on the clients. These two files can lookup where and how it can mount /home. On my testing-infrastructure I just used netgroups for the NFS-exports, so I have now an example what they’re for. After all I realized that my speech will also be about “How do I plan a network in a company”, so I realized that the security of LDAP is also very important. Fortunately OpenLDAP can be secured quite hard, it supports SASL and KERBEROS_V4 authentication mechanisms! When I would realize now a network i would do the following:

• Most parts of the directory readable anonymously
• Use SSL for LDAP
• The password writable by “self” (the “dn” can write the password itself when authenticated as it)
• Create an NFS Share, writable by some netgroups

I still ask me how i can protect my netgroups so that nobody can plug computer “x” out and plug in his computer, take the IP of “x” and mount the NFS-shares. Kerberos as user authentication would be a solution, but I don’t really like it. As I already told I prefer libpam-mount, with it’s help it would be possible that every user gets his share mounted maybe over smb. Now I will search for a solution how i can properly secure NFS, so it does only allow connections from users@boxes authenticated with LDAP and not user@host-which-can-be-faked. If you have an idea feel free to write it down here in my blog! In the evening I went to “Biel” by train to join the LUG-Seeland meeting. Beat Fasel explained me a lot about genetical biology, his work and the evolution, this was very interesting. Later we talked about the future of our community, I will prepare bylaws for the next meeting which we will sign. Other ideas were about preparing an install event and a mailinglist because we don’t have communication over the internet and this is quite complicated if we want to organise something in teamwork.

Yesterday I discovered some inadequacies in the stock Debian Etch initscripts when it came to filesystems on ATA over Ethernet block devices. Overnight I got a few emails, but it was a one Joshua J. Kugler who brought my attention to the _netdev mount option. The mount(8) manpage really is a treasure-trove of information that should be read often. So, in a nutshell, here's what I'm doing:

• added _netdev to the existing options for the filesystem that is on the ATA over Ethernet device in /etc/fstab
• /etc/init.d/mountall.sh modified to call mount with an extra -O no_netdev argument. Also filed #383073 to get this fixed for real.
• modified /etc/init.d/mountnfs.sh (I know this is a bit off-purpose, but the comments mention smbfs being mounted, even though there's no other traces in the file) to also mount devices with the _netdev option (essentially just mount -a -O _netdev). Also filed #383123 to get this fixed for real.
• modified /etc/init.d/umountnfs to unconditionally call umount -a -O _netdev (so the filesystem is unmounted before the network gets hauled out from underneath it). Also filed #383124 to get this fixed for real.

Finally, I found I had to either load the aoe kernel module as a post-up operation for the Ethernet interface in /etc/network/interfaces or load it in /etc/modules and call /usr/sbin/aoe-discover as a post-up operation in /etc/network/interfaces, because just loading it from /etc/modules resulted in the driver being loaded a fair while before the network came up, and once the network had come up, it took the driver longer than the time for /etc/init.d/mountnfs.sh to be run to realise there were ATA over Ethernet devices on the LAN and arrange for the relevant entries to appear in /dev/etherd. I've currently opted for running aoe-discover from /etc/network/interfaces. So I'm now very happy. I can reboot my MythTV server and everything comes back without any intervention whatsoever. Sweet. It'd be nice if these fixes can make it into Etch, as all of the other components will be there for ATA over Ethernet support. The kernel is new enough to have the driver as standard, and vblade is in Etch. Just the support to mount it all properly isn't there. Oh, and nothing I've done (with the exception of the post-up stuff in /etc/network/interfaces) is ATAoE-specific. It'll work equally well for NBD for example, and presumably other SAN-like stuff. I certainly find this approach far less gross than what Coraid suggests, and it'd also be cool if Debian were the first distro to do it the right way. I wonder if Red Hat/Fedora are already? I suppose I could fire up Xen or VMware and find out if I get curious enough.

I had tried to read Wilt already twice, it’s quite a short book, but somehow I had failed to get past the first pages both times. I failed to even open it when I took it to my holidays in the beach, but I’d to come back home earlier than my friends, so I had a boring three hour bus ride ahead, which was a great opportunity to finally read it. Wilt (Tom Sharpe, 1976) is a novel about an underachieving assistant lecturer (Henry Wilt), which grows increasingly demoralized by both his simpleminded wife (which he dreams to kill) and his frustrating experiences teaching literature to a bunch of uninterested construction apprentices in a South England community college. Everything gets messed up when Mrs. Wilt gets lured by Sally Pringsheim’s liberal ideas, and talks Henry into attending a party hosted by the Pringsheims, a bizarre hip couple. Henry ends up with an inflatable sex doll overinflated around his genitals, and Mrs. Wilt gets extremely infuriated. From this moment the story goes wild and I won’t spoil it

A small idea for increasing the performance of peer-to-peer communications for highly popular files. Imagine a BitTorrent file that becomes popular overnight. There are thousands of clients trying to download it, so for each client there are thousands of possible source to get the file from. However, one client can only connect to a small number of source at any one time (50-100). Due to randominess of that connection, such sources could be from the house next doors or could be from China or Australia. This creates excessive stress on the whole Internet as the parts of the file fly all around the world over and over. I suggest to introduce a horizon effect in clients when the number of peers is much higher then the max number of connections the client can make at one time. The easiest would be to make the decisions based on ping speed, but other criteria could also be useful. For an example implementation consider this: a new client joins in downloading a very popular file on a P2P network, he discovers enough peers to enable the horizon feature, the client connects to 100 random peers and takes their ping times, a distribution is calculated and a horizon is established - all peers with ping higher then the horizon value are deemed to be too far and are disconnected, freeing the resources for finding closer peers. The algorithm would continue to update horizon calculations and eventually would converge down to a set of closest peers and that would give both optimum performance to the client and less stress for the global Internet backbones. Separate horizons could be set up for seeders (clients with fully downloaded file) and lechers (clients with only partially downloaded file), so that we do not have a whole section of the network stagnating with no seeds in the area. Anyone up to make this work in, for example, Azureus?