Search Results: "ag"

24 October 2007

Russell Coker: Blog Ethics

Reporters Sans Frontiers (AKA RSF AKA Reporters Without Borders) has an interesting document about blogging [1]. They are specifically focussed on blogging as a way of reporting news. Their definition of a blog states that it is “a personal website” (there are many corporate blogs run by teams) and that it contains “mostly news” (most blogs that I read contain mostly technial information about computers and any “news” is mostly about the author). They also describe a blog as being set up with an “interactive tool” - most blogs are written using web-based tools, but some are written with plain text and a compilation system. Some of their technical information is simply wrong. For example they say that RSS “alerts users whenever their favourite blogs are updated” (this can be done through RPC notification which then requires RSS to pull the data - but a user will almost always have a tool polling their favourite RSS feeds). Their section listing the various options for blogging platforms mentions LiveJournal, Blogger, and MSNSpaces but doesn’t mention Wordpress.com which seems to be a serious omission (although it does mention civiblog.org which is a useful hosting resource for grassroots political campaigning). But these are minor problems, they are reporters primarily not programmers. Their document gets interesting when it gives links to pages about blogging ethics. One page that was not directly linked (presumably because it mainly concerns non-journalistic blogging) is Rebecca Blood’s document about blogging ethics [2]. She gives a brief coverage to conflicts of interest and gives most space to the topic of maintaining a record of changes. One thing I have been considering is having a separate instance of Wordpress for documents that change [3]. This way regular blog posts (such as this one) can be relied on to preserve a list of changes (any change other than correcting a typo will be documented) so if you link to a post you can rely on the original intent being preserved. But for posts which have ongoing value as live documents they will be kept current without a significant change-log. Items that go into the main blog would include commentary on news and predictions, and the document blog would contain technical information about Linux, the science-fiction stories that I will eventually write, etc. When I wrote my previous blog post about this issue I was mainly considering technical issues, but when considering the ethical issues it becomes clear that I need a separate blog (or other CMS) for such things. The site etbe.coker.com.au needs to be a reliable record of what I wrote while another site such as docs.coker.com.au could contain live documents with no such record. The essential factor here is the ability of the user to know what type of document they are seeing. Another really interesting point that Rebecca makes is in regard to known bad sources. Sometimes a known bad source can produce worthy data and is worth referencing, but you have to note the context (which among other things means that if the worthy data gets any reasonable number of hits it may get replaced by something entirely different). If a blogger cites a reference on a known bad site and explains the reason for the reference (and the fact that the nature of the site is known) then the reader reaction will be quite different to just a reference with no explanation. As a contrast to Rebecca Blood’s document the cyberjournalist.net code of ethics [4] covers the traditional issues of journalistic integrity, “public interest” etc. One interesting issue that they raise (but apparently don’t address) is the definition of “private people“. While it is generally agreed that movie stars and similar people deserve less protection as they have chosen to be in the public eye the situation regarding bloggers is less clear. Is a blogger with a high ranking on technorati.com in a similar position to a Hollywood star when it comes to privacy? A claim is sometimes made that blogs are unreliable because they can change (based on the actions of some bloggers who change posts with no notification and turn sections of the blogsphere into a swamp). Magazines and newspapers are held as an example of unchanging content and people who publish online magazines sometimes express horror at the idea of changing a past issue. The fact is that newspapers and magazines never changed past issues because it is impossible to recall hundreds of thousands of printed documents that are distributed around a country or the world. When it’s possible to fix old issues (as it is online) then the requirement is to document the changes not keep bad versions. It probably would be a useful feature for a blogging package to have the ability to display (and link to) old versions of a page. This would allow the users to easily see the changes and anyone who references a post can reference a particular version of it. In some situations it may make sense to use a Wiki server instead of a Blog server for some data to preserve the change history. Maybe I should consider a wiki in which only I have write access for my documents repository. No-one seems to cover the issue of how to deal with fiction. Obviously for a blog such as 365tomorrows.com (a science-fiction blog) it’s assumed that all content is fictional unless stated otherwise. The problem comes when someone includes fiction amongst the non-fiction content of a blog. I recently unsubscribed from a blog feed when the author revealed that what appeared to have been a diary entry was actually a fiction story (and a poor one at that). If your blog has mostly non-fiction then any fiction must be clearly marked, probably including the word “fiction” in the title and the permalink would be required to avoid all potential of confusion. For my documents CMS/blog I am considering having a category of “fiction” and to use the category name in the permalink address of each post. Of course science-fiction is sometimes obvious as fiction, but to avoid problems with fiction that is set in current times I think it’s best to clearly mark all such posts. The old-fashioned media seems to have dealt with this, tabloid womens magazines (which almost everyone reads occasionally as they are often the only reading materiel in a waiting room) traditionally have a fiction story in every issue which is clearly marked. Another type of blogging is corporate blogging. I wonder whether that needs to be covered separately in an ethics code. One thing that the three documents about ethics have in common (as a side-note in each) is the fact that the personal reputation of the writer depends on their ethics. If you are known for writing truthfully and fairly then people will treat your posts more seriously than that of writers who lie or act unfairly. There are direct personal benefits in acting ethically. The RSF document claims that the sole purpose of ethics standards is to instill trust in the readership which directly benefits the writer - not for any abstract reasons. Whenever ethics is mentioned in terms of writing there is always someone who claims that it can’t be enforced, well many people in your audience will figure out whether you are acting ethically and decide whether they want to continue reading your materiel. I wonder whether Planet installations should have ethics codes and remove blogs that violate them. In conclusion I think that a complete code of ethics for blogging needs to have some IF clauses to cover the different types of blog. I may have to write my own. Please let me know if you have any suggestions.

MJ Ray: Sky and Setanta Ad Complaints Highlight Differences

Both Sky and Setanta have had complaints upheld against their adverts by ASA this week. Sky for misleading criticism of Virgin Media's new-customer offers (Sky runs new-customers offers too) and Setanta for misleading advertising of their Premiership football package (or formally, Sky's "ad breached CAP Code clauses 3.1 (Substantiation), 7.1 (Truthfulness), 15.1 (Prices) and 18.1 (Comparisons)" and Setanta's "ad breached CAP Code clause 7.1 (Truthfulness)" ). But it's more interesting to look at who made the complaints:-
"Sky TV believed the [Setanta] ad was misleading"
but
"one complainant who was already a Sky customer"
and
"an existing customer"
were among the five complaints about Sky's ad. Sky's biggest critics seem to be their own customers. That's gotta hurt.

Rob Bradford: Anjuta 2.2.2

Packages for Anjuta 2.2.2 were uploaded to Debian unstable yesterday. I also built packages for Ubuntu’s ‘gutsy’ and put them in a PPA on Launchpad. The packages currently available in ‘gutsy’ are liable to crash due to issues with the plugins. The sources.list line for these packages is:
deb http://ppa.launchpad.net/robster/ubuntu gutsy universe

23 October 2007

Raphael Geissert: Homepage as a control field debian/control stats

I think by now everybody knows that Homepage: is now a valid debian/control field.

So, let's see how many packages in sid/main/binary-i386 are using Homepage as a description field and how many as a control field.


We fetch Packages.bz2 and unpack...
$ wget http://localhost:3142/ftp.us.debian.org/debian/dists/sid/main/binary-i386/Packages.bz2
$ bunzip2 Packages.bz2 


Now, we count the number of packages using Homepage as a control field:
$ egrep "^Homep" < Packages   sort -u   wc -l

(note the usage of sort -u because I'd say packages with the same Homepage come from the same source package).

So we have our first number: 335
Doesn't look bad, does it? :)

But let's compare it with this:
$ egrep "^[ \t]+Homep" < Packages   sort -u   wc -l


Our second number is: 2716
Wow, that's a big difference.

But hey? how many packages are out there?

$ wget http://localhost:3142/ftp.us.debian.org/debian/dists/sid/main/sources/Sources.bz2
$ bunzip2 Sources.bz2
$ egrep "^Package:" Sources  wc -l


Our third number is: 11820

So, if we make those number percentages:

Norbert Tretkowski: Backport of MySQL 5.1 package for etch

For those of you who want to use MySQL 5.1 on etch, I prepared a backport from the experimental package of MySQL 5.1.22-rc. It's available for amd64 and i386, but will not be uploaded to the backports.org archive until there's a stable release in lenny.

Take a look at the list of Open Issues in MySQL 5.1 before upgrading your servers.

Simon Richter: Folding Ubuntu's changes into Debian

Once again there is a discussion whether uClibc based systems should get their own architecture, or whether it would be more like Ubuntu -- they, too, have a different C library (with stack-protector support; this simplifies the gcc bootstrap sequence a bit). Thinking further, why not have generic support for keeping different configurations for various derived distributions in a single source tree, so the same source package could be used on Debian, Ubuntu, emdebian and all the others which all build different binaries? The gcc package does this, to some extent, but it looks at the build system to find out which configuration to use -- a standard mechanism, for example a DEB_BUILD_VARIANT environment variable would be nice.

Christian Perrier: Give credit where due...

I recently ranted about Ubuntu not giving much credit to Debian. I have to complete this by thanking the Merge-o-Matic feature which (for a few months now) sends to package maintainers a diff file with the Debian package when a new Ubuntu package is uploaded. This sometimes make it quite easy to spot modifications done by Ubuntu and eventually adopt them. This already bringed me, as maintainer, a few interesting patches for iso-codes, console-data and geneweb. In the case of iso-codes, samba and geneweb, the Ubuntu patch was even reported manually as a Debian bug by a Ubuntu contributor/developer (no later than today, Martin Pitt took care to suggest us to adopt bzip2 compression for iso-codes and Mathias Gug proposed a patch to enable usershares in samba). So, well, even if things can still be improved, many Good collaboration also often happens.

Wouter Verhelst: Automounters

You know the problem? Something happens, and all your systems go down—scheduled or otherwise. When you boot them up, one box is faster than the other; and as a result, some network mounts don't really work correctly. While this doesn't really happen at work, I've seen it after a power failure at home. I thought this was a feature that Windows had but Linux doesn't. I really should've known better...
wouter@country:~$ mount
/dev/hda5 on / type ext3 (rw,errors=remount-ro,commit=0)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
(...)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
automount(pid15173) on /autofs type autofs (rw,fd=5,pgrp=15173,minproto=2,maxproto=4)
wouter@country:~$ ls /mnt/stuff
(lots of private data snipped...)
wouter@country:~$ mount
/dev/hda5 on / type ext3 (rw,errors=remount-ro,commit=0)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
(...)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
automount(pid15173) on /autofs type autofs (rw,fd=5,pgrp=15173,minproto=2,maxproto=4)
goa:/stuff on /autofs/stuff type nfs (rw,addr=192.168.119.23)
And after I wait a minute, that last line is gone. Automounters are cool. And not too hard to set up, either, if only a bit confusing at first. My /etc/auto.master contains the following (not counting commented-out examples):
/autofs	/etc/auto.autofs --timeout=60
I then need to have a directory /autofs, on which the automounter mounts a special filesystem if it's started; this is the "automount" in the above mount output. The autofs needs a second automounter map, which looks like this in my case:
cdrom	-fstype=iso9660,ro,nosuid,nodev	:/dev/cdrom
stuff	-fstype=nfs			goa:/stuff
camerab	-fstype=vfat,ro,umask=133,dmask=22 :/dev/disk/by-id/usb-GENERIC_USB_Storage-SDC_200606191719424ND-0:1-part1
floppy	-fstype=auto			:/dev/disk/by-id/usb-Y-E_DATA_USB_Floppy_Drive
Occasionally, yes, from the above lines it can be inferred that this indeed also works for my USB floppy drive, my multicardreader, and my CD-ROM drive. Finally, I need a symlink into the automount filesystem, like so:
wouter@country:~$ ls -l /mnt/stuff
lrwxrwxrwx 1 root root 13 2007-10-23 19:21 /mnt/stuff -> /autofs/stuff
et voila: if I now try to access anything below /autofs/stuff, the automounter kicks in, transparently mounts the filesystem, and without any further delay or ado, I can get at my files. Well—unless the server is down, that is.

Adeodato Sim : Coming out (the bipolar version)

I’d like to make somewhat more visible what some of you already know, that I have bipolar disorder. I was diagnosed a bit more than a year ago. I’m doing this mainly because I think visibility matters. First and most important, because in my experience it can be helpful and comforting to hear about other people whose names you recognize that can relate to your situation. And second, in case it helps a tiny bit in detabooizing mental ilnesses and their drugs. (But fortunately geek circles seem to need less of that than other, more prejudiced environments.) On a more personal note, I can’t but be grateful after six months of straight euthymia, during which I’ve regained control of my life, written code, attended events, and retaken long term projects. I also feel, after many years, less fearful of the future. And finally, let me add I can’t imagine having made through it without one particular friend, who doesn’t even read this blog. Should everybody be so lucky.

Wouter Verhelst: Printer on fire

Seems this message (which I have seen once or twice on my Linux systems) originated from a real failure mode in which the printer could, indeed, catch fire. Whoa. Mean time, my own old Kyocera FS-1500 is dead; the drum is damaged to a level that confuses the printer somehow, so that it now always thinks there's no toner left. If this were any type of modern printer, I'd just replace the drum. In this particular case, that's not going to happen, though; this printer is from 1992, and neither the toner container nor the drum are replaceable; instead, you have to clean the drum with some device that's part of the toner kit, and you have to refill the toner container using the contents of a box of toner that's inside the kit. Trust me, toner powder is messy.

MJ Ray: the Cooperative Group Half-Yearly Meeting: Q+A

the co-operative half-yearly meeting, Bristol Belatedly, as promised, here are the questions from the floor at last week's half-year meeting of the Co-operative Group Kennet and Avon area. All errors and omissions are mine, of course.
How do we defend against demutualisation and private equity attacks?
Our rules are now written so that demutualisation would require assets to be transferred to other co-operatives. Please comment on other ways to the current constitutional review consultation.
How can ordinary members get informed and involved in running the group?
Get items into network, call the regional office.
The consitutional review should give more power to meetings and make it easier for motions and binding promises.
will forward this to the review.
Why did the group secretary suddenly leave?
The job had changed post-merger, with different responsibilities.
How can we stop the payments to voluntarily-retiring directors?
The area has made its point. The chief executive of CooperativesUK will look into this and there was a congress motion to look into this issue.
There's been no mortgage advisor in one local bank branch for months. Is this indicative of the bank's attitude to mortgages?
No, it's disappointing. Please send details to the area secretary.
Please can we use geographic numbers instead of the expensive 0845 for the membership enquiries?
Will forward and ask for a geographic number.
Is the constitutional review being rushed?
It's streamlined because the merger gives a pressing need, but we want member input.
Can we or do we promote credit unions in-store?
Yes, we can and do promote them to members already. Can set aside some space in-store if needed.
Does the co-operative travel insurance discriminate against over 75s?
will forward the answer when we have it.
The top management post-merger are all from United: is this a take-over?
no, 2 executives had clear intentions to retire already and the lower levels are more mixed.
Which? gave the co-operative food a D for organic produce: why and what will we do about it?
Report from today's Guardian not yet seen. Expect to respond in next few days. The report may have compared convenience stores with larger stores.
Will ex-United travel agents become the co-operative travel stores?
Yes. There's a big big change coming, to be more of a community travel agent, restructured, with more phone and internet options.
Is the energy-saving policy to include refrigeration, lighting and IT?
This is a top goal, to improve refrigeration and lighting control. It saves the earth and saves money. Nearly all electricity is already "greener" electricity.
Do we own the freehold on the land previously occupied by a store at Shirehampton?
Yes. Only a non-food retailer will be allowed in.
Why do elected representatives have to retire at 70 if still capable?
Please submit comments on it to the constitutional review?
Why was there no transport from Weston-super-Mare?
Will check the office records for requests.
Anyone see these themes appear at other meetings? Any comments on these themes?

Enrico Zini: axi-query-pkgtype

apt-xapian-index: adding simple result filters to the query I've recently posted an introduction of apt-xapian-index and an example of how to query it. Today I'll show how to add simple Debtags-based result filters to a query. I'll expand axi-query-simple.py with a --type comman line option that allows to filter for package type. As a starter I'll implement filtering for "game", "gui", "cmdline" or "editor". If you feel like reimplementing my examples in another language, let me know and I'll include it to the post. The filters are implemented using Xapian queries. Xapian allows to build queries as well as combine existing queries using a large amount of operators. This allows us to create a little database of ready made queries:
# This is our little database of simple Debtags filters we provide: the name
# entered by the user in "--type" maps to a piece of Xapian query
filterdb = dict(
    # We can do simple AND queries...
    game = xapian.Query(xapian.Query.OP_AND, ('XTuse::gameplaying', 'XTrole::program')),
    # Or we can do complicate binary expressions...
    gui = xapian.Query(xapian.Query.OP_AND, xapian.Query('XTrole::program'),
        xapian.Query(xapian.Query.OP_OR, 'XTinterface::x11', 'XTinterface::3d')),
    cmdline = xapian.Query(xapian.Query.OP_AND, 'XTrole::program', 'XTinterface::commandline'),
    editor = xapian.Query(xapian.Query.OP_AND, 'XTrole::program', 'XTuse::editing'))
    # Feel free to invent more
It's now trivial to lookup a query and AND it to the main query:
# Build the query by ORing together various terms
query = xapian.Query(xapian.Query.OP_OR, termlist)
# If a filter was requested, AND it with the query
if extrafilter:
    query = xapian.Query(xapian.Query.OP_AND, query, filterdb[extrafilter])
That's it. Simple, trivial feature to add to a search, but very useful and simple to use. You can use the wsvn interface to get to the full source code and the module it uses. Try running it as:
axi-query-pkgtype.py --type=gui image editor
axi-query-pkgtype.py --type=cmdline image editor
axi-query-pkgtype.py --type=game image editor
you will see that the results change radically. What are the games that match "image editor", you would ask?
74% xjig - An X11 jigsaw puzzle
69% wmpuzzle - WindowMaker dock app 4x4 puzzle
66% xflip - programs to mirror-image or melt your display
Jigsaws, of course :) The code is now colourised thanks to Simon Ward who pointed me at the syntax plugin for ikiwiki. Next in the series: suggesting new terms for improving the query.

Isaac Clerencia: Open aLANtejo 2007

Last weekend I had the chance to attend the Open aLANtejo conference, in beautiful vora, Portugal. I went there to give a presentation about Open Source Game Development and had the chance to meet interesting people that I knew through somebody else or just from the community lore :P To be honest I had a really better time than I expected :) Some of the speakers were the Bitrock people, Jono Bacon (Ubuntu Community Manager) and Juanjo from Igalia. I’m really looking forward to meet up again. The organisation team was also really helpful and passionate about the event, so if you get invited, don’t think twice about going :)

Raphael Geissert: Nice badvista photo

I just came across this photo (which, by the way, is in public domain):



P.D. sorry about the size, but bigger looks better :)

22 October 2007

Joerg Jaspert: Clusterssh

Some people did ask me to add a “tech” category to my blog, and randomly post about useful little things that help making life easier. Well, Im not making a dedicated category for it, but one where I will put all my not-strictly-debian related “tech” content in. Up to now my blog only knew a debian category… :) Well, so the first thing i have a little to write about is clusterssh. It is similar to dsh, with the difference that clusterssh opens an xterm for every host you want to control, while dsh simply takes one commandline and executes it on the remote host(s). This way clusterssh allows you to step in and control a single host, if that one needs a little extra action, while keeping the connection to all other hosts open. After you are done with the extra action - just go back to control all hosts at once. Very helpful if you edit configuration files, and one hosts needs a little different value, while most stuff is the same. Or doing security upgrades and find out one of your colleagues was so “clever” to install an extra package which now makes aptitude want to remove half of your system. You can step in and fix the situation and then simply continue with all hosts. Usage of clusterssh is pretty simple - after you installed it type
  cssh machine1 machine2 machine3
and clusterssh will open 3 xterms and one control window. Whatever you type into that control window is then send to all 3 machines. cssh example1 As you can see in the picture I first executed one command on both hosts, by typing it into the CSSH window, and then selected one host directly, typing a command only this host sees. Now, we are all lazy, so lets make cssh easier to use. Noone wants to type hundreds of hostnames all the time. Luckily clusterssh has a concept where you map tags to servernames. If you are the admin of your machine and want to setup such a mapping for all of your users you edit /etc/clusters, if you just want to do it for yourself edit $HOME/.csshrc. Actually - do not edit it, but run
  cssh -u >$HOME/.csshrc
and edit the generated file. Ignore all of the config stuff for now, defaults are nice and good. Go to the bottom and add something similar to the following lines:
 clusters = foo bar
 foo = miau wuff:2222 root@milk:98 root@elsewhere
 bar = root@miau:443 web@ftp:443
Replace with hosts you can actually login to. The above defines two set of hosts, foo and bar. Now all I need to do to login to the defined machines is to type
 cssh foo
and clusterssh will open 4 xterms and login to the machines. At work I usually have up to 200-250 machines open with clusterssh, so it can handle large numbers of hosts at once. In such cases you might want to consider adding the -G parameter to the commandline, to not get clusterssh to resize the xterms. A last thing I like to do is - give the xterms opened by clusterssh a different layout. My window manager is setup to draw all my terminals maximized and also has some other settings that do not really help clusterssh terminals . But there is a simple way round this. Edit the following options in your .csshrc to look similar like
terminal_allow_send_events=-xrm '*.VT100.allowSendEvents:true'
terminal_args=-class CSSH
This will put all xterms into the class CSSH. As your normal settings usually apply to class XTerm this already ignores them. If you want to setup some more things, like a blue background and red foreground in CSSH windows, just add the following into your $HOME/.Xdefaults file (or do whatever steps your window manager might like to configure such things):
CSSH*background: Blue
CSSH*foreground: Red
Activate with xrdb -load ~/.Xdefaults, done.

Bernd Zeimetz: short package update

short package update Didn't find the time to blog lately, so here's the very short summary of (hopefully) all news regarding the more important packages I take care of: gimp-plugin-registry / gimp-refocus gimp-plugin-registry is at version 0.4.2-1 now, including several updates and new plugins since my last blog entry. Also I've moved the refocus plugin into gimp-plugin-registry - now gimp-refocus is only a transitional package, which will install gimp-plugin-registry. gimp-refocus will be removed from Debian before Lenny will be frozen. As it was not shipped with Etch there's no need keep the transitional package in the release. gpsd Successfully hijacked gpsd, the new upload fixed a lot of bugs, including one old RC bug which was the reason why gpsd was removed form testing. If you're using gpsd please check the packages in unstable - comments appreciated. If you've used libgps form the gpsd package before, you'll have to look into the libgps-dev and libgps16 packages now. Bugs against packages using the library were filed in time. Also I plan to get a beta version of 2.35 into experimental during the next weeks, which will come with a more or less completely rewritten NMEA layer and a lot of fixes and improvements. radiance Finally integrated in Debian! Development of radiance started in 1985 and is still actively developed, but nobody every thought about including it in Debian. I'll blog about radiance later, in the meantime please read this wikipedia article. viking Today the latest version 0.9.3 was uploaded to unstable. Most important addition: Realtime GPS tracking, using gpsd. If you're interested in OpenStreetMap, geocaching or any other fun with GPS devices, take a look into viking.

Simon Richter: How "WTF" code is produced

You cannot store a CComPtr inside an STL list. You can, however, wrap it in a struct, and store that in the list. Careful study of the error message led to the conclusion that apparently some STL implementations frown upon classes that provide an unary operator&.

Lior Kaplan: Status of the CVS bug triage

The CVS bug triage was started two and a half weeks ago. Considering the time frame, it’s going great. I started with 145 open bugs, and 3 forwarded to upstream (which doesn’t keep track of old bugs, AFAIK). And now only 53 (about 1/3) of the bugs still wait for confirmation about their status. During the triage, 38 bugs were closed and around 50 bugs were verified to still be relevant. All these numbers means that CVS has 57 bugs which are pending and verified to occur in the latest version. At least 10 of them can be solved quite easily as they have patch or an explanation good enough to be considered as a patch. The rest should probably require more work from the maintainers and upstream. During the triage, a lot of work was invested in reproducing bugs when the submitters said they can’t do that themselves anymore (not using CVS or not using Debian). Another bunch of of bugs got better tags (mostly patch tag) and up-to-date version info to ease the maintainers’ future work on the package. The bugs waiting for verification bugs were user tagged so they can be followed easily. Help with those is always welcome.

Holger Levsen: Small report from the meeting in Extremadura

The last five days I've spent at the combined Debian Edu, FAI and CDD meeting in M rida in Extremadura, together with 17 others. The Debian Edu / Skolelinux crowd was the biggest, followed by FAI, followed by CDD people, though many were part of more than one project. "As usual" it was great to meet many known faces and some unknown ones and to be able to discuss stuff in RL.

Besides various small different things, I worked on the Debian Edu etch r1 release, our lenny roadmap (see the bottom of our roadmap page which also covers etch r1) and the (technical side of the) german translation of the Debian Edu documentation. Non edu related the new setupharddisk2 for FAI took most of my brain activity. And I've signed all keys allready :-)

We stayed at a very nice hotel in the even much nicer city of Merida, which features a lot of roman ruins and history. For example, we walked over the biggest roman bridge in the world! The only thing that was bigger was the amount of good food we were forced to eat every day... Incredible. This was my second visit to Merida and this time I finally managed to visit the amphitheatre (with seats for 15000 people) and the Roman theatre. (Unfortunatly the english page about Emerita Augusta is a stub.) If you happen to come around that area make sure to spend some time sight seeing, it is really worth it! People living in Merida literally fear to rebuild their houses, because as soon as they start to dig, they find something historical and have to turn their property into a museum :-)

In case you have not seen it yet, you should watch this small movie from euro news about the free software efforts in Extremadura. Insightful and very good promotion, also to show around.

Many thanks to C sar, Jos Luis and the Junta de Extremdura for hosting such a great meeting! You rock!

On a unrelated note, spam has gone crazy yesterday while traveling back, 15000 mails were not catched by my so far well working setup. ARG. I still used to have a catchall setting, which was useful to give out addresses like $company@mydomain, but now I will explicitly list those addresses and disable catchall. And luckily for you I will not blog about every spam mail I got... :) Generally I dont consider spam (the source of) evil, it's money: as long as money rules the world, there will be spam. So since it's a social problem, it is hardly solvable by technology. (That is, as long as you want to receive mails from strangers, which I do.) IMO what's worse than spammers are autoresponders to spam. Most of those 15000 mails were auto-replies to spam. OMFSM.

Enrico Zini: axi-simplequery

apt-xapian-index: performing a simple query I've recently posted an introduction of apt-xapian-index. Today I'll show how to make simple queries to apt-xapian-index. If you feel like reimplementing my examples in another language, let me know and I'll include it to the post. What I'm going to build is a replacement for apt-cache search that: This is just a beginning: in future blog posts I'll show how to enhance a search with interesting advanced features. First thing, we need to import the Xapian module, found in the package python-xapian. Documentation on the Python Xapian API can be found in /usr/share/doc/python-xapian.
import xapian
Then we open the apt-xapian-index database:
# Instantiate a xapian.Database object for read only access to the index
db = xapian.Database("/var/lib/apt-xapian-index/index")
Now we build a query from the command line arguments. We'll assume that if an argument is in the form foo::bar, then it's a Debtags tag instead of a normal keyword. For normal keywords, we also search for the stemmed version, so that we can, for example, find "editing" when the user searches for "edit".
# Stemmer function to generate stemmed search keywords
stemmer = xapian.Stem("english")
# Build the terms that will go in the query
terms = []
for word in args:
    if word.islower() and word.find("::") != -1:
        # According to /var/lib/apt-xapian-index/README, Debtags tags are
        # indexed with the 'XT' prefix.
        terms.append("XT"+word)
    else:
        # If it is not a Debtags tag, then we consider it a normal keyword.
        word = word.lower()  # The index stores keyword all in lowercase
        terms.append(word)
        # If the word has a stemmed version, add it to the query.
        # /var/lib/apt-xapian-index/README tells us that stemmed terms have a
        # 'Z' prefix.
        stem = stemmer(word)
        if stem != word:
            terms.append("Z"+stem)
Now we have the terms for the query, and we can create a query that ORs them together. One may ask, why OR and not AND? The reason is that, contrarily to apt-cache, Xapian scores results according to how well they matched. Matches that match all the terms will score higher than the others, so if we build an OR query what we really have is an AND query that gracefully degenerates to closer matches when they run out of perfect results. This allows stemmed searches to work nicely: if you look for 'editing', then the query will be 'editing OR Zedit'. Packages with the word 'editing' will match both and score higher, and packages with the word 'edited' will still match 'Zedit' and get included in the results.
# OR the terms together into a Xapian query.
#
query = xapian.Query(xapian.Query.OP_OR, terms)
We then run the query. Queries are run through a xapian.Enquire object:
# Perform the query
enquire = xapian.Enquire(db)
enquire.set_query(query)
The Enquire object returns results as an mset. An mset represents a view of the result set, and can be iterated to access the resulting documents. Here we iterate the mset and output the result of the query, looking up short descriptions with apt:
# Display the top 20 results, sorted by how well they match
matches = enquire.get_mset(0, 20)
print "%i results found." % matches.get_matches_estimated()
print "Results 1-%i:" % matches.size()
for m in matches:
    # /var/lib/apt-xapian-index/README tells us that the Xapian document data
    # is the package name.
    name = m[xapian.MSET_DOCUMENT].get_data()
    # Get the package record out of the Apt cache, so we can retrieve the short
    # description
    pkg = cache[name]
    # Print the match, together with the short description
    print "%i%% %s - %s" % (m[xapian.MSET_PERCENT], name, pkg.summary)
This is it. You can use the wsvn interface to get to the full source code. You can run the code passing keywords and Debtags tags. Try running it as:
    ./axi-query-simple.py role::program image edit
    ./axi-query-simple.py role::program game::arcade
    ./axi-query-simple.py kernel image
You can search Debtags tags using debtags tagsearch. In a later blog post, I'll show how to use apt-xapian-index to implement a better-than-you-would-ever-have-thought-possible tag search. Next in the series: adding simple result filters to the query.

Next.

Previous.