Search Results: "xam"

24 October 2007

Russell Coker: Blog Ethics

Reporters Sans Frontiers (AKA RSF AKA Reporters Without Borders) has an interesting document about blogging [1]. They are specifically focussed on blogging as a way of reporting news. Their definition of a blog states that it is “a personal website” (there are many corporate blogs run by teams) and that it contains “mostly news” (most blogs that I read contain mostly technial information about computers and any “news” is mostly about the author). They also describe a blog as being set up with an “interactive tool” - most blogs are written using web-based tools, but some are written with plain text and a compilation system. Some of their technical information is simply wrong. For example they say that RSS “alerts users whenever their favourite blogs are updated” (this can be done through RPC notification which then requires RSS to pull the data - but a user will almost always have a tool polling their favourite RSS feeds). Their section listing the various options for blogging platforms mentions LiveJournal, Blogger, and MSNSpaces but doesn’t mention Wordpress.com which seems to be a serious omission (although it does mention civiblog.org which is a useful hosting resource for grassroots political campaigning). But these are minor problems, they are reporters primarily not programmers. Their document gets interesting when it gives links to pages about blogging ethics. One page that was not directly linked (presumably because it mainly concerns non-journalistic blogging) is Rebecca Blood’s document about blogging ethics [2]. She gives a brief coverage to conflicts of interest and gives most space to the topic of maintaining a record of changes. One thing I have been considering is having a separate instance of Wordpress for documents that change [3]. This way regular blog posts (such as this one) can be relied on to preserve a list of changes (any change other than correcting a typo will be documented) so if you link to a post you can rely on the original intent being preserved. But for posts which have ongoing value as live documents they will be kept current without a significant change-log. Items that go into the main blog would include commentary on news and predictions, and the document blog would contain technical information about Linux, the science-fiction stories that I will eventually write, etc. When I wrote my previous blog post about this issue I was mainly considering technical issues, but when considering the ethical issues it becomes clear that I need a separate blog (or other CMS) for such things. The site etbe.coker.com.au needs to be a reliable record of what I wrote while another site such as docs.coker.com.au could contain live documents with no such record. The essential factor here is the ability of the user to know what type of document they are seeing. Another really interesting point that Rebecca makes is in regard to known bad sources. Sometimes a known bad source can produce worthy data and is worth referencing, but you have to note the context (which among other things means that if the worthy data gets any reasonable number of hits it may get replaced by something entirely different). If a blogger cites a reference on a known bad site and explains the reason for the reference (and the fact that the nature of the site is known) then the reader reaction will be quite different to just a reference with no explanation. As a contrast to Rebecca Blood’s document the cyberjournalist.net code of ethics [4] covers the traditional issues of journalistic integrity, “public interest” etc. One interesting issue that they raise (but apparently don’t address) is the definition of “private people“. While it is generally agreed that movie stars and similar people deserve less protection as they have chosen to be in the public eye the situation regarding bloggers is less clear. Is a blogger with a high ranking on technorati.com in a similar position to a Hollywood star when it comes to privacy? A claim is sometimes made that blogs are unreliable because they can change (based on the actions of some bloggers who change posts with no notification and turn sections of the blogsphere into a swamp). Magazines and newspapers are held as an example of unchanging content and people who publish online magazines sometimes express horror at the idea of changing a past issue. The fact is that newspapers and magazines never changed past issues because it is impossible to recall hundreds of thousands of printed documents that are distributed around a country or the world. When it’s possible to fix old issues (as it is online) then the requirement is to document the changes not keep bad versions. It probably would be a useful feature for a blogging package to have the ability to display (and link to) old versions of a page. This would allow the users to easily see the changes and anyone who references a post can reference a particular version of it. In some situations it may make sense to use a Wiki server instead of a Blog server for some data to preserve the change history. Maybe I should consider a wiki in which only I have write access for my documents repository. No-one seems to cover the issue of how to deal with fiction. Obviously for a blog such as 365tomorrows.com (a science-fiction blog) it’s assumed that all content is fictional unless stated otherwise. The problem comes when someone includes fiction amongst the non-fiction content of a blog. I recently unsubscribed from a blog feed when the author revealed that what appeared to have been a diary entry was actually a fiction story (and a poor one at that). If your blog has mostly non-fiction then any fiction must be clearly marked, probably including the word “fiction” in the title and the permalink would be required to avoid all potential of confusion. For my documents CMS/blog I am considering having a category of “fiction” and to use the category name in the permalink address of each post. Of course science-fiction is sometimes obvious as fiction, but to avoid problems with fiction that is set in current times I think it’s best to clearly mark all such posts. The old-fashioned media seems to have dealt with this, tabloid womens magazines (which almost everyone reads occasionally as they are often the only reading materiel in a waiting room) traditionally have a fiction story in every issue which is clearly marked. Another type of blogging is corporate blogging. I wonder whether that needs to be covered separately in an ethics code. One thing that the three documents about ethics have in common (as a side-note in each) is the fact that the personal reputation of the writer depends on their ethics. If you are known for writing truthfully and fairly then people will treat your posts more seriously than that of writers who lie or act unfairly. There are direct personal benefits in acting ethically. The RSF document claims that the sole purpose of ethics standards is to instill trust in the readership which directly benefits the writer - not for any abstract reasons. Whenever ethics is mentioned in terms of writing there is always someone who claims that it can’t be enforced, well many people in your audience will figure out whether you are acting ethically and decide whether they want to continue reading your materiel. I wonder whether Planet installations should have ethics codes and remove blogs that violate them. In conclusion I think that a complete code of ethics for blogging needs to have some IF clauses to cover the different types of blog. I may have to write my own. Please let me know if you have any suggestions.

Share This

23 October 2007

Simon Richter: Folding Ubuntu's changes into Debian

Once again there is a discussion whether uClibc based systems should get their own architecture, or whether it would be more like Ubuntu -- they, too, have a different C library (with stack-protector support; this simplifies the gcc bootstrap sequence a bit). Thinking further, why not have generic support for keeping different configurations for various derived distributions in a single source tree, so the same source package could be used on Debian, Ubuntu, emdebian and all the others which all build different binaries? The gcc package does this, to some extent, but it looks at the build system to find out which configuration to use -- a standard mechanism, for example a DEB_BUILD_VARIANT environment variable would be nice.

Wouter Verhelst: Automounters

You know the problem? Something happens, and all your systems go down—scheduled or otherwise. When you boot them up, one box is faster than the other; and as a result, some network mounts don't really work correctly. While this doesn't really happen at work, I've seen it after a power failure at home. I thought this was a feature that Windows had but Linux doesn't. I really should've known better... 
wouter@country:~$ mount
/dev/hda5 on / type ext3 (rw,errors=remount-ro,commit=0)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
(...) 
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
automount(pid15173) on /autofs type autofs (rw,fd=5,pgrp=15173,minproto=2,maxproto=4)
wouter@country:~$ ls /mnt/stuff
(lots of private data snipped...) 
wouter@country:~$ mount
/dev/hda5 on / type ext3 (rw,errors=remount-ro,commit=0)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
(...) 
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
automount(pid15173) on /autofs type autofs (rw,fd=5,pgrp=15173,minproto=2,maxproto=4)
goa:/stuff on /autofs/stuff type nfs (rw,addr=192.168.119.23)
And after I wait a minute, that last line is gone. Automounters are cool. And not too hard to set up, either, if only a bit confusing at first. My /etc/auto.master contains the following (not counting commented-out examples): 
/autofs	/etc/auto.autofs --timeout=60
I then need to have a directory /autofs, on which the automounter mounts a special filesystem if it's started; this is the "automount" in the above mount output. The autofs needs a second automounter map, which looks like this in my case: 
cdrom	-fstype=iso9660,ro,nosuid,nodev	:/dev/cdrom
stuff	-fstype=nfs			goa:/stuff
camerab	-fstype=vfat,ro,umask=133,dmask=22 :/dev/disk/by-id/usb-GENERIC_USB_Storage-SDC_200606191719424ND-0:1-part1
floppy	-fstype=auto			:/dev/disk/by-id/usb-Y-E_DATA_USB_Floppy_Drive
Occasionally, yes, from the above lines it can be inferred that this indeed also works for my USB floppy drive, my multicardreader, and my CD-ROM drive. Finally, I need a symlink into the automount filesystem, like so: 
wouter@country:~$ ls -l /mnt/stuff
lrwxrwxrwx 1 root root 13 2007-10-23 19:21 /mnt/stuff -> /autofs/stuff
et voila: if I now try to access anything below /autofs/stuff, the automounter kicks in, transparently mounts the filesystem, and without any further delay or ado, I can get at my files. Well—unless the server is down, that is.

Enrico Zini: axi-query-pkgtype

apt-xapian-index: adding simple result filters to the query I've recently posted an introduction of apt-xapian-index and an example of how to query it. Today I'll show how to add simple Debtags-based result filters to a query. I'll expand axi-query-simple.py with a --type comman line option that allows to filter for package type. As a starter I'll implement filtering for "game", "gui", "cmdline" or "editor". If you feel like reimplementing my examples in another language, let me know and I'll include it to the post. The filters are implemented using Xapian queries. Xapian allows to build queries as well as combine existing queries using a large amount of operators. This allows us to create a little database of ready made queries: 
# This is our little database of simple Debtags filters we provide: the name
# entered by the user in "--type" maps to a piece of Xapian query
filterdb = dict(
    # We can do simple AND queries...
    game = xapian.Query(xapian.Query.OP_AND, ('XTuse::gameplaying', 'XTrole::program')),
    # Or we can do complicate binary expressions...
    gui = xapian.Query(xapian.Query.OP_AND, xapian.Query('XTrole::program'),
        xapian.Query(xapian.Query.OP_OR, 'XTinterface::x11', 'XTinterface::3d')),
    cmdline = xapian.Query(xapian.Query.OP_AND, 'XTrole::program', 'XTinterface::commandline'),
    editor = xapian.Query(xapian.Query.OP_AND, 'XTrole::program', 'XTuse::editing'))
    # Feel free to invent more
It's now trivial to lookup a query and AND it to the main query: 
# Build the query by ORing together various terms
query = xapian.Query(xapian.Query.OP_OR, termlist)
# If a filter was requested, AND it with the query
if extrafilter:
    query = xapian.Query(xapian.Query.OP_AND, query, filterdb[extrafilter])
That's it. Simple, trivial feature to add to a search, but very useful and simple to use. You can use the wsvn interface to get to the full source code and the module it uses. Try running it as: 
axi-query-pkgtype.py --type=gui image editor
axi-query-pkgtype.py --type=cmdline image editor
axi-query-pkgtype.py --type=game image editor
you will see that the results change radically. What are the games that match "image editor", you would ask? 
74% xjig - An X11 jigsaw puzzle
69% wmpuzzle - WindowMaker dock app 4x4 puzzle
66% xflip - programs to mirror-image or melt your display
Jigsaws, of course :) The code is now colourised thanks to Simon Ward who pointed me at the syntax plugin for ikiwiki. Next in the series: suggesting new terms for improving the query.

22 October 2007

Joerg Jaspert: Clusterssh

Some people did ask me to add a “tech” category to my blog, and randomly post about useful little things that help making life easier. Well, Im not making a dedicated category for it, but one where I will put all my not-strictly-debian related “tech” content in. Up to now my blog only knew a debian category… :) Well, so the first thing i have a little to write about is clusterssh. It is similar to dsh, with the difference that clusterssh opens an xterm for every host you want to control, while dsh simply takes one commandline and executes it on the remote host(s). This way clusterssh allows you to step in and control a single host, if that one needs a little extra action, while keeping the connection to all other hosts open. After you are done with the extra action - just go back to control all hosts at once. Very helpful if you edit configuration files, and one hosts needs a little different value, while most stuff is the same. Or doing security upgrades and find out one of your colleagues was so “clever” to install an extra package which now makes aptitude want to remove half of your system. You can step in and fix the situation and then simply continue with all hosts. Usage of clusterssh is pretty simple - after you installed it type 
  cssh machine1 machine2 machine3
and clusterssh will open 3 xterms and one control window. Whatever you type into that control window is then send to all 3 machines. cssh example1 As you can see in the picture I first executed one command on both hosts, by typing it into the CSSH window, and then selected one host directly, typing a command only this host sees. Now, we are all lazy, so lets make cssh easier to use. Noone wants to type hundreds of hostnames all the time. Luckily clusterssh has a concept where you map tags to servernames. If you are the admin of your machine and want to setup such a mapping for all of your users you edit /etc/clusters, if you just want to do it for yourself edit $HOME/.csshrc. Actually - do not edit it, but run 
  cssh -u >$HOME/.csshrc
and edit the generated file. Ignore all of the config stuff for now, defaults are nice and good. Go to the bottom and add something similar to the following lines: 
 clusters = foo bar
 foo = miau wuff:2222 root@milk:98 root@elsewhere
 bar = root@miau:443 web@ftp:443
Replace with hosts you can actually login to. The above defines two set of hosts, foo and bar. Now all I need to do to login to the defined machines is to type 
 cssh foo
and clusterssh will open 4 xterms and login to the machines. At work I usually have up to 200-250 machines open with clusterssh, so it can handle large numbers of hosts at once. In such cases you might want to consider adding the -G parameter to the commandline, to not get clusterssh to resize the xterms. A last thing I like to do is - give the xterms opened by clusterssh a different layout. My window manager is setup to draw all my terminals maximized and also has some other settings that do not really help clusterssh terminals . But there is a simple way round this. Edit the following options in your .csshrc to look similar like 
terminal_allow_send_events=-xrm '*.VT100.allowSendEvents:true'
terminal_args=-class CSSH
This will put all xterms into the class CSSH. As your normal settings usually apply to class XTerm this already ignores them. If you want to setup some more things, like a blue background and red foreground in CSSH windows, just add the following into your $HOME/.Xdefaults file (or do whatever steps your window manager might like to configure such things): 
CSSH*background: Blue
CSSH*foreground: Red
Activate with xrdb -load ~/.Xdefaults, done.

Sven Mueller: RE Anthonys some fun post

I really dislike posts like (sorry AJ, you are just one example) AJ Towns blog post
“Some fun”. What I dislike? Well, the post lacks critical information: Which slashdot post inspired him? What data is he talking about? How did he turn the data into those graphics? Sorry AJ, your post is just the latest example of this style of post, and I really got frustrated over such posts, this is not meant as a personal attack. Edit:
So to make my wish clear: Please, fellow bloggers, don’t assume that your readers are following your favourite web resources as closely as you do (and with the same specific interests). Explicitly say what you are writing about, reference resources needed to understand what you are doing, at least give readers a chance to find out what you did. In AJs case, it would probably have been enough to reference the /. article or comment which inspired him.

Holger Levsen: Small report from the meeting in Extremadura

The last five days I've spent at the combined Debian Edu, FAI and CDD meeting in M rida in Extremadura, together with 17 others. The Debian Edu / Skolelinux crowd was the biggest, followed by FAI, followed by CDD people, though many were part of more than one project. "As usual" it was great to meet many known faces and some unknown ones and to be able to discuss stuff in RL.

Besides various small different things, I worked on the Debian Edu etch r1 release, our lenny roadmap (see the bottom of our roadmap page which also covers etch r1) and the (technical side of the) german translation of the Debian Edu documentation. Non edu related the new setupharddisk2 for FAI took most of my brain activity. And I've signed all keys allready :-)

We stayed at a very nice hotel in the even much nicer city of Merida, which features a lot of roman ruins and history. For example, we walked over the biggest roman bridge in the world! The only thing that was bigger was the amount of good food we were forced to eat every day... Incredible. This was my second visit to Merida and this time I finally managed to visit the amphitheatre (with seats for 15000 people) and the Roman theatre. (Unfortunatly the english page about Emerita Augusta is a stub.) If you happen to come around that area make sure to spend some time sight seeing, it is really worth it! People living in Merida literally fear to rebuild their houses, because as soon as they start to dig, they find something historical and have to turn their property into a museum :-)

In case you have not seen it yet, you should watch this small movie from euro news about the free software efforts in Extremadura. Insightful and very good promotion, also to show around.

Many thanks to C sar, Jos Luis and the Junta de Extremdura for hosting such a great meeting! You rock!

On a unrelated note, spam has gone crazy yesterday while traveling back, 15000 mails were not catched by my so far well working setup. ARG. I still used to have a catchall setting, which was useful to give out addresses like $company@mydomain, but now I will explicitly list those addresses and disable catchall. And luckily for you I will not blog about every spam mail I got... :) Generally I dont consider spam (the source of) evil, it's money: as long as money rules the world, there will be spam. So since it's a social problem, it is hardly solvable by technology. (That is, as long as you want to receive mails from strangers, which I do.) IMO what's worse than spammers are autoresponders to spam. Most of those 15000 mails were auto-replies to spam. OMFSM.

Enrico Zini: axi-simplequery

apt-xapian-index: performing a simple query I've recently posted an introduction of apt-xapian-index. Today I'll show how to make simple queries to apt-xapian-index. If you feel like reimplementing my examples in another language, let me know and I'll include it to the post. What I'm going to build is a replacement for apt-cache search that: This is just a beginning: in future blog posts I'll show how to enhance a search with interesting advanced features. First thing, we need to import the Xapian module, found in the package python-xapian. Documentation on the Python Xapian API can be found in /usr/share/doc/python-xapian. 
import xapian
Then we open the apt-xapian-index database: 
# Instantiate a xapian.Database object for read only access to the index
db = xapian.Database("/var/lib/apt-xapian-index/index")
Now we build a query from the command line arguments. We'll assume that if an argument is in the form foo::bar, then it's a Debtags tag instead of a normal keyword. For normal keywords, we also search for the stemmed version, so that we can, for example, find "editing" when the user searches for "edit". 
# Stemmer function to generate stemmed search keywords
stemmer = xapian.Stem("english")
# Build the terms that will go in the query
terms = []
for word in args:
    if word.islower() and word.find("::") != -1:
        # According to /var/lib/apt-xapian-index/README, Debtags tags are
        # indexed with the 'XT' prefix.
        terms.append("XT"+word)
    else:
        # If it is not a Debtags tag, then we consider it a normal keyword.
        word = word.lower()  # The index stores keyword all in lowercase
        terms.append(word)
        # If the word has a stemmed version, add it to the query.
        # /var/lib/apt-xapian-index/README tells us that stemmed terms have a
        # 'Z' prefix.
        stem = stemmer(word)
        if stem != word:
            terms.append("Z"+stem)
Now we have the terms for the query, and we can create a query that ORs them together. One may ask, why OR and not AND? The reason is that, contrarily to apt-cache, Xapian scores results according to how well they matched. Matches that match all the terms will score higher than the others, so if we build an OR query what we really have is an AND query that gracefully degenerates to closer matches when they run out of perfect results. This allows stemmed searches to work nicely: if you look for 'editing', then the query will be 'editing OR Zedit'. Packages with the word 'editing' will match both and score higher, and packages with the word 'edited' will still match 'Zedit' and get included in the results. 
# OR the terms together into a Xapian query.
#
query = xapian.Query(xapian.Query.OP_OR, terms)
We then run the query. Queries are run through a xapian.Enquire object: 
# Perform the query
enquire = xapian.Enquire(db)
enquire.set_query(query)
The Enquire object returns results as an mset. An mset represents a view of the result set, and can be iterated to access the resulting documents. Here we iterate the mset and output the result of the query, looking up short descriptions with apt: 
# Display the top 20 results, sorted by how well they match
matches = enquire.get_mset(0, 20)
print "%i results found." % matches.get_matches_estimated()
print "Results 1-%i:" % matches.size()
for m in matches:
    # /var/lib/apt-xapian-index/README tells us that the Xapian document data
    # is the package name.
    name = m[xapian.MSET_DOCUMENT].get_data()
    # Get the package record out of the Apt cache, so we can retrieve the short
    # description
    pkg = cache[name]
    # Print the match, together with the short description
    print "%i%% %s - %s" % (m[xapian.MSET_PERCENT], name, pkg.summary)
This is it. You can use the wsvn interface to get to the full source code. You can run the code passing keywords and Debtags tags. Try running it as: 
    ./axi-query-simple.py role::program image edit
    ./axi-query-simple.py role::program game::arcade
    ./axi-query-simple.py kernel image
You can search Debtags tags using debtags tagsearch. In a later blog post, I'll show how to use apt-xapian-index to implement a better-than-you-would-ever-have-thought-possible tag search. Next in the series: adding simple result filters to the query.

21 October 2007

Jeff Bailey: School, moving, and Leif.

I spent most of yesterday working on my Athabasca challenge courses. On top of two completely overlapping all-week meetings this week, I also have three exams and three papers to do. My bad on the scheduling. One of the assignments is done now, with the outline for the second one looking good. I'm hopeful about getting everything at least past draft this weekend.

And hey, while we're at it, lets move houses! Some of last night was unpacking boxes so that we could use stay here. While the corporate housing was a total economic lifesaver, the pillow-top bed was so soft that i would wake several times through the night with pretty severe back pain. We now have pur posturepedic back and i'm much happier this morning!

I also put leif on the toilet for the first time this morning! It's something angie's been doing since we were in montreal, but this is the first time that I caught the signs right and took him over to the toilet. It's nice to save a diaper here and there. I think she's usually putting him on the toilet at least once per day.

Russell Coker: Open Hardware - What Would Change

I have just read an interesting post speculating about the possibility of open source hardware [1]. To some extent things have been following a trend in that direction. Back in the bad old days every computer manufacturer wanted to totally control their market segment and prevent anyone else from “stealing their business”. Anti-competitive practices were standard in the computer industry, when you bought a mainframe you were effectively making a commitment to buy all peripherals and parts from the same company. The problems were alleviated by government action, but the real change came from the popularity of PC clones. White-box clones where every part came from a different company truly opened up the hardware development, and it wasn’t all good. When running a simple single-tasking OS such as MS-DOS the problems were largely hidden. But when running a reliable multi-tasking OS such as Linux hardware problems became apparent. The PCI bus (which autoconfigured most things) reduced the scope of the problem but there are still ways that white-box machines can fail you. Now when I get a white-box machine I give it away to members of my local LUG. My time is too valuable to waste on debugging white-box hardware, I would rather stick to machines from IBM and HP which tend to just work. Nowadays I buy only name-brand machines, all the parts were designed and tested to work together - this doesn’t guarantee that the machine will be reliable but it does significantly improve the probability. Fortunately modern hardware is much faster than I require for the work I do, so buying second-hand name-brand machines (for less money than a new white-box machine) is a viable option. The PCI bus [2] standard from Intel can be compared to some of the “Open Source” licenses from companies where anyone can use the software but only one company can really be involved in developing it. One significant impediment to open hardware development is the proprietary nature of CPU manufacture. Currently there are only a few companies that have the ability to fabricate high-end CPUs so projects such as OpenRISC [3] which develop free CPU designs will be limited to having their CPUs implemented with older technology (which means lower clock speeds). However this doesn’t mean that they aren’t useful, tailoring factors such as the number of registers, the bus width of the CPU, and the cache size to match the target application has the potential to offset the performance loss from a lower clock speed. But this doesn’t mean that an OpenRISC or similar open core would be ideal for your typical desktop machine. If companies such as Intel and AMD were compelled to fabricate any competing CPU design at a reasonable cost (legislation in this regard is a possibility as the two companies collectively dominate the world computer industry and it would be easy for them to form a cartel) then designs such as OpenRISC could be used to implement new CPUs for general purpose servers. Another issue is the quality of support for some optional extra features which are essential for some operations. For example Linux software RAID is quite good for what it does (basic mirroring, striping, RAID-5 and RAID-6), but it doesn’t compare well with some hardware RAID implementations (which actually are implemented in software with a CPU on the RAID controller). For example with a HP hardware RAID device you can start with two disks in a RAID-1 and then add a third disk to make it a RAID-5 (I’ve done it). Adding further disks to make it a larger RAID-5 is possible too. Linux software RAID does not support such things (and I’m not aware of any free software RAID implementation which does). It would certainly be possible to write such code but no-one has done so - and HP seem happy to make heaps of money selling their servers with the RAID features as a selling point. Finally there’s the issue of demand. When hardware without free software support (such as some video cards which need binary-only drivers for best performance) is discussed there is always a significant group of people who want it. The binary-only drivers in question are of low quality, often don’t support the latest kernels, and have such a history of causing crashes that kernel developers won’t accept bug reports from people who use them, but still people use them. In the short-term at least I expect that an open hardware design would deliver less performance and in spite of the fact that it would have the potential to offer better reliability the majority of the market would not accept it. The production volume of electronics gear is the major factor determining the price so it would cost more. I think that both IBM and HP provide hardware that is open enough for my requirements, they both have engineers working on their Linux support and the interfaces are well enough documented that we generally don’t have any problems with them. Both Intel and AMD and the major system vendors are all working on making things more open, so I expect some small bug significant improvements in the near future.

Share This

20 October 2007

Jan Wagner: wordpress plugin and widget packages

Yesterday I was going through the plugins and widgets, if there are any updates or such. I realized again, that updating of plugins (and maybe searching for the relocated download source) makes me mad. Also testing and switching back is annoying much.
So I decided to package plugins and widgets I use. Since some plugins and widgets are not maintained long enought for any debian release cycle or getting obsolet with new wordpress releases, I decided to maintain the packages outside of the debian project. Anyways … some people are also scared about the security situation of wordpress. I guess this isn’t much better for wordpress extensions. Next pity, some extensions or any equivalents aren’t available for wordpress 2.3 (yet). If you run multiple blogs via the stock debian package and wan’t to make use of the new wordpress version in any of your sites, you have update all at once … which maybe not a way you want to go. I decided to repackage wordpress 2.3.0 as wordpress23, which allows you to install wordpress (<<2.3.0) and wordpress23 at same time. If you want to use debian plugin, widget or theme packages or use these from your normal wordpress installation (/usr/share/wordpress) with wordpress23, you need to symlink the dirs. For example:

# mv /usr/share/wordpress23/wp-content/plugins/ /usr/share/wordpress23/wp-content/plugins.orig
# mv /usr/share/wordpress23/wp-content/themes/ /usr/share/wordpress23/wp-content/themes.orig
# ln -s /usr/share/wordpress23/wp-content/plugins /usr/share/wordpress/wp-content/plugins
# ln -s /usr/share/wordpress23/wp-content/themes /usr/share/wordpress/wp-content/themes Remember that package updates of wordpress or wordpress23 may overwrite parts of /usr/share/wordpress/wp-content/[plugins themes] in such situation. BE CAREFULL! You can get the stuff via apt-line:

deb ftp://ftp.cyconet.org/debian wordpress main non-free contrib Or directly from: http://ftp.cyconet.org/debian/archive/wordpress/ The actual package list is: wordpress-plugin-af-extended-live-archive
wordpress-plugin-ald-spa
wordpress-plugin-all-in-one-seo-pack
wordpress-plugin-exec-php
wordpress-plugin-google-sitemap-generator
wordpress-plugin-landingsites
wordpress-plugin-lightbox-2
wordpress-plugin-now-reading
wordpress-plugin-page-links-to
wordpress-plugin-photopress
wordpress-plugin-related-posts
wordpress-plugin-rolemanager
wordpress-plugin-sideblog
wordpress-plugin-spam-karma
wordpress-plugin-stats
wordpress-plugin-ubernyms
wordpress-plugin-utw
wordpress-plugin-utwgooglesitemaps
wordpress-plugins
wordpress-widget-category-cloud
wordpress-widget-samsarin-php
wordpress-widget-utw-cloud
wordpress-widgets
wordpress23 If you want a plugin or widget included and it makes sence to do so, drop me a mail, where to grab. Be aware that UTW isn’t supported since wordpress 2.3 anymore. A tag system was included into wordpress and you need to search replacements for all the nice UTW extension! Stay tuned! , , , , ,

Adam Rosi-Kessel: Multitasking and Classical Music

Two recent print-media articles begging to be linked: First, the Autumn of the Multitaskers in this month’s Atlantic. Despite the age-of-Internet theme, the full article is only available to subscribers. The thesis is one I’ve seen before (e.g., in GTD). Namely: although we might feel like we’re accomplishing more when multitasking, we actually end up less productive, and eventually we’ll lose our minds if we continue to overload our sense. The writer takes the idea further than I’ve seen before, incorporating several astonishing indicators from the zeitgeist, including this NSFW quote from Jennifer Connelly that I won’t reprint, and several foreign policy and domestic political examples of how multitasking is failing our culture and our country. There is a generational divide at work here. Most in previous generations (say, people over 40) didn’t come online until the Internet was nearly full-blown with spam, entertainment news, popups, advertising, streaming stock quotes, rich media services, IM, Facebook, Flash, etc.. These people generally deal poorly with information and sensory overflow because they had no incremental introduction. The younger generations (people under 25) don’t remember the world before the Internet and multipurpose portable devices, and thus think it entirely normal to watch TV, surf the web with a few open windows, talk on the phone (perhaps VoIP), and IM at the same time. At least. And they’re bored if they don’t have at least have three or four tasks going simultaneously. This isn’t to say that these gen-Yers are actually doing a good job at it — I find myself often answering computer-related questions from my younger siblings because they can’t actually read a focused page or two of dense text on the computer (e.g., a readme file) in one sitting — but rather that they don’t experience the flow as psychologically overwhelming. That leaves my (in between) generation, generally people born in the 1970’s. Those of us who grew up with early low-bandwidth wide area networks — I started on BITNET, USENET, and BBS’s in the early- to mid-1980’s — have better mechanisms for coping and prioritizing. Perhaps because our multitasking options were relatively limited, we haven’t leaped on the bandwagon to the same extent as others. I personally don’t mind two simultaneous tasks — e.g., perusing RSS headlines and listening to MP3s — but more than that doesn’t interest me. (I don’t count it as “multitasking” in the same sense where one of the tasks is just waiting, e.g., for code to compile or a large upgrade to download). Please feel free to destroy my broad generalizations here in the comments. While we’re on the subject of multitasking, I fall into the “subtle exception” category described in Getting Things Done:
The Multitasking Exception There’s a subtle exception to the one-item-at-a-time rule. Some personality types really need to shift their focus away from some task for at least a minute in order to make a decision about it. When I see this going on with someone, I let him take two or sometimes three things out at once as he’s processing. It’s then easier and faster for him to make a choice about the action required.
I find this trick particularly helpful in getting unstuck. Frequently, when drafting briefs, I’ve rewritten a sentence eight or nine times and it still doesn’t seem to get across the necessary idea efficiently and without making the reader do any hard work. Switching to another (usually simpler) task, if only for one or two minutes, is often enough to come back and do it right in one swoop. In this situation (and probably only in this situation), multitasking actually saves me time. Second, the Well Tempered Web (by Alex Ross, music critic with a blog) in this week’s New Yorker. This one is more naturally available in complete form online. Ross manages to weave together Wagner, Snakes on a Plane, the Long Tail, and Rick Rubin. Worth reading. (And while you’re at it, check out Adam Gopnik’s piece on abridgments, which unfortunately is not available online.) Lest you all think all I read is the New Yorker and the Atlantic: it’s not true. I’m just not at liberty to disclose my other sources at this time.

Share This

Holger Levsen: mediawiki diffs and lazyweb

Dear lazyplanet, I follow two mediawikis via reading the changes with RSS, and I cannot really follow them well, as the diffs are not very readable. For moinmoin wikis this is no issue, there are nice diff mails one can subscribe to. Does anybody know a way to configure mediawiki, so that it's possible to read the diffs? Or, an example of such a setup? If you do, please contact me. Thanks.

Enrico Zini: apt-xapian-index

Introducing apt-xapian-index apt-xapian-index has just been approved into experimental, and in the next days I'm going to blog more about it. The package contains a tool called update-apt-xapian-index that indexes Debian package metadata into a Xapian index located at /var/lib/apt-xapian-index/index. The index is read-only, except for update-apt-xapian-index; however, it is world-readable: every user can query it, all the time, at the same time, even during index updates. The index can contain more than package descriptions. update-apt-xapian-index indexes data using plugins located in /usr/share/apt-xapian-index/plugins, and any package can add their own. For example, debtags will provide a plugin to index tags. Since Xapian can index numeric values as well, if anyone makes a popcon package that downloads popcon information, they can provide a plugin to index popcon values. If anyone makes an iterating package that downloads ratings, they can provide a plugin to index ratings. Another plugin could be a specialised Debian stemmer that generates token such as foo out of libfoo-dev. I think you get the idea: it's very extensible. You can have a look at the initial set of plugins in subversion. The index is also self-documenting, so that one can keep track of all the intresting things that can be found in it. update-apt-xapian-index does not only maintain the index, but also the file /var/lib/apt-xapian-index/README that aggregates documentation provided by the plugins. To query the index, you just use Xapian. Debian contains Xapian bindings for various languages: In the next days I am going to post various example queries and interesting tricks that the index allows you to do. It's going to be fun.

19 October 2007

Ian Haken: Long Time, No Update

So I've clearly neglected giving frequent updates, so I'll try to summarize everything that's been going on and maybe try to post more frequently in the future (I feel like I've said that many times before).

The summer ended very well, I think. My GSoC project vlosuts concluded with a fully working version, and so hopefully some guys over at Quality Assurance can be goaded into giving it a try. Once Lars and I find some time (it seems like both our lives have been quite hectic) we'll be submitting it to the unstable branch. I intend to continue supporting and developing it; in fact last night I just checked in the first update for VirtualBox support, as recommended by Jon in my comments. It does seem to work faster than QEMU and Innotek did a quality job (in my opinion at least) of packaging it. The biggest kink is that it does not support raw disk images, and so vlosuts gets slowed down by having to convert to VMDK and back, for which I am using qemu-img (and does anyone know why VBoxManage unregisterimage doesn't seem to be working for VMDKs?). If it's possible to do a loopback mount of VMDK (without VMware's utility) it would probably make more sense to use VMDK instead of a raw image to begin with, but I'm not familiar with any way to do this. At any rate, first version of VirtualBox is there for anyone who'd like to give it a shot.

Anyone not concerned with the personal going-ons of my life should stop reading right about now. Since my semester started up again at the end of August, I've pretty much hit the ground running. I'm taking 20 units (I think the average is around 15), one of which is a mathematics graduate course. I'm also doing a research project in computer science, working a part time job (which I try to do for about 10 hours a week), and getting my graduate school applications taken care of. For anyone who is curious, the GRE is a terrible thing. Graduate school and fellowship applications have been a particular headache, but fortunately I'm just about finished up with the essays. For the curious I'll be applying to mathematics departments with an emphasis in logic and foundations at the following universities (list subject to change frequently): MIT, Berkeley, Stanford, Carnegie Mellon, Urbana-Champaign, UMich.

Enrico Zini: filtering-planet

Filtering planet entries Here is how to setup liferea not to show me some entries in Planet Debian:
  1. Create a script that reads the rss from stdin, removes the entries you don't want and then writes the rss to stdout;
  2. From the feed properties in liferea, choose the source tab, enable the conversion filter and point that at your script.
Now you just need a simple script that filters the RSS. Here is mine: 
#!/usr/bin/python
# Copyright (C) 2007 Enrico Zini <enrico@debian.org>
# This software is licensed under the therms of the GNU General Public
# License, version 2 or later.
import libxml2, re
# What links we should filter out
unwanted = re.compile(r"^(http://feed1.example.com http://feed2.example.com)")
doc = libxml2.parseFile("-")
root = doc.getRootElement()
# Create an xpath context and register the namespaces
xpc = doc.xpathNewContext()
for d in root.nsDefs():
    if d.name == None:
        xpc.xpathRegisterNs("rss", d.content)
    else:
        xpc.xpathRegisterNs(d.name, d.content)
# Remove unwanted items from the channel list
for x in xpc.xpathEval("/rdf:RDF/rss:channel/rss:items/rdf:Seq/rdf:li"):
    res = x.nsProp("resource", "http://www.w3.org/1999/02/22-rdf-syntax-ns#")
    if unwanted.match(res):
        x.unlinkNode()
        x.freeNode()
# Remove unwanted items from the item list
for x in xpc.xpathEval("/rdf:RDF/rss:item"):
    res = x.nsProp("about", "http://www.w3.org/1999/02/22-rdf-syntax-ns#")
    if unwanted.match(res):
        x.unlinkNode()
        x.freeNode()
# Serialize the result
print doc.saveFormatFile("-", True)
Now, getting to this simple script took some spitting blood. Basically, in Debian we seem to have lots of simple libraries for: I tried, in order:

Russell Coker: Banking with an Infected Computer

Bruce Schneier summarised a series of articles about banking security [1]. He mentioned the fact that banks don’t seem to care about small losses and would rather just deal with the problem (presumably by increasing their fees to account for losses). There are some other interesting bits in the article, for example banks are planning a strategy of securing transactions with an infected computer [2]! Now there are some possible solutions to this, for example if the bank issued a hardware device that allowed the customer to enter the account number, amount to transfer, destination account, and PIN number and then produced a cryptographically secure hash (based in part on a rolling code) that the user could type in. The only way that you are going to do anything securely with an infected host is if everything is offloaded into an external device. In which case why not just do the Internet banking on the external device? It’s not difficult to make a hardware device that is small enough to carry everywhere, has a display, an input device, net access, and which is reasonably difficult to crack externally. Consider for example a typical mobile phone which has more RAM, CPU power, and storage than a low-end machine that was used for web browsing in 1996. Mobile phones have a good history of not being hacked remotely and are difficult for the owner to “unlock”. A locked-down mobile phone would be a good platform for Internet banking, it has wireless net access in most places (and with a Java application on the phone it could do banking by encrypted SMS). Being locked down to prevent the user from reconfiguring the software (or installing new software) will solve most of the security problems that plague Windows. If when signing up for a new phone contract I was offered the possibility of getting a phone with secure banking software installed for a small extra fee then I would be very interested. Of course we would want some external auditing of the software development to make sure that it’s not like some of the stupid ideas that banks have implemented. Here is a classic example of banking stupidity [3]. They display a selected word and picture for the user when they login to try and prevent phishing (of course a proxy or a key-logger on the local machine will defeat that). They also ask for an extra password (of the simple challenge phrase variety) if you use a different IP address, of course as the typical broadband user doesn’t know when their IP address changes they wouldn’t know if their data was being proxied and dial-up users will enter it every time. A google search for “internet banking” picture password turns up a bunch of banks that implement such ideas.

Share This

Raphael Geissert: DeBaBaReTools: The de-what?

It's been a while since I wanted to post about a project I started.
But today when I read Russell Coker's post about his SE Linux repository I decided it was time to blog about DeBaBaReTools.

So, what is/are DeBaBaReTools?
is: a project which tries to unify different packages-building systems (pbuilder, buildd, etc) and different repository tools (debarchiver, reprepro, etc) in a single system by providing an incoming/ system.

are: Debian Bash-Based Repository Tools.

Or in other words:
DeBaBaReTools handles new packages by:


DeBaBaReTools is NOT a rewrite of a repository tool (like reprepro, debpool or debarchiver), neither yet an other building tool (like buildd or pbuilder). It is rather a glue between those two kind of tools.

So, how does DeBaBaReTools work?
As already said, they are a set of scripts which do all the work.
These are some of the scripts (more might be written later, they are under development):



Note that at the moment these are the only supported backends: gpgv, du, md5sums (all built in checkSums), pbuilder and reprepro.

Sounds interesting, doesn't it? :)

So, here's the magic:

The backends system
DeBaBaReTools doesn't try to re-invent the wheel, but it makes use of all the available tools so it works as less as possible.

This is where the backends take action. They are even more shell (even tough DeBaBaReTools clearly specifies they are Bash scripts they should work on any other POSIX-compatible shell) scripts which are loaded on demand and according to the user settings.

This opens a wide range of possible combinations of backends; making DeBaBaReTools work with almost anything.

The probe.sh script searches different directories for the specified backend (and will soon support user-hinted backends).

I think an example says more than a thousand words:
When needsBuild finds a package that needs to be built in a given architecture, the toBuild shell function is called. In order to provide the already said flexibility, needsBuild will at startup time call probeFile "buildQueue" which will then check several directories for the required backend (buildQueue in this case). Once it finds the given backend (backends/buildQueue/) it will check for any user hint (which will specify the real backend to be used) and then for a default one (default.sh which is a symlink to a real backend, plaintext.sh by default).

So, if somebody at some point wants to use a sqlite database to store all that information, that person would have to write a file called backends/buildQueue/sqlite.sh (or make it available in any of the include paths which can be extended via the EXTRA_PROBE_DIRS env variable) and either set a user hint or make default.sh point to it.
The sqlite.sh script can then call any external tools (or even more scripts) to insert the information in the database, read it and more.

I know this is a lot of blahblahblah but no single line of code is shown.
I've already requested an alioth project but it was denied because the package isn't already in the Debian archive.
If you are a DD and you think this sounds like a good project please contact the Alioth admins and convince them to create a project account for DeBaBaReTools.
Once created, many other people will be able to work on it and make it work with virtually any tool.

Any kind of feedback is very welcomed but I hope they are constructive ideas.

Why did I start this crazy project?
Because AFAIR only dak is able to do such job but I think dak is just too complex for a simple system to manage an a repository with an incoming system. And AFAIK there's no script which is able to do this job besides dak.

I hope this post makes sense because I don't want to review it again (I'm tired) :)

:bye: all

18 October 2007

Jeff Bailey: One step closer to here

Angie and I wrote out written tests today for the California driver's license. We've been issued temporary licenses because we had current out-of-country licenses. We've been talking about possibly finding a drivers-ed instructor for a couple hours of private lessons to make sure that we'll actually pass the road test. After 13 years of driving, am I still likely to be able to pass a road test? Who knows? =)

I've thought before that the road test should be redone every 5 years. If the fees cover the cost of administering the test, then all it can do is make the roads safer and reduce the traffic on the road. Maybe if it were that much hassle, the demand from transit would increase enough that people wanted to use it.

I remember that I got my license so I could take a friend of mine to her prom. I wonder how Alicia's doing? For all the old faces on Facebook, I haven't come across her yet.

Next step: Move into new condo on Friday/Saturday (address is on Facebook, I'll update Orkut and the Debian DB shortly), followed by 3 university exams next week. And there's still a dayjob tucked in and around all of this! =)

17 October 2007

Russell Coker: Insider Threats and Small Storage Devices

Danny Angus writes about the potential threat posed by small storage devices with large capacity [1]. His post was prompted by a BBC article about Hitachi’s plans for new hard drives [2], they are aiming for 4TB of data on a single drive by 2011 and a 1TB laptop drive. One thing I noticed about the article is that they made the false claim that current drives are limited to 1TB, the storage capacity is determined by the total surface area which is proportional to the square of the radius and the height of the drive (AFAIK there are no practical limits to the number of platters apart from the height of the drive). So if a 5.25 inch hard drive was to be manufactured with today’s technology it should get a capacity equivalent to at least three times the capacity of the larger 3.5 inch drive. The reason that 5.25 inch drives are not manufactured is that for best performance you want multiple spindles so that multiple operations can be performed concurrently. Using 3.5 inch drives in servers allows the use of more disks for the same amount of space in the rack and the same amount of power. The latest trend is towards 2.5 inch (Small Form Factor AKA SFF) disks for servers to allow more drives for better performance. With 3.5 inch disks a 1U system was limited to 3 disks and a 2U system was often limited to 4 or 5 disks. But with 2.5 inch drives a 2U server can have 10 drives or more. I know of one hardware vendor that plans to entirely cease using 3.5 inch drives and claims that 2.5 inch disks will give better performance, capacity, and power use! In regard to Danny’s claim (which is entirely correct) about the threat posed by insiders. I don’t believe that a laptop with 1TB of capacity is the threat. In a server room people notice where laptops get connected and there are often strictly enforced policies about connecting machines that don’t belong to the company. I believe that the greatest threat is posed by USB flash devices. For example let’s consider a database with customer name (~20B), birth-date (10B), address (~80B), phone number (~12B), card type (1B), card number (16B), card expiry (5B), and card CVV code (3B). That’s ~155 bytes per record in CSV or TSV format. If you have data for a million customers that’s 155M uncompressed and probably about 50M when compressed with gzip or WinZip (depending on which platform is being ripped). No-one even sells a USB flash device that is smaller than 50M, I recently bought a 2G flash device that was physically very small and cheap (it was in the bargain bin). The next issue is, what data might be worth stealing that is large enough to not fit on a USB device? I guess that if you want to copy entire network file shares from a corporation then you would need more than the 16G that seems to be the maximum capacity of a USB device at the moment. Another theoretical possibility would be to copy the entire mail spool of a medium to large ISP. For the case of a corporate file server you could probably get the data at reasonable speed, 1TB of data would take 10,000 seconds or 2.8 hours to transfer at gigabit Ethernet speeds (if you max out a GigE link - it could be as much as five times that if the network is congested or if the server is slow). It’s doable, but it would be a rather tense three or more hours waiting by an illegally connected laptop. For the mail server of a large ISP there is often no chance of getting anywhere near line speed, it’s lots of small reads and seek performance is the bottleneck, such servers are usually running close to capacity (and trying to copy data fast would hurt performance and draw unwanted attention). Another possibility might be to copy the storage of an Intranet search device. If a company has a Google appliance or similar device indexing much of their secret data then copying the indexes would be very useful. It would allow offline searches of the corporate data to prepare a list of files to retrieve later. It would probably be more useful to get online access to the data from a remote site. I expect that an unethical person could sell remote access to someone who is out of range of extradition. All that would be required would be to intentionally leave a flaw in the security of the system. In most large corporations this could be done in a way that is impossible to prove. For example if management decrees that the Internet servers run some software that is known to be of low quality then a hostile insider could make configuration changes to increase the risk - it would look like an innocent mistake if the problem was ever discovered (the blame would entirely go to the buggy software and the person who recommended it). A large part of the solution to this problem is to hire good employees. The common checks performed grudgingly by financial companies are grossly inadequate for this. Checking whether a potential employee has a criminal record does not prevent hiring criminals, it merely prevents hiring unsuccessful criminals and people who have not yet been tempted enough! The best way to assess whether HR people are being smart about this is to ask them for an estimate of how many criminals are employed by the company. If you have a company that’s not incredibly small then it’s inevitable that some criminals will be employed. Anyone who thinks that it is possible to avoid hiring criminals simply isn’t thinking about the issues. I may write more about this issue in a future post. Another significant part of the solution to the problem is to grant minimum privileges to access data. Everyone should only be granted access to data that they need for their work so that the only people who can really compromise the company are senior managers and sys-admins, and for best security different departments or groups should have different sys-admin teams and separate server rooms. Of course this does increase the cost of doing business, and probably most managers would rather have it be cheap than secure.

Share This

Next.

Previous.