A few days ago, Henrik Bengtsson reported that my digest package breaks under R-devel, i.e. what will be R 2.4.0 come October. Being a good sport, he also debugged and patched this :-). I made a new version (0.2.2) available that is waiting in CRAN's incoming area until the CRAN maintainers come back from vacation. In the meantime, it is also available on a simple digest page I finally added to the code section here. So as an belated into, digest provides hash digests of arbitrary R objects by relying on the wonderss of R's serialize() function, and a simple interface to (C-level) md5, sha-1 and crc32 functions. Clearly, digest is not meant for crypto but rather to quickly summarize, compare or even cache complex R objects that may have arisen from larger computations or simulations. Apparently, Henrik uses digest to compute hash functions of arbitrary R objects that can be cached using his R.cache package.

Duffman thrusts in the direction of the problem. Oh yeah!

On Debian Policy

When your package contains GDFL docs
or when it break other programs
it stores passwords in mode 666
has incorrect dependencies
that's RC.

When you think that rpath is justified
just because libtool inserted it there
and don't use -fPIC,
libtool is a fool
and you are retarded.

When you have to use Minix
and there are conf files in /usr/local/etc,
Lintian will say,
you had it coming.

I finally took some time to migrate my blog from a (very old) Wordpress to a fairly new Ikiwiki. As I write these lines, the version of Ikiwiki I'm using isn't even released, and it includes two patches made by yours truly (and submitted upstream, of course, as bugs #380680 and #380743), allowing the creation of monthly archives and the possibility of making these archives RSS-less (it doesn't make much sense to have an RSS feed for something that's supposed to be static). The whole blog is now static, since I haven't investigated the comments system. I will probably enable the comments eventually, and I will of course migrate the existing ones (much like I migrated the existing articles, with a combination of a Perl script and some hand-tuning). In the meantime, I'll enjoy a spamless blog. The other regression is the lack of a search facility, but from what I can see it was hardly ever used anyway. Note to readers: old URLs should still work (thanks to Apache's magic mod_rewrite), but please update your bookmarks and feed readers and agregators and so on to use the new ones, since I'll probably disable URLs at some point. Update: I also hacked a new plugin, linkbar.pm, which appends a static Markdown string to all *.mdwn pages before they are rendered. So you should now see the lovely navigation menu on all pages, including the article pages.

On foreign policy

When you defend the death of a civilian
and say
he should have known better
he should have left his home
and his life
in the land of your enemy
that is hate. When you say the boy's death was justified,
for daring to be born in that country
with such people
and such policies
it is because you hate. When the war comes to your town
and you are caught in the cross fire,
I too will say,
you had it coming.

I am quite happy to see that we will be able to make GConf FHS-compliant in etch.

Here are the packages still shipping files in /etc/gconf.

That makes really few, given that:

• goobox has been fixed,


• yank will probably be removed from the archive,


• we will probably be able to remove gconf (the GNOME 1 version) now that gnucash 2.0 has entered unstable.

This leaves only 2 bugs : #380728 (apt-watch, I only noticed it today) and #369858 (gnome-panel). Gnome-panel is the only case that couldn't be handled by dh_gconf, and I'll try to push one of the solutions that were thought of ASAP.

Thanks to everyone who made this possible, especially package maintainers who updated their packages so quickly (e.g. one hour for goobox).

I’ve just upload new upstream (new for Debian, not really brand new! ) of libgnucrypto-java, libcommons-httpclient-java. The first upload will solve the libjessie-java FTBFS. I’ll work on new upstream of libcommons-net-java, then, I’ll track down Tomcat5 and Tomcat5.5 bugs.

There have not been updates for an extended period of time as I have taken a completely unplanned and unexpected holiday. Now I'm back on track. Work I have accomplished since the last update:

• updated TranslationUnit and the .po parser. In the end I decided that it's not worth making it too different from Pootle's pofile. Comments are now dealt with and the header is treated specially.
• merging templates. I defined an interface for objects able to merge translation stores and provided a simple implementation. Pootle already has an implementation of this too, I'll try to encapsulate that as well.
• help with move to Subversion for the Wordforge project. The Subversion repository is already operational at https://svn.sourceforge.net/svnroot/translate/trunk, although it's read-only at this moment.
• some small shufflings in the API. Generally the defined interface are holding out fairly well new code, so only small modifications are needed.

Here's what I want to do next week:

• Wire this thing up to DDTP. Should be easy, I just need to ask about a good way to fetch DDTP translations. Probably this will be a download-only demo in the beginning.
• Same for debconf l10n templates.
• Come up with a translation review and approval model. This is one of the cornerstones of the API and may be tricky to get right. I think that it is essential to have a sort of a 'diff' tool working at translation unit level (much like ordinary 'diff' works on text). To get usable diffs several copies of the template (upstream & local) should be stored. With this in place it shouldn't be hard to implement pushing of new DDTP / debconf translations upstream.
• further updates to the API as flaws and missing features pop up.

My latest music CDs (of course already ripped to my Ogg Vorbis player... CDs suck usability-wise, actually...):
Sing when you're Swinging Two CDs - the second is titled "(If you can't sing it) You'll have to swing it" - with recordings of the swing era. Some slow, some better for dancing. Rather cheap, and well worth the money unless you have many songs of that era already.
Pop swings This CD is a mixed bag. It's swing covers of well-known songs of the 70s-90s. Some work great, others don't. A lot is personal taste, depending on the memories you have of the originals or how you liked them. If you didn't like the original, you might like the swing cover. If you loved the original, you'll likely hate the cover, just like any cover. The funniest thing is that one artist, Paul Anka, is on the CD in both roles, as cover artist (Smells like teen spirit) and as original artist (Diana). This is a CD you can't really listen to on-block (thats why CDs suck) or play on a party. You'll usually pick some specific songs.
Perfect for parties The full title is

Bear Family
"Perfect for parties
Highlight Album

This CD was the surprise CD. Very cheap, but very convincing. It's odd and fun at the same time. Actually it's probably as much country as it's rockability and swing. But who cares when you can have lots of fun with it? The song I like currently like best is "Burn that candle" by Charline Arthur. Note: above amazon.de links will earn me a tiny commission. It would be nice if you'd use them if you plan to order some of these CDs. Thanks. It will certainly never pay off, but when I set links I can as well set some that might allow me to buy myself a CD from it sometime later, too. ;-)

I've recently used bittorrent a bit. For example, I loaded the Ubuntu installer CD with it. For power users, I think rtorrent is a good client. I tried freeloader, but apart from some obvious bugs (it doesn't save the maximum bandwidth settings value), i'm lacking interesting information such as the number of peers or of distributed copies. When toying around with rtorrent and it's options I noticed a couple of interesting things. First of all, if you're firewalled, you'll be much slower. Allowing incoming connetions (and having a port forwarding in your router) makes a huge difference in the number of peers. My estimates are that 70% of peers are firewalled; if both of you are, you can't transfer data! The other thing is that apparently BitTorrent clients use some kind of "Tit for Tat" strategy. They request a packet, you request a packet. If you don't send them data, they'll stop sending you any. They'll still eventually retry (maybe you were overloaded), but largely your download speed will be related to your upload speed. I don't know how much peers (and seeds) trade information on who has which parts or is sharing how much upload speed. I guess they do somehow to deflect cheaters. Assuming that the illegal file sharing tools work similarly, this gives some interesting (theoretical?) ways to recude illegal file sharing:

• Give people free routers - which don't have an easy-to-use port forwarding option (especially not UPnP, which software could auto-configure...)
• Tell them that downloading isn't that dangerous for being sued, but uploading is, so they should set their upload speed limit to the minimum.

You thought that buglist was soon over ? sadly for you, I've tracked (using the Contents.gz of the archive) all the packages that seemed to have private python modules. I've found many that were even not compliant with the previous python policy (and that shiped public extensions btw), and a big lot that need update. I'm way to tired to launch the big mass bug now, because I've worked more than 2 hours straight on that, looking at almost a hundred of packages list files in the eyes, to know if there was a tiiiiiiny python module hidden in there. Still expect it soon Sadly, it does not looks so good: my first list was 265 packages long, I suppose there is a whole bunch of false-positives, but I fear there will remain in the 200 bugs. Though, this is the final step. If you do maintain a package that depends upon python, please help ! Migrating your package to the new policy is often easy if you use debhelper/cdbs/ and other high-level packaging tools.

Some might have seen the videos of Microsoft PhotoSynth. That of course makes many people go all "ooooh, must, have!". Actually, the videos from University of Washingtons PhotoTour are much more impressive. And it actually tells you how it works. Read the PDF of the paper published at this years SigGRAPH, also linked from that site, for more details. The Microsoft video only shows the UI; while the Washington video gives a bit more information, but still doesn't really talk about the "backend requirements". You can get some facts from the published paper, though. Basically it's a smart combination of existing technologies, together with some good optimization (I guess). It uses SIFT feature extraction and matching (use also by panotools: autopano-sift, but unfortunately this is patent encumbered in the US; I doubt it's patentable in Germany because of it's very mathematic nature) and basically does the same as panorama tools, except in 3D - using "Structure from Motion" approaches. So you might wonder why I still call this vapourware, when there is a sigGRAPH paper backing it, and an interesting demo? Well, in the paper they mentioned their test machine to be a 3.4 GHz computer. And that the Notre Dame photoset took two week to process, and that 597 of the 2635 images were actually placed; the others maybe had too much other stuff on them, or didn't match properly. This yields the impression that for this to work you need

1. lots of CPU time
2. lots of well-suited images (clean projection, not too many moving people on it etc.)
3. no gaps inbetween pictures, but a complete covering

I don't want to talk down the scientific work done here, but the Microsoft annoucement makes it sound (well, just look at who they imply in the video that their users are) like it would be a product for everyone to use within a year. It won't. Thats why I call it vapourware. It's good research, but not a realistic product. It will be very interesting for professionals, however. They can set their camera to a well-behaved angle (no fisheye; I don't expect the software to like fisheye artifacts much) and take tons of photos, trying to cover all of a building. The software will then eventually allow them to construct and texture a 3D model, with some weeks of computation. The viewer is still interesting, for data sets computed by someone with access to excess CPU power. E.g. for a company to offer virtual sightseeing.

I've been watching the Lebanese bloodshed since it started. It really makes me sick. I have discussed this with many friends. There was little discussion though: we all thought this is a plain war against civilian targets. I tried discussing with people who could maybe be more supportive of the Israeli points of view. I tried talking to some of my right-winger relatives... and I haven't been able to find anyone that can say a word of support of what Israel is doing in Lebanon. In three weeks, I have seen how Hizbullah killed some Israeli soldiers in some area around the border between Lebanon and Israel. As far as my (as an international online paper reader) judgement can tell, the location where this happened is disputed. Some say it was inside Lebanese territory, the others say it was an Hizbullah incursion into Israeli soldiers. I really don't care. The immediate result of this action was a campaign of air strikes against Beirut and other Lebanese cities aiming to give a measured response (cache, and beware this is explicit) to Hizbulla's aggression. This, apparently, included destroying any vital infrastructure in heavily populated areas of Beirut, including bridges, power plants, hospitals and apparently random buildings. While the Israeli army said the military operations were aimed to get rid of the Hizbullah camps and infrastructures around the border where they fire their rockets against Haifa and other Israeli cities, and their headquarters in Beirut and other southern cities, what the world espectated during the first three or four days of air strikes was pure horror. Hizbullah-controlled areas were like ghost quarters, so any bombing of those were very unlikely to take any Hizbullah victims. On the other hand, the rising number of civilian victims was the first bits of news that I could hear in my radio as I woke up every morning. All of this, with a great amount of cinism and unlimitted cruelty. The Israeli army declared that the civilian population in the South should be leaving their homes (measured response), while they knew they were doing exactly what was needed to prevent this, by locking them up around destroyed bridges, harbours, airport and roads. The first columns of the lucky refugees who could afford fleeing to the North were greeted by Israeli bombs, leaving a good number of scattered bodies. In Spain, the news shocked the public opinion and Zapatero, along with Chirac (IIRC), declared those actions were untolerable and unproportionate, and should be stopped immediately. This brought some local politics fun, when on one side the Israel ambassador in Spain accussed Zapatero of being antisemitic, and the right-wing leader Rajoy called the thousands of people who spontaneously demonstrated against the war in a few cities "clowns" and "ignorant". Watching what the UN has done to stop these warcrimes, I really think it's a great moment to shut down that useless organisation and let some big corporation buy their nice building in New York City to build their offices in it. Once again, when some countries tried to come up with some resolution that forced Israel to stop, the United States waived their vetto flag. Once again, the Security Council has done nothing to put an end to another chapter of violence in the Middle East. The vettoing powers of some of the members is so ridiculous that it makes it totally inoperative. So, as the UK and US governments were not going to do anything to stop the air strikes, Israel had carte blanche to do whatever they wanted for a week. And then, Israel's measured response destroyed one of the UN's buildings in Lebanon, killing four annoying observers, despite their calls warning that the bombs were falling too near their location. Only then, the United States decided to send Condie to have a look around the area, surprise visit to Beirut included. I wonder, what phone number must the innocent women and children who cannot escape their cities and towns dial in order to warn the Israeli army about the proximity of their bombed targets? After two visits of Condie to the region, nothing has changed, as the fourth week of operations starts. This is getting on my nerves. I don't, and won't, justify any of the Hizbullah attacks on civilian targets using Katyushas, or the kidnapping of soldiers, or whatever. What I see is that unless Israel takes a radically different approach to their problem, nothing will change either. Violence isn't at least. Apparently, their military operation isn't going well at all, and is not damaging much of Hizbullah's military capacity. What are they going to do? Will they occupy southern Lebanon for 20 more years? The social movements this kind of actions are generating is also very, very worrying. On one hand, I read that around 85% of the Israeli population approve this war and think this is what their government needs to do to make them safe. This is pretty shocking. On the other hand, everyone still in Lebanon, even the Christian community, are united against the attack, helping each other in their survival quest, while the popular support to Hizbullah is rising, with big concentrations of people cheering them in the streets. This is just two years after similar demonstrations in Beirut forced the Syrian military out of the country. While the Israeli war against Lebanon (I won't buy the "It's against Hizbullah" bullshit), sadly all the media, cameras and journalists have given their back on Gaza, which is under similar, but probably more brutal and savage military operations by Israel, as another soldier was kidnapped just one week before all of the Lebanon stuff started. The press doesn't have any news of how things are going in Gaza. The last thing I remember is that the UN was warning about the threat of a humanitarian disaster due to the lack of any kind of supplies in Palestine. If Palestinian kids are starving, the media are not letting us know. Yesterday, 54 civilians were killed in an improvised underground bomb shelter in Qana, most of them being children. The Lebanese civilian death toll must be well over 500 as of this writing, with a ratio of at least 1:10 compared to Israeli military victims. When will Olmert have enough? The only way I see this will end up for good is when the Israeli public realises being at war in three fronts is quite expensive and damages their home economies. Maybe, at that point, people start hearing what the Meretz party minority in the country have been saying for years. Israel has the right to defend itself. This is the official motto of Israeli and American leaders. When this right is extended and abused to the point the main targets and victims are civilians, Israel walks quickly away from a perdurable solution. Today, hostility against the state of Israel is even greater than last month. Hizbullah in Lebanon, Hamas in Palestine, will probably be the winners of the conflict when the bombings end. Not the best recipe for a peaceful future.

Ninth day in Addis SSH To enable remote logins with ssh apt-get install openssh-server Then you can login with:

$ ssh efossnet@proxy.dream.edu.et

To verify the host key fingerprint of a machine:

$ ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub

Note: you need to verify it before logging in! More information at http://www.securityfocus.com/infocus/1806 Example ssh usages To log in:

    $ ssh efossnet@proxy

To run a command in the remote computer:

    $ ssh efossnet@proxy "cat /etc/hosts"

To copy a file to the remote computer:

    $ scp Desktop/july-18.tar.gz efossnet@proxy:

To copy a file from the remote computer:

    $ scp efossnet@proxy:july-18.tar.gz /tmp/

Beware of brute-force login attempts Warning about SSH: there are people who run automated scans for ssh servers and try to login using commonly used easy passwords. If you have an SSH server on the network, use strong passwords, or if you can it's even better to disable password authentication: in /etc/ssh/sshd_config, add:

    PasswordAuthentication no

To log in using public/private keys:

1. Create your key:

   ssh-keygen -t rsa
   

2. Copy your public key to the machine where you want to log in:

   ssh-copy-id -i .ssh/id_rsa.pub efossnet@proxy
   

3. Now you can ssh using your RSA key

If you use ssh often, read these:

• http://mah.everybody.org/docs/ssh
• http://www.securityfocus.com/infocus/1812
• http://www.sshkeychain.org/mirrors/SSH-with-Keys-HOWTO/SSH-with-Keys-HOWTO-6.html

proxy Problems we had today with the proxy: ssl does not work Reason: squid tries to directly connect to the ssl server, but the AAU network wants us to go through their proxy. Ideal solution: none. There is no way to tell squid to use a parent proxy for SSL connections. Solution: update the documentation for the Dream university users telling to setup a different proxy for SSL connections. Longer term solution: get the AAU network admins to enable outgoing SSL connections from the Dream university proxy. Other things that can be done:

• report a bug on squid reporting the need and requesting the feature
• download squid source code and implement the feature ourselves, then submit the patch to the squid people

Browsing normal pages returns an error of 'Connection refused'. In the logs, the line is:

1153294204.912    887 192.168.0.200 TCP_MISS/503 1441 GET http://www.google.com.et/search? - NONE/- text/html

That "/503" is one of the HTTP error codes. Explanation of the error codes:

• http://www.w3.org/Protocols/HTTP/HTRESP.html
• http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
• http://offline.web.cern.ch/offline/web/http_error_codes.html

Reason: the other proxy is refusing connections from our proxy. Solution: none so far. Will need to get in touch with the admins of the other proxy to try to find out why it refuses connection to our proxy, and how we can fix the problem. postfix on smtp.dream.edu.et Basic information is at http://www.postfix.org/basic.html. Difference between mail name and smarthost:

• The mail name is the name of the mail server you're setting up (TODO: need more details on what's it used for)
• The smarthost is the name of the mail server that will relay mail for you.

Quick way to send test mails:

apt-get install mailx
echo ciao   mail efossnet@localhost

To configure a workstation not to do any mail delivery locally and send all mail produced locally to smtp.dream.edu.et:

1. install postfix choosing "Satellite system"
2. put smtp.dream.edu.et as a smarthost.

To setup a webmail: apt-get install squirrelmail (on a working apache setup). To setup mailing lists: apt-get install mailman, then follow the instructions in /usr/share/doc. Mail server issues we encountered When a mail is sent to efossnet@localhost, the system tries to send it to efossnet@yoseph.org Investigation:

• "yoseph.org" does not appear anywhere in /etc or /var/spool/postfix
• postfix configuration has been reloaded
• postfix logs show that the mail has been 'forwarded'

Cause: the user efossnet had forgotten that he or she had setup a .forward file in the home directory. Solution:

 rm ~efossnet/.forward

Apache To add a new website:

1. cd /etc/apache2/sites-available
2. sudo cp default course
3. sudo vi course:
   1. Remove the first line
   2. Add a ServerName directive with the address of your server: ServerName course.dream.edu.et
   3. Customize the rest as needed: you at least want to remove the support for browsing /usr/share/doc and you want to use a different document root.
4. sudo a2ensite course
5. sudo /etc/init.d/apache2 reload

More VIM Undo: u (in command mode) Redo: ^R (in command mode) You can undo and redo multiple times. To recover a lost password for root or for the ubuntu admin user Boot with a live CD, mount the system on the hard disk (the live CD usually does it automatically), then edit the file /etc/shadow, removing the password:

enrico:$1$3AJfasjJFHa234dfh230:13343:0:99999:7:::

becomes:

enrico::13343:0:99999:7:::

You can edit the file because, in the live CD system, you can always become root. After you do this, reboot the system: you can log in without password, and set yourself a new password using the command passwd. Installing packages not on the CDs To get a package for installing when offline:

1. apt-get --print-uris install dnsmasq
2. Manually download the packages at the URLs that it gives you

Otherwise, apt-get --download-only install dnsmasq will download the package for you in /var/cache/apt/archives. You can install various previously downloaded debian packages with:

dpkg -i *.deb

Backups There are various ways:

• dump (for ext2/ext3 file systems) or xfsdump (for xfs file systems). Makes a low-level dump of the file system. It must be used for every different partition. It makes the most exact backup possible, including inode numbers. It can do full and incremental backups. To see the type of the filesystems, use 'mount' with no parameters. To restore: restore or xfsrestore.
• tar Filesystem independent. It can work accross partitions. It correctly backups permissions and hard links. It can do full and incremental backups. Example:

  tar lzcpf backup.tar.gz /home /var /etc /usr/local
  tar lzcpf root.tar.gz /
  

  To restore:

  tar zxpf backup.tar.gz
  

• faubackup Filesystem independent. Uses hard drive as backup storage. Always incremental. It cannot do compression. Unchanged files in new backups are just links to old backups, and do not occupy space. Any old backup can be deleted at any time without compromising the others. It can be used to provided a "yesterday's files" service to users (both locally and exported as a read-only samba share...). To restore, just copy the files from the backup area.
• amanda

  apt-get install amanda-client amanda-server
  

  It is a network backup system. It can do full and incremental backups. You can have a backup server which handles the storage and various backup clients that send the files to backup to the server. It takes some studying to set up. To restore: it has its own tool.

Some data requires exporting before backing it up:

• To save the list of installed packages and the answer to configuration questions:

  dpkg --get-selections > pkglist
  debconf-get-selections > pkgconfig
  

  To restore:

  dpkg --set-selections < list
  debconf-set-selections < pkgconfig
  apt-get dselect-upgrade
  

  If you do this, they you only need to backup /etc, /home, /usr/local, /var.
• To save the contents of a MySQL database:

  mysqldump name-of-database   gzip > name-of-database.dump.gz
  

  To restore:

  zcat name-of-database.dump.gz   mysql
  

You can schedule these dumps to be made one hour before the time you make backups. Scheduling tasks As a user:

crontab -e

As root: add a file in one of the /etc/cron.* directories. In cron. hourly,daily,weekly,monthly you put scripts. In the other directories you put crontab files (man 5 crontab). If the system is turned off during normal maintainance hours, you can do two things:

1. Change /etc/crontab to use different maintanance hours
2. Install anacron (it's installed by default in ubuntu)

For scheduling one-shot tasks, use at(1):

$ at 17:40
echo "Please tell Enrico that the lesson is finished"   mail efossnet@dream.edu.et
^D

When and how to automate

1. First, you manage to do it yourself
2. Then, you document it
3. Then, you automate it

Start at step 1 and go to 2 or 3 if/when you actually need it. (credits to sto@debian.org: he's the one from which I heard it for the first time, said so well). Interesting programs to schedule during maintanance

• rkhunter, chkrootkit
• checksecurity
• debsecan
• tiger

Important keys to know in a Unix terminal These are special keys that work on Unix terminals:

• ^C: interrupt (sends SIGTERM)
• ^\: interrupt (send SIGQUIT)
• ^D: end of input
• ^S: stop scrolling
• ^Q: resume scrolling

Therefore, if the terminal looks like it got stuck, try hitting ^Q. Problems we had today with postfix

• Problem: mail to efossnet@dream.edu.et is accepted only if sent locally. Reason:

  $ host -t mx dream.edu.et
  Host dream.edu.et not found: 3(NXDOMAIN)
  

  Solution: tell dnsmasq to handle a MX record also for dream.edu.et: mx-host=dream.edu.et,smtp.dream.edu.et,50
• The problem not solved with the previous solution. Reason: postfix was making complaints which mentioned localhost as a domain name. Solution: fixed by changing 'myhostname' in main.cf to something different than localhost. Note: solved by luck. Investigate why this happened.

Problems found yesterday and today

• there is no way to tell squid to use another proxy for SSL connections: it only does them directly
• if you want to configure evolution to get mail from /var/mail/user, you need to explicitly enter the path. It would be trivially easier if evolution presented a good default, since it's easy to compute. It would also be useful if below the "Path" entry there were some text telling what path is being requested: the mail spool? the evolution mail storage?
• In Evolution: IMAP or IMAPv4r1? What is the difference? Why should I care?
• apt-get --print-uris doesn't print the URIs if the package is in the local cache, and there seems to be no way to have it do it.
• in /etc/apache2/sites-available/default, is the NameVirtualHost * directive appropriate there? It gets in the way when using 'default' as a template for new sites. Otherwise, one can add a new (disabled) site that can be used as a template for new sites instead of default.
• the default comments put by crontab -e are not that easy to read.

For a long time I have considered F-Spot to be a nice concept application, but not really fit to be the center of Linux digital photography experience due to general bloat and (most importantly) due to the lack of many important features that would compensate the interface lock in that this application demands. In this post I will try to list all the deficiencies and lacking features of F-Spot that I have found while using it to manage my photo flow.
First let’s talk about the bloat. I will compare F-Spot to GThumb. There are few points of comparison: first start up, viewing a bunch of photos and importing a bunch of photos. When starting up GThumb uses 17 Mb RSS memory (10 of them shared with all other Gnome programs) while F-Spot uses 33 Mb RSS memory (17 of them shared with other Mono programs, of which I do not have any). When viewing a folder with couple hundred photos GThumb uses 25 Mb RSS memory while F-Spot grows to 55 Mb RSS. On import GThumb calls gphoto that adds 3-5 Mb RSS to the memory usage, while F-Spot seems to implement some/most of the photo protocol internally and can easily grow in memory up to the size of the photos being imported (which is kind of problematic while importing a gigabyte of photos), but usually the memory usage on import hangs around 250-300 Mb and hovers around that level for some time after the import is complete. For me personally it does not make much a difference (I have 2 Gb RAM now), but conservative memory usage is still critical for a good Linux application because people expect Linux to be faster and high memory usage can slow down anything. In my opinion no photo management operation should take more then 50 Mb of private RSS memory (even importing several gigs of RAW files from the camera).
Now let me try to describe the optimal work process of a (semi)professional photographer, so that we can see what is missing in F-Spot to accommodate such work-flow and what is done right.

1. The photographer takes one (or several) cameras, lenses, flashes, memory cars, spare batteries and tripods and goes to an event/photoshoot/modelling session/walk in the park/wedding/…
2. He takes 3-5 Gb of pictures from lots of different angles with different cameras, different lenses on different memory cards with different settings. Most of them will be the highest quality available on the camera, usually those will be RAW images.
3. Upon getting back to his computer, the photographer wants to drop all the data to a staging place dedicated to this event. He wants to do it fast and simple without sorting trough the photos at the moment, but at the same time he might want to split the staging areas (multiple models within one session) or join them (one model from different cameras). Sometimes he might not even have time for that and would just want to dump the data now and run to another event.
4. After several shooting events the photographer finally has reserved the time to sift trough some of the photos from the staging area (modelling jobs first, then the friends wedding and the walk in the park as last)
5. First the photographer decides to check if the times on the photographs are correct and discovers that at some shooting sessions one of the cameras was set to another timezone. He select all shots from that camera across several (but not all) staging areas and adjusts the time.
6. Now he rechecks the logical groups of photos (which photos should be in which staging area) by dragging the photos around, splitting and joining the staging areas
7. When the overall structure is clear, the photographer selects one (most important) staging area and starts working with images in that staging area not caring about any other images
8. First he browses in full screen trough all the photos rating them in quality scores form 1 to 9 by pressing corresponding number keys. Rating a picture automatically advances to the next picture, which happens very fast because the software precaches several photos ahead for speed and displays without any flicker. Speeds in the order of 2 photos per second are expected on good enough hardware. One key operations for “view 100% zoom”, “rotate”, “view exif” and “quick note” are available. RAW images are automatically translated for preview with default settings.
9. With couple clicks bad photos (with score of 4 or under) are selected and erased
10. Now the photographer starts working with individual photos
11. First step is the RAW->Jpeg conversion. An appropriate tool is brought up with default setting already applied and a preview of the photo showing up (like in dcraw). The photographer adjusts the conversion parameters (including and option to make a HDR-like effect by compressing the dynamic range of high bitness pictures). The converted Jpeg is saved as a version of the original photo and becomes the default for operations that do not understand RAW.
12. A separate plugin is used to reduce digital photo noise from the pictures (new version)
13. Photographer makes adjustments to the image inside F-Spot. Each modification makes a new version.
14. Still not satisfied, the photographer starts Gimp from inside F-Spot and make further edits. Each time Gimp saves the file, F-Spot makes another version.
15. The photographer erases some versions of the picture and also joins multiple near-duplicate shots into one picture as versions of that picture. One version is always marked as the top version and the photographer can easily change it.
16. The good photos are tagged and imported to the main collection
17. A subset of the photos is exported to: Flickr, personal Gallery installation, plain html photo album, a zip file with small (1024x…) versions of the photos, … Only the top versions of photos are exported. If unconverted RAWs are top versions, then they are converted to Jpeg with default settings. In most cases photos are also downsized before export according to the preferences. Highest quality downsizing is used (anti-aliasing) and also additionally small unsharp mask is run on each photo
18. The photographer does advanced searches and advanced tag selections to select and display or export subsets of his main collection
19. The photographer has three instances of F-Spot: his laptop, his main work PC and a backup hard drive that can be attached to either of them. He needs to keep his F-Spot photographs synchronised between the three locations for backups and for distributed work. Also after some point the main photo archive becomes too big to store it (with all versions) on the laptop or even on the main PC, so the photographer selects a set of older photos and selects an option to archive them. The next time a connection is established with the backup drive, all versions of archived photos are copied to the backup harddrive and the laptop and the main PC only keep 90% quality Jpeg format picture of the top version of each photo downsized to exactly fit the display resolution of the device. If even that takes too much space then “Archive Only” option is selected which insures that the backup harddrive has the copy of the photo and then erases it from primary data stores completely.
20. When the backup harddrive is connected it is also very simple to make DVD/HDDVD/Blueray backup copies of subsets of photos from the archive with all the metadata and versions in such a way that other software (and even dedicated media devices) could easily browse the top versions of photos directly from the backup media. Of course, restoring from such backups also must be tested.

So, to summarise, the changes needed for F-Spot are:

• Support for the concept of staging areas and staging area manipulation
• Much better performance on import
• Camera timestamp correction tool (#)
• Quick rating tool (#)
• RAW support for controlled conversion and automatic conversion at export (#)
• Support for external editor with automatic version control and format conversion (#)
• Noise reduction plugin support (#)
• Support for version manipulation - join (#, #) and split (#) version stacks
• Preferences for export methods with resize and quality options and automatic enhancement options at export
• Multiple instances and data sync (#)
• Archival support and total archival option
• Backup (also from archive) support (#, #)
• More powerful search and filter options (#) with automatic options from exif data (#): People/Mike AND Events/wedding AND (camera/RebelXT OR camera/SonyF717) AND setting/noflash AND lens/wide

As you can see, many of those issues are already reported to F-Spot bugzilla, but have recieved little attention. So what shall we do about it?

I am in the airport in Amsterdam on my way home to the US after a pretty decent nights sleep on the KLM from South Africa. This trip has been simply amazing, probably because I was taken around by locals, most of them who are U of C anthro students or graduate, all of them from South Africa. I have already started to write a few entries but will finish them when I get back. In the meantime, since I started the trip with an entry about an adapter, here is some more information on them. When I was researching the type of adapter needed for South Africa, much of the information on the web indicated that while South Africa took the M plug, one could get by with the British adapter as that is quite common here. I have to say that I never saw that type of plug at all… There seems to be one and only one type of plug so if you do come here, do purchase the South Africa-specific adapter.

Hello Planet Debian! Goodbye University Vacations!:-)

[Debian]

Thanks to Luk Claes, new translate-docformat was uploaded to fix Policy 7.6 Violation. madcoder has a list of affected packages. During the weekend I work on my packages, I'm waiting for my sponsor to get them uploaded to the archive closing the "adoption process" and fixing a few bugs and lintian warnings. There are some news from the DDTP and no news from the CVS pserver for non-DDs (and unfortunately, in this case: no news, bad news). I also sent a mail do debian-l10n-portuguese about the future of our list, we need to change a few points, create and update documentation, change our coordinators in some areas (they are MIA for quite a while now) and structure the team to be ready for future changes and also to work focused to get as much translations as possible in "etch".

[Life]

That's it... end of my University Vacation. Back to classes, busy schedule for the semester, I hope to manage everything in a very good way, so I can keep up-to-date with my Debian work.

Thanks to Cl ment Stenac (zorglub), my nice AM, this diary now appears in Planet Debian. Two old posts already appeared, but this is the first post that I made being aware that it will appear in Planet. :-) Thanks to Cl ment and Paul Wise.

I’ve been thinking a lot about software above the level of a single device lately, so I set out this weekend to finally get two-way email synchronization working on my Blackberry. My main goal is to have a single inbox I can access from multiple devices without having to ever deal with the same message twice (i.e., if the message is read/filed/deleted/etc. on one client, it’s marked read/filed/deleted/etc. on all clients), with full support for disconnected operation. (I care much less about the oft ballyhooed push email feature—I’m perfectly content to wait 15-20 minutes for email to be delivered.) Since what I want more or less perfectly describes IMAP, I figured Blackberry Internet Service, which uses standard Internet protocols for mail delivery and supports IMAP, would be sufficient, and that I wouldn’t need Blackberry Enterprise Server, which hooks into Exchange and other “enterprise” messaging systems. Unfortunately, I appear to have been wrong. While messages read and deleted on the Blackberry are reflected on the IMAP server, the reverse is, surprisingly, not true (surprisingly because, unlike POP, which behaves similarly in BIS, IMAP is a bidirectional protocol). In other words, when messages are read or deleted on another client (e.g., the Thunderbird running on my laptop), those changes aren’t reflected on the Blackberry; so, as a result, the Blackberry accumulates every message I receive when it’s not in use, making a periodic “Delete Prior” necessary to get rid of all the accumulated junk (which obviously deletes the useful messages too, like, ahem, the ones with my flight information in it). Worst of all, after several days of trying to figure out why IMAP wasn’t working the way I was expecting it to, I discovered this is the expected behavior (see note 5—”Messages deleted from the source mailbox are not automatically deleted from the device”), rather than a bug. What to do? I could obviously break down and upgrade to Blackberry Enterprise Server. I’m not too keen on setting up an Exchange server just for this (which would cost hundreds of dollars, particularly after the cascading upgrades typical of Microsoft products, e.g., I’d need to get Windows Server too). As an alternative, there appear to be a number of companies that offer a BES hosting service. However, the costs add up pretty quickly—another $10/mo. to T-Mobile to allow BES traffic through (the “business tax”), $10/mo. for BES hosting, and $10-15/mo. for Exchange hosting on top of that (BES hosting is an add-on to Exchange hosting in every plan I’ve seen). That’s a grand total of $35/mo. just to get email marked read or deleted on my Blackberry when I read or delete it in Thunderbird. Hrmmph. Another idea is to find a Blackberry IMAP client, but there seem to be a dearth of these. The main one that comes up both in web searches and that’s described in Blackberry Hacks is from a company called Reqwireless which, tantalizingly, was acquired by Google last year but whose products, frustratingly, are no longer available. In any event, I want something that integrates with the Blackberry’s excellent inbox, which aggregates email, SMS, missed calls, Google Talk messages, etc. in a single stream, and I’m not sure if a separate IMAP client will be able to do that. Other observations: This page says that “two-way [Blackberry] sync is coming soon” to Yahoo Mail. However, “coming soon” doesn’t really help me. Funambol looks very promising, but it’s going to take me a little while to wrap my head around what exactly I need to do to deploy it. Not exactly point and click. Scalix offers Scalix Wireless Solution that works with Blackberry and is integrated with the Enterprise and Small Business Editions. At $60/user with no minimum number of users and a free trial, this looks promising too. Zimbra offers Zimbra Mobile, which “includes the ability to leverage Zimbra’s third party mobile partners that offer two-way sync to Blackberry and other devices” (whatever that means). However, I’m assuming this comes bundled with the Network Edition, which is sold with a minimum of 25 users, and that probably makes it unaffordable to an individual user like me. Any other suggestions?

As I predicted on the 27 of June Telecom hasn’t changed a single bit. All their rhetoric about embracing the new competitive environment and voluntarily separating themselves has been posturing and PR as evidenced by the following article in this mornings Computerworld.

• Telecom: Xtra is not an ISP

Basically Telecom is saying Xtra is a brand, not a separate business unit, so it will continue to receive services in a different (favourable) manner to every other ISP. The only hope now is for the Government to actually muster up the courage to enforce a physical separation on Telecom… We live in hope.

As greylistd does not seem to be developed anymore and I wanted to have --auto-whitelist-clients I have switched to postgrey. I am using postgrey-integration into exim4 written by Guy Antony Halse with some minor changes to minimize the differences to my greylistd setup. - I greylist on /27 instead of /24 networks and will immediately reject invalid recipients instead of greylisting them. postgrey runs with --unix=/var/run/postgrey/socket --retry-window=4 --lookup-by-host --auto-whitelist-clients=10 --exim.

  defer
    log_message = greylisted host $sender_host_address
    !senders = : postmaster@*
    domains = +local_domains : +relay_to_domains
    !hosts = /etc/exim4/no-greylist-hosts
    !authenticated = *
    verify = recipient/callout=20s,use_sender,defer_ok
    set acl_m3 = request=smtpd_access_policy\nprotocol_state=RCPT\n\
                 protocol_name=$ uc:$received_protocol \n\
                 helo_name=$sender_helo_name\n\
                 client_address=$ substr_-3:$ mask:$sender_host_address/27 \n\
                 client_name=$sender_host_name\n\
                 sender=$sender_address\n\
                 recipient=$local_part@$domain\n\n
    set acl_m3 = $ sg \
                     $ readsocket /var/run/postgrey/socket $acl_m3 \
                            5s action=DUNNO \
                      action= 
    message = $ sg $acl_m3 ^\\w+\\s* 
    condition = $ if eq $ uc:$ substr 0 5 $acl_m3 DEFER true false 
  # add "greylisted by ..seconds" header to mail which has successfully
  # passed the greylisting.
  warn
    !senders = : postmaster@*
    domains = +local_domains : +relay_to_domains
    !hosts = /etc/exim4/no-greylist-hosts
    !authenticated = *
    message = $ sg $acl_m3 ^\\w+\\s* 
    condition = $ if eq $ uc:$ substr_0_7:$acl_m3 PREPEND true false