So Mandriva apparently is including a new 3D desktop named "Metisse" in their next release. It seems to be a real 3D thing, not like Compiz and Beryl which are just about some fancy effects for your 2D desktop. However, I'm not convinced that this is a good idea. First of all, many people have problems with thinking in 3D. "Engineers" like us are used to thinking in 3D, but many people are not. They might be better off with a 2D thing. Also supposedly 2D perception is easier for the mind, and should better allow us to focus on other things. Secondly, optimizing display for 2D is much better. Many people won't want to miss the crisp and sharp fonts we have today. Subpixel Antialiasing together with hinting does a good job at optimizing fonts for readability. Don't exepct a 3D desktop to be able to offer this, as soon as you turn perspective you'll end up with blurred fonts. Instead I'd consider doing mostly away with the concept of resizable and moveable windows. Most of my applications (except my terminal windows) are running full-screen. If I'd be working a next generation Desktop thing, I'd be investigating application switching (Expose for example seems to be good), some intuitive way of automatically tiling windows (ion sucks, sorry) and making the concept of multiple desktops easier for users. IMHO that is one of the most powerful (and yet often unused) functions on our systems. Note that metisse does seem to enable people to use multiple desktop a bit easier; I like their overview screen. I also like the "folding away" concept when temporarily marking text in a different window, but how is it supposed to know I'm going back to the previous window again afterwards? Most of what they show off is totally useless and has bad usability. Where is the actual use of being able to 3D-turn two windows? I can barely read the location bar of the browser afterwards! Just like Compiz and Beryl, they're just showing off fancy effects, but I don't see the usability benefit. (I havn't tried metisse, this opinion is solely from watching their demo videos.) For example, I never need to move my windows around on a pixel-level. If I ever resize my windows away from their default size, it's about having two windows visible at the same time. I could imagine having a window chooser such as Expose, and by selecting multiple windows with the right mouse button I can get a tiled view of them, as opposed to selecting a single window with the left mouse button.

I’ve always wondered why ill-intentioned hackers don’t set up fake free WiFi hotspots in public places like airports to collect passwords and other personal information. As it turns out, they do:

Authentium did an in-depth survey of the ad hoc networks found at O’Hare, visiting on three different occasions. It found more than 20 ad hoc networks each time, with 80% of them advertising free Wi-Fi access. The company also found that many of the networks were displaying fake or misleading MAC addresses, a clear sign that they were bent on mischief.

The Computerworld article focuses on the dangers of associating with ad-hoc (peer-to-peer) wireless networks and provides a number of tips for Windows users to avoid unintentionally connecting to an ad-hoc network. It seems to me just a matter of time, though, before malicious agents set up “real” (infrastructure-mode) access points in public places to collect such data. In fact, based on the airport study above, I would be shocked if it weren’t already happening. Numerous trivial variations could be very difficult to detect for the average user: for example, someone could set up a WiFi access point advertising itself as a T-Mobile HotSpot, and copy the T-Mobile web interface. If they wanted to be especially crafty, they could sit outside a Starbucks (or other known T-Mobile HotSpot) and simply use a different channel than the existing hot spot. Even a sophisticated user might have trouble telling the difference. The lesson is that you need to expect that all network traffic is insecure if not encrypted and signed at both ends. Although even in that case there are possible man-in-the-middle attacks, at least we have a cryptographic certificate/certificate authority infrastructure to mitigate the risk. (That sort of attack is also a bit more difficult to pull off, thus perhaps deterring the simplest “script-kiddie” type attackers.)

I’ve always wondered why ill-intentioned hackers don’t set up fake free WiFi hotspots in public places like airports to collect passwords and other personal information. As it turns out, they do:

Authentium did an in-depth survey of the ad hoc networks found at O’Hare, visiting on three different occasions. It found more than 20 ad hoc networks each time, with 80% of them advertising free Wi-Fi access. The company also found that many of the networks were displaying fake or misleading MAC addresses, a clear sign that they were bent on mischief.

The Computerworld article focuses on the dangers of associating with ad-hoc (peer-to-peer) wireless networks and provides a number of tips for Windows users to avoid unintentionally connecting to an ad-hoc network. It seems to me just a matter of time, though, before malicious agents set up “real” (infrastructure-mode) access points in public places to collect such data. In fact, based on the airport study above, I would be shocked if it weren’t already happening. Numerous trivial variations could be very difficult to detect for the average user: for example, someone could set up a WiFi access point advertising itself as a T-Mobile HotSpot, and copy the T-Mobile web interface. If they wanted to be especially crafty, they could sit outside a Starbucks (or other known T-Mobile HotSpot) and simply use a different channel than the existing hot spot. Even a sophisticated user might have trouble telling the difference. The lesson is that you need to expect that all network traffic is insecure if not encrypted and signed at both ends. Although even in that case there are possible man-in-the-middle attacks, at least we have a cryptographic certificate/certificate authority infrastructure to mitigate the risk. (That sort of attack is also a bit more difficult to pull off, thus perhaps deterring the simplest “script-kiddie” type attackers.)

Recently I complained about using dial-up in PR and I believe in response to my post, Joey Hess provided some great tips for making the dial-up experience a little more bearable. He, of course, was providing a set of solution to constraints, which is what the likes of Joey Hess do a lot of their waking (and according to a recent IRC conversation, even dreaming) hours. I as an academic am not involved so much in the business of finding real hard solutions but when I go to Puerto Rico, that is what I DO spend a lot of time doing. And I came up with 3 solutions this year, 2 of which I was particularly proud of (one of which may be useful to the blog readers), so I thought I would share them here: As I wrote about a few weeks ago, my mom has been plagued with a problem of choking on her saliva because she is not swallowing. It causes her and those around her a lot of angst. I tired many things but when I stopped trying and just started watching and observing to see when it seemed to be worse, I finally noticed that if she wore her dentures, the problem, while not totally resolved, was like 70-80% better. With her dentures, she would never choke on her spit and often times there would be not spit. I think the dentures changes the shape of her mouth so that she can swallow with less effort. I was pretty proud to notice this because it was the overwhelming problem of the season. Now, there is, of course, another problem. She loathes wearing those teeth and well, it has been a battle to convince her it helps (I think she sort of knows but does not want to admit it) but she is wearing them a lot more than she used to…. Maybe I will come up with a long-term solution to that problem later but I have failed in all of my attempts while still there which stretched from begging, to coy deception, to outright bribery. So in PR, we have an awesome mutt, who goes by one of three names, Isabela, Gordita (look at her tummy in the pictures to see why we call her chubster ) and Pucha. She is literally the joy of the house and she truly revels in this role. She is, in fact, like a hovercraft of love , in the sense that she will hover over you, as you are, for example, trying to get work done, and she will whimper, or lift her little paw, or more audaciously butt her head against your hand, until you assume the position and pet her, or preferably hug her. And she wants demands this sort of attention for as long as you can humanely sustain it. Ok this is not the problem part (though it can be a problem when trying to complete work and I usually just give up and give in to her or kick her out of the room). The problem was that the pucha would routinely despoil the same spot on the floor, usually every day, usually in dramatic response to leaving her in the house alone for more than 10 minutes. And I know this is willful because she is otherwise quite potty trained. She just likes to remind those who shower her with affection and love that she wants it to rain harder. The problem was that she was soaking and thus really rotting the wooden trimming (and poking around led me to discovering the third huge problem, termites, but more on that soon) and this had just had to stop. Giving her a light whallop in her chubby behind never helped (maybe she was too well padded?) and obedience school seemed like way too much effort and money. Eventually, I just threw her little bed right where she peed, and vualla, she stopped peeing there and thankfully she did not have the cojones to find another spot. I am not sure if this is long-term solution, but for now I am satisfied. Through the pucha, I found out that the wooden trimming in our house was ingested with termites. This caused a mini-heart attack, because well, termintes are a bitch to eradicate and they are a real problem in the tropics. I called the exterminators just to find out some information on termites and they assured me that once I ripped out the molding (which was my low-tech solution), I would not be putting the other wood in danger by accidentally flinging one of the little chubby white grubs over to another piece of wood and there is a lot of wood at home. Thankfully, since that is unlikely, with some help , I proceeded to tear most of the molding out only to disover a cornupia of bugs living there. It was so so so so so gross, really, gross that of course I am not replacing the trimming with wood but with some other bug-resistanant material. But that will have to wait till another trip.

On the past Monday, Yoshinori TAKESAKO, the one of the developers of Namazu, talked about Plagger in FSIJ monthly seminar. Plagger is a very useful and powerful application, and I could make a planet site in my office. It is apt-gettable but not official. The apt-line is:

deb http://debian.hansode.org/ ./ 

and the package name is libplagger-perl. However, it has no "assets", the essential components for Plagger, so you need to get them from the svn repo.

svn co http://svn.bulknews.net/repos/plagger/trunk/plagger/assets/

The following is the example of YAML:

global:
  timezone: Asia/Tokyo
  assets_path: /home/knok/plagger/assets
plugins:
  - module: Subscription::Config
    config:
      feed:
        - http://internal.host/~knok/index.rdf
  - module: Bundle::Planet
    config:
      title: Planet Office
      dir: /home/knok/planet
      url: http://internal.host/planet/
      theme: sixapart-std
      duration: 30 days
      description: The internal RSS aggregation.

I suggested in a previous blog entry that conferences should provide computers that speakers can use for their presentations. The reason for this is that getting one computer working with the beamer in each room is an easy task, while getting the laptop of every speaker to work is much more difficult.

It seems that my idea has been rejected by almost everyone who read it, so I'll document some tips for getting a laptop working.

 SZ:    Pixels          Physical       Refresh
*0   1400 x 1050   ( 474mm x 356mm )  *50
4    640 x 480    ( 474mm x 356mm )   50
5    800 x 600    ( 474mm x 356mm )   50
6   1024 x 768    ( 474mm x 356mm )   50
8   1280 x 960    ( 474mm x 356mm )   50
9   1280 x 1024   ( 474mm x 356mm )   50

Firstly there is the command xrandr which can be used to change the resolution without logging out. Above are the most useful lines produced by running xrandr with no options on my Thinkpad T41p. The left column is the index to the list of resolutions. For example I run xrandr -s 9 to use mode 1280x1024 and xrandr -s 0 to use mode 1400x1050. This takes much less time than editing an X config file!

The next thing to note is that my Thinkpad has a refresh rate of 50Hz, apparently most beamers expect at least 60Hz, this explains why I have had ongoing problems in getting my Thinkpad to correctly work for presentations for the entire time that I have owned it. If you own such a Thinkpad then I recommend that you just bring another laptop to do your presentation on the assumption that the display possibly won't work and probably won't work properly! I had developed this habit anyway after repeated problems in getting my Thinkpad working (occurring on a number of occasions in several countries). It's good to now know the reason for this (thanks Keith).

When setting the resolution there are often tweaks that can be used. For example in my talk for the Debian Miniconf of LCA 2007 I used mode 800x600 (I think - Keith set it up and I didn't look closely after verifying that things basically worked). Even though the beamer didn't have good support for a low refresh rate it worked when the resolution was low enough. Fortunately the xrandr program allows changing resolution fast enough that all 13 resolutions could be attempted in about a minute.

The support for better display detection and configuration is steadily improving. Hopefully this year the problems will be solved (which means that for the Debian and RHEL releases in 2008 the problem will be solved).

A possible work-around is to use Xephyr (the replacement for Xnest). In a previous blog entry I described how to get Xephyr going for use by Xen images. It seems to be a common symptom of display synchronization problems that the edges of the screen will be clipped. The most common work-around for this is to not use the full-screen mode of OpenOffice - which means that instead of having a small amount of text clipped there is a large amount of OpenOffice menus etc on the screen. As Xephyr accepts any resolution it should not be difficult to arrange for it to use 98% of the screen space and then run the presentation full-screen in the Xephyr window. This will be particularly useful for programs such as MagicPoint (my favorite presentation program) which don't support a windowed mode of operation.

If you have any other suggestions on how to solve or work around display problems with laptops then please leave comments.

Recently someone asked me for advice on what they can do to improve their career without getting a degree.

I have performed a quick poll of some people I know and found that for experienced people there seems to be little need for a degree. People who have extensive experience but no degree report no problems in finding work, and employers don't report any reluctance to hire someone who has the skills but no degree.

One thing that a degree is very good for is making a career jump. This is most notable when you get your first professional job, school results and references from part-time work don't help and a degree is a massive benefit. But if you have proven your abilities in the field then most employers will be more interested in checking references and the interview process than in qualifications. If you are only interested in getting a job that is one level above where you are at the moment then lacking a degree should not be a problem.

Another possibility for someone who lacks a degree is certification such as the Linux Professional Institute (LPI) provides and the Red Hat Certified Engineer (RHCE). One advantage of the RHCE certification is that it is based on fixing misconfigured Linux systems, no theoretical questions, just the type of work that real sys-admins do for their job - this means that people who do badly in traditional exams can be expected to do well, and it also means that the RHCE certification accurately depicts real skills in fixing problems (and it should therefore be more valuable to employers). The LPI exams can be taken by anyone, but to sit for an RHCE you have to be sponsored by an employer.

There are ways of getting career benefits without strictly going upwards. One way of doing this is to move to a region where the pay scales are different. Some years ago I moved from Melbourne, Australia to London to increase my salary. When in London I did work that was a lot less challenging and was paid considerably better for doing so. One thing I discovered is that in London Australians were widely admired for working really hard, I don't think that Australians work harder than British people on average, but people who will move to the other side of the world to advance their career are generally prepared to work hard!

If you spend some time working in another region and then decide to return home you will probably find that employers are more interested in hiring you for what you have learned in another region. Whether you actually learn things that are of value to potential employers when working in another country is debatable, it probably depends on the individual. But when applying for a job you want to make the most of every opportunity that is available - if someone wants to hire you for the special skills you learned in another country then that's OK. ;)

Another possibility is moving to a different industry sector. Some industries have career bottlenecks at different levels. If there is no possibility of moving upwards in the area where you work then getting a job with the same skill requirements in a different industry might open up more opportunities. An example of this is working as a sys-admin in a medium sized company that is not IT based. If you are the only sys-admin in the company then there is no possibility of promotion, moving from such a company to an ISP (or other IT based company) would then give the possibility of becoming a senior sys-admin, team leader, or even the manager of the ops team (if management is your thing).

A final option that few people consider is becoming a contractor. Contractors tend to earn significantly more than permanent employees when they do the same work (so becoming a contractor provides a significant immediate benefit) and as the duration of contracts is usually small there is less attention paid to degrees etc (what does it matter if the contractor will only be there for three months?). Of course most contracts last significantly longer than the initial term, some contractors end up working in the same position for 10 years or more!

There are some down-sides to being a contractor, one is that they get less interesting work (offering someone a choice of projects if they become a permanent employee or the project that is deemed to be least interesting if they insist on being a contractor is not uncommon). Another down-side to being a contractor is the way that contractors are used. The ideal way of running a company is to have mostly permanent employees and to use contractors for special skills, short-term projects, and for emergencies when they can't hire permanent employees. When a company has almost no permanent employees it usually means that something is going badly wrong. This means that if you select a random contract role there is a good chance that it will be one where things are going badly wrong. The money from contracting is good, but it can be depressing when projects fail.

Improving package managers I noticed two posts on improving package managers none of which mentions Debtags. Daniel Burrows mentions various issues:

• the current sections in Synaptic are useless
• there are better keyword search technologies than strstr()
• we could use popularity contest data to sort results
• it would be cool to do amazon-like things using popcon data

David Nusinov mentions that the ideal package manager should look like Google, where you search for things using just a simple one line text entry and pick from the results what you want to install. I should probably do a bit of recap of things that have been going on. I'll go through that list again:

• The current sections in Synaptic are useless

Agreed. This used to be a bug about this, which has been closed by Debtags more than one year ago. We now have much more useful category data for about 73% of the archive (including experimental), but what we lack is software using it. Here's a quick trick to try:

1. install debtags, and this gives you an easy to read text file in /var/lib/debtags/package-tags.
2. from that file, pick packages that have the tags role::program, scope::application and interface::x11.
3. display the results, and use the tags works-with::* and use::* to navigate the results.

There is a python-debian package in experimental that has a debtags module you could play with. Why is that that so far noone has written a simple package manager just for gamers, which uses only the game::* tags? Do you think Debtags gives you too many tags? Then check out:

• The Debtags smart search, and especially how it does not show you all the tags, but it is able to infer the tags you want from your google-like query (hi David!).
• The Debtags tag editor, and especially the search-as-you type feature on all the tags and the tag search (analogous to the Debtags smart search, but it only searches tags.
• The Debtags tag cloud, and if you don't like that one try to make your own: there are countless ways of generating tag clouds from Debtags data.

To summarise so far, we not only do have better categories, but also a number of cool algorithms to use them, and some interface prototypes as well. Just don't expect me to write a package manager as well: that's a job that so far I decided to leave to someone else. adept gave it a try, with positive results.

• there are better keyword search technologies than strstr()

Indeed, Xapian for example. I use it as part of the backend of the Debtags smart search, and here's our Xapian-powered normal keyword based package search interface which does stemming, indexing and all you want to ask from a serious full text index. In that page you don't see all the nice features of Xapian, but only the ones that I needed for my Debtags evil plans. Have a look at the documentation and give it a try. Here is a way to see Xapian's similarity matching in action:

1. go to the Go tagging! page
2. click on a random untagged package
3. the system gives you a rather relevant selection of tags
4. look at it again: the package was untagged: how could the web engine possibly figure those tags out?

What is happening under the scenes is that:

1. I ask Xapian: "what packages are similar to this one?".
2. I aggregate the tags of the resulting packages.
3. I rank the tags by how many resulting packages have them.

While we are on this topic, why don't we decide that we maintain a Xapian index of our package descriptions in, for example, /var/lib/apt/fulltext/, so that various applications can share it?

• we could use popularity contest data to sort results

Indeed. Anyone would like to implement this little "popcon" tool? Having the data easily accessible locally can encourage people to use them. The Debtags Go tagging! page already uses popcon data to show the most common untagged packages at the top, with double reason: it shows packages that more people are likely to know (and therefore likely to categorise) and it pushes for the most common packages to be tagged more urgently.

• it would be cool to do amazon-like things using popcon data

Indeed. Anyone volunteers to implement a prototype? The full unaggregated (but anonymised) popcon data are accessible to every Debian Developer on the host gluck.debian.org in the directory /org/popcon.debian.org/popcon-mail/all-popcon-results. Ideally one can do many interesting things with this concept: besides tag suggestions, one could identify the packages that are most representative of an installed system, and also offer negative suggestions like: "people who have packages like yours usually don't have this package: would you like to remove it?". There is more than all this that could be done. Recently, almost by accident, I had the idea of querying packages by example, like pointing to a file and find packages that can work with it. I've asked Jeroen to have Mole collect info on all files that could possibly get installed in /usr/lib/mime/packages/ (as suggested by Bernhard R. Link), to see if that prototype can be made more accurate. Query by similarity would be nice: I don't like this program, but what else do we have that does the same job? This is best implemented using Debtags data, since it directly maps to semantic properties. Note that you don't have to show a single tag to the user to implement this kind of interface. Do we have a way to point at the X window of an application and get the name of the package that installed it? Wouldn't it be about time to have it? Why don't we have a system updater utility that shows the Debian weather? Why aren't more people playing with semantic web? But more generally, the problem with package managers is that we seem to be irrationally compulsive in wanting to make the one and only big easy and complete interface for everyone. Other more reasonable people would tell you that if you have two very different kinds of users you may want to consider having two different user interfaces. Ubuntu for example installs by default 3 package manager interfaces: Synaptic; the thing that you access from the application menu to add applications to it; and the update manager. Does it sound like a waste? To me it makes lots of sense. We have lots of interesting, usable metadata; we have algorithms; we have prototypes; we have ideas for lots of cool, implementable features. The question is, are we able to write applications that just combines what is needed from all this treasure to provide the right interface(s) for our base(s) of users? Even if my English in 2004 wasn't easy to understand, a read here might still be useful. There is so much really cool stuff to be written, just within reach.

According to Wikipedia: “Burnout is a psychological term for the experience of long-term exhaustion and diminished interest (depersonalisation or cynicism), usually in the work context. Burnout is often construed as the result of a period of expending too much effort at work while having too little recovery, but it is sometimes argued that workers with particular personality traits (especially neuroticism) are more prone to experiencing burnout.” In this web page there’s a global overview of burnout that might be useful: Burnout is a chronic condition that happens when your body or mind can no longer cope with overwhelmingly high demands. You are trapped in a state of emotional exhaustion, and it is hard to get out of that state. You stop caring about what you do, even though you may feel guilty about that fact. Even if you still continue working, it seems to be hard to make progress. You hardly accomplish anything significant, just go through the motions. Certain categories of people and professions are particularly susceptible to job burnout. Most often these are people who are highly committed and motivated, who have high standards and idealistic dedication to their jobs. Common burnout causes include:

• An overwhelming workload. Could be due to insufficient time management skills, especially lack of planning, prioritizing, or delegation skills.
• Hard work with no clear goals. You work hard and hard, but no matter how long you keep at it, you cannot see any progress. But how could you see that you have got closer if you don’t know your destination?
• Powerlessness to change something important to you. Something that you are very much emotionally attached to, but that is at the same time beyond your control.
• Forcing yourself to make the impossible happen. For example, solving problems without having the necessary resources.
• A conflict between your personal values and the values of the company your working for. You don’t believe in or disagree with what your are doing, but you feel the circumstances force you to keep doing it anyway.
• Hitting the invisible ceiling. No matter how good or competent you become, there is hardly any chance of recognition or promotional opportunities.

For all of those burnout causes what is important is not as much the external factors that fall on you, but how you interpret them, what you say to yourself, and what actions you take in response. Finally, it is important to understand the risks of burnout in your personal or job situations. Once you are its victim, it may not be easy to get things back on track. That condition does not go away in a day. You may not be able to recover by yourself, and you may need to have drastic changes in your attitudes and life style. You are much better off preventing it now than putting your life back together later. Some links:

• How Can We Avoid Burnout?
• Volunteer Burnout

If you want another example, one from a company who any developer would agree is an outstanding engineering organization, here's one: "Sun has maintained binary compatibility between operating system releases for nearly a decade, enabling existing Solaris applications to run unmodified on Solaris 10. This means that Solaris applications developed 10 years ago will run on Solaris 10 unchanged, taking full advantage of new and advanced Solaris features."

Yes, it's true that there is outstanding backward binary compatibility in Solaris. However, if you're going to try use that as an argument in favour of Solaris, try again. Have you ever actually used a Solaris system?

• The ugliness on a Solaris system starts with $PATH, which is all over the place: /usr/ucb/bin for the compiler, /usr/gnu/bin (IIRC) for bash, /usr/bin for sed. The FHS is nonexistent.
• Everyone who writes code for the Berkeley socket API knows that Solaris has to be different:
  

  #ifndef sun
  int yes=1;
  #else
  char yes='1';
  #endif
  

• Even if you do it that way, on Solaris you have to say -lsocket.
• I've heard claims that Solaris, partially because of the backward compatibility, is rather slow; that Linux beats the shit out of Solaris on the hardware (SPARC) that was more or less specifically designed for Solaris (and vice versa). Having used it for a short while once, I cannot think of a counterargument for that one.

If you're going to say that backwards compatibility is good and that we should incorporate backwards compatibility into Linux somehow, then please do not point people towards the two ugliest operating systems on the earth with a lot of garbage baggage for the sake of backwards compatibility...

Ever noticed the following with Skype: Go offline while several of your friends are online. Suspend your computer. A few hours later, resume your computer and go back online with Skype. Notice how it claims all the buddys that were online when you disconnected just came online? And a few seconds later, some of them all go offline? And several minutes later, they even come back? Skype (well, at least version 1.3.0.53, YMMV) obviously caches which users it has seen and just assumes they are still online. When it then has actually connected to the Skype network, it notice that none of its peers has them and decides that they maybe aren't online after all. Then it actually searches them throughout the network, and when it has found them puts them back online again. Thats what I call cheating. Other IM services are much more reliable here. And other IM services - at least ICQ and many Jabber servers, though not Google Talk - also have offline messages. I.e. you can write messages that will reach the receipient once he goes online again. Skype again cheats on these: if you send such a message, it will after some time display the notice that sending it failed. The next time both of you are online, it will actually be delivered. This can result in suprisingly old messages, and makes connecting to people much harder: for putting someone onto your list, you also have to be online at the same time. Assuming that you are using Skype only during work hours and going offline inbetween, this makes communicating with people in different timezones a pain. File transfers with Skype are also a game of chance: If both of you are somewhat firewalled (e.g. behin a DSL-modem-router combo), it often can't transfer directly. Instead it picks a random "super node" with large bandwith for transferring the files. However, the super nodes bandwidth isn't always as good as expected by Skype, so sometimes file transfers with Skype are awfully slow, even when your line isn't busy. MSN for example (though I do NOT suggest using MSN), is using it's own, well-powered servers to handle such file transfers (or maybe all; thats why I don't really suggest using MSN, since I've only seen it transfer my files through microsofts servers...). But usually file transfers via MSN - especially in firewalled situations - are much faster than with Skype. (A drawback of them is the short timeout) So consider using XMPP/Jabber (= Google Talk) instead. They are more reliable, and an open, well-defined, extensible standard.

Ever noticed the following with Skype: Go offline while several of your friends are online. Suspend your computer. A few hours later, resume your computer and go back online with Skype. Notice how it claims all the buddys that were online when you disconnected just came online? And a few seconds later, some of them all go offline? And several minutes later, they even come back? Skype (well, at least version 1.3.0.53, YMMV) obviously caches which users it has seen and just assumes they are still online. When it then has actually connected to the Skype network, it notice that none of its peers has them and decides that they maybe aren't online after all. Then it actually searches them throughout the network, and when it has found them puts them back online again. Thats what I call cheating. Other IM services are much more reliable here. And other IM services - at least ICQ and many Jabber servers, though not Google Talk - also have offline messages. I.e. you can write messages that will reach the receipient once he goes online again. Skype again cheats on these: if you send such a message, it will after some time display the notice that sending it failed. The next time both of you are online, it will actually be delivered. This can result in suprisingly old messages, and makes connecting to people much harder: for putting someone onto your list, you also have to be online at the same time. Assuming that you are using Skype only during work hours and going offline inbetween, this makes communicating with people in different timezones a pain. File transfers with Skype are also a game of chance: If both of you are somewhat firewalled (e.g. behin a DSL-modem-router combo), it often can't transfer directly. Instead it picks a random "super node" with large bandwith for transferring the files. However, the super nodes bandwidth isn't always as good as expected by Skype, so sometimes file transfers with Skype are awfully slow, even when your line isn't busy. MSN for example (though I do NOT suggest using MSN), is using it's own, well-powered servers to handle such file transfers (or maybe all; thats why I don't really suggest using MSN, since I've only seen it transfer my files through microsofts servers...). But usually file transfers via MSN - especially in firewalled situations - are much faster than with Skype. (A drawback of them is the short timeout) So consider using XMPP/Jabber (= Google Talk) instead. They are more reliable, and an open, well-defined, extensible standard.

There were a lot of good comments on my post about the importance of backward compatibility the other day (both here and in the blogosphere), and a lot more of them were positive than I was expecting, which I find encouraging. A fair number of people called me out for using such a bad example—come on, changing the OS to fix a buggy application? Fair enough. Perhaps I diminished the point I was trying to make by referencing that extreme example, but it’s a worthwhile example for one reason: At Microsoft, the user experience comes first, notbefore developer sensibilities. Fortunately, as others (rightly) point out, it’s hard to imagine a situation where such extremes would be necessary in the Linux world—for one thing, we don’t have the sheer number of legacy binary applications to deal with, nor do we have the same volume or average user profile as Windows. But the point stands. User experience should always win. If you want another example, one from a company who any developer would agree is an outstanding engineering organization, here’s one: “Sun has maintained binary compatibility between operating system releases for nearly a decade, enabling existing Solaris applications to run unmodified on Solaris 10. This means that Solaris applications developed ten years ago will run on Solaris 10 unchanged, taking full advantage of new and advanced Solaris features.”

(17.1.2007) I haven't written a blog in aeons. I have been busy with life, as many of you know. For one, i have a girlfriend (her name is Lucy)... And, no matter what i have thought not so long ago, things are better than ever before. She's probably all that i have been looking for. We spend most of the weekends together (she lives and studies in western Slovakia, while i'm in the west-neighbouring Czech Republic (Moravia) -- somewhat surprizingly, this doesn't seem to be a problem so far). Anyway, that's probably boring for those of you that aren't me (ha), so to move on to other things. Work and school is busy these days (exam tomorrow (aaagh)), so i don't have much more free time than that i spend with Lucy. Despite that, i have done some cleaning-up on libept and adept codebases after importing them (with tailor) into darcs. Will blog about that separately i suppose, when i have more time and more stuff to report on. The exam tomorrow is on graph algorithms, and i should be actually studying instead of writing this, but i never change, do i? (19.1.2007) So the results of the exams will be available following monday. No idea whether i'm going to pass or not though. Anyway, i am sitting in a train to Slovakia, alone in a compartment, the window open wide, wind in hair. Sun is obscured by clouds at times, air is somewhat colder than it used to be. Maybe winter will finally arrive? Another exam is monday (same day as i get the results from yesterday's one)... Algorithms for NP hard problems. I'm still somewhat undermotivated to learn, and apart from that i should be hacking dmeventd (work stuff), working on my bachelor's thesis, learning to play a (soprano) recorder, learning russian and probably a ton of other things. Ah and adept, libept and friends need some attention too. I suppose i am a lost existence, given my total laziness. The train is speeding towards Breclav, last (and the only one apart from Brno on my route) station in Czech Republic. As usual, i intended to blog a ton of things, but can't recall of one them... That's life, i will hopefully get to blog a bit more again.

Python "commands" module The standard python library has a module called commands with three nice simple functions to run a command and get its output and its status. The first one is called commands.getstatusoutput(cmd), it runs the given command and returns its exit status and its output. The second one is called commands.getoutput(cmd), it runs the given command and returns its output, ignoring the exit status. The third one is called commands.getstatus(cmd), it runs the given command and returns its exit status, ignoring the output. Actually, no. The third one is called getstatus(file), it runs the command ls -ld file (with proper shell escaping) and returns its output. Seriously.

>>> commands.getstatusoutput("echo ciao")
(0, 'ciao')
>>> commands.getoutput("echo ciao")
'ciao'
>>> commands.getstatus("echo ciao")
'ls: echo ciao: No such file or directory'
>>> commands.getstatus("/bin/echo")
'-rwxr-xr-x 1 root root 19304 2006-08-31 14:25 /bin/echo'

Every time I think of this getstatus, I quickly convince myself that I misread the documentation. Then I rerun the example above.

In a previous post, I pointed out some ways that you can acheive the "compute this, now save it and compute this, ..." style of coding in a functional language such as Haskell without going mad. This is a continuation of that post.

There's one other purely syntactic feature of Haskell that I rather like for readability: Haskell lets you define variables after you use them.

GoToTheNextTrickyBit someVal
   where someVal = DoThisAndThisAndThat...

Although it isn't always appropriate to do this, there are times when I find it extremely intuitive to define my variables after they're used. For instance, to take a "practical" example, compare

mailTo customer =
   let customerName = getCustomerName customer
       customerAddr = getCustomerAddr customer
   in
       makeMailingTo customerName customerAddr

with

mailTo customer = makeMailingTo customerName customerAddr
   where
      customerName = getCustomerName customer
      customerAddr = getCustomerAddr customer

I don't know about anyone else, but when I read the first definition I mentally go "huh. Why are we extracting the customer name?", then skip down to where the name gets used, and finally re-parse the name in that context. The "where" form lets me write the code in the same order that I would read it, giving the code a natural flow. It's very important, of course, to keep the number of auxiliary definitions small; otherwise you leave too many "open questions" for the reader of your code to carry around as she works her way down to the definitions.

But it isn't all roses at the low syntactic level. IMO, the biggest practical problem with writing Haskell code is the syntactically significant indentation.

Python, of course, also has significant whitespace -- but it works mainly because it's extremely simple. Any line ending in a ":" introduces a new block, and statements within a block have the same amount of indentation. In contrast, Haskell's rule...well, just read that link above. You can write code that has the same meaning under any indentation, but it's not idiomatic to do so -- and worse, since the statement terminators are not obligatory, a missed terminator might work fine, might change the meaning of your code, or might fail to compile in an obscure way.

A side effect of the complicated indentation rules in Haskell is that the Emacs Haskell mode doesn't work very well at indenting my code. It usually chooses an indentation that I don't want, and often one that doesn't even compile. I'm sure Emacs could do better, but I would be willing to live with a little more punctuation if it meant that the code was easier for programming tools to process correctly. Functional coding advocates assert, correctly IMO, that side-effects are problematic because even smart programmers will write more bugs in their presence. I think the same logic should apply to syntax: making the language harder for humans and machines to parse has negative overall effects, even if it is possible with sufficient effort to parse it correctly.

I think that conceptually, Haskell is a great language -- the problems I see with it are issues of syntax, tool support, libraries and speed (laziness has a high price). Oh, and also the fact that, as with most programming languages that are pleasant to use, you can't get a real paying job writing in it unless you have a PhD. :-(

David Welton writes that the problem with functional languages is that you don't get to mentally "catch your breath" between computations; piling function invocations on function invocations makes it hard to understand what's actually being calculated. He has an amusing summary of how functional code can become unreadable:

Do this with the result of this with what this produces when given this input that is derived from this function which takes the result of this other thing after calculating the result of something that is derived from the output of a function that ...

Indeed, I've seen functional code like this (much of it written by myself). Here's a particular gem:

f splitPoints = map fst $ tail $ scanl (flip ($) . snd) ([], toSplit) $
          map splitAt splitPoints

I wrote this code a year ago, and I don't know what it does! It's hard to read and hard to understand. So, however, is imperative code that says

Add 5 to A, now if B is not zero we jump to where we subtract one from B and if C is less than 3 we add 4 to C otherwise we divide C by 2 and then add it to A, then we ...

The reason both pieces of pseudocode are hard to understand is that they don't use the abstraction features built into the language. Good imperative code uses meaningful variable names, uses the structured-programming primitives the language provides, and abstracts complex operations into functions.

My rule of thumb (which is, heh, not always religiously followed) is that a piece of program structure that does more than about 3-6 things is probably too complicated; it seems like that's about as much local context as I can hold in my head comfortably (and the number is more 3 than 6 these days, if you get my drift). As David puts it,

If you'll bear with a poor comparison, sometimes programming is like a song. You do the tricky section, then you go back to an easier section and catch your breath for a bit.

The problem seems to be that people trained on imperative languages, when they're tossed into the ocean of functional coding, can't find the guideposts and landmarks that would help them impose some structure on the water. Should I put this complicated expression inside a "let", or break it out into another function? When do I use a higher-order function to unify two dissimilar pieces of code? Should I write this code in terms of higher-order library functions or directly? It's easy to wander off down a chain of nested invocations of anonymous higher-order functions, or composed partially applied library functions, just "because you can" and end up forgetting what you were trying to do in the first place.

Luckily, some of the basic rules of keeping your code readable remain more or less the same: assign intermediate results to variables with meaningful names, and abstract complicated operations into separate functions. To tackle David's example:

You get sections with DoThisAndThisAndThatWithTheResultsOfSelectThisFromTableXYZ that you have to slow down to think through.

...

GoToTheNextTrickyBit where ManyThingsHappen.

All in all, you wind up with a sort of ebb and flow that works out pretty well if you're in tune with the code being written, going from intense calculations, to a line or two where you do no more than store things in a variable, take a breather, and prepare for the next important stanza.

The way to "take a breath" in a functional language is generally to assign an intermediate result to a variable, or to extract a complicated operation into a function and give it a meaningful name. One Haskell approximation to David's example is

do DoThis
   AndThis 
   results - SelectThisFromTableXYZ
   someVal - AndThat results
   GoToTheNextTrickyBit someval

If you're writing pure functional code, you might write

let
   results = SelectThisFromTableXYZ
   someVal = AndThat results
in
   GoToTheNextTrickyBit someVal

Of course, this will become unreadable if you write a huge complicated nested "let" expression in place of "AndThat". For a truly dreadful example of this kind of coding, check out the Gnucash Scheme code that generates the "Cash Flow" report (warning: read this in short bursts, or you WILL go blind). This is exactly analogous to writing too many nested loops and conditionals in imperative code, and the fix is the same: find logically distinct bits of code and extract them into separate functions.

I have a few more thoughts, but this post is getting too long, so I'll inflict them on the world in another post. The rule that short things are easier to understand applies to blog posts as well as code. :-)

I just got the Free Your Phone announcement from the OpenMoKo announce list. Looks like they'll have preview versions of the new Linux-based, Open Source phone platform available Feb 11, generally available March 11, and ready for public distribution September 11 (!). FIC is under a lot of pressure to make this product the Open Source answer to the iPhone, and I think they have a very good story. At this point I think the success of the platform is going to depend on three things. First, FIC's execution in getting developer versions out the door and ready to use. If the OpenMoKo platform stays vapourware for much past the first part of 2007, it won't be able to capitalize on the iPhone hype and current excitement. Second, FIC is going to have to resist the temptation to hold back parts of the OS or the application stack as proprietary software. Credibility in the Free Software fanatic community is going to be key for its first wave of usage, and "free enough" isn't going to be good enough for this product. A good cautionary case would be Trolltech's Qtopia. Trolltech's reputation for holding back parts of its stack has probably been the main reason this platform hasn't been taken up by Linux aficionados. Third, and perhaps most importantly, support from more hardware vendors. "Open Source" is sometimes a synonym for "second source" -- it gives customers and developers the security of knowing that the success of a platform is not tied to the success of a particular company. OpenMoKo can take the Free Software niche in the smartphone OS market -- making a place among Symbian, Pocket PC, and Palm OS that appeals to hackers. I think if they can get the hackers, they can get the kind of UI quality and application availability that will get the rest of the population interested. Firefox is an example of an Open Source project that has done this recently. All in all, I think FIC has a lot of challenges ahead -- mostly internal. But I also think that if this is played right, OpenMoKo or something like it could change smartphones the way Linux and the *BSD family have changed servers. tags: openmoko greenphone qtopia neo1973 phones mobile

Hillary's In News bulletin: Hillary's In. She's in to win. She's in to win the Big Spin. She's in up to her chin. She's in and so is her twin. She's in with a grin and a tin of aspirin. Hillary's in by the skin of her shin. She's in with the thin fin of an elfin dolphin. So it begins. tags: hillary in

Cutting Free: An interview with gNewSense founders Paul O'Malley and Brian Brazil

"if your hardware reacts badly to gNewSense, is it time for you to examine what the vendor is doing to you? If they offer some piece of hardware in exchange for your freedom and then they don't trust you to run that hardware correctly. They don't trust you, why should you trust them? As you don't have the specifications could it be the case that they are pretending that commodity is really something luxurious? How can you know that?" [note: bad site for the visually-impaired, bad colours and a graphical eyetest for comments]

Chip and SPIN! Examining the technology behind the "Chip and PIN" initiative...

"The purpose of this site is not to provide totally balanced comment on the move, but to give proper airing to the drawbacks. It's a always a tough job for onlookers to assess the truth behind anything controversial, but the greater the quantity of information, and the greater the variety of sources the better."

The Huffington Post: James Love: WIPO and the global consolidation of rights in audiovisual works

"This makes it more difficult, risky and expensive to copy, republish, remix or reuse works, for any purpose. This is a bad idea in general, but an insane idea at the very moment when we are witnessing such an explosion of new ways of creating, using, sharing and re-purposing information."

OPENSOURCE petition to the Prime Minister

"We the undersigned petition the Prime Minister to Improve Open Source use in govenment and local govenment work places."

I found a really nice way to trace back the execution stack inside a program, without the need of a debugger like gdb. I’ve made some test code in C that seems to work all right in my system. I doubt if it will just work in i386 systems, but it’ll do perfect for the debugging of the code I’m currently doing. It’s great.