Search Results: "xam"

13 February 2009

Rob Bradford: GNOME web services integration: Introducing librest

When OpenedHand was first acquired by Intel I was asked to look into ways of making it easier to access web services from within GLIBish (i.e. Clutter & GTK) applications. We figured that as web services more important to the users of Moblin, and thus by extension, GNOME we needed some way to help developers access them. So, I did some research, read up on RESTful. And thought yay, lots of services are claiming to be RESTful, let s see if we can come up with some kind of service description language that we can autogenerate code from (ala, dbus-glib). Turns out that very few services actually adhere to this RESTful definition and my attempts to produce some kind of description language resulted in more lines of XML than code, hmmm. Anyway, I persevered and looked at building an API that I would fine easy to drive RESTful web services with. After a few iterations I decided upon designing something with a somewhat Cairo-esque API.



Marco thinking RESTful thoughts.
The API is comprised of two parts, the first half is for making the requests (and is based on libsoup.) Here is an example: 
  proxy = rest_proxy_new ("http://www.flickr.com/services/rest/", FALSE);
  call = rest_proxy_new_call (proxy);
  rest_proxy_call_add_params (call,
                              "method", "flickr.test.echo",
                              "api_key", "",
                              "format", "xml",
                              NULL);
  rest_proxy_call_run (call, NULL, NULL); /* obviously can do async instead */
  payload = rest_proxy_call_get_payload (call);
  len = rest_proxy_call_get_payload_length (call);
Easy, huh? Anyway, enough rambling for today s blog post, next week i ll talk about the really interesting stuff we re (Ross and I) are working on to make parsing XML fun (yes, really, honestly, fun.) Oh and and the source code can be found in git and it s licensed under LGPLv2.1, etc etc. (Yes, we have too many libraries, when mature perhaps we can move this into libsoup.)

12 February 2009

David Welton: Javascript Charts

I've been doing some work on LangPop.com, and one of the things I'd like to do is update the chart software. What I have now, Plotr seems to work ok, but being the tinkering type, I want to fix things, even if they're not broke. Truth be told, my worry is that Plotr isn't maintained any more, so it would be a good idea to find something that's receiving a bit of attention. The candidates: These seem to be fairly up to date in the sense that someone worked on them recently. There are some older ones like PlotKit, that do not appear to be maintained any more. It is entirely possible that I missed a good one. Another possibility would be to use Google's chart API, but I'd rather be a little bit more in control of things than to farm that out, and I also am planning on doing some interactive features in the near(ish) future. So let's have a look: Flot Based on JQuery, this one looks fairly complete, and has a lot of different, nice looking charts. Since I don't really care what library it's based on JQuery seems as good as any other, being quite popular these days. Installation is pretty simple, and, having good defaults, it's easy to get nice looking data up on the screen in short order. Flotr Modeled after Flot, Flotr uses Prototype (the Rails default) instead of JQuery, and is the follow up to Plotr. I'm not quite sure what the motivation was behind the name/project change, but this seems to be where the guy is spending his time. Since he did a good job with Plotr, this one seemed worth a look too. ProtoChart This one is based on Prototype too. It immediately annoyed me by uncompressing from the .zip file in the current directory, scattering files around. I've always found this behavior a bit antisocial. This code claims to be "motivated by Flot, Flotr and PlotKit libraries", which indicate that it's fairly recent. However, my feeling is that the problem is not necessarily how old code is in these projects, but how quickly they spring up, bloom, and then stagnate. I'd like to use something that's got some staying power... But anyway, ProtoChart looks like pretty good code, even if the distribution is a bit minimal, and doesn't come with examples. If you look at LangPop.com, you'll notice something that's very important for my choice: I need a barchart that has horizontal bars, rather than the more traditional vertical bars. This is because it would be quite difficult to squeeze so many languages across the screen horizontally. This ended up being the deciding factor: like Plotr, Flotr supports horizontally oriented bar charts, making it the obvious choice. The other two libraries looked pretty good too - Flot, in particular, looks like quite solid code. So, there you have it - very brief reviews of an incomplete selection of libraries! In my defense, the goal is to attempt to dominate my maximizing nature, pick one, and get on with doing some cool stuff with LangPop. I do, however, welcome comments and suggestions.

Julian Andres Klode: debimg core: creating a repository


As I wrote Monday in debimg core example , I have an extended version which creates a repository. The problem on Monday was that the code was not far enough to be published. Not much has changed yet, but I considered to create a temporary branch temp/repository to be able to show you the example, and to give you an impression of what the final API will look like (it s not complete yet, trust me). Now I will present you the example. It is not much longer than the pool example I gave on monday, but it does much more - it creates a repository (and it has some commandline options). The concept of the Repository is very simple, too. We have two classes, Repository and Distribution, and a Repository object contains multiple Distribution objects, which are responsible for packages and related stuff. 
#!/usr/bin/python
# Copyright (C) 2009 Julian Andres Klode.
#
# Released under the terms of the GNU General Publice License, version 3
# or (at your option) any later version.
#
"""Example to demonstrate the power of debimg's repository management.
This example creates a directory example.debimg, with two subdirectories:
    - pool: This holds debimg's file pool (see debimg.core.files.Pool)
    - repo: This holds the created repository (debimg.core.repository)
It creates a repository of all required packages, using your local apt sources.
"""
from __future__ import with_statement
import operator
import optparse
import sys
from debimg.core.resolver import Resolver
from debimg.core.files import Pool
from debimg.core.repository import Distribution, Repository
def main():
    """Called when the script is executed."""
    parser = optparse.OptionParser()
    parser.add_option('-u', '--user', help='Same as in GPG')
    parser.add_option('-c', '--contents', action="store_true",
                     help='Create Contents-*.gz files')
    opts, args = parser.parse_args()
    # Create a pool, which manages the access to the files.
    pool = Pool('example.debimg/pool')
    # Create a resolver using your local apt configuration.
    pkgs = Resolver()
    # Create a new repository
    repo = Repository(pool, 'example.debimg/repo')
    # Create a new distribution and store it in dist.
    dist = repo.add_distribution('lenny')
    # Add all packages with priority required to the resolver
    pkgs.add_priority('required')
    # Add the packages from the resolver to the distribution
    for group in pkgs.groups():
        for package in group:
            dist.add_package(package)
    pool.compact()   # Optional step
    pool.fetch()     # Fetch all the packages from the mirror
    dist.finalize_files() # Link the files into the repository
    dist.finalize_packages() # Create Packages files (uncompressed and gzip)
    ##
    # If we want to create Contents-*.gz files, we can do this as well:
    # but we don't want to
    if opts.contents:
        dist.finalize_contents()
    # Now create our Release files, and sign them using the key provided on
    # the commandline. If no key has been provided, do not sign it.
    dist.finalize_release(Suite='testing', key=opts.user)
    for file in sorted(pool._files, key=operator.attrgetter('uri')):
        print file   # Print information about every file, sorted by URI.
if __name__ == '__main__':
    main()
To try the code, checkout the temp/repository branch at git://git.debian.org/users/jak/debimg.git and execute the script named test-repository, which is the same as printed above. If you have questions, please ask. Also read the docstring of debimg.core.repository to see which changes are not yet done (eg. common base class for Distribution and Repository, and of course adding files manually). Posted in Debian, debimg

Ingo Juergensmann: Automatically restore files from lost+found

I already wrote two days ago about it, but the problem then was to find a way of unique identifiers for files. Using md5sum and openssl sha1 seemed to deliver multiple files for each hash, which confused me.
A closer look later revealed that there were in fact multiple times the same file, so the same hash is of course correct.

Anywa, I'm now using the following two scripts to create an index file about the content of my filesystems and to be able to restore files from lost+found.

CODE:
#!/bin/bash
#
# Usage: ./make-lsLR.sh
#
# Purpose:
# to make a file that is parseable for recovering
# a filled /lost+found directory by parsing
# filesize, md5sum, permisions and path+filename
#
# Author: Ingo Juergensman - http://blog.windfluechter.net
# License: GPL v2, see http://gnu.org for details.
#
# first: get all directories
nice -15 find / -path /sys -prune -o \
-path /proc -prune -o \
-path /var/lib/backuppc -prune -o \
-path /var/spool/squid -prune -o \
-type d -print > /root/ls-md5sum-dirs.txt


# next: get all relevant information
nice -15 find / -path /sys -prune -o \
-path /proc -prune -o \
-path /var/lib/backuppc -prune -o \
-path /var/spool/squid -prune -o \
-type f -printf "%s %U:%G %#m " \
-exec nice -15 md5sum \; tr -s " " > /root/ls-md5sum-files.txt


CODE:
#!/bin/bash
#
# usage: check_lostfound.sh <pathtolost+found> [make_it_happen]
#
# Purpose: to find files in lost+found and trying to restore
# original files by comparing ls-md5sum-files.txt (generated by
# make-lsLR.sh
# Option make_it_happen cause the data actually being written/moved
# whereas the script runs in dry mode per default.
#
# Author: Ingo Juergensman - http://blog.windfluechter.net
# License: GPL v2, see http://gnu.org for details.
#
PFAD=$1
dry=$2

# generating md5sum/information of lost+found
echo "Examing $ PFAD /lost+found..."
find $PFAD/lost+found -type f -printf "%s %U:%G %#m " -exec nice -15 md5sum \; > /root/lostfound-files.txt


# creating missing directories if necessary by parsing ls-md5sum-dirs.txt
echo "Creating missing directories..."
for dir in cat /root/ls-md5sum-dirs.txt sed -e 's/\ /,/g' ; do
dir= echo $ dir sed -e 's/,/\ /g'
if [ ! -d "/space/check_lostfound/$ dir " ] ; then
echo " Missing dir "$dir" - creating..."
if [ "$dry" = "make_it_happen" ]; then
mkdir -p "/space/check_lostfound/$ dir "
fi
fi
done


# next, get the md5sum of files in lost+found and compare it
# against stored md5sum in ls-md5sum-files.txt
echo "Restoring/moving files..."
for z in cat /root/lostfound-files.txt tr -s " " sed -e 's/\ /,/g' ; do
z= echo $z sed -e 's/,/\ /g'

size1= echo $z awk ' print $1 '
ugid1= echo $z awk ' print $2 '
perm1= echo $z awk ' print $3 '
md5s1= echo $z awk ' print $4 '
path1= echo $z awk ' print $5 '

file= grep -m 1 $md5s1 /root/ls-md5sum-files.txt
if [ ! -z "$ file " ] ; then
file= echo $file sed -e 's/,/\ /g'
size2= echo $file awk ' print $1 '
ugid2= echo $z awk ' print $2 '
perm2= echo $file awk ' print $3 '
#md5s2= echo $file awk ' print $4 '
path2= echo $file awk ' print $5 '

if [ ! -e /space/check_lostfound/$ path2 ]; then
echo "$path1 -=> $path2"
if [ "$dry" = "make_it_happen" ]; then
cp "$ path1 " "/space/check_lostfound/$ path2 "
fi
fi
fi
done


It's actually working for me, but use them at your own risk - as always! ;)

Anyway, there are some drawbacks with these versions:

  1. it is dog slow. Grepping through 75M of ls-md5sum-files.txt for each file found in lost+found is CPU-intensive and slow.

  2. files with spaces in their names are not handled well. In fact they are not handled properly at all. You'll get "file not found" errors for each filename with spaces.

  3. currently grep -m 1 stops grepping when the first match was found.



If anyone has an idea how to speed up the script or improve it otherwise, please let me know! I already though by myself to remove found files from ls-md5sum-files.txt to reduce the workload on grepping every time all of the 75M and to work around the last point from above. But rewriting the file everytime a fiel was found and moved will put extra workload on disk I/O, which - of course - will get better over time because the file gets smaller. Another idea was to store the file within an array in memory, but then I would prefer to use a Python script for this purpose.

So, when you have some ideas, tips or improvemts, please comment!

11 February 2009

Enrico Zini: DDE, Debian Data Export

DDE, Debian Data Export After testing the idea and the prototype with my presentation at Fosdem, I'm proud to announce DDE, Debian Data Export. The problem In Debian, we publish all sort of information, using all sorts of data formats (often ad hoc and obscure), in obscure places. Try to think of an application, for example, that wants to access all this information together: A nightmare, uh? The solution DDE is a way to make it simple to publish and download data. The aim is to be able to access all sorts of Debian information without worrying about data formats, protocols and access control, and to make it easy to discover what data is available. DDE exports data as a big virtual tree. You can pick a node in the tree by its URL and download all the data that it contains, in a format of your choice: currently it supports JSON/JSONP, YAML, CSV and Python pickled objects. This means that you can now get Debian data using a trivial HTTP client tool or library, and read it using commonly available decoders: both should be available in almost any programming language nowadays. Since JSON and JSONP are supported, this even includes JavaScript in Ajaxy web pages. DDE is not a competitor to UDD: UDD is about creating a central location where all the data can be accessed, while DDE is about giving people a simple way to access data or subsets of data. In a way, DDE and UDD complete each other: the more data enters UDD, the more data is available for DDE. In turn, DDE gives a simple interface to the most popular and useful UDD queries. The dream Here are some hints at what can be done with this: A call for action You can add data to the DDE tree by just putting a data file in yaml, json or pickle format under ~/.dde (I've written a specific guide to this on the Debian wiki). If you wish to create new and fancy Debian statistics or compute other sorts of data, or if you already maintain tools that generate Debian data, including but not limited to, for example: Then please have a look at http://wiki.debian.org/DDE/StaticData and try to publish your data in ~/.dde on merkel.debian.org. For more complicated cases (like accessing a remote database), it is possible to extend DDE via python plugins. You can get in touch with me if you need to go that way.

10 February 2009

Vincent Fourmond: A makefile snippet for generaring ray-traced images from pymol

For my research, to create neat images of the molecules I study, I use Pymol. It's free, part of Debian, command-driven, very powerful and it produces really good-looking pictures. Although, admittedly, if you don't buy a subscription, the documentation is rather sparse. I'm using scripts to generate ray-traced pictures, but I can't include the ray command directly in the scripts, as Pymol would not give me the opportunity to change anything before tracing (and things I trace take loads of time...). So I write a ray command, but I comment it out, such as in the following example, cytochrome.pml: 
load http://www.rcsb.org/pdb/files/1c6s.pdb
hide everything
show sticks, (resname HEC)
color orange, (resname HEC)
show cartoon
bg_color white
set_view (\
     0.939850032,    0.167392358,    0.297758341,\
     0.336208642,   -0.299271941,   -0.892972171,\
    -0.060366094,    0.939367712,   -0.337551534,\
     0.000000000,    0.000000000, -106.161231995,\
    33.668910980,   -4.749122620,  -24.822664261,\
    -9.742469788,  222.064941406,    0.000000000 )
# ray 800,800
So, to easily generate PNG images from such a file, I wrote the following makefile snippet, that does the job (as demonstrated by the figure on the right, produced by this very snippet ran on the cytochrome.pml file above): 
%.png: %.pml
 cd $(dir $<); \
 ray_trace_args= grep -o 'ray.*' $(notdir $<)  tail -n1   egrep -o '([0-9, ]+)'   sed 's/ //g'  ; \
 pymol -c -d "@$(notdir $<)" -d "cmd.ray($$ray_trace_args)" -g $@
I hope this snippet will prove useful for others. At any rate, it will prove useful for me ! Just for the record, this protein, represented in cartoon view, is called cytochrome c6 and is one of the proteins implied in the photosynthesis of some organisms.

Ingo Juergensmann: Unique fingerprints of files?

End of last year I migrated from PowerPC arch to i386 which made it necessary to reformat/reparition/relabel the old 250 GB disk from Amiga disk label to PC disk label. Therefor I backupped all necessary files from the 250 GB disk to a 160 GB disk.
After installing the system on the 250 GB disk, I wanted to copy over the backup from the other disk, but accidently issued a pvcreate on the backup partition. Big mistake. ;) No need to tell that lots of files ended in lost+found: 28 GB in total. Of course it's troublesome to sort all those files and directories back to their original places.

Because I don't know any tool yet, I wrote some scripts to address this issue. The first script is to be run to get a list of directories and files and a unique file identifier for the files.

CODE:
#!/bin/bash
#
# Purpose of this file:
# to make a file that is parseable for recovering
# a filled /lost+found directory by parsing
# filesize, md5sum, permisions and path+filename
#

# first: get all directories
nice -15 find / -path /sys -prune -o \
-path /proc -prune -o \
-path /var/lib/backuppc -prune -o \
-path /var/spool/squid -prune -o \
-type d -print > /root/ls-md5sum-dirs.txt


# next: get all relevant information
nice -15 find / -path /sys -prune -o \
-path /proc -prune -o \
-path /var/lib/backuppc -prune -o \
-path /var/spool/squid -prune -o \
-type f -printf "%s %m " \
-exec openssl sha1 \; sed -e 's/SHA1(//g' -e 's/)=//g' > /root/ls-sha1sum-files.txt
#-exec nice -15 md5sum \; > /root/ls-md5sum-files.txt


This will create two files: /root/ls-md5sum-dirs.txt and /root/ls-md5sum-files.txt. The above is the second version of the file with openssl sha1 instead of md5sum, because I discovered that md5sum is giving me double hashes:

CODE:
muaddib:~# wc -l ls-md5sum-files.txt
712251
muaddib:~# cat ls-md5sum-files.txt awk ' print $3 ' sort -u wc -l
576539


That makes a difference of ~140000 files. The same script with openssl sha1 is giving a better approximation, but still a gap between those two numbers:

CODE:
muaddib:~# wc -l ls-sha1sum-files.txt
712137 ls-sha1sum-files.txt
muaddib:~# cat ls-sha1sum-files.txt awk ' print $4 ' uniq wc -l
694200


The difference is much smaller with openssl but it takes a lot of more time to generate the resulting file.

Anyway, my second script assumes that the files can be identified by an unique hash. Currently it looks like this:

CODE:
#!/bin/bash

PFAD=$1
dry=$2

# generating md5sum/information of lost+found
echo "Examing $ PFAD /lost+found..."
find $PFAD/lost+found -type f -printf "%s %m " -exec nice -15 md5sum \; > /root/lostfound-files.txt


# creating missing directories if necessary by parsing ls-md5sum-dirs.txt
echo "Creating missing directories..."
for dir in cat /root/ls-md5sum-dirs.txt sed -e 's/\ /,/g' ; do
dir= echo $ dir sed -e 's/,/\ /g'
if [ ! -d "$ dir " ] ; then
echo " Missing dir "$dir" - creating..."
if [ "$dry" = "make_it_happen" ]; then
echo mkdir "$ dir "
fi
fi
done


# next, get the md5sum of files in lost+found and compare it
# against stored md5sum in ls-md5sum-files.txt
echo "Restoring/moving files..."
for z in cat /root/lostfound-files.txt tr -s " " sed -e 's/\ /,/g' ; do
z= echo $z sed -e 's/,/\ /g'

#size1= echo $z awk ' print $1 '
#perm1= echo $z awk ' print $2 '
md5s1= echo $z awk ' print $3 '
path1= echo $z awk ' print $4 '

file= grep $md5s1 /root/ls-md5sum-files.txt
if [ ! -z "$ file " ] ; then
file= echo $file sed -e 's/,/\ /g'
#size2= echo $file awk ' print $1 '
#perm2= echo $file awk ' print $2 '
#md5s2= echo $file awk ' print $3 '
path2= echo $file awk ' print $4 '

echo "$path1 -=> $path2"
if [ "$dry" = "make_it_happen" ]; then
echo mv "$ path1 " "$ path2 "
fi
fi
done


This script generates a similar list as the first script and compares each hash or file and tries to move it back to its original path. Because of the above mentioned difference between total number of lines and unique liines this approach seems a little bit broken.

So, dear lazyweb, is there something that can generate unique fingerprints in a reasonable amount of time (like md5sum) or is there even an already existing application out there that I'm not aware of? And yes: I know that there are backup applications, but that is another topic. In this case, the backup medium has been corrupt so the problem still exists: how to identify files in lost+found and move them back in an automatic way, if possible?

I'm appreciating help, tips, patches, ideas, ... :)

9 February 2009

Steve Kemp: It is an army bred for a single purpose

It is funny the way things work out when you're looking for help. Recently I was working on a Ruby + FUSE based filesystem and as part of the development I added simple diagnostic output via trivial code such as this: 
@debug && puts "called foo(# param );"
That was adequate for minimal interactive use, but not so good for real live use. In real live use I started outputing messages to a dedicated logfile, but in practise became overwhelmed by thousands of lines of output describing everything ever applied to the filesystem. I figured the natural solution was to have a ring-buffer. (Everybody knows what a ringbuffer is, right?) It could keep the last 500 messages and newer debug information would just replace older entreis. That'd be just enough to be useful if I had a problem, but not so overwhelming it would get ignored. In Perl I found a nice ringbuffer library, but for Ruby nothing. Locking a region of shared memory via shmget, shmset and keeping an array of a few hunded strings would be simple, but it seems odd I have to code this myself. I started searching around and I accidentally stumbled upon the unrelated IPC::DirQueue perl module. Not useful for my ringbuffer logging problem, but beautifully useful. There is no package for Debian but that was easily created: 
dh-make-perl --build --cpan IPC::DirQueue
Already I have a million and one uses for it - not least to solve my problem of maintaining a centralised quarantine for all the spam mail rejected by N MX machines. (Which currently uses a combination of rsync and lockfiles.) This is the reason why sites like Perl Advent Calendar are useful - they introduce a useful module every day or two, and introduce you to thinks that you can use in the future. Of course keeping a sustainable site like that up and running is hard which is why sites like debaday struggle to attract contributors, for example. Anyway random happyness. ObFilm: Lord of the rings: Two Towers

Julian Andres Klode: debimg core example


The following example demonstrates the features of debimg core, and how it can be used to fetch a some packages. As you will see when you run this example, debimg uses SHA1 filenames for the downloaded files. This may be changed in a future version. There is also an improved version of this example, which creates a repository, but the needed module (debimg.core.repository) is not public yet, because its far from being finished. I expect to complete repository code on Wednesday. 
#!/usr/bin/python
# Copyright (C) 2009 Julian Andres Klode.
#
# Released under the terms of the GNU General Publice License, version 3
# or (at your option) any later version.
#
"""Example to demonstrate the power of debimg's pool
This example creates a directory example.debimg, with two subdirectories:
- pool: This holds debimg's file pool (see debimg.core.files.Pool)
"""
from debimg.core.resolver import Resolver
from debimg.core.files import Pool
def main():
    """Called when the script is executed."""
    # Create a pool, which manages the access to the files.
    pool = Pool('example.debimg/pool')
    # Create a resolver using your local apt configuration.
    pkgs = Resolver()
    # Add all packages with priority required to the resolver
    pkgs.add_priority('required')
    # Add the packages from the resolver to the pool
    for group in pkgs.groups():
        for package in group:
            pool.add_package(package)
    pool.fetch() # Fetch all the packages from the mirror
    for file in sorted(pool._files, key=lambda k: k.uri):
        print file # Print information about every file, sorted by URI.
if __name__ == '__main__':
    main()
Posted in Debian, debimg

Julian Andres Klode: Really, don t use codenames for pinning.


You can not use codenames for pinning packages to a specific release. Everything like this just fails, and can result in apt not finding any candidate for a package. You can see this for example in Bug#512318, and Bug#513864 seems to be related to that one. If you want to target Lenny and you don t have Etch in your sources.list, pin stable at a higher priority than testing. This means that the pin automatically migrates once Lenny becomes stable (which happens hopefully in 2 weeks). Posted in Debian

8 February 2009

Brandon Holtsclaw: Twitter vs. SMS ( text ) Messages

You know, the more I ve been using Twitter the last week or so ( @imbrandon ) I have noticed many things that I ll be blogging about. But one of those such things is the language used on Twitter by large is not the same Language that is used in SMS messages even though the two technologies while very different have quite a bit that would effect the language used in common such as the 140 character limit on tweets and that largely the people that use twitter also make use of SMS messages too. SMS messages always seem to be much more abbreviated than Tweets , here is an example ( fictional to explain the point better ) : Twitter to Mark : @MarkRosky Me and Sarah are having coffee at Starbucks on 4th and Grand, come join us if you have time.
SMS to Mark : grabbin cup o java at da spot. get down here Now I m curious if this is because most people started ( and continue ) to use SMS via limited phone keyboards instead of fuller featured ones on Blackberries, iPhones, and other smart phones ? Is it a mindset when using a device other than a full computer , e.g. when you send a tweet via SMS does it follow the SMS language or Twitter s ? Or am I mistaken and its a whole different demographic ? I d love to hear your thoughts in the comments

Russell Coker: Normalising Wages

John Robb writes about the normalisation of salaries that is driven by the use of the Internet and global corporations [1]. He cites an example of IBM forcing many of it s employees to work in developing countries for lower wages. It seems to me that IBM is leading the field in this regard and many other companies will do the same. The computer industry which has been very well paid for the last 20 years seems likely to have some significant reductions in wages. Some computer jobs can t be immediately outsourced to other companies such as network cabling, hardware repair, and training. But they will experience a pay reduction due to competition from software engineers and other people who s primary work is easily outsourced. For a job hunter it seems that the best thing to do is to look for work that requires site visits and customer contact. One of my long-term clients has a large business in installing wireless devices. For a long time they have had an open offer for me to climb up towers and install wireless devices. If the rates for system administration and software engineering in Australia (and the outsourced work from the US that I sometimes do) drops to Indian pay standards then I might do some of that wireless work. When recommending that my clients hire people to do software engineering or programming work I am considering seeking out people in low-wages countries that I know through the free software community. I believe that through an organisation like Debian I can find people who are as good as the people I have been finding through my local LUG but who can feel happy earning a much lower rate. My clients seem to periodically need PHP work, so if you live in a low wages country, have good PHP skills, and are well known in the free software community to a degree that I can feel happy in skipping personal meetings then you can email me your CV. Don Marti has written about the Linux vs Windows situation on Netbooks [2]. He suggests that neither Intel nor Microsoft is set up for a Netbook world. If both Intel and Microsoft were to pay the majority of their employees rates that are only slightly greater than typical wages in India then things might look a little better for them. During the dot-com boom I was working in Europe (firstly in London and then later in Amsterdam). It was a lot of fun in many ways with large amounts of money, easy work, and parties. The down-side was that I had to work with druggies and other people who were not suitable employees as the companies I worked for felt that they had no choice. Since the dot-com crash the quality of the people I have worked with has increased significantly which was a good compensation for the lower pay. Also I believe that the lack of silly money was one factor that helped Linux and other free software increase market share after the crash. I expect that the economic depression that we are now entering will have a similar effect. I will earn less money, have fewer parties, and work harder. But generally the quality of the staff in the IT industry will improve and the usage of free software will increase. As an aside, Don suggests that people who create web sites etc will want more expensive machines. I regularly use my EeePC for running servers in several data centers, I do all types of system administration and system programming on it. The screen resolution is not that great, but it shouldn t be difficult to design a Netbook that can drive a 1920*1200 external display (as is being commonly deployed in new hotel rooms - it seems that all new large-screen TVs have VGA and DVI input). A netbook which can drive a 1920*1200 display and a full-size USB keyboard could allow me to do some very effective work in a hotel room while also allowing me to work when traveling in public transport. Now all I need is for the hotel booking sites such as www.wotif.com to allow me to search for rooms that have such a display. A business hotel could even provide a USB keyboard in the room to allow guests to travel light. Of course it would be possible to design a slightly larger laptop at a Netbook price point, the extra plastic and metal needed to make a larger frame and keyboard costs almost nothing and given the low prices of large desktop TFT displays I find it difficult to believe that the factor of two difference in price between the cheaper and more expensive Netbooks is due to the display. My main machine is a Thinkpad T41p, I really don t need a high-end machine on my desktop. I am considering the practice of avoiding purchasing expensive machines as a matter of principle. Maybe if we try and avoid buying expensive machines we can help drive the market towards Netbooks where Linux has an advantage.

Joey Hess: focusing on the screen

Biella asks if I can read novels from a screen. Yes, I've read what has to be getting close to a thousand novels that way, over the past 8+ years. I was thinking about this just the other day as I read The once and future e-book: on reading in the digital age. That article does a good job arguing against many of the common complaints about ebooks, such as "I can't read an ebook in the bathtub".
my pile of paper
Right now, I have a pile of paper books I want to read or am in the middle of reading. But if I want something new to read, I don't reach for that pile, but for my (much larger) directory of ebooks. The pile of paper is reserved for special occasions: plane trips, camping, etc. I started reading ebooks in circumstances where it was too hard to get access to paper ones. Traveling in the third world, hauling along a lot of paper books wasn't practical. Living rural, where every book had to be backpacked in. I've stuck with it because it works for me and because I've enjoyed stretching my muscles of concentration on the screen. To me, the focus required to read an ebook from the screen is similar to the focus required to work on a program. And completly different than skimming through a web page. The best example of this that I have right now is Real World Haskell, which I've almost finished (hurrah!) reading in a combination of paper and web editions. I was only able to stick with reading this book when I got the paper version. I use the web version for reference, or for looking for help on a challenging part of the book. For reading, it's just too much a web page; too easy to skim, or be distracted by the comments that can be attached to every paragraph, or jump off to Wikipedia.
screenshot of my ebook reader
Just as most programmers tune their development environment so that it uniquely suites them, I've put a lot of thought, trial, and error into tuning my ebook reading environment. The most important thing seemed to be getting rid of anything that interfered with focusing on the text. Since I wasn't (easily) able to use that environment for reading Real World Haskell, I wasn't able to read it from the screen.

Biella Coleman: People of the screen

Geeks are creatures of the screen. Many, I have found, are also great lovers of books. I have peered into the homes and apartment of many and books usually adorn the walls and the tables. The exist in abundance. Though I am more or less paid to read for a living but these days, I don t seem to the time to do extra reading and I often mourn the fact that I spend so much time getting scraps of stories and information from the net as opposed to delving into a great book, over the course of weeks and weeks (and as opposed to reading, often hurriedly but totally intensely, for class).
I do manage to get in some good articles and last night, I pushed through the heaviness of sleepiness to read a pretty interesting article People of the Screen by Christine Rosen. Though a bit too alarmist for my taste, I enjoyed reading it, largely because it was written well and also brought some interesting questions and points to the table about the transition from a print culture to a digital one. One point, that I was not surprised to read, is the most avid of screen users (programmers, the digerati), are also avid readers. But among the less (economically) privileged, who have different educational relationships to books and who also may not have the time to read, reading in the traditional sense, is may soon be a more or less historical fact. What I also enjoyed reading about and contemplating are the different material properties of the screen vs print book and the ways in which these affordances might create a different user/intellectual (almost existential, the author would argue) experience. For example, she argues that the emotional relationship to treeware books are more profound, or that reading a long novel (vs playing a video game) is about submitting your will at least for a while to the narrative and the story. I don t agree with all of her assessments but I certainly agree that there are phenomenological differences between reading words on sheets of paper and the pages of a compact book, and then reading from the blue hue of a screen, often sitting upright, and whose text, at least for me, does not seem quite alive as it is when imprinted on bounded paper. But I don t think this necessarily has anything to do with some inherent properties of the screen. For example, writing on the screen does nothing but enliven text for me. When it comes to writing, I cannot imagine doing it any way but via the screen and the keyboard. I can play around with some words and erase with impunity. I can move around a whole paragraph here and there, split it into two or three and as such it feels dynamic and quite alive. I am sure I have lost, or there is something to lose, by not going straight from brain to pen but I never did all that much serious writing without the keyboard and so perhaps there was nothing to lose, no bodily transitional for me to undergo, and thus nothing to really mourn. So, I suspect many readers of this blog are voracious writers and readers, on screen and off screen. Do you, can you read novels on the screen? If you love the printed book, why? Are you concerned about the loss of this medium? Or is there nothing to really worry about since technologists will eventually create a new digital medium that will surpass, in terms of a good user experience, the printed book?

Biella Coleman: People of the screen

Geeks are creatures of the screen. Many, I have found, are also great lovers of books. I have peered into the homes and apartment of many and books usually adorn the walls and the tables. The exist in abundance. Though I am more or less paid to read for a living but these days, I don t seem to the time to do extra reading and I often mourn the fact that I spend so much time getting scraps of stories and information from the net as opposed to delving into a great book, over the course of weeks and weeks (and as opposed to reading, often hurriedly but totally intensely, for class).
I do manage to get in some good articles and last night, I pushed through the heaviness of sleepiness to read a pretty interesting article People of the Screen by Christine Rosen. Though a bit too alarmist for my taste, I enjoyed reading it, largely because it was written well and also brought some interesting questions and points to the table about the transition from a print culture to a digital one. One point, that I was not surprised to read, is the most avid of screen users (programmers, the digerati), are also avid readers. But among the less (economically) privileged, who have different educational relationships to books and who also may not have the time to read, reading in the traditional sense, is may soon be a more or less historical fact. What I also enjoyed reading about and contemplating are the different material properties of the screen vs print book and the ways in which these affordances might create a different user/intellectual (almost existential, the author would argue) experience. For example, she argues that the emotional relationship to treeware books are more profound, or that reading a long novel (vs playing a video game) is about submitting your will at least for a while to the narrative and the story. I don t agree with all of her assessments but I certainly agree that there are phenomenological differences between reading words on sheets of paper and the pages of a compact book, and then reading from the blue hue of a screen, often sitting upright, and whose text, at least for me, does not seem quite alive as it is when imprinted on bounded paper. But I don t think this necessarily has anything to do with some inherent properties of the screen. For example, writing on the screen does nothing but enliven text for me. When it comes to writing, I cannot imagine doing it any way but via the screen and the keyboard. I can play around with some words and erase with impunity. I can move around a whole paragraph here and there, split it into two or three and as such it feels dynamic and quite alive. I am sure I have lost, or there is something to lose, by not going straight from brain to pen but I never did all that much serious writing without the keyboard and so perhaps there was nothing to lose, no bodily transitional for me to undergo, and thus nothing to really mourn. So, I suspect many readers of this blog are voracious writers and readers, on screen and off screen. Do you, can you read novels on the screen? If you love the printed book, why? Are you concerned about the loss of this medium? Or is there nothing to really worry about since technologists will eventually create a new digital medium that will surpass, in terms of a good user experience, the printed book?

Uwe Hermann: List of my Creative Commons licensed photos being used elsewhere

As you may know I maintain a Creative Commons licensed photoblog at my website. I'm also cross-posting some of the better photos to my flickr page. Even with my humble, and not really widely-known little photoblog, you can already see the Creative Commons license's effects on media sharing and remixing/reusing kick in. Quite a number of my photos have already been used by other people for various different purposes (blogs posts, articles, even album covers), including some of the "bigger" sites such as the Wall Street Journal Blog or Cult of Mac... Here's the list of places I know of where my photos are used. Please leave a comment if you spot more of them in the wild. I intend to keep this list updated as more of my photos appear elsewhere. (Oh, and I have no idea why people seem to be so obsessed with my "Sugar" photo...) Sugar Sugar
Clock Clock
Autumn Leaf Autumn Leaf
Scissors Scrissors
Soccer World Championship 2006 Soccer World Championship 2006
Organized Organized
Dandelion Dandelion II
Intel Celeron CPU Intel Celeron CPU
Smoke Smoke
Sun and trees Sun and Trees
POSIX.1g Posix.1g

Matthew Palmer: English Comprehension Skills

The phrase "Total Fire Ban" is bandied around a lot these days -- mostly on days with high temperatures, low humidity, and high winds. To me, it isn't a particularly hard concept to understand, either: in the immortal words of Metallica, "Fire bad, Aircon goood!". And yet, some people just don't seem to get it. This afternoon, my brigade got a page for a fire call to a reported "smoking log" behind town. When we got to where the log was, indeed, smoking, it was obvious that a group of people had had a large party around a bonfire last night. Given the prevalence of large-scale fire incidents at the moment, anyone who feels that lighting up a bonfire, cracking open a large quantity of alcohol, and then leaving the damn fire going in the morning, is a good idea, really needs to have their head examined. In other firefighting-related news, despite being the on-call brigade for our district this week, we've not been called to assist at any of the fires currently burning both to our north and south. It's frustrating that having been trained to fight bushfires, and with so many areas currently needing some serious firefighting, my district isn't getting asked to help out. On the other hand, they don't appear to be hurting for resources at these fires (with several hundred RFS personnel, not to mention NSWFB and NPWS people, at the Peats Ridge fire; also, some 200 RFS personnel have been dispatched to Victoria), so it's good to know that there's plenty of bush firefighting volunteers around if things get bad around here.

5 February 2009

Meike Reichle: Looking for free projects!

Dear all, I am currently assembling a talk that discusses if and how the principles of free software can be applied to topics beyond operating systems and software. The first things that come to mind are surely the Creative Commons, free music, free films, free books etc. but I am sure that there is more, also even beyond the digital world. I already know a couple projects but I am looking for many many more! So, what examples do you know for applications of the free software philosophy that go beyond the usual software/OS area? I'd be happy for any hints, urls, projects, authors, bands, names etc. that come to your mind! To contact me, simply drop me an e-mail or prod me on irc (alphascorpii on OFTC and freenode).

Frans Pop: Debian on HP 2510p

In August I got a very nice HP 2510p notebook which is now my main system for development. This took a while for two reasons. First of all the system did not resume 100% reliably. This has now been solved, although the final patches needed for that will only be in 2.6.29, but that's no problem as I run upstream kernels anyway (I do quite a bit of kernel testing). Kudos have to go to Rafael J. Wysocki who has been doing a huge amount of work to improve the suspend/resume code in the kernel. Second of all I wanted a docking station so I could continue to use my 19" monitor, with the laptop's LCD as second display. I bought the docking station in December. The challenge then was to automatically (de)activate the external display when the notebook is (un)docked. Unfortunately there are no ACPI events, but the hp-wmi module (written by Mattew Garrett) sends docking events to the input subsystem. I wrote a small program to catch these events and a script that uses xrandr to switch displays. There were two issues with that setup. The first docking event was getting lost, but I managed to fix that. And the X server would crash when starting some applications (einstein and virtualbox) after undocking, but there's a patch for that too now. The notebook is now really well supported and stable. The system is currently running Debian/lenny with a KDE desktop and a 2.6.29-rc3 kernel. Working with upstream developers to get this far has really been worthwhile, and fun.

Chris Lamb: Scrap your boilerplate; POSIX shell edition

I don't remember where I saw this first - probably in the Debian Installer code. Anyway, given an n-tuple it can be somewhat unwieldy to unpack it: 
VAR="Aho Sethi Ullman"
ONE="$(echo $ VAR    cut -d' ' -f1)"
TWO="$(echo $ VAR    cut -d' ' -f2)"
THREE="$(echo $ VAR    cut -d' ' -f3)"
The trick being presented is to generate the boilerplate code and then eval it: 
VAR="Aho Sethi Ullman"
eval $(echo $ VAR    awk '  print "ONE=" $1 " TWO=" $2 " THREE=" $3  ')
Here's a more concrete example, with apologies to Python: 
for PAIR in \
           "one    Aho" \
           "two    Sethi" \
           "three  Ullman"
do
    eval $(echo $ PAIR    awk '  print "k=" $1 " v=" $2  ')
    # .. do something with $k and $v
done

Next.

Previous.