Search Results: "ag"

3 July 2011

Benjamin Drung: libkibi 0.1 released

The first version of libkibi is released. This library is designed for formatting sizes in bytes for display. The user can configure a preferred prefix style. Packages for Debian unstable and Ubuntu 11.04 (maverick) are uploaded. It contains a README for developers and a byteprefix man page for users, which can be read with man 5 byteprefix once installed. Thanks to Stefano Rivera for writing the man page! For a demonstration how this library used by an application can look like, read my previous post.

Benjamin Drung: How to purge Wine completely

I had Wine installed and wanted to get rid of it completely. I removed the Wine Debian package and what belong to it, but the Windows applications still appeared in the GNOME main menu. I removed the .wine directory and all local wine-related desktop files:

rm -rf ~/.wine ~/.local/share/applications/wine

rm -f ~/.local/share/applications/wine-* ~/.local/share/applications/mimeinfo.cache

Now there is no sign left that Wine was installed.

Benjamin Drung: Symptoms of a dying graphic card

All started in December last year. I saw a red and a green pixel on my monitor. Moving the window containing the wrong-colors pixels corrected them. The wrong-colored pixels reappear from time to time in increased quantity. Then a second symptom appeared: The screen went black and came back one second later. This was triggered by moving a window and scrolling. I wasn t sure what the reason was. Either it was a bug somewhere in the X stack or some hardware was dying. dmesg showed multiple problems with the radeon driver:

[283808.667454] radeon 0000:01:00.0: ffff88021f815c00 unpin not necessary
[283808.667820] radeon 0000:01:00.0: GPU softreset
[283808.667823] radeon 0000:01:00.0: R_008010_GRBM_STATUS=0xE57024A4
[283808.667825] radeon 0000:01:00.0: R_008014_GRBM_STATUS2=0 00330302
[283808.667826] radeon 0000:01:00.0: R_000E50_SRBM_STATUS=0x200000C0
[283808.667832] radeon 0000:01:00.0: R_008020_GRBM_SOFT_RESET=0x00007FEE
[283808.682844] radeon 0000:01:00.0: R_008020_GRBM_SOFT_RESET=0 00000001
[283808.698840] radeon 0000:01:00.0: R_008010_GRBM_STATUS=0 00003028
[283808.698843] radeon 0000:01:00.0: R_008014_GRBM_STATUS2=0 00000002
[283808.698845] radeon 0000:01:00.0: R_000E50_SRBM_STATUS=0x200000C0
[283808.699845] radeon 0000:01:00.0: GPU reset succeed
[283808.717570] [drm] Clocks initialized !
[283808.765829] [drm] ring test succeeded in 0 usecs
[283808.765838] [drm] ib test succeeded in 1 usecs
[283808.765840] [drm] Enabling audio support
[283812.521265] radeon 0000:01:00.0: GPU lockup CP stall for more than 1000msec
[283812.521269] [ cut here ]
[283812.521294] WARNING: at /build/buildd/linux-2.6.35/drivers/gpu/drm/radeon/radeon_fence.c:235 radeon_fence_wait+0 365/0x3d0 [radeon]()
[283812.521297] Hardware name:
[283812.521299] GPU lockup (waiting for 0x00AE670A last fence id 0x00AE6705)
[283812.521301] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs nls_utf8 udf ip6table_filter ip6_tables binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_intel kvm parport_pc ppdev snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_emu10k1 snd_ac97_codec ac97_bus snd_pcm snd_page_alloc snd_util_mem snd_hwdep snd_seq_midi radeon snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device ttm snd pl2303 drm_kms_helper coretemp usbserial joydev psmouse soundcore drm serio_raw intel_agp i2c_algo_bit lp parport hid_cherry usbhid hid firewire_ohci firewire_core usb_storage crc_itu_t e1000e ahci libahci pata_marvell
[283812.521360] Pid: 2194, comm: compiz Tainted: G W 2.6.35-24-generic #42-Ubuntu
[283812.521362] Call Trace:
[283812.521370] [<ffffffff8106089f>] warn_slowpath_common+0x7f/0xc0
[283812.521374] [<ffffffff81060996>] warn_slowpath_fmt+0 46/0 50
[283812.521390] [<ffffffffa01bd775>] radeon_fence_wait+0 365/0x3d0 [radeon]
[283812.521394] [<ffffffff8107f730>] ? autoremove_wake_function+0 0/0 40
[283812.521410] [<ffffffffa01bdf71>] radeon_sync_obj_wait+0 11/0 20 [radeon]
[283812.521418] [<ffffffffa01751a3>] ttm_bo_wait+0 103/0x1c0 [ttm]
[283812.521435] [<ffffffffa01d4e1a>] radeon_gem_wait_idle_ioctl+0x9a/0 150 [radeon]
[283812.521447] [<ffffffffa010f433>] drm_ioctl+0 463/0 520 [drm]
[283812.521465] [<ffffffffa01d4d80>] ? radeon_gem_wait_idle_ioctl+0 0/0 150 [radeon]
[283812.521470] [<ffffffff81162f0d>] vfs_ioctl+0x3d/0xd0
[283812.521473] [<ffffffff811637e1>] do_vfs_ioctl+0 81/0 340
[283812.521477] [<ffffffff811535f1>] ? vfs_read+0 181/0x1a0
[283812.521480] [<ffffffff81163b21>] sys_ioctl+0 81/0xa0
[283812.521484] [<ffffffff8100a0f2>] system_call_fastpath+0 16/0x1b
[283812.521487] [ end trace 6d5e03bab743abfa ]
[283812.521493] [drm] Disabling audio support
[283812.525575] [drm:radeon_ib_schedule] *ERROR* radeon: couldn t schedule IB(10).
[283812.525579] [drm:radeon_cs_ioctl] *ERROR* Faild to schedule IB !
[283812.527021] [drm:radeon_ib_schedule] *ERROR* radeon: couldn t schedule IB(11).
[283812.527024] [drm:radeon_cs_ioctl] *ERROR* Faild to schedule IB !
[283812.527921] [drm:radeon_ib_schedule] *ERROR* radeon: couldn t schedule IB(12).
[283812.527923] [drm:radeon_cs_ioctl] *ERROR* Faild to schedule IB !

The time between going black reduces every time. The system wasn t usable any more in the end. After testing that the screen was going blank on other systems like a live CD of Ubuntu and a not noteworthy proprietary system, it was clear that some hardware component was dying. My first assumption was verified after replacing it: My graphic card, a Radeon HD 4670, died. This was three weeks ago and around 25 month after I bought the graphics card. So I didn t have to worry if replacing the fan voided the guarantee, because the guarantee lasts only 24 month. Conclusion:

Not every error is a software bug.
The hardware component with the highest failure rate is the graphics card, followed by the motherboard. Four graphics cards died last year (in four different systems owned by four different people, in three different households).
Not every error is a software bug.

Mesutcan Kurt: Python Multi-Build for Python Extensions Packaging - Report 3

Hello,

This is the third report of my GSoC project for Debian. During last two weeks

I've done most of things about the main tool. Before that weeks I have been

working on project's API. API is almost finished. Still I find missing/wrong

parts during the coding and changing the API design.

Like I said, main tool's coding process is nearly finished. For now It is doing

its job for building Python3.X packages. You can find the source code at [1].

I also coded an example plugin (actually only return values) and plugins/__init__.py

file which contains like Base class, exception classes and plugin listing method.

When I started coding I realized that I need extra parameters. I've added --install-dir

parameter. You can specify the install directory if there is no install directory the tools

pretends that the install directory is "./debian/tmp". After that it checks whether if the

directory exists. If not it creates the directory.

I've changed py3build.py a bit. First, I've thought for every process I invoke py3build's

methods but they are redundant. I've removed them. Now they are called in main

method of py3build.

I use logging module for printing messages, errors etc.

get_build_system() method retieves list from plugins/__init__.py file. In that file there is

scan_for_plugins() method. It finds all plugins in plugins/ directory which files match for

"[a-zA-Z].py". It keeps the list in plugins_list variable and returns that. When I'm writing

scan_for_plugins() method I've read codes (f.e. [2]) about that and decided to name all

plugins' class name as *BuildSystem*. Now, detecting method works with that way.

(all plugins' class' are called BuildSystem) And I've changed base class name to

BuildSystemBase.

get_build_system() method is used by determine_build_system() method. If there is given

build system (by parameter) it tries to use that. If that build system is not usable then give

error and exits. If there is no given build system. It invokes get_build_system() and iterates

that methods return value which is a list. When iterating It create an instance of that build

system class (if can not create then it gives warning and skips that build system plugin)

and invokes detect method and append plugin name and detect method's return value to

the bs_list which contains plugin list which are available. And then sorts it. If there is error

while sorting list or there is no usable plugin then give error and exits. If there is multiple

highest detect values then lists that plugins and exists. (maintainer should select one of

these plugins to build extension.)

In main method, after getting arguments, it checks for verbose mode (or 'DH_VERBOSE'

environment variable) and sets log level (logging.DEBUG or logging.WARN)

If there is no given build_dir then it pretends it is "./".

After that It try to determine build system. If can not then prints error and exits.

And finally it starts to build extension. Firstly it checks if there is given version list. If not,

it uses default values and iterates build loop with that values.

First cleans, configures, builds (if there is no --test-only arg) and tests (if there is --test-only

arg or there is not --no-tests argument)

If there is error (raises plugin.Error) while invoking that method then prints error and exits.

In verbose mode tool prints every process. (like "configuring for PythonX.Y" etc.)

I've started writing a plugin for detecting *distutils*. For now, I only wrote its detect method. It

returns between 0-1.0 floating point value. If everything is OK it returns 1.0. Other methods does

nothing, only returns True but this week I'll finish this plugin and work for others.

For detecting distutils I've used the code at [3]. First I try to import that module if not, I've copied

required methods to plugins/_packaging.py file, I use them.

I don't use is_distutils() method directly. For give probability I invoke what it invokes and give them

a number. (f.e. _has_setup_py has 0.5, _has_pkg_info has 0.25 and _has_distutil_text has 0.25.

But if _has_setup_py returns False then detect method doesn't look other methods since they

require setup.py.)

For now that's all. See you at next time.

[1] git://git.debian.org/~mesutcan-guest/py3multibuild.git
[2] http://svn.berlios.de/viewvc/griffith/trunk/lib/plugins/extensions/

[3] http://hg.python.org/cpython/file/a7e00934baf1/Lib/packaging/util.py

--
Mesutcan Kurt

Asheesh Laroia: How I feel about Google+ (not thrilled)

Image credit: Lake Shimmer by (nc-sa) cobalt123. The beautiful photo above is what reflections should look like. If you don't like Google's shimmering +1 buttons, try the convenient AdBlock filter Tom Morris wrote. I also block this image.

I have been reading a lot about Google+. I'm sure a lot of people put a lot of hard work into it. Lots of people seem to enjoy using it and appreciate its features. For my part, right now, when I see Google+ mentioned, the smile vanishes from my face. I just have a few thoughts. It's anti-competitive. If you could ask Google, "How much for a shimmering +1 button next to every search result?" you would be laughed out of the room. But they managed to offer themselves that unique marketing opportunity. I know that (1) most regulators who see this anti-competitive behavior won't see it as an anti-trust violation, and (2) the damage is already done, and no retribution in the future can act fast enough to fix the unfair advantage Google gave itself in the Internet-mediated personal communication (and ad sales) markets. The shimmering is distracting, making my online search experience worse. I know that Google is holding nothing back; they are willing to distract me from my intended use of Google in order to branch out their business. (If you haven't seen the shimmering +1 button, it's next to every web search result, and if you mouse over any part of the result, it does a wavy dance.) The most-touted feature is a clone of Diaspora. This by itself is fine; software should help users by having the best features imaginable. But the enthusiasm I've seen within the free software community (like when I read about Google+ on Slashdot and Planet Debian) should at least recognize that Google "circles" are a clone of Diaspora's Aspects. Centralization on the web feels like a personal attack. Corporations on the web are like unstoppable machines rolling us toward a future of corporate eavesdropping, central points of failure, and end-users sold en masse to advertisers. It enables a future of individual "platform owners" who can change the lived experience of untold hundreds of millions with tweaks that benefit their actual customers (the advertisers). There's not much I can do about that. I happen to take this personally: I am part of a small culture of people who run their own mail and web servers and understand the importance of software freedom. This is a part of my identity. I watch as its traces online vanish, replaced by something more efficient and terrifying. It makes me sad. I'm probably going to end up using it despite all these pain points. That's how I know that I am being steamrolled. The reasons I have given above are not reasons to not use Google+. They're just reasons that I'm going to frown all the way to the address bar. In weeks to months to years, the service will grow an enthusiastic userbase. At that point, personally, I'll have to make a choice between connecting with those people on their terms, or not staying in touch with them. Already, professionally, the idea has already come up on #openhatch that Google+ could help us reach people willing to contribute to open source. Are my gripes worth holding OpenHatch back, or worth isolating myself over? Probably not.

Vincent Bernat: Dealing with missing locales on remote hosts

On my system, I happen to set LANG to fr_FR.utf8 and LC_MESSAGES to en_US.utf8. This means that applications should follow French cultural conventions for most things except for messages which should be displayed in US English. On my own system, /etc/locale.gen contains those two locales. However, when I connect to some random remote system, they may be unavailable. Most applications will fallback silently to C. However, Perl can be quite noisy:

$ perl -e 'print "Hello\n";'
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LC_MESSAGES = "en_US.utf8",
        LANG = "fr_FR.utf8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Hello

This is an incredibly annoying message. I don t understand why we still need to bear it. Perl documentation explains how to get rid of this message. The simplest way is to set PERL_BADLANG environment variable to 0. Problem solved.

$ PERL_BADLANG=0 perl -e 'print "Hello\n";'
Hello

Well, no. Keep reading. When you connect to a remote host with ssh, all your environment variables are thrown away, except those defined in AcceptEnv directive in /etc/ssh/sshd_config file on the remote host. On Debian, this directive defaults to LANG LC_*. Since I cannot install my favourite locales on all hosts or fiddle environment variables on them, I use something like this in my .zshrc:

ssh()  
  [ -t 1 ] && echo -ne "\033]0;$@\007"
  LANG=C LC_MESSAGES=C =ssh "$@"

Unfortunately, on systems where my locales are present, I still fallback to the crappy C locale. I could just unset LANG and LC_MESSAGES to allow fallback to the proper default locale but if the remote system starts to speak Spanish, I will not be pleased. Moreover, the behaviour of C locale (also known as POSIX locale) is undefined with characters not in the portable character set. From IEEE Std 1003.1:

Conforming systems shall provide a POSIX locale, also known as the C locale. The behavior of standard utilities and functions in the POSIX locale shall be as if the locale was defined via the localedef utility with input data from the POSIX locale tables in Locale Definition. The tables in Locale Definition describe the characteristics and behavior of the POSIX locale for data consisting entirely of characters from the portable character set and the control character set. For other characters, the behavior is unspecified. For C-language programs, the POSIX locale shall be the default locale when the setlocale() function is not called. The POSIX locale can be specified by assigning to the appropriate environment variables the values C or POSIX.

There is no way to detect the locales installed on the remote system unless you connect to it. Starting a connection just to check the locale is too expensive. On my own systems, I use another snippet in .zshrc to reconfigure the locales to my favourite ones if they are available:

export LANG=C
export LC_MESSAGES=C
(( $+commands[locale] )) && function  
    local locales
    locales=( "LANG fr_FR.utf8 en_US.utf8 C.UTF-8 C" \
              "LC_MESSAGES en_US.utf8 fr_FR.utf8 C.UTF-8 C" )
    for locale in $locales; do
        for l in $=locale[(w)2,-1]; do
            if locale -a   grep -qx $l; then
                export $locale[(w)1]=$l
                break
            fi
        done
    done
  2> /dev/null

This is an interesting snippet which uses some ZSH features.

The use of (( $+commands[locale] )) is a way to check if the locale command exists. Briefer than if $(which locale >& /dev/null)
The use of an anonymous function to avoid to clutter the global environment with local variables.
The use of word splitting. $locale[(w)1] converts $locale to an array and pops the first item. $=locale[(w)2,-1] converts $locale to an array, keeps the tail and apply word splitting to it (by default, zsh does not automatically apply word splitting to non quoted variables).

Feel free to tell me if you have found a better way to handle this problem!

Dominique Dumont: Generate a configuration editor from a config template file with Perl: lcdproc example

Hello Creating a configuration model for Config::Model can be a boring task: When a configuration system has a lot of parameters, a lot of details need to be specified to have a complete configuration validation tool. So, all tricks are permitted to make this task more bearable. Let s see how this can be applied to lcdproc. This project requires a configuration file that supports more than 40 drivers. Each driver has it own set of parameters for a total of about 250 parameters. That s a lot to specify.. Fortunately, LCDd.conf template file is written in a way which makes it relatively easy to parse and get all required information to build a model. All drivers are listed in this template file, most parameters have default values and legal values are written in comments in a uniform way. Hence this file (and comments) can be parsed to retrieve information required for the model. This blog will describe how this task can be implemented in Perl using Config::Model modules. The code will be slightly trimmed down to keep this blog (relatively) short. All initialisation code and error checks are removed. You can get the whole script on sourceforge. The main steps are:

prepare a parser for LCDd.conf template
parse LCDd.conf template
prepare a LCDd model skeleton
translate LCDd.conf template information in a format suitable to create a model.
Write the resulting LCDd model

First, here s the necessary incantation to start properly a Perl program:

use strict;
use warnings;

Then load all required libraries. Their use will be described later:

use Config::Model;
use Config::Model::Itself 1.225;
use Config::Model::Backend::IniFile;
use 5.10.0;
use IO::File;
use IO::String;

Perform some initialisation to avoid upsetting the logging system:

use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init($WARN);

Let s also initialize a model object that will be very useful later:

my $model = Config::Model->new();

Now we can really start the fun part. Step 1: Create a parser for LCDd.conf (INI file)
Problem: LCDd.conf comments must also be analysed to get more semantic information. The comments must be retrieved and associated with the INI classes and parameters found in this file. Fortunately, Config::Model::Backend::IniFile is able to associate comments with the relevant parameters. But Config::Model::Backend::IniFile must store its values in a configuration tree. So let s create a model suitable for LCDd.conf template that accepts any INI class and any INI parameter Dummy::Class is used to store any parameter found in an INI class. The accept keyword with .* regular expression means that any INI parameter will be accepted in each INI class.

$model->create_config_class(
  name   => 'Dummy::Class',
  accept => [
    '.*' =>  
       type => 'leaf',
       value_type => 'uniline'
        ,
    ],
);

Now the Dummy model must provide a configuration class to hold all INI classes:

$model->create_config_class(
    name   => 'Dummy',
    accept => [
       '.*' =>  
           type => 'node',
           config_class_name => 'Dummy::Class'
        ,
    ],
);

Note that an INI backend could be specified directly here. But, some useful parameters are commented out in LCD.conf. Some some processing is required to be able to create a model with these commented parameters. See below for this processing. Step 2: Parse LCDd.conf Now the dummy configuration class is created. Let s create a configuration tree from this dummy class to store the data from LCDd.conf

my $dummy = $model->instance(
    instance_name   => 'dummy',
    root_class_name => 'Dummy',
)-> config_root;

Now, read LCDd.conf:

my $lcd_file
  = IO::File->new('examples/lcdproc/LCDd.conf');
my @lines    = $lcd_file->getlines;

And perform the pre-processing on LCDd.conf mentioned above. Just un-commenting commented parameters is required:

foreach (@lines)   s/^#(\w+=)/$1/

Store the pre-processed LCDd.conf in a IO::Handle usable by INI backend

my $ioh = IO::String->new( join( '', @lines ) );

Create an INI backend to read the data from pre-processed LCDd.conf and store it in the configuration tree:

my $ini_backend = Config::Model::Backend::IniFile
  ->new( node => $dummy );
$ini_backend->read( io_handle => $ioh );

Step 3: Prepare LCDd configuration model Your attention please. You may remember that we invoked instance on object to create the dummy configuration tree to hold LCDd.conf data. The instance method used the Dummy configuration class. Now, we must invoke instance again on object. But this time, instance will use Config::Model::Itself. This model is Config::Model s way of eating its own dog food. The configuration tree created there will be able to hold LCDd model. Hence the root of this special configuration tree will be called meta_root to avoid confusion with the other configuration root defined above (for the Dummy model). Still with me? Good, let s create this meta tree that will contain LCDd model:

my $meta_root = $model->instance(
    root_class_name => 'Itself::Model',
    instance_name   => 'meta_model',
) -> config_root;

Create the main LCDd configuration class and store the first comment from LCDd.conf as class description:

my $first_comment = $dummy->annotation ;
$meta_root
  ->grab("class:LCDd class_description")
  ->store( $first_comment );

Append my own text in this description:

my $extra_description =
    "Model information extracted from template /etc/LCDd.conf"
  . "\n\n=head1 BUGS\n\nThis model does not support to load "
  . "several drivers. Loading several drivers is probably a "
  . "marginal case. Please complain to the author if this "
  . "assumption is false";
$meta_root->load(
     qq!class:LCDd
        class_description.="\n\n$extra_description"!
);

Add legal stuff in the model:

$meta_root->load( qq!
  class:LCDd
    copyright:0="2011, Dominique Dumont"
    copyright:1="1999-2011, William Ferrell and others"
    license="GPL-2"
!
);

Add INI backend so configuration object created from LCDd model will be able to read user s INI files:

$meta_root->load( qq!
  class:LCDd
    read_config:0
      backend=ini_file
      config_dir="/etc"
      file="LCDd.conf"
!
);

Note that all the load calls above could be done as one call. They were split for better clarity. Step 4:Extracting LCDd.conf information Now that all information from LCDd.conf template file is stored in $dummy configuration tree, it s time to use this information to enhance the skeleton of LCDd model created above. Let s first find all the INI classes defined in LCDd.conf:

my @ini_classes = $dummy->get_element_name;

Now before actually mining LCDd.conf information, we must prepare several subs to handle them. This is done using a dispatch table:

my %dispatch;

First create the default entry which will be used for most parameters. This subs is passed: the INI class name, the INI parameter name, the comment attached to the parameter, the INI value, and an optional value type. It will return a string describing a model for simple elements. For instance: class:"LCDd::MD8800" element:Device type=leaf value_type=uniline default="/dev/ttyS1" description="device to use [default: /dev/ttyS1]" Here s the subroutine invoked by default. Let s create a an anonymous subroutine and store it in the dispatch table:

$dispatch _default_  = sub  
    my ( $ini_class, $ini_param, $ini_note,
         $ini_v, $value_type ) = @_;
    $value_type  = 'uniline';

Now, prepare a string that will be used later to create the ini_class model. This string uses Config::Model s serialisation format described in Config::Model::Loader:

    my $load = qq!class:"$ini_class"
                  element:$ini_param type=leaf !;

Get semantic information from LCDd.conf template comments (written between square brackets):

    my $info = $ini_note =~ /\[(.*)\]/ ? $1 : '';
    $info =~ s/\s+//g;

Use this semantic information to better specify the parameter:

    $load .=
        $info =~ /legal:(\d+)-(\d+)/
           ? " value_type=integer min=$1 max=$2"
      : $info =~ /legal:([\w\,]+)/
           ? " value_type=enum choice=$1"
      :    " value_type=$value_type";
    if ( $info =~ /default:(\w+)/ )  
        # specify upstream default value
        # if it was found in the comment
        $load .= qq! upstream_default="$1"!
           if length($1);
     
    else  
        # or use the value found in INI
        # file as default
        $ini_v =~ s/^"//g;
        $ini_v =~ s/"$//g;
        $load .= qq! default="$ini_v"!
           if length($ini_v);

The model (in serialised format) is ready, let s return it and exit the subroutine:

    return $load;
 ;

Now let s take care of the special cases. This one deals with Driver parameter found in INI [server] class. This sub will create a string for an enum element. The possible values of the enum are extracted from the comment found in the LCDd.conf file:

$dispatch "LCDd::server" Driver  = sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    my $load = qq!class:"$class"
                    element:$elt
                      type=leaf
                      value_type=enum
    !;
    my @drivers = split /\W+/, $info;
    while ( @drivers
            and ( shift @drivers ) !~ /supported/ )  
      # nothing, the work is done by shift operator
      # in loop test to skip all words in $info until
      # 'The following drivers are supported:'.
      # ok, that's a hack
     
    $load .= 'choice='
          . join( ',', @drivers ) . ' ';
    return $load;
 ;

This sub invokes the default sub and add a small feature to the model to ensure that DriverPath ends with / . This is ensured by the match="/$" snippet:

$dispatch "LCDd::server" DriverPath  = sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    return   $dispatch _default_ ->(@_)
           . q! match="/$" !;
 ;

Likewise, this sub invokes the default sub with the optional 4th parameter that forces the value_type of the element. In this case, we enforce an integer element:

 $dispatch "LCDd::server" WaitTime 
 = $dispatch "LCDd::server" ReportLevel 
 = $dispatch "LCDd::server" Port 
 = sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    return $dispatch _default_ ->( @_, 'integer' );
   ;

This sub returns a string for a list element with default values:

  $dispatch "LCDd::server" GoodBye 
= $dispatch "LCDd::server" Hello 
= sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    my $ret = qq( class:"$class"
                    element:$elt
                      type=list
                ) ;
    $ret .= 'cargo type=leaf
             value_type=uniline - ' ;
    $ret .= 'default_with_init:0="\" '.$elt.'\"" ' ;
    $ret .= 'default_with_init:1="\" LCDproc!\""';
    return $ret ;
 ;

Now that the dispatch table is ready, let s really mine LCDd.conf information. First handle all INI classes in the foreach loop:

foreach my $ini_class (@ini_classes)

Let s retrieve parameters for this INI class from the LCDd.conf template:

  my $ini_obj = $dummy->grab($ini_class);

This will be the name of the lcdproc driver configuration class (e.g. LCDd::imonlcd):

  my $config_class   = "LCDd::$ini_class";

The configuration class will also be created in a serialised format. Let s create config class in case there s no parameter in INI file:

  $meta_root->load(qq!class:"LCDd::$ini_class"!);

Now, loop over all INI parameters and create LCDd::$ini_class elements:

  my @all_ini_params = $ini_obj->get_element_name ;
  foreach my $ini_param ( @all_ini_params )

Retrieve the value provided in template LCDd.conf:

    my $ini_v = $ini_obj->grab_value($ini_param);

Retrieve the comment attached to the value provided in template LCDd.conf:

    my $ini_comment
      = $ini_obj->grab($ini_param)->annotation;
    # escape embedded quotes
    $ini_comment =~ s/"/\\"/g;

Retrieve the correct sub from dispatch table:

    my $sub = $dispatch $config_class $ini_param 
              $dispatch _default_ ;

Run the sub to get the model string:

    my $model_spec
      = $sub->( $config_class, $ini_param,
                   $ini_comment, $ini_v );

Attach the comment from LCDd.conf as class description:

    $model_spec .= qq! description="$ini_comment"!
      if length($ini_comment);

Then, last but not least, load class specification in the model, and end the loop aroung INI parameters:

    $meta_root->load($model_spec);

Now create a an $ini_class element in LCDd class (to link LCDd class and LCDd::$ini_class):

  my $driver_class_spec = qq!
    class:LCDd
      element:$ini_class
  ! ;

Since server and menu are not drivers, they have a special treatment:

  if (   $ini_class eq 'server'
      or $ini_class eq 'menu' )  
    $driver_class_spec .= qq!
      type=node
      config_class_name="LCDd::$ini_class"
    ! ;

Here s the treatment for LCDd drivers. Setup the model with a warped node so that a driver configuration class is shown only if the driver was selected in the [server] class:

  else  
    $driver_class_spec .= qq!
      type=warped_node
      config_class_name="LCDd::$ini_class"
      level=hidden
      follow:selected="- server Driver"
      rules:"\$selected eq '$ini_class'"
        level=normal
    !;

Now we can load the LCDd driver model in meta root:

  $meta_root->load($driver_class_spec);

Step 5: save LCDd model Still with me ? Good. Here s the last part, now that the model was created in memory, let s save it in files. Config::Model::Itself constructor returns an object to read or write the data:

my $rw_obj = Config::Model::Itself
     ->new( model_object => $meta_root );
say "Writing all models in file (please wait)";
$rw_obj->write_all(
    model_dir => 'lib/Config/Model/models/'
);

That s it. Once this script is run, the directory lib/Config/Model/models/ contains a bunch of files:

The main LCDd model
All models for the drivers

You can see the resulting editor in my first blog on lcdproc. Conclusion and future steps (with you?) As mentioned at the end of this blog, the result of this model generation process is far from perfect, because the information retrieved from LCDd.conf template are not complete for this task (which is normal since these comments were targeted for users). Now, I d really like to discuss with project developers to improve configuration template to generate better models. For project owner, such an improvement would give them:

command lines to validate configuration
graphical configuration editor with integrated help
Configuration doc generated from the information stored in the template file

Please leave a comment on this blog (or send a message to config-model-users at lists.sourceforge.net) if you are interested in configuration tool generation for a project you contribute to. All the best

Raphaël Hertzog: My Debian activities in June 2011

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (195 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects.

Dropbox for Debian This is not free software but Dropbox is very popular and they did only provide an Ubuntu package that did not work on Debian. So I created an official package. I have been in touch with Dropbox developers and they have been very helpful. They ll shortly release a signature mechanism (with GPG) so that we can further improve the package by verifying the origin of the downloaded binaries. SAT Britney At the start of the month, I continued my work on the britney reimplementation (the software that creates testing out of unstable) but I quickly stalled it because the release managers asked the feedback of Stefano Zacchiroli and Ralf Treinen (who have extensive knowledge on the topic with their research work on Mancoosi) and I did not want to invest further work in case they would identify a major flow the feedback came only very late this month and while it was somewhat negative, I still think it s worth pursuing the effort for a bit longer. Converted ftplib to multiarch While dpkg still doesn t support multiarch (no news from Guillem and no visible sign of progress

), unstable got all the remaining bits allowing us to convert libraries to multiarch (see the announce). As soon as the required libc6 landed in unstable, I looked into converting the only library package that I maintain. I had no major problem but I still identified 2 issues in Lintian (filed as #630164 and quickly fixed by Niels Thykier). build-arch / build-indep support For the 42th time in the last 10 years, the idea of using build-arch/build-indep targets in the rules file has surfaced again. I had already decided some time ago that I would accept a patch implementing a new field Build-Features to enable dpkg-buildpackage to use those targets and this time Bill Allombert completed such a patch so I merged it. The technical committee also decided that it would take a final decision on this topic (see #629385). Roger Leigh provided useful input by doing an archive-wide rebuild with the various solutions suggested. Given that the majority would like to make the target mandatory at some point in the future, I provided the dpkg patch for my preferred solution. We would use auto-detection as a temporary measure until all packages have been converted to have the targets. The technical committee has not yet taken any decision even though the discussion stalled since the 12th of June. But that s usual with that body. I m sure it will be solved during Debconf.

Misc dpkg work

Modified dpkg-source --after-build to automatically unapply patches if they have been applied by dpkg-source --before-build.
Lots of small bug fixes (#628726, #629582, #630996, #631435, #631439, #631547, #632168) and that s just to keep with the flow of incoming bug reports!
Added 2 supplementary Perl modules to the supported API for the benefit of Lintian.
Spent an evening to track down the possible causes of an long-standing and annoying assertion failure related to triggers.
Updated my branch with improved triggers directives to take into account the feedback of Guillem, and merged it.
While doing this I discovered a design flaw with the usage of prerm failed-upgrade and merged a fix.
Discussed integration of dpkg-buildflags with debhelper in #544844 and decided of further improvements for dpkg-buildflags as a result.

Hamster applet update Hamster-applet is a GNOME application which did not have a 3.0 release, but it had a development release (2.91.x). I checked out whether it was possible to package this version for experimental and have the applet work with the GNOME fallback mode. Apparently not, the code was not yet updated to be compatible with the newer panel. Instead I uploaded the latest stable version (2.32.1) to unstable. It has some nice improvements in the standalone version (and the name of the executable changed). For usage with GNOME 3, I have created a custom shortcut to start it quickly (with gconf-editor set /apps/metacity/global_keybindings/run_command_1 to <Mod4>t and /apps/metacity/keybinding_commands/command_1 to hamster-time-tracker because the GNOME 3 control panel does not seem to work to set custom keybindings currently). Translated my professional website into English While I m grateful for all the people who are supporting my work, I m still far from my goal to have one third of my time funded through donations and sales of products on this blog. So I decided to also bring more visibility to my company and in particular to its Debian-related service offering. It was only available in French up to now so I translated it and expanded it a bit. My support page on this blog now also links to my company s website. If your company needs help to create Debian packages, or needs Debian technical support by email, you just found the right partner.

BTW, I have discounted prices for individuals and non-profits who would like to benefit from my help to create Debian packages. The Debian Administrator s Handbook This is the title of the upcoming translation of my book. The project now has a dedicated website: debian-handbook.info. You can subscribe to its RSS feed to keep up with the latest news. The full table of contents is online along with a FAQ. I m actively looking for partners to help me promote the fundraising once it goes live. If you can reach a large set of readers interested by a good Debian book, get in touch with me to join the affiliate program. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Russell Coker: Can Online Dating make You Depressed?

Anne Rettenberg wrote an article for Psychology Today that is critical of the idea of online dating [1], she cites one example of a man who visited a prostitute due to being depressed at his lack of success in online dating to support her claim. The first big problem with her article is that she doesn t mention the different experiences that male and female customers presumably have on online dating sites. I don t know what it s like for women on the dating sites so I can t comment on that. But I m sure that someone who works as a counselor could provide some useful insight into this matter. Also she didn t even give a mention to the issue of gay/lesbian dating sites. The next issue is that she didn t offer any good advice for who should use online dating sites and what their aims should be. Rejection In dating in real life (IRL) it s expected that the man make the first move, and therefore women end up rejecting lots of guys for various reasons. Anne seems to think that rejection online is somehow worse than rejection IRL, it probably is for some people but that certainly isn t the case for everyone. The way that lots of dating sites seem to work is that women place adverts, men respond to them, and then the women reply to a small subset of the email that they receive. Rejection in this case isn t a matter of telling someone that you aren t interested, but of merely not replying to their mail. From my discussions with a few men who ve used online dating sites the strategy seems to be to send out initial messages to a few dozen women every day and then maybe get a few responses a day. For the messages that get no response you will never know whether the other person found someone else first, wasn t interested in you for some reason, or just didn t bother checking their email. The only comparison to IRL rejection is that which happens after phone numbers have been exchanged, which isn t going to be that common (and has the same issues regardless of how the people met). Guys, relax about the women who don t respond to your first message. They probably get 100+ responses to their advert and don t have time to even read half of them. If you get rejected later in the process then you can look through your email archive at a later date to try and discover what went wrong. The Aim of Online Dating The fact that Anne s client visited a prostitute suggests that maybe he wasn t really after a relationship. In which case using one of the many online services for finding sex partners might have been a better option. Generally it seems that a good strategy is to try and have fun. I don t know any men who have married someone from a dating site, but the general opinion seems to be that they are still worth using. If you meet someone in a bar then you might end up having a drunken conversation that is drowned out by loud music. If you meet someone over the Internet then you can have a quiet conversation over the phone which seems to be a better way to get to know someone (and generally more pleasant for anyone who s not an alcoholic). I think that men who have no immediate aim other than finding someone nice to talk to will do better than those who aim to score quickly. Of the men I know who married women they met over the Internet (but not through online dating sites) I wonder how many of them would have ended up married if they hadn t used the online dating sites first. It seems that men who regularly communicate with women outside formal situations (work etc) will have a better chance of impressing someone that they like than those who lack such experience. Introverts There are a lot of people who really can t function in a bar. With the way our society works it seems that anyone who can t handle the bar scene really should consider online dating. How Counselors can Help It seems to me that someone who is seeing a counselor and who is considering a new way of finding a SO should ask their counselor for advice first. It also seems to me that a good option might be to ask their counselor for advice in online dating. Instead of being unsuccessful and depressed a man who was seeing a female counselor could do well to ask her advice for how to impress women on the Internet. This is probably a business opportunity for female counselors who can advise men on such things, among other things it seems that seeing an online relationship coach would be perceived in a more positive manner than seeing a counselor or psychologist for the more traditional reasons.

[1] http://www.psychologytoday.com/blog/beyond-don-juan/201107/can-online-dating-make-you-depressed

Erich Schubert: Google vs. Facebook

Let the games begin. It looks like Google and Facebook are going to fight it out.

Given the spectacular failures of Wave and Buzz, people are of course skeptic about the success of Google Plus. However, I'm rather confident it is going to stay. Here are some things I like about it:

Privacy: as far as I can tell, Plus design had started with privacy in mind, whereas for Facebook it is still an unloved child, a spare tire. Facebook keeps on getting bad reviews here; people don't get their UI and mis-share things. I read somewhere that FB is actually losing users in the US: kids who leave Facebook because their parents are getting on, and they don't want them to see what they shared with their friends.
UI: the Circles UI is very easy to use. The same functionality on Facebook is awful to manage. And the notifications UI in Plus also are a lot better than the tiny indicator in Facebook.
New stuff: Hangout and group chats are pretty interesting (well, actually I don't like Video at all ...). This puts pressure on Facebook to move, and it looks like they might present some Skype integration soon. But they need to make this really good to keep up with hangout. Just adding "Call me" buttons for all users that have specified a Skype account in their profile won't do the trick.
Future integration: Wave already had many of these things, but the wrong way. You would put a map into a chat; the Plus way will probably be to share a map session with a circle and add chat and collaborative editing there. Google "Office" is another place where it is trivial but very useful to add Plus.

Some people think that Google will not stand a chance against social giant Facebook. But after all, Google has more users - and they have tons of services people like to use. So when Google Maps has Plus integration, will the users use it to chat about where to meet, or will they go the long way and post the map URL on Facebook, without the option of updating it collaboratively?

Googles position is much stronger than many people believe, once you think about integration possibilities. The current Plus is just a fragment, the missing puzzle piece connecting the other apps. But imagine that Google now Plus-connects its various services: YouTube, Maps, Mail, Talk, ... - for them, this is just a matter of some engineering. I guess probably half of these are already in internal testing. And Facebook just can't keep up there. Sure, they do have Facebook Video. But actually, most people prefer YouTube. And while Google can integrate plus all the way there, Facebook cannot. And while Google is the master of search, Facebook is particularly weak there - they can't even properly search their own data. Google however will at some point offer an "find things that interest me" button; Sparks is just the beginning where you have to manually define your interests (which will probably remain an option due to privacy concerns!); it is way too static right now.

So essentially, Google doesn't need to copy Facebook. They just need to do what is obvious on their own products, and Facebook will have a hard time keeping up.

Plus, in my opinion, Google got the timing just right. Users aren't too happy with Facebook these days, there is just no big alternative around anymore; their friends are on Facebook and not on some other social network. Facebook doesn't seem to evolve much anymore. The mail functionality opened more security holes (apparently you can post to a group with a fake user name when you spoof the senders email address) than it contributed to functionality and usefulness. Privacy is still not in line with all countries such as Germany; but Facebook keeps telling those users essentially that they don't care. Spam and fraud still reappears every month following the same pattern again and again (Clickjacking). The search function of Facebook is still usually described as "useless" ... People waste time in games and annoy their friends with random game invitations and posts. Facebook should better make a major move now, too. More than a demo of Skype integration. But whenever Facebook changed, their users complained ...

Of course, Google+ still has a long way to go, too. There are still many things missing here, too. For example groups and events. I figure Google is already testing them in "dogfood", and they'll actually come out within the month. With groups I do not refer to the existing Google product, but to what would be "public circles" that you need to join and that are accessible to all the circle members instead of just the creator. And events are also a key function; probably one of the most used on Facebook. These may require much more careful design to integrate well with Calendar. But given the visual update of Calender these days, this may just be around the corner, too.

Rog rio Brito: Installing Meego on the N900

My last cellphone was a dumb phone. I seem to remember that it was called something like Samsung Voicer. It didn't have those hip things like SIM cards. OK. Fast forward to December of 2010, and after a long hiatus, I decided to get a new phone. It was this shiny Nokia N900. The biggest thing that made me want this one was that some people at DebConf 10 were praising it for having a Debian-like distribution in it, and that it worked very well. I saw that Phil Hands was enthusiastically talking about the bargain of getting one in New York and that it would have cost him many pounds in the UK and, so, he was happy with it. Ot vio Salvador also got one in NY and during one of the nights of that DebConf with him, Tiago Vaz, Daniel Baumann, and Chris Lamb, I asked him to call my parents here in Brazil and he let me use his N900. I had more time with it and played a little bit with the command line, opened the stock media player and played a beautiful trailer of 9 (2009 film), leaving me a very good impression. I bought one, and, indeed, it is a very good machine that can even make phone calls. The ability to run Linux on it (even if it contains some non-free pieces of software) was decisive in me getting it. It runs this distribution called Maemo 5 which is loosely based on Debian. But now, Maemo is not the "cool thing" to run, regarding Linux on these portable devices. Meego is. But with the advent of many Internet forums has brought many people writing some long, convoluted howto documents for things that would be better done as, say, providing a script or, better yet, preparing a package. In the case of getting Meego running on the N900, only three things are required:

Install uboot-pr13 while in Maemo.
Uncompress the MeeGo 1.2 Community Edition for Nokia N900 image to a SD card.
Turn the phone off, plug in the SD card, and PUT THE BACK COVER on the phone.

The last bit of the third step is crucial, as, otherwise, the SD card won't be detected and you will get kernel panics with the device trying to mount the root filesystem from a device that is not there. As an aside, the official documentation tells us one should uncompress the available pre-made images and write them with dd to the device. In my experience, it is completely unnecessary to use it and, in fact, it is so 6 times slower than using a simple shell redirection. That is, instead of:

bzip2 -d < mg-handset-armv7nhl-n900-whateverwhatever-mmcblk0p.raw.bz2   dd bs=4096 of=/dev/mmcblk0

you can get better results with the simpler:

bzip2 -d < mg-handset-armv7nhl-n900-whateverwhatever-mmcblk0p.raw.bz2 > /dev/mmcblk0

I actually used lbzip2 instead of bzip2, but that shouldn't matter. The use of the small block size for the dd command is probably the culprit, but I don't see the need to write the uncompressed data in chunks this small. If there is a problem, I would love to be informed of that. As trying it with uboot and a SD card doesn't mess with your "safe" Maemo installation, this is a good way to play with the successor of Maemo. Perhaps, if we give good feedback to the project, we can influence the direction that it is taking. It will surely be nice to learn about this new "consumer-oriented" distribution.

Gunnar Wolf: DebConf13 at home? Why not?

We are few weeks away from the start of DebConf11. Excitement runs high in Debian-land. The two most worthy weeks of the year, every year, loom close. Our Bosnian friends have done a great job of finding and defending an amazing proposal, and are now facing the hard work and permanent adrenaline levels of being in charge of the closest I have seen to a herd of (well-behaved but wild and untamable) cats. I have organized DebConf in my country. It was hellish, but at the same time, it's one of my most cherished experiences. And I'm sure the same will be said by the leaders of each successive bid It is one of the most rewarding experiences you can imagine. Next year, DebConf will be held in tropical Managua, Nicaragua. But, where will we meet in 2013? Well, that depends on you, my dear reader! Do you want to work your ass off for Debian and have utter fun? Do you want to show and share your country with this huge family of developers? Start thinking about pushing for a DebConf13 bid! Do you have to be at Banja Luka to propose your bid? No. You can proxy via somebody I'd suggest to do it via somebody who knows the location you are suggesting, but basically, choose a friend that you trust that trusts you. Of course, you can participate in the presentation session via IRC. Do you have to be a Debian Developer to propose a bid? No. For DebConf9, none of the C ceres guys was a DD; for DebConf10, some of the people most involved from the local New Yorkers were not DDs. For DC11, none of our dear and overworked hosts in Bosnia are DDs. And for DC12, the Nicaraguan crew is also made from people interested in getting closer to the Debian project, but not DDs. Do you have to decide now? No. This is just a call for a first presentation, but the decision regarding DC13 will be taken probably around March 2012. However, giving a nice presentation at DebConf helps a lot, gives you visibility, and will get the ball rolling. Is there a geographical bias? Slight. So far, and since the second DebConf, we have kept the tradition not to repeat continents on two successive DebConfs. This is not a hard condition, however! What do you need to start thinking about? Go visit our prospective location checklist at http://wiki.debconf.org/wiki/LocationCheckList. You can also look at what other teams have historically presented. Finally, I just learnt about the existence of http://wiki.debconf.org/wiki/DebConf13 Register there, even if you are just in the early phases of finding data. We will be holding a DebConf13 bids presentation session, most probably (the schedule is close to being presented officially!) on 30-07-2011, at 17:00.

2 July 2011

Martin Meredith: Google+: Part 1 Circles

So, I haven t read the planets in a couple of days but I m going to guess that, like twitter, a lot of people are going to be raving about Google+. For those of you who haven t heard about Google+, it s a new Social Networking site from Google and it brings a new take to the world of social networking. In real life, we communicate differently to the different people in our social contexts. For example, most people will converse differently to their work colleagues than they will to their family. They ll converse differently with their drinking buddies than they will with their Family. There are things that you want only certain people to know, and there are aspects of your personality that you only want to show to certain people. I think giving an example here might be better. This is an example of a friend of mine, who we ll call Fred. Fred is a teacher. Fred also likes to go out of a weekend and party till dawn, while consuming lots of tequila. I, as Fred s friend, am on a night out of drunken debauchery, and I ve my camera with me, and decide to post lots of pictures online of the night s revelry. The next day, one of Fred s colleagues logs into a Social Networking site, and sees said pictures in Fred s profile. Now things aren t looking so bright for Fred. He no longer gives across that professional image that he should do at work. Some may say that this is Fred s fault for going out and doing these things, but everyone s entitled to a personal life, right? At the moment, on Fred s social networking site, it s an all or nothing option. If he allows you access to his profile, he allows you access to see anything and everything that might show up on there. (It s a bit more complicated than this, but let s keep it simple for now). Every time Fred has someone add him as a contact he has to make that decision Do I want this person to be able to see everything I may or may not get up to . Put simply, how many of you have refused someone to add you on the Social Networking site du jour because what you show on their may be harmful to you, or give over an impression that isn t what you want to give across. Enter Circles. Circles brings to the social networking scene the way we interact with people in real life. We show a different aspect of our personality to different people we interact with. I m a bit of an organisational freak, and I know that there are different groups of people that either for the fact of portraying a persona, or not annoying everyone else, I might want to push in different ways. For example, I might want to send out a BBQ at my house, bring lots of alcohol message to all my friends, but don t really want my co-workers coming along and making it so I have to be on my best behaviour. Below is a screenshot of my (current) circles.

As you can see I have a fair few, and only a relative few people. Now, let s say that I wanted to send out that BBQ message

As you can see I ve the option to send this to the 5 people that are in my Friends group (it s currently mostly geeks on there there are only a few of my real life friends on there so far!) Now with only a few clicks, I ve managed to send out a message to just those few people that I want to

Over the next few days, I ll write a bit more about Google+, but I thought I d start with an introduction into the Circles feature. I ll be giving a bit of feedback about what I like, what I dislike, and also making a few suggestions for what I d like to see happen in the future (and probably use the word siloing a fair bit!) For now, toodles.

Joey Hess: databranches: using git as a database

I've just released git-annex version 3, which stops cluttering the filesystem with .git-annex directories. Instead it stores its data in a git-annex branch, which it manages entirely transparently to the user. It is essentially now using git as a distributed NOSQL database. Let's call it a databranch. This is not an unheard of thing to do with git. The git notes built into recent git does something similar, using a dynamically balanced tree in a hidden branch to store notes. My own pristine-tar injects data into a git branch. (Thanks to Alexander Wirt for showing me how to do that when I was a git newbie.) Some distributed bug trackers store their data in git in various ways. What I think takes git-annex beyond these is that it not only injects data into git, but it does it in a way that's efficient for large quantities of changing data, and it automates merging remote changes into its databranch. This is novel enough to write up how I did it, especially the latter which tends to be a weak spot in things that use git this way. Indeed, it's important to approach your design for using git as a database from the perspective of automated merging. Get the merging right and the rest will follow. I've chosen to use the simplest possible merge, the union merge: When merging parent trees A and B, the result will have all files that are in either A or B, and files present in both will have their lines merged (and possibly reordered or uniqed). The main thing git-annex stores in its databranch is a bunch of presence logs. Each log file corresponds to one item, and has lines with this form:

timestamp [0 1] id

This records whether the item was present at the specified id at a given time. It can be easily union merged, since only the newest timestamp for an id is relevant. Older lines can be compacted away whenever the log is updated. Generalizing this technique for other kinds of data is probably an interesting problem. :) While git can union merge changes into the currently checked out branch, when using git as a database, you want to merge into your internal-use databranch instead, and maintaining a checkout of that branch is inefficient. So git-annex includes a general purpose git-union-merge command that can union merge changes into a git branch, efficiently, without needing the branch to be checked out. Another problem is how to trigger the merge when git pulls changes from remotes. There is no suitible git hook (post-merge won't do because the checked out branch may not change at all). git-annex works around this problem by automatically merging */git-annex into git-annex each time it is run. I hope that git might eventually get such capabilities built into it to better support this type of thing. So that's the data. Now, how to efficiently inject it into your databranch? And how to efficiently retrieve it? The second question is easier to answer, although it took me a while to find the right way ... Which is two orders of magnitude faster than the wrong way, and fairly close in speed to reading data files directly from the filesystem. The right choice is to use git-cat-file --batch; starting it up the first time data is requested, and leaving it running for further queries. This would be straightforward, except git-cat-file --batch is a little difficult when a file is requested that does not exist. To detect that, you'll have to examine its stderr for error messages too. Perhaps git-cat-file --batch could be improved to print something machine parseable to stdout when it cannot find a file. Takes some careful parsing, but straightforward. Efficiently injecting changes into the databranch was another place where my first attempt was an order of magnitude slower than my final code. The key trick is to maintain a separate index file for the branch. (Set GIT_INDEX_FILE to make git use it.) Then changes can be fed into git by using git hash-object, and those hashes recorded into the branch's index file with git update-index --index-info. Finally, just commit the separate index file and update the branch's ref. That works ok, but the sad truth is that git's index files don't scale well as the number of files in the tree grows. Once you have a hundred thousand or so files, updating an index file becomes slow, since for every update, git has to rewrite the entire file. I hope that git will be improved to scale better, perhaps by some git wizard who understands index files (does anyone except Junio and Linus?) arranging for them to be modified in-place. In the meantime, I use a workaround: Each change that will be committed to the databranch is first recorded into a journal file, and when git-annex shuts down, it runs git hash-object just once, passing it all the journal files, and feeds the resulting hashes into a single call to

git
update-index

. Of course, my database code has to make sure to check the journal when retrieving data. And of course, it has to deal with possibly being interrupted in the middle of updating the journal, or before it can commit it, and so forth. If gory details interest you, the complete code for using a git branch as a database, with journaling, is here. After all that, git-annex turned out to be nearly as fast as before when it was simply reading files from the filesystem, and actually faster in some cases. And without the clutter of the .git-annex/ directory, git use is overall faster, commits are uncluttered, and there's no difficulty with branching. Using a git branch as a database is not always the right choice, and git's plumbing could be improved to better support it, but it is an interesting technique.

Bernd Zeimetz: plusone button plugin for ikiwiki

Just published an ikiwiki plugin to add google's +1 buttons. See ikiwiki.info/plugins/contrib/plusone/ for details. And if you enable html5 in your ikiwiki settings, it won't show up on planet debian as ugly g:plusone tag.

Guillaume Mazoyer: Status report Jigsaw num. 3 for GSoC 2011

At beginning of the last two weeks, I was working on the final package of JTReg. I was working on generating man pages and I finally found a solution. I decided to use help2man to generate the man pages but I had to patch the two scripts jtreg.sh and jtdiff.sh . Basically, these two scripts launch main classes of the jtreg.jar JAR file. The problem with them is that they don t know where they can find the JAR file on Debian. So I patched the two scripts shell so they can locate JAR files in /usr/share/java .
To be able to generate man pages with help2man, I needed to be able to use the scripts. But with jtdiff.sh and jtreg.sh using jtreg.jar which is not in /usr/share/java during the build, I had to find a way to make the scripts work. I decided to patch the scripts (again) and make them depend of some environment variables. So in each script, there are 4 used variables:

JAVA_HOME, which is the path where Java is,
JTREG_HOME, which is the path where jtreg.jar is,
JTHAR_HOME, which is the path where jtharness JAR are, and,
JAR_HOME, which is the path where other JAR are.

So during the packaging only JTREG_HOME needed to be changed to be able to use the script. Once the scripts patch done, I finally wrote the rules file of the package. Once again, I used javahelper so the rules file contains the following code:

#!/usr/bin/make -f
 JAVA_HOME = /usr/lib/jvm/default-java

 override_dh_auto_build:
  ant -f make/build.xml
  dh_auto_build
  JTREG_HOME=./dist/jtreg/lib/ help2man \
  --name="Regression Test Harness" \
  --help-option="-help" \
  ./dist/jtreg/linux/bin/jtdiff > jtdiff.1
  JTREG_HOME=./dist/jtreg/lib/ help2man \
  --name="Regression Test Harness" \
  --help-option="-help" \
  ./dist/jtreg/linux/bin/jtreg > jtreg.1

  override_dh_auto_clean:
  rm -r dist   :
  rm -r build  :
  rm jtdiff.1  :
  rm jtreg.1   :
  dh_auto_clean

  %:
  dh $@ --with javahelper

As you can see it is not really complicated. The most interesting part is the one when the man pages are generated. The JTREG_HOME variable is set to ./dist/jtreg/lib/ because it is the path where jtreg.jar is once it is built. I also used the help-option option of help2man because the help option of jtreg and jtdiff is -help and not help .
The jtreg package is now on the Debian Java team SVN so anyone can get it using:

svn checkout svn+ssh://$ SVN_USER @svn.debian.org/svn/pkg-java/trunk/jtreg

Thanks to Sylestre Ledru, jtreg is now available in Sid and the ITP is now closed.

The second part of my work consisted in testing Jigsaw and fixed tests if necessary. All the packaging work is now useful since the tests need JTReg to be run. The problem with the current build system of Jigsaw is that it is enough generic to be used on any system. So I had to patch several makefiles. To find them:

cd jigsaw-tests
find . -name 'Makefile' grep '/test/Makefile'

Then, the path to find jtreg has to be modified to match /usr/bin/jtreg . After that it is possible to run the tests with the following command:

make test

Passing all the tests is something which takes a lot of time. After that, I identified 42 (that s a cool number right?) failing tests. Some of them came from the lack of X11 server and other from unreachable network hosts. Some hackers of IcedTea gave me some ideas to fix several tests. They told me to look here and here because I just had the same problems that they had before.
With this, I was able to fix the tests related to unreachable hosts and to the lack of X server. I used Xvfb to fix the X server depending tests.
After running the tests again, only 13 tests were still failing. That is a good result. Tom Marble and me decided to apply the patch of Alan Bateman and re-run the tests to see if it will break something:

cd jigsaw-tests
 mkdir patches && cd patches
 wget http://cr.openjdk.java.net/~alanb/jigsaw-mp-prototype1/webrev/jdk.patch
 cd jdk
 patch -p1 < ../patches/jdk.patch
 make clean
 make sanity
 make all
 make modules
 xvfb-run -e xvfb-errors -a -s -ac make test -k

15 tests failed against 13 before. The 2 new failing tests are:

java/lang/management/MemoryMXBean/LowMemoryTestConcMarkSweepGC.sh
java/util/Locale/Bug6989440.java

I didn t go further but I ll try to understand why we have 15 failing tests in few days. All the modifications that I have made on the source code of Jigsaw can be found in a patch format here:

This 2 weeks were also dedicated to the understanding of Jigsaw and its modules. Reading the jigsaw-dev mailing list, I have found some interesting conversation about jigsaw and how to write modules. So I decided to write a module to see how it works. I followed the instruction of the quick start guide but when compiling the module didn t work. So I tried to understand why and then to fix it. I eventually compiled and installed my module and here is how I did it:

mkdir -p src/com.greetings/com/greetings/
 mkdir -p src/org.astro/org/astro/
 mkdir modules
 vim src/com.greetings/module-info.java
 vim src/com.greetings/com/greetings/Hello.java
 vim src/org.astro/module-info.java
 vim src/org.astro/org/astro/World.java
 ./jigsaw/build/linux-amd64/bin/javac -d modules \
  -modulepath ./jigsaw/build/linux-amd64/modules/modules \
  -sourcepath src  find src -name '*.java' 
 ./jigsaw/build/linux-amd64/bin/jmod create -L mlib
 ./jigsaw/build/linux-amd64/bin/jmod install \
  modules org.astro com.greetings -L mlib
 ./jigsaw/build/linux-amd64/bin/java -L mlib -m com.greetings

And here it is, we have a nice hello world module. The documentation says to put the module-info.java of org.astro in src/org.astro/org/astro/ but it didn t work for me so I tried to put it in src/org.astro/ and it worked. Also the quick start guide says that we have to use the -modulepath option to specify where the modules are. I first tried to use ./jigsaw/build/linux-amd64/modules/ but javac told me he could not find java.lang so I noticed that there is a subdirectory called modules in ./jigsaw/build/linux-amd64/modules/ so I used it as module path and it worked.

Writing a module and seeing how Jigsaw modules are made started to make me think about the packaging of Jigsaw. Dependencies being available, knowing how to build and test Jigsaw and seeing how modules are made, I think that I will soon try to start the packaging of Jigsaw and getting started with the packaging will be my goal for the next two weeks.

1 July 2011

Marco Silva: Some C++ stuff

I've been playing with C++ lately, and it's an interesting language. Some random thoughts go here. First, in which situation does "... int x; x = 0 ..." is different from "... int x = 0 ... "? The answer on the end of the post. Second, consider this code:

class A  ;
// void f(const A &a)  
// void f(A a)  
void f(A &a)  
int main()  
  f(A());
  // A a;
  // f(a);

It doesn't build under g++ or clang. Some say it builds under MSVC. If the definition of f is swapped by any of the commented ones, it works. If the declaration and call to f on function main is swapped by the commented lines, it also works. Third, dynamic dispatch only works with pointers. Consider this code:

class A  
 public:
  virtual void f()  
    printf("A");
   
 ;
class B : public A  
  void f()  
    printf("B");
   
 ;
int main()  
  A a = B();
  a.f();
  A *p = new B();
  p->f();
  delete p;
  B b;
  A *q = &b
  q->f();

It prints ABB. Some things to remember. The answer of the first point is: those codes are different if there is a static before them. This can only happen when they are in the body of a function. In this case, the first would assign x to 0 at each call to the function, and the other only at the first call.

David Moreno: DebConf 11

Last year, I had all the intention in the world to go to Bosnia and Herzegovina this summer to attend DebConf 11, but given my non-existent involvement on the project for the last couple of years, there's no reason for me to do it. Even last year when the event was held in town, I was not entirely thrilled about it. Life has changed quite a bit in the last few years, allowing me to just find and allocate my priorities on their right places. I hope everybody attending have a great time and snap lots of photos for me to see and regret not being there.

Gunnar Wolf: Yet another flurry of invites

Umh, so Google created yet-another-beta-service with yet-another-invite-only stage. Whee! And this time they choose a name yet more stupid to their previous attempts (Wave, Buzz now it's simply Google+). And yes, some of my friends have mailed me invites (or offered them to me by chat) so I can join the fun and love, and lose the few free time I still have. Guys, I care about you. I even love you all! But no, I don't care the least for so-called-social-network invites. I am quite a mess managing my time as it stands. And I cannot reply this to you in person, as Google+ sends mails with a very helpful sender (noreply-929ac25c@plus.google.com or variations of it), I have just added this to my .procmailrc:

:0
* ^From:.*noreply.*@plus.google.com
/dev/null

What does this mean in human? That instead of manually ignoring your invitations and feeling bad about not answering to you, dear friends, I'll leave that job to my computer. I will not receive any Google+ invites. Just as I don't receive Twitter and Facebook spam, FWIW. Thanks for thinking about me!

Michael Casadevall: Pandaboard Netboot Images Now Available

As I mentioned in my previous blog post, OMAP4 netboot images were available, but non-functional. I'm pleased to announce that these bugs have now been resolved and it is possible to have a functional install on OMAP4. This also has the added advantage of allowing one special partitioning layouts such as RAID, LVM, or simply having a non-SD based root device. The images are available here: http://ports.ubuntu.com/ubuntu-ports/dists/oneiric/main/installer-armel/current/images/omap4/netboot/

To use, simply dd boot.img-serial or boot.img-fb to an SD card, pop it in, and run, and the installer will pop up.

There is still a known bug that partman will not properly create the necessary boot partition. During the partitioning step, you must select manual partitioning, then create a 72 MiB FAT32 partition, with no mount point, and the Bootable flag must be set to 'on'. This partition must be the first partition on the device. flash-kernel-installer will be able to find the partition on its own.

Next.

Previous.