Search Results: "will"

4 July 2011

Lars Wirzenius: strncpy? just say no

This is a rant. I've made it in various forms in various places over the years, and I think it's time to put it somewhere so I can point people at it, and stop repeating myself. Here's the pebble in my shoe: people keep claiming that strncpy is a safer alternative to strcpy. It's not. It's a slower function that is not going to make you significantly safer. The problem with strcpy is that it's somewhat easy to overflow the target array: 
char src[] = "hello, world";
char dst[5];
strcpy(dst, src); // Oops.
The suggested alternative with strncpy would be this: 
strncpy(dst, src, sizeof(dst));
strncpy gets the size of the destination array as a parameter, and can therefore make sure the string will fit there. Problem solved? Not so much. Every time you do any string handling with C's native string types and functions, you need to make sure you have allocated enough memory for each string, you need to keep track of how much memory you've allocated, and you neet to make sure the zero byte ('NUL', '\0') is put there correctly. This is a lot of details to keep track of, and it's not surprising things sometimes go wrong. If your arrays are allocated statically, which is the simplest, but least useful case, you need to something like this with strcpy: 
if (strlen(src) + 1 > sizeof(dst))
    panic(); // or however you want to handle your errors
strcpy(dst, src);
The +1 after strlen is to make room for the zero byte (NUL, '\0') that is used in C to indicate the end of a string. With strncpy, your code is slightly simpler. This is its attraction. Problem is, this only works if dst is an array defined within the scope of the code. If, as is common, you have received dst as an argument to a function, sizeof(dst) is the size of a pointer to an element of the array, not the size of the array. So now you need to track the size explicitly in a second variable: 
strncpy(dst, src, dst_size);
Further, strncpy has an unfortunate error handling mode: it merely fills the destination array and stops. It does not indicate failure in any way. It does not terminate the target array with a zero byte. Since all C string processing assume the zero byte is there, as soon as you try to use dst as a string, you're in trouble. But luckily you can check for that before you call strncpy: 
if (strlen(src) + 1 > dst_size)
    panic(); // or whatever
strncpy(dst, src, dst_size);
By this time, the code is equally hard whether you use strcpy or strncpy. With strncpy you often waste some extra time, though, filling the unused parts of the target array with zero bytes: they're almost never necessary. Alternatively, you can force the zero byte there yourself, silently truncating a string. 
strncpy(dst, src, dst_size);
dst[dst_size - 1] = '\0'; // let's hope dst_size > 0
If your string handling is OK with strings being randomly shorter than they should be, possibly you could replace all of them with empty strings, and save yourself a world of pain. Instead of strncpy, use snprintf. It has sensible error behavior, and is much more versatile: 
n = snprintf(dst, dst_size, "%s", src);
if (n >= dst_size)
    panic();
snprintf still isn't very nice to use. It still requires you to allocate memory yourself, and keep track of the size of each array manually. It's also slower for small destination buffers (but faster for large ones). The real solution is to abandon C's antiquated string handling, and use a more sensible helper library instead. There's a bunch of them around, though none have gained much popularity. I'm not going to name any of them, but just imagine how much nicer it would be to write code like this instead: 
Dynstr *src = dynstr_new_from_cstring("hello, world\n");
Dynstr *cat = dynstr_cat_many(src, src, src, src, src, NULL);
Dynstr *dog = dynstr_substr(cat, 15, 5);
Let the library take care of memory allocations and related tracking, and concentrate on real problem solving instead. We did this in Kannel, back in 1999, and it wiped out all the buffer overflow problems (until a co-worker decided it was "more efficient" to use native C strings instead, promptly introducing a buffer overflow). All of this string handling is, of course, way nicer in higher level languages. In fact, whenever you need to deal with string handling in C, it is probably a sign that you should consider switching to a higher level language instead. But whatever you do, stop claiming that strncpy is safe or solves any real problems strcpy has. Update: Fixed "It does terminate" to say "It does not terminate". Thanks to Matthew Garrett for pointing that out. Update2: Fixed example call to snprintf. Update3: Fixed the test for snprintf example. Thanks, Redditor axylone. Also pointed out that snprintf is slower than the rest.

Cosimo Alfarano: Facebook, Google+ and Diaspora

Google+ opened to the small public some days ago, I'm not really going to comment anything about it for now.

The purpose of my post is actually reminding people of another interesting project, Diaspora, which actually fits several of my views over social networking.

I really hope that when Google will release API for its G+, there will be some chance for Diaspora to integrate with it and, most important, an opening for G+ to operate with Diaspora.

Most of us use(d) Facebook, now almost any geek I know is joining G+, some of them rejoicing its advent as an opportunity to leave Facebook for good.
That's not really my intention and I don't think it's the point of the whole social thing.

What about, instead, allowing the whole social sites to inter-communicate, exchanging information and letting users being able to see each other independently of the platform used?

That's clearly not possible so far :-), but it is the reason for which years and years ago (Internet-time rules apply) I got interested in FOAF, last year I crossed my fingers with Diaspora and now I'm doing the same about Google receiving some enlightenment bit and getting the right path about profile sharing.

I'm aware that Google point is to re-factor the whole Google experience around Google Accounts and the profiles associated with them, and the major point of it is making G+ a bigger part of G profiles (if you are a Google Profile user, you'll probably noticed that your profile is now your public view of G+, with your public posts for instance).

My point is: Google is aware that not everyone agrees with Google wanting to extract information from your profile, your habits, etc and some of us is not really using Google 100% for this reason.
Also, Google started the Data Liberation front, so why not taking the next step and allowing people to inter-operate with other open [1] social sites?

The big part of the work is not helping Diaspora evolving and gaining power, IMHO, but pretty much the other way round: helping the idea of inter-communication to be a central point of the social experience: this idea getting stuck in some important and enlightened mind so that a bridge will be offered.

But I'm just a noob in the subject: it's just a foolish idea that has been in my mind for long enough and I needed to express :-D

Raphaël Hertzog: How to prepare patches for Debian packages

You want to start contributing to Debian and/or Ubuntu, you decided to help a package maintainer and you re now looking for how to change a source package and how to submit your changes. 1. Retrieve the source package and install build-dependencies The first step is to retrieve the latest version of the source package and to install the required build-dependencies. I already covered how to do this with apt-get in the article explaining how to rebuild a source package. If you prefer, you can use dget (from the devscripts package) to directly grab the source package. You can find the URL of the .dsc files in the Package Tracking System for example. Sometimes apt-get will warn you that the source package is maintained in a VCS repository, like this: 
$ apt-get source wordpress
[...]
NOTICE: 'wordpress' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/git/collab-maint/wordpress.git
[...]
In that case, you can use debcheckout to retrieve the VCS repository instead (provided that you have the corresponding VCS installed): 
$ debcheckout wordpress
declared git repository at git://git.debian.org/git/collab-maint/wordpress.git
git clone git://git.debian.org/git/collab-maint/wordpress.git wordpress ...
Cloning into wordpress...
Note however that some maintainers use their VCS in a way that s not really compatible with the explanations that I will give below. It s also a good idea to install the package packaging-dev . It s a meta-package depending on the most common tools that are used for Debian packaging work. 2. Do the changes Execute dch --nmu to record the fact that you re working on an update prepared by someone who is not the maintainer (NMU means Non Maintainer Upload). This also ensures that if we build the package, we won t overwrite the original source package that we downloaded, thus making it possible to generate a diff between both versions. 2.1. Modify Debian packaging files Now fire your text editor and do the required changes in the debian sub-directory. You will probably run dch -a multiple times to document each subsequent change. 2.2. Modify upstream files If you have to modify upstream files, the proper way to do it depends on the source package format ( 1.0 vs 3.0 (quilt) vs 3.0 (native) , see the debian/source/format file) and on the presence or not of a patch system (the what-patch can help you identify it). In this explanation, I ll assume that the package is using the recommended format: 3.0 (quilt) . (It also works for 1.0 if quilt is used and if you configured ~/.quiltrc as recommended by /usr/share/doc/quilt/README.source). First you should ensure that all patches have been applied with quilt push -a. If there s no patch yet, you want to create the debian/patches directory (with mkdir debian/patches). 2.2.1. Import a patch If the upstream changes are already in a patch file (say /tmp/patch that you downloaded from the upstream VCS repository) you can import that patch like this: 
$ quilt import -P fix-foobar.patch /tmp/patch
Importing patch /tmp/patch (stored as fix-foobar.patch)
$ quilt push
Applying patch fix-foobar.patch
[...]
Now at patch fix-foobar.patch
The -P option allows to select the name of the patch file created in debian/patches/. As you see, the new patch file is recorded in debian/patches/series but not applied by default, we re thus doing it with quilt push. 2.2.1. Create a new patch If the upstream changes that you want to make are not in a patch yet, you should tell quilt that you re going to create one: 
$ quilt new fix-foobar.patch
Patch fix-foobar.patch is now on top
Then you must record every file that you re going to modify with a quilt add invocation. quilt then makes a backup of those files so that it can generate the patch later on. If you re going to modify the files with your text editor you can just do quilt edit file-to-modify, it s the same than quilt add file-to-modify followed by sensible-editor file-to-modify. 
$ quilt edit foobar.c
File foobar.c added to patch fix-foobar.patch
The last step is tell quilt to generate the patch: 
$ quilt refresh
Refreshed patch fix-foobar.patch
3. Test your changes You should build your modified package with debuild -us -uc . You can easily install the resulting package with debi . Verify that everything works as expected. If not, continue your modifications until you re satisfied with the result. 4. Generate a patch and mail it If you followed the instructions, you should have two .dsc files in the parent directory, like this: 
$ cd ..
$ ls wordpress_*.dsc
../wordpress_3.0.5+dfsg-1.1.dsc
../wordpress_3.0.5+dfsg-1.dsc
Generating the patch to send to the maintainer is then just a matter of running debdiff: 
$ debdiff wordpress_3.0.5+dfsg-1.dsc wordpress_3.0.5+dfsg-1.1.dsc >/tmp/wp-debdiff
You can send the /tmp/wp-debdiff file to the wordpress maintainer. Usually you send it via the bugreport that your update is fixing and you add the patch tag to the report. This can be automated with the nmudiff utility. By default it assumes that you re using mutt but it can also directly feed the resulting mail to sendmail. The default text that nmudiff proposes assumes that you re actually performing an NMU and that the result has been uploaded. If that s not the case, you should edit the text and make it clear that you re just sending a patch. If you have been working in a VCS repository, instead of using debdiff you can simply use the diff feature integrated in your VCS (git diff, svn diff, etc.). But note that with a distributed VCS (like git/bzr/mercurial, unlike svn) you should probably have committed all individual changes in separate changesets. And instead of sending a single patch, you re probably going to send a series of patches (though it might be easier to just upload your branch in a public repository and give the corresponding URL to the maintainer).
Found it useful? Be sure to not miss other packaging tips (or lessons), click here to subscribe to my free newsletter and get new articles by email.

No comment Liked this article? Click here. My blog is Flattr-enabled.

3 July 2011

Andrea Veri: Automatic Gmail s Trash & Spam folders cleanup

Since some time I ve been thinking about a possible way to delete my Gmail s Trash & Spam folders content automatically without having to bother doing it manually every single time I wanted to check my mail and clean it up. (I love keeping everything in place and having my Trash&Spam folders empty as they should be makes me pretty happy) A few years ago when Mutt was my main mail client I had the need to filter my mail through IMAP and while googling around for that I found out a great piece of software: imapfilter. Today while analyzing the above quoted issue I suddenly told myself: Hey, but why don t you use your dear and old friend imapfilter to fulfil your needs? After a few minutes I came up with a small lua script that was doing exactly what I wanted: my Trash&Spam folders are no longer crowded and I finally don t have to delete mails twice! But here they come a few details about my script: 
options.timeout = 120
options.subscribe = true
account = IMAP  
 server = 'imap.gmail.com',
 username = 'example@gmail.com',
 password = 'password',
 ssl = 'ssl3'
  
trash = account['[Gmail]/Trash']:is_undeleted()
account['[Gmail]/Trash']:delete_messages(trash)
spam = account['[Gmail]/Spam']:is_unanswered()
account['[Gmail]/Spam']:delete_messages(spam)
The script does two things:
  1. It checks whether a mail is not marked as deleted (moving an e-mail into the Trash does not mark it as to be deleted automatically) already and removes it.
  2. It checks whether a mail on the Spam folder has been answered (I never had to answer a single e-mail contained into my Spam folder) and if not removes it.
Using the above script is really easy (you should run imapfilter on interactive mode first to generate Gmail s certificates, do that before having cron to run the script for you or otherwise it ll just hang), just make sure to have imapfilter installed on your system and then run it through cron every half an hour or less depending on your needs: 
crontab -e

*/30 * * * * imapfilter -c /home/user/imapfilter.lua >> /home/user/imapfilter.log
Please also remember to setup appropriate permissions on the config file since it contains your Gmail s password and most of all make sure that your Spam folder is visible through IMAP (this option can be found on the label menu available under your Gmail s settings) otherwise imapfilter will just report an error. Enjoy!

Clint Adams: A very little bit about how herb works

This information is pieced together from no fewer than five informants. As such it is likely to contain inaccuracies. Every year, through the conference registration system, a fork of an old version of Pentabarf, attendees declare that they need sponsorship. They state their cost of travel as well as how much of this amount they can afford to pay themselves. In addition, there is a text field in which one is to answer what you do for Debian and why you request sponsorship. Next a committee is assembled. The head of the committee hand-picks its members. In 2011, twenty-four people were asked if they would like to participate, thirteen people were named to the committee, and twelve of those ended up voting. This committee is named herb , after Herbert Powell. The members of herb each individually rate the applicants on two factors: contribution to Debian and amount requested. Each factor is rated on a five-point scale (-2 to +2). Raters may leave, at their option, a comment regarding the applicant. The raters are able to see previous ratings and comments. In the interest of avoiding the appearance of conflict of interest, each rater will refrain voluntarily from self-rating. Individual raters use exercise judgment during this process.
Contribution to Debian is very subjective, and some raters will vote either on total travel cost, requested amount, the ratio of the two, or something else. At a certain point in time, a weighted average of all the ratings is used to order all the applicants into a list. At the top will be the people ranked most highly. The herb team will then conduct an IRC meeting in which ranking may be arbitrarily changed and two thresholds will be established. The result of this process is three lists of people: queue A, for people who should definitely get sponsorship; queue B, for people that might get sponsorship if people from queue A cancel or other circumstances arise; and queue C, which is not really a queue, but a rejection pile. In some cases, the structure of these queues are decided before definite budget numbers are known. Applicants are emailed to let them know which queues they are in, and their rankings within those queues.

Benjamin Drung: Nautilus with libkibi

In this Ubuntu cycle, I work on getting the units policy implemented. For this I wrote a library called libkibi. Here are some screen shots how nautilus looks like with libkibi. Some changes are highlighted in red. The file properties will show the file size in base10 and base2: PS: I failed to launch nautilus in English. Therefore the screen shots are in German. PS : You can grab the modified nautilus package for Ubuntu 10.10 (maverick) from my experimental PPA (at your own risk!).

Asheesh Laroia: How I feel about Google+ (not thrilled)

Image credit: Lake Shimmer by (nc-sa) cobalt123. The beautiful photo above is what reflections should look like. If you don't like Google's shimmering +1 buttons, try the convenient AdBlock filter Tom Morris wrote. I also block this image.

I have been reading a lot about Google+. I'm sure a lot of people put a lot of hard work into it. Lots of people seem to enjoy using it and appreciate its features. For my part, right now, when I see Google+ mentioned, the smile vanishes from my face. I just have a few thoughts. It's anti-competitive. If you could ask Google, "How much for a shimmering +1 button next to every search result?" you would be laughed out of the room. But they managed to offer themselves that unique marketing opportunity. I know that (1) most regulators who see this anti-competitive behavior won't see it as an anti-trust violation, and (2) the damage is already done, and no retribution in the future can act fast enough to fix the unfair advantage Google gave itself in the Internet-mediated personal communication (and ad sales) markets. The shimmering is distracting, making my online search experience worse. I know that Google is holding nothing back; they are willing to distract me from my intended use of Google in order to branch out their business. (If you haven't seen the shimmering +1 button, it's next to every web search result, and if you mouse over any part of the result, it does a wavy dance.) The most-touted feature is a clone of Diaspora. This by itself is fine; software should help users by having the best features imaginable. But the enthusiasm I've seen within the free software community (like when I read about Google+ on Slashdot and Planet Debian) should at least recognize that Google "circles" are a clone of Diaspora's Aspects. Centralization on the web feels like a personal attack. Corporations on the web are like unstoppable machines rolling us toward a future of corporate eavesdropping, central points of failure, and end-users sold en masse to advertisers. It enables a future of individual "platform owners" who can change the lived experience of untold hundreds of millions with tweaks that benefit their actual customers (the advertisers). There's not much I can do about that. I happen to take this personally: I am part of a small culture of people who run their own mail and web servers and understand the importance of software freedom. This is a part of my identity. I watch as its traces online vanish, replaced by something more efficient and terrifying. It makes me sad. I'm probably going to end up using it despite all these pain points. That's how I know that I am being steamrolled. The reasons I have given above are not reasons to not use Google+. They're just reasons that I'm going to frown all the way to the address bar. In weeks to months to years, the service will grow an enthusiastic userbase. At that point, personally, I'll have to make a choice between connecting with those people on their terms, or not staying in touch with them. Already, professionally, the idea has already come up on #openhatch that Google+ could help us reach people willing to contribute to open source. Are my gripes worth holding OpenHatch back, or worth isolating myself over? Probably not.


Vincent Bernat: Dealing with missing locales on remote hosts

On my system, I happen to set LANG to fr_FR.utf8 and LC_MESSAGES to en_US.utf8. This means that applications should follow French cultural conventions for most things except for messages which should be displayed in US English. On my own system, /etc/locale.gen contains those two locales. However, when I connect to some random remote system, they may be unavailable. Most applications will fallback silently to C. However, Perl can be quite noisy: 
$ perl -e 'print "Hello\n";'
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LC_MESSAGES = "en_US.utf8",
        LANG = "fr_FR.utf8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Hello
This is an incredibly annoying message. I don t understand why we still need to bear it. Perl documentation explains how to get rid of this message. The simplest way is to set PERL_BADLANG environment variable to 0. Problem solved. 
$ PERL_BADLANG=0 perl -e 'print "Hello\n";'
Hello
Well, no. Keep reading. When you connect to a remote host with ssh, all your environment variables are thrown away, except those defined in AcceptEnv directive in /etc/ssh/sshd_config file on the remote host. On Debian, this directive defaults to LANG LC_*. Since I cannot install my favourite locales on all hosts or fiddle environment variables on them, I use something like this in my .zshrc: 
ssh()  
  [ -t 1 ] && echo -ne "\033]0;$@\007"
  LANG=C LC_MESSAGES=C =ssh "$@"
Unfortunately, on systems where my locales are present, I still fallback to the crappy C locale. I could just unset LANG and LC_MESSAGES to allow fallback to the proper default locale but if the remote system starts to speak Spanish, I will not be pleased. Moreover, the behaviour of C locale (also known as POSIX locale) is undefined with characters not in the portable character set. From IEEE Std 1003.1:
Conforming systems shall provide a POSIX locale, also known as the C locale. The behavior of standard utilities and functions in the POSIX locale shall be as if the locale was defined via the localedef utility with input data from the POSIX locale tables in Locale Definition. The tables in Locale Definition describe the characteristics and behavior of the POSIX locale for data consisting entirely of characters from the portable character set and the control character set. For other characters, the behavior is unspecified. For C-language programs, the POSIX locale shall be the default locale when the setlocale() function is not called. The POSIX locale can be specified by assigning to the appropriate environment variables the values C or POSIX.
There is no way to detect the locales installed on the remote system unless you connect to it. Starting a connection just to check the locale is too expensive. On my own systems, I use another snippet in .zshrc to reconfigure the locales to my favourite ones if they are available: 
export LANG=C
export LC_MESSAGES=C
(( $+commands[locale] )) && function  
    local locales
    locales=( "LANG fr_FR.utf8 en_US.utf8 C.UTF-8 C" \
              "LC_MESSAGES en_US.utf8 fr_FR.utf8 C.UTF-8 C" )
    for locale in $locales; do
        for l in $=locale[(w)2,-1]; do
            if locale -a   grep -qx $l; then
                export $locale[(w)1]=$l
                break
            fi
        done
    done
  2> /dev/null
This is an interesting snippet which uses some ZSH features. Feel free to tell me if you have found a better way to handle this problem!

Dominique Dumont: Generate a configuration editor from a config template file with Perl: lcdproc example

Hello Creating a configuration model for Config::Model can be a boring task: When a configuration system has a lot of parameters, a lot of details need to be specified to have a complete configuration validation tool. So, all tricks are permitted to make this task more bearable. Let s see how this can be applied to lcdproc. This project requires a configuration file that supports more than 40 drivers. Each driver has it own set of parameters for a total of about 250 parameters. That s a lot to specify.. Fortunately, LCDd.conf template file is written in a way which makes it relatively easy to parse and get all required information to build a model. All drivers are listed in this template file, most parameters have default values and legal values are written in comments in a uniform way. Hence this file (and comments) can be parsed to retrieve information required for the model. This blog will describe how this task can be implemented in Perl using Config::Model modules. The code will be slightly trimmed down to keep this blog (relatively) short. All initialisation code and error checks are removed. You can get the whole script on sourceforge. The main steps are:
  1. prepare a parser for LCDd.conf template
  2. parse LCDd.conf template
  3. prepare a LCDd model skeleton
  4. translate LCDd.conf template information in a format suitable to create a model.
  5. Write the resulting LCDd model
First, here s the necessary incantation to start properly a Perl program: 
use strict;
use warnings;
Then load all required libraries. Their use will be described later: 
use Config::Model;
use Config::Model::Itself 1.225;
use Config::Model::Backend::IniFile;
use 5.10.0;
use IO::File;
use IO::String;
Perform some initialisation to avoid upsetting the logging system: 
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init($WARN);
Let s also initialize a model object that will be very useful later: 
my $model = Config::Model->new();
Now we can really start the fun part. Step 1: Create a parser for LCDd.conf (INI file)
Problem: LCDd.conf comments must also be analysed to get more semantic information. The comments must be retrieved and associated with the INI classes and parameters found in this file. Fortunately, Config::Model::Backend::IniFile is able to associate comments with the relevant parameters. But Config::Model::Backend::IniFile must store its values in a configuration tree. So let s create a model suitable for LCDd.conf template that accepts any INI class and any INI parameter Dummy::Class is used to store any parameter found in an INI class. The accept keyword with .* regular expression means that any INI parameter will be accepted in each INI class. 
$model->create_config_class(
  name   => 'Dummy::Class',
  accept => [
    '.*' =>  
       type => 'leaf',
       value_type => 'uniline'
        ,
    ],
);
Now the Dummy model must provide a configuration class to hold all INI classes: 
$model->create_config_class(
    name   => 'Dummy',
    accept => [
       '.*' =>  
           type => 'node',
           config_class_name => 'Dummy::Class'
        ,
    ],
);
Note that an INI backend could be specified directly here. But, some useful parameters are commented out in LCD.conf. Some some processing is required to be able to create a model with these commented parameters. See below for this processing. Step 2: Parse LCDd.conf Now the dummy configuration class is created. Let s create a configuration tree from this dummy class to store the data from LCDd.conf 
my $dummy = $model->instance(
    instance_name   => 'dummy',
    root_class_name => 'Dummy',
)-> config_root;
Now, read LCDd.conf: 
my $lcd_file
  = IO::File->new('examples/lcdproc/LCDd.conf');
my @lines    = $lcd_file->getlines;
And perform the pre-processing on LCDd.conf mentioned above. Just un-commenting commented parameters is required: 
foreach (@lines)   s/^#(\w+=)/$1/
Store the pre-processed LCDd.conf in a IO::Handle usable by INI backend 
my $ioh = IO::String->new( join( '', @lines ) );
Create an INI backend to read the data from pre-processed LCDd.conf and store it in the configuration tree: 
my $ini_backend = Config::Model::Backend::IniFile
  ->new( node => $dummy );
$ini_backend->read( io_handle => $ioh );
Step 3: Prepare LCDd configuration model Your attention please. You may remember that we invoked instance on object to create the dummy configuration tree to hold LCDd.conf data. The instance method used the Dummy configuration class. Now, we must invoke instance again on object. But this time, instance will use Config::Model::Itself. This model is Config::Model s way of eating its own dog food. The configuration tree created there will be able to hold LCDd model. Hence the root of this special configuration tree will be called meta_root to avoid confusion with the other configuration root defined above (for the Dummy model). Still with me? Good, let s create this meta tree that will contain LCDd model: 
my $meta_root = $model->instance(
    root_class_name => 'Itself::Model',
    instance_name   => 'meta_model',
) -> config_root;
Create the main LCDd configuration class and store the first comment from LCDd.conf as class description: 
my $first_comment = $dummy->annotation ;
$meta_root
  ->grab("class:LCDd class_description")
  ->store( $first_comment );
Append my own text in this description: 
my $extra_description =
    "Model information extracted from template /etc/LCDd.conf"
  . "\n\n=head1 BUGS\n\nThis model does not support to load "
  . "several drivers. Loading several drivers is probably a "
  . "marginal case. Please complain to the author if this "
  . "assumption is false";
$meta_root->load(
     qq!class:LCDd
        class_description.="\n\n$extra_description"!
);
Add legal stuff in the model: 
$meta_root->load( qq!
  class:LCDd
    copyright:0="2011, Dominique Dumont"
    copyright:1="1999-2011, William Ferrell and others"
    license="GPL-2"
!
);
Add INI backend so configuration object created from LCDd model will be able to read user s INI files: 
$meta_root->load( qq!
  class:LCDd
    read_config:0
      backend=ini_file
      config_dir="/etc"
      file="LCDd.conf"
!
);
Note that all the load calls above could be done as one call. They were split for better clarity. Step 4:Extracting LCDd.conf information Now that all information from LCDd.conf template file is stored in $dummy configuration tree, it s time to use this information to enhance the skeleton of LCDd model created above. Let s first find all the INI classes defined in LCDd.conf: 
my @ini_classes = $dummy->get_element_name;
Now before actually mining LCDd.conf information, we must prepare several subs to handle them. This is done using a dispatch table: 
my %dispatch;
First create the default entry which will be used for most parameters. This subs is passed: the INI class name, the INI parameter name, the comment attached to the parameter, the INI value, and an optional value type. It will return a string describing a model for simple elements. For instance: class:"LCDd::MD8800" element:Device type=leaf value_type=uniline default="/dev/ttyS1" description="device to use [default: /dev/ttyS1]" Here s the subroutine invoked by default. Let s create a an anonymous subroutine and store it in the dispatch table: 
$dispatch _default_  = sub  
    my ( $ini_class, $ini_param, $ini_note,
         $ini_v, $value_type ) = @_;
    $value_type  = 'uniline';
Now, prepare a string that will be used later to create the ini_class model. This string uses Config::Model s serialisation format described in Config::Model::Loader: 
    my $load = qq!class:"$ini_class"
                  element:$ini_param type=leaf !;
Get semantic information from LCDd.conf template comments (written between square brackets): 
    my $info = $ini_note =~ /\[(.*)\]/ ? $1 : '';
    $info =~ s/\s+//g;
Use this semantic information to better specify the parameter: 
    $load .=
        $info =~ /legal:(\d+)-(\d+)/
           ? " value_type=integer min=$1 max=$2"
      : $info =~ /legal:([\w\,]+)/
           ? " value_type=enum choice=$1"
      :    " value_type=$value_type";
    if ( $info =~ /default:(\w+)/ )  
        # specify upstream default value
        # if it was found in the comment
        $load .= qq! upstream_default="$1"!
           if length($1);
     
    else  
        # or use the value found in INI
        # file as default
        $ini_v =~ s/^"//g;
        $ini_v =~ s/"$//g;
        $load .= qq! default="$ini_v"!
           if length($ini_v);
The model (in serialised format) is ready, let s return it and exit the subroutine: 
    return $load;
 ;
Now let s take care of the special cases. This one deals with Driver parameter found in INI [server] class. This sub will create a string for an enum element. The possible values of the enum are extracted from the comment found in the LCDd.conf file: 
$dispatch "LCDd::server" Driver  = sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    my $load = qq!class:"$class"
                    element:$elt
                      type=leaf
                      value_type=enum
    !;
    my @drivers = split /\W+/, $info;
    while ( @drivers
            and ( shift @drivers ) !~ /supported/ )  
      # nothing, the work is done by shift operator
      # in loop test to skip all words in $info until
      # 'The following drivers are supported:'.
      # ok, that's a hack
     
    $load .= 'choice='
          . join( ',', @drivers ) . ' ';
    return $load;
 ;
This sub invokes the default sub and add a small feature to the model to ensure that DriverPath ends with / . This is ensured by the match="/$" snippet: 
$dispatch "LCDd::server" DriverPath  = sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    return   $dispatch _default_ ->(@_)
           . q! match="/$" !;
 ;
Likewise, this sub invokes the default sub with the optional 4th parameter that forces the value_type of the element. In this case, we enforce an integer element: 
 $dispatch "LCDd::server" WaitTime 
 = $dispatch "LCDd::server" ReportLevel 
 = $dispatch "LCDd::server" Port 
 = sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    return $dispatch _default_ ->( @_, 'integer' );
   ;
This sub returns a string for a list element with default values: 
  $dispatch "LCDd::server" GoodBye 
= $dispatch "LCDd::server" Hello 
= sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    my $ret = qq( class:"$class"
                    element:$elt
                      type=list
                ) ;
    $ret .= 'cargo type=leaf
             value_type=uniline - ' ;
    $ret .= 'default_with_init:0="\" '.$elt.'\"" ' ;
    $ret .= 'default_with_init:1="\" LCDproc!\""';
    return $ret ;
 ;
Now that the dispatch table is ready, let s really mine LCDd.conf information. First handle all INI classes in the foreach loop: 
foreach my $ini_class (@ini_classes)
Let s retrieve parameters for this INI class from the LCDd.conf template: 
  my $ini_obj = $dummy->grab($ini_class);
This will be the name of the lcdproc driver configuration class (e.g. LCDd::imonlcd): 
  my $config_class   = "LCDd::$ini_class";
The configuration class will also be created in a serialised format. Let s create config class in case there s no parameter in INI file: 
  $meta_root->load(qq!class:"LCDd::$ini_class"!);
Now, loop over all INI parameters and create LCDd::$ini_class elements: 
  my @all_ini_params = $ini_obj->get_element_name ;
  foreach my $ini_param ( @all_ini_params )
Retrieve the value provided in template LCDd.conf: 
    my $ini_v = $ini_obj->grab_value($ini_param);
Retrieve the comment attached to the value provided in template LCDd.conf: 
    my $ini_comment
      = $ini_obj->grab($ini_param)->annotation;
    # escape embedded quotes
    $ini_comment =~ s/"/\\"/g;
Retrieve the correct sub from dispatch table: 
    my $sub = $dispatch $config_class $ini_param 
              $dispatch _default_ ;
Run the sub to get the model string: 
    my $model_spec
      = $sub->( $config_class, $ini_param,
                   $ini_comment, $ini_v );
Attach the comment from LCDd.conf as class description: 
    $model_spec .= qq! description="$ini_comment"!
      if length($ini_comment);
Then, last but not least, load class specification in the model, and end the loop aroung INI parameters: 
    $meta_root->load($model_spec);
Now create a an $ini_class element in LCDd class (to link LCDd class and LCDd::$ini_class): 
  my $driver_class_spec = qq!
    class:LCDd
      element:$ini_class
  ! ;
Since server and menu are not drivers, they have a special treatment: 
  if (   $ini_class eq 'server'
      or $ini_class eq 'menu' )  
    $driver_class_spec .= qq!
      type=node
      config_class_name="LCDd::$ini_class"
    ! ;
Here s the treatment for LCDd drivers. Setup the model with a warped node so that a driver configuration class is shown only if the driver was selected in the [server] class: 
  else  
    $driver_class_spec .= qq!
      type=warped_node
      config_class_name="LCDd::$ini_class"
      level=hidden
      follow:selected="- server Driver"
      rules:"\$selected eq '$ini_class'"
        level=normal
    !;
Now we can load the LCDd driver model in meta root: 
  $meta_root->load($driver_class_spec);
Step 5: save LCDd model Still with me ? Good. Here s the last part, now that the model was created in memory, let s save it in files. Config::Model::Itself constructor returns an object to read or write the data: 
my $rw_obj = Config::Model::Itself
     ->new( model_object => $meta_root );
say "Writing all models in file (please wait)";
$rw_obj->write_all(
    model_dir => 'lib/Config/Model/models/'
);
That s it. Once this script is run, the directory lib/Config/Model/models/ contains a bunch of files: You can see the resulting editor in my first blog on lcdproc. Conclusion and future steps (with you?) As mentioned at the end of this blog, the result of this model generation process is far from perfect, because the information retrieved from LCDd.conf template are not complete for this task (which is normal since these comments were targeted for users). Now, I d really like to discuss with project developers to improve configuration template to generate better models. For project owner, such an improvement would give them: Please leave a comment on this blog (or send a message to config-model-users at lists.sourceforge.net) if you are interested in configuration tool generation for a project you contribute to. All the best

Raphaël Hertzog: My Debian activities in June 2011

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (195 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Dropbox for Debian This is not free software but Dropbox is very popular and they did only provide an Ubuntu package that did not work on Debian. So I created an official package. I have been in touch with Dropbox developers and they have been very helpful. They ll shortly release a signature mechanism (with GPG) so that we can further improve the package by verifying the origin of the downloaded binaries. SAT Britney At the start of the month, I continued my work on the britney reimplementation (the software that creates testing out of unstable) but I quickly stalled it because the release managers asked the feedback of Stefano Zacchiroli and Ralf Treinen (who have extensive knowledge on the topic with their research work on Mancoosi) and I did not want to invest further work in case they would identify a major flow the feedback came only very late this month and while it was somewhat negative, I still think it s worth pursuing the effort for a bit longer. Converted ftplib to multiarch While dpkg still doesn t support multiarch (no news from Guillem and no visible sign of progress :-( ), unstable got all the remaining bits allowing us to convert libraries to multiarch (see the announce). As soon as the required libc6 landed in unstable, I looked into converting the only library package that I maintain. I had no major problem but I still identified 2 issues in Lintian (filed as #630164 and quickly fixed by Niels Thykier). build-arch / build-indep support For the 42th time in the last 10 years, the idea of using build-arch/build-indep targets in the rules file has surfaced again. I had already decided some time ago that I would accept a patch implementing a new field Build-Features to enable dpkg-buildpackage to use those targets and this time Bill Allombert completed such a patch so I merged it. The technical committee also decided that it would take a final decision on this topic (see #629385). Roger Leigh provided useful input by doing an archive-wide rebuild with the various solutions suggested. Given that the majority would like to make the target mandatory at some point in the future, I provided the dpkg patch for my preferred solution. We would use auto-detection as a temporary measure until all packages have been converted to have the targets. The technical committee has not yet taken any decision even though the discussion stalled since the 12th of June. But that s usual with that body. I m sure it will be solved during Debconf. ;-) Misc dpkg work Hamster applet update Hamster-applet is a GNOME application which did not have a 3.0 release, but it had a development release (2.91.x). I checked out whether it was possible to package this version for experimental and have the applet work with the GNOME fallback mode. Apparently not, the code was not yet updated to be compatible with the newer panel. Instead I uploaded the latest stable version (2.32.1) to unstable. It has some nice improvements in the standalone version (and the name of the executable changed). For usage with GNOME 3, I have created a custom shortcut to start it quickly (with gconf-editor set /apps/metacity/global_keybindings/run_command_1 to <Mod4>t and /apps/metacity/keybinding_commands/command_1 to hamster-time-tracker because the GNOME 3 control panel does not seem to work to set custom keybindings currently). Translated my professional website into English While I m grateful for all the people who are supporting my work, I m still far from my goal to have one third of my time funded through donations and sales of products on this blog. So I decided to also bring more visibility to my company and in particular to its Debian-related service offering. It was only available in French up to now so I translated it and expanded it a bit. My support page on this blog now also links to my company s website. If your company needs help to create Debian packages, or needs Debian technical support by email, you just found the right partner. :-) BTW, I have discounted prices for individuals and non-profits who would like to benefit from my help to create Debian packages. The Debian Administrator s Handbook This is the title of the upcoming translation of my book. The project now has a dedicated website: debian-handbook.info. You can subscribe to its RSS feed to keep up with the latest news. The full table of contents is online along with a FAQ. I m actively looking for partners to help me promote the fundraising once it goes live. If you can reach a large set of readers interested by a good Debian book, get in touch with me to join the affiliate program. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Russell Coker: Can Online Dating make You Depressed?

Anne Rettenberg wrote an article for Psychology Today that is critical of the idea of online dating [1], she cites one example of a man who visited a prostitute due to being depressed at his lack of success in online dating to support her claim. The first big problem with her article is that she doesn t mention the different experiences that male and female customers presumably have on online dating sites. I don t know what it s like for women on the dating sites so I can t comment on that. But I m sure that someone who works as a counselor could provide some useful insight into this matter. Also she didn t even give a mention to the issue of gay/lesbian dating sites. The next issue is that she didn t offer any good advice for who should use online dating sites and what their aims should be. Rejection In dating in real life (IRL) it s expected that the man make the first move, and therefore women end up rejecting lots of guys for various reasons. Anne seems to think that rejection online is somehow worse than rejection IRL, it probably is for some people but that certainly isn t the case for everyone. The way that lots of dating sites seem to work is that women place adverts, men respond to them, and then the women reply to a small subset of the email that they receive. Rejection in this case isn t a matter of telling someone that you aren t interested, but of merely not replying to their mail. From my discussions with a few men who ve used online dating sites the strategy seems to be to send out initial messages to a few dozen women every day and then maybe get a few responses a day. For the messages that get no response you will never know whether the other person found someone else first, wasn t interested in you for some reason, or just didn t bother checking their email. The only comparison to IRL rejection is that which happens after phone numbers have been exchanged, which isn t going to be that common (and has the same issues regardless of how the people met). Guys, relax about the women who don t respond to your first message. They probably get 100+ responses to their advert and don t have time to even read half of them. If you get rejected later in the process then you can look through your email archive at a later date to try and discover what went wrong. The Aim of Online Dating The fact that Anne s client visited a prostitute suggests that maybe he wasn t really after a relationship. In which case using one of the many online services for finding sex partners might have been a better option. Generally it seems that a good strategy is to try and have fun. I don t know any men who have married someone from a dating site, but the general opinion seems to be that they are still worth using. If you meet someone in a bar then you might end up having a drunken conversation that is drowned out by loud music. If you meet someone over the Internet then you can have a quiet conversation over the phone which seems to be a better way to get to know someone (and generally more pleasant for anyone who s not an alcoholic). I think that men who have no immediate aim other than finding someone nice to talk to will do better than those who aim to score quickly. Of the men I know who married women they met over the Internet (but not through online dating sites) I wonder how many of them would have ended up married if they hadn t used the online dating sites first. It seems that men who regularly communicate with women outside formal situations (work etc) will have a better chance of impressing someone that they like than those who lack such experience. Introverts There are a lot of people who really can t function in a bar. With the way our society works it seems that anyone who can t handle the bar scene really should consider online dating. How Counselors can Help It seems to me that someone who is seeing a counselor and who is considering a new way of finding a SO should ask their counselor for advice first. It also seems to me that a good option might be to ask their counselor for advice in online dating. Instead of being unsuccessful and depressed a man who was seeing a female counselor could do well to ask her advice for how to impress women on the Internet. This is probably a business opportunity for female counselors who can advise men on such things, among other things it seems that seeing an online relationship coach would be perceived in a more positive manner than seeing a counselor or psychologist for the more traditional reasons.

Erich Schubert: Google vs. Facebook

Let the games begin. It looks like Google and Facebook are going to fight it out.
Given the spectacular failures of Wave and Buzz, people are of course skeptic about the success of Google Plus. However, I'm rather confident it is going to stay. Here are some things I like about it:
Some people think that Google will not stand a chance against social giant Facebook. But after all, Google has more users - and they have tons of services people like to use. So when Google Maps has Plus integration, will the users use it to chat about where to meet, or will they go the long way and post the map URL on Facebook, without the option of updating it collaboratively?
Googles position is much stronger than many people believe, once you think about integration possibilities. The current Plus is just a fragment, the missing puzzle piece connecting the other apps. But imagine that Google now Plus-connects its various services: YouTube, Maps, Mail, Talk, ... - for them, this is just a matter of some engineering. I guess probably half of these are already in internal testing. And Facebook just can't keep up there. Sure, they do have Facebook Video. But actually, most people prefer YouTube. And while Google can integrate plus all the way there, Facebook cannot. And while Google is the master of search, Facebook is particularly weak there - they can't even properly search their own data. Google however will at some point offer an "find things that interest me" button; Sparks is just the beginning where you have to manually define your interests (which will probably remain an option due to privacy concerns!); it is way too static right now.
So essentially, Google doesn't need to copy Facebook. They just need to do what is obvious on their own products, and Facebook will have a hard time keeping up.
Plus, in my opinion, Google got the timing just right. Users aren't too happy with Facebook these days, there is just no big alternative around anymore; their friends are on Facebook and not on some other social network. Facebook doesn't seem to evolve much anymore. The mail functionality opened more security holes (apparently you can post to a group with a fake user name when you spoof the senders email address) than it contributed to functionality and usefulness. Privacy is still not in line with all countries such as Germany; but Facebook keeps telling those users essentially that they don't care. Spam and fraud still reappears every month following the same pattern again and again (Clickjacking). The search function of Facebook is still usually described as "useless" ... People waste time in games and annoy their friends with random game invitations and posts. Facebook should better make a major move now, too. More than a demo of Skype integration. But whenever Facebook changed, their users complained ...
Of course, Google+ still has a long way to go, too. There are still many things missing here, too. For example groups and events. I figure Google is already testing them in "dogfood", and they'll actually come out within the month. With groups I do not refer to the existing Google product, but to what would be "public circles" that you need to join and that are accessible to all the circle members instead of just the creator. And events are also a key function; probably one of the most used on Facebook. These may require much more careful design to integrate well with Calendar. But given the visual update of Calender these days, this may just be around the corner, too.

Rog rio Brito: Installing Meego on the N900

My last cellphone was a dumb phone. I seem to remember that it was called something like Samsung Voicer. It didn't have those hip things like SIM cards. OK. Fast forward to December of 2010, and after a long hiatus, I decided to get a new phone. It was this shiny Nokia N900. The biggest thing that made me want this one was that some people at DebConf 10 were praising it for having a Debian-like distribution in it, and that it worked very well. I saw that Phil Hands was enthusiastically talking about the bargain of getting one in New York and that it would have cost him many pounds in the UK and, so, he was happy with it. Ot vio Salvador also got one in NY and during one of the nights of that DebConf with him, Tiago Vaz, Daniel Baumann, and Chris Lamb, I asked him to call my parents here in Brazil and he let me use his N900. I had more time with it and played a little bit with the command line, opened the stock media player and played a beautiful trailer of 9 (2009 film), leaving me a very good impression. I bought one, and, indeed, it is a very good machine that can even make phone calls. The ability to run Linux on it (even if it contains some non-free pieces of software) was decisive in me getting it. It runs this distribution called Maemo 5 which is loosely based on Debian. But now, Maemo is not the "cool thing" to run, regarding Linux on these portable devices. Meego is. But with the advent of many Internet forums has brought many people writing some long, convoluted howto documents for things that would be better done as, say, providing a script or, better yet, preparing a package. In the case of getting Meego running on the N900, only three things are required:
  1. Install uboot-pr13 while in Maemo.
  2. Uncompress the MeeGo 1.2 Community Edition for Nokia N900 image to a SD card.
  3. Turn the phone off, plug in the SD card, and PUT THE BACK COVER on the phone.
The last bit of the third step is crucial, as, otherwise, the SD card won't be detected and you will get kernel panics with the device trying to mount the root filesystem from a device that is not there. As an aside, the official documentation tells us one should uncompress the available pre-made images and write them with dd to the device. In my experience, it is completely unnecessary to use it and, in fact, it is so 6 times slower than using a simple shell redirection. That is, instead of: 
bzip2 -d < mg-handset-armv7nhl-n900-whateverwhatever-mmcblk0p.raw.bz2   dd bs=4096 of=/dev/mmcblk0
you can get better results with the simpler: 
bzip2 -d < mg-handset-armv7nhl-n900-whateverwhatever-mmcblk0p.raw.bz2 > /dev/mmcblk0
I actually used lbzip2 instead of bzip2, but that shouldn't matter. The use of the small block size for the dd command is probably the culprit, but I don't see the need to write the uncompressed data in chunks this small. If there is a problem, I would love to be informed of that. As trying it with uboot and a SD card doesn't mess with your "safe" Maemo installation, this is a good way to play with the successor of Maemo. Perhaps, if we give good feedback to the project, we can influence the direction that it is taking. It will surely be nice to learn about this new "consumer-oriented" distribution.

Gunnar Wolf: DebConf13 at home? Why not?

We are few weeks away from the start of DebConf11. Excitement runs high in Debian-land. The two most worthy weeks of the year, every year, loom close. Our Bosnian friends have done a great job of finding and defending an amazing proposal, and are now facing the hard work and permanent adrenaline levels of being in charge of the closest I have seen to a herd of (well-behaved but wild and untamable) cats. I have organized DebConf in my country. It was hellish, but at the same time, it's one of my most cherished experiences. And I'm sure the same will be said by the leaders of each successive bid It is one of the most rewarding experiences you can imagine. Next year, DebConf will be held in tropical Managua, Nicaragua. But, where will we meet in 2013? Well, that depends on you, my dear reader! Do you want to work your ass off for Debian and have utter fun? Do you want to show and share your country with this huge family of developers? Start thinking about pushing for a DebConf13 bid! Do you have to be at Banja Luka to propose your bid? No. You can proxy via somebody I'd suggest to do it via somebody who knows the location you are suggesting, but basically, choose a friend that you trust that trusts you. Of course, you can participate in the presentation session via IRC. Do you have to be a Debian Developer to propose a bid? No. For DebConf9, none of the C ceres guys was a DD; for DebConf10, some of the people most involved from the local New Yorkers were not DDs. For DC11, none of our dear and overworked hosts in Bosnia are DDs. And for DC12, the Nicaraguan crew is also made from people interested in getting closer to the Debian project, but not DDs. Do you have to decide now? No. This is just a call for a first presentation, but the decision regarding DC13 will be taken probably around March 2012. However, giving a nice presentation at DebConf helps a lot, gives you visibility, and will get the ball rolling. Is there a geographical bias? Slight. So far, and since the second DebConf, we have kept the tradition not to repeat continents on two successive DebConfs. This is not a hard condition, however! What do you need to start thinking about? Go visit our prospective location checklist at http://wiki.debconf.org/wiki/LocationCheckList. You can also look at what other teams have historically presented. Finally, I just learnt about the existence of http://wiki.debconf.org/wiki/DebConf13 Register there, even if you are just in the early phases of finding data. We will be holding a DebConf13 bids presentation session, most probably (the schedule is close to being presented officially!) on 30-07-2011, at 17:00.

2 July 2011

Martin Meredith: Google+: Part 1 Circles

So, I haven t read the planets in a couple of days but I m going to guess that, like twitter, a lot of people are going to be raving about Google+. For those of you who haven t heard about Google+, it s a new Social Networking site from Google and it brings a new take to the world of social networking. In real life, we communicate differently to the different people in our social contexts. For example, most people will converse differently to their work colleagues than they will to their family. They ll converse differently with their drinking buddies than they will with their Family. There are things that you want only certain people to know, and there are aspects of your personality that you only want to show to certain people. I think giving an example here might be better. This is an example of a friend of mine, who we ll call Fred. Fred is a teacher. Fred also likes to go out of a weekend and party till dawn, while consuming lots of tequila. I, as Fred s friend, am on a night out of drunken debauchery, and I ve my camera with me, and decide to post lots of pictures online of the night s revelry. The next day, one of Fred s colleagues logs into a Social Networking site, and sees said pictures in Fred s profile. Now things aren t looking so bright for Fred. He no longer gives across that professional image that he should do at work. Some may say that this is Fred s fault for going out and doing these things, but everyone s entitled to a personal life, right? At the moment, on Fred s social networking site, it s an all or nothing option. If he allows you access to his profile, he allows you access to see anything and everything that might show up on there. (It s a bit more complicated than this, but let s keep it simple for now). Every time Fred has someone add him as a contact he has to make that decision Do I want this person to be able to see everything I may or may not get up to . Put simply, how many of you have refused someone to add you on the Social Networking site du jour because what you show on their may be harmful to you, or give over an impression that isn t what you want to give across. Enter Circles. Circles brings to the social networking scene the way we interact with people in real life. We show a different aspect of our personality to different people we interact with. I m a bit of an organisational freak, and I know that there are different groups of people that either for the fact of portraying a persona, or not annoying everyone else, I might want to push in different ways. For example, I might want to send out a BBQ at my house, bring lots of alcohol message to all my friends, but don t really want my co-workers coming along and making it so I have to be on my best behaviour. Below is a screenshot of my (current) circles. As you can see I have a fair few, and only a relative few people. Now, let s say that I wanted to send out that BBQ message As you can see I ve the option to send this to the 5 people that are in my Friends group (it s currently mostly geeks on there there are only a few of my real life friends on there so far!) Now with only a few clicks, I ve managed to send out a message to just those few people that I want to :) Over the next few days, I ll write a bit more about Google+, but I thought I d start with an introduction into the Circles feature. I ll be giving a bit of feedback about what I like, what I dislike, and also making a few suggestions for what I d like to see happen in the future (and probably use the word siloing a fair bit!) For now, toodles.

Joey Hess: databranches: using git as a database

I've just released git-annex version 3, which stops cluttering the filesystem with .git-annex directories. Instead it stores its data in a git-annex branch, which it manages entirely transparently to the user. It is essentially now using git as a distributed NOSQL database. Let's call it a databranch. This is not an unheard of thing to do with git. The git notes built into recent git does something similar, using a dynamically balanced tree in a hidden branch to store notes. My own pristine-tar injects data into a git branch. (Thanks to Alexander Wirt for showing me how to do that when I was a git newbie.) Some distributed bug trackers store their data in git in various ways. What I think takes git-annex beyond these is that it not only injects data into git, but it does it in a way that's efficient for large quantities of changing data, and it automates merging remote changes into its databranch. This is novel enough to write up how I did it, especially the latter which tends to be a weak spot in things that use git this way. Indeed, it's important to approach your design for using git as a database from the perspective of automated merging. Get the merging right and the rest will follow. I've chosen to use the simplest possible merge, the union merge: When merging parent trees A and B, the result will have all files that are in either A or B, and files present in both will have their lines merged (and possibly reordered or uniqed). The main thing git-annex stores in its databranch is a bunch of presence logs. Each log file corresponds to one item, and has lines with this form: 
timestamp [0 1] id
This records whether the item was present at the specified id at a given time. It can be easily union merged, since only the newest timestamp for an id is relevant. Older lines can be compacted away whenever the log is updated. Generalizing this technique for other kinds of data is probably an interesting problem. :) While git can union merge changes into the currently checked out branch, when using git as a database, you want to merge into your internal-use databranch instead, and maintaining a checkout of that branch is inefficient. So git-annex includes a general purpose git-union-merge command that can union merge changes into a git branch, efficiently, without needing the branch to be checked out. Another problem is how to trigger the merge when git pulls changes from remotes. There is no suitible git hook (post-merge won't do because the checked out branch may not change at all). git-annex works around this problem by automatically merging */git-annex into git-annex each time it is run. I hope that git might eventually get such capabilities built into it to better support this type of thing. So that's the data. Now, how to efficiently inject it into your databranch? And how to efficiently retrieve it? The second question is easier to answer, although it took me a while to find the right way ... Which is two orders of magnitude faster than the wrong way, and fairly close in speed to reading data files directly from the filesystem. The right choice is to use git-cat-file --batch; starting it up the first time data is requested, and leaving it running for further queries. This would be straightforward, except git-cat-file --batch is a little difficult when a file is requested that does not exist. To detect that, you'll have to examine its stderr for error messages too. Perhaps git-cat-file --batch could be improved to print something machine parseable to stdout when it cannot find a file. Takes some careful parsing, but straightforward. Efficiently injecting changes into the databranch was another place where my first attempt was an order of magnitude slower than my final code. The key trick is to maintain a separate index file for the branch. (Set GIT_INDEX_FILE to make git use it.) Then changes can be fed into git by using git hash-object, and those hashes recorded into the branch's index file with git update-index --index-info. Finally, just commit the separate index file and update the branch's ref. That works ok, but the sad truth is that git's index files don't scale well as the number of files in the tree grows. Once you have a hundred thousand or so files, updating an index file becomes slow, since for every update, git has to rewrite the entire file. I hope that git will be improved to scale better, perhaps by some git wizard who understands index files (does anyone except Junio and Linus?) arranging for them to be modified in-place. In the meantime, I use a workaround: Each change that will be committed to the databranch is first recorded into a journal file, and when git-annex shuts down, it runs git hash-object just once, passing it all the journal files, and feeds the resulting hashes into a single call to git update-index. Of course, my database code has to make sure to check the journal when retrieving data. And of course, it has to deal with possibly being interrupted in the middle of updating the journal, or before it can commit it, and so forth. If gory details interest you, the complete code for using a git branch as a database, with journaling, is here. After all that, git-annex turned out to be nearly as fast as before when it was simply reading files from the filesystem, and actually faster in some cases. And without the clutter of the .git-annex/ directory, git use is overall faster, commits are uncluttered, and there's no difficulty with branching. Using a git branch as a database is not always the right choice, and git's plumbing could be improved to better support it, but it is an interesting technique.

Guillaume Mazoyer: Status report Jigsaw num. 3 for GSoC 2011

code.pngAt beginning of the last two weeks, I was working on the final package of JTReg. I was working on generating man pages and I finally found a solution. I decided to use help2man to generate the man pages but I had to patch the two scripts jtreg.sh and jtdiff.sh . Basically, these two scripts launch main classes of the jtreg.jar JAR file. The problem with them is that they don t know where they can find the JAR file on Debian. So I patched the two scripts shell so they can locate JAR files in /usr/share/java .
To be able to generate man pages with help2man, I needed to be able to use the scripts. But with jtdiff.sh and jtreg.sh using jtreg.jar which is not in /usr/share/java during the build, I had to find a way to make the scripts work. I decided to patch the scripts (again) and make them depend of some environment variables. So in each script, there are 4 used variables: So during the packaging only JTREG_HOME needed to be changed to be able to use the script. Once the scripts patch done, I finally wrote the rules file of the package. Once again, I used javahelper so the rules file contains the following code:

#!/usr/bin/make -f
JAVA_HOME = /usr/lib/jvm/default-java

override_dh_auto_build:
ant -f make/build.xml
dh_auto_build
JTREG_HOME=./dist/jtreg/lib/ help2man \
--name="Regression Test Harness" \
--help-option="-help" \
./dist/jtreg/linux/bin/jtdiff > jtdiff.1
JTREG_HOME=./dist/jtreg/lib/ help2man \
--name="Regression Test Harness" \
--help-option="-help" \
./dist/jtreg/linux/bin/jtreg > jtreg.1

override_dh_auto_clean:
rm -r dist :
rm -r build :
rm jtdiff.1 :
rm jtreg.1 :
dh_auto_clean

%:
dh $@ --with javahelper

As you can see it is not really complicated. The most interesting part is the one when the man pages are generated. The JTREG_HOME variable is set to ./dist/jtreg/lib/ because it is the path where jtreg.jar is once it is built. I also used the help-option option of help2man because the help option of jtreg and jtdiff is -help and not help .
The jtreg package is now on the Debian Java team SVN so anyone can get it using:

svn checkout svn+ssh://$ SVN_USER @svn.debian.org/svn/pkg-java/trunk/jtreg

Thanks to Sylestre Ledru, jtreg is now available in Sid and the ITP is now closed.

The second part of my work consisted in testing Jigsaw and fixed tests if necessary. All the packaging work is now useful since the tests need JTReg to be run. The problem with the current build system of Jigsaw is that it is enough generic to be used on any system. So I had to patch several makefiles. To find them:

cd jigsaw-tests
find . -name 'Makefile' grep '/test/Makefile'

Then, the path to find jtreg has to be modified to match /usr/bin/jtreg . After that it is possible to run the tests with the following command:

make test

Passing all the tests is something which takes a lot of time. After that, I identified 42 (that s a cool number right?) failing tests. Some of them came from the lack of X11 server and other from unreachable network hosts. Some hackers of IcedTea gave me some ideas to fix several tests. They told me to look here and here because I just had the same problems that they had before.
With this, I was able to fix the tests related to unreachable hosts and to the lack of X server. I used Xvfb to fix the X server depending tests.
After running the tests again, only 13 tests were still failing. That is a good result. Tom Marble and me decided to apply the patch of Alan Bateman and re-run the tests to see if it will break something:

cd jigsaw-tests
mkdir patches && cd patches
wget http://cr.openjdk.java.net/~alanb/jigsaw-mp-prototype1/webrev/jdk.patch
cd jdk
patch -p1 < ../patches/jdk.patch
make clean
make sanity
make all
make modules
xvfb-run -e xvfb-errors -a -s -ac make test -k

15 tests failed against 13 before. The 2 new failing tests are: I didn t go further but I ll try to understand why we have 15 failing tests in few days. All the modifications that I have made on the source code of Jigsaw can be found in a patch format here:

This 2 weeks were also dedicated to the understanding of Jigsaw and its modules. Reading the jigsaw-dev mailing list, I have found some interesting conversation about jigsaw and how to write modules. So I decided to write a module to see how it works. I followed the instruction of the quick start guide but when compiling the module didn t work. So I tried to understand why and then to fix it. I eventually compiled and installed my module and here is how I did it:

mkdir -p src/com.greetings/com/greetings/
mkdir -p src/org.astro/org/astro/
mkdir modules
vim src/com.greetings/module-info.java
vim src/com.greetings/com/greetings/Hello.java
vim src/org.astro/module-info.java
vim src/org.astro/org/astro/World.java
./jigsaw/build/linux-amd64/bin/javac -d modules \
-modulepath ./jigsaw/build/linux-amd64/modules/modules \
-sourcepath src find src -name '*.java'
./jigsaw/build/linux-amd64/bin/jmod create -L mlib
./jigsaw/build/linux-amd64/bin/jmod install \
modules org.astro com.greetings -L mlib
./jigsaw/build/linux-amd64/bin/java -L mlib -m com.greetings

And here it is, we have a nice hello world module. The documentation says to put the module-info.java of org.astro in src/org.astro/org/astro/ but it didn t work for me so I tried to put it in src/org.astro/ and it worked. Also the quick start guide says that we have to use the -modulepath option to specify where the modules are. I first tried to use ./jigsaw/build/linux-amd64/modules/ but javac told me he could not find java.lang so I noticed that there is a subdirectory called modules in ./jigsaw/build/linux-amd64/modules/ so I used it as module path and it worked.

Writing a module and seeing how Jigsaw modules are made started to make me think about the packaging of Jigsaw. Dependencies being available, knowing how to build and test Jigsaw and seeing how modules are made, I think that I will soon try to start the packaging of Jigsaw and getting started with the packaging will be my goal for the next two weeks.

1 July 2011

Gunnar Wolf: Yet another flurry of invites

Umh, so Google created yet-another-beta-service with yet-another-invite-only stage. Whee! And this time they choose a name yet more stupid to their previous attempts (Wave, Buzz now it's simply Google+). And yes, some of my friends have mailed me invites (or offered them to me by chat) so I can join the fun and love, and lose the few free time I still have. Guys, I care about you. I even love you all! But no, I don't care the least for so-called-social-network invites. I am quite a mess managing my time as it stands. And I cannot reply this to you in person, as Google+ sends mails with a very helpful sender (noreply-929ac25c@plus.google.com or variations of it), I have just added this to my .procmailrc: 
  1. :0
  2. * ^From:.*noreply.*@plus.google.com
  3. /dev/null
What does this mean in human? That instead of manually ignoring your invitations and feeling bad about not answering to you, dear friends, I'll leave that job to my computer. I will not receive any Google+ invites. Just as I don't receive Twitter and Facebook spam, FWIW. Thanks for thinking about me!

Michael Casadevall: Pandaboard Netboot Images Now Available

As I mentioned in my previous blog post, OMAP4 netboot images were available, but non-functional. I'm pleased to announce that these bugs have now been resolved and it is possible to have a functional install on OMAP4. This also has the added advantage of allowing one special partitioning layouts such as RAID, LVM, or simply having a non-SD based root device. The images are available here: http://ports.ubuntu.com/ubuntu-ports/dists/oneiric/main/installer-armel/current/images/omap4/netboot/

To use, simply dd boot.img-serial or boot.img-fb to an SD card, pop it in, and run, and the installer will pop up.

There is still a known bug that partman will not properly create the necessary boot partition. During the partitioning step, you must select manual partitioning, then create a 72 MiB FAT32 partition, with no mount point, and the Bootable flag must be set to 'on'. This partition must be the first partition on the device. flash-kernel-installer will be able to find the partition on its own.

Michal &#268;iha&#345;: Not going to DebConf

Year ago, when DebConf 11 in Banja Luka has been announced, I was really happy for the location and was sure I'm going to visit it. I applied early, requested sponsorship, but then things got slightly more complicated and in the end I had to say no quite shortly before reconfirmation deadline. So I will not meet all the great people behind Debian this year and will wait for another chance (probably in two years, as Nicaragua sounds attractive, but when I would be so far away I'd like to spend at least month in the area and time and money is always problem in such case :-)).

Filed under: Debian English 0 comments Flattr this!

Next.

Previous.