Search Results: "tar"

6 July 2011

Ritesh Raj Sarraf: Console Gaming - Sony Playstation 3

I have been thinking of this topic for some time. I am a happy owner of the old Phat PS3 Model. At one time, I use to do a fair amount of gaming on my computers. So when Sony marketed the PlayStation 3 product, it was cool to see a machine that did:
  1. Gaming
  2. Movies, Music, Pictures
  3. Blu-Ray
  4. Ran Linux
I convinced myself to buy a PlayStation 3, that I'd use it for playing with Linux apart from gaming. The regular gaming did not excite any more, perhaps it had to do with the limitation of time. On running Debian on it, was when I realized that the Linux support was merely a joke. The Linux support included running a Linux distribution in a VM with not much access to the full hardware resources. Not a Great Experienceread more

John Goerzen: Geeks, Hobbies, and Free/Open Source: Feedback Wanted

I ve been thinking lately about ways to improve ways in which I interact with Free Software projects, and ways in which they interact with me. Before I proceed to take steps or make suggestions, I d like to see if others share my traits and observations. Here are some questions I have been thinking of. If you d like to help give me anecdotal evidence, please post a comment below this post. Identify the question numbers you are answering. It helps me if you can give specific examples, but if you don t have the time or memory for that, no problem. I will post my own answers in a day or two, but the point of this post is listening, not talking, so I ll not post them immediately. Hobbies (General any geeks) Examples: video games might be a challenging hobby (depending on the person) but in most cases aren t educational. A hobby might be video game playing or being a Debian developer. A sub-hobby might be playing GTA IV , playing RPGs , or maintaining mutt . Free/Open Source Hobbies Examples on F9: If, say, you are a long-time Perl user and have gone to Perl conferences, but now you are interested in Ruby, would your involvement with the Perl community cause you to avoid taking up the Ruby programming hobby? Or would it cause you to cut your ties with Perl less quickly than your changing interest might dictate? (This is a completely arbitrary example and isn t meant to start a $LANGUAGE thread.) Changes over time

Dirk Eddelbuettel: Even faster linear model fits with R using RcppEigen

Linear regression models are a major component of every applied researcher's toolbox. Obtaining results more quickly is therefore of central importance, particularly when many such models have to be fit. Common examples in this context are Monte Carlo simulation or bootstrapping. My talks introducing High Performance Computing with R (see e.g. these slides from a five-hour workshop at the ISM in Tokyo) frequently feature an example of how to extend R with dedicated compiled code for linear regressions. Romain and I also frequently use this a motivating examples with our Rcpp package for seamless R and C++ integration. In fact, the examples directory for Rcpp still contains an earlier version of a benchmark for fastLm(), a faster alternative for R's lm() and lm.fit() functions. We have also extended this with the RcppArmadillo package which brings Conrad Sanderson's excellent Armadillo library with templated C++ code for linear algebra to R, as well as a simple integration to the GNU GSL via our RcppGSL package. The Rcpp section on my blog contains several posts about fastLm benchmarks. Doug Bates has been a key Rcpp contributor, helping particularly with the initial Armadillo integration. His research, however, also requires highly performing sparse matrix operations which Armadillo does not yet offer. So Doug has started to explore the Eigen project---a free C++ template math library mainly focused on vectors, matrices, and linear algebra (note that we will refer to the Eigen, Eigen2 and Eigen3 APIs as just 'Eigen' here, focusing on the latest version, Eigen3). Better still, Doug went to work and pretty much single-handedly wrote a new package RcppEigen which integrates the templated C++ library Eigen with R using Rcpp. RcppEigen also provides a fastLm implementation and benchmark script. In fact, it contains a full six different implementations as Doug is keenly interested in rank-revealing decompositions which can guard against ill-conditioned model matrices. Some more background information on this is also available in Doug's article on Least Squares Calculations in R in R News 4(1). Doug's implementation also uses an elegant design. It comprises a base class with common functionality, and six subclasses which specialize accordingly for these six different decompositions approaches: On my server, the result of running the included benchmark script lmBenchmark is as follows:
lm benchmark for n = 100000 and p = 40: nrep = 20
     test   relative elapsed user.self sys.self
7     LLt   1.000000   0.918      0.91     0.00
3    LDLt   1.002179   0.920      0.92     0.00
5 SymmEig   3.021786   2.774      2.19     0.57
6      QR   5.136166   4.715      4.24     0.48
2   PivQR   5.303922   4.869      4.27     0.58
8    arma   6.592593   6.052      6.03     0.02
1  lm.fit   9.386710   8.617      7.14     1.45
4     SVD  33.858388  31.082     30.19     0.84
9     GSL 114.972767 105.545    104.79     0.63
From this first set of results, the preferred method may be 'PivQR', the pivoted QR. Strictly-speaking, it is the only one we can compare to lm.fit() which also uses a pivoting scheme. In the case of a degenerated model matrix, all the other methods, including the four fastest approaches, are susceptible to producing incorrect estimates. Doug plans to make SVD and SymmEig rank-revealing too. As for pure speed, the LL and LDL decomposition have almost identical performance, and are clearly faster than the other approaches. Compared to lm.fit(), which is the best one could do with just R, we see an improvement by a factor of eight which is quite impressive (albeit not robust to rank-deficient model matrices). Apart from the SVD, all approaches using Eigen are faster than the one using Armadillo, which itself is still faster than R's lm.fit(). Doug and I were very surprised by the poor performance of the GNU GSL (which also uses SVD) via RcppGSL. Now, Eigen uses its own code for all linear algebra operations, bypassing BLAS and LAPACK libraries. The results above were achieved with the current Atlas package in Ubuntu. If we take advantage of the BLAS / LAPACK plug-in architecture offered on Debian / Ubuntu systems (see the vignette in my gcbd package for more) and use Goto BLAS which provide tuning as well as parallelism on multi-core machines, the results are as follow:
lm benchmark for n = 100000 and p = 40: nrep = 20
     test   relative elapsed user.self sys.self
3    LDLt   1.000000   0.907      0.90     0.00
7     LLt   1.000000   0.907      0.91     0.00
5 SymmEig   2.981257   2.704      2.14     0.56
6      QR   5.004410   4.539      4.03     0.50
8    arma   5.072767   4.601     15.30     3.05
2   PivQR   5.307607   4.814      4.27     0.55
1  lm.fit   8.302095   7.530      9.55    12.25
4     SVD  33.015436  29.945     29.06     0.85
9     GSL 195.413451 177.240    244.64   319.89
We see that the BLAS-using Armadillo approach improves a little and moves just slightly ahead of the pivoted QR. On the other hand, lm.fit(), which also uses a pivoting scheme and hence only level 1 BLAS operations, changes less. GSL performs even worse (and it is unclear why). Doug's post announcing RcppEigen on the Eigen list has a few more sets of results. This post has illustrated some of the performance gains that can be obtained from using Eigen via RcppEigen. When not using rank-revealing methods, computing time can be reduced by up to eight times relative to lm.fit(). Rank-revealing method can still improve by almost a factor of two. The main disadvantage of Eigen may be one of the reasons behind its impressive performance: its heavily templated code does not use BLAS, and the resulting object code (as e.g. in RcppEigen) becomes enormous (when compiling with debugging symbols). As one illustration, the shared library for RcppEigen on my Ubuntu 64-bit system has a size of 24.6 mb whereas RcppArmadillo comes in at a mere 0.78 mb; without debugging symbols it is a more reasonable 0.52 mb. The performance of Eigen is certainly intriguiging, and its API is rather complete. It seems safe to say that we may see more R projects going to make use of Eigen thanks to the RcppEigen wrapper.

Update: Clarified statement about large object size which was entirely due to building with debugging support.

5 July 2011

MJ Ray: #coops14 continues: Guardian, Facebook, Cyberunions and Breakfast

Thanks to the Guardian for publishing Why co-operatives matter written by two members of our co-op. Co-operative Congress 2011Also today, I m pictured being given a share by Giles Simon of our national co-ordinating co-op on the Co-operatives Fortnight Facebook page. I m not a big fan of facebook and I still want to try the more sharing alternatives that cyberunions reviewed. Speaking of cyberunions, apparently they answer the criticisms from my review in episode six which I still haven t found time to listen to yet and there s an episode seven out already. Maybe I ll start to catch up later today (Tuesday), if I don t go to the Bristol Wireless Annual Meeting. Tomorrow is another busy day, hopefully including the Koha Town Hall Meeting. And there s still half a week of Co-operatives Fortnight to go, so we re not done by a long way yet! If you d like to take part, there s a Co-operative Business Breakfast in Weston-super-Mare on Friday morning at 9am please leave me a comment if you d like to come along.

Rudy Godoy: Compute Clusters Integration for Debian Development and Testing Report 3

Hello this is the third report regarding my project. As usual, quick
summary first. From my last report I have progressed much little than I
would like but, as stated previously also, I m finishing semester
and between exams and projects it takes almost all my time. However, I m
free of academic duties next week and I ll double the speed on the project. What was done so far

- Started working on the modifications to the Eucalyptus code, as
expected.
I ve almost complete the code for the Libvirt s XML Domain
definition. I ve abstracted the wrapper in order to support a broader
number of arches. Previously it was tailored for supporting x86/amd64
based machines. The idea is to have a list of supported arches and
generate the XML file according that. For ARM some parameters and
elements are different than for the others. This would allow for
people to support another architectures, given they are well supported
by KVM-Qemu.
Started working on changes to util/data.c to make sure the correct
parameters are being passed to the XML generator and to libvirt kvm
calls.
- Define and start to setup the testing / demo environment. Basically
involves having one machine to run KVM as hypervisor so the Eucalyptus
handler uses it to run ARM kvm-qemu images.
- Over this weekend I ll be publising the code changes on my
website. Steffen suggested to create some docs regarding the progress
too. I ll do that once I feel I m on track again for my current
pending work, that s coding. What I ll be working over the next two weeks

- My primary goal is to catch up on the work pace and finish the neccesary
code for Eucalyptus to support ARM images. This means to finish
modifications to:
node/handlers_kvm.c handling nc creation. Little changes are need
here from what I ve indentified already.
util/data. c,h handling instance creation. This is the part where
most of the work shall be done. I ve started to work here already.
- Test the code I ve complete already and send the patches to the teams
involved. To date I m still working on a local environment, because no
usable patches have been generated already, just little pieces here
and there.
- Since this is a sort of gluing project, that is code here and there
and make it work together. I m in the process of finding a machine to
be able to test parts of it once they are complete. I m still
contacting people to get such machine, in anycase I gues I can manage
to get one myself. I can t use my current machine since it involves
reinstalling and working at the OS level.
- For the project s demo I ll use both machines, one as the front-end
and the kvm one for NCs (nodes). That s the goal. What is required after that

- I d like to release a package that includes my changes, so it can be
installed either from official packages or my own packages, this
involves:
Updating the packaging bits to the pkg-Eucalyptus repo.
Talking to upstream about it.
- Write the reference Steffen suggested. I believe that I can finish the first part in one week, because of the
work I ve done in the past weeks to understand the Eucalyptus code. I m
also will be talking more to the Eucalyptus team and also to the
maintainers about such changes. I guess that s all for now.

John Goerzen: The Lives of Others

It s not very often that I watch a movie anymore. It s been a few years since I ve actually purchased one (normally I see them from Netflix). But yesterday I saw one that may change that. The Lives of Others is an incredible film set in the former East Germany (GDR/DDR) mostly in 1984. The authenticity of it is incredible and so is the story. It s subtitled, but if you re an American wary of subtitled European films, don t be wary of this one. It is easy to watch and worth every minute. The story revolves around the Stasi, the GDR Ministry for State Security ( secret police ). It is an incredible picture of what living in a police state was like, and how many of the informants were victims of the regime too. My breath caught near the beginning of the film, showing the inside of a Stasi building. A prisoner was being interrogated for helping someone attempt to escape to the west. But the reason my breath caught was this incredible feeling of I was there . Last year, Terah and I were in Leipzig and visited the Stasi museum there, Museum in der Runden Ecke . I always have an incredible sense of history when being in a preserved place, and this building was literally the Stasi headquarters for Leipzig. Much of it was preserved intact, and seeing it in the film brought home even more vividly the terrible things that happened in that building, and others like it, not so very long ago. IMG_2717 We watched the special features on the Blu-Ray disc, and one of them was an interview with director Florian Henckel von Donnersmarck. He described how he spent a lot of time interviewing both victims of the Stasi, as well as ex-Stasi officers. One of the most disturbing things to me was his almost offhand comment that most of the former Stasi officers still had some pride in performing their jobs well. Even now, freed of the state s ideology, they were proud of the work they did which could be put most charitably as ruining people s lives. What leads a person to view life that way? How can we try to make sure it doesn t happen again elsewhere? I am happy to say that most of us have never experienced anything like the Stasi. And yet, small reflections of that mindset can be seen almost everywhere. Societies at wartime or feeling under threat, even Western democracies, can drum up those feelings. In the USA, for instance, the McCarthyism era saw people s careers ruined for alleged anti-state behavior. Contemporary examples include the indefinite detention (I hate that word; shouldn t we say imprisonment ?) of terrorism suspects at Guantanamo Bay, and the terrible treatment of Bradley Manning, who revealed some true but embarrassing things about the US military which really needed to be revealed. Even tobacco farmers and companies are selling a product they know ruins lives, but somehow keep doing it. And there are still members of the public that try to make life difficult for people that don t think like they do. From organizing campaigns of telephone harassment of colleges that don t perform the American national anthem before sporting events, to tossing about the term un-American (a loaded McCarthyist one, which many may not even be aware) at an inflated rate, we are not immune from attempts at forcing conformity or silence in others, and blind loyalty to state. I am never in a particularly celebratory mood on July 4, the biggest day for American boasting, faux patriotism, militarism, and general flag-waving. We do have a lot to be proud of and thankful for, but it seems that we celebrate all the wrong things on July 4, and see it as an occasion to proclaim American exceptionalism rather than as one to see how far we ve come and bolster hope for how far we can, and should, yet go. No, I don t think that the land of the free ought to have operated secret prisons in Europe (nor the Europeans to have been complicit in it), or that the American military was defending our freedom 100% of the time they were deployed, or that it is right for governments to mandate daily recitation of an untrue document (the pledge of allegiance) in schools. And yet, I am mindful that I have a lot to be thankful for stability, lack of much internal violent conflict, etc. And this particular day I am happy that a post like this is not something that gets the attention of some government agency and mostly that I will have a handful of angry emails to delete.

4 July 2011

Nathan Handler: Google Summer of Code Report #3

Since my last report, a lot of visible changes have been made. First, if you edit the value of any of the text inputs on the dashboard table, your changes will be saved when you hit the submit button. This feature still needs a bit of work though. For example, I noticed during testing that if you submit the form multiple times, you will sometimes end up with approximately duplicate entries in the changes file (the file that stores user-submitted changes to the table). I also need to implement some checks to ensure that the values submitted are actually valid (i.e. we probably only want to allow a few possible values for the status column). Another big improvement is the addition of a graph at the bottom of each project s page. This graph tracks the number of open tasks versus time. The graph is re-generated once each day based on a progress file. This file simply lists a date and the number of open tasks on each line. While the graph does correctly show a line tracking the progress, it still has some issues to sort out. For example, the x-axis does not properly display a time increment. Work on the graph was blocked for a while due to not having a fully-functioning dashboard. Now that the dashboard is functioning, I should be able to start working on fixing these graph issues. Several people approached me to request the ability to link directly to a particular project. That way, users do not need to mess around with the select boxes. Above each table is a title. If you click on the title, the URL should update to link directly to the project. This is done with the distro and project query parameters. For example, a link to dex.html?distro=ubuntu&project=ancient-patches will take you directly to Ubuntu s ancient-patches project. Attempting to specify an invalid distro or project will cause the query parameters to be ignored. While looking at the table title, you will also notice a new (Graph) link. This link will take you directly to the bottom of the page so you can view the graph. Currently, it is not possible to link directly to the graph anchor tag of a specific project. Attempting to do so will cause the #graph portion of the URL to be ignored. However, if you click on the actual graph, it will load a full-size graph.png that you can link to. One rather significant issue that has shown up is that the table is very wide. On most monitors, you will have to scroll horizontally to view all of the columns. This is due to the fact that most of the cells are made up of text inputs (they don t have a background or border, but if you click on some text in the table, you should see the box) rather than plain text. The size parameter for a text input was meant to specify how many characters should be visible. However, while I have the size parameter set correctly for each input, it is showing up much wider than the text. I will continue to play around with this, since excluding the size parameter results in some values getting cut-off. As mentioned in my last report, I have continued to work with Allison Randal to make the dashboard more useful. I have received feedback directly from her and indirectly from other people working on the python2.7 project. During my last phone call with my mentor, Stefano Zacchiroli, we agreed that it would be very helpful to spend some time documenting how to go about adding a new project to the dashboard and how to utilize all of the dashboard s features. The goal of this is to not require people to go through me or any other individual to be able to use the dashboard. So while the Summer of Code is meant to focus on programming, I will probably spend some time adding some documentation, as a program is rather useless if nobody knows how to use it. Another bug that I will be working on has to do with sorting the table. I had this feature working early on in the project. However, the sorting broke when I starting using text inputs instead of plain text. I have done some research, and I hope to get the sorting re-implemented in the near future. One other feature that has been requested by several people is the ability to hide completed tasks. This has been on my todo list for a while. Matt Zimmerman wanted to take this a step farther. He felt that the status of the task should be separate from the bug. This means that you can have an open task with a closed bug or a closed task with an open bug. The user could then have some more control over what data they are seeing. I will probably attempt to add some check boxes to control whether certain information is displayed. The user will also be able to specify this directly in the URL. This means that a team could link directly to all open tasks in a project. It is also worth noting that thanks to some help from the fantastic Alioth admins, I now have a live instance of the dashboard running on Alioth. While the URL might change in the future, I hope to keep it running on Alioth. If you are interested in helping out with DEX or dextools, my code is available in a git repository on alioth. You can also join the debian-derivatives mailing list, join #debian-derivatives on OFTC, or contact me directly via email or on IRC (nhandler).

Lars Wirzenius: strncpy? just say no

This is a rant. I've made it in various forms in various places over the years, and I think it's time to put it somewhere so I can point people at it, and stop repeating myself. Here's the pebble in my shoe: people keep claiming that strncpy is a safer alternative to strcpy. It's not. It's a slower function that is not going to make you significantly safer. The problem with strcpy is that it's somewhat easy to overflow the target array:
char src[] = "hello, world";
char dst[5];
strcpy(dst, src); // Oops.
The suggested alternative with strncpy would be this:
strncpy(dst, src, sizeof(dst));
strncpy gets the size of the destination array as a parameter, and can therefore make sure the string will fit there. Problem solved? Not so much. Every time you do any string handling with C's native string types and functions, you need to make sure you have allocated enough memory for each string, you need to keep track of how much memory you've allocated, and you neet to make sure the zero byte ('NUL', '\0') is put there correctly. This is a lot of details to keep track of, and it's not surprising things sometimes go wrong. If your arrays are allocated statically, which is the simplest, but least useful case, you need to something like this with strcpy:
if (strlen(src) + 1 > sizeof(dst))
    panic(); // or however you want to handle your errors
strcpy(dst, src);
The +1 after strlen is to make room for the zero byte (NUL, '\0') that is used in C to indicate the end of a string. With strncpy, your code is slightly simpler. This is its attraction. Problem is, this only works if dst is an array defined within the scope of the code. If, as is common, you have received dst as an argument to a function, sizeof(dst) is the size of a pointer to an element of the array, not the size of the array. So now you need to track the size explicitly in a second variable:
strncpy(dst, src, dst_size);
Further, strncpy has an unfortunate error handling mode: it merely fills the destination array and stops. It does not indicate failure in any way. It does not terminate the target array with a zero byte. Since all C string processing assume the zero byte is there, as soon as you try to use dst as a string, you're in trouble. But luckily you can check for that before you call strncpy:
if (strlen(src) + 1 > dst_size)
    panic(); // or whatever
strncpy(dst, src, dst_size);
By this time, the code is equally hard whether you use strcpy or strncpy. With strncpy you often waste some extra time, though, filling the unused parts of the target array with zero bytes: they're almost never necessary. Alternatively, you can force the zero byte there yourself, silently truncating a string.
strncpy(dst, src, dst_size);
dst[dst_size - 1] = '\0'; // let's hope dst_size > 0
If your string handling is OK with strings being randomly shorter than they should be, possibly you could replace all of them with empty strings, and save yourself a world of pain. Instead of strncpy, use snprintf. It has sensible error behavior, and is much more versatile:
n = snprintf(dst, dst_size, "%s", src);
if (n >= dst_size)
    panic();
snprintf still isn't very nice to use. It still requires you to allocate memory yourself, and keep track of the size of each array manually. It's also slower for small destination buffers (but faster for large ones). The real solution is to abandon C's antiquated string handling, and use a more sensible helper library instead. There's a bunch of them around, though none have gained much popularity. I'm not going to name any of them, but just imagine how much nicer it would be to write code like this instead:
Dynstr *src = dynstr_new_from_cstring("hello, world\n");
Dynstr *cat = dynstr_cat_many(src, src, src, src, src, NULL);
Dynstr *dog = dynstr_substr(cat, 15, 5);
Let the library take care of memory allocations and related tracking, and concentrate on real problem solving instead. We did this in Kannel, back in 1999, and it wiped out all the buffer overflow problems (until a co-worker decided it was "more efficient" to use native C strings instead, promptly introducing a buffer overflow). All of this string handling is, of course, way nicer in higher level languages. In fact, whenever you need to deal with string handling in C, it is probably a sign that you should consider switching to a higher level language instead. But whatever you do, stop claiming that strncpy is safe or solves any real problems strcpy has. Update: Fixed "It does terminate" to say "It does not terminate". Thanks to Matthew Garrett for pointing that out. Update2: Fixed example call to snprintf. Update3: Fixed the test for snprintf example. Thanks, Redditor axylone. Also pointed out that snprintf is slower than the rest.

Cosimo Alfarano: Facebook, Google+ and Diaspora

Google+ opened to the small public some days ago, I'm not really going to comment anything about it for now.

The purpose of my post is actually reminding people of another interesting project, Diaspora, which actually fits several of my views over social networking.

I really hope that when Google will release API for its G+, there will be some chance for Diaspora to integrate with it and, most important, an opening for G+ to operate with Diaspora.

Most of us use(d) Facebook, now almost any geek I know is joining G+, some of them rejoicing its advent as an opportunity to leave Facebook for good.
That's not really my intention and I don't think it's the point of the whole social thing.

What about, instead, allowing the whole social sites to inter-communicate, exchanging information and letting users being able to see each other independently of the platform used?

That's clearly not possible so far :-), but it is the reason for which years and years ago (Internet-time rules apply) I got interested in FOAF, last year I crossed my fingers with Diaspora and now I'm doing the same about Google receiving some enlightenment bit and getting the right path about profile sharing.

I'm aware that Google point is to re-factor the whole Google experience around Google Accounts and the profiles associated with them, and the major point of it is making G+ a bigger part of G profiles (if you are a Google Profile user, you'll probably noticed that your profile is now your public view of G+, with your public posts for instance).

My point is: Google is aware that not everyone agrees with Google wanting to extract information from your profile, your habits, etc and some of us is not really using Google 100% for this reason.
Also, Google started the Data Liberation front, so why not taking the next step and allowing people to inter-operate with other open [1] social sites?

The big part of the work is not helping Diaspora evolving and gaining power, IMHO, but pretty much the other way round: helping the idea of inter-communication to be a central point of the social experience: this idea getting stuck in some important and enlightened mind so that a bridge will be offered.

But I'm just a noob in the subject: it's just a foolish idea that has been in my mind for long enough and I needed to express :-D

Raphaël Hertzog: How to prepare patches for Debian packages

You want to start contributing to Debian and/or Ubuntu, you decided to help a package maintainer and you re now looking for how to change a source package and how to submit your changes. 1. Retrieve the source package and install build-dependencies The first step is to retrieve the latest version of the source package and to install the required build-dependencies. I already covered how to do this with apt-get in the article explaining how to rebuild a source package. If you prefer, you can use dget (from the devscripts package) to directly grab the source package. You can find the URL of the .dsc files in the Package Tracking System for example. Sometimes apt-get will warn you that the source package is maintained in a VCS repository, like this:
$ apt-get source wordpress
[...]
NOTICE: 'wordpress' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/git/collab-maint/wordpress.git
[...]
In that case, you can use debcheckout to retrieve the VCS repository instead (provided that you have the corresponding VCS installed):
$ debcheckout wordpress
declared git repository at git://git.debian.org/git/collab-maint/wordpress.git
git clone git://git.debian.org/git/collab-maint/wordpress.git wordpress ...
Cloning into wordpress...
Note however that some maintainers use their VCS in a way that s not really compatible with the explanations that I will give below. It s also a good idea to install the package packaging-dev . It s a meta-package depending on the most common tools that are used for Debian packaging work. 2. Do the changes Execute dch --nmu to record the fact that you re working on an update prepared by someone who is not the maintainer (NMU means Non Maintainer Upload). This also ensures that if we build the package, we won t overwrite the original source package that we downloaded, thus making it possible to generate a diff between both versions. 2.1. Modify Debian packaging files Now fire your text editor and do the required changes in the debian sub-directory. You will probably run dch -a multiple times to document each subsequent change. 2.2. Modify upstream files If you have to modify upstream files, the proper way to do it depends on the source package format ( 1.0 vs 3.0 (quilt) vs 3.0 (native) , see the debian/source/format file) and on the presence or not of a patch system (the what-patch can help you identify it). In this explanation, I ll assume that the package is using the recommended format: 3.0 (quilt) . (It also works for 1.0 if quilt is used and if you configured ~/.quiltrc as recommended by /usr/share/doc/quilt/README.source). First you should ensure that all patches have been applied with quilt push -a. If there s no patch yet, you want to create the debian/patches directory (with mkdir debian/patches). 2.2.1. Import a patch If the upstream changes are already in a patch file (say /tmp/patch that you downloaded from the upstream VCS repository) you can import that patch like this:
$ quilt import -P fix-foobar.patch /tmp/patch
Importing patch /tmp/patch (stored as fix-foobar.patch)
$ quilt push
Applying patch fix-foobar.patch
[...]
Now at patch fix-foobar.patch
The -P option allows to select the name of the patch file created in debian/patches/. As you see, the new patch file is recorded in debian/patches/series but not applied by default, we re thus doing it with quilt push. 2.2.1. Create a new patch If the upstream changes that you want to make are not in a patch yet, you should tell quilt that you re going to create one:
$ quilt new fix-foobar.patch
Patch fix-foobar.patch is now on top
Then you must record every file that you re going to modify with a quilt add invocation. quilt then makes a backup of those files so that it can generate the patch later on. If you re going to modify the files with your text editor you can just do quilt edit file-to-modify, it s the same than quilt add file-to-modify followed by sensible-editor file-to-modify.
$ quilt edit foobar.c
File foobar.c added to patch fix-foobar.patch
The last step is tell quilt to generate the patch:
$ quilt refresh
Refreshed patch fix-foobar.patch
3. Test your changes You should build your modified package with debuild -us -uc . You can easily install the resulting package with debi . Verify that everything works as expected. If not, continue your modifications until you re satisfied with the result. 4. Generate a patch and mail it If you followed the instructions, you should have two .dsc files in the parent directory, like this:
$ cd ..
$ ls wordpress_*.dsc
../wordpress_3.0.5+dfsg-1.1.dsc
../wordpress_3.0.5+dfsg-1.dsc
Generating the patch to send to the maintainer is then just a matter of running debdiff:
$ debdiff wordpress_3.0.5+dfsg-1.dsc wordpress_3.0.5+dfsg-1.1.dsc >/tmp/wp-debdiff
You can send the /tmp/wp-debdiff file to the wordpress maintainer. Usually you send it via the bugreport that your update is fixing and you add the patch tag to the report. This can be automated with the nmudiff utility. By default it assumes that you re using mutt but it can also directly feed the resulting mail to sendmail. The default text that nmudiff proposes assumes that you re actually performing an NMU and that the result has been uploaded. If that s not the case, you should edit the text and make it clear that you re just sending a patch. If you have been working in a VCS repository, instead of using debdiff you can simply use the diff feature integrated in your VCS (git diff, svn diff, etc.). But note that with a distributed VCS (like git/bzr/mercurial, unlike svn) you should probably have committed all individual changes in separate changesets. And instead of sending a single patch, you re probably going to send a series of patches (though it might be easier to just upload your branch in a public repository and give the corresponding URL to the maintainer).
Found it useful? Be sure to not miss other packaging tips (or lessons), click here to subscribe to my free newsletter and get new articles by email.

No comment Liked this article? Click here. My blog is Flattr-enabled.

3 July 2011

Andrea Veri: Automatic Gmail s Trash & Spam folders cleanup

Since some time I ve been thinking about a possible way to delete my Gmail s Trash & Spam folders content automatically without having to bother doing it manually every single time I wanted to check my mail and clean it up. (I love keeping everything in place and having my Trash&Spam folders empty as they should be makes me pretty happy) A few years ago when Mutt was my main mail client I had the need to filter my mail through IMAP and while googling around for that I found out a great piece of software: imapfilter. Today while analyzing the above quoted issue I suddenly told myself: Hey, but why don t you use your dear and old friend imapfilter to fulfil your needs? After a few minutes I came up with a small lua script that was doing exactly what I wanted: my Trash&Spam folders are no longer crowded and I finally don t have to delete mails twice! But here they come a few details about my script:
options.timeout = 120
options.subscribe = true
account = IMAP  
 server = 'imap.gmail.com',
 username = 'example@gmail.com',
 password = 'password',
 ssl = 'ssl3'
  
trash = account['[Gmail]/Trash']:is_undeleted()
account['[Gmail]/Trash']:delete_messages(trash)
spam = account['[Gmail]/Spam']:is_unanswered()
account['[Gmail]/Spam']:delete_messages(spam)
The script does two things:
  1. It checks whether a mail is not marked as deleted (moving an e-mail into the Trash does not mark it as to be deleted automatically) already and removes it.
  2. It checks whether a mail on the Spam folder has been answered (I never had to answer a single e-mail contained into my Spam folder) and if not removes it.
Using the above script is really easy (you should run imapfilter on interactive mode first to generate Gmail s certificates, do that before having cron to run the script for you or otherwise it ll just hang), just make sure to have imapfilter installed on your system and then run it through cron every half an hour or less depending on your needs:
crontab -e
*/30 * * * * imapfilter -c /home/user/imapfilter.lua >> /home/user/imapfilter.log
Please also remember to setup appropriate permissions on the config file since it contains your Gmail s password and most of all make sure that your Spam folder is visible through IMAP (this option can be found on the label menu available under your Gmail s settings) otherwise imapfilter will just report an error. Enjoy!

Clint Adams: A very little bit about how herb works

This information is pieced together from no fewer than five informants. As such it is likely to contain inaccuracies. Every year, through the conference registration system, a fork of an old version of Pentabarf, attendees declare that they need sponsorship. They state their cost of travel as well as how much of this amount they can afford to pay themselves. In addition, there is a text field in which one is to answer what you do for Debian and why you request sponsorship. Next a committee is assembled. The head of the committee hand-picks its members. In 2011, twenty-four people were asked if they would like to participate, thirteen people were named to the committee, and twelve of those ended up voting. This committee is named herb , after Herbert Powell. The members of herb each individually rate the applicants on two factors: contribution to Debian and amount requested. Each factor is rated on a five-point scale (-2 to +2). Raters may leave, at their option, a comment regarding the applicant. The raters are able to see previous ratings and comments. In the interest of avoiding the appearance of conflict of interest, each rater will refrain voluntarily from self-rating. Individual raters use exercise judgment during this process.
Contribution to Debian is very subjective, and some raters will vote either on total travel cost, requested amount, the ratio of the two, or something else. At a certain point in time, a weighted average of all the ratings is used to order all the applicants into a list. At the top will be the people ranked most highly. The herb team will then conduct an IRC meeting in which ranking may be arbitrarily changed and two thresholds will be established. The result of this process is three lists of people: queue A, for people who should definitely get sponsorship; queue B, for people that might get sponsorship if people from queue A cancel or other circumstances arise; and queue C, which is not really a queue, but a rejection pile. In some cases, the structure of these queues are decided before definite budget numbers are known. Applicants are emailed to let them know which queues they are in, and their rankings within those queues.

Benjamin Drung: Symptoms of a dying graphic card

All started in December last year. I saw a red and a green pixel on my monitor. Moving the window containing the wrong-colors pixels corrected them. The wrong-colored pixels reappear from time to time in increased quantity. Then a second symptom appeared: The screen went black and came back one second later. This was triggered by moving a window and scrolling. I wasn t sure what the reason was. Either it was a bug somewhere in the X stack or some hardware was dying. dmesg showed multiple problems with the radeon driver:
[283808.667454] radeon 0000:01:00.0: ffff88021f815c00 unpin not necessary
[283808.667820] radeon 0000:01:00.0: GPU softreset
[283808.667823] radeon 0000:01:00.0: R_008010_GRBM_STATUS=0xE57024A4
[283808.667825] radeon 0000:01:00.0: R_008014_GRBM_STATUS2=0 00330302
[283808.667826] radeon 0000:01:00.0: R_000E50_SRBM_STATUS=0x200000C0
[283808.667832] radeon 0000:01:00.0: R_008020_GRBM_SOFT_RESET=0x00007FEE
[283808.682844] radeon 0000:01:00.0: R_008020_GRBM_SOFT_RESET=0 00000001
[283808.698840] radeon 0000:01:00.0: R_008010_GRBM_STATUS=0 00003028
[283808.698843] radeon 0000:01:00.0: R_008014_GRBM_STATUS2=0 00000002
[283808.698845] radeon 0000:01:00.0: R_000E50_SRBM_STATUS=0x200000C0
[283808.699845] radeon 0000:01:00.0: GPU reset succeed
[283808.717570] [drm] Clocks initialized !
[283808.765829] [drm] ring test succeeded in 0 usecs
[283808.765838] [drm] ib test succeeded in 1 usecs
[283808.765840] [drm] Enabling audio support
[283812.521265] radeon 0000:01:00.0: GPU lockup CP stall for more than 1000msec
[283812.521269] [ cut here ]
[283812.521294] WARNING: at /build/buildd/linux-2.6.35/drivers/gpu/drm/radeon/radeon_fence.c:235 radeon_fence_wait+0 365/0x3d0 [radeon]()
[283812.521297] Hardware name:
[283812.521299] GPU lockup (waiting for 0x00AE670A last fence id 0x00AE6705)
[283812.521301] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs nls_utf8 udf ip6table_filter ip6_tables binfmt_misc ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_intel kvm parport_pc ppdev snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_emu10k1 snd_ac97_codec ac97_bus snd_pcm snd_page_alloc snd_util_mem snd_hwdep snd_seq_midi radeon snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device ttm snd pl2303 drm_kms_helper coretemp usbserial joydev psmouse soundcore drm serio_raw intel_agp i2c_algo_bit lp parport hid_cherry usbhid hid firewire_ohci firewire_core usb_storage crc_itu_t e1000e ahci libahci pata_marvell
[283812.521360] Pid: 2194, comm: compiz Tainted: G W 2.6.35-24-generic #42-Ubuntu
[283812.521362] Call Trace:
[283812.521370] [<ffffffff8106089f>] warn_slowpath_common+0x7f/0xc0
[283812.521374] [<ffffffff81060996>] warn_slowpath_fmt+0 46/0 50
[283812.521390] [<ffffffffa01bd775>] radeon_fence_wait+0 365/0x3d0 [radeon]
[283812.521394] [<ffffffff8107f730>] ? autoremove_wake_function+0 0/0 40
[283812.521410] [<ffffffffa01bdf71>] radeon_sync_obj_wait+0 11/0 20 [radeon]
[283812.521418] [<ffffffffa01751a3>] ttm_bo_wait+0 103/0x1c0 [ttm]
[283812.521435] [<ffffffffa01d4e1a>] radeon_gem_wait_idle_ioctl+0x9a/0 150 [radeon]
[283812.521447] [<ffffffffa010f433>] drm_ioctl+0 463/0 520 [drm]
[283812.521465] [<ffffffffa01d4d80>] ? radeon_gem_wait_idle_ioctl+0 0/0 150 [radeon]
[283812.521470] [<ffffffff81162f0d>] vfs_ioctl+0x3d/0xd0
[283812.521473] [<ffffffff811637e1>] do_vfs_ioctl+0 81/0 340
[283812.521477] [<ffffffff811535f1>] ? vfs_read+0 181/0x1a0
[283812.521480] [<ffffffff81163b21>] sys_ioctl+0 81/0xa0
[283812.521484] [<ffffffff8100a0f2>] system_call_fastpath+0 16/0x1b
[283812.521487] [ end trace 6d5e03bab743abfa ]
[283812.521493] [drm] Disabling audio support
[283812.525575] [drm:radeon_ib_schedule] *ERROR* radeon: couldn t schedule IB(10).
[283812.525579] [drm:radeon_cs_ioctl] *ERROR* Faild to schedule IB !
[283812.527021] [drm:radeon_ib_schedule] *ERROR* radeon: couldn t schedule IB(11).
[283812.527024] [drm:radeon_cs_ioctl] *ERROR* Faild to schedule IB !
[283812.527921] [drm:radeon_ib_schedule] *ERROR* radeon: couldn t schedule IB(12).
[283812.527923] [drm:radeon_cs_ioctl] *ERROR* Faild to schedule IB !
The time between going black reduces every time. The system wasn t usable any more in the end. After testing that the screen was going blank on other systems like a live CD of Ubuntu and a not noteworthy proprietary system, it was clear that some hardware component was dying. My first assumption was verified after replacing it: My graphic card, a Radeon HD 4670, died. This was three weeks ago and around 25 month after I bought the graphics card. So I didn t have to worry if replacing the fan voided the guarantee, because the guarantee lasts only 24 month. Conclusion:
  1. Not every error is a software bug.
  2. The hardware component with the highest failure rate is the graphics card, followed by the motherboard. Four graphics cards died last year (in four different systems owned by four different people, in three different households).
  3. Not every error is a software bug.

Mesutcan Kurt: Python Multi-Build for Python Extensions Packaging - Report 3

<meta content="text/html; charset=utf-8" equiv="content-type">
Hello,

This is the third report of my GSoC project for Debian. During last two weeks
I've done most of things about the main tool. Before that weeks I have been
working on project's API. API is almost finished. Still I find missing/wrong
parts during the coding and changing the API design.

Like I said, main tool's coding process is nearly finished. For now It is doing
its job for building Python3.X packages. You can find the source code at [1].

I also coded an example plugin (actually only return values) and plugins/__init__.py
file which contains like Base class, exception classes and plugin listing method.

When I started coding I realized that I need extra parameters. I've added --install-dir
parameter. You can specify the install directory if there is no install directory the tools
pretends that the install directory is "./debian/tmp". After that it checks whether if the
directory exists. If not it creates the directory.

I've changed py3build.py a bit. First, I've thought for every process I invoke py3build's
methods but they are redundant. I've removed them. Now they are called in main
method of py3build.

I use logging module for printing messages, errors etc.

get_build_system() method retieves list from plugins/__init__.py file. In that file there is
scan_for_plugins() method. It finds all plugins in plugins/ directory which files match for
"[a-zA-Z].py". It keeps the list in plugins_list variable and returns that. When I'm writing
scan_for_plugins() method I've read codes (f.e. [2]) about that and decided to name all
plugins' class name as *BuildSystem*. Now, detecting method works with that way.
(all plugins' class' are called BuildSystem) And I've changed base class name to
BuildSystemBase.

get_build_system() method is used by determine_build_system() method. If there is given
build system (by parameter) it tries to use that. If that build system is not usable then give
error and exits. If there is no given build system. It invokes get_build_system() and iterates
that methods return value which is a list. When iterating It create an instance of that build
system class (if can not create then it gives warning and skips that build system plugin)
and invokes detect method and append plugin name and detect method's return value to
the bs_list which contains plugin list which are available. And then sorts it. If there is error
while sorting list or there is no usable plugin then give error and exits. If there is multiple
highest detect values then lists that plugins and exists. (maintainer should select one of
these plugins to build extension.)

In main method, after getting arguments, it checks for verbose mode (or 'DH_VERBOSE'
environment variable) and sets log level (logging.DEBUG or logging.WARN)

If there is no given build_dir then it pretends it is "./".

After that It try to determine build system. If can not then prints error and exits.

And finally it starts to build extension. Firstly it checks if there is given version list. If not,
it uses default values and iterates build loop with that values.

First cleans, configures, builds (if there is no --test-only arg) and tests (if there is --test-only
arg or there is not --no-tests argument)

If there is error (raises plugin.Error) while invoking that method then prints error and exits.

In verbose mode tool prints every process. (like "configuring for PythonX.Y" etc.)

I've started writing a plugin for detecting *distutils*. For now, I only wrote its detect method. It
returns between 0-1.0 floating point value. If everything is OK it returns 1.0. Other methods does
nothing, only returns True but this week I'll finish this plugin and work for others.

For detecting distutils I've used the code at [3]. First I try to import that module if not, I've copied
required methods to plugins/_packaging.py file, I use them.

I don't use is_distutils() method directly. For give probability I invoke what it invokes and give them
a number. (f.e. _has_setup_py has 0.5, _has_pkg_info has 0.25 and _has_distutil_text has 0.25.
But if _has_setup_py returns False then detect method doesn't look other methods since they
require setup.py.)

For now that's all. See you at next time.

[1] git://git.debian.org/~<wbr>mesutcan-guest/py3multibuild.<wbr>git
[2] http://svn.berlios.de/<wbr>viewvc/griffith/trunk/lib/<wbr>plugins/extensions/
[3] http://hg.python.org/<wbr>cpython/file/a7e00934baf1/Lib/<wbr>packaging/util.py

--
Mesutcan Kurt

Vincent Bernat: Dealing with missing locales on remote hosts

On my system, I happen to set LANG to fr_FR.utf8 and LC_MESSAGES to en_US.utf8. This means that applications should follow French cultural conventions for most things except for messages which should be displayed in US English. On my own system, /etc/locale.gen contains those two locales. However, when I connect to some random remote system, they may be unavailable. Most applications will fallback silently to C. However, Perl can be quite noisy:
$ perl -e 'print "Hello\n";'
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LC_MESSAGES = "en_US.utf8",
        LANG = "fr_FR.utf8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Hello
This is an incredibly annoying message. I don t understand why we still need to bear it. Perl documentation explains how to get rid of this message. The simplest way is to set PERL_BADLANG environment variable to 0. Problem solved.
$ PERL_BADLANG=0 perl -e 'print "Hello\n";'
Hello
Well, no. Keep reading. When you connect to a remote host with ssh, all your environment variables are thrown away, except those defined in AcceptEnv directive in /etc/ssh/sshd_config file on the remote host. On Debian, this directive defaults to LANG LC_*. Since I cannot install my favourite locales on all hosts or fiddle environment variables on them, I use something like this in my .zshrc:
ssh()  
  [ -t 1 ] && echo -ne "\033]0;$@\007"
  LANG=C LC_MESSAGES=C =ssh "$@"
 
Unfortunately, on systems where my locales are present, I still fallback to the crappy C locale. I could just unset LANG and LC_MESSAGES to allow fallback to the proper default locale but if the remote system starts to speak Spanish, I will not be pleased. Moreover, the behaviour of C locale (also known as POSIX locale) is undefined with characters not in the portable character set. From IEEE Std 1003.1:
Conforming systems shall provide a POSIX locale, also known as the C locale. The behavior of standard utilities and functions in the POSIX locale shall be as if the locale was defined via the localedef utility with input data from the POSIX locale tables in Locale Definition. The tables in Locale Definition describe the characteristics and behavior of the POSIX locale for data consisting entirely of characters from the portable character set and the control character set. For other characters, the behavior is unspecified. For C-language programs, the POSIX locale shall be the default locale when the setlocale() function is not called. The POSIX locale can be specified by assigning to the appropriate environment variables the values C or POSIX.
There is no way to detect the locales installed on the remote system unless you connect to it. Starting a connection just to check the locale is too expensive. On my own systems, I use another snippet in .zshrc to reconfigure the locales to my favourite ones if they are available:
export LANG=C
export LC_MESSAGES=C
(( $+commands[locale] )) && function  
    local locales
    locales=( "LANG fr_FR.utf8 en_US.utf8 C.UTF-8 C" \
              "LC_MESSAGES en_US.utf8 fr_FR.utf8 C.UTF-8 C" )
    for locale in $locales; do
        for l in $=locale[(w)2,-1]; do
            if locale -a   grep -qx $l; then
                export $locale[(w)1]=$l
                break
            fi
        done
    done
  2> /dev/null
This is an interesting snippet which uses some ZSH features. Feel free to tell me if you have found a better way to handle this problem!

Dominique Dumont: Generate a configuration editor from a config template file with Perl: lcdproc example

Hello Creating a configuration model for Config::Model can be a boring task: When a configuration system has a lot of parameters, a lot of details need to be specified to have a complete configuration validation tool. So, all tricks are permitted to make this task more bearable. Let s see how this can be applied to lcdproc. This project requires a configuration file that supports more than 40 drivers. Each driver has it own set of parameters for a total of about 250 parameters. That s a lot to specify.. Fortunately, LCDd.conf template file is written in a way which makes it relatively easy to parse and get all required information to build a model. All drivers are listed in this template file, most parameters have default values and legal values are written in comments in a uniform way. Hence this file (and comments) can be parsed to retrieve information required for the model. This blog will describe how this task can be implemented in Perl using Config::Model modules. The code will be slightly trimmed down to keep this blog (relatively) short. All initialisation code and error checks are removed. You can get the whole script on sourceforge. The main steps are:
  1. prepare a parser for LCDd.conf template
  2. parse LCDd.conf template
  3. prepare a LCDd model skeleton
  4. translate LCDd.conf template information in a format suitable to create a model.
  5. Write the resulting LCDd model
First, here s the necessary incantation to start properly a Perl program:
use strict;
use warnings;
Then load all required libraries. Their use will be described later:
use Config::Model;
use Config::Model::Itself 1.225;
use Config::Model::Backend::IniFile;
use 5.10.0;
use IO::File;
use IO::String;
Perform some initialisation to avoid upsetting the logging system:
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init($WARN);
Let s also initialize a model object that will be very useful later:
my $model = Config::Model->new();
Now we can really start the fun part. Step 1: Create a parser for LCDd.conf (INI file)
Problem: LCDd.conf comments must also be analysed to get more semantic information. The comments must be retrieved and associated with the INI classes and parameters found in this file. Fortunately, Config::Model::Backend::IniFile is able to associate comments with the relevant parameters. But Config::Model::Backend::IniFile must store its values in a configuration tree. So let s create a model suitable for LCDd.conf template that accepts any INI class and any INI parameter Dummy::Class is used to store any parameter found in an INI class. The accept keyword with .* regular expression means that any INI parameter will be accepted in each INI class.
$model->create_config_class(
  name   => 'Dummy::Class',
  accept => [
    '.*' =>  
       type => 'leaf',
       value_type => 'uniline'
        ,
    ],
);
Now the Dummy model must provide a configuration class to hold all INI classes:
$model->create_config_class(
    name   => 'Dummy',
    accept => [
       '.*' =>  
           type => 'node',
           config_class_name => 'Dummy::Class'
        ,
    ],
);
Note that an INI backend could be specified directly here. But, some useful parameters are commented out in LCD.conf. Some some processing is required to be able to create a model with these commented parameters. See below for this processing. Step 2: Parse LCDd.conf Now the dummy configuration class is created. Let s create a configuration tree from this dummy class to store the data from LCDd.conf
my $dummy = $model->instance(
    instance_name   => 'dummy',
    root_class_name => 'Dummy',
)-> config_root;
Now, read LCDd.conf:
my $lcd_file
  = IO::File->new('examples/lcdproc/LCDd.conf');
my @lines    = $lcd_file->getlines;
And perform the pre-processing on LCDd.conf mentioned above. Just un-commenting commented parameters is required:
foreach (@lines)   s/^#(\w+=)/$1/  
Store the pre-processed LCDd.conf in a IO::Handle usable by INI backend
my $ioh = IO::String->new( join( '', @lines ) );
Create an INI backend to read the data from pre-processed LCDd.conf and store it in the configuration tree:
my $ini_backend = Config::Model::Backend::IniFile
  ->new( node => $dummy );
$ini_backend->read( io_handle => $ioh );
Step 3: Prepare LCDd configuration model Your attention please. You may remember that we invoked instance on object to create the dummy configuration tree to hold LCDd.conf data. The instance method used the Dummy configuration class. Now, we must invoke instance again on object. But this time, instance will use Config::Model::Itself. This model is Config::Model s way of eating its own dog food. The configuration tree created there will be able to hold LCDd model. Hence the root of this special configuration tree will be called meta_root to avoid confusion with the other configuration root defined above (for the Dummy model). Still with me? Good, let s create this meta tree that will contain LCDd model:
my $meta_root = $model->instance(
    root_class_name => 'Itself::Model',
    instance_name   => 'meta_model',
) -> config_root;
Create the main LCDd configuration class and store the first comment from LCDd.conf as class description:
my $first_comment = $dummy->annotation ;
$meta_root
  ->grab("class:LCDd class_description")
  ->store( $first_comment );
Append my own text in this description:
my $extra_description =
    "Model information extracted from template /etc/LCDd.conf"
  . "\n\n=head1 BUGS\n\nThis model does not support to load "
  . "several drivers. Loading several drivers is probably a "
  . "marginal case. Please complain to the author if this "
  . "assumption is false";
$meta_root->load(
     qq!class:LCDd
        class_description.="\n\n$extra_description"!
);
Add legal stuff in the model:
$meta_root->load( qq!
  class:LCDd
    copyright:0="2011, Dominique Dumont"
    copyright:1="1999-2011, William Ferrell and others"
    license="GPL-2"
!
);
Add INI backend so configuration object created from LCDd model will be able to read user s INI files:
$meta_root->load( qq!
  class:LCDd
    read_config:0
      backend=ini_file
      config_dir="/etc"
      file="LCDd.conf"
!
);
Note that all the load calls above could be done as one call. They were split for better clarity. Step 4:Extracting LCDd.conf information Now that all information from LCDd.conf template file is stored in $dummy configuration tree, it s time to use this information to enhance the skeleton of LCDd model created above. Let s first find all the INI classes defined in LCDd.conf:
my @ini_classes = $dummy->get_element_name;
Now before actually mining LCDd.conf information, we must prepare several subs to handle them. This is done using a dispatch table:
my %dispatch;
First create the default entry which will be used for most parameters. This subs is passed: the INI class name, the INI parameter name, the comment attached to the parameter, the INI value, and an optional value type. It will return a string describing a model for simple elements. For instance: class:"LCDd::MD8800" element:Device type=leaf value_type=uniline default="/dev/ttyS1" description="device to use [default: /dev/ttyS1]" Here s the subroutine invoked by default. Let s create a an anonymous subroutine and store it in the dispatch table:
$dispatch _default_  = sub  
    my ( $ini_class, $ini_param, $ini_note,
         $ini_v, $value_type ) = @_;
    $value_type  = 'uniline';
Now, prepare a string that will be used later to create the ini_class model. This string uses Config::Model s serialisation format described in Config::Model::Loader:
    my $load = qq!class:"$ini_class"
                  element:$ini_param type=leaf !;
Get semantic information from LCDd.conf template comments (written between square brackets):
    my $info = $ini_note =~ /\[(.*)\]/ ? $1 : '';
    $info =~ s/\s+//g;
Use this semantic information to better specify the parameter:
    $load .=
        $info =~ /legal:(\d+)-(\d+)/
           ? " value_type=integer min=$1 max=$2"
      : $info =~ /legal:([\w\,]+)/
           ? " value_type=enum choice=$1"
      :    " value_type=$value_type";
    if ( $info =~ /default:(\w+)/ )  
        # specify upstream default value
        # if it was found in the comment
        $load .= qq! upstream_default="$1"!
           if length($1);
     
    else  
        # or use the value found in INI
        # file as default
        $ini_v =~ s/^"//g;
        $ini_v =~ s/"$//g;
        $load .= qq! default="$ini_v"!
           if length($ini_v);
     
The model (in serialised format) is ready, let s return it and exit the subroutine:
    return $load;
 ;
Now let s take care of the special cases. This one deals with Driver parameter found in INI [server] class. This sub will create a string for an enum element. The possible values of the enum are extracted from the comment found in the LCDd.conf file:
$dispatch "LCDd::server" Driver  = sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    my $load = qq!class:"$class"
                    element:$elt
                      type=leaf
                      value_type=enum
    !;
    my @drivers = split /\W+/, $info;
    while ( @drivers
            and ( shift @drivers ) !~ /supported/ )  
      # nothing, the work is done by shift operator
      # in loop test to skip all words in $info until
      # 'The following drivers are supported:'.
      # ok, that's a hack
     
    $load .= 'choice='
          . join( ',', @drivers ) . ' ';
    return $load;
 ;
This sub invokes the default sub and add a small feature to the model to ensure that DriverPath ends with / . This is ensured by the match="/$" snippet:
$dispatch "LCDd::server" DriverPath  = sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    return   $dispatch _default_ ->(@_)
           . q! match="/$" !;
 ;
Likewise, this sub invokes the default sub with the optional 4th parameter that forces the value_type of the element. In this case, we enforce an integer element:
 $dispatch "LCDd::server" WaitTime 
 = $dispatch "LCDd::server" ReportLevel 
 = $dispatch "LCDd::server" Port 
 = sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    return $dispatch _default_ ->( @_, 'integer' );
   ;
This sub returns a string for a list element with default values:
  $dispatch "LCDd::server" GoodBye 
= $dispatch "LCDd::server" Hello 
= sub  
    my ( $class, $elt, $info, $ini_v ) = @_;
    my $ret = qq( class:"$class"
                    element:$elt
                      type=list
                ) ;
    $ret .= 'cargo type=leaf
             value_type=uniline - ' ;
    $ret .= 'default_with_init:0="\" '.$elt.'\"" ' ;
    $ret .= 'default_with_init:1="\" LCDproc!\""';
    return $ret ;
 ;
Now that the dispatch table is ready, let s really mine LCDd.conf information. First handle all INI classes in the foreach loop:
foreach my $ini_class (@ini_classes)  
Let s retrieve parameters for this INI class from the LCDd.conf template:
  my $ini_obj = $dummy->grab($ini_class);
This will be the name of the lcdproc driver configuration class (e.g. LCDd::imonlcd):
  my $config_class   = "LCDd::$ini_class";
The configuration class will also be created in a serialised format. Let s create config class in case there s no parameter in INI file:
  $meta_root->load(qq!class:"LCDd::$ini_class"!);
Now, loop over all INI parameters and create LCDd::$ini_class elements:
  my @all_ini_params = $ini_obj->get_element_name ;
  foreach my $ini_param ( @all_ini_params )  
Retrieve the value provided in template LCDd.conf:
    my $ini_v = $ini_obj->grab_value($ini_param);
Retrieve the comment attached to the value provided in template LCDd.conf:
    my $ini_comment
      = $ini_obj->grab($ini_param)->annotation;
    # escape embedded quotes
    $ini_comment =~ s/"/\\"/g;
Retrieve the correct sub from dispatch table:
    my $sub = $dispatch $config_class $ini_param 
              $dispatch _default_ ;
Run the sub to get the model string:
    my $model_spec
      = $sub->( $config_class, $ini_param,
                   $ini_comment, $ini_v );
Attach the comment from LCDd.conf as class description:
    $model_spec .= qq! description="$ini_comment"!
      if length($ini_comment);
Then, last but not least, load class specification in the model, and end the loop aroung INI parameters:
    $meta_root->load($model_spec);
   
Now create a an $ini_class element in LCDd class (to link LCDd class and LCDd::$ini_class):
  my $driver_class_spec = qq!
    class:LCDd
      element:$ini_class
  ! ;
Since server and menu are not drivers, they have a special treatment:
  if (   $ini_class eq 'server'
      or $ini_class eq 'menu' )  
    $driver_class_spec .= qq!
      type=node
      config_class_name="LCDd::$ini_class"
    ! ;
   
Here s the treatment for LCDd drivers. Setup the model with a warped node so that a driver configuration class is shown only if the driver was selected in the [server] class:
  else  
    $driver_class_spec .= qq!
      type=warped_node
      config_class_name="LCDd::$ini_class"
      level=hidden
      follow:selected="- server Driver"
      rules:"\$selected eq '$ini_class'"
        level=normal
    !;
   
Now we can load the LCDd driver model in meta root:
  $meta_root->load($driver_class_spec);
 
Step 5: save LCDd model Still with me ? Good. Here s the last part, now that the model was created in memory, let s save it in files. Config::Model::Itself constructor returns an object to read or write the data:
my $rw_obj = Config::Model::Itself
     ->new( model_object => $meta_root );
say "Writing all models in file (please wait)";
$rw_obj->write_all(
    model_dir => 'lib/Config/Model/models/'
);
That s it. Once this script is run, the directory lib/Config/Model/models/ contains a bunch of files: You can see the resulting editor in my first blog on lcdproc. Conclusion and future steps (with you?) As mentioned at the end of this blog, the result of this model generation process is far from perfect, because the information retrieved from LCDd.conf template are not complete for this task (which is normal since these comments were targeted for users). Now, I d really like to discuss with project developers to improve configuration template to generate better models. For project owner, such an improvement would give them: Please leave a comment on this blog (or send a message to config-model-users at lists.sourceforge.net) if you are interested in configuration tool generation for a project you contribute to. All the best

Rapha&#235;l Hertzog: My Debian activities in June 2011

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (195 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Dropbox for Debian This is not free software but Dropbox is very popular and they did only provide an Ubuntu package that did not work on Debian. So I created an official package. I have been in touch with Dropbox developers and they have been very helpful. They ll shortly release a signature mechanism (with GPG) so that we can further improve the package by verifying the origin of the downloaded binaries. SAT Britney At the start of the month, I continued my work on the britney reimplementation (the software that creates testing out of unstable) but I quickly stalled it because the release managers asked the feedback of Stefano Zacchiroli and Ralf Treinen (who have extensive knowledge on the topic with their research work on Mancoosi) and I did not want to invest further work in case they would identify a major flow the feedback came only very late this month and while it was somewhat negative, I still think it s worth pursuing the effort for a bit longer. Converted ftplib to multiarch While dpkg still doesn t support multiarch (no news from Guillem and no visible sign of progress :-( ), unstable got all the remaining bits allowing us to convert libraries to multiarch (see the announce). As soon as the required libc6 landed in unstable, I looked into converting the only library package that I maintain. I had no major problem but I still identified 2 issues in Lintian (filed as #630164 and quickly fixed by Niels Thykier). build-arch / build-indep support For the 42th time in the last 10 years, the idea of using build-arch/build-indep targets in the rules file has surfaced again. I had already decided some time ago that I would accept a patch implementing a new field Build-Features to enable dpkg-buildpackage to use those targets and this time Bill Allombert completed such a patch so I merged it. The technical committee also decided that it would take a final decision on this topic (see #629385). Roger Leigh provided useful input by doing an archive-wide rebuild with the various solutions suggested. Given that the majority would like to make the target mandatory at some point in the future, I provided the dpkg patch for my preferred solution. We would use auto-detection as a temporary measure until all packages have been converted to have the targets. The technical committee has not yet taken any decision even though the discussion stalled since the 12th of June. But that s usual with that body. I m sure it will be solved during Debconf. ;-) Misc dpkg work Hamster applet update Hamster-applet is a GNOME application which did not have a 3.0 release, but it had a development release (2.91.x). I checked out whether it was possible to package this version for experimental and have the applet work with the GNOME fallback mode. Apparently not, the code was not yet updated to be compatible with the newer panel. Instead I uploaded the latest stable version (2.32.1) to unstable. It has some nice improvements in the standalone version (and the name of the executable changed). For usage with GNOME 3, I have created a custom shortcut to start it quickly (with gconf-editor set /apps/metacity/global_keybindings/run_command_1 to <Mod4>t and /apps/metacity/keybinding_commands/command_1 to hamster-time-tracker because the GNOME 3 control panel does not seem to work to set custom keybindings currently). Translated my professional website into English While I m grateful for all the people who are supporting my work, I m still far from my goal to have one third of my time funded through donations and sales of products on this blog. So I decided to also bring more visibility to my company and in particular to its Debian-related service offering. It was only available in French up to now so I translated it and expanded it a bit. My support page on this blog now also links to my company s website. If your company needs help to create Debian packages, or needs Debian technical support by email, you just found the right partner. :-) BTW, I have discounted prices for individuals and non-profits who would like to benefit from my help to create Debian packages. The Debian Administrator s Handbook This is the title of the upcoming translation of my book. The project now has a dedicated website: debian-handbook.info. You can subscribe to its RSS feed to keep up with the latest news. The full table of contents is online along with a FAQ. I m actively looking for partners to help me promote the fundraising once it goes live. If you can reach a large set of readers interested by a good Debian book, get in touch with me to join the affiliate program. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Erich Schubert: Google vs. Facebook

Let the games begin. It looks like Google and Facebook are going to fight it out.
Given the spectacular failures of Wave and Buzz, people are of course skeptic about the success of Google Plus. However, I'm rather confident it is going to stay. Here are some things I like about it:
Some people think that Google will not stand a chance against social giant Facebook. But after all, Google has more users - and they have tons of services people like to use. So when Google Maps has Plus integration, will the users use it to chat about where to meet, or will they go the long way and post the map URL on Facebook, without the option of updating it collaboratively?
Googles position is much stronger than many people believe, once you think about integration possibilities. The current Plus is just a fragment, the missing puzzle piece connecting the other apps. But imagine that Google now Plus-connects its various services: YouTube, Maps, Mail, Talk, ... - for them, this is just a matter of some engineering. I guess probably half of these are already in internal testing. And Facebook just can't keep up there. Sure, they do have Facebook Video. But actually, most people prefer YouTube. And while Google can integrate plus all the way there, Facebook cannot. And while Google is the master of search, Facebook is particularly weak there - they can't even properly search their own data. Google however will at some point offer an "find things that interest me" button; Sparks is just the beginning where you have to manually define your interests (which will probably remain an option due to privacy concerns!); it is way too static right now.
So essentially, Google doesn't need to copy Facebook. They just need to do what is obvious on their own products, and Facebook will have a hard time keeping up.
Plus, in my opinion, Google got the timing just right. Users aren't too happy with Facebook these days, there is just no big alternative around anymore; their friends are on Facebook and not on some other social network. Facebook doesn't seem to evolve much anymore. The mail functionality opened more security holes (apparently you can post to a group with a fake user name when you spoof the senders email address) than it contributed to functionality and usefulness. Privacy is still not in line with all countries such as Germany; but Facebook keeps telling those users essentially that they don't care. Spam and fraud still reappears every month following the same pattern again and again (Clickjacking). The search function of Facebook is still usually described as "useless" ... People waste time in games and annoy their friends with random game invitations and posts. Facebook should better make a major move now, too. More than a demo of Skype integration. But whenever Facebook changed, their users complained ...

Of course, Google+ still has a long way to go, too. There are still many things missing here, too. For example groups and events. I figure Google is already testing them in "dogfood", and they'll actually come out within the month. With groups I do not refer to the existing Google product, but to what would be "public circles" that you need to join and that are accessible to all the circle members instead of just the creator. And events are also a key function; probably one of the most used on Facebook. These may require much more careful design to integrate well with Calendar. But given the visual update of Calender these days, this may just be around the corner, too.

Gunnar Wolf: DebConf13 at home? Why not?

We are few weeks away from the start of DebConf11. Excitement runs high in Debian-land. The two most worthy weeks of the year, every year, loom close. Our Bosnian friends have done a great job of finding and defending an amazing proposal, and are now facing the hard work and permanent adrenaline levels of being in charge of the closest I have seen to a herd of (well-behaved but wild and untamable) cats. I have organized DebConf in my country. It was hellish, but at the same time, it's one of my most cherished experiences. And I'm sure the same will be said by the leaders of each successive bid It is one of the most rewarding experiences you can imagine. Next year, DebConf will be held in tropical Managua, Nicaragua. But, where will we meet in 2013? Well, that depends on you, my dear reader! Do you want to work your ass off for Debian and have utter fun? Do you want to show and share your country with this huge family of developers? Start thinking about pushing for a DebConf13 bid! Do you have to be at Banja Luka to propose your bid? No. You can proxy via somebody I'd suggest to do it via somebody who knows the location you are suggesting, but basically, choose a friend that you trust that trusts you. Of course, you can participate in the presentation session via IRC. Do you have to be a Debian Developer to propose a bid? No. For DebConf9, none of the C ceres guys was a DD; for DebConf10, some of the people most involved from the local New Yorkers were not DDs. For DC11, none of our dear and overworked hosts in Bosnia are DDs. And for DC12, the Nicaraguan crew is also made from people interested in getting closer to the Debian project, but not DDs. Do you have to decide now? No. This is just a call for a first presentation, but the decision regarding DC13 will be taken probably around March 2012. However, giving a nice presentation at DebConf helps a lot, gives you visibility, and will get the ball rolling. Is there a geographical bias? Slight. So far, and since the second DebConf, we have kept the tradition not to repeat continents on two successive DebConfs. This is not a hard condition, however! What do you need to start thinking about? Go visit our prospective location checklist at http://wiki.debconf.org/wiki/LocationCheckList. You can also look at what other teams have historically presented. Finally, I just learnt about the existence of http://wiki.debconf.org/wiki/DebConf13 Register there, even if you are just in the early phases of finding data. We will be holding a DebConf13 bids presentation session, most probably (the schedule is close to being presented officially!) on 30-07-2011, at 17:00.

2 July 2011

Martin Meredith: Google+: Part 1 Circles

So, I haven t read the planets in a couple of days but I m going to guess that, like twitter, a lot of people are going to be raving about Google+. For those of you who haven t heard about Google+, it s a new Social Networking site from Google and it brings a new take to the world of social networking. In real life, we communicate differently to the different people in our social contexts. For example, most people will converse differently to their work colleagues than they will to their family. They ll converse differently with their drinking buddies than they will with their Family. There are things that you want only certain people to know, and there are aspects of your personality that you only want to show to certain people. I think giving an example here might be better. This is an example of a friend of mine, who we ll call Fred. Fred is a teacher. Fred also likes to go out of a weekend and party till dawn, while consuming lots of tequila. I, as Fred s friend, am on a night out of drunken debauchery, and I ve my camera with me, and decide to post lots of pictures online of the night s revelry. The next day, one of Fred s colleagues logs into a Social Networking site, and sees said pictures in Fred s profile. Now things aren t looking so bright for Fred. He no longer gives across that professional image that he should do at work. Some may say that this is Fred s fault for going out and doing these things, but everyone s entitled to a personal life, right? At the moment, on Fred s social networking site, it s an all or nothing option. If he allows you access to his profile, he allows you access to see anything and everything that might show up on there. (It s a bit more complicated than this, but let s keep it simple for now). Every time Fred has someone add him as a contact he has to make that decision Do I want this person to be able to see everything I may or may not get up to . Put simply, how many of you have refused someone to add you on the Social Networking site du jour because what you show on their may be harmful to you, or give over an impression that isn t what you want to give across. Enter Circles. Circles brings to the social networking scene the way we interact with people in real life. We show a different aspect of our personality to different people we interact with. I m a bit of an organisational freak, and I know that there are different groups of people that either for the fact of portraying a persona, or not annoying everyone else, I might want to push in different ways. For example, I might want to send out a BBQ at my house, bring lots of alcohol message to all my friends, but don t really want my co-workers coming along and making it so I have to be on my best behaviour. Below is a screenshot of my (current) circles. As you can see I have a fair few, and only a relative few people. Now, let s say that I wanted to send out that BBQ message As you can see I ve the option to send this to the 5 people that are in my Friends group (it s currently mostly geeks on there there are only a few of my real life friends on there so far!) Now with only a few clicks, I ve managed to send out a message to just those few people that I want to :) Over the next few days, I ll write a bit more about Google+, but I thought I d start with an introduction into the Circles feature. I ll be giving a bit of feedback about what I like, what I dislike, and also making a few suggestions for what I d like to see happen in the future (and probably use the word siloing a fair bit!) For now, toodles.

Next.

Previous.