We released Rcpp 0.10.0 earlier today. This post will just provide a simple example for one of the smaller new features -- the new namespace for functions from Rmath.h -- and illustrate one of the key features (Rcpp attributes) in passing. R, as a statistical language and environment, has very well written and tested statistical distribution functions providing probability density, cumulative distribution, quantiles and random number draws for dozens of common and not so common distribution functions. This code is used inside R, and available for use from standalone C or C++ programs via the standalone R math library which Debian / Ubuntu have as a package r-mathlib (and which can be built from R sources). User sometimes write code against this interface, and then want to combine the code with other code, possibly even with Rcpp. We allowed for this, but it required a bit of an ugly interface. R provides a C interface; these have no namespaces. Identifiers can clash, and to be safe one can enable a generic prefix Rf_. So functions which could clash such as length or error become Rf_length and Rf_error and are less likely to conflict with symbols from other libraries. Unfortunately, the side-effect is that calling, say, the probability distribution function for the Normal distribution becomes Rf_pnorm5() (with the 5 denoting the five parameters: quantile, mean, std.deviation, lowerTail, logValue). Not pretty, and not obvious. So one of the things we added was another layer of indirection by adding a namespace R with a bunch of inline'd wrapper functions (as well as several handful of unit tests to make sure we avoided typos and argument transposition and what not). The short example below shows this for a simple function taking a vector, and returning its pnorm computed three different ways:

#include <Rcpp.h>
// [[Rcpp::export]]
Rcpp::DataFrame mypnorm(Rcpp::NumericVector x)  
    int n = x.size();
    Rcpp::NumericVector y1(n), y2(n), y3(n);
    for (int i=0; i<n; i++)  
        // the way we used to do this
        y1[i] = ::Rf_pnorm5(x[i], 0.0, 1.0, 1, 0);
        // the way we can do it now
        y2[i] = R::pnorm(x[i], 0.0, 1.0, 1, 0);
     
    // or using Rcpp sugar in one go
    y3 = Rcpp::pnorm(x);
    return Rcpp::DataFrame::create(Rcpp::Named("Rold")  = y1,
                                   Rcpp::Named("Rnew")  = y2,
                                   Rcpp::Named("sugar") = y3);
 

This example also uses the new Rcpp attributes described briefly in the announcement blog post and of course in more detail in the corresponding vignette. Let us just state here that we simply provide a complete C++ function, using standard Rcpp types -- along with one 'attribute' declaration of an export via Rcpp. That's it -- even easier than using inline. Now in R we simply do

R> sourceCpp("mypnorm.cpp")

to obtain a callable R function with the C++ code just shown behind it. No Makefile, no command-line tool invocation -- nothing but a single call to sourceCpp() which takes care of things --- and brings us a compiled C++ function to R just given the source file with its attribute declaration. We can now use the new function to compute the probaility distribution both the old way, the new way with the 'cleaner' R::pnorm(), and of course the Rcpp sugar way in a single call. We build a data frame in C++, and assert that all three variants are the same:

R> x <- seq(0, 1, length=1e3)
R> res <- mypnorm(x)
R> head(res)
      Rold     Rnew    sugar
1 0.500000 0.500000 0.500000
2 0.500399 0.500399 0.500399
3 0.500799 0.500799 0.500799
4 0.501198 0.501198 0.501198
5 0.501597 0.501597 0.501597
6 0.501997 0.501997 0.501997
R> all.equal(res[,1], res[,2], res[,3])
[1] TRUE
R> 

This example hopefully helped to illustrate how Rcpp 0.10.0 brings both something really powerful (Rcpp attributes -- more on this another time, hopefully) and convenient in the new namespace for statistical functions.

TL;DR: There s a summary at the end of the article. Today I wanted to see why a dependency in a .deb-package from an external APT repository changed so that it became uninstallable. While dpkg-deb --info foobar.deb easily shows the control information, the changelog is in the filesystem part of the package. I could extract that one dpkg-deb, too, but I d have to extract either to some temporary directory or pipe it into tar which then can extract a single file from the archive and sent it to STDOUT:

dpkg-deb --fsys-tarfile foobar.deb   tar xOf - ./usr/share/doc/foobar/changelog.Debian.gz   zless

But that s tedious to type. The following command is clearly less to type and way easier to remember:

acat foobar.deb ./usr/share/doc/foobar/changelog.Debian.gz   zless

acat stands for archive cat is part of the atool suite of commands:

als

lists files in an archive.

$ als foobar.tgz
drwxr-xr-x abe/abe           0 2012-11-15 00:19 foobar/
-rw-r--r-- abe/abe          13 2012-11-15 00:20 foobar/bar
-rw-r--r-- abe/abe          13 2012-11-15 00:20 foobar/foo

acat

extracts files in an archive to standard out.

$ acat foobar.tgz foobar/foo foobar/bar
foobar/bar
bar contents
foobar/foo
foo contents

adiff

generates a diff between two archives using diff(1).

$ als quux.zip
Archive:  quux.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2012-11-15 00:23   quux/
       16  2012-11-15 00:22   quux/foo
       13  2012-11-15 00:20   quux/bar
---------                     -------
       29                     3 files
$ adiff foobar.tgz quux.zip
diff -ru Unpack-3594/foobar/foo Unpack-7862/quux/foo
--- Unpack-3594/foobar/foo      2012-11-15 00:20:46.000000000 +0100
+++ Unpack-7862/quux/foo        2012-11-15 00:22:56.000000000 +0100
@@ -1 +1 @@
-foo contents
+foobar contents

arepack

repacks archives to a different format. It does this by first extracting all files of the old archive into a temporary directory, then packing all files extracted to that directory to the new archive. Use the --each (-e) option in combination with --format (-F) to repack multiple archives using a single invocation of atool. Note that arepack will not remove the old archive.

$ arepack foobar.tgz foobar.txz
foobar.tgz: extracted to  Unpack-7121/foobar'
foobar.txz: grew 36 bytes

apack

creates archives (or compresses files). If no file arguments are specified, filenames to add are read from standard in.

aunpack

extracts files from an archive. Often one wants to extract all files in an archive to a single subdirectory. However, some archives contain multiple files in their root directories. The aunpack program overcomes this problem by first extracting files to a unique (temporary) directory, and then moving its contents back if possible. This also prevents local files from being overwritten by mistake.

(atool subcommand descriptions from the atool man page which is licensed under GPLv3+. Examples by me.) I though miss the existence of an agrep subcommand. Guess why? atool supports a wealth of archive types: tar (gzip-, bzip-, bzip2-, compress-/Z-, lzip-, lzop-, xz-, and 7zip-compressed), zip, jar/war, rar, lha/lzh, 7zip, alzip/alz, ace, ar, arj, arc, rpm, deb, cab, gzip, bzip, bzip2, compress/Z, lzip, lzop, xz, rzip, lrzip and cpio. (Not all subcommands support all archive types.) Similar Utilities There are some utilities which cover parts of what atool does, too: Tools from the mtools package Yes, they come from the handle MS-DOS floppy disks tool package, don t ask me why. :-)

uz

gunzips and extracts a gzip d tar d archives

Advantage over aunpack: Less to type. :-)

Disadvantage compared to aunpack: Supports only one archive format.

lz

gunzips and shows a listing of a gzip d tar d archive

Advantage over als: One character less to type. :-)

Disadvantage compared to als: Supports only one archive format.

unp unp extracts one or more files given as arguments on the command line.

$ unp -s
Known archive formats and tools:
7z:           p7zip or p7zip-full
ace:          unace
ar,deb:       binutils
arj:          arj
bz2:          bzip2
cab:          cabextract
chm:          libchm-bin or archmage
cpio,afio:    cpio or afio
dat:          tnef
dms:          xdms
exe:          maybe orange or unzip or unrar or unarj or lha 
gz:           gzip
hqx:          macutils
lha,lzh:      lha
lz:           lzip
lzma:         xz-utils or lzma
lzo:          lzop
lzx:          unlzx
mbox:         formail and mpack
pmd:          ppmd
rar:          rar or unrar or unrar-free
rpm:          rpm2cpio and cpio
sea,sea.bin:  macutils
shar:         sharutils
tar:          tar
tar.bz2,tbz2: tar with bzip2
tar.lzip:     tar with lzip
tar.lzop,tzo: tar with lzop
tar.xz,txz:   tar with xz-utils
tar.z:        tar with compress
tgz,tar.gz:   tar with gzip
uu:           sharutils
xz:           xz-utils
zip,cbz,cbr,jar,war,ear,xpi,adf: unzip
zoo:          zoo

So it s very similar to aunpack, just a shorter command and it supports some more exotic archive formats which atool doesn t support. Also part of the unp package is ucat which does more or less the same as acat, just with unp as backend. dtrx From the man page of dtrx:

In addition to providing one command to extract many different archive types, dtrx also aids the user by extracting contents consistently. By default, everything will be written to a dedicated directory that s named after the archive. dtrx will also change the permissions to ensure that the owner can read and write all those files. Supported archive formats: tar, zip (including self-extracting .exe files), cpio, rpm, deb, gem, 7z, cab, rar, and InstallShield. It can also decompress files compressed with gzip, bzip2, lzma, or compress.

dtrx -l lists the contents of an archive, i.e. works like als or lz. dtrx has two features not present in the other tools mentioned so far:

• It can extract metadata instead of the normal contents from .deb and .gem files.
• It can extract archives recursively, i.e. can extract archives inside of archives.

Unfortunately you can t mix those two features. But you can use the following tool for that purpose: deepfind deepfind is a command from the package strigi-utils and recursively lists files in archives, including archives in archives. I ve already written a detailed blog-posting about deepfind and its friend deepgrep. tardiff tardiff was written to check what changed in source code tarballs from one release to another. By default it just lists the differences in the file lists, not in the files contents and hence works different than adiff. Summary atool and friends are probably the first choice when it comes to DWIM archive handling, also because they have an easy to remember subcommand scheme. uz and lz and the shortest way to extract or list the contents of a .tar.gz file. But nothing more. And you have to install mtools even if you don t have a floppy drive. unp comes in handy for exotic archive formats atool doesn t support. And it s way easier to remember and type than aunpack. dtrx is neat if you want to extract archives in archives or if you want to extract metadata from some package files with just a few keystrokes. For listing all files in recursive archives, use deepfind.

Here is another interview with one of the people in the Debian Edu and Skolelinux community. I am running short on people willing to be interviewed, so if you know about someone I should interview, Please send me an email. After asking for many months, I finally managed to lure another one of the people behind the German "IT-Zukunft Schule" project out from maternity leave to conduct an interview. Give a warm welcome to Angela Fu . :) Who are you, and how do you spend your days? I am a 39-year-old woman living in the very north of Germany near Denmark. I live in a patchwork family with "my man" Mike Gabriel, my two daughters, Mikes daughter and Mikes and my rather newborn son. At the moment - because of our little baby - I am spending most of the day by being a caring and organising mom for all the kids. Besides that I am really involved into and occupied with several inner growth processes: New born souls always bring the whole familiar system into movement and that needs time and focus ;-). We are also in the middle of buying a house and moving to it. In 2013 I will work again in my job in a German foundation for nature conservation. I am doing public relation work there. Besides that - and that is the connection to Skolelinux / Debian Edu - I am working in our own school project "IT-Zukunft Schule" in North Germany. I am responsible for the quality assurance, the customer relationship management and the communication processes in the project. Since 2001 I constantly have been training myself in communication and leadership. Besides that I am a forester, a landscaping gardener and a yoga teacher. How did you get in contact with the Skolelinux / Debian Edu project? I fell in love with Mike ;-). Very soon after getting to know him I was completely enrolled into Free Software. At this time Mike did IT-services for one newly founded school in Kiel. Other schools in Kiel needed concepts for their IT environment. Often when Mike came home from working at the newly founded school I found myself listening to his complaints about several points where the communication with the schools head or the teachers did not work. So we were clear that he would not work for one more school if we did not set up a structure for communication between him, the schools head, the teachers, the students and the parents. Together with our friend and hardware supplier Andreas Buchholz we started to get an overview of free software solutions suitable for schools. One day before Christmas 2010 Mike and I had a date with Kurt Gramlich in G tersloh. As Kurt and I are really interested in building networks of people and in being in communication we dived into Skolelinux and brought it to the first grammar schools in Northern Germany. For information about our school project you can read the interview with Mike Gabriel. What do you see as the advantages of Skolelinux / Debian Edu? First I have to say: I cannot answer this question technically. My answer comes rather from a social point of view. The biggest advantage of Skolelinux / Debian Edu I see is the large and strong international community of Debian Developers in the background which is very alive and connected over mailinglists, blogs and meetings. My constant feeling for the Debian Community is: If something does not work they will somehow fix it. All is well ;-). This is of course a user experience. What I also get as a big advantage of Skolelinux / Debian Edu is that everybody who uses it and works with it can also contribute to it - that includes students, teachers, parents... What do you see as the disadvantages of Skolelinux / Debian Edu? I will answer this question relating to the internal structure of Skolelinux / Debian Edu. What I see as a major disadvantage is that there is a gap between the group of developers for Debian Edu and the people who make the marketing, that means the people that bring Skolelinux to the schools. There is a lack of communication between these two groups and I think that does not really work for Skolelinux / Debian Edu. Further I appreciate that Skolelinux / Debian Edu is known as a do-ocracy. Nevertheless I keep asking myself if at some points a democracy or some kind of hierarchical project structure would be good and helpful. I am also missing some kind of contact between the Skolelinux / Debian Edu communities in Europe or on an international level. I think it would be good if there was more sharing between the different countries using Skolelinux / Debian Edu. Which free software do you use daily? On my laptop I am still using an Ubuntu 10.04 with a Gnome Desktop on. As applications I use Openoffice.org, Gedit, Firefox, Pidgin, LaTeX and GnuCash. For mails I am using Horde. And I am really fond of my N900 running with Maemo. Which strategy do you believe is the right one to use to get schools to use free software? I am really convinced that in our school project "IT-Zukunft Schule" we have developed (and keep developing) a great way to get schools to use Free Software. We have written a detailed concept for that so I cannot explain the whole thing here. But in a nutshell the strategy has three crucial pillars:

• We really take time to get what sort of stories, questions and concerns the schools head and the teachers have about using different kinds of IT and we take time to enrol them into Free Software.
• Our solution for schools is never just technical. In the centre are always the people who are going to use the software. From the very beginning of the planning for a school, we tell the schools head that they are paying us not only for a technical solution for their school, they also pay us for leading all the communication processes needed. If they do not want that, we are not working with them because we cannot give a guarantee for the quality of our work then.
• Another focus lies in the training of teachers and students in co-administrating the IT-System at their school. They start getting in contact with the Skolelinux / Debian Edu community and they get the offer to become more and more independent from us.

Rcpp release 0.10.0 is now on CRAN and being uploaded to Debian. This is a new feature release, and we are very exciting about the changes, notably Rcpp attributes which make using C++ from R even easier than inline (see below as well as the new vignette for details and first examples), the extensions to Rcpp modules (see below) and more as for example new Rcpp sugar functions, a new error output device syncing to R, and a new namespace R> for the statistical functions from Rmath.h. The complete NEWS entry for 0.10.0 is below; more details are in the ChangeLog file in the package and on the Rcpp Changelog page.

Changes in Rcpp version 0.10.0 (2012-11-13)

• Support for C++11 style attributes (embedded in comments) to enable use of C++ within interactive sessions and to automatically generate module declarations for packages:
  • Rcpp::export attribute to export a C++ function to R
  • sourceCpp() function to source exported functions from a file
  • cppFunction() and evalCpp() functions for inline declarations and execution
  • compileAttribtes() function to generate Rcpp modules from exported functions within a package
  • Rcpp::depends attribute for specifying additional build dependencies for sourceCpp()
  • Rcpp::interfaces attribute to specify the external bindings compileAttributes() should generate (defaults to R-only but a C++ include file using R_GetCCallable can also be generated)
  • New vignette "Rcpp-attribute"
• Rcpp modules feature set has been expanded:
  • Functions and methods can now return objects from classes that are exposed through modules. This uses the make_new_object template internally. This feature requires that some class traits are declared to indicate Rcpp's wrap/as system that these classes are covered by modules. The macro RCPP_EXPOSED_CLASS and RCPP_EXPOSED_CLASS_NODECL can be used to declared these type traits.
  • Classes exposed through modules can also be used as parameters of exposed functions or methods.
  • Exposed classes can declare factories with ".factory". A factory is a c++ function that returns a pointer to the target class. It is assumed that these objects are allocated with new on the factory. On the R side, factories are called just like other constructors, with the "new" function. This feature allows an alternative way to construct objects.
  • "converter" can be used to declare a way to convert an object of a type to another type. This gets translated to the appropriate "as" method on the R side.
  • Inheritance. A class can now declare that it inherits from another class with the .derives<Parent>( "Parent" ) notation. As a result the exposed class gains methods and properties (fields) from its parent class.
• New sugar functions:
  • which_min implements which.min. Traversing the sugar expression and returning the index of the first time the minimum value is found.
  • which_max idem
  • unique uses unordered_set to find unique values. In particular, the version for CharacterVector is found to be more efficient than R's version
  • sort_unique calculates unique values and then sorts them.
• Improvements to output facilities:
  • Implemented sync() so that flushing output streams works
  • Added Rcerr output stream (forwarding to REprintf)
• Provide a namespace 'R' for the standalone Rmath library so that Rcpp users can access those functions too; also added unit tests
• Development releases sets variable RunAllRcppTests to yes to run all tests (unless it was alredy set to 'no'); CRAN releases do not and still require setting which helps with the desired CRAN default of less testing at the CRAN server farm.

Thanks to CRANberries, you can also look at a diff to the previous release 0.9.15. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

Update: One link corrected.

Dear lazyweb, we need a new printer-scanner combination for our new office, ideally a laser printer and a high-resolution scanner (1200dpi). It must integrate nicely with CUPS and SANE, without requiring any proprietary, closed-licence drivers or plugins. It also has to come with an automatic document feeder in addition to the plain flatbed scanner. Additional, would-be-nice features in decreasing order of preference are

• Colour printing (separate cartridges a must)
• Duplex printing (real duplex preferred over soft solutions)
• Direct scanning to PDF (distribution by E-mail, Samba or FTP)

I have investigated the Samsung CLX-6220FX, but the driver situation was such a nightmare (and the printer so loud), that we returned it right away. I should have read this article by Peter beforehand I have recently seen the HP Laserjet Pro 200 M276n, the successor of the HP Laserjet CM1415nf which looks promising, but according to LinuxPrinting.org, a proprietary plugin is required (WTF HP?). Does anyone have experience with these HP models and can recommend them for use with Debian stable? Does anyone have another recommendation? Please send me a message. Thanks!

Tomorrow (Thursday 15th November), there will be a PGP key signing and CAcert assurance party in Prague. It will be held in SUSE offices and it will start at 17:00. If you are interesting in having your PGP key signed, bring papers with your key fingerprint (you can use for example these slips) and some government issued ID. For CAcert assurance it is recommended to bring printed out assurance papers (you can get them from CAcert website) and of course IDs as well. You can find more information on Facebook or Google+ event pages.

Filed under: English Suse 2 comments Flattr this!

I am organizing two signing parties in France:

• Saturday, November 17th at 12:15 at the JDLL in Lyon;
• Sunday, November 25th at 11:45 at the Mini-DebConf in Paris.

Lyon, November 17th This will be an informal key signing party: just come with your ID documents and paper strips with your public key fingerprint. The signing party will occur at 2012-11-17 12:15+01:00 in the Maison Pour Tous in Lyon, France.

Paris, November 25th This will be a formally organized key signing party. If you want to participate, you should follow the indications given in the event wiki page. Basically:

1. send me your public key;
2. print some copies of your key fingerprint for people that did not register;
3. print the attendees list I shall send you before the event;
4. come to the signing party with all that, ID documents and a pen.

The signing party will occurr at 2012-11-25 11:45+01:00 in the pita school in Le Kremlin-Bic tre, France.

Tips To print strips with your public key fingerprint, you can use the utility gpg-key2ps from the package signing-party. If you are a user of the CAcert.org SSL certification authority, you can come to these signing parties with some copies of the accreditation form: it will be an opportunity to get or give CAcert.org accreditation.

Weblate is close to 1.3 release (should happen on Friday if nothing urgent appears) and it's quite last chance for translators to catch up. Weblate is of course translated using Weblate, but you can of course also translate Gettext po files directly and either upload them into Weblate or use Github issue tracker for that. If you don't know Weblate yet, it is web based tool for translating with Git integration. You can also call it crowdsourcing platform if you wish :-).

Filed under: English Phpmyadmin Weblate 2 comments Flattr this!

I do like the way programmers think in terms of DRY, for instance, among other forms of optimization. A couple of days ago I wanted to test different implementations of the same algorithm in C++. The usual approach would be to implement those algorithms, call them in main() and see the results. A more interesting approach would be to write a validator for each function and print whether the result is correct or not. Since both are boring and break the rule of DRY, I ve decided to go complicated but productive, so I ve implemented a function testing method using function pointers that will allow me to write a single function and use it to test the algorithms using different input and expected results. Let s say I have 3 algorithms that implement Modular Exponentiation.
uint64 modexp1new(const uint64 &a, const uint64 &_p, const uint64 &n)
...


uint64 modexp2(const uint64 &_a, const uint64 &_p, const uint64 &_n)
...


uint64 modexp3(const uint64 &a, const uint64 &p, const uint64 &n)
...

If you are not familiar with the syntax, const means that the parameter is constant, thus it will not be modified inside the function. The & means that the parameter is passed as a reference, read the memory address of the calling variable, so we don t overload RAM with new variables. Now, the testing part. For my very own purposes I just want to test those particular functions and know whether if the result they output conforms what I do expect. Some people call this unit testing.
I do also want to test it in different scenarios, meaning input and output. So, I m creating a function that will get the three parameters required to call the functions, and a fourth parameter that is the value I do expect to be the correct answer. Now, since I don t want to repeat myself writing a call for each case, I m going to create a function pointer that is an array and has the function s address as their value. That way I can call them in a loop, and voila! we are done. Finally, after calling the function I check the result with the expected value and print an OK or FAIL. Tricky part for this could be understanding the function pointer. Two things to consider: the returning value from the referenced functions has to be the same for all. Second: the input definition for each function has to be similar too. This is important because function pointers are just pointers and they need to know the size of the data type in order to navigate the memory. For this sample code the uint64 is a typedef for long long, of course. Full code is below.
void testAlgos(const uint64 &a, const uint64 &e, const uint64 &n, const uint64 &r)
uint64 (*fP[3])(const uint64&, const uint64&, const uint64&) = &modexp1new, &modexp2, &modexp3 ;
uint64 t; for(int i=0; i<3; i++) t = fP[i](a, e, n); std::cout < < "F(" << i << ") " << t << "\t";
if (t == r)
std::cout << "OK";
else
std::cout << "FAIL";

std::cout << std::endl;

Now that I have this, I can use it in main this way.
int main() uint64 a = 3740332;
uint64 e = 44383;
uint64 n = 3130164467;
uint64 r = 1976425102; testAlgos(a, e, n, r); a = 404137263;
r = 2520752541; testAlgos(a, e, n, r); a = 21;
e = 3;
n = 1003;
r = 234;
testAlgos(a, e, n, r); return 0;
Resulting in:
F(0) 657519405 FAIL
F(1) 1976425102 OK
F(2) 657519405 FAIL
F(0) -2752082808 FAIL
F(1) 2520752541 OK
F(2) -2752082808 FAIL
F(0) 234 OK
F(1) 234 OK
F(2) 234 OK
I m happy about not having to write tests like this for all use cases:
std::cout < < "f1: " << modexp1new(a, e, n) << std::endl;
std::cout << "f2: " << modexp2(a, e, n) << std::endl;
std::cout << "f3: " << modexp3(a, e, n) << std::endl;
Now, happiness can be more exciting if I do use templates so I can test any function independent of the data type for the returning and input values. You know, more abstraction. Homework if time allows! Have fun!
Related posts:

1. Compute Clusters Integration for Debian Development and Building Report 4 Hello, this is the fourth report for the project. First,...
2. Compute Clusters Integration for Debian Development and Testing Report 3 Hello this is the third report regarding my project. As...
3. Compute clusters for Debian development and building final report Compute clusters for Debian development and building final report...

Ever since I passed the age of 18, I've used my legal right in the UK to vote in every poll I could: local council, British Parliament, European Parliament and even the messed-up referendum on the Alternative Vote that we had last year. In each of those cases, I made a point of considering my available options and I voted accordingly. I went along and marked my ballot paper in person where possible, or I filled in a postal vote for the last Westminster election in 2010 as I was on vacation in Florida when it was held. Unfortunately, it seems that my diligence in voting makes me unusual amongst the UK population these days. The long-term trend for voter turnout is downwards (as you can see). However, the latest election that's happening in the UK (well, England and Wales outside London) this week is for the newly-created posts of Police and Crime Commissioners. It's a textbook example of how not to organise an election, as pointed out eloquently by the Electoral Reform Society. This election has been incredibly badly managed and publicised by the Government, and lots of early polls are suggesting a tiny turnout of less than 20%. Despite the political crap about "democracy" being spouted by some Tory ministers, it's clear that very few people want this change in how the police are run, and it has simply been imposed from the top. Too few people care about the results of these elections for them to be valid - most people don't want the police politicised. After a lot of thought about the issue, I've decided how I'm going to vote this time. For the first time ever, I'm explicitly going to turn up and spoil my ballot paper as a protest - I do not want this crap. I urge other people in England and Wales to consider doing the same.

As I've already written before, phpMyAdmin is getting new documentation. The basic conversion has been already done and merged, but there are always things to improve. Right now I'm spending my free time on improving that and pushing the documentation forward. But hey, you can help in this area as well - just look at it on http://docs.phpmyadmin.net/ and in case you see any problems, fix them or at least report them. The easiest way to contribute a fix is to use online editor Github provides. Just navigate to the documentation sources, choose appropriate file, click on edit (what automatically forks the project for you) and once you're satisfied with your changes, open a pull request. In case this still looks too hard for you, just open a bug report or write us on mailing list or IRC. And you can of course also contribute to documentation translations, which did unfortunately suffer quite a lot from the documentation conversion.

Filed under: English Phpmyadmin 0 comments Flattr this!

New Laptop: Thinkpad x120e My travels the last few days have not been to anywhere new. I spent a few days at Tumco, then spent today in Algodones and am staying the night at Quechen Casino. Since I never described my new computer due to network problems, I figured now would be a good time to do so. I purchased a discontinued Lenovo Thinkpad x120e laptop. It has 2gb ram, a 320gb hard drive, dual-core AMD processor, and no optical drive. The first thing I did was wipe windows off of it and install Debian Squeeze (stable), then attempt to upgrade to Debian Wheezy (testing). (Squeeze lacked the drivers for some hardware.) Wheezy lacked the drivers for the display, so during the boot sequence it would switch to an unreadable screen. I eventually found that Sid (unstable) had the driver for the display in non-free. (To install it I needed to boot with the Squeeze kernel.) The most significant thing I have not yet debugged is hibernate mode. One thing I really like about this system is the 4-6 hour battery life. This is enough that I can charge during the day when my solar is in absorption mode (meaning the panels are producing more power than the house battery can safely take) and use it at night and in the morning when there is no spare solar power without depleting the house battery. The 12v charger that I had purchased for my old Thinkpad (which died, prompting the replacement) works fine on my new one. I purchased a USB powered DVD writer to give me optical capability when I need it. This uses two of the three USB ports. The network problem I had this summer was the Millenicom 3g USB stick glitched and needed to be reset with their proprietary software that is only available for Windows and Mac. Lacking access to a Windows system that I could install software on, this meant that I was without my network connection. I used the Sandy Library (fast, free wifi) 30 miles from my worksite and occasionally wifi at a closer coffee shop or cafe which were both unreliable.

For the last couple of years, one of the topics I have tackled with several columns and a couple of articles published, several presentations on conferences and many comments wherever I can is raising awareness on why security experts around the world oppose electronic voting. Slightly over a year ago, I started gathering all the published news (in Spanish, with some notes in English if I feel them relevant enough) that came to my greedy hands and archiving them, categorized as well as I could, into what I came to call an Electronic voting observatory. I don't feed it with the frequency I'd like, but overall I do feel there is an interesting amount of information in there. About a month and a half ago, via Twitterfeed, I set up a Twitter account where my bot echoes everything I get on that regard. So, while you can follow the observatory via RSS (as probably most readers of my blog will do), I know there are many Twitter users among you @e_voto might be interesting to you if you speak Spanish and are interested in the topic. I try not to be partisan on the information I copy there; of course I am subjective (I try not to refer to news which don't provide new insight or data points, or to obvious repeats), but I am doing this effort to follow the development as it unfolds, not to push my viewpoints.

This is slightly old news now, but I've decided to starting blogging and this seems like a good place to start. At this year's debconf in Nicaragua I finally got around to applying to be a Debian Maintainer. Thanks to the various people who advocated me (and prodded me to apply in the first place!). My key was added to the DM keyring in mid-August. I intend to continue to maintain the various IVTV capture card packages that I maintain (ivtv-utils and xserver-xorg-video-ivtv) as well as continuing my involvement with the packaging of Xen and associated stuff like the kernel and installer. I also plan keep working on improved support for the DreamPlug, for which I have a few fixes queued up for post-Wheezy already. More recently I've bought a QNAP TS-210 (which is a nice little NAS that can run Debian) and ended up adopting the qcontrol package, which ended up being my first actual upload to Debian! Anyway, hopefully I'll manage to post here at least semi-regularly.

again, not one of my best RCBW weeks. still, here's the short report:

• #683100 pnp4nagios-web: "pnp4nagios-web: libjs-jquery-ui (>= 1.8.14) may be not enough"
  add a comment to the bug report
• #683138 pnp4nagios-web: "pnp4nagios-web: crashes immediately due to debian/patches/adjust-template-path"
  add a comment to the bug report
• #689687 src:guile-1.6: "guile-1.6: FTBFS on i386, kfreebsd-i386: testsuite failures"
  sponsor NMU prepared by Daniel Hartwig, upload to DELAYED/2
• #690128 inn2: "inn2: conffile disappearing during squeeze->wheezy upgrade: /etc/news/motd.news"
  some triaging
• #691215 pinpoint: "X request failed: BadDrawable"
  try to reproduce, unsuccessful
• #691535 mumble: "Plugins: Failed to load libmumble.so: Cannot load library /usr/lib/mumble/libmumble.so: (/usr/lib/mumble/libmumble.so: undefined symbol: glPopClientAttrib)"
  send a patch proposal to the BTS, then upload to unstable with maintainer's approval

Today, I officially got approved by the Debian Account Managers as a Debian Developer (still waiting on keyring-maint and DSA). Over the years, I have seen many people complain about the New Member Process. The most common complaint was with regards to the (usually) long amount of time the process can take to complete. I am writing this blog post to provide one more perspective on this process. Hopefully, it will prove useful to people considering starting the New Member Process. The most difficult part about the New Member Process for me had to do with getting my GPG key signed by Debian Developers. I have not been able to attend any large conferences, which are great places to get your key signed. I also have not been able to meet up with the few Debian Developers living in/around Chicago. As a result, I was forced to patiently wait to start the NM process. This waiting period lasted a couple of years. It wasn't until this October, at the [Association for Computing Machinery at Urbana-Champaign's Reflections Projections Conference], that this changed. Stefano Zacchiroli was present to give a talk about Debian. Asheesh Laroia was also present to lead an OpenHatch Workshop about contributing to open source projects. Both of these Developers were more than willing to sign my key when I asked. If you look at my key, you will see that these signatures were made on October 7 and October 9, 2012. With the signatures out of the way, the next step in the process was to actually apply. Since I did not already have an account in the system, I had to send an email to the Front Desk and have them enter my information into the system. Details on this step, along with a sample email are available here. Once I was in the system, the next step was to get some Debian Developers to serve as my advocates. Advocates should be Debian Developers you have worked closely with, and usually include your sponsor(s). If these people believe you are ready to become a Debian Developer, they write a message describing the work you have been doing with them and why they feel you are ready. Paul Tagliamonte had helped review and sponsor a number of my uploads. I had been working with him for a number of years, and he really helped encourage and help me to reach this milestone. He served as my first advocate. Gregor Herrmann is responsible for getting me started in contributing to Debian. When I first tried to get involved, I had a hard time finding sponsors for my uploads and bugs to work on. Eventually, I discovered the Debian Perl Group. This team collectively maintains most of the Perl modules that are included in the Debian repositories. Gregor and the other Debian Developers on the team were really good about reviewing and sponsoring uploads in a very timely manner. This allowed me to learn quickly and make a number of contributions to Debian. He served as my second advocate. With my advocations taken care of, the next step in the process was for the Front Desk to assign me an Application Manager and for the Application Manager to accept the appointment. Thijs Kinkhorst was appointed as my Application Manager. He also agreed to take on this task. For those of you who might not know, the Application Manager is in charge of asking the applicant questions, collecting information, and ultimately making a recommendation to the Debian Account Managers about whether or not they should accept the applicant as a Developer. They can go about this in a variety of ways, but most choose to utilize a set of template questions that are adjusted slightly on a per-applicant basis. Remember that period of waiting to get my GPG key signed? I had used that time to go through and prepare answers to most of the template questions. This served two purposes. First, it allowed me to prove to myself that I had the knowledge to become a Debian Developer. Second, it helped to greatly speed up the New Member process once I actually applied. There were some questions that were added/removed/modified, but by answering the template questions befrehoand, I had become quite familiar with documents such as the Debian Policy and the Debian Developer's Rerference. These documents are the basis for almost all questions that are asked. After several rounds of questions, testing my knowledge of Debian's philosophy and procedures as well as my technical knowledge and skills, some of my uploads were reviewed. This is a pretty standard step. Be prepared to explain any changes you made (or chose not to make) in your uploads. If you have any outstanding bugs or issues with your packages, you might also be asked to resolve them. Eventually, once your Application Manager has collected enough information to ensure you are capable of becoming a Debian Developer, they will send their recommendation and a brief biography about you to the debian-newmaint mailing list and forward all information and emails from you to the Debian Account Managers (after the Front Desk checks and confirms that all of the important information is present). The Debian Account Managers have the actual authority to approve new Debian Developers. They will review all information sent to them and reach their own decision. If they approve your application, they will submit requests for your GPG key to be added to the Debian Keyring and for an account to be created for you in the Debian systems. At this point, the New Member process is complete. For me, it took exactly 1 month from the time I officially applied to become a Debian Developer until the time of my application being approved by the Debian Account Managers. Hopefully, it will not be long until my GPG key is added to the keyring and my account is created. I feel the entire process went by very quickly and was pain-free. Hopefully, this blog post will help to encourage more people to apply to become Debian Developers.

I've just released initial version of gettext-inspector, a tool for checking gettext PO/POT/MO files. While it's in an early stage of development, it's already able to detect wide rage of problems. For example, this is what it emits on my system:

$ gettext-inspector /usr/share/locale/*/LC_MESSAGES/*.mo   cut -d ' ' -f 1,3   sort   uniq -c   sort -rn
   1902 P: no-language-header-field
   1601 P: no-version-in-project-id-version
   1372 W: no-report-msgid-bugs-to-header-field
    273 P: invalid-content-transfer-encoding
    201 W: invalid-date
     78 W: syntax-error-in-plural-forms
     77 I: no-package-name-in-project-id-version
     63 W: boilerplate-in-report-msgid-bugs-to
     50 W: language-disparity
     47 I: unknown-header-field
     38 W: invalid-language
     25 W: boilerplate-in-project-id-version
     10 I: unknown-poedit-language
      8 W: no-date-header-field
      5 W: no-project-id-version-header-field
      5 W: c1-control-characters
      3 P: no-mime-version-header-field
      3 P: no-content-transfer-encoding-header-field
      2 W: non-portable-encoding
      1 W: invalid-report-msgid-bugs-to
      1 W: ancient-date
      1 I: unable-to-determine-language

While going through the NM Process, I spent a lot of time on https://nm.debian.org. At the bottom of the page, there is a TODO list of some new features they would like to implement. One item on the list jumped out at me as something I was capable of completing: IRC bot to update stats in the #debian-newmaint channel topic. I immediately reached out to Enrico Zini who was very supportive of the idea. He also explained how he wanted to expand on that idea and have the bot send updates to the channel whenever the progress of an applicant changes. Thanks to Paul Tagliamonte, I was able to get my hands on a copy of the nm.debian.org database (with private information replaced with dummy data). This allowed me to create some code to parse the database and create counts for the number of people waiting in the various queues. I also created a list of events that occurred in the last n days. Enrico then took this code and modified it to integrate it into the website. You can specify the number of days to show events for, and even have the information produced in JSON format. This information is generated upon requesting the page, so it is always up-to-date. It took a couple of rounds of revisions to ensure that the website was exposing all of the necessary information in the JSON export. At this stage, I converted the code to be an IRC bot. Based on prior experience, I decided to implement this as an irssi5 script. The bot is currently running as nmbot in #debian-newmaint on OFTC. Every minute, it fetches the JSON statistics to see if any new events have occurred. If they have, it updates the topic and sends announcements as necessary. While the bot is running, there are still a few more things on its TODO list. First, we need to move it to a stable and permanent home. Running it off of my personal laptop is fine for testing, but it is not a long term solution. Luckily, Joerg Jaspert (Ganneff) has graciously offerred to host the bot. He also made the suggestion of converting the bot to a Supybot plugin so that it could be integrated into the existing ftpmaster bot (dak). The bot's code is currently pretty simple, so I do not expect too much difficulty in converting it to Python/supybot. One last item on the list is something that Enrico is working on implementing. He is going to have the website generate static versions of the JSON output whenever an applicant's progress changes. The bot could then fetch this file, which would reduce the number of times the site needs to generate the JSON. The code for the bot is available in a public git repository, and feedback/suggestions are always appreciated.