A new version of our RcppExamples, package is now on CRAN. RcppExamples contains a few illustrations of how to use Rcpp. It grew out of documentation for the classic API (now in its own package RcppClassic) and we added more functions documenting how to do the same with the new API we have been focusing on for the last year or so. One of the things I added in the last few days was the example below showing how to use Rcpp::List with lookups to replace use of the old and deprecated RcppParams. It also show how to return values to R rather easily <quote>

#include <Rcpp.h>
RcppExport SEXP newRcppParamsExample(SEXP params)  
    try                                         // or use BEGIN_RCPP macro
        Rcpp::List rparam(params);              // Get parameters in params.
        std::string method   = Rcpp::as<std::string>(rparam["method"]);
        double tolerance     = Rcpp::as<double>(rparam["tolerance"]);
        int    maxIter       = Rcpp::as<int>(rparam["maxIter"]);
        Rcpp::Date startDate = Rcpp::Date(Rcpp::as<int>(rparam["startDate"])); // ctor from int
        
        Rprintf("\nIn C++, seeing the following value\n");
        Rprintf("Method argument    : %s\n", method.c_str());
        Rprintf("Tolerance argument : %f\n", tolerance);
        Rprintf("MaxIter argument   : %d\n", maxIter);
        Rprintf("Start date argument: %04d-%02d-%02d\n", 
                startDate.getYear(), startDate.getMonth(), startDate.getDay());
        return Rcpp::List::create(Rcpp::Named("method", method),
                                  Rcpp::Named("tolerance", tolerance),
                                  Rcpp::Named("maxIter", maxIter),
                                  Rcpp::Named("startDate", startDate),
                                  Rcpp::Named("params", params));  // or use rparam
      catch( std::exception &ex )               // or use END_RCPP macro
        forward_exception_to_r( ex );
      catch(...)   
        ::Rf_error( "c++ exception (unknown reason)" ); 
     
    return R_NilValue; // -Wall
 

</quote> The package is work-in-progress and needs way more general usage examples for Rcpp and particularly the new API. But it's a start. A few more details on the page are on the RcppExamples page.

(This is part of a series of posts on Why Linux?) I've run a number of distros in my time. I ended up on Debian near the end of 1999, and part of the drive for that was the number of packages available in one centralised location. Decent package management is a definite strength of Linux (or FreeBSD) over proprietary operating systems. It derives from the freedom aspect, but means you can end up with one source for all (or most) of your software, that's compiled against the same set of libraries, with one way to track what owns what. This may not seem like a big thing, especially if you're a hobbyist or are coming from a Windows background. Reinstalling is often seen as a necessary regular requirement. Personally I've got better things to do with my time. If I want to try out a piece of software I want to be able to install it safe in the knowledge I know exactly what files it owns and where there are. And I want it to be able to tell me what other common components it needs that I might not already have. Then if I decide it's not for me I can cleanly remove it and anything else it pulled in that I no longer need. Don't underestimate this. This is useful on all of my machines. I can query the version number of everything installed. I can check for updates with one command (no need for every piece of installed software to have its own updater implementation). Software can share libraries correctly rather than stashing their own private copies, meaning I get bug fixes and security updates. (Yes, sometimes authors bundle even in the Linux world. Stop it.) I'm a developer. I tend to interact with a lot of different systems, of different types. It's really handy to have access to a wide range of tools to help me with that, know that there's legally no problem with me installing them, be able to do so with a single command and, should they turn out to be unsuitable, know I can cleanly remove them with another single command. This is a definite win in the work context. Equally I've been a sysadmin for multiple machines at once. Being able to login to each of them and check that everything is up to date is damn handy. Being able to easily install software for customers tends to make you popular too. And being able to rebuild boxes (or build additional boxes to share load) with the same setup is a lot easier with a decent package manager too. And, to pre-empt any responses about how a lot of this is possible under, say, Windows, yes, it is. I've spent some time in the past building packages for commercial deployment using Novadigm's Radia tool. I'm aware that Windows integral package management has also got better over time. I still think dpkg/apt (or rpm/yum) is far more powerful. And, for the end user, mostly easier as well - distros are building pre-prepared packages for you, rather than you having to do it yourself like with Radia.

A few months ago, my company made the decision to switch from our previous mail provider, to use Google Apps. It was a no-brainer really, most of the company were using it already, for the calendars, and Documents, and well we were a bit fed up with our previous mail provider. So we made the move. As an online retailer, we obviously get a lot of Financial related emails, and our Accountancy Department has an email address setup as a group, where the group sends to all the members in the Accounts Team. A pretty simple setup. However, for the last 2 months, I ve been fighting an uphill battle to get this to work properly. When an email gets delivered to a Google Group, it stops there, and then Google re-sends the email to the people in the group. Somehow, this triggers the receiving account to believe that it s originated from the Google Groups servers, rather than the actual originating server. Google Apps emails have a pretty nifty spam filtering service. For those well-known services on the web, it ll check whether it actually came from them (I think through some combination of DKIM and SPF), and bounce it if not. Can you see where I m going yet? Google Groups has a nifty feature to stop it attempting to send emails to addresses that don t exist. If an email address bounces at a certain rate, it ll flag it as undeliverable. If all the emails in a group are flagged as undeliverable, it ll bounce that email to the original party, so that they know no-one received it. The Accounts team are setup to receive information regarding Paypal Payments, Disputes, etc etc. A couple of days in, I had our Financial controller tell us that the Accounts team had stopped receiving emails. Well, being who I am I sent a test email out guess what I got back? Hello <redacted>, Your message could not be delivered because of previous failures during delivery attempts to this mailing list. Please update the list with valid addresses. This is an example of some of the bounces we received: ----------------------------------------------
Recipient: pa.......@mobilefun.co.uk 550 550 5.7.1 Unauthenticated email is not accepted from this domain. u22si29486554yba.55 (state 18).
Message-Id: <redacted> ----------------------------------------------
Recipient: ni........@mobilefun.co.uk 550 550 5.7.1 Unauthenticated email is not accepted from this domain. u22si29486554yba.55 (state 18).
Message-Id: <redacted> ----------------------------------------------
Recipient: lu..........@mobilefun.co.uk 550 550 5.7.1 Unauthenticated email is not accepted from this domain. u22si29486554yba.55 (state 18).
Message-Id: <redacted> *sigh* time to play with the group. I change the members of the group to use one of our alias domains instead of the @mobilefun.co.uk this gets emails working again. 2 days later . our Financial Controller comes to me and complains that clients are complaining that they re receiving bounce emails when sending to his team I check, and once again, get the same message. Time to open a support ticket. That was in October since then we ve tried everything Changing to user-defined groups and switching off the spam filters, whitelisting the groups, setting up DKIM on our domain, changing the spam levels, clearing bounce statuses all to no avail I ve been back and forthing with Google Enterprise Support for 2 months and have still not found a suitable solution *sigh*. I must say however, that the person dealing with the case at Google (Josephine H) has been professional and helpful all the way along, even with my rising frustration at the issue. Today I ended up calling their service unusable number, after having tried a couple of times to change the groups so that the Accounts Team could receive emails again and finding that none of my previous tricks worked. While on hold, I found that I could use a local part extension to an email address, and the group would recognise it as a new email address and therefore have no bounce status. I ve now made a script using the Provisioning APIs, and a bit of python-fu that will generate a local part extension based on the current date/time, and replace the users in the group for the Accounts team with those. Say for example, the primary email address for a user was test@example.com it d add a user of test+20101221235641@example.com. This is set to run each day so it s pretty much the same as resetting the bounce status (which fixes things for a short while) on a daily basis. According to my latest email from Google Enterprise support:-

At the moment there is an incentive going on to fix this outright as messages from Paypal etc have been causing bounce s for other domains. This fix is supposedly due very early in the New Year and will solve this problem indefinitely.

I wait with baited breath but for now, I m happy with my hack.

As of tonight, all translatable debconf templates for packages in squeeze are translated in German and French, after udev entered testing. Both teams are very happy to join the Russian team who was the very first to reach that goal on November 5th. Unfortunately, for our Russian friends, debian-edu-config is currently missing for them now. We of course hope for Debian Edu developers to correct this before squeeze. The Swedish team should join the 100% gang as soon as recently uploaded console-setup reaches testing. The Portuguese team is now only missing two complete packages to reach that 100%: sympa and (again) udev. udev is likely to be updated but, unfortunately, this is much less sure for sympa. The Czech team is only missing bugzilla. Again, unfortunately, that package needs an upload to testing-proposed-updates (that should fix and RC bug) and I am technically unable to do it (and the maintainer is apparently not motivated for doing it, arguing that a more recent upstream version should go in instead, which is unacceptable for the release team). The Spanish team made a great effort, but I'm not sure that the 16 packages they're missing will make it. Next time, folks! Full list for the "7 languages trying to reach 100%" at http://i18n.debian.net/debian-l10n/l10n-testing.txt

I would appreciate if you could test the proposed fixes for the exim4 privilege escalation bug CVE-2010-4345. Preliminary binary and source packages for squeeze/sid and lenny are available here:
deb http://www.bebt.de/debian/ sid exim4+cve
deb-src http://www.bebt.de/debian/ sid exim4+cve
deb http://www.bebt.de/debian/ lenny exim4+cve
deb-src http://www.bebt.de/debian/ lenny exim4+cve
You can also browse the changes in SVN (lenny and sid) or build your own binaries.

---

Changelog:

• 67_unnecessaryCopt.dpatch: Do not use exim's -C option in utility scripts. This would not work with ALT_CONFIG_PREFIX.
• Pull changes related to fixing CVE-2010-4345 from exim 4.73 rc1. Closes: #606527
  • 1_cfile_norw_eximuid: Don't allow a configure file which is writeable by the Exim user or group.
  • 2_permcheck_configurefile: Check configure file permissions even for non-default files if still privileged.
  • 3_remove_ALT_CONFIG_ROOT_ONLY: Remove ALT_CONFIG_ROOT_ONLY build option, effectively making it always true.
  • 4_FD_CLOEXEC: Set FD_CLOEXEC on SMTP sockets after forking in the daemon, to ensure that rogue child processes cannot use them.
  • 5_TRUSTED_CONFIG_LIST: Add TRUSTED_CONFIG_LIST compile option.
  • 6_nonroot_system_filter_user: If the system filter needs to be run as root, let that be explicitly configured. The default is now the Exim run-time user.
  • 7_filter_D_option: Add a (compiletime) whitelist of acceptable values for the -D option.
  • 8_updatedocumentation: Update documentation to reflect the changes.
• 4_FD_CLOEXEC replaces 80_fdleak.dpatch, drop the latter.
• Build with WHITELIST_D_MACROS=OUTGOING. Post patch 7_filter_D_option exim will not regain root privileges (usually necessary for local delivery) if the -D option was used. Macro identifiers listed in WHITELIST_D_MACROS are exempted from this restriction. mailscanner (4.79.11-2.2) uses -DOUTGOING.
• Build with TRUSTED_CONFIG_LIST=/etc/exim4/trusted_configs. Post patch 3_remove_ALT_CONFIG_ROOT_ONLY exim will not re-gain root privileges (usually necessary for local delivery) if the -C option was used. This makes it impossible to start a fully functional damon with an alternate configuration file. /etc/exim4/trusted_configs (can) contain a list of filenames (one per line, full path given) to which this restriction does not apply.

NEWS entry: Exim versions up to and including 4.72 are vulnerable to CVE-2010-4345. This is a privilege escalation issue that allows the exim user to gain root privileges by specifying an alternate configuration file using the -C option. The macro override facility (-D) might also be misused for this purpose. In reaction to this security vulnerability upstream has made a number of user visible changes. This package includes these changes. If exim is invoked with the -C or -D option the daemon will not regain root privileges though re-execution. This is usually necessary for local delivery, though. Therefore it is generally not possible anymore to run an exim daemon with -D or -C options. However this version of exim has been built with TRUSTED_CONFIG_LIST=/etc/exim4/trusted_configs. TRUSTED_CONFIG_LIST defines a list of configuration files which are trusted; if a config file is owned by root and matches a pathname in the list, then it may be invoked by the Exim build-time user without Exim relinquishing root privileges. As a hotfix to not break existing installations of mailscanner we have also set WHITELIST_D_MACROS=OUTGOING. i.e. it is still possible to start exim with -DOUTGOING while being able to do local deliveries. If you previously were using -D switches you will need to change your setup to use a separate configuration file. The ".include" mechanism makes this easy. The system filter is run as exim_user instead of root by default. If your setup requies root privileges when running the system filter you will need to set the system_filter_user exim main configuration option.

Recently on IRC the issue of PDF's usage restrictions flags came up again. Should free software implement them? Here's my take on this. It is unethical for software to restrict what I can do with a PDF file that I have. The PDF format allows, for example, the creator to set a flag that prevents the user from saving a copy of the file. That is not a power that the creator should have, and it is wrong for software to implement the feature. Now, I'm fine with software implementing it as a suggestion: "the creator of the document would prefer it if you didn't save a copy of this file". Preventing the user from saving a copy, however, is not acceptable. I would welcome a Debian policy change to make software that implements these PDF features, or any other form of DRM, in a manner that makes them more than suggestions, unsuitable for the main part of the archive. Such software should go into non-free. Software that is used to directly undermine people's freedom should not be considered pro-freedom, even if it licensed using a DFSG-free license.

Sven has been very busy getting a new Debian ncurses packages into order. With the multiple changes made and the fact that so many things depend on the shared libraries, we decided that it is best to put the first cut of these packages into experimental.

We're now using the Quilt 3.0 source format for our packages, which means that we should have a better handle on the patches, especially as they all now have Dep-3 patch tags .

So what has changed, why should you go over and try the experimental packages?

• First, it will help us with checking the packages actually work, Sven has tried them out (I will be soon) but the more testing the better.
• We've updated the upstream patch level (basically the upstream version) on this set of packages.
• New package! ncurses-examples which are all the example programs that shows you what ncurses can do.

When reviewing packages from mentors.debian.net I like to have them under version control so I can easily compare different versions, write patches and clean up autogenerated files. So when looking at a package I do a

git-import-dsc --download http://mentors.debian.net/debian/pool/main/x/xul-ext-firetray/xul-ext-firetray_0.2.8-1.dsc

This command first fetches the files via dget from devscripts and then imports the upstream sources onto the upstream branch (upstream by default) and the debian modifications onto the debian branch (master by default). I can then invoke git-buildpackage as usual. Since this works incrementally I can import newer versions into the same repository and use git diff to see the changes. If I'd like to hack on a random Debian package (say xul-ext-nostalgy) I usually use:

git-import-dsc --download xul-ext-nostalgy

This uses apt-get source to fetch the source package and imports it into git using the same branch layout as described above so I can use all the familiar git commands to generate patches, etc.. This blog is flattr enabled.

Debian unstable is not conceived as a product for end-users, and for very good reasons. There seems to be some misunderstanding and people trying to push end-users to use unstable. This blog post tries to address the claims raised and put them into proper light. 1. It contains mainly stable versions of the software The critical part here is the term mainly. Yes, developers are advised to only upload packages to unstable that they deem to be suitable for the next stable release. This is no hard requirement though and no one actually playing a gate-keeper enforcing this recommendation. Also, there is a fine amount of packages that follow either VCS snapshots or development branches, and only time can tell how stable those releases actually are. That's actually why there is a delay of several days before a package can transition over to testing. 2. It doesn't break badly every other day That's right but if you look at it, it also means that it does break badly eventually. And if you don't know how to move on from there, including knowing the location of maintainer scripts and how to edit them in those cases, or even resort to a rescue system, you are in troubles. 3. It's the basis of other distributions That's right in itself too, but it doesn't address the fact that those other distributions do put a lot of effort into quality assurance to work around the most nasty and annoying bugs that do affect unstable every other day. 4. It's not inherently less secure than Stable or Testing It's not inherently more secure than stable or testing either. And this is also ignoring the fact that some security bugs don't even get into stable or testing, they only affect unstable and have to get addressed just there. 5. I use it on my main computer This is the reason that you should ignore as most. People using unstable are often enough deeply involved into Debian maintenance, know how to write maintainer scripts, know where they are located in case of troubles, know how to use a rescue system in case of bootloader or kernel troubles. Just think about whether you would consider yourself to be at the same knowledge level than the person who wrote the blog article you read that used this as a convincing argument. Conclusion: Stories always have several truths attached to them. If you feel adventurous, like to understand what's going on and have a tendency for digging into things that go wrong, you most probably are using unstable already anyway and are reporting bugs that you find along your way. If you on the other hand rather just want to use your computer and don't want to work around (smaller) bugs every other day, you rather should stick with the release that is actually provided as purpose to suit the needs of end-users: stable.

/debian permanent link Comments: 19 Flattr this

A new release 0.9.0 of Rcpp is now available at CRAN and has just been uploaded to Debian. As always, sources are also available from my local directory here. With this release, the older API which we have been referring to as the classic Rcpp API has been split off into its own new package RcppClassic to ensure backwards compatibility. Rcpp will now contain only the new API. We also fixes a number a minor bugs and applied a few contributed patches which extended functionality or documentation as detailed below in the NEWS entry:

0.9.0   2010-12-19
    o   The classic API was factored out into its own package RcppClassic which
        is released concurrently with this version.
    o   If an object is created but not initialized, attempting to use
        it now gives a more sensible error message (by forwarding an
        Rcpp::not_initialized exception to R).
    o   SubMatrix fixed, and Matrix types now have a nested ::Sub typedef.
  
    o   New unexported function SHLIB() to aid in creating a shared library on
        the command-line or in Makefile (similar to CxxFlags() / LdFlags()).
    o   Module gets a seven-argument ctor thanks to a patch from Tama Ma.
    o   The (still incomplete) QuickRef vignette has grown thanks to a patch
        by Christian Gunning.
    o   Added a sprintf template intended for logging and error messages.
    o   Date::getYear() corrected (where addition of 1900 was not called for); 
        corresponding change in constructor from three ints made as well.
    o   Date() and Datetime() constructors from string received a missing
        conversion to int and double following strptime. The default format
        string for the Datetime() strptime call was also corrected.
    o   A few minor fixes throughout, see ChangeLog.

Thanks to CRANberries, there is also a diff to the previous release 0.8.9. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

Debian Unstable (also known as sid) is one of the 3 distributions that Debian provides (along with Stable and Testing). It s not conceived as a product for end-users, instead it s the place where contributors are uploading newer packages. Daily. Yes that means that Unstable is a quickly moving target and it s not for everybody. But you can use it and your computer won t explode. 1. It contains mainly stable versions of the software Yes, you read it right. Unstable is not full of development versions of the various software. It happens on some software but then it s usually a conscious decision of the maintainer who believes that this specific version is already better than the previous one. The packages in sid are supposed to migrate to testing, the place where the next Debian stable release is prepared. So maintainers are advised to only upload stuff that is of release quality, the rest should be uploaded to experimental instead. 2. It doesn t break badly every other day Breakages happen but they are not a big deal usually. It has been long time since I could not reboot my computer after an upgrade or since the graphical interface was no longer working. The kind of breakages that you have is that one software stops working, or triggers an annoying bug, or that a few packages are uninstallable. In most cases, you can save yourself by downgrading to the version available in Testing. Or by finding a work-around in the bug tracking system. Or by not upgrading because you have apt-listbugs installed and you have been warned about the problem. 3. It s the basis of other distributions If Debian Unstable was really so bad, it would not be a good basis to build a derivative distribution, isn t it? But Ubuntu and SiduxAptosid (to name only two) are based on Debian Sid. 4. It s not inherently less secure than Stable or Testing High impact security vulnerabilities will usually be quickly fixed in Stable and Unstable. The stable upload is done by the security team while the unstable one is made by the maintainer. Testing will usually get the fix through the package uploaded to Unstable, so testing users get security updates with a delay. For less serious vulnerabilities, it s entirely possible that stable does not get any update at all. In that case, unstable/testing users are better served since they will get the fix with the next upstream version anyway. Of course, it happens that maintainers are busy or that something falls through the cracks, but there are other people watching RC bugs who will fix this if the maintainer doesn t react at all. 5. I use it on my main computer And many other people do the same. And you can do the same if you meet the criteria below:

• you can work on the command-line (enough to downgrade a problematic package, to edit configuration files, etc.);
• you know how to work with APT and multiple distributions in /etc/apt/sources.list;
• you are able to read/write English so that you can read/file bug reports when needed;
• you have another computer connected to the Internet that you can use to lookup documentation (or the bug tracking system, or the support mailing lists) when your usual computer is off-line for a reason that you don t understand.

If you feel you are not ready for the jump, click here to subscribe to this blog (or here via the RSS feed), I ll surely teach some of the required skills in future articles. PS: All that said, if you have a working sid installation, do not upgrade it just before an important presentation, or before a trip. It will always break at the most annoying time. Unless you like to live dangerously, of course.

41 comments Liked this article? Click here. My blog is Flattr-enabled.

A new release 0.9.0 of Rcpp is now available at CRAN and has just been uploaded to Debian. As always, sources are also available from my local directory here. With this release, the older API which we have been referring to as the classic Rcpp API has been split off into its own new package RcppClassic to ensure backwards compatibility. Rcpp will now contain only the new API. We also fixes a number a minor bugs and applied a few contributed patches which extended functionality or documentation as detailed below in the NEWS entry:

0.9.0   2010-12-19
    o   The classic API was factored out into its own package RcppClassic which
        is released concurrently with this version.
    o   If an object is created but not initialized, attempting to use
        it now gives a more sensible error message (by forwarding an
        Rcpp::not_initialized exception to R).
    o   SubMatrix fixed, and Matrix types now have a nested ::Sub typedef.
  
    o   New unexported function SHLIB() to aid in creating a shared library on
        the command-line or in Makefile (similar to CxxFlags() / LdFlags()).
    o   Module gets a seven-argument ctor thanks to a patch from Tama Ma.
    o   The (still incomplete) QuickRef vignette has grown thanks to a patch
        by Christian Gunning.
    o   Added a sprintf template intended for logging and error messages.
    o   Date::getYear() corrected (where addition of 1900 was not called for); 
        corresponding change in constructor from three ints made as well.
    o   Date() and Datetime() constructors from string received a missing
        conversion to int and double following strptime. The default format
        string for the Datetime() strptime call was also corrected.
    o   A few minor fixes throughout, see ChangeLog.

Thanks to CRANberries, there is also a diff to the previous release 0.8.9. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

Russell complains about locked down phones and horrible telcos, in particular about not getting a discount on your monthly contract if you don't get a phone with it. This hasn't been my experience, either in the UK or since I moved to the US. In the UK I ended up on an O2 Simplicity (month-by-month) plan which provided more minutes, SMSes and data allowance that I needed for 20/month (note that I only use the data for the phone, I didn't tether it to my laptop). Originally I chose this because I wasn't sure about coverage where I lived (that's why I was changing provider), but it turned out to be a pretty good deal, saving me at least 10/month over a contract that I'd have been tied into. When the G1 was launched I wasn't interested in moving to T-Mobile, who I knew had no 3G coverage outside of Belfast, so I ended up with one off eBay (received as a gift) and kept my O2 contract. When I moved to the US I signed up to Simple Mobile mainly because I could get a SIM from eBay before I left the UK, and it was PAYG (so the fact I'd no credit record didn't matter) but still included unlimited voice/SMS/data. Significantly more expensive at $60/month than I was used to paying in the UK, but seemed to be the going rate even for a contract. Then the G2 launched back in October. I resisted for 2 or 3 weeks, then decided it had to be mine. The G1's battery was even worse than it had been (to be fair it had lasted 2 years), and although Cyanogen provided Android 2.2 the hardware isn't really up to it. I decided to go with T-Mobile; Simple Mobile use their network, so I knew the coverage would be fine, and I figured a contract was probably a good way to help get a credit record here. Except, the pricing was a bit weird. $200 for the phone with a 2 year $80/month contract or $500 for the phone with an identical contract but no tie in and $60/month. Er, what? I pay up front and I save $180 and I can walk away whenever I want? Ok. As it turned out this was the smart choice. Firstly $60/month means $60/month plus taxes[0], so I was paying more than I paid Simple Mobile. I figured I could bear that for a few months to get the credit history, plus the free network unlock after 3 months. Except then it became clear that international SMS wasn't included in the unlimited SMS (it is with Simple). Most of my SMS is international. Now, T-Mobile have a $5/month bolt on to cover that, but not if you're on their flexpay scheme because they found themselves unable to verify your SSN. So I cancelled the contract after the first month and moved back to Simple. I didn't even need to unlock the phone due to the fact it's the same network (though I have now in preparation for my trip back to the UK over Christmas). Surprisingly T-Mobile didn't try and keep me by sorting out the international SMS bolt on. I guess US mobile customers are used to being screwed over (certainly the pricing suggests that). Er, sorry, that turned into a bit of a T-Mobile rant. My original point was that all of my recent mobile contracts have been month by month, not involved a subsidised phone, and saved me money over being tied in. And even if they hadn't my experiences with the flexibility offered by not being tied in (worries about coverage, discovering the deal isn't as good as you thought) mean that I'm pretty much convinced that contract-free is the way to go anyway. [0] Dear America, for all your complaints about VAT it's not really a lot different from sales tax and at least the prices in shops/online actually include it. Also it's the same everywhere in the country.

We had a small celebration with friends at our local Italian restaurant, Mario's Italiano today, to mark Sarah's 30th birthday. The reason this milestone is a big deal is because, to put it simply, two of her direct ancestors didn't make it to it. The undetermined genetic connective tissue disorder that Sarah has, claimed the lives of her father and paternal aunt when they were both in their twenties, suddenly and two weeks apart from each other. The case made the journal of the American College of Chest Physicians at the time. We're just very fortunate that with advances in medical diagnostic and surgical technology, Sarah was able to be diagnosed and have her aortic aneurysm repaired before it dissected. So there's the heart thing. That's a big deal. Then there's all the other crap that she's had to go through in only thirty years of life. She's had to endure more tragedies and medical challenges than some people have to go through in a lifetime. The fact that she's still such a happy, positive person is a testament to just what an amazing person she is, and it is an honour to be able to call her my wife. So happy birthday Sarah, here's to the next thirty years being a whole lot less eventful than the first thirty, and the last three in particular.

On the last weekend my team and I participated in the Mobile App Camp (link in Lithuanian) organized by Omnitel, which is one of the dominant mobile operators in Lithuania. The event seems to be part of Omnitel's push for increasing the share of smartphones in the Lithuanian market, which has been lagging behind the European trends.

Our team consisted of Povilas Kytra, who is behind the TV.LT project, Mantas Kanaporis from A-Gain and me. In the weekend we built an app that shows the TV programme for the coming day for all the Lithuanian TV channels. (The app is not yet available on the Android Market, but we are working on it.)

Here's a screenshot of the main screen of the application:

When I first started using Debian properly, I played with gpg-agent and pinentry but I didn t really understand the various bash initialisation scripts, and my botched setup annoyed me so much I disabled it again quite quickly (for example, if I left the machine logged in to GDM at home then logged in through SSH, pinentry dialogs stayed in GDM and had to be killed). Now I ve finally had time to get it sorted out, and this is part aide-memoire and part idiot check (hello, lazyweb). The brief:

• gpg-agent should take care of both GPG keys and SSH keys, transparently;
• pinentry should ask with pinentry-gtk2 at home, or at the current SSH session remotely;
• nothing should be left lying around after a session.

I assume appropriate packages are already installed at a minimum, gpg-agent and whichever pinentry you prefer. Installing gpg-agent drops a hook into /etc/X11/Xsession.d, but now there are both gpg-agent and ssh-agent running. Disable ssh-agent because we re going to let gpg-agent take care of everything: in /etc/X11/Xsession.options, comment out the line use-ssh-agent. In /etc/X11/Xsession.d/90gpg-agent, add the enable-ssh-support parameter so the STARTUP variable looks something like:

STARTUP="$GPGAGENT --daemon --sh --enable-ssh-support --write-env-file=$PID_FILE $STARTUP"

and also disable ssh in gnome-keyring:

$ gconftool-2 --set -t bool /apps/gnome-keyring/daemon-components/ssh false

Now (after restarting your X session) you can use ssh-add to import a key into gpg-agent, and you should see a pinentry dialog asking for a password. That takes care of X sessions, but it won t work for SSH. This is where the different bash login scripts matter: ~/.bashrc is run for all non-login shells (that is, ones spawned from e.g. gnome-terminal) but ~/.profile is only run for login shells (that is, ones where you have to authenticate, such as SSH connections). So, at the bottom of ~/.profile, we just start another gpg-agent who doesn t know about X:

eval $(gpg-agent --enable-ssh-support --daemon)

After restarting your SSH session and triggering a key operation, like signing a document, pinentry-curses should ask for your passphrase at the SSH shell and not leave dialogs stranded in the X session. Finally, that second agent shouldn t be left hanging around after we ve finished, or there ll be hundreds of them soon. Fortunately, the PID is recorded in the environment variable GPG_AGENT_INFO, but parsing that is a bit awkward when it s also recorded alone in SSH_AGENT_PID. So in ~/.bash_logout, we just

kill $SSH_AGENT_PID

(Most of the pointers for this setup came from http://wiki.kumina.nl/index.php/Managing_ssh_keys_with_gpg-agent and http://stefaanlippens.net/bashrc_and_others)

I was looking for a simple free software solution which would allow me to have a video call with someone else (I don't care about sound since I've already got that working through Asterisk) and I ended up writing a Gstreamer-based poor man's videoconf solution because I wasn't satisfied with the other options I considered.

EmpathyEmpathy was my first choice since it seems to be the preferred GNOME communication software nowadays.

While the quality of the video was excellent, the latency between New Zealand and Canada was unbearable: a full 6 seconds. I suspect that this is due to the fact that it runs everything through the Google Talk STUN server and I couldn't find how to force it to go directly from one host to the other.

EkigaEkiga was my second choice since I had used it succesfully in the past.

It was not too bad latency-wise, but the quality of the video was not as good as Empathy (it was smaller and choppier). Also, given that it was running over SIP, it was interfering with my VoIP phone.

Direct peer-to-peer streamingGiven that I wasn't gonna use the voice features of these video-conference tools, I figured that there must be an easy way to just stream video from one peer to the other. That's when I thought of looking into Gstreamer (apt-get install gstreamer0.10-tools on Debian/Ubuntu).

To stream video from my webcam onto port 5000, I ran:

gst-launch v4l2src device=/dev/video0 ! videorate ! video/x-raw-yuv,width=640,height=480,framerate=6/1 ! jpegenc quality=30 ! multipartmux ! tcpserversink port=5000

which is the best I could do within 85 kbps (100-120 kbps is about the maximum reliable synchronous bandwidth I get between New Zealand and Canada):

• resolution of 640x480
• 6 frames per second
• jpeg quality of 30%

On the other computer, I simply ran this to connect and display the remote stream:

gst-launch tcpclientsrc host=stream.example.com port=5000 ! multipartdemux ! jpegdec ! autovideosink

Then I swapped the roles around to also stream video the other way around. That's it: two-way peer-to-peer video link!

Small tweaks to the Gstreamer pipelineThere are quite a few plugins that can be used within Gstreamer pipelines.

If you have problems with autovideosink refusing to load (I did on one of the two computers), you can also install the gstreamer0.10-sdl package and replace autovideosink with sdlvideosink:

gst-launch tcpclientsrc host=example.com port=5000 ! multipartdemux ! jpegdec ! sdlvideosink

Another change I had to make on one of the machines was to flip the image coming out of the webcam (which insists on giving me a mirror image instead of acting like a real camera):

gst-launch v4l2src device=/dev/video0 ! videorate ! video/x-raw-yuv,width=640,height=480,framerate=6/1 ! videoflip method=horizontal-flip ! jpegenc quality=30 ! multipartmux ! tcpserversink port=5000

Possible improvementsI got down to about 1-2 seconds of latency, which isn't bad considering the processing to be done and the distance bits have to travel, but I would love to further reduce this.

Using jpegenc was a lot better than theoraenc which added an extra 3-4 seconds of latency. Is there a better codec I should be using?

Another thing I thought of trying was to switch from TCP to UDP. I'm currently using tcpserversink and tcpclientsrc but since I don't care about having a few dropped frames, maybe I should look into the udp and rtp plugins. It seems like it might help but it also seems to be quite a bit more complicated and I have yet to find an easy way to make use of the RTP stack in Gstreamer.

Please feel free leave a comment if you can suggest ways of improving my quick 'n dirty solution.

Choosing a PhoneI was considering renewing my Three contract and getting a HTC Desire HD [1]. What I need is a phone that is good for being a ssh client on 3G networks, has a good camera, and has all the fancy Google Apps.In the comments Lon recommended a Norwegian review of phone cameras which gave the Sony Ericsson Xperia X10 a much better review than the HTC Desire HD [2] the Xperia was the highest rating Android camera phone while the Nokia N8 was the best overall.Also the Xperia is a lot cheaper, I can get it on a $29 monthly cap from Virgin as opposed to $44 per month from Three. So just on hardware and price the Xperia beats the Desire HD.One of the advantages of the Xperia from Virgin being cheap is that my wife and I can use the same model of phone. This avoids having to solve two sets of phone related problems and also allows us to do things like swap batteries between the phones based on who can most conveniently charge their phone.The Need for RootBut one problem with e Xperia is that the CyanogenMod images for rooted Android phones can t be installed on an Xperia because the boot loader hasn t yet been cracked [3]. I would like to run CyanogenMod so I can get wireless proxy support, and support for tunneling IP over DNS, ICMP, and OpenVPN.The Three web site claims that they have 3G phone and net access in Bendigo, but on a recent holiday my phone said that it was roaming all the time and I couldn t get net access. I ended up having to use McDonalds Wifi net access which had ports such as 22 blocked and thus forced me to use Iodine IP over DNS to get proper net access. To avoid having to talk my mother through rebooting servers in future I need to have a mobile ssh client that can use all possible protocols. I could carry my EeePC with me all the time, but sometimes it s good to travel light.OTOH, as I feel compelled to fiddle with all my computers it would probably give me a more reliable mobile experience if I was unable to mess with my phone.Why Buying a Phone Outright Isn t ViableA $29 monthly plan is probably the cheapest plan that will do for anyone who uses a phone regularly, I have had a Three $29 plan for the last four years which allows up to $150 of calls to be made in a month and typically use about $60. So any plan which doesn t have such a cap will have to be no more than half the price of Three on a per-minute basis to compete. If I m going to pay $29 per month ($696 over a 24 month contract) then I can use a free phone. If I was to buy a phone then it would cost at least $500 for anything that I like and maybe a lot more.Buying a phone independently of a contract would about double the cost of owning a phone. It s really not a viable option.Therefore I am compelled to buy a phone that is on offer from a Telco. Things like the Nokia N900 are nice devices but as the Telcos don t offer them I can t consider them.No Discount if you Don t get a PhoneThe annoying thing is that the Telcos don t offer a discount if you choose not to get a phone. Obviously buying the hardware costs them some money, so a $29 cap with a phone included should have a matching offer of something less than $29 if you choose not to get the phone from them. I currently have a $29 per month contract with Three, I can renew that for another two years at the same rate and get a half-decent phone for free or I can renew for two years on a $19 per month contract and get a low-end phone for free but I can t get a price that is lower than $19 per month if I decide to keep my current phone.If Three was to offer such a discount then I would consider buying a phone outright over the net and staying with them. But as it is they don t provide good deals for buying a phone and give me an economic incentive to go to another provider. So I will probably use Virgin when my contract runs out in January.Locked PhonesMany Telcos still sell locked phones on a contract. When that happens it s really difficult to get a phone unlocked as the Telco employees usually aren t very helpful. There are a variety of web sites claiming to generate unlock codes for phones, most seem to charge $10 or more for this service and the free ones have a very small range of phones, so getting the unlock code from the Telco seems to be the only option for a phone at the end of it s contract period as it s not worth enough to justify the $10 expense.While some Telcos sell unlocked phones on plans the ones that lock their phones have a chilling effect on the industry. Most people don t test whether their old phone can be used with a different provider they just throw it out the phone stores conveniently provide bins for old phones that are apparently recycled for some good cause.ConclusionIf you make serious use of mobile phones (EG being ready to fix errors reported by Nagios 24*7) then choosing a new phone and plan is one of the most difficult things there is to do. All the plans are quite complicated and every Telco offers a different set of phones. The Telco web sites are usually poorly done, most of them don t have an option to search for Android phones or for phones with a certain minimum resolution they usually don t even state the resolution and use terms such as WQVGA which don t even have a fixed meaning in pixels. When it comes to choosing a plan most Telcos don t have a clear comparison of the different plans, writing your own spreadsheet comparing plan costs is a good idea.The fact that Telcos such as Virgin and Three/Vodaphone allow free calls to other people using the same company makes it even more tricky. I have to discuss my phone plans with several relatives as there is a good incentive for everyone to use the same provider.I think that we need government regulation on the way that phones are bundled. The market for phones that aren t associated with Telco contracts has been destroyed by the anti-competitive behavior of the Telcos.

• [1] http://etbe.coker.com.au/2010/11/17/mobile-phone-sysadmin/
• [2] http://www.amobil.no/artikler/norges_beste_kameramobil/79081
• [3] http://www.cyanogenmod.com/

(This is part of a series of posts on Why Linux?) Linux, to me as an end user, is cheap. Even taking into account the fact most PCs come with a Windows licence included it's still cheaper for me to run Linux. I paid Steve minimal amounts for my first set of Debian install CDs. These days I can burn my own netinst CDs and pull the rest over the internet. I legally have access to a tremendous range of excellent software, for nothing. All the apps I need are available without having to shell out more. How is that not awesome? I get free updates, both for bugs and also for major features. I'm not left with the option of paying lots of money for the latest and greatest, or dealing with an unsupported old release with known bugs. The counter argument from my Windows using friends is often about how they didn't pay for their Windows updates nor their copy of Office. I'm unimpressed with anyone who tells me Windows is a better option, but is unprepared to pay for it. If you have to illegally obtain it in order for it to compete with Linux then you're not really comparing on equal terms, are you? Also don't tell me that Free software takes away jobs from software engineers and then pirate software, eh? Cost isn't just about the money though. I've put many hours into being involved in Debian. I've provided project resources when I was in a position to do so. I've contributed to the Linux kernel. Not quite the same as paying for it, but I think does indicate that I'm trying to give back a little too. I also accept that at an organisational level the basic cost of the software licences is often negligible compared to things like hardware, training and support. I still think cost is a compelling argument for the home user, and for decisions at an organisational level. As mentioned I realise there are issues with training and support, but I don't believe these costs are any higher than for alternative OSes. Linux also makes it remarkably easy to remotely administer machines, and perform common actions across an entire installed estate, without needing extra bolt ons from 3rd parties. Cost doesn't provide sufficient justification for an individual desktop in an organisation that has site licences for an alternative however (and in fact running Linux requires extra work on my part to do the install and maintenance compared to allowing central IT to manage my machine). So that's not a good enough reason.

Did you ever have to analyse a mail queue in postfix - more than typing 'mailq' in a shell and hope after some time the queue empties? I thought there'd be a command in postfix to show the message body or header from a specified message id. But I was disappointed, only exim has this with -Mvb or -Mvh. Postfix ships postcat to show queue files, but it needs the absolute path to the queued file - so I can use less or cat instead. With a Google search I found a small bash script called pfcat to do this job. It walks through all queue directories and uses postcat to display the message with the given message id. BTW: I found another nice tool called qshape to display all queue content in a table view. This one is shipped with standard postfix.