Next week end, November 24th and 25th, the mini Debconf will take place at EPITA in Paris. Mehdi Dogguy and I worked hard on the organization and on building a great program for this 2012 conference. This mini debconf will cover many subjects like Gnome (both as upstream and downstream), the Release team, how the Linux is packaged in Debian, etc.
The keynote will be "Free software and Debian, 20 years after" by Roberto Di Cosmo. Also, some merchandising will be proposed during the event by the association Debian France. Here is a quick list: Polos

Long-sleeved polo
40 euros - 10 pieces
Sleeveless polo
25 euros - 100 pieces Thanks to Tanguy Ortolo for taking take of the order. Buff

Buff Debian
18 euros - 75 pieces. It is described as Original Multifunctional Headwear. Real life example in video.
I love them! Finally, we will sell two kinds of Debian branded Sticker Portable:

Sticker Debian
1 euro - 100 pieces. They can be also ordered on the it2l website. Thanks to J r me Lemaire for providing these great products. If these products are successful, we will probably produce more for FOSDEM! For more information about the conference:
The official website
Wiki page for subscription

Last week, just before the branching of 3.6.4 was the peak of my recent work in LibreOffice. The motivation from attending the conference and working with others lasted long for me with the help of a few collaborations done since (thanks Caolan and Eike). My own patches were approved just in time to make it to the 3-6 branch, which, along other fixes, pretty much makes 3.6.4 a better release to the RTL users in general, and Hebrew users specifically. Things aren t perfect, but they re improving nicely (on going status is at RTL bugs wiki page). I m also very happy that most RTL related fixes since the conference have been cherry-picked from master to 3-6 branch, so the users won t have to wait till February for the 4.0 release. Last year I had to wait till the 3.5 release to see some of my work fruits, now I can see them in much shorter intervals. The motivation and involvement go hand in hand, each enhances the other for me. I ve started reviewing patches sent for RTL bugs to verify the suggested fixes with private builds and marking bugs as verified when the patches do get accepted. Compering to a month ago, I m now busy with helping the developers do their part instead of just reporting problems. Seeing things go forward (and helping them do so) is much more motivating than waiting for someone to pick the bug report and try solve it. I hope having fast response to their code changes is also motivating for the developers, in that case everyone enjoys the collaboration.
Filed under: LibreOffice

Since converting the phpMyAdmin documentation to new format and updating it's content, I was quite frightened how this will affect translated versions of it. Fortunately giving it more attractive format and more advertisement lead to increased contributions. Since Friday, we already have one language which has reached 100% - Spanish. Mat as Bellone simply did there awesome job and went through all fuzzy and new strings in just few days. Now I'm looking forward to other contributors as well, I hope we will be able to deliver translated manual for phpMyAdmin 4.0 in many languages. I'll definitely try to translate to Czech, but any help with that is welcome.

Filed under: English Phpmyadmin 2 comments Flattr this!

A minor bug-fix release 3.4.4 of Armadillo came out upstream a few days ago. RcppArmadillo, our wrapper for R and Armadillo, is now on CRAN with its corresponding version 0.3.4.4. No R level or interface changes were made and the upstream changes are summarized below.

Changes in RcppArmadillo version 0.3.4.4 (2012-11-15)

• Upgraded to Armadillo release 3.4.4
  • fix for handling complex numbers by sparse matrices
  • fix for minor memory leak by sparse matrices

Courtesy of CRANberries, there is also a diffstat report for 0.3.4.4 relative to 0.3.4.3 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

In case you care, there's a short discussion about when to publish this info and how it relates to DWN on the Debian publicity mailing list. The UDD bugs interface currently knows about the following release critical bugs:

• In Total: 906
  • Affecting wheezy: 401 That's the number we need to get down to zero before the release. They can be split in two big categories:
    • Affecting wheezy and unstable: 258 Those need someone to find a fix, or to finish the work to upload a fix to unstable:
      • 41 bugs are tagged 'patch'. Please help by reviewing the patches, and (if you are a DD) by uploading them.
      • 7 bugs are marked as done, but still affect unstable. This can happen due to missing builds on some architectures, for example. Help investigate!
      • 213 bugs are neither tagged patch, nor marked done. Help make a first step towards resolution!
    • Affecting wheezy only: 143 Those are already fixed in unstable, but the fix still needs to migrate to wheezy. You can help by submitting unblock requests for fixed packages, by investigating why packages do not migrate, or by reviewing submitted unblock requests.

How do we compare to the squeeze cycle ? Yes, how indeed. There's not automatic way to generate this data, yet. Stay tuned for now.

After my last post I got curious about booting the Macbook in EFI mode. Because, well, why not? So I googled for a while and found this blog post which served as a great reference and comment on two other. I understand EFI is somewhat buggy and what works for me might not work for you. So be careful, have a rescue disk at hand. On that topic, I should say that I could not get this to work with Debian sid s 3.2 kernel, but 3.5 from experimental worked fine for me.

Hopefully, you after booting EFI succesfully. My main question before doing this was What works? , I had heard some weird rumours and mentions that you had to run X in framebuffer mode and weird stuff like that. This is bullshit. Everything works fine, with the same standard software and setup. Features:

• GNOME Shell, hence acceleration, works
• All drivers work: wireless, bluetooth, etc.
• No need to force AHCI mode on the disk controller
• No graphic flicker during the boot process
• A feeling of catching up with the times

So, here we go. I am posting my notes in shell-like since I find little benefit in turning them into an essay or something more textual. Good luck! # Install grub-efi-$arch.
# Usually $arch depends on your processor.
# Best bet: you can use OSX to check for this:
#
# $ ioreg -l -p IODeviceTree grep firmware-abi
# It will print EFI32, or EFI64. You know what to do.
$ sudo apt-get install grub-efi-amd64 # Now we need to mount the system EFI partition for GRUB2 to install
# itself on it.
# NOTE: you have to do this, installing the package is NOT enough.
$ sudo mkdir /boot/efi
$ sudo mount /dev/sda1 /boot/efi # Did you get the right partition?
# It should have an EFI directory with an APPLE directory inside.
# DO NOT DELETE THE APPLE DIRECTORY, PRICK.
$ ls /boot/efi
EFI/
$ ls /boot/efi/EFI/
APPLE/ # Create a new directory, debian will do.
# Run grub-install so GRUB2 drops its EFI code in the new directory.
$ sudo mkdir /boot/efi/EFI/debian
$ sudo grub-install
$ ls /boot/efi/EFI/
$ ls /boot/efi/EFI/debian/ # Help rEFIt know this is a GNU/Linux boot.
# rEFIt assigns icons depending on the name of the EFI file, grub64x is not in its list
$ sudo mv /boot/efi/EFI/debian/grub64x.efi /boot/efi/EFI/debian/e.efi # IMPORTANT: Add the proper modeset.
# For EFI-only systems: efi_uga
# For UEFI systems: efi_gop
# Without this you won t have any graphics.
$ sudo vim /etc/default/grub
...
GRUB_VIDEO_BACKEND="efi_uga"
... # Update grub, reboot. Hope that everything worked.
$ sudo update-grub Extra credit
You can clean your MBR from the old GRUB2-pc installation (if you installed it to the hybryd-MBR, like a sane person) from OSX:
$ sudo fdisk -u /dev/disk(tab-to-complete, verify it is the right disk) (Read the man page for your fdisk to be sure -u still means fix MBR while keeping partitions )

Quite on updated schedule (ie. one month later than originally planned), Weblate 1.3 has been released today. It comes with lot of improvements, especially in support of non gettext files, new quality checks and improved performance. Full list of changes for 1.3:

• Compatibility with PostgreSQL database backend.
• Removes languages removed in upstream git repository.
• Improved consistency checks processing.
• Added new checks (BB code, XML markup and newlines).
• Support for optional rebasing instead of merge.
• Possibility to relocate Weblate (eg. to run it under /weblate path).
• Support for manually choosing file type in case autodetection fails.
• Better support for Android resources.
• Support for generating SSH key from web interface.
• More visible data exports.
• New buttons to enter some special characters.
• Support for exporting dictionary.
• Support for locking down whole Weblate installation.
• Checks for source strings and support for source strings review.
• Support for user comments for both translations and source strings.
• Better changes log tracking.
• Changes can now be monitored using RSS.
• Improved support for RTL languages.

You can find more information about Weblate on it's website, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Ready to run appliances will be soon available in SUSE Studio Gallery. Weblate is also being used https://l10n.cihar.com/ as official translating service for phpMyAdmin, Gammu, Weblate itself and others. If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you (this will be decided case by case as my hosting space is limited).

Filed under: English Phpmyadmin Weblate 0 comments Flattr this!

As I've mentioned before, there was a keysigning party in Prague yesterday. Though we were afraid that not so many people will arrive, in the end I have signed 19 keys and assured 13 people for CAcert. The nice outcome of this is that I can now give up to 25 points when assuring for CAcert and altogether we are able to give you over 50 points in the Prague SUSE office. Unfortunately this also reminded me my problems with Unicode on CAcert and quick look at the bug tracker showed me that my bugs 769 and 770 are still open, valid and did not receive any attention :-(. PS: Of course after key signing, we had some fun with the openSUSE beer.

Filed under: English Suse 0 comments Flattr this!

A few months ago, I wrote about my reverse engineering attempt to Logitech Unifying devices. Back then, I concluded my post with big hopes on the future after receiving a document with some part of the specification of the HID++ 2.0 from Logitech. A couple of weeks ago, some of my summer work has been merged to UPower, adding battery support for some Logitech devices. HID++ As I discovered late in my first reverse engineering attempt, Logitech developed a custom HID protocol named HID++. This protocol exists in two versions, 1.0 and 2.0. Some devices talk with version 1 of the protocol (like my M705 mouse) and some others talk with version 2 of the protocol (like my K750 keyboard). Recently, I've been able to be in touch with a Logitech engineer who worked on the Linux support for the Unifying receiver, and he has been really helpful and exposed me some details about this protocol. Logitech made the decision to publish their HID++ specification publicly about a year ago, but still didn't do it. The internal review needed to publish such documents hasn't be done yet. The only published draft is just an extract of the specification, with even some typo in it as I discovered. Some other documents have been recently published, but I didn't have the time to review them. They contains HID++ 1.0 specifications and some details I asked for about the K750 keyboard. UPower support It took me sometime to get a full understanding of the protocol, its different version etc. After reverse engineering my K750 keyboard, I've also reverse engineered the data stream used to get my M705 mouse battery status. I've also received some information about the HID++ 1.0 protocol, so I've been able to discover a bit more on what the packets mean. Most of my discoveries are now used to do proper #define in up-lg-unifying.c so the code makes more sense. My first patch implements a new property for UPower devices, named luminosity, that use with K750 keyboard to report the light level received. The second patch add support for Logitech Unifying devices (over USB only) and should work with at least Logitech M705 and K750 devices. This should be available with the next version of UPower, which should be 0.9.19. So far, Logitech has been kind enough to help me understanding part of the protocol and even sent me a few devices so I can play and test my work with them. Unfortunately, this will probably requires some work and time, and so far Logitech was not able to help with that. There should be enough information out there to at least add support for battery to HID++ 2.0 devices, and probably a few other things too. I hope I'd get the time do this at some point, but feel free to beat me in this race!

As a followup to my toy over at debtree, I ve taken some suggestions from my buddy Arno and found some time in the hotel during a visit for work up to Canada with the Open North folks hacking on open13. The hack is called debfriends (I need to purge personal details from the source tree before I can push it to VCS, since it uses the nm.d.o dump. Sorry, I should have been better about that.) I ve posted a cute front-end on my staging machine to play with over at graph.lucifer.pault.ag There s also an API, but it has all the data in the NM dump exposed. If you re a DD who will treat this data right, I can add a key (it s not secure, like, at all). I m going to publish the source ASAP so that others can stand up an instance if they d like. It s a cute toy, though! In the tradition of using nhandler for my examples, here s an example path from pabs to Nathan: pabs > nhandler

• nhandler was advocated by paultag
• paultag was advocated by mako
• micah was advocated by mako
• pabs was advocated by micah

Also, we re a shockingly well connected bunch. Well done :) Feel free to play with it!

When someone wants to give you access to a Cisco VPN, they might give you a .mobileconfig file. This is apparently used by MacOS and iOS to encapsulate the configuration parameters needed to connect to a VPN. You should be able to connect to it with open source software (such as NetworkManager and vpnc) as long as you have the right configuration. Some helpful soul has tried to give you that configuration, but it s wrapped up in an Apple-specific container. Here s how you rip it open and get the goodies. File format A .mobileconfig appears to contain:

1. Some binary garbage which is safe to ignore
2. An XML document containing the good bits, i.e.:
   1. The local identifier (i.e. IPsec group name)
   2. The remote address (i.e. IPsec gateway host)
   3. The shared secret (base64 encoded)
3. Some more binary garbage which is safe to ignore

and it looks like this:

<plist version="1.0">
<dict>
  <key>PayloadContent</key>
  <array>
    <dict>
      <key>IPSec</key>
      <dict>
        <key>AuthenticationMethod</key>
        <string>SharedSecret</string>
        <key>LocalIdentifier</key>
        <string>LOCAL_IDENTIFIER_HERE</string>
        <key>LocalIdentifierType</key>
        <string>KeyID</string>
        <key>RemoteAddress</key>
        <string>REMOTE_ADDRESS_HERE</string>
        <key>SharedSecret</key>
        <data>
        BASE64_ENCODED_SHARED_SECRET_HERE
        </data>
      </dict>
      <key>IPv4</key>
      <dict>
        <key>OverridePrimary</key>
        <integer>0</integer>
      </dict>
      <key>PayloadDescription</key>
      <string>...</string>
      <key>PayloadDisplayName</key>
      <string>...</string>
      <key>PayloadIdentifier</key>
      <string>...</string>
      <key>PayloadOrganization</key>
      <string>...</string>
      <key>PayloadType</key>
      <string>com.apple.vpn.managed</string>
      <key>PayloadUUID</key>
      <string>...</string>
      <key>PayloadVersion</key>
      <integer>1</integer>
      <key>Proxies</key>
      <dict>
        <key>HTTPEnable</key>
        <integer>0</integer>
        <key>HTTPSEnable</key>
        <integer>0</integer>
        <key>ProxyAutoConfigEnable</key>
        <integer>0</integer>
        <key>ProxyAutoDiscoveryEnable</key>
        <integer>0</integer>
      </dict>
      <key>UserDefinedName</key>
      <string>...</string>
      <key>VPNType</key>
      <string>IPSec</string>
    </dict>
  </array>
  <key>PayloadDescription</key>
  <string>...</string>
  <key>PayloadDisplayName</key>
  <string>...</string>
  <key>PayloadIdentifier</key>
  <string>...</string>
  <key>PayloadOrganization</key>
  <string>...</string>
  <key>PayloadRemovalDisallowed</key>
  <false/>
  <key>PayloadType</key>
  <string>Configuration</string>
  <key>PayloadUUID</key>
  <string>...</string>
  <key>PayloadVersion</key>
  <integer>1</integer>
</dict>
</plist>

The shared secret is base64-encoded, so you can decode it with: $ echo -n 'BASE64_ENCODED_SECRET_HERE' base64 -d Network Manager configuration

1. Make sure you have network-manager-vpnc installed
2. Click the Network Manager icon, select VPN Connections , Configure VPN
3. Create a Cisco-compatible (vpnc) connection
   
   

   Create a Cisco-compatible (vpnc) VPN connection

4. Configure the connection settings as follows:
   
   

   Configure the connection settings

   • Enter the remote address in the Gateway field
   • Enter the local identifier in the Group name field
   • Enter the shared secret in the Group password field
5. To connect, click the Network Manager icon, select VPN Connections , and select the connection you just configured

Good luck and enjoy!

A year ago I wrote in Useful but Unknown Unix Tools: How wdiff and colordiff help to choose the right Swiss Army Knife about using wdiff and colordiff together. Colordiff ed wdiff output looks like this:

$ wdiff foobar.txt barfoo.txt   colordiff
[-foo-]bar fnord
gnarz hurz quux
bla  +foo+  fasel

But if you have colour, why still having these hard to read wdiff markers still in the text? There exists a tool named dwdiff which can do word diffs in colour without textual markers and with even less to type (and without being git diff --color-words ;-). Actually it looks like git diff --color-words, just without the git:

$ dwdiff -c foobar.txt barfoo.txt
foo bar fnord
gnarz hurz quux
bla foo fasel

Another cool thing about dwdiff (and its name giving feature) is that you can defined what you consider whitespace, i.e. which character(s) delimit the words. So lets do the example above again, but this time declare that f is considered the only whitespace character:

$ dwdiff -W f -c foobar.txt barfoo.txt
foo bar bar fnord
gnarz hurz quux
bla foo fasel

dwdiff can also show line numbers:

$ dwdiff -c -L foobar.txt barfoo.txt
   1:1    foo bar fnord
   2:2    gnarz hurz quux
   3:3    bla foo fasel
$ dwdiff -c -L foobar.txt quux.txt
   1:1    foo bar fnord
   1:2    foobar floedeldoe
   2:3    gnarz hurz quux
   3:4    bla foo fasel

(coloured shell screenshots by aha)

I had a dream that many prominent members of the Linux community were dressed in Nazi SS uniforms on the Arctic ice, clubbing baby seals to death. I can t bear the thought that so many Linux people may secretly be Nazi seal murderers, so I demand that they be excluded from any future Linux-related events or conferences that I might wish to attend. My nightmare and the strength of my personal reaction to it are conclusive proof that they are guilty. I can t remember all those who were identified in my dream right now, so I need to be able to point the finger at any time in the future and have any individual shunned excluded and banned from events, conferences, mailing lists, and open source projects in general with any who communicate with them (or, worse, support the evil-doers by pretending to a moderate, reasonable, rational, and/or evidence-based stance) being likewise exiled. After all, remember that even in these enlightened times there is still a seething culture of penguin-supremacism in the Linux community overtly pushing the agenda of the pure penguin race over the pernicious and corrosive evil of seals .this culture of seal-murder must be stamped out by whatever means necessary. Zero tolerance for seal-murder apologists! Thank you for understanding and supporting me in my very real fears. You make the world a safer place for everyone. I had a dream is a post from: Errata

Contrary to http://blog.einval.com/2012/11/13, I ve just been and voted (four times, or 2 2 one for me, one as a proxy) for a police and crime commissioner. I agree with much of what Steve writes and more besides. It s a very bad election. But I voted for three main reasons: Firstly, parties I don t like will get their core vote out anyway. If we don t turn put, the election becomes more about party structures, less the popular wish. Secondly, democracy is imperfect, but it s the best we ve got. I ve interacted with two appointed police authorities and they re rubbish and they re untouchable/unaccountable. Thirdly, I expect policing to be a hot topic here if the government do actually try to build the incinerator that 65,000 people/90+% voted against. Best try to get a non-gov commissioner now, I think. So that s why I ve voted and encouraged other members of our co-op to vote.

No sooner had Henrique de Moraes Holschuh announced the availability of improved microcode updating in Wheezy (sadly non-free) than Stephan Seitz pointed out that this support was broken for systems running Xen, which is unfortunate since as Henrique notes the impact of not applying such an update could be pretty much anything. There's a bit of a saga here: a patch has been available for several years which adds Xen support to the kernel's microcode driver in a nicely self contained way. However the Linux maintainers have decided that loading microcode from the kernel is too late and it should be done earlier (e.g. by the bootloader, or some shim between the bootloader and the kernel) in order to be effective and on that basis rejected the Xen patch implementing the existing kernel based scheme. In the meantime Xen has implemented early loading of microcode via an additional multiboot module passed by the bootloader and this was released in Xen 4.2 in September. (so far I've not seen any progress on making any change to the native side but that's by the by). However Wheezy is going to be shipping with Xen 4.1 which doesn't yet have this support in it. So we are basically faced with two choices, either we backport the support from Xen 4.2 or we apply the kernel patch. The downside of backporting rather than taking the kernel patch is that the backport is non-trivial (although not totally unwieldy) while the kernel patch is pretty much entirely self-contained. The other downside of early the early loading scheme in general (for either native or Xen) is the need to update tools (e.g. update-grub) as well as modifying the microcode packaging to put the files somewhere that the bootloader can see them (so /boot). This certainly can't happen for Wheezy at this stage which basically rules out the backport approach. So, the upshot is I've filed Debian bug #693053 and applied the kernel patch to our tree. It's all a bit unsatisfactory but at least microcode loading will work for users running Xen on Wheezy. With any luck for Jessie the native early microcode loading code will have landed as well and the pain of the infrastructure updates can be shared.

As systems developer I like just being able to pick the most useful Perl modules from CPAN and preferably use quite recent versions of the modules I need. On the other hand our systems administrators prefer running Debian Stable and deploy software using .deb packages. I do not want to maintain hundred small packages by hand. I just want a single package with all needed the Perl dependencies for a given project. After looking at different possibilities for bundling Perl dependencies I finally found Carton written by Tatsuhiko Miyagawa. Given a list of your direct Perl dependencies, Carton can maintain the full dependency chain and manage which versions of each module you deploy. One of the nice features is that you get a reproducible set of versions each time and not just the latest, greatest, and recently broken versions. The initial step is to set Carton up to install the dependencies for my project. This part is done just like the Carton documentation describes. To ensure independence from whatever is available on CPAN I end by calling carton bundle. The next step is to write a small Perl module making sure you load modules from the right place. I end up with installing modules in /usr/lib/projectname. My module looks like this:

package My::Bundle::ProjectName;
=head1 NAME
My::Bundle::ProjectName - Bundle of CPAN modules used by ProjectName
=cut
our $VERSION = 1.0;
use Config;
use Carp;
use Carp::Heavy;  # Ensure that all parts of Carp are loaded before @INC mangling
use File::Spec;
our $BASE = '/usr/lib/projectname';
sub import  
    carp (__PACKAGE__ . " bundle not installed")
        unless -d $BASE;
    unshift @INC, 
        File::Spec->catdir( $BASE, 'lib', 'perl5' ),
        File::Spec->catdir( $BASE, 'lib', 'perl5', $Config archname );
 
1;

Now I need to install this module and all the modules maintained by Carton. I do this by a enhanced Module::Build script:

use v5.10.0;
use strict;
use warnings;
use Module::Build;
my $class = Module::Build->subclass(
    class => "My::Builder",
    code  => q 
        sub ACTION_build  
            my $self = shift;
            $self->SUPER::ACTION_build;
            !system 'carton', 'install', '--cached';
         
        sub ACTION_install  
            my $self = shift;
            $self->SUPER::ACTION_install;
            
            my $installdir = $self->destdir . "/usr/lib/projectname";
            system 'install', '-d', $installdir;
            !system 'cp', '-a', 'local/lib', $installdir;
         
     ,
);
$class->new(
    module_name       => 'My::Bundle::ProjectName',
    dist_author       => 'Peter Makholm <peter>',
    dist_version_from => 'lib/My/Bundle/ProjectName.pm',
    add_to_cleanup    => [ 'local/lib', 'local/bin' ],
)->create_build_script;

The last step is to create a .deb package. This is mostly just a standard debian/ directory using debhelper. You just need to override the dh_perl to pick up the correct binary dependencies. So this is my debian/rules:

#!/usr/bin/make -f
%:
        dh $@
override_dh_perl:
        dh_perl /usr/lib/projectname/lib

That s it. Then all the scripts and applications making up my project just needes to use My::Bundle::ProjectName and the rest is automatic. I am thinking about enhancing Carton to provide individual .deb packages instead of just installing the modules. This would make it quite easy to maintain the packages in a more The Debian Perl Group-compatible way. That could be cool.

With python-dbusmock you can provide mocks for arbitrary D-BUS services for your test suites or if you want to reproduce a bug. However, when writing actual tests for gnome-settings-daemon etc. I noticed that it is rather cumbersome to always have to set up the skeleton of common services such as UPower. python-dbusmock 0.2 now introduces the concept of templates which provide those skeletons for common standard services so that your code only needs to set up the particular properties and specific D-BUS objects that you need. These templates can be parameterized for common customizations, and they can provide additional convenience methods on the org.freedesktop.DBus.Mock interface to provide more abstract functionality like add a battery . So if you want to pretend you have one AC and a half-charged battery, you can now simply do

  def setUp(self):
     (self.p_mock, self.obj_upower) = self.spawn_server_template('upower',  )
  def test_ac_bat(self):
     self.obj_upower.AddAC('mock_AC', 'Mock AC')
     self.obj_upower.AddChargingBattery('mock_BAT', 'Mock Battery', 50.0, 1200)

Or, if your code is not in Python, use the CLI/D-BUS interface, like in shell:

  # start a fake system bus
  eval  dbus-launch 
  export DBUS_SYSTEM_BUS_ADDRESS=$DBUS_SESSION_BUS_ADDRESS
  # start mock upower on the fake bus
  python3 -m dbusmock --template upower &
  # add devices
  gdbus call --system -d org.freedesktop.UPower -o /org/freedesktop/UPower \
      -m org.freedesktop.DBus.Mock.AddAC mock_ac 'Mock AC'
  gdbus call --system -d org.freedesktop.UPower -o /org/freedesktop/UPower \
      -m org.freedesktop.DBus.Mock.AddChargingBattery mock_bat 'Mock Bat' 50.0 1200

In both cases upower --dump or gnome-power-statistics will show you the expected devices (of course you need to run that within the environment of the fake $DBUS_SYSTEM_BUS_ADDRESS, or run the mock on the real system bus as root). Iftikhar Ahmad contributed a template for NetworkManager, which allows you to easily set up ethernet and wifi devices and wifi access points. See pydoc3 dbusmock.templates.networkmanager for details and the test cases for how this looks like in practice. I just released python-dbusmock 0.2.1 and uploaded the new version to Debian experimental. I will sync it into Ubuntu Raring in a few hours.

A (well, now former) coworker let me know about a problem he was having with a Lenovo Thinkcentre M92p. It booted Fedora UEFI install media fine, but after an apparently successful installation refused to boot. UEFI installs on Windows worked perfectly. Secure Boot was quickly ruled out, but this could still have been a number of things. The most interesting observation was that the Fedora boot option didn't appear in the firmware boot menu at all, but Windows did. We spent a little while comparing the variable contents, gradually ruling out potential issues - Linux was writing an entry that had an extra 6 bytes in a structure, for instance[1], and a sufficiently paranoid firmware implementation may have been tripping up on that. Fixing that didn't help, though. Finally we tried just taking the Windows entry and changing the descriptive string. And it broke.

Every UEFI boot entry has a descriptive string. This is used by the firmware when it's presenting a menu to users - instead of "Hard drive 0" and "USB drive 3", the firmware can list "Windows Boot Manager" and "Fedora Linux". There's no reason at all for the firmware to be parsing these strings. But the evidence seemed pretty strong - given two identical boot entries, one saying "Windows Boot Manager" and one not, only the first would work. At this point I downloaded a copy of the firmware and started poking at it. Turns out that yes, actually, there is a function that compares the descriptive string against "Windows Boot Manager" and appears to return an error if it doesn't match. What's stranger is that it also checks for "Red Hat Enterprise Linux" and lets that one work as well.

This is, obviously, bizarre. A vendor appears to have actually written additional code to check whether an OS claims to be Windows before it'll let it boot. Someone then presumably tested booting RHEL on it and discovered that it didn't work. Rather than take out that check, they then addded another check to let RHEL boot as well. We haven't yet verified whether this is an absolute string match or whether a prefix of "Red Hat Enterprise Linux" is sufficient, and further examination of the code may reveal further workarounds. For now, if you want to run Fedora[2] on these systems you're probably best off changing the firmware to perform a legacy boot.

[1] src/include/efi.h: uint8_t padding[6]; /* Emperically needed */, says the efibootmgr source code. Unhelpful.
[2] Or Ubuntu, or Suse, or

comments

We released Rcpp 0.10.0 earlier today. This post will just provide a simple example for one of the smaller new features -- the new namespace for functions from Rmath.h -- and illustrate one of the key features (Rcpp attributes) in passing. R, as a statistical language and environment, has very well written and tested statistical distribution functions providing probability density, cumulative distribution, quantiles and random number draws for dozens of common and not so common distribution functions. This code is used inside R, and available for use from standalone C or C++ programs via the standalone R math library which Debian / Ubuntu have as a package r-mathlib (and which can be built from R sources). User sometimes write code against this interface, and then want to combine the code with other code, possibly even with Rcpp. We allowed for this, but it required a bit of an ugly interface. R provides a C interface; these have no namespaces. Identifiers can clash, and to be safe one can enable a generic prefix Rf_. So functions which could clash such as length or error become Rf_length and Rf_error and are less likely to conflict with symbols from other libraries. Unfortunately, the side-effect is that calling, say, the probability distribution function for the Normal distribution becomes Rf_pnorm5() (with the 5 denoting the five parameters: quantile, mean, std.deviation, lowerTail, logValue). Not pretty, and not obvious. So one of the things we added was another layer of indirection by adding a namespace R with a bunch of inline'd wrapper functions (as well as several handful of unit tests to make sure we avoided typos and argument transposition and what not). The short example below shows this for a simple function taking a vector, and returning its pnorm computed three different ways:

#include <Rcpp.h>
// [[Rcpp::export]]
Rcpp::DataFrame mypnorm(Rcpp::NumericVector x)  
    int n = x.size();
    Rcpp::NumericVector y1(n), y2(n), y3(n);
    for (int i=0; i<n; i++)  
        // the way we used to do this
        y1[i] = ::Rf_pnorm5(x[i], 0.0, 1.0, 1, 0);
        // the way we can do it now
        y2[i] = R::pnorm(x[i], 0.0, 1.0, 1, 0);
     
    // or using Rcpp sugar in one go
    y3 = Rcpp::pnorm(x);
    return Rcpp::DataFrame::create(Rcpp::Named("Rold")  = y1,
                                   Rcpp::Named("Rnew")  = y2,
                                   Rcpp::Named("sugar") = y3);
 

This example also uses the new Rcpp attributes described briefly in the announcement blog post and of course in more detail in the corresponding vignette. Let us just state here that we simply provide a complete C++ function, using standard Rcpp types -- along with one 'attribute' declaration of an export via Rcpp. That's it -- even easier than using inline. Now in R we simply do

R> sourceCpp("mypnorm.cpp")

to obtain a callable R function with the C++ code just shown behind it. No Makefile, no command-line tool invocation -- nothing but a single call to sourceCpp() which takes care of things --- and brings us a compiled C++ function to R just given the source file with its attribute declaration. We can now use the new function to compute the probaility distribution both the old way, the new way with the 'cleaner' R::pnorm(), and of course the Rcpp sugar way in a single call. We build a data frame in C++, and assert that all three variants are the same:

R> x <- seq(0, 1, length=1e3)
R> res <- mypnorm(x)
R> head(res)
      Rold     Rnew    sugar
1 0.500000 0.500000 0.500000
2 0.500399 0.500399 0.500399
3 0.500799 0.500799 0.500799
4 0.501198 0.501198 0.501198
5 0.501597 0.501597 0.501597
6 0.501997 0.501997 0.501997
R> all.equal(res[,1], res[,2], res[,3])
[1] TRUE
R> 

This example hopefully helped to illustrate how Rcpp 0.10.0 brings both something really powerful (Rcpp attributes -- more on this another time, hopefully) and convenient in the new namespace for statistical functions.

TL;DR: There s a summary at the end of the article. Today I wanted to see why a dependency in a .deb-package from an external APT repository changed so that it became uninstallable. While dpkg-deb --info foobar.deb easily shows the control information, the changelog is in the filesystem part of the package. I could extract that one dpkg-deb, too, but I d have to extract either to some temporary directory or pipe it into tar which then can extract a single file from the archive and sent it to STDOUT:

dpkg-deb --fsys-tarfile foobar.deb   tar xOf - ./usr/share/doc/foobar/changelog.Debian.gz   zless

But that s tedious to type. The following command is clearly less to type and way easier to remember:

acat foobar.deb ./usr/share/doc/foobar/changelog.Debian.gz   zless

acat stands for archive cat is part of the atool suite of commands:

als

lists files in an archive.

$ als foobar.tgz
drwxr-xr-x abe/abe           0 2012-11-15 00:19 foobar/
-rw-r--r-- abe/abe          13 2012-11-15 00:20 foobar/bar
-rw-r--r-- abe/abe          13 2012-11-15 00:20 foobar/foo

acat

extracts files in an archive to standard out.

$ acat foobar.tgz foobar/foo foobar/bar
foobar/bar
bar contents
foobar/foo
foo contents

adiff

generates a diff between two archives using diff(1).

$ als quux.zip
Archive:  quux.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2012-11-15 00:23   quux/
       16  2012-11-15 00:22   quux/foo
       13  2012-11-15 00:20   quux/bar
---------                     -------
       29                     3 files
$ adiff foobar.tgz quux.zip
diff -ru Unpack-3594/foobar/foo Unpack-7862/quux/foo
--- Unpack-3594/foobar/foo      2012-11-15 00:20:46.000000000 +0100
+++ Unpack-7862/quux/foo        2012-11-15 00:22:56.000000000 +0100
@@ -1 +1 @@
-foo contents
+foobar contents

arepack

repacks archives to a different format. It does this by first extracting all files of the old archive into a temporary directory, then packing all files extracted to that directory to the new archive. Use the --each (-e) option in combination with --format (-F) to repack multiple archives using a single invocation of atool. Note that arepack will not remove the old archive.

$ arepack foobar.tgz foobar.txz
foobar.tgz: extracted to  Unpack-7121/foobar'
foobar.txz: grew 36 bytes

apack

creates archives (or compresses files). If no file arguments are specified, filenames to add are read from standard in.

aunpack

extracts files from an archive. Often one wants to extract all files in an archive to a single subdirectory. However, some archives contain multiple files in their root directories. The aunpack program overcomes this problem by first extracting files to a unique (temporary) directory, and then moving its contents back if possible. This also prevents local files from being overwritten by mistake.

(atool subcommand descriptions from the atool man page which is licensed under GPLv3+. Examples by me.) I though miss the existence of an agrep subcommand. Guess why? atool supports a wealth of archive types: tar (gzip-, bzip-, bzip2-, compress-/Z-, lzip-, lzop-, xz-, and 7zip-compressed), zip, jar/war, rar, lha/lzh, 7zip, alzip/alz, ace, ar, arj, arc, rpm, deb, cab, gzip, bzip, bzip2, compress/Z, lzip, lzop, xz, rzip, lrzip and cpio. (Not all subcommands support all archive types.) Similar Utilities There are some utilities which cover parts of what atool does, too: Tools from the mtools package Yes, they come from the handle MS-DOS floppy disks tool package, don t ask me why. :-)

uz

gunzips and extracts a gzip d tar d archives

Advantage over aunpack: Less to type. :-)

Disadvantage compared to aunpack: Supports only one archive format.

lz

gunzips and shows a listing of a gzip d tar d archive

Advantage over als: One character less to type. :-)

Disadvantage compared to als: Supports only one archive format.

unp unp extracts one or more files given as arguments on the command line.

$ unp -s
Known archive formats and tools:
7z:           p7zip or p7zip-full
ace:          unace
ar,deb:       binutils
arj:          arj
bz2:          bzip2
cab:          cabextract
chm:          libchm-bin or archmage
cpio,afio:    cpio or afio
dat:          tnef
dms:          xdms
exe:          maybe orange or unzip or unrar or unarj or lha 
gz:           gzip
hqx:          macutils
lha,lzh:      lha
lz:           lzip
lzma:         xz-utils or lzma
lzo:          lzop
lzx:          unlzx
mbox:         formail and mpack
pmd:          ppmd
rar:          rar or unrar or unrar-free
rpm:          rpm2cpio and cpio
sea,sea.bin:  macutils
shar:         sharutils
tar:          tar
tar.bz2,tbz2: tar with bzip2
tar.lzip:     tar with lzip
tar.lzop,tzo: tar with lzop
tar.xz,txz:   tar with xz-utils
tar.z:        tar with compress
tgz,tar.gz:   tar with gzip
uu:           sharutils
xz:           xz-utils
zip,cbz,cbr,jar,war,ear,xpi,adf: unzip
zoo:          zoo

So it s very similar to aunpack, just a shorter command and it supports some more exotic archive formats which atool doesn t support. Also part of the unp package is ucat which does more or less the same as acat, just with unp as backend. dtrx From the man page of dtrx:

In addition to providing one command to extract many different archive types, dtrx also aids the user by extracting contents consistently. By default, everything will be written to a dedicated directory that s named after the archive. dtrx will also change the permissions to ensure that the owner can read and write all those files. Supported archive formats: tar, zip (including self-extracting .exe files), cpio, rpm, deb, gem, 7z, cab, rar, and InstallShield. It can also decompress files compressed with gzip, bzip2, lzma, or compress.

dtrx -l lists the contents of an archive, i.e. works like als or lz. dtrx has two features not present in the other tools mentioned so far:

• It can extract metadata instead of the normal contents from .deb and .gem files.
• It can extract archives recursively, i.e. can extract archives inside of archives.

Unfortunately you can t mix those two features. But you can use the following tool for that purpose: deepfind deepfind is a command from the package strigi-utils and recursively lists files in archives, including archives in archives. I ve already written a detailed blog-posting about deepfind and its friend deepgrep. tardiff tardiff was written to check what changed in source code tarballs from one release to another. By default it just lists the differences in the file lists, not in the files contents and hence works different than adiff. Summary atool and friends are probably the first choice when it comes to DWIM archive handling, also because they have an easy to remember subcommand scheme. uz and lz and the shortest way to extract or list the contents of a .tar.gz file. But nothing more. And you have to install mtools even if you don t have a floppy drive. unp comes in handy for exotic archive formats atool doesn t support. And it s way easier to remember and type than aunpack. dtrx is neat if you want to extract archives in archives or if you want to extract metadata from some package files with just a few keystrokes. For listing all files in recursive archives, use deepfind.