Search Results: "will"

27 February 2013

Julien Danjou: OpenStack Ceilometer and Heat projects graduated

The OpenStack Technical Committee have voted these last weeks about graduation of Heat and Ceilometer, to change their status from incubation to integrated. The details of the discussion can be found in the TC IRC meetings logs for the brave. The results are:

Approve graduation of Heat (to be integrated in common Havana release)?: yes: 10, abstain: 1, no: 1
Approve graduation of Ceilometer (to be integrated in common Havana release)?: yes: 11, abstain: 1

Therefore both projects have been graduated from Incubation to Integrated status. That means that Heat and Ceilometer will be released as part as OpenStack for the next release cycle Havana, due in Autumn 2013. For people being curious, we the Ceilometer team put up a nice wiki page about our status and what we think we were ready to jump. For the curious, The OpenStack Technical Committee charter has some explanations about the incubation and integration process. What about Grizzly? Both projects will be released with Grizzly too, obviously, since they already follow the release process of OpenStack. What about core?

The question that has been raised several times to me is if that means the projects are becoming Core projects. The answer is no, because how to become a Core project is still under discussion and is more a matter for the Board of Directors than the Technical Committee. But this is definitely a step in this direction. Anyway, from a technical point of view, this means both projects are now onboard with other OpenStack components so you can enjoy them!

Matthew Garrett: Supporting third-party keys in a Secure Boot world

It's fairly straightforward to boot a UEFI Secure Boot system using something like Shim or the Linux Foundation's loader, and for distributions using either the LF loader or the generic version of Shim that's pretty much all you need to care about. The physically-present end user has had to explicitly install new keys or hashes, and that means that you no longer need to care about Microsoft's security policies or (assuming there's no exploitable flaws in the bootloader itself) fear any kind of revocation.

But what about if you're a distribution that cares about booting without the user having to install keys? There's several reasons to want that (convenience for naive users, ability to netboot, that kind of thing), but it has the downside that your system can now be used as an attack vector against other operating systems. Do you care about that? It depends how you weigh the risks. First, someone would have to use your system to attack another. Second, Microsoft would have to care enough to revoke your signature. The first hasn't happened yet, so we have no real idea how likely the second is. However, it doesn't seem awfully unlikely that Microsoft would be willing to revoke a distribution signature if that distribution were being used to attack Windows.

How do you avoid that scenario? There's various bits of security work you need to do, but one of them is to require that all your kernel modules be signed. That's easy for the modules in the distribution, since you just sign them all before shipping them. But how about third party modules? There's three main options here:

Don't support third party modules on Secure Boot systems
Have the distribution sign the modules
Have the vendor sign the modules

The first option is easy, but not likely to please users. Or hardware vendors. Not ideal.

The second option is irritating for a bunch of reasons, and a pretty significant one is license-related. If you sign a module, does that mean you're endorsing it in some way? Does signing the nvidia driver mean that you think there's no license concerns? Even ignoring that, how do you decide whose drivers to sign? We can probably assume that companies like AMD and nvidia are fairly reputable, but how about Honest John's Driver Emporium? Verifying someone's identity is astonishingly expensive to do a good job of yourself, and not hugely cheaper if you farm it out to a third party. It's also irritating for the driver vendor, who needs a separate signature for every distribution they support. So, while possible, this isn't an attractive solution.

The third option pushes the responsibility out to other people, and it's always nice to get other people to do work instead of you. The problem then is deciding whose keys you trust. You can push that off to the user, but it's not the friendliest solution. The alternative is to trust any keys that are signed with a trusted key. But what is a trusted key? Having the distribution sign keys just pushes us back to option (2) - you need to verify everyone's identity, and they need a separate signing key for every distribution they support. In an ideal world, there'd be a key that we already trust and which is owned by someone willing to sign things with it.

The good news is that such a key exists. The bad news is that it's owned by Microsoft.

The recent discussion on LKML was about a patchset that allowed the kernel to install new keys if they were inside a PE/COFF binary signed by a trusted key. It's worth emphasising that this patchset doesn't change the set of keys that the kernel trusts - the kernel trusts keys that are installed in your system firmware, so if your system firmware trusts the Microsoft key then your kernel already trusts the Microsoft key. The reasoning here is pretty straightforward. If your firmware trusts things signed by Microsoft, and if a bad person can get things signed by Microsoft, the bad person can already give you a package containing a backdoored bootloader. Letting them sign kernel modules doesn't alter the power they already have over your system. Microsoft will sign PE/COFF binaries, so a vendor would just have to sign up with Microsoft, pay $99 to Symantec to get their ID verified, wrap their key in a PE/COFF binary and then get it signed by Microsoft. The kernel would see that this object was signed by a trusted key and extract and install the key.

Linus is, to put it mildly, unenthusiastic about this idea. It adds some extra complexity to the kernel in the form of a binary parser that would only be used to load keys from userspace, and the kernel already has an interface for that in the form of X509 certificates. The problem we have is that Microsoft won't sign X509 certificates, and there's no way to turn a PE/COFF signature into an X509 signature. Someone would have to re-sign the keys, which starts getting us back to option (2). One way around this would be to have an automated service that accepts PE/COFF objects, verifies that they're signed by Microsoft, extracts the key, re-signs it with a new private key and spits out an X509 certificate. That avoids having to add any new code to the kernel, but it means that there would have to be someone to run that service and it means that their public key would have to be trusted by the kernel by default.

Who would that third party be? The logical choice might be the Linux Foundation, but since we have members of the Linux Foundation Technical Advisory Board saying that they think module signing is unnecessary and that there's no real risk of revocation, it doesn't seem likely that they'll be enthusiastic. A distribution could do it, but there'd be arguments about putting one distribution in a more privileged position than others. So far, nobody's stood up to do this.

A possible outcome is that the distributions who care about signed modules will all just carry this patchset anyway, and the ones who don't won't. That's probably going to be interpreted by many as giving too much responsibility to Microsoft, but it's worth emphasising that these patches change nothing in that respect - if your firmware trusts Microsoft, you already trust Microsoft. If your firmware doesn't trust Microsoft, these patches will not cause your kernel to trust Microsoft. If you've set up your own chain of trust instead, anything signed by Microsoft will be rejected.

What's next? It wouldn't surprise me too much if nothing happens until someone demonstrates how to use a signed Linux system to attack Windows. Microsoft's response to that will probably determine whether anyone ends up caring.

comments

Neil McGovern: Let s get physical

Two recent announcements have been made about how it's viewed that people should interact with each other. Both, in my opinion, are misguided. Firstly, Yahoo's chief executive, Marissa Mayer has announced that she's banning staff from remote working. The idea behind this announcement is simple - that "some of the best decisions and insights come from hallway and cafeteria discussions, meeting new people, and impromptu team meetings". This is absolutely spot on, but the next sentence in the leaked internal memo is more problematic: "Speed and quality are often sacrificed when we work from home".
Don't get me wrong, working remotely has some special challenges, but they are by no means insurmountable. There's lots of tips for people working remotely which will turn into a future blog post at some point (I'm running an internal training about how to do it in a couple of weeks), but three simple rules are stay connected, set a routine and take care of yourself. The second decision is one by Ubuntu to move Ubuntu Developer Summits to a purely online meeting, ditching the physical meeting. This misses the point of conferences. If we simply wanted to listen to talks and presentations, why meet up at all? Webcasts have been around for the last 20 years, and yet conferences still exist. The most important part of a conference isn't the talks, it's the "hallway track" - it's the ability for people to meet up, chat and socialise. Be this an impromptu meeting in the corridor, or over a few nice beers. Without this component, why schedule a time at all? Simply publish a list of talks over the coming 3 months, and anyone can pick the best time for them to attend. At Collabora, many of our engineers work remotely. One of the perks we offer is the ability to attend conferences, and to "touch base" and visit and work from one of the offices. It is important to recognise the importance of collaboration physically and it shouldn't be discounted the way Ubuntu has done. But it should not be seen as a silver bullet to an organisation, like Yahoo seem to be implying. Both extremes are wrong, and a balance must be struck to ensure the best outcome for productivity and innovation. (Title from an iconic 80s song)

26 February 2013

Daniel Pocock: Announcing DruCall, the WebRTC module for Drupal blogs, CMS and e-commerce sites

Building on my previous work to put convenient WebRTC solutions into the hands of the free software community, I've just created the DruCall WebRTC module for Drupal. DruCall makes it easier than ever to put click-to-call functionality into virtually any Drupal powered web site, whether your run a personal blog or the United States Government (the White House really does use Drupal) People wishing to deploy DruCall will need a WebRTC (SIP over WebSockets) capable SIP server. Both the major SIP proxies have this capability now in their development versions:

repro / reSIProcate WebRTC support, just check out and compile the b-webrtc branch
Kamailio WebSockets support, also see Peter Dunkley's blog for more details

Big thanks to the developers of SIPml5 for providing a full SIP over WebSockets client in JavaScript that is used as the foundation for the DruCall project

Olivier Berger: Python RDF example of a MySQL triple store thanks to redland/librdf

In a way similar to my previous post about persisting graphs in PostGreSQL in PHP, here s similar examples for Python and MySQL. Note that you may need to adjust the MySQL table creation templates if you re running librdf version 1.0.16 (see this bug for details). The first example will load data from a file, and store the parsed model to the RDBMS, while the second example, will reload the model and issue queries on it. These examples don t do much, but hopefully, they ll help you get started if you re interested in using redland/librdf. One potential benefit can be interoperability with other languages like PHP, for instance if you re running harvesters in Python that will save facts in a triple store, which is then accessed by a PHP application that will publish a Web interface.

Ian Campbell: LinaroConnect 2013, Talks and Demos

My colleague Stefano Stabellini and I are in the process of putting the finishing touches to our presentation which we are giving at Linaro Connect next week in Hong Kong. If you are there come and see Introducing XEN on ARM on Monday at 1100, we'll be running through some of the basics of Xen as well as the specifics of how this translates into use on the ARM virtualised platform and talking about our status and roadmap going forward. As well as that we will also be demoing Xen on ARM, both on v7 hardware platforms and on v8 software models, at Demo Friday. I've been mostly working on the 64-bit port recently and the initial set of 45(!) patches was committed at the end of last week --- just in time! The other main Xen on ARM event at this years Connect is a keynote on the Tuesday morning given by Lars Kurth (Xen.org Community Manager) and Mark Heath (XenServer VP of Engineering, and also my boss). Other than that my schedule seems to be running over with other interesting sounding talks and there are certainly going to be plenty of people to catch up with, but hopefully I'll have enough time to put the finishing touches onto ARMv8 64-bit guest support in the hackrooms like I'm planning... Looking forward to seeing some of you there!

Craig Small: python and rrdtool

RRDTool is a neat utility for collecting and graphing statistics such as server loads or network traffic. There are two main modules for interfacing with RRDTool files within python; rrdpython and pyrrd. rrdpython is the basic bindings of the rrdtool library within python. The API is very familiar for people who program in C or use the command line tools which for me is both so it works well. However if you were expecting a pythonic API you will be disappointed. As there is the direct binding, you have to have either a pre-compiled module or compile it yourself with librrd-dev package installed. Depending on your setup this could be trivial or a real pain. pyrrd initially looks good as it is a object-orientated style and (supposedly) a pure python code, so no trying to compile things but! Well, the problem is its not pure python. The hooks are there for it (in backend.py) for it to be implemented in python but it falls back to using the external method, and external method is a bunch of Popen calls. pyrrd also does not
support the full set of rrdtool commands. My intention is to keep using rrdpython despite the compile hassle and possibly even use a fancier graph setup such as High Charts though there is a problem with their license for me.

Ingo Juergensmann: Xen problems with VMs on 2.6.32-5-xen-amd64?

On Saturday there were some updates for Squeeze/Stable, for example a new/updated version of Xen hypervisor and kernels were downloaded and installed:

gate:~# dir /var/cache/apt/archives/
base-files_6.0squeeze7_amd64.deb                 libxenstore3.0_4.0.1-5.6_amd64.deb
bind9-host_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb   linux-base_2.6.32-48_all.deb
dbus_1.2.24-4+squeeze2_amd64.deb                 linux-image-2.6.32-5-amd64_2.6.32-48_amd64.deb
dbus-x11_1.2.24-4+squeeze2_amd64.deb             linux-image-2.6.32-5-xen-amd64_2.6.32-48_amd64.deb
firmware-linux-free_2.6.32-48_all.deb            lock
gzip_1.3.12-9+squeeze1_amd64.deb                 openssh-client_1%3a5.5p1-6+squeeze3_amd64.deb
host_1%3a9.7.3.dfsg-1~squeeze9_all.deb           openssh-server_1%3a5.5p1-6+squeeze3_amd64.deb
libbind9-60_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb  openssl_0.9.8o-4squeeze14_amd64.deb
libcups2_1.4.4-7+squeeze3_amd64.deb              partial
libdbus-1-3_1.2.24-4+squeeze2_amd64.deb          perl_5.10.1-17squeeze5_amd64.deb
libdbus-glib-1-2_0.88-2.1+squeeze1_amd64.deb     perl-base_5.10.1-17squeeze5_amd64.deb
libdns69_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb     perl-modules_5.10.1-17squeeze5_all.deb
libisc62_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb     ssh_1%3a5.5p1-6+squeeze3_all.deb
libisccc60_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb   tzdata_2012g-0squeeze1_all.deb
libisccfg62_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb  xen-hypervisor-4.0-amd64_4.0.1-5.6_amd64.deb
libldap-2.4-2_2.4.23-7.3_amd64.deb               xen-linux-system-2.6.32-5-xen-amd64_2.6.32-48_amd64.deb
liblwres60_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb   xenstore-utils_4.0.1-5.6_amd64.deb
libperl5.10_5.10.1-17squeeze5_amd64.deb          xen-utils-4.0_4.0.1-5.6_amd64.deb
libssl0.9.8_0.9.8o-4squeeze14_amd64.deb

Unfortunately this update appears to be problematic on my Xen hosting server. This night it happened the second time that some of the virtual network interfaces disappeared or turned out to be non-working. For example I have two VMs: one running the webserver and one running the databases. Between these two VMs there's a bridge on the dom0 and both VMs have a VIF to that (internal) bridge. What happens is that this bridge becomes inaccessible from within the webserver VM. Sadly there's not much to see in the log files. I just spotted this on dom0:

Feb 26 01:01:29 gate kernel: [12697.907512] vif3.1: Frag is bigger than frame.
Feb 26 01:01:29 gate kernel: [12697.907550] vif3.1: fatal error; disabling device
Feb 26 01:01:29 gate kernel: [12697.919921] xenbr1: port 3(vif3.1) entering disabled state
Feb 26 01:22:00 gate kernel: [13928.644888] vif2.1: Frag is bigger than frame.
Feb 26 01:22:00 gate kernel: [13928.644920] vif2.1: fatal error; disabling device
Feb 26 01:22:00 gate kernel: [13928.663571] xenbr1: port 2(vif2.1) entering disabled state
Feb 26 01:40:44 gate kernel: [15052.629280] vif7.1: Frag is bigger than frame.
Feb 26 01:40:44 gate kernel: [15052.629314] vif7.1: fatal error; disabling device
Feb 26 01:40:44 gate kernel: [15052.641725] xenbr1: port 6(vif7.1) entering disabled state

This corresponds to the number of VMs having lost their internal connection to the bridge. On the webserver VM I see this output:

Feb 26 01:59:01 vserv1 kernel: [16113.539767] IPv6: sending pkt_too_big to self
Feb 26 01:59:01 vserv1 kernel: [16113.539794] IPv6: sending pkt_too_big to self
Feb 26 02:30:54 vserv1 kernel: [18026.407517] IPv6: sending pkt_too_big to self
Feb 26 02:30:54 vserv1 kernel: [18026.407546] IPv6: sending pkt_too_big to self
Feb 26 02:30:54 vserv1 kernel: [18026.434761] IPv6: sending pkt_too_big to self
Feb 26 02:30:54 vserv1 kernel: [18026.434787] IPv6: sending pkt_too_big to self
Feb 26 03:39:16 vserv1 kernel: [22128.768214] IPv6: sending pkt_too_big to self
Feb 26 03:39:16 vserv1 kernel: [22128.768240] IPv6: sending pkt_too_big to self
Feb 26 04:39:51 vserv1 kernel: [25764.250170] IPv6: sending pkt_too_big to self
Feb 26 04:39:51 vserv1 kernel: [25764.250196] IPv6: sending pkt_too_big to self

Rebooting the VMs will result in a non-working VM as it will get paused on creation and Xen scripts complain about not working hotplug scripts and Xen logs shows this:

[2013-02-25 13:06:34 5470] DEBUG (XendDomainInfo:101)
XendDomainInfo.create(['vm', ['name', 'vserv1'], ['memory', '2048'],
['on_poweroff', 'destroy'], ['on_reboot', 'restart'], ['on_crash',
'restart'], ['on_xend_start', 'ignore'], ['on_xend_stop', 'ignore'],
['vcpus', '2'], ['oos', 1], ['bootloader', '/usr/lib/xen-4.0/bin/pygrub'],
['bootloader_args', ''], ['image', ['linux', ['root', '/dev/xvdb '],
['videoram', 4], ['tsc_mode', 0], ['nomigrate', 0]]], ['s3_integrity', 1],
['device', ['vbd', ['uname', 'phy:/dev/lv/vserv1-boot'], ['dev', 'xvda'],
['mode', 'w']]], ['device', ['vbd', ['uname', 'phy:/dev/lv/vserv1-disk'],
['dev', 'xvdb'], ['mode', 'w']]], ['device', ['vbd', ['uname',
'phy:/dev/lv/vserv1-swap'], ['dev', 'xvdc'], ['mode', 'w']]], ['device',
['vbd', ['uname', 'phy:/dev/lv/vserv1mirror'], ['dev', 'xvdd'], ['mode',
'w']]]])
[2013-02-25 13:06:34 5470] DEBUG (XendDomainInfo:2508)
XendDomainInfo.constructDomain
[2013-02-25 13:06:34 5470] DEBUG (balloon:220) Balloon: 2100000 KiB free;
need 16384; done.
[2013-02-25 13:06:34 5470] DEBUG (XendDomain:464) Adding Domain: 39
[2013-02-25 13:06:34 5470] DEBUG (XendDomainInfo:2818)
XendDomainInfo.initDomain: 39 256
[2013-02-25 13:06:34 5781] DEBUG (XendBootloader:113) Launching bootloader
as ['/usr/lib/xen-4.0/bin/pygrub', '--args=root=/dev/xvdb  ',
'--output=/var/run/xend/boot/xenbl.6040', '/dev/lv/vserv1-boot'].
[2013-02-25 13:06:39 5470] DEBUG (XendDomainInfo:2845)
_initDomain:shadow_memory=0x0, memory_static_max=0x80000000,
memory_static_min=0x0.
[2013-02-25 13:06:39 5470] INFO (image:182) buildDomain os=linux dom=39
vcpus=2
[2013-02-25 13:06:39 5470] DEBUG (image:721) domid	    = 39
[2013-02-25 13:06:39 5470] DEBUG (image:722) memsize	    = 2048
[2013-02-25 13:06:39 5470] DEBUG (image:723) image	    =
/var/run/xend/boot/boot_kernel.xj7W_t
[2013-02-25 13:06:39 5470] DEBUG (image:724) store_evtchn   = 1
[2013-02-25 13:06:39 5470] DEBUG (image:725) console_evtchn = 2
[2013-02-25 13:06:39 5470] DEBUG (image:726) cmdline	    =
root=UUID=ed71a39f-fd2e-4035-8557-493686baa151 ro root=/dev/xvdb
[2013-02-25 13:06:39 5470] DEBUG (image:727) ramdisk	    =
/var/run/xend/boot/boot_ramdisk.QavuAo
[2013-02-25 13:06:39 5470] DEBUG (image:728) vcpus	    = 2
[2013-02-25 13:06:39 5470] DEBUG (image:729) features	    =
[2013-02-25 13:06:39 5470] DEBUG (image:730) flags	    = 0
[2013-02-25 13:06:39 5470] DEBUG (image:731) superpages     = 0
[2013-02-25 13:06:40 5470] INFO (XendDomainInfo:2367) createDevice: vbd :
 'uuid': '04d99772-cf27-aecf-2d1b-c73eaf657410', 'bootable': 1, 'driver':
'paravirtualised', 'dev': 'xvda', 'uname': 'phy:/dev/lv/vserv1-boot',
'mode': 'w' 
[2013-02-25 13:06:40 5470] DEBUG (DevController:95) DevController: writing
 'virtual-device': '51712', 'device-type': 'disk', 'protocol':
'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend':
'/local/domain/0/backend/vbd/39/51712'  to
/local/domain/39/device/vbd/51712.
[2013-02-25 13:06:40 5470] DEBUG (DevController:97) DevController: writing
 'domain': 'vserv1', 'frontend': '/local/domain/39/device/vbd/51712',
'uuid': '04d99772-cf27-aecf-2d1b-c73eaf657410', 'bootable': '1', 'dev':
'xvda', 'state': '1', 'params': '/dev/lv/vserv1-boot', 'mode': 'w',
'online': '1', 'frontend-id': '39', 'type': 'phy'  to
/local/domain/0/backend/vbd/39/51712.
[2013-02-25 13:06:40 5470] INFO (XendDomainInfo:2367) createDevice: vbd :
 'uuid': 'e46cb89f-3e54-41d2-53bd-759ed6c690d2', 'bootable': 0, 'driver':
'paravirtualised', 'dev': 'xvdb', 'uname': 'phy:/dev/lv/vserv1-disk',
'mode': 'w' 
[2013-02-25 13:06:40 5470] DEBUG (DevController:95) DevController: writing
 'virtual-device': '51728', 'device-type': 'disk', 'protocol':
'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend':
'/local/domain/0/backend/vbd/39/51728'  to
/local/domain/39/device/vbd/51728.
[2013-02-25 13:06:40 5470] DEBUG (DevController:97) DevController: writing
 'domain': 'vserv1', 'frontend': '/local/domain/39/device/vbd/51728',
'uuid': 'e46cb89f-3e54-41d2-53bd-759ed6c690d2', 'bootable': '0', 'dev':
'xvdb', 'state': '1', 'params': '/dev/lv/vserv1-disk', 'mode': 'w',
'online': '1', 'frontend-id': '39', 'type': 'phy'  to
/local/domain/0/backend/vbd/39/51728.
[2013-02-25 13:06:40 5470] INFO (XendDomainInfo:2367) createDevice: vbd :
 'uuid': 'e2d61860-7448-1843-3935-6b63c5d2878e', 'bootable': 0, 'driver':
'paravirtualised', 'dev': 'xvdc', 'uname': 'phy:/dev/lv/vserv1-swap',
'mode': 'w' 
[2013-02-25 13:06:40 5470] DEBUG (DevController:95) DevController: writing
 'virtual-device': '51744', 'device-type': 'disk', 'protocol':
'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend':
'/local/domain/0/backend/vbd/39/51744'  to
/local/domain/39/device/vbd/51744.
[2013-02-25 13:06:40 5470] DEBUG (DevController:97) DevController: writing
 'domain': 'vserv1', 'frontend': '/local/domain/39/device/vbd/51744',
'uuid': 'e2d61860-7448-1843-3935-6b63c5d2878e', 'bootable': '0', 'dev':
'xvdc', 'state': '1', 'params': '/dev/lv/vserv1-swap', 'mode': 'w',
'online': '1', 'frontend-id': '39', 'type': 'phy'  to
/local/domain/0/backend/vbd/39/51744.
[2013-02-25 13:06:40 5470] INFO (XendDomainInfo:2367) createDevice: vbd :
 'uuid': 'd314a46e-1ce9-0e8d-b009-3f08e29735f5', 'bootable': 0, 'driver':
'paravirtualised', 'dev': 'xvdd', 'uname': 'phy:/dev/lv/vserv1mirror',
'mode': 'w' 
[2013-02-25 13:06:40 5470] DEBUG (DevController:95) DevController: writing
 'virtual-device': '51760', 'device-type': 'disk', 'protocol':
'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend':
'/local/domain/0/backend/vbd/39/51760'  to
/local/domain/39/device/vbd/51760.
[2013-02-25 13:06:40 5470] DEBUG (DevController:97) DevController: writing
 'domain': 'vserv1', 'frontend': '/local/domain/39/device/vbd/51760',
'uuid': 'd314a46e-1ce9-0e8d-b009-3f08e29735f5', 'bootable': '0', 'dev':
'xvdd', 'state': '1', 'params': '/dev/lv/vserv1mirror', 'mode': 'w',
'online': '1', 'frontend-id': '39', 'type': 'phy'  to
/local/domain/0/backend/vbd/39/51760.
[2013-02-25 13:06:40 5470] DEBUG (XendDomainInfo:3400) Storing VM details:
 'on_xend_stop': 'ignore', 'shadow_memory': '0', 'uuid':
'04541225-6d3c-3cae-a4c4-0b6d4ccfac7a', 'on_reboot': 'restart',
'start_time': '1361794000.37', 'on_poweroff': 'destroy', 'bootloader_args':
'', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count':
'0', 'vcpus': '2', 'vcpu_avail': '3', 'bootloader':
'/usr/lib/xen-4.0/bin/pygrub', 'image': "(linux (kernel ) (args
'root=/dev/xvdb  ') (superpages 0) (tsc_mode 0) (videoram 4) (pci ())
(nomigrate 0) (notes (HV_START_LOW 18446603336221196288) (FEATURES
'!writable_page_tables pae_pgdir_above_4gb') (VIRT_BASE
18446744071562067968) (GUEST_VERSION 2.6) (PADDR_OFFSET 0) (GUEST_OS linux)
(HYPERCALL_PAGE 18446744071578882048) (LOADER generic) (SUSPEND_CANCEL 1)
(PAE_MODE yes) (ENTRY 18446744071584289280) (XEN_VERSION xen-3.0)))",
'name': 'vserv1' 
[2013-02-25 13:06:40 5470] DEBUG (XendDomainInfo:1804) Storing domain
details:  'console/ring-ref': '2143834', 'image/entry':
'18446744071584289280', 'console/port': '2', 'store/ring-ref': '2143835',
'image/loader': 'generic', 'vm':
'/vm/04541225-6d3c-3cae-a4c4-0b6d4ccfac7a',
'control/platform-feature-multiprocessor-suspend': '1',
'image/hv-start-low': '18446603336221196288', 'image/guest-os': 'linux',
'cpu/1/availability': 'online', 'image/virt-base': '18446744071562067968',
'memory/target': '2097152', 'image/guest-version': '2.6', 'image/pae-mode':
'yes', 'description': '', 'console/limit': '1048576', 'image/paddr-offset':
'0', 'image/hypercall-page': '18446744071578882048',
'image/suspend-cancel': '1', 'cpu/0/availability': 'online',
'image/features/pae-pgdir-above-4gb': '1',
'image/features/writable-page-tables': '0', 'console/type': 'xenconsoled',
'name': 'vserv1', 'domid': '39', 'image/xen-version': 'xen-3.0',
'store/port': '1' 
[2013-02-25 13:06:40 5470] DEBUG (DevController:95) DevController: writing
 'protocol': 'x86_64-abi', 'state': '1', 'backend-id': '0', 'backend':
'/local/domain/0/backend/console/39/0'  to
/local/domain/39/device/console/0.
[2013-02-25 13:06:40 5470] DEBUG (DevController:97) DevController: writing
 'domain': 'vserv1', 'frontend': '/local/domain/39/device/console/0',
'uuid': 'c8819aed-c78f-02b8-0ef7-1600abd15add', 'frontend-id': '39',
'state': '1', 'location': '2', 'online': '1', 'protocol': 'vt100'  to
/local/domain/0/backend/console/39/0.
[2013-02-25 13:06:40 5470] DEBUG (XendDomainInfo:1891)
XendDomainInfo.handleShutdownWatch
[2013-02-25 13:06:40 5470] DEBUG (DevController:139) Waiting for devices
vif2.
[2013-02-25 13:06:40 5470] DEBUG (DevController:139) Waiting for devices
vif.
[2013-02-25 13:06:40 5470] DEBUG (DevController:139) Waiting for devices
vscsi.
[2013-02-25 13:06:40 5470] DEBUG (DevController:139) Waiting for devices
vbd.
[2013-02-25 13:06:40 5470] DEBUG (DevController:144) Waiting for 51712.
[2013-02-25 13:06:40 5470] DEBUG (DevController:628) hotplugStatusCallback
/local/domain/0/backend/vbd/39/51712/hotplug-status.

From my point of view, either Xen hypervisor or the kernel seems to be broken, but it's hard to tell for me. Maybe it would be easier to update the system from Squeeze to Wheezy and get rid off this problem that way? On the other hand this would solve the problem. Are there any other people experiencing problems with that latest update of Xen and kernel? UPDATE: Bug #701744 filed. UPDATE 2: Downgrading kernel and hypervisor on dom0 to the following packages from snapshot.debian.org seems to have solved that problem.

xen-hypervisor-4.0-amd64_4.0.1-5.4_amd64.deb
linux-image-2.6.32-5-xen-amd64_2.6.32-46_amd64.deb

Note that I haven't tested yet with the newest kernel update from DSA-2632-1. UPDATE 3: After running with older versions of hypervisor and kernel I now have upgraded the hypervisor to xen-hypervisor-4.0-amd64_4.0.1-5.6_amd64.deb and rebooted. Let's see whether it's running stable or not. If not it's the hypervisor, if yes, it's the kernel. UPDATE 4: apparently it's the kernel which is buggy. And the kernel from DSA-2632-1 is affected as well. So, the current workaround is to downgrad to linux-image-2.6.32-5-xen-amd64_2.6.32-46_amd64.deb.

Kategorie:

Tags:

25 February 2013

Francesca Ciceri: Three Harry Potter crossovers you want to read

...at least, if you are into fan fictions and Harry Potter. I love fan fictions, and I'll shamelessly admit that I like Harry Potter too, but I usually avoid crossovers. That is because however interesting and exciting is - in theory - to have your favourite characters from different fandoms together, it often ends bad. So, it's a public duty of every fanfic reader to recommend good crossovers, if they stumble upon them. Here are some I found recently (all the quotes are from the stories' summaries).

Not Quite A Maia (Harry Potter x Lord of the Rings)
"Middle Earth has a problem: Gandalf the Grey is AWOL in Time & Space after destroying the Balrog of Khazad-d m. But who will take his place in the Quest to defeat the Dark Lord Sauron? Not who you think..."
This is a very well written fic, and one of the best HP/LOTR crossover I've ever read. Bonus points for not featuring main characters of HP-verse (someone else will have to save the day in Middle Earth, not the usual Golden Trio). And if there are some minor glitches in the plot (especially in the last chapters), the author has undoubtedly a knack for hilarious situations. If you can read through Augusta's chapters without laughing out loud you are not human. Really.
Evil Be Thou My Good (Harry Potter x Hellraiser)
"Nine years ago Vernon Dursley brought home a certain puzzle box. His nephew managed to open it, changing his destiny. Now, in the midst of Voldemort's second rise, Harry Potter has decided to recreate the Lament Configuration... and open it... again."
Even if I am a notorious addicted to horror and b-movies (and more so to horror b-movies) I've never seen Hellraiser. But you don't have to, to enjoy this story. It's well written, it explains the backstory for the Hellraiser universe (or at least what you need to know about it) and features a refreshing Dark!Harry. The only thing I didn't like is a slightly OOC (Out Of Character) Voldemort, who's described in canon as intelligent and cunning and here seems to me a bit gullible. But apart from that, the story is worth a read.
That Which Holds The Image (Harry Potter x Doctor Who).
"Harry Potter faces a boggart that doesn't turn into a Dementor or even Voldermort, but into a horror from his childhood. Now the boggart isn't even a boggart anymore. There's no imitation. That which holds the image of an Angel, becomes itself an Angel."
I have a soft spot for the Weeping Angels. They are pure evil genius. If you don't know what I'm talking about stop reading now and go watch Doctor Who episode 3x10 "Blink". It's really a masterpiece. So Weeping Angels in Potterverse would be awesome even if the plot of this story weren't as good as actually is. The only thing to make it better would be to have 10th Doctor instead of 11th. But then, you can't have everything, I guess.

If you read any of these and want to talk about it or if you have more crossovers to recommend, drop me a mail. For more fan fiction recommendations, I suggest you to check this detailed index on tvtropes.org.

Russ Allbery: Review: The Hare with Amber Eyes

Review: The Hare with Amber Eyes, by Edmund de Waal

Publisher:	Farrar, Straus and Giroux
Copyright:	2010
ISBN:	0-374-10597-9
Format:	Kindle
Pages:	351

A netsuke ( ) is a miniature sculpture, often made from ivory or boxwood, generally rounded and no larger than a matchbox. They were a part of Japanese clothing starting from the 17th century: traditional garments had no pockets, but people needed a way to carry small objects around with them, so they wore pouches or small boxes attached to a cord. The cord could then be passed under the sash of the garment and be held in place there by a netsuke attached to the end of the cord, since the netsuke was too large to slip under the sash. The hare with amber eyes is one of 264 netsuke. It, and all the rest, belonged to Edmund de Waal's uncle Iggie. They sat in a vitrine in his home in Japan, where de Waal went for a two-year scholarship and was working on a book on japonisme: the passionate and creative Western misunderstanding of Japan. When Iggie later died in 1991, de Waal inherited the collection. This book is his attempt to tell their history. The Hare with Amber Eyes falls into the genre of family memoir or biography, but it's one told from an unusual angle. Edmund de Waal is a potter by profession: an artist who creates physical objects that then pass from his studio out into the world. The physicality of the netsuke their occupation of time and space and their passsage through the world fascinates him, and he tries to express that fascination to the reader and uses it as a focal point for the memoir. He tells, here, the story of his own family, but he does so via the netsuke, following them from their purchase in Paris in 1871 through one branch of the family and then another until they reach Iggie and, finally, him. Along the way, he tries to put them into context. At first, the context is primarily artistic, and the early parts of this book are something of a tour of late 19th century artistic culture in Paris. But, as the story continues, the connection becomes more emotional and the context becomes more complicated. Anyone who is a fan of the TV program Antiques Roadshow or any similar effort to trace the history of specific objects will see the immediate appeal in this presentation. Antiques are fascinating in their mute path through lives and history; attempting to reconstruct the rooms they've sat in and the conversations they've witnessed is something I've always found intriguing. But it's also very difficult to do. Most of our objects, and most owners of objects, simply don't have that sort of documentation. I was wondering, at the start of this book, how de Waal was going to manage to construct his history. The answer is that his family is, or at least was, famous enough to have letters, papers, and history in libraries throughout Europe. That's what makes the parts of the book before Iggie's living memory possible. It's a bit disappointing, if unsurprising; de Waal doesn't have any special technique to find hidden history, and most objects could never have this sort of history written about them. The first owner of the netsuke in his family is Charles Ephrussi, a critic, historian, and art historian who was a patron of the Impressionists and who was important enough in Parisian art culture that he is the inspiration of a character in Marcel Proust's Remembrance of Things Past. This means that quite a lot of this book is about people who moved through social circles and lived in ways that I find quite foreign, which made it hard to invest too much in the first part of the book. The Ephrussi family was originally from Odessa and made a fortune as grain merchants some time prior to the start of the history, and by the time de Waal opens their story, they have vast inherited wealth. There is no sign in the book that Charles needs to work in any way. The first part of the book paints a picture of the culture of art patronage partly via descriptions of Charles's rooms and personal collection, to which the netsuke were added during the surge in interest in Japanese art in Europe produced by the opening of Japan during the Meiji period. It's interesting, but for me in a distant and intellectual way. (People with a deeper interest in Impressionist art or other late 19th century art may have a different reaction.) De Waal writes at length at the start of this book about avoiding sentimentality and trying to take an honest and immersive look at the history of the netsuke. The emotional intensity that he brings to this goal seemed a bit odd, since little in the early part of the book seems to tempt towards sentimentality. But then the Dreyfus Affair enters Charles's story, followed by his gift of the vitrine and the netsuke to a cousin in Austria as a marriage gift, and I started to realize the implications of following a possession of a prominent Jewish family in Europe during the first half of the twentieth century. This is the sort of non-fiction book that you can spoil, so I won't give a detailed description of the rest of the book. But there is a specific event in the history of the netsuke that de Waal tells that would be the story most people would start with, and I now understand the sentimentality that de Waal was trying to avoid. It's a great story, but one of the things I like about this book is that de Waal puts it into a broader context and complicates the story. He includes the long periods of time when the netsuke were mostly ignored, he ties them into the history of western obsession with (and exploitation of) Japanese art, and he shows how the sturdy nature of the netsuke, meant for physical handling rather than display, led to them creating the emotional connection that made them important to the family. It's a book that deals with anti-Semitism with accuracy and specifics without being either lurid or overwhelming. It's a book that traces a line through one of the most overwritten and well-trod areas of history and manages to stay on its own path. It's a book that made me angry, and made me cry, and then made me step back and think about those emotions in a broader context and think about the benefits of leaving those emotions behind. I've never read a book quite like this before. It's slow in places (particularly if one is not very interested in art history), utterly compelling in others, and carefully avoids any simple themes or grand stories. It's entangled with some of the darkest events of recent history, but de Waal stays tightly focused on the family through the lens of the netsuke, which is valuable for keeping the story from drifting into well-worn and less unique material. I was not as completely taken by this book as the review that got me to buy it (it's an excellent review, better than this one). But I'm glad that I read it, and I feel like I now know more about the world. Rating: 7 out of 10

Matthew Palmer: libvirt errors that are not helpful

Since there is absolutely zero Google juice on this problem, here s some hints in case someone else is out there beating their heads on their keyboard in frustration. The problem: when trying to define a storage pool (or 90+% of other virsh commands), you get this sort of result:

# virsh pool-define /tmp/pooldef
error: Failed to define pool from /tmp/pooldef
error: this function is not supported by the connection driver: virStoragePoolDefineXML

Or this:

# virsh pool-create /tmp/pooldef
error: Failed to create pool from /tmp/pooldef
error: this function is not supported by the connection driver: virStoragePoolCreateXML

Not helpful at all. The problem is (or, at least it was for me) that I have both KVM and virtualbox installed (I prefer KVM, but vagrant uses virtualbox and I m playing around with it). It would appear that libvirt is preferring to use virtualbox over KVM, which is stupid because virtualbox doesn t appear to be fully supported (as evidenced by the extensive set of functions that are not supported by the virtualbox connection driver). The solution: edit /etc/libvirt/libvirt.conf, and ensure that the following line is defined:

uri_default = "qemu:///system"

This will tell libvirt to use KVM (via qemu) rather than virtualbox, and you can play with pools to your hearts content.

24 February 2013

Andrew Pollock: [tech] On owning a Nissan Leaf

I'll soon be disposing of the Nissan Leaf that we leased a few months ago, so I thought it a useful exercise to write about my experiences with it. I am not a car man. I am a gadget man. For me, driving is a means to an end, and I'm much more interested in what I call the "cabin" experience, than the "driving" experience, so this is going to slanted much more that way. That said, I found the Nissan Leaf a fun car to drive, from my limited experiences of having fun driving cars. I liked how responsive it was when you put your foot down. It has two modes of operation, "D" and "Eco". I've actually taken to driving it in "Eco" mode most of the time, as it squeezes more range out of the batteries, but occasionally I'll pop it back into "D" to have a bit of fun. The main difference between the two modes, from a driving perspective, is it seems to limit the responsiveness of the accelerator. In "Eco" mode it feels more like you're driving in molasses, whereas in "D" mode, when you put your foot down, it responses instantly. "D" is great for dragging people off at the lights. It's a very zippy little car in "D" mode. It feels lighter. I've noticed that it has a bit of a penchant for over steering. Or maybe that's just my driving. If I have floored it a bit to take a right turn into oncoming traffic, I've noticed slight over steering. That's about all the driving type "real car stuff" I'll talk about. Now to the driver's "cabin experience". It's absolutely fabulous. I love sitting in the driver's seat of this car. Firstly, the seat itself is heated (in fact all of them are). As is the steering wheel. Nissan has gone to great lengths to allow you to avoid needing to run the car's heating system to heat the car, as doing so immediately drops at least 10 miles off the range. Unfortunately I found the windscreen had a tendency to fog up in the winter rainy periods, so I'd have to intermittently fire up the air conditioning to defog the windscreen. Of course, in the summer months, you're going to want to run the AC to cool down, so the range hit in that situation is unavoidable. I've only had this car from late Autumn until late Winter so far, so that hasn't been an issue I've had to contend with. The dashboard is all digital, and looks relatively high tech, which appeals to my inner geek. It's a dashboard. It tells you stuff. The stuff you'd expect it to tell you. Enough said. The audio system is nice. It supports Bluetooth audio, so one can be streaming Pandora from one's phone, through the sound system, for example. Or listening to audio from the phone. There's also a USB port, and it will play various audio files from USB storage. I found the way it chose to sort the files on a USB stick to be slightly surprising though. I haven't invested the time to figure out how I should be naming files so that they play in the order I expect them to play. The ability to play from USB storage compensates nicely for the fact that it only has a single CD player. (We have a 6 disc stacker in our 2006 Prius). The car also came with a 3 month free trial of Sirius XM satellite radio. This was fun. The only dance music FM station in the Bay Area has a weak signal in Mountain View, and I hate dance music with static, whereas there was an excellent electronic music station that I could listen to in glorious high definition. As long as I wasn't driving under a bridge. There's no way I'd pay money for satellite radio, but it was a fun gimmick to try out. The navigation system is really, really good. I haven't bothered using Google Maps on my phone at all. It gives such good spoken directions, that you don't even need to have the map on the screen. It names all the streets. I couldn't figure out a way to get distances in metric. The telematics service, Carwings, is probably my favourite feature. This is what really makes it feel like a car of the future. Through the companion Android application, I can view the charging status (if it's plugged in) or just check the available range (if it's not plugged in). From a web browser, I can plan a route, and push the route to the vehicle's navigation system. If the car is plugged in, I can also remotely turn on the vehicle's climate control system, pre-warming or cooling the car. It's a little thing, but the door unlocking annoyed me a little bit. I'm used to the Prius, where if you unlock the boot (that's trunk, Americans), or the front passenger door, all the doors unlocked. This was a convenient way of unlocking the car for multiple people as you approached it. With the Leaf, unlocking the boot only unlocks the boot. Unlocking the front passenger door only unlocks that door. It requires a second unlock action to unlock all the doors. I've found this to be slightly cumbersome when trying to unlock the car for everyone all at once, quickly (like when it's raining). Another minor annoyance is the headlights. I've gotten into the habit of driving with the headlights on all the time, because I believe it's safer. In the Prius, one could just leave the headlights switched to the "on" position, and they'd turn off whenever the driver's door was opened after the car was switched off. If you try that in the Leaf, the car beeps at you to warn you you've left the headlights on. It has an "auto" mode, where the car will choose when to turn the headlights on, based on ambient light conditions. In that case, when you turn the car off, it'll leave the headlights on for a configurable period of time and then turn them off. This is actually slightly unsettling, because it makes you think you've left your headlights on. The default timeout was quite long as well, something like 60 seconds. The way multiple Bluetooth phones are handled is just as annoying in the Leaf as it is in the Prius, which disappoints me, given 6 years have passed. The way I'd like to see multiple phones handled is the car chooses to pair with whichever phone is in range, or if multiple phones are in range, it asks or uses the last one it paired with. In reality, it tries to pair with whatever it paired with last time, and one has to press far too many buttons to switch it to one of the other phones it knows about. Range anxiety is definitely something of a concern. It can be managed by using the GPS navigation for long or otherwise anxiety-inducing trips, and then one can compare the "miles remaining" on the GPS with the "miles remaining" on the battery range, and reassure oneself that they will indeed make it to where they're trying to go. The worst case I had was getting to within 5 miles of empty. The car started complaining at me. The charging infrastructure in the Bay Area is pretty good. There are plenty of charging stations in San Francisco and San Jose. I'm spoiled in that I have free charging available at work (including a building at the end of my street, so I never bothered with getting a home charger installed). I've almost never had to pay for charging, so it's been great while gas prices have been on the rise. The car's navigation system knows about some charging stations, so you can plan a route with the charging stations it knows about in mind. The only problem is it doesn't know if the charging stations are in use. If you use the ChargePoint Android app, you can see if the charging stations are in use, but then you have to do this cumbersome dance to find an available charging station and plug the address into the vehicle's navigation system. Of course, what can then happen is in the time you're driving to the charging station, someone else starts using it. I actually got bitten by this yesterday. Would I buy a Leaf again? Not as my sole car. It makes a perfect second/commuter car, but as a primary vehicle, it's too limited by its range. They're also ridiculously expensive in Australia, and Brisbane has absolutely no charging infrastructure.

23 February 2013

Dirk Eddelbuettel: Two papers about RcppEigen and RcppArmadillo published

Two papers got published recently. The first one is Bates and Eddelbuettel (2013). It is titled Fast and Elegant Numerical Linear Algebra Using the RcppEigen Package, and provides a pretty thorough introduction to our RcppEigen package which uses Rcpp to provide access to the Eigen C++ template library from GNU R. The paper is out as Volume 50, Issue 5 at the (all electronic, open, and generally awesome) Journal of Statistical Software. A bibtex entry is available. The second paper is Eddelbuettel and Sanderson (2013). This one is titled RcppArmadillo: Accelerating R with high-performance C++ linear algebra and introduces the RcppArmadillo package which brings Conrad Sanderson's Armadillo C++ template library to GNU R by deploying Rcpp. The paper is currently "in press" at Computational Statistics & Data Analysis but the DOI 10.1016/j.csda.2013.02.005 will remain once a volume and issue is assigned by CSDA. Preprints of both papers are available via my papers page, and as vignettes in the corresponding packages. The upcoming Rcpp class in New York will feature Rcpp, RcppArmadillo and RcppEigen. Space is still available.

Sylvain Le Gall: OUnit 2.0 progress

I have recently started to work on OUnit 2.0. The point of this new version will be to adapt OUnit to improve speed and compatibility with third party system:

better configuration setup (through the use of environment variable, command line options and configuration files)
systematic logging (verbose always on), but output log in a file
allow vim quickfix to jump in the log file where the error has happened
output HTML report
output JUnit report
run tests in parallel
automatic selection of tests (choose the one that failed in the last run, before re-running the one that was ok)

Only the 3 last point needs to be completed. Here is a screenshot of Jenkins reading JUnit output of an OUnit test: Jenkins, OUnit test results

So if you have a long standing issue with OUnit, this is the time to submit a bug. OUnit BTS

Russell Coker: Serious Begging

This evening I was driving through one of the inner suburbs of Melbourne when a man flagged me down. He said that his mother was dying and he needed a taxi ride to some hospital far away and needed to borrow $200. He was saying something about his phone, I wasn t sure if he was planning to give me his phone number so I could call him to ask for repayment or offering his phone as collateral on the loan (incidentally a well known scam is to offer a stolen phone as collateral for a loan, it s a way of selling a locked phone that doesn t have cables). I ve encountered many beggars over the years, but he was by far the most serious about it he demonstrated the level of desperation that I ve only previously seen documented in history books and reports from travelers who visited developing countries. I will never know if his mother was dying, there are lots of other reasons why someone might urgently need cash (most of which won t get much sympathy). I gave him $20 as a gift. If his story was legitimate then I gave him 10% of what he needed so he only had to find another 9 people willing to do the same. If he was lying then I can afford to lose $20. In any case I definitely wasn t going to do what he asked and withdraw hundreds of dollars from an ATM for him. Also regardless of whether he was telling the truth I didn t want to have him repay me, if he s telling the truth then I m happy to give money to him and if he s not then I m better off avoiding him in future. If I had $50 I would probably have given it to him, but $200 is too much. As I drove off I looked in my rear-vision mirror and saw him running between cars on the road trying to flag someone else down. Running through moving traffic on a Saturday night is another indication of how serious he was, generally someone who s in a good state of mind and wants a long and healthy life won t do that.

Daniel Pocock: Checking authors.txt before migrating to github

Ensuring that contributors are correctly recognised for their work is a cornerstone of the free and open source software community. Here I present a convenient script to help. When projects migrate to git (and usually Github these days), there is usually quite a lot of enthusiasm to get up and running and the authors.txt file is not looked at closely, or not at all. Github does a great job of summarising the free software contributions of each member. To do this, it needs to cross-reference their commits with their email addresses. This possibility is not unique to Github though: as git is a distributed version control system, it is quite possible that other services will seek to mirror and report on code contributions in the future. authors.txt can be tedious For a large project with a long history, there may be more than 100 committers, many of them designated by SVN user IDs that are unique to the project and not easily mapped to the user email address. If the project is hosted on Google Code SVN, then Google often makes dummy gmail.com accounts for the committers - and many people don't even use those accounts for anything else, not even as an email account. Including these accounts in a git repository is a bad idea if it is not the person's preferred email address. If they don't have that address associated with their Github account, then their commits won't be easily attributed to them at all. Often, building an accurate authors.txt can be tedious, but here are some steps to simplify it:

Use a script like svn-authors-extract to quickly build a template file by scanning the SVN repository.
Scan email list history - if necessary, download the email archive as a mailbox, and use regular expressions to extract all email addresses
Find out if the SVN server keeps a list of email addresses with the SVN user IDs somewhere
Use searches in Google, LinkedIn and various other places to try and find out where the person is now
Finally, use the script below to test the results

Testing the results against Github automatically How do you know which email addresses are already matched to a Github account and which need further research? I've written a convenient script to help It creates a dummy repository with one commit per committer. Upload this dummy repository as a new project on Github, view the commit log page, and all the matched user IDs will be highlighted, so the missing ones will stand out and you can quickly focus your search for missing email addresses. I've tested this with the authors.txt file I'm building for the Sipdroid/Lumicall projects. This project comes from Google Code SVN and almost all the identities are obscured by gmail.com addresses that may or may not be the real/preferred email addresses. I created a dummy project on Github called sipdroid-users and pushed the dummy commits there so I can see how many of the contributors are correctly mapped to a Github account. A full copy of the script is below, you can also find it in the sync2git repository:

#!/bin/bash
set -e
if [ $# -lt 2 ];
then
  echo "$0 <authors file> <dest repo for push>"
  exit 1
fi
AUTHORS_FILE=" pwd /$1"
DEST_REPO="$2"
if [ ! -e "$ AUTHORS_FILE " ];
then
  echo "can't find $ AUTHORS_FILE , aborting"
  exit 1
fi
TMP_REPO= mktemp -d 
TARGET_FILE="testing.txt"
cd "$ TMP_REPO "
git init .
cat "$ AUTHORS_FILE "   while read ;
do
  full_info= echo "$ REPLY "   cut -f2 -d'='   cut -b2- 
  an=" echo "$ full_info "   sed -e "s/ <.*\$//\" "
  am=" echo "$ full_info "   sed -e "s/^.*</</\" "
  cn="$an"
  cm="$am"
  echo "$REPLY" >> "$ TARGET_FILE "
  git add "$ TARGET_FILE "
  export GIT_AUTHOR_NAME="$an"
  export GIT_AUTHOR_EMAIL="$am"
  export GIT_COMMITTER_NAME="$cn"
  export GIT_COMMITTER_EMAIL="$cm"
  git commit -m "Test adding $ REPLY "
done
git remote add origin "$ DEST_REPO "
git push -u origin master

Robert Collins: Simpler is better a single event type for StreamResult

StreamResult, covered in my last few blog posts, has panned out pretty well. Until that is, that I sat down to do a serialised version of it. It became fairly clear that the wire protocol can be very simple just one event type that has a bunch of optional fields test ids, routing code, file data, mime-type etc. It is up to the recipient at the far end of a stream to derive semantic meaning, which means that encoding a lot of rules (such as a data packet can have either a test status or file data) into the wire protocol isn t called for. If the wire protocol doesn t have those rules, Python parsers that convert a bytestream into StreamResult API calls will have to manually split packets that have both status() and file() data in them this means it would be impossible to create many legitimate bytestreams via the normal StreamResult API. That seems to be an unnecessary restriction, and thinking about it, having a very simple here is an event about a test run API that carries any information we have and maps down a very simple wire protocol should be about as easy to work with as the current file or status API. Most combinations of file+status parameters is trivially interpretable, but there is one that had no prior definition a test_status with no test id specified. Files with no testid are easily considered as global scope for their source, so perhaps test_status should be treated the same way? [Feedback in comments or email please]. For now I m going to leave the meaning undefined and unconstrained. So I m preparing a change to my patchset for StreamResult to:

Drop the file() method altogether.
Add file_bytes, mime_type and eof parameters to status().
Make the test_id and test_status parameters to status() optional.

This will make the API trivially serialisable (both to JSON or protobufs or whatever, or to the custom binary format I m considering for subunit), and equally trivially parsable, which I think is a good thing.

22 February 2013

Richard Hartmann: Finland I

Finland. Helsinki, Lahti, streets Arriving at Helsinki airport, we filed a claim with Lufthansa as a hard shell suitcase had a splintered corner. We were surprised that so many Finns arrived from Munich with skis, more on that later. We picked up our car and started on our way towards Koli; driving with a top speed of 100 km/h and often being limited to 80 km/h or even 60 km/h is... unusual... Finnish police/authorities seem to be obsessed with enforcing those speed limits as there are a lot of speed cameras along the way. Finnish people seem to be similarly obsessed with slot machines; there is an incredible amount of them at gas stations and a constant stream of people playing them. From an outsider's perspective, it's weird that a country as strict about one form of addiction, alcohol, and working against it vigorously, by means of taxes, would allow another form of addiction, gambling, run as freely and this allow so many slot machines. Speaking of taxes on alcohol: a single 0.33 l bottle of beer is more expensive in a Finnish supermarket than 0.5 l of beer in a German restaurant. Which also explains why supermarkets tend to have a rather large section with relatively cheap alcohol free beer. Anyway, coming back to streets: Highway intersections don't have continuous on/off ramps from which you change from one highway to another; you drive off of the highway, stop at a traffic light, and then continue onto the other highway. Weird system, but given the amount of traffic we witnessed, it's probably Good Enough (tm). Stopping for a short time in Lohti simply because it's apparently famous for winter sports competitions, we arrived at Future Freetime in Koli national park after about five to six gruelling hours of net driving through somewhat bad weather and behind slow drivers. Koli Hiking up to Ukko-Koli and its sister peaks proved to be rather exhausting as we kept on breaking through the snow cover to our knees and sometimes even our hips. Once we were up there, we realized that even though you couldn't see it in between the trees, there was fog all over the plains so we couldn't see anything. Still, it was a nice hike even if somewhat short. Note to self: Even when a trail is marked locally, if OpenStreetMap does not know about it... don't walk along it. Especially not when the going's rough already. And if there's a sign suggesting you wear snow shoes... wear snow shoes. Returning to Koli Hotel and the museum next to it, we walked over to the ski slope. The highest peak within Koli,Ukko-Koli, is 347 meters high, the local ski slope starts a good way below that. This would explain why a lot of Finns came back from Munich with their skis... Afterwards, we rented a snow mobile, without guide or supervision, and drove from Loma-Koli over lake Pielien towards Purnuniemi and in a large circle down towards lake Ryyn skyl where we turned around and went back the same way. If we thought Finnish streets don't have a lot of signs we soon realized that snow mobile tracks have even less. There are at most two or three signs pointing you in the right direction, but on the plus side, there are no posted speed limits for snow mobiles, either. In somewhat related news, snow mobiles can go at least 95 km/h. At that point, the scratched and dirty visor of your rental helmet will keep slamming down, forcing you to take one hand off the handle and thus stop accelerating to maintain stability. To round off the day, we heated up the sauna built into our little wooden hut. Running outside three times to rub myself off with snow from head to toes, I almost slipped and fell while standing still. When your feet are too hot for the snowy ground, you'll start to melt your own little pools of slippery water/snow mush within seconds. File that one under "I would never have guessed unless I had experienced it myself". Generic The MarkDown source of this blog post is not even 5 kiB in size; even in a worst case scenario, pushing this to my ikiwiki instance via git will eat up less 10 kiB of mobile data. Which is good because I have 78 MiB of international data left on this plan. This is also the reason why there are no links in this blog post: I am writing everything off line and don't want to search for the correct URLs to link to. I really wish EU regulators would start to tackle data roaming now that SMS and voice calls are being forced down into somewhat sane pricing regions by regulations. PS:

-rw-r--r-- 1 richih richih 4.6K Feb 11 22:55 11-Finland-I.mdwn
[...]
Writing objects: 100% (7/7), 2.79 KiB, done.

Richard Hartmann: Finland II

Second part; bullet points to save time

For the first time in a long time, I used something that comes "from Nokia": Nokia_n tires with spikes. Nokia_n means "from Nokia", i.e. a former business unit which Nokia sold and allowed the new owner to keep the modified name. I don't expect to use anything else from Nokia any time soon, which is a pity imo.
Roads are often plowed, but a lot of ice still remains. This is not a problem because of the spikes; they, in turn, will be a problem for non-icy roads as they turn tarmac into gravel and potholes... At least while I was there, most streets, both within and without cities, were covered with either snow or ice.
Cars moving snow are not purpose-built, but re-use existing machinery. Plows tend to be smaller dump trucks (~20-30 tons?) while actual shovelling is done by tractors and small diggers.
Plow drivers expect you to make way. In narrow streets, they will not slow down even if their shovel extends well onto your lane. You get out of the way, and quickly, or your car is scrap.
Finland is a nation of tailgaters. Distance between cars driving at around 80 km/h on icy roads averages at about 10-20 meters.
One reason why tailgating is somewhat understandable, if still highly dangerous, is that a single slow driver can accumulate literally a dozen or more car behind them as there's almost no chance to overtake on those wavy single-lane roads. I witnessed a car driving at a constant 50 km/h in a 80 km/h zone. As they refused to stop at the side to let others pass, this was... infuriating.
Overtaking is generally done at the lowest speed delta possible without crashing into oncoming traffic. This ensures that only one car per overtaking opportunity gets to overtake.
Licorice all the things. Licorice ice cream. Yay.
There are a lot of body building products in Finnish super markets. Even the smallest stores always have their own section.
Lactose and gluten free food is a Big Thing.
Foreign loanwords always have at least one letter doubled; tack an "i" at the end and you're good to go.
Finns are fiercely proud of their own country and its achievements. Most if not all of them sport a Nokia phone. Try and find a door hinge not made by Fiskars.
Angry. Birds. Everything. There is even Angry Birds coffee beans on sale.
Keys are half-round, of a system I have never seen before, and tend to jam in the locks.
Self-locking doors are not opened automagically by depressing the inner handle; you are forced to turn an extra knob. The same system is used for doors you lock manually. Said knob needs to be turned clock-wise to open and vice versa to close. The German system follows standard screws (clock-wise to tighten/close) and thus is the exact opposite. If you are used to the exact opposite of a system like that, you get to run into doors a lot.
Energy seems to be free or nearly so. Doors of many heated buildings just stay open all the time. Lights burn continuously. Both cabins I stayed at were heated with electric stoves of all things.
Drafty (thus non-insulated) windows are apparently common enough that you can buy plastic tape marketed specifically to tape over drafty windows.

Russell Coker: iPhone vs Android

A friend who s a long-time iPhone user just asked for my advice about whether to get a Samsung Galaxy S3, a Samsung Galaxy Note 2, or a iPhone 5. Advantages for Android I think that liberty should be the first consideration, I ve previously written about how Android phones won t necessarily give you as much freedom as you desire if you buy on the basis of price and features [1]. But even the least free Android options are way better than the iOS (iPhone and iPad) environment. This isn t necessarily a big deal for my friend, like most of the population he usually just wants things to work being able to hack them isn t such an issue. However unlike most of the population he does make a reasonable portion of his income from software development and it could be that he will have a contract for developing an app on a mobile device in which case the freedom to tinker on Android will help him. He could use an iPhone for his personal use and develop on an Android platform for his clients, but generally it s more efficient if your personal use of technology is similar to that of your clients. The Nexus devices are very good for liberty and they also have nice hardware at a low price, I ve just got a Nexus 4 for my wife and it s very nice. The next issue is that of hardware standards, I ve previously written about the potential for developing a standard form factor for Android phones [2] although this doesn t seem likely to be implemented in the near future. The wide range of Android hardware means that the range of cases etc on the market is rather small. But the advantage of the wide range is that with an Android phone you can have a device that s bigger, smaller, cheaper, cuter, or faster than an iPhone. There are Android devices which have a higher resolution, more RAM, more storage (if you include SD storage), or has other benefits over an iPhone. For whatever reasonable range of specs appeal to you you can probably find a device to match. I ve previously written about the way the ideal size for devices depends on your hand size and your preferred manner of gripping the device [3], so the lack of size range in Apple devices is not just a limitation on personal choice but also a failure to properly support people with different size hands. Depending on the preferred manner of gripping a phone the iPhone 5 is either too big for an average woman or too small for a tall man. The Google Play store apparently has more applications than the iPhone/iPad App Market. This difference can be expected to increase now that the Samsung Galaxy S3 is outselling the iPhone 5. Comparing the number of unit sales of the iPhone vs Android phones is no longer interesting, comparing Samsung to Apple is the interesting thing. Advantages for the iPhone By all accounts it s quite an easy process to backup and restore all iPhone settings. You can expect that after losing an iPhone you can just connect the new one to your PC and have it work in exactly the same way after all the data is transferred. Trying to do such things on Android is merely difficult if you have root access to your phone and the source and destination phones are of exactly the same make and model. But if you have different versions of the phone or if you don t have root access then it may be impossible. I welcome comments from anyone who knows of good solutions to this problem. The iPhone achieved a reasonable share of the smart-phone market before Android really started going well so there are a lot of people who are used to the iPhone. Simply by being unfamiliar Android will be a more difficult option for people who have used the iPhone such as my friend. But it is possible to learn other systems. Generally I think that this may be a big issue for people who use Macs for all their other computing. But if the only Apple product you use is an iPhone then switching to Android shouldn t be a big deal. Update: One feature of the iPhone that is very important to my friend is the ability to add arbitrary tags in the contacts. In addition to name, address, phone number, etc he wants to add arbitrary notes related to his business. While he could put that sort of thing into the Notes field in Android he would rather have several fields with his own names. Android 4.1.x definitely doesn t have this and I can t test Android 4.2.x at this time. Is there any way of doing such things on Android? Conclusion It seems to me that Android devices are better in every way apart from backup, restore, and general management. If I was about to buy 100 phones then I d probably consider the iPhone (not necessarily buy but definitely consider). But for a single user I definitely recommend Android devices. The Android devices which seem good at the moment are the Galaxy S3 (which I m using now), the Nexus 4 (which is really good apart from being unable to change the battery or add more storage), and the Galaxy Note 2 (which is about the biggest phone available). One of the things that my friend wants to do is to use a phone instead of a tablet or laptop. I think that the Galaxy Note 2 is the only option for him.

Next.

Previous.