Search Results: "reg"

4 February 2014

Russell Coker: Clothing and Phone Cameras

In 2012 I wrote about my jeans from Rivers that fit the largest available phones (and the smaller tablets) in their pockets [1]. Those jeans are still working well for me, I can add the fact that they don t wear out quickly to the list of positive attributes. Recently my sister asked for advice on getting a new phone, she was considering the Samsung Galaxy Note 2 (the phone I m using now) because it apparently takes better pictures than the Nexus 4 she s using. I ve used both those phones and I hadn t noticed a difference in picture quality, but there is some variation in manufacturing and it could be that I ve got a below average Note 2 and a better than average Nexus 4 so I ll assume for the sake of discussion that my sister would actually get an improvement in picture quality by using a Note 2. If you have a phone that doesn t have the picture quality you desire then one option is to buy a phone with a better camera, but you will be limited by issues of physics. A thin phone has a short focal length which means that the lens has to be small and therefore the amount of light that gets to the sensor is small. The Nokia Lumia 1020 has some of the best camera hardware that you ll find in a phone, but it s still only 14.5mm thick where the camera is and that will limit the quality a lot. Any compact camera should be able to beat all phone cameras in terms of picture quality in most areas. The Samsung Galaxy Camera [2] is also worth considering, it has more features than a typical compact camera and good GUI that allows novice photographers to take advantage of it. Also being able to blog your photos directly from the camera could be a useful feature. But the big down-side of a compact camera is that it s not that compact. Most people won t find it convenient to carry a compact camera with them at all times and therefore they might miss a good opportunity to take a photo. The Galaxy Note series of phones also suffer in this regard because they are larger than most phones. If your phone won t fit in your pocket and you have it in your backpack when on the move or on a bench at home then you will probably miss some good photos. As I was at a Rivers store recently I tested my Note 2 in the pockets of women s jeans. Rivers scored very poorly in this regard, one pair of women s jeans had fake pockets (this is just wrong for working clothes) and of the rest only one pair could fit a Note 2. The pair that fit a Note 2 didn t completely enclose the phone, one corner was sticking out, this would probably give a risk of having the phone fall out of the pocket and cause some discomfort to the wearer. I have a pair of shorts with similar size pockets and find it very annoying with the Note 2 in the pocket (for about 10 months of the year I wear jeans so this isn t a big deal). Rivers jeans only count as geeky jeans for male geeks. It s disappointing that with about a dozen different styles of women s jeans there didn t seem to be a single one with pockets of comparable size to the men s jeans. I had to recommend that my sister not get a phone from the Galaxy Note series if taking pictures is a priority due to the apparent difficulty in getting it to fit in a pocket and the probability that she would miss good photos due to this. In past discussions of phone size there have been mentions of the possibility of getting clothing altered. Does anyone have a good experience in getting clothes altered to have bigger pockets or in the case of women s clothing to have fake pockets replaced with real ones?

3 February 2014

Wouter Verhelst: Waking up

Today is the monday after FOSDEM 2014, and I'm slowly waking up. Being in charge of the video team, and after having spent a whole week preparing followed by a whole weekend stressing about, today is a day of slowly waking up from hibernation and returning gear to various rental companies. At FOSDEM 2013, we had recorded five rooms. Even though we did lose a few talks, I felt pretty good about the whole thing, and I think we did pretty well then. It had been my plan to increase the number of rooms to be recorded from 5 to a reasonably higher number say, 10 or so but someone convinced me to do all of them. I'll readily admit I was afraid. Scratch that, I was terrified. Petrified, that we wouldn't be able to pull it off. For previous editions, we'd kept the number conservative, since video work is a high-risk business: if something goes wrong with volunteers work (say, the heralding volunteer doesn't show up), you find a replacement, or you have a staff member introduce a speaker, and the show will go on. Not so with video work: if something minor goes wrong, you will most likely lose content. If something minor goes wrong in one devroom, I'll deal with it. If something minor goes wrong in 20% of devrooms, that's 4 to 5 devrooms, and I can't deal with it. Moreover, finding enough volunteers to manage the video for five rooms was a challenge last year; I couldn't even begin to imagine how to do so for 22 of them. We did work out a way that didn't seem too far-fetched, and that might work: why not look for volunteers from the devrooms themselves? But then, they couldn't be expected to have the required experience, so we'd have to do a lot of handholding. The week before FOSDEM, four of us sat down in an office in Mechelen, and started preparing. We had a lot to do; and when you do, more jobs keep being added to the pile; TODO lists tend to grow longer, not shorter. Installing servers, testing cameras, setting twinpact dipswitches, configuring laptops. Changing a few tests in the config management system, and having to go over the laptops again. Buying extra gear. Buying yet more extra gear, several times. Buying cardboard boxes when it appeared that the plastic foldable ones we'd ordered wouldn't make it in time, and unloading the pallet of them when it did, after all, so we could load them into the van with all of the other gear. Laminating instruction sheets. Receiving shipments of rented gear. Calling another supplier last minute when one of our other suppliers had contacted us to let us know one of their cameras that they'd promised us had broken down, and couldn't be repaired in time. Calling that last-minute supplier again when we found out that some of the cameras we'd rented could do HDV only (and not DV), and we needed to find more than the ones we'd already asked for. Discovering that his cameras were of the same problematic model and so also couldn't be used. The feeling of relief when he called me a few hours later and told me he'd managed to scrounge up enough cameras to help us out after all. And then it was time to go to the ULB and start setting up everything. On friday, we had a little bit of help, but did have to do most of it ourselves. After setting up everything in every room and testing as much as possible within the time constraints that we had, we picked up the most expensive bits of the gear, brought it down to a safe place, and went home. On saturday and sunday, for me it was mostly a matter of handing out gear to various devrooms, running around to fix various issues, and then in the evening receiving boxes back, using a checklist to ensure everything expected was in the box (and that things not needed would remain in the room, or be sent back). It was exhausting, to the extent that on saturday afternoon, I had to take a bit of time off so I could go and rest. I think it's safe to say that something like this hasn't been done before. FOSDEM is a huge conference; most conferences with multiple rooms don't have more somewhere between 5 and 10 of them. The fact that we have so many more, and the fact that we record all of them, puts FOSDEM video work in a class of its own, to the extent that the professional cameramen I talked to in order to get the right cameras would incredulously ask "what do you need that for?" when I gave them the numbers we needed. As such, I didn't expect, nor was aiming for, perfection. Unfortunately, with video work it is close to impossible to attain perfection; I have told multiple people during the event that I would be happy if we reached 85% of talks recorded at acceptable quality; and though it is much too soon to say for certain, my gut feeling tells m that we've probably achieved that. One thing we hadn't planned for was streaming. During the past three years, FOSDEM enlisted the help and sponsorship of flumotion to get streaming going. Unfortunately, that did not work out this year; and since we already had far too much on our plates with everything else, we decided to forego streaming and focus on recording, only. A few days before the event, however, Steinar Gunderson took it upon himself to fix that. While we couldn't support him a great deal, we could give him access to the secondary laptops (which weren't otherwise doing much anyway) on which he could then do his thing. This was mostly transparent to us; we did communicate to some extent (e.g., when a machine that had been down had fixed and was working again), but he mostly did his thing while we did ours. Full details, for those who want it, in the linked blog post. Today, then, I spent most of my time handling hardware: waiting for other members of FOSDEM to drive the van up to my office; waiting for the laptop rental company to retrieve most of the laptops; waiting for two of the camera rental companies to retrieve their cameras and tripods; driving over to the final camera rental company to return their cameras and tripods; and finally, driving up to the ULB to retrieve the last laptops who had been copying files all night. As I write this, some video files have already been uploaded to the FOSDEM video archive. However, these are extremely low-quality renditions of video snippets that need to be reviewed; they are not ready yet for public consumption. When those files exist, expect an announcement through the main FOSDEM website.

Kees Cook: compiler hardening in Ubuntu and Debian

Back in 2006, the compiler in Ubuntu was patched to enable most build-time security-hardening features (relro, stack protector, fortify source). I wasn t able to convince Debian to do the same, so Debian went the route of other distributions, adding security hardening flags during package builds only. I remain disappointed in this approach, because it means that someone who builds software without using the packaging tools on a non-Ubuntu system won t get those hardening features. Think of a sysadmin trying the latest nginx, or a vendor like Valve building games for distribution. On Ubuntu, when you do that ./configure && make you ll get the features automatically. Debian, at the time, didn t have a good way forward even for package builds since it lacked a concept of global package build flags . Happily, a solution (via dh) was developed about 2 years ago, and Debian package maintainers have been working to adopt it ever since. So, while I don t think any distro can match Ubuntu s method of security hardening compiler defaults, it is valuable to see the results of global package build flags in Debian on the package archive. I ve had an on-going graph of the state of build hardening on both Ubuntu and Debian for a while, but only recently did I put together a comparison of a default install. Very few people have all the packages in the archive installed, so it s a bit silly to only look at the archive statistics. But let s start there, just to describe what s being measured. Here s today s snapshot of Ubuntu s development archive for the past year (you can see development opening after a release every 6 months with an influx of new packages):

Here s today s snapshot of Debian s unstable archive for the past year (at the start of May you can see the archive unfreezing after the Wheezy release; the gaps were my analysis tool failing):

Ubuntu s lines are relatively flat because everything that can be built with hardening already is. Debian s graph is on a slow upward trend as more packages get migrated to dh to gain knowledge of the global flags. Each line in the graphs represents the count of source packages that contain binary packages that have at least 1 hit for a given category. ELF is just that: a source package that ultimately produces at least 1 binary package with at least 1 ELF binary in it (i.e. produces a compiled output). The Read-only Relocations ( relro ) hardening feature is almost always done for an ELF, excepting uncommon situations. As a result, the count of ELF and relro are close on Ubuntu. In fact, examining relro is a good indication of whether or not a source package got built with hardening of any kind. So, in Ubuntu, 91.5% of the archive is built with hardening, with Debian at 55.2%. The stack protector and fortify source features depend on characteristics of the source itself, and may not always be present in package s binaries even when hardening is enabled for the build (e.g. no functions got selected for stack protection, or no fortified glibc functions were used). Really these lines mostly indicate the count of packages that have a sufficiently high level of complexity that would trigger such protections. The PIE and immediate binding ( bind_now ) features are specifically enabled by a package maintainer. PIE can have a noticeable performance impact on CPU-register-starved architectures like i386 (ia32), so it is neither patched on in Ubuntu, nor part of the default flags in Debian. (And bind_now doesn t make much sense without PIE, so they usually go together.) It s worth noting, however, that it probably should be the default on amd64 (x86_64), which has plenty of available registers. Here is a comparison of default installed packages between the most recent stable releases of Ubuntu (13.10) and Debian (Wheezy). It s clear that what the average user gets with a default fresh install is better than what the archive-to-archive comparison shows. Debian s showing is better (74% built with hardening), though it is still clearly lagging behind Ubuntu (99%):

2014, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

Dirk Eddelbuettel: Rcpp 0.11.0

A new major release 0.11.0 of Rcpp is now on the CRAN network for GNU R and binaries for Debian have been uploaded as well. Before you read on, please note that this release will most likely require a reinstallation of all your packages using Rcpp as it now works without a user-facing shared library. The biggest change in this version is that it is now effectively headers-only. Before you wonder, there is still compiled code provided by Rcpp for use by packages. But this uses the function registration mechanism of GNU R instead via instantiation at package startup. This does make package building easier, will remove the need to query GNU R for the Rcpp library file in src/Makevars when compiling, and should generally avoid building issues such as the dreaded fails on paths with spaces still annoying users of a certain OS. There are a number of other fixes, upgrades and other extensions detailed in NEWS file extract below, in the ChangeLog file in the package and on the Rcpp Changelog page as well as in a release announcement I'll post later.

Changes in Rcpp version 0.11.0 (2014-02-02)

Changes in Rcpp API:

Functions provided/used by Rcpp are now registered with R and instantiated by client package alleviating the need for explicit linking against libRcpp which is therefore no longer created.

Updated the Rcpp.package.skeleton() function accordingly.

New class StretchyList for pair lists with fast addition of elements at the front and back. This abstracts the 3 functions NewList, GrowList and Insert used in various packages and in parsers in R.

The function dnt, pnt, qnt sugar functions were incorrectly expanding to the no-degree-of-freedoms variant.

Unit tests for pnt were added.

The sugar table function did not handle NAs and NaNs properly for numeric vectors. Fixed and tests added.

The internal coercion mechanism mapping numerics to strings has been updated to better match R (specifically with Inf, -Inf, and NaN.)

Applied two bug fixes to Vector sort() and RObject definition spotted and corrected by Kevin Ushey

New checkUserInterrupt() function that provides a C++ friendly implementation of R_CheckUserInterrupt.

Changes in Rcpp attributes:

Embedded R code chunks in sourceCpp are now executed within the working directory of the C++ source file.

Embedded R code chunks in sourceCpp can now be disabled.

Changes in Rcpp documentation:

The Rcpp-FAQ and Rcpp-package vignettes have been updated and expanded.

Vignettes are now typeset with grey background for code boxes.

The bibtex reference file has been update to reflexct current package versions.

Changes in Rcpp unit tests:

The file tests/doRUnit.R was rewritten following the pattern deployed in RProtoBuf which is due to Murray Stokely

The function test() was rewritten; it provides an easy entry point to running unit tests of the installed package

Thanks to CRANberries, you can also look at a diff to the previous release 0.10.6. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

2 February 2014

Gunnar Wolf: CuBox-i4Pro

Somewhere back in August or September, I pre-ordered a CuBox-i A nicely finished, completely hackable, and reasonably powerful ARM system, nicely packaged and meant to be used to hack on. A sweet deal! There are four models (you can see the different models' specs here) I went for the top one, and bought a CuBox-i4Pro. That means, I have a US$130 nice little box, with 4 ARM7 cores, 2GB RAM, WiFi, and... well, all of its basic goodies and features. For some more details, look at the CuBox-i block diagram. I got it delivered by early January, and (with no real ARM experience on my side) I finally got to a point where I can, I believe, contribute something to its adoption/usage: How to get a basic Debian system installed and running in it. The ARM world is quite different to the x86 one: Compatibility is much harder, the computing platform does not self-describe properly, and a kernel must first understand how a specific subarchitecture is before being able to boot on it. Somewhere in the CuBox forums (or was it the IRC channel?) I learnt that the upstream Linux kernel does not yet boot on the i.MX6 chip (although support is rumored to be merged for the 3.14 release), so I am using both a kernel and an uBoot bootloader not built for (or by) Debian people. Besides that, the result I will describe is a kosher Debian install. Yes, I know that my orthodox friends and family will say that 99% kosher is taref... But remember I'm never ever that dogmatic. (yeah, right!) Note that there is a prebuilt image you can run if you are so inclined: In the CuBox-i forums and wiki, you will find links to a pre-installed Debian image you can use... But I cannot advise to do so. First, it is IMO quite bloated (you need a 4GB card for a very basic Debian install? Seriously?) Second, it has a whole desktop environment (LXDE, if I recall correctly) and a whole set of packages I will probably not use in this little box. Third, there is a preinstalled user, and that's a no-no (user: debian, password: debian). But, most importantly, fourth: It is a nightly build of the Testing (Jessie) suite... Built back in December. So no, as a Debian Developer, it's not something we should recommend our users to run! So, in the end and after quite a bit of frustration due to my lack of knowledge, here goes the list of steps I followed:

Using the CuBox

On the i2 and i4 models, you can use it either with a USB keyboard and a HDMI monitor, or by a serial consoles (smaller models do not have a serial console). I don't have a HDMI monitor handy (only a projector), so I prefer to use the serial terminal. Important details to avoid frustration: The USB keyboard has to be connected to the lower USB port, or it will be ignored during the boot process. And make sure your serial terminal is configured not to use hardware flow control. Minicom is configured by default to use hardware flow control, so it was not sending any characters to the CuBox. ^A-O gets you to the Minicom configuration, select Serial port setup, and disable it.

Set up the SD card

I created a 2GB partition, but much less can suffice; I'd leave it at least to 1GB to do the base install, although it can be less once the system is set up (more on this later). Partition and format using your usual tools (fdisk+mke2fs, or gparted, or whatever suits your style).

Install the bootloader

I followed up the instructions on this CuBox-i forums thread to get the SPL and uBoot bootloader running. In short, from this Google Drive folder, download the SPL-U-Boot.img.xz file, uncompress it (xz --decompress SPL-U-Boot.img.xz), and write it to the SD card just after the partition map: As root,
# dd if=SPL-U-Boot.img of=/dev/mmcblk0 bs=1024 seek=1.
Actually, to be honest: As I wanted something basic to be able to debug from, I downloaded (from the same Google Drive) the busybox.img.gz file. That's a bit easier to install from: xz --decompress busybox.img.xz, and just dump it into the SD from the beginning (as it does already include a partition table):
# dd if=busybox.img of=/dev/mmcblk0
This card is already bootable and minimal, and allows to debug some bits from the CuBox-i itself (as we will see shortly).
After this step, I created a second partition, as I said earlier. So, my mmcblk0p1 partition holds Busybox, and the second will hold Debian. We are still working from the x86 system, so we mount the SD card in /media/mmcblk0p2

Installing the base system

Without debian-installer to do the heavy lifting, I went for debootstrap. As I ran it from my PC, debootstrap's role will be for this first stage only to download and do a very initial pre-unpacking of the files: Bootstrapping a foreign architecture implies, right, using the --foreign switch:
debootstrap --foreign --arch=armhf wheezy /media/mmcblk0p2 http://http.debian.net/debian
You can add some packages you often use by specifying --include=foo,bar,baz
So, take note notes: This board is capable of running the armhf architecture (HF for Hardware Float). It can also run armel, but I understand it is way slower.

First boot (with busybox)

So, once debootstrap finishes, you are good to go to the real hardware! Unmount the SD card, put it in the little guy, plug your favorite console in (I'm using the serial port), and plug the power in! You should immediately see something like:

U-Boot SPL 2013.10-rc4-gd05c5c7-dirty (Jan 12 2014 - 02:18:28)
Boot Device: SD1                                              
reading u-boot.img
Load image from RAW...
 
 
U-Boot 2013.10-rc4-gd05c5c7-dirty (Jan 12 2014 - 02:18:28)
 
CPU:   Freescale i.MX6Q rev1.2 at 792 MHz
Reset cause: POR                         
Board: MX6-CuBox-i
DRAM:  2 GiB      
MMC:   FSL_SDHC: 0
In:    serial     
Out:   vga   
Err:   vga
Net:   phydev = 0x0
Phy not found      
PHY reset timed out
FEC                
(Re)start USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
       scanning usb for ethernet devices... 0 Ethernet Device(s) found
Hit any key to stop autoboot:  3

Let it boot (that means, don't stop autoboot), and you will soon see a familiar #, showing you are root in the busybox environment. Great! Now, mount the Debian partition:
# mount /dev/mmcblk0p2 /mnt

Finishing debootstrap's task

With everything in place, it's time for debootstrap to work. Chroot into the Debian partition:
# chroot /mnt
And ask Debootstrap to finish what it started:
# debootstrap --second-stage
Be patient, as this step takes quite a bit to be finished.

Some extra touches...

After this is done, your Debian system is almost ready to be booted into. Why almost? Because it still does not have any users, does not know its own name nor knows I want to use it via a serial terminal. Three very simple tasks to fix. First two:

# passwd
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully
# echo cubox-i.gwolf.org > /etc/hostname

For the second one, add a line to /etc/inittab specifying the details of the serial console. You can just do this:
# echo 'T0:23:respawn:/sbin/getty -L ttymxc0 115200 vt100' >> /etc/inittab

Boot into Debian!

So, ready to boot Debian? Ok, first exit the chroot shell, to go back to the Busybox shell, unmount the Debian partition, and set the root partition read-only:

# exit
# umount /mnt
# mount / -o remount,ro

Disconnect and connect power, and now, do interrupt the boot process when you see the Hit any key to stop automount prompt. To see the configuration of uboot, you can type printenv We will only modify the parameters given to the kernel:

CuBox-i U-Boot > setenv root /dev/mmcblk0p2 rootfstype=ext3 ro rootwait
CuBox-i U-Boot > boot

So, the kernel will load, and a minimal Debian system will be initialized. In my case, I get the following output:

** File not found /boot/busyEnv.txt **
4703740 bytes read in 390 ms (11.5 MiB/s)
## Booting kernel from Legacy Image at 10000000 ...
   Image Name:   Linux-3.0.35-8                    
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    4703676 Bytes = 4.5 MiB              
   Load Address: 10008000               
   Entry Point:  10008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
 
Starting kernel ...
 
Unable to get enet.0 clock
pwm-backlight pwm-backlight.0: unable to request PWM for backlight
pwm-backlight pwm-backlight.1: unable to request PWM for backlight
_regulator_get: get() with no identifier
mxc_sdc_fb mxc_sdc_fb.2: NO mxc display driver found!
INIT: version 2.88 booting
[info] Using makefile-style concurrent boot in runlevel S.
[....] Starting the hotplug events dispatcher: udevd. ok 
[....] Synthesizing the initial hotplug events...done.
[....] Waiting for /dev to be fully populated...done.
[....] Activating swap...done.
[....] Cleaning up temporary files... /tmp. ok 
[....] Activating lvm and md swap...done.
[....] Checking file systems...fsck from util-linux 2.20.1
done.
[....] Mounting local filesystems...done.
[....] Activating swapfile swap...done.
[....] Cleaning up temporary files.... ok 
[....] Setting kernel variables ...done.
[....] Configuring network interfaces...done.
[....] Cleaning up temporary files.... ok 
[....] Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix. ok 
INIT: Entering runlevel: 2
[info] Using makefile-style concurrent boot in runlevel 2.
[....] Starting enhanced syslogd: rsyslogd. ok 
[....] Starting periodic command scheduler: cron. ok 
 
Debian GNU/Linux 7 cubox-i.gwolf.org ttymxc0
 
cubox-i login:

And that's it, the system is live and ready for my commands!

So, how big is this minimal Debian installed system? I cheated a bit on this, as I had already added emacs and screen to the system, so yours will be a small bit smaller. But anyway Lets clear our cache of downloaded packages, and see the disk usage information:

root@cubox-i:~# apt-get clean
root@cubox-i:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs         1008M  347M  611M  37% /
/dev/root      1008M  347M  611M  37% /
devtmpfs        881M     0  881M   0% /dev
tmpfs           177M  132K  177M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           353M     0  353M   0% /run/shm

So, instead of a 4GB install, we have a 350MB one. Great improvement! Now, lets get it to do something useful, in a most Debianic way!

31 January 2014

Daniel Pocock: If you thought systemd proponents were bad....

Just imagine for a moment if something like the iPhone App Store approval process was applied to every application for a domain name. Delays and unexplained rejections killing many great ideas before they even get off the ground. In the worse cases, like that of Greg Hughes, the great firewall of the app store has been used to quash ideas that it appears Apple may have been planning to compete with or copy. If domain applicants had to go through such a process or if other core technologies fell into a rut like that then it would throw Charles Darwin's long established theory of evolution out the window and society would end up eating each other. Once companies had established their position and were happily making money they would do everything possible to stomp out any innovation that may undermine them. This is exactly the kind of non-evolution that has plagued the telecoms sector for many years now. You don't have to look past the price of a simple SMS to see the truth of this situation. Your 160 bytes can cost anywhere between $US0.10 - $US0.50 throughout the western world, even on a network that is otherwise charging maybe $10 per 1MiB (the equivalent of 6,000 text messages) when roaming. One of the compelling features of free software is that it empowers innovation and efficiency. Telecoms executives may argue that by charging you these extortionate fees, they are keeping spam out of your phone. Think again: technologies like XMPP and SIP are doing that even more effectively and without any price to pay. Software for managing buddy lists and algorithms for spam detection put those telecoms executives out of a job. If SIP, XMPP or both become dominant, evolution may succeed yet. Despite having these great technologies available for messaging and multimedia communications and despite having a range of free software success stories in products such as Firefox or whole distributions like Fedora, Debian and its derivatives like Ubuntu, non-free communications solutions like Skype and Viber continue to run rampant in the voice and video domain. As it turns out, this is a very dangerous area for the free software community to neglect. How many people know somebody who tried running Linux but went back to a proprietary solution so they could use a non-free softphone? In these cases, it is not the phone software itself that appeals to the user: it is the people that software lets them communicate with, their friends, family, professional community and beyond. For people who don't truly perceive the benefits of free software or those who don't have the confidence or experience to defend those benefits, the power of Metcalfe's law hits them like a tsunami and violently sweeps away all the rest of the free software they had started to use. While Skype is currently available for some distributions of Linux, it is not free software: amongst other things, there is no published source code and no independent review of the code has been possible. Leveraging the power of Metcalfe's law, the publisher of that software gains significant power over those who choose to install it. Any day now, they could even use it to offer users a one-click "free trial" upgrade to some "Windows Lite" or install other components that obstruct effective upgrades to newer versions of the free operating system. To make an analogy, the potential for dirty tricks in this area is far more scary than the things systemd proponents have been accused of. For those with an interest in solving these problems or at least giving free real-time communications software a try,

Drop by the real-time lounge at FOSDEM this weekend
Try the Debian SIP service
Join the Free RTC mailing list sponsored by FSF Europe
and see some of my previous blogs for more detail about projects in this area of free software

On the init system issue itself Note: the reference to systemd is not itself an opinion for or against any particular init system in Debian nor is it a suggestion that any one side is badder than any other.

29 January 2014

Daniel Pocock: Lumicall: giving you choice when you dial

One of the distinguishing features of Lumicall, the free and open SIP/VoIP app for Android, is the unique dialing experience. In fact, even regular softphones for the desktop/laptop don't currently offer the same power when you want to make a call. In my previous blog, I looked at the ENUM technology that underpins part of this compelling new user experience. Here we can step back and look at the bigger picture. Choice is important, sneak preview of Sunday's talk at FOSDEM The fundamental reason for the dialing popup is to give the user choice. Choice is important. With that in mind, one of the key revelations that I will discuss during my talk about Lumicall this Sunday at FOSDEM is that I would like to give users the opportunity to use the Lumicall dialer with other VoIP apps, including Jitsi for Android and CSipSimple. To facilitate this, I hope to separate the dialing logic from the app and make it available as a standalone app or library module that can be used in any free communications software. Users expect convenience, Lumicall tries to deliver Have you noticed how many people will pick up their mobile phone to make a call even when they have a regular phone within reach? Have you found yourself calling your friends' mobile numbers even though you have their home and office numbers in your address book too? These mobile network calls are often at least ten times more expensive than landline calls and by definition they are infinitely more expensive than free calls made over the Internet. So why do people behave this way? The answer is simple: convenience. The caller does not want to make multiple efforts to dial, nor does he want to lose one or two minutes speaking to a secretary, colleague or spouse of the person he is trying to call. Dialing from the address book in his own mobile phone and calling the mobile number of the person he wants to speak with is the shortest path to success. It is efficient, it is compelling and it is what users expect. Free software products wanting to make an impact need to carefully align themselves with these user expectations and that is exactly what Lumicall tries to do. Choose your own adventure: dial the digits, use the address book or surf your call history Any of these everyday dialing practices triggers the Lumicall dialer. In this example, the user is trying to call Bob from the address book:

While the user may have simply pressed the "call" button, Lumicall alerts them to several ways that they may proceed:

The first choice is a SIP address found in the ENUM directory, sip:+4456190455234@sip5060.net. Lumicall will take all the phone numbers associated with Bob in the address book and report any ENUM results associated with them. Here, just a single ENUM result is display.
The second choice is a SIP address constructed from Bob's email address. Lumicall found the email address in the address book entry and discovered that Bob's domain, lumicall.org, has a NAPTR record for SIP. Based on this discovery, Lumicall feels there is a good probability the email address is also a SIP address. If multiple email addresses are in the address book, they will all be checked for NAPTR.
(Not shown in the picture) if Lumicall has been configured to use a SIP-to-PSTN gateway account, it will display the option to call Bob's number through the gateway in blue
Finally, at the bottom of the list, Lumicall offers the option to call Bob's number in the normal manner, through the GSM network. This option is colour-coded in red to emphasize that it is usually the most expensive.

The tip of the iceberg This experience is a big improvement over the experience of the old rotary dial but it could also be taken even further. People don't always respond to their SIP address, especially if they live in a country where mobile data is not yet the norm. These are areas where Lumicall could be improved. For example, we could use SIP presence to make sure that Bob really is online before displaying the SIP option. This would reduce frustration and boost confidence in SIP. We could even go beyond that: Bob could use the SIP presence mechanism to advertise whether or not he is on a slow mobile data network or a fast network like 4G or wifi. You may choose to defer your call to Bob until he enters wifi coverage where the call quality is likely to be better and maybe video will be possible. This latter opportunity also has privacy implications: Bob may only choose to share that live insight into his network coverage with contacts he trusts. Try it at FOSDEM I will spend some time around the real-time lounge at FOSDEM and would be delighted to get feedback from users who are trying Lumicall, FreePhoneBox.net, JSCommunicator, DruCall or the new Debian SIP service.

Emanuele Rocca: Antifeatures and Antibugs

Software Engineering distinguishes between software features and software bugs. It is usually understood that features are positive, expected characteristics of a computer program. Features make users happy by allowing them to do something useful, interesting, or fun. Something good, anyways. Bugs are instead undesirable and annoying. You're sitting there at your computer writing a long email and the software crashes right before your email is sent. Bad stuff. Features are generally implemented by programmers on purpose, whereas bugs are purely unintentional. They are mistakes. You don't make a mistake on purpose. We might at this point be inclined to think that: i) what is good for users is done on purpose by software manufacturers; ii) what is bad for users was not meant to be. It happened by mistake. Here is a handy table to visualize this idea:

	On purpose	By mistake
Good	Feature
Bad		Bug

It seems to make a lot of sense. But you might have noticed that two cells of the table are empty. Right! In a great talk titled When Free Software isn't better, Benjamin Mako Hill mentions the concept of antifeatures, and how they relate to Free Software. Antifeatures are features that make the software do something users will hate. Something they will hate so much they would pay to have those features removed, if that's an option. Microsoft Windows 7 is used in the talk to provide some examples of software antifeatures: the Starter Edition does not allow users to change their background image. Also, it limits the amount of usable memory on the computer to 2GBs, regardless of how much memory the system actually has. Two antifeatures engineered to afflict users to the point that they will purchase a more expensive version of the software, if they have the means to do that. I have another nice example. The Spotify music streaming service plays advertisements between songs every now and then. To make sure users are annoyed as much as possible, Spotify automatically pauses an advertisement if it detects that the volume is being lowered. A poor Spotify user even tried to report the bug on The Spotify Community forum, only to find out that what she naively considered as a software error was "intentional behavior". A spectacular antifeature indeed. Whenever a piece of technology does something you most definitely do not want it to do, such as allowing the NSA to take complete control of your Apple iPhone, including turning on its microphone and camera against your will, that's an antifeature.

	On purpose	By mistake
Good	Feature
Bad	Antifeature	Bug

Both bugs and antifeatures are bad for users. The difference between them is that antifeatures are engineered. Time and money are spent to make sure the goal is reached. A testing methodology is followed. "Are we really sure customers cannot change their wallpaper even if they try very very hard?" Engineering processes, of course, can fail. If the poor devils at Microsoft who implemented those harassments would have made a mistake that allows users to somehow change their wallpaper on Windows Starter... Well, I would call that a glorious antibug.

	On purpose	By mistake
Good	Feature	Antibug
Bad	Antifeature	Bug

There is no place for antifeatures in Free and Open Source Software. Free Software gives users control over what their software does. Imagine Mozilla adding a feature to Firefox that sets your speakers volume to 11 and starts playing a random song from the black metal artist Burzum every time you add a bookmark, unless you pay for Mozilla Firefox Premium Edition. The source code for Firefox is available under a free license. People who are not into Burzum's music would immediately remove this neat antifeature. I have spent many years of my life advocating Free and Open Source Software, perhaps using the wrong arguments. Mako's talk made me think about all this (thanks mate!). All these years I've been preaching about the technical superiority of Free Software, despite evidence of thousands of bugs and usability issues in the very programs I am using, and contributing to develop. Free Software is not better than Proprietary Software per se. Sometimes it is, sometimes it's not. But it gives you control, and freedom. When it annoys you, when it doesn't do what you expect and want, you can be sure it's not on purpose. And we can fix it together.

Petter Reinholdtsen: A fist full of non-anonymous Bitcoins

Bitcoin is a incredible use of peer to peer communication and encryption, allowing direct and immediate money transfer without any central control. It is sometimes claimed to be ideal for illegal activity, which I believe is quite a long way from the truth. At least I would not conduct illegal money transfers using a system where the details of every transaction are kept forever. This point is investigated in USENIX ;login: from December 2013, in the article "A Fistful of Bitcoins - Characterizing Payments Among Men with No Names" by Sarah Meiklejohn, Marjori Pomarole,Grant Jordan, Kirill Levchenko, Damon McCoy, Geoffrey M. Voelker, and Stefan Savage. They analyse the transaction log in the Bitcoin system, using it to find addresses belong to individuals and organisations and follow the flow of money from both Bitcoin theft and trades on Silk Road to where the money end up. This is how they wrap up their article:

"To demonstrate the usefulness of this type of analysis, we turned our attention to criminal activity. In the Bitcoin economy, criminal activity can appear in a number of forms, such as dealing drugs on Silk Road or simply stealing someone else s bitcoins. We followed the flow of bitcoins out of Silk Road (in particular, from one notorious address) and from a number of highly publicized thefts to see whether we could track the bitcoins to known services. Although some of the thieves attempted to use sophisticated mixing techniques (or possibly mix services) to obscure the flow of bitcoins, for the most part tracking the bitcoins was quite straightforward, and we ultimately saw large quantities of bitcoins flow to a variety of exchanges directly from the point of theft (or the withdrawal from Silk Road). As acknowledged above, following stolen bitcoins to the point at which they are deposited into an exchange does not in itself identify the thief; however, it does enable further de-anonymization in the case in which certain agencies can determine (through, for example, subpoena power) the real-world owner of the account into which the stolen bitcoins were deposited. Because such exchanges seem to serve as chokepoints into and out of the Bitcoin economy (i.e., there are few alternative ways to cash out), we conclude that using Bitcoin for money laundering or other illicit purposes does not (at least at present) seem to be particularly attractive."

These researches are not the first to analyse the Bitcoin transaction log. The 2011 paper "An Analysis of Anonymity in the Bitcoin System" by Fergal Reid and Martin Harrigan is summarized like this:

"Anonymity in Bitcoin, a peer-to-peer electronic currency system, is a complicated issue. Within the system, users are identified by public-keys only. An attacker wishing to de-anonymize its users will attempt to construct the one-to-many mapping between users and public-keys and associate information external to the system with the users. Bitcoin tries to prevent this attack by storing the mapping of a user to his or her public-keys on that user's node only and by allowing each user to generate as many public-keys as required. In this chapter we consider the topological structure of two networks derived from Bitcoin's public transaction history. We show that the two networks have a non-trivial topological structure, provide complementary views of the Bitcoin system and have implications for anonymity. We combine these structures with external information and techniques such as context discovery and flow analysis to investigate an alleged theft of Bitcoins, which, at the time of the theft, had a market value of approximately half a million U.S. dollars."

I hope these references can help kill the urban myth that Bitcoin is anonymous. It isn't really a good fit for illegal activites. Use cash if you need to stay anonymous, at least until regular DNA sampling of notes and coins become the norm. :) As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Russ Allbery: rra-c-util 5.2

Another release of my general collection of utility libraries, Autoconf probes, general test cases, and similar things. The major change in this release is the addition of Autoconf probes and portability wrappers for libevent. Now included are Autoconf macros to probe for libevent (mandatory or optional) either via pkg-config or not, following the pattern of other probes in rra-c-util. Also included is a portability wrapper, probes, and some additional glue code that allows programs to be written against the libevent 2.x API and work (mostly) with the libevent 1.4 API, albeit with some missing functionality. This release fixes the systemd unit directory probe to not produce shell errors if pkg-config isn't available on the local system. It also fixes a syntax error when building portable/krb5.h with a C++ compiler. There are multiple improvements to the Kerberos and process TAP add-ons: memory is properly freed in all processes, making exhaustive memory allocation analysis easier; process output is flushed after stopping the process instead of before; the method used to stop processes is cleaner; a memory leak when running via fakeroot has been fixed; and full use is made of the new test_cleanup_register API in C TAP Harness. This release also adds a new message_handlers_reset function that restores all message handlers to their defaults and frees any memory allocated to set different handlers. This again is mostly useful when doing exhaustive memory allocation testing. You can get the latest release from the rra-c-util distribution page.

Russ Allbery: C TAP Harness 3.0

This is my test harness and C and shell library for writing test suites using the Test Anything Protocol (also used by Perl). The changes in this release aren't that major, but I changed one API in a non-backward-compatible way, so I bumped the major revision. That change is to test_cleanup_register, whose callback functions now take two arguments instead of one. The first, as before, indicates whether the test case succeeded or failed. The second is now true if the callback is running in the primary process (the one in which plan or plan_lazy was called), and false otherwise. Callback functions are now called for all processes, not just the primary one. This allows cleanup of per-process resources, like memory, in the child processes, which allows for easier valgrind analysis. Also in this release, the test harness was modified to reopen standard input to /dev/null when running a set of tests from a list, rather than inheriting standard input from the test driver. The test harness also now cleans up all extraneous copies of open file descriptors used during the setup of standard input, output, and error to avoid leaking harmless but noticeable file descriptors to tests. You can get the latest release from the C TAP Harness distribution page.

28 January 2014

Andrea Colangelo: The (brand new) Debian 3D-Printing Team is hiring!

The 3D-printing technology has gained quite a big momentum in recent times. Its use is spreading widely even among hobbyists and many sources report daily several (sometimes weird) examples of its usage. The thing I like most, as a Free Software activist, is that several 3D-printing-related free and open source software are already available and quite good, and some printers are even released as free hardware. Unfortunately, Debian is quite lagged behind with regard to the availability of such software in our archive, especially compared to other distributions who already ship much of it. Hopefully this situation will change drastically thanks to the brand new Debian 3D-Printing team, whose purpose is to make Debian a rocking platform to do some serious 3D-printing. The team is bootstrapping right now, and we are looking for volunteers who are willing to help us. Packagers are greatly welcomed, but also triagers, users and passionate people with some experience in the field are extremely valuable. People willing to join the team can take a look at our (in-progress) wiki page and apply on the team page on Alioth. As with any good packaging team, a mailing list is available to share ideas, feel free to subscribe it, and feel free to join #debian-3dprinting on OFTC if you want to hang out with other team members.

Andrew Pollock: [life] Day 1, first day of Kindergarten

Yesterday was the first day of my new routine, except I haven't really established the new routine yet. I think that's going to take a few weeks. It was, however, the first day of Zoe's new routine, she started Kindergarten. "Kindergarten" is a bit of an ambiguous term, even within Australia, it seems. Back when I went to school, there was "pre-school", which was, as best I can determine, what "Kindergarten" is now, except a year later. It was optional, half a week, and the year preceding the start of primary ("elementary") school. I think every other state was already doing an extra year of school (which I think they called "Kindergarten", "Kinder", or something starting with "K") so Queensland blew up "pre-school", created "Prep" and now that's the first year of primary school. Zoe will start "Prep" next year. Then the Queensland education boffins realised that kids were missing out on the stuff that "pre-school" used to offer, so they reintroduced it in the form of "Kindergarten" (by Queensland standards). It's supposed to be play-based, and isn't really "long day care". Zoe does Monday, Tuesday and every other Wednesday, from 8:30am until 2:20pm. I'd only heard about C&K from a friend after moving back to Brisbane, so we were very fortunate to secure a place without having camped out on a waiting list from birth. We got very lucky, and ended up getting multiple offers, but we're pretty thrilled with the one that Zoe got into. It's a relatively new facility, right next to a state school and some bushland. They take the kids on excursions into the bushland, and up to the state school where there's some chickens. The Kindergarten facility itself is really lovely. Anyway, yesterday was Zoe's first day. I biked there yesterday morning (without Zoe) and it took me about 11 minutes. It took me more like 15 minutes to pick her up with the bike trailer. It's uphill on the way there, but not too much of a slog. It's been really great that Zoe's BFF from day care, Megan, also scored a place in the same Kindergarten, as I think this will help the transition for both girls greatly. I was able to stay for an extended period of time to help settle Zoe in, and I was able to leave without her being particularly distressed. I then biked home, running a few errands along the way. I then got stuck into a woodworking project (I'm trying to make a little wooden clothesline for Zoe). It turns out trying to cut a halved joint with a hand saw and chisel is a lot of work. I've managed to aggravate an old rotator cuff injury and the two pieces are a tad too tight, so I'm thinking I'll redo it and cut the joint out with an electric router instead. I probably spent at least a couple of hours faffing around with that. So I banged away with a hammer and chisel for a bit, got a haircut, had a late lunch and next thing I knew it was 2pm and time to head back on the bike to pick up Zoe. She was still asleep on her stretcher when I arrived to pick her up, which wasn't that surprising, since she tends to sleep later and longer at day care. I got a good opportunity to chat with her teacher about her first day and then gently woke her up with a kiss, and we headed home. I introduced Zoe to one of our new neighbours, and after a snack, we drove into Fortitude Valley where I had to pick up some mail from my post office box. When we got back home, I started on dinner while Zoe watched a little bit of TV. I spontaneously invited my cousin over to join us for dinner, and we walked down to the Hawthorne Garage for a babyccino after dinner. It was a good day. I realised that I'm going to have to be super regimented with my time on the days Zoe is in Kindergarten, because 2pm is going to roll around very quickly. "Scarcity brings clarity", as Sergey says...

Dirk Eddelbuettel: RQuantLib 0.3.11

A new minor / maintenance release RQuantLib 0.3.11 is now on CRAN and in Debian. Like the RcppClassic upload two days ago and the RcppZiggurat and RcppEigen uploads yesterday, this release was motivated at least in part by an upcoming Rcpp release about which I should have more to day soon. A few things got polished and updated, and we added a plugin which should make use via Rcpp Attribute easier. RQuantLib combines (some of) the quantitative analytics of QuantLib with the R statistical computing environment and language. Thanks to CRANberries, there is also a diff to the previous release 0.3.10. Full changelog details, examples and more details about this package are at my RQuantLib page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

27 January 2014

Kees Cook: -fstack-protector-strong

There will be a new option in gcc 4.9 named -fstack-protector-strong , which offers an improved version of -fstack-protector without going all the way to -fstack-protector-all . The stack protector feature itself adds a known canary to the stack during function preamble, and checks it when the function returns. If it changed, there was a stack overflow, and the program aborts. This is fine, but figuring out when to include it is the reason behind the various options. Since traditionally stack overflows happen with string-based manipulations, the default (-fstack-protector), only includes the canary code when a function defines an 8 (--param=ssp-buffer-size=N, N=8 by default) or more byte local character array. This means just a few functions get the checking, but they re probably the most likely to need it, so it s an okay balance. Various distributions ended up lowering their default --param=ssp-buffer-size option down to 4, since there were still cases of functions that should have been protected but the conservative gcc upstream default of 8 wasn t covering them. However, even with the increased function coverage, there are rare cases when a stack overflow happens on other kinds of stack variables. To handle this more paranoid concern, -fstack-protector-all was defined to add the canary to all functions. This results in substantial use of stack space for saving the canary on deep stack users, and measurable (though surprisingly still relatively low) performance hit due to all the saving/checking. For a long time, Chrome OS used this, since we re paranoid. :) In the interest of gaining back some of the lost performance and not hitting our Chrome OS build images with such a giant stack-protector hammer, Han Shen from the Chrome OS compiler team created the new option -fstack-protector-strong, which enables the canary in many more conditions:

local variable s address used as part of the right hand side of an assignment or function argument
local variable is an array (or union containing an array), regardless of array type or length
uses register local variables

This meant we were covering all the more paranoid conditions that might lead to a stack overflow. Chrome OS has been using this option instead of -fstack-protector-all for about 10 months now. As a quick demonstration of the options, you can see this example program under various conditions. It tries to show off an example of shoving serialized data into a non-character variable, like might happen in some network address manipulations or streaming data parsing. Since I m using memcpy here for clarity, the builds will need to turn off FORTIFY_SOURCE, which would also notice the overflow.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
struct no_chars  
    unsigned int len;
    unsigned int data;
 ;
int main(int argc, char * argv[])
 
    struct no_chars info =    ;
    if (argc < 3)  
        fprintf(stderr, "Usage: %s LENGTH DATA...\n", argv[0]);
        return 1;
     
    info.len = atoi(argv[1]);
    memcpy(&info.data, argv[2], info.len);
    return 0;

Built with everything disabled, this faults trying to return to an invalid VMA:

$ gcc -Wall -O2 -U_FORTIFY_SOURCE -fno-stack-protector /tmp/boom.c -o /tmp/boom
$ /tmp/boom 64 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Segmentation fault (core dumped)

Built with FORTIFY_SOURCE enabled, we see the expected catch of the overflow in memcpy:

$ gcc -Wall -O2 -D_FORTIFY_SOURCE=2 -fno-stack-protector /tmp/boom.c -o /tmp/boom
$ /tmp/boom 64 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
*** buffer overflow detected ***: /tmp/boom terminated
...

So, we ll leave FORTIFY_SOURCE disabled for our comparisons. With pre-4.9 gcc, we can see that -fstack-protector does not get triggered to protect this function:

$ gcc -Wall -O2 -U_FORTIFY_SOURCE -fstack-protector /tmp/boom.c -o /tmp/boom
$ /tmp/boom 64 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Segmentation fault (core dumped)

However, using -fstack-protector-all does trigger the protection, as expected:

$ gcc -Wall -O2 -U_FORTIFY_SOURCE -fstack-protector-all /tmp/boom.c -o /tmp/boom
$ /tmp/boom 64 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
*** stack smashing detected ***: /tmp/boom terminated
Aborted (core dumped)

And finally, using the gcc snapshot of 4.9, here is -fstack-protector-strong doing its job:

$ /usr/lib/gcc-snapshot/bin/gcc -Wall -O2 -U_FORTIFY_SOURCE -fstack-protector-strong /tmp/boom.c -o /tmp/boom
$ /tmp/boom 64 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
*** stack smashing detected ***: /tmp/boom terminated
Aborted (core dumped)

For Linux 3.14, I ve added support for -fstack-protector-strong via the new CONFIG_CC_STACKPROTECTOR_STRONG option. The old CONFIG_CC_STACKPROTECTOR will be available as CONFIG_CC_STACKPROTECTOR_REGULAR. When comparing the results on builds via size and objdump -d analysis, here s what I found with gcc 4.9: A normal x86_64 defconfig build, without stack protector had a kernel text size of 11430641 bytes with 36110 function bodies. Adding CONFIG_CC_STACKPROTECTOR_REGULAR increased the kernel text size to 11468490 (a +0.33% change), with 1015 of 36110 functions stack-protected (2.81%). Using CONFIG_CC_STACKPROTECTOR_STRONG increased the kernel text size to 11692790 (+2.24%), with 7401 of 36110 functions stack-protected (20.5%). And 20% is a far-cry from 100% if support for -fstack-protector-all was added back to the kernel. The next bit of work will be figuring out the best way to detect the version of gcc in use when doing Debian package builds, and using -fstack-protector-strong instead of -fstack-protector. For Ubuntu, it s much simpler because it ll just be the compiler default.

2014, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

DebConf team: Call for sponsorship for DebConf14 (Posted by Steve Langasek)

With DebConf13 behind us, the team is looking forward to DebConf14 this August in Portland, Oregon, USA. As a community-oriented conference, DebConf relies on its network of generous sponsors for funding. Please help us make DebConf14 a success - Become a sponsor today!

26 January 2014

Dirk Eddelbuettel: RcppEigen 0.3.2.0.2

A new maintenance release of RcppEigen is now on CRAN. And just like yesterday's RcppClassic upload and today's RcppZiggurat upload, the timing was motivated chiefly by some minor caretaking for an upcoming Rcpp release -- hopefully soon, and with more details to follow. But this release also contains some minor cleanups. The NEWS file entry shown below lists all changes.

Changes in RcppEigen version 0.3.2.0.2 (2014-01-26)

Converted three unused unit test files to RUnit and removed Suggests: of testthat

Add declaration to import a symbol from Rcpp to NAMESPACE to ensure proper instantiation with the upcoming Rcpp version

Retire SHLIB.maker function

Courtesy of CRANberries, there are diffstat reports for the most recent release relative to the preceding version. Questions, comments etc about RcppEigen should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel: RcppZiggurat 0.1.2

A new maintenance release of RcppZiggurat is now on CRAN. And just like yesterday's RcppClassic upload, this is motivated chiefly by some minor caretaking for an upcoming Rcpp release -- hopefully soon, and with more details to follow. This release also contains some small extensions and work-in-progress. By using R's uniform generator, we can fall back to R's seeding of the RNG -- but going through the R data structures has a performance hit. Similarly, and somewhat surprisingly, the default generators in both Boost and the C++11 library are not all that speedy. The NEWS file entry shown below lists all changes.

Changes in version 0.1.2 (2013-12-28)

Add a new generator deploying R's unif_rand which faciliates use of Ziggurat as the user-supplied generator for N(0,1)

Update a 'local' demo comparing normal distribution RNGs from Boost, C++11 and Armadillo none of which are particularly speedy

Add declaration to import a symbol from Rcpp to NAMESPACE to ensure proper instantiation with the upcoming Rcpp version

Courtesy of CRANberries, there are diffstat reports for the most recent release relative to the preceding version. More detailed information is on the RcppZiggurat page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel: RcppClassic 0.9.5

A maintenance release of RcppClassic, now at version 0.9.5, went out to CRAN today. Nothing changed in user-facing code. The build was triggered by a change in an upcoming Rcpp release (more about that another time), and a few minor updates were needed for extended checks done by R. Courtesy of CRANberries, there is the set of changes relative to the previous release 0.9.4. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

25 January 2014

Ben Hutchings: Looking back on my time at Solarflare, part 1

Friday was my last day at Solarflare Communications, which I joined in 2006 (originally as Level 5 Networks). When I started, Level 5 Networks (L5N) had a 2-port gigabit Ethernet controller called EF1 and a user-level network stack called EtherFabric which ran on Linux. It had nearly completed the Falcon project to develop a 10-gigabit controller (also supporing a 2-port gigabit mode) and was porting EtherFabric to Windows. Later that year, however, L5N merged with Solarflare Communications (SF), which was developing a 10GBASE-T PHY (for 10-gigabit Ethernet over twisted-pair cables). The investors and management saw the opportunity to create an integrated 10GBASE-T LAN-on-motherboard chip that would be attractive to OEMs and so had the potential for mass sales. EtherFabric was put on the back-burner, but regular net drivers for Linux, Windows and other operating systems became more important. In a change from my previous jobs, my initial role at L5N was to maintain and extend the in-house test automation software, Runbench, written in Python. I worked with Dickon Reed who had written most of Runbench up to then. One of my first major tasks was to extend it to cover DUTs running Windows. Aside from changing Runbench itself, I wrote a simple SSH server to be installed on DUTs, using the Twisted.Python framework. (Cygwin's port of OpenSSH wasn't suitable for running a lot of native Windows commands.) I rewrote the test-installation script for the Windows driver itself, converting it to Python (with a C extension to get around the WOW64 nonsense). As Windows does not allow loading a driver just once (like insmod in Linux), except by using the kernel debugger, I added a time check in the driver so the test-installation script can make it fail early when reloaded after a reboot, so that a crasher bug in the probe path would usually be recoverable. In 2007 I persuaded management in Cambridge to loan several older servers to DebConf. I took them there and back by car, and they were used to convert video for live streams and recordings. (This was repeated in 2009.) I was also asked to work on other small development tasks, including some cleanup on the Linux net driver (sfc). In mid-2007, when SF decided it was time to get sfc upstream (in-tree), I joined Steve Hodgson and Robert Stonehouse in working on that. Out-of-tree (separately distributed) drivers for Linux normally need a large number of preprocessor conditions for compatibility with Linux's kernel module API across a range of supported versions. They may also implement some features with a driver-specific interface that should be replaced with a standard interface (possibly including the work to define that standard). They may not comply with the kernel coding style or unwritten conventions for kernel code. All of these problems were present and had to be dealt with. As part of this work, I extended unifdef so that it could be used to remove all of the backward-compatibility and not-for-upstream code without introducing extra blank lines. I could then export the out-of-tree driver from CVS(!) into a kernel source tree automatically and turn it into a patch or patch series that would stand a chance of being acceptable upstream. Our first few attempts got little response, but eventually we newbies got the message that the driver was simply too big for a single submission and that putting patches on a web site because they're too big for the list was not a solution. In the next week, I removed about half of the code (on a git branch), resulting in a relatively lean driver that could be sent to the netdev list. And finally the 9th (I think) submission was accepted. Following this, I continued to work on sfc but was also brought into the Siena project. Siena (SFL9021) was the LAN-on-motherboard chip that had been planned following the merger. It combined a dual-port controller based on Falcon with a 10GBASE-T PHY and a management controller (MC) to support Lights-Out Management (LOM). (Since 10GBASE-T has not taken off, it is now mostly sold as the SFC9020 variant in which the PHY is disabled.) I worked on some of the test framework and test cases for validating the controller design in software simulation and FPGA. In the course of this I learned to read and (somewhat) understand the chip design written in Verilog, but only made a single trivial fix to it. In August 2009, the first Siena ASICs came back from the fab. There were a few ASIC-specific bugs but all of them were quickly worked around in firmware, so it would soon be ready for production. But there was much work still to be done on the driver and firmware. The firmware had until then been running in a smaller block of RAM with no peripherals to manage, while sfc had earlier been modified just enough to configure a sim or FPGA for running a userland test application. sfc now had many special cases scattered around it, and would tell the MC firmware to peek and poke registers that were no longer directly accessible from the host. The software and firmware developers had agreed that, in order to support LOM, the MC would be responsible for managing the ports and all peripherals and drivers would send higher-level requests to the MC. As a bonus, this removed the need for the driver to know the details of each new board, so that it would not be necessary to backport sfc into distribution kernels very often. Over the next few months, the firmware team did an excellent job of defining and implementing those operations, including writing the driver-side functions to invoke them. Meanwhile, Steve and I concentrated on refactoring sfc so most of the Falcon/Siena differences could be abstracted through a few structures with function pointers. The refactoring and new code for Siena were completed and submitted upstream in several large patch series in October and November. In fact, there were so many patches that Solarflare appeared in LWN's table of who wrote Linux 2.6.33 (as did I, thanks also to another patch series adding firmware metadata to many drivers). Sadly we had missed Linux 2.6.32 which was a longterm stable branch and used in many distributions, but I was able to get this version backported into all the major distributions over the next year. (To be continued.)

Next.

Previous.