In no particular order, here are my 4 picks for the DPL thunderdome:
For some reason that is not very important, I was looking for a better
solution for GPSes with maps.
Side-note: if you ask why Android is not good enough , here's the
story: last year, knowing that Google Maps supports caching (finally),
I upgraded Maps, preloaded London, and went on a trip. Only to exit
the Victoria underground station and to stare ~20 minutes in light
rain at my phone, wondering why it doesn't get a fix. I gave up in the
end, and used the cached maps for getting around. Later I learned the
reason: Maps was not broken, but without data, Android can't get a
lock easily; and more-over, the recent Maps update without GPS active
even once afterwards, to do whatever it needed, meant I couldn't get a
lock until back in Switzerland.
Or, alternatively: while on a trip to Japan, it turns out that Google
Maps caching is not supported there, probably because of rights over
mapping data. So I had location, but only temporary cached maps (from
the wifi in the hotels).
Due to this kind of problems makes I consider Android maps a toy, so I
wanted a GPS device with proper maps (on the device, not in the
cloud), and with good GPS capabilities (no fancy A-GPS or anything,
just plain good standalone GPS).
Having had good experiences with the sport line of Garmin (but which
doesn't have maps, so it's out of the question), I looked at
Garmin. Many people complained about some things (same all over the
place):
Summary
So, sorry again for sounding like an ad (I don't have any relation
with Garmin the company), but I find this device awesome - powerful,
rugged, and working (so far) very well. I'm glad that there still
exist companies who don't believe they know better than their
customers, and instead offer you with a tool that you can make it work
just like you want. Or almost 
Garmin's tag line: Big and Tough Goes anywhere, everywhere . I tend
to agree
That s a great tool, but don t you fear it makes unjustified overrides too easy ?In my experience, people sometimes have issues writing overrides (justified or not). In fact it sometimes leaves them really frustrated with Lintian. If that frustration eventually leads to them reject Lintian, then we are doing the project a disservice. I will quote Russ Allbery as I believe he covered when he wrote:
I care most about all of the regular Debian developers [...] continuing to use Lintian, so that Lintian can stay as effective as it is now at getting people to make archive-wide changes. [...] This only works if we can get nearly everyone uploading packages to run Lintian all the time.If a handful of people adds overrides for tags they should not have, then we can fix that (e.g. by filing a bug against their packages). But it is unlikely that we will ever make them run Lintian again once they have boycott it.
I'd like to respond (also) here to inquiries I'm receiving these days: I will not run again as DPL. So you have about 20 days to mob\^Wconvince other DDs to run, or decide to run yourself. Do not to wait for the vary last minute, as that makes for lousy campaigns.Ladies and gentlemen, I am pleased to present you... THE DPL GAME GOALS:
. We have the idea of arbitrary attachments for tests. Perhaps arbitrary attachments outside of test scope would be better than specifying stdout/stderr as specific things. On the other hand stdout and stderr are well known things.
Proposal version 2
A packetised length prefixed binary protocol, with each packet containing a small signature, length, routing code, a binary timestamp in UTC, a set of UTF8 tags (active only, no negative tags), a content tag one of (estimate + number, class TestInfo(object):
id = unicode
status = ('exists', 'inprogress', 'xfail', 'xsuccess', 'success', 'fail', 'error', 'skip')
runnable = boolean
class StreamingResult(object):
def startTestRun(self):
pass
def stopTestRun(self):
passs
def estimate(self, count, route_code=None, timestamp=None):
pass
def file(self, name, bytes, eof=False, mime=None, test_info=None, route_code=None, timestamp=None):
"""Inform the result about the contents of an attachment."""
def status(self, test_info, route_code=None, timestamp=None):
"""Inform the result about a test status with no attached data."""
This would permit the full semantics of a subunit stream to be represented I think, while being a narrow interface that should be easy to implement.
Please provide feedback! I ll probably start implementing this soon.
revno: 1 committer: Robert Collins <robertc@robertcollins.net> timestamp: Sat 2005-08-27 15:01:20 +1000 message: design up a protocol with kfishIt has proved remarkably resilient as a protocol the basic nature hasn t changed at all, even though we ve added tags, timestamps, support for attachments of arbitrary size. However a growing number of irritations have been building up with it. I think it is time to design another iteration of the protocol, one that will retain the positive qualities of the current protocol, while helping it become suitable for the next 7 years. Ideally we can keep compatibility and make it possible for a single stream to be represented in any format. Existing design The existing design is a mostly human readable line orientated protocol that can be sniffed out from the regular output of make or other build systems. Binary attachments are done using HTTP chunking, and the parser has to maintain state about the current test, tags, timing data and test progression [a simple stack of progress counters]. How to arrange subunit output is undefined, how to select tests to run is undefined. This makes writing a parser quite easy, and the tagging and timestamp facility allow multiplexing streams from two or more concurrent test runs into one with good fidelity but also requires that state be buffered until the end of a test, as two tests cannot be executing at once. Dealing with debuggers The initial protocol was intended to support dropping into a debugger just pass each line read through to stdout, and connect stdin to the test process, and voila, you have a working debugger connection. This works, but the current line based parsers make using it tedious the line buffered nature of it makes feedback on what has been typed fiddly, and stdout tends to be buffered, leading to an inability to see print statements and the like. All in-principle fixable, right ? When running two or more test processes, which test process should stdin be connected to? What if two or more drop into a debugger at once? What is being typed to which process is more luck than anything else. We ve added some idioms in testrepository that control test execution by a similar but different format one test per line to list tests, and have runners permit listing and selecting by a list. This works well, but the inconsistency with subunit itself is a little annoying you need two parsers, and two output formats. Good points The current protocol is extremely easy to implement for emitters, and the arbitrary attachments and tagging features have worked extremely well. There is a comprehensive Python parser which maps everything into Python unittest API calls (an extended version of the standard, with good backwards compatibility). Pain points The debugging support was a total failure, and the way the parser depraminates it s toys when a test process corrupts an outcome line is extremely frustrating. (other tests execute but the parser sees them as non-subunit chatter and passes the lines on through stdout). Dealing with concurrency The original design didn t cater for concurrency. There are three concurrency issues the corruption issue (see below for more detail) and multiplexing. Consider two levels of nested concurrency: A supervisor process such as testrepository starts 2 (or more but 2 is sufficient to reason about the issue) subsidiary worker processes (I1 and I2), each of which starts 2 subsidiary processes of their own (W1, W2, W3, W4). Each of the 4 leaf processes is outputting subunit which gets multiplexed in the 2 intermediary processes, and then again in the supervisor. Why would there be two layers? A concrete example is using testrepository to coordinate test runs on multiple machines at once, with each machine running a local testrepository to broker tests amongst the local CPUs. This could be done with 4 separate ssh sessions and no intermediaries, but that only removes a fraction of the issues. What issues? Well, consider some stdout chatter that W1 outputs. That will get passed to I1 and from there to the supervisor and captured. But there is nothing marking the chatter as belonging to W1: there is no way to tell where it came from. If W1 happened to fail, and there was a diagnostic message printed, we ve lost information. Or at best muddled it all up. Secondly, buffering imagine that a test on W1 hangs. I1 will know that W1 is running a test, but has no way to tell the supervisor (and thus the user) that this is the case, without writing to stdout [and causing a *lot* of noise if that happens a lot]. We could have I1 write to stdout only if W1 s test is taking more than 5 seconds or something but this is a workaround for a limitation of the protocol. Adding to the confusion, the clock on W1 and W3 may be very skewed, so timestamps for everything have to be carefully synchronised by the multiplexer. Thirdly, scheduling if W1/W2 are on a faster machine than W3/W4 then a partition of equal-timed tests onto each machine will lead one idle before the other finishes. It would be nice to be able to pass tests to run to the faster machine when it goes idle, rather than having to start a new runner each time. Lastly, what to do when W1 and W2 both wait for user input on stdin (e.g. passphrases, debugger input, $other). Naively connecting stdin to all processes doesn t work well. A GUI supervisor could connect a separate fd to each of I1 and I2, but that doesn t help when it is W1 and W2 reading from stdin. So additional requirement over baseline subunit:
test: foo starting serversuccess:fooThis is a classic example of corruption: the test foo started a server and helpfully wrote to stdout explaining that it did that, but missed the newline. As a result the success message for the test wasn t printed on a line of its own, and the subunit parser will believe that foo never completed. Every subsequent test is then ignored. This is usually easy to identify and fix, but its a head-scratcher when it happens. Another way it can happen is when a build tool like make runs tests in parallel, and they output subunit onto the same stdout file handle. A third way is when a build tool like make runs two separate test scripts serially, and the first one starts a test but errors hard and doesn t finish it. That looks like:
test: foo test: bar success: barOne way that this sort of corruption can be mitigated is to put subunit on it s own file descriptor, but this has several caveats: it is harder to tunnel through things like ssh and it doesn t solve the failing test script case. I think it is unreasonable to require a protocol where arbitrary interleaving of bytes between different test runner streams will work so the make -j2 case can be ignored at the wire level though we should create a simple way to safely mux the output from such tests when the execute. The root of the issue is that a dropped update leaves bad state in the parser and it never recovers. So some way to recover, or less state to carry in the parser, would neatly solve things. I favour reducing parser state as that should shift stateful complexity onto end nodes / complex processors, rather than being carried by every node in the transmission path. Dependencies Various suggestions have been made JSON, Protobufs, etc A key design goal of the first subunit was a low barrier to entry. We keep that by being backward compatible, but being easy to work with for the new revision is also a worthy goal. High level proposal A packetised length prefixed binary protocol, with each packet containing a small signature, length, routing code, a binary timestamp in UTC, a set of UTF8 tags (active only, no negative tags), a content tag one of (estimate + number, stdin, stdout, stderr, test- + test id), test status (one of exists/inprogress/xfail/xsuccess/success/fail/skip), an attachment name, mime type, a last-block marker and a block of bytes. The content tags:
class StreamingResult(object):
def startTestRun(self):
pass
def stopTestRun(self):
pass
def estimate(self, count, route_code=None):
pass
def stdin(self, bytes, route_code=None):
pass
def stdout(self, bytes, route_code=None):
pass
def test(self, test_id, status, attachment_name=None, attachment_mime=None, attachment_eof=None, attachment_bytes=None):
pass
This would support just-in-time debugging by wiring up pdb to the stdin/stdout handlers of the result object, rather than actual stdin/stdout of the process a simple matter once written. Alternatively, the test runner could replace sys.stdin/stdout etc with thunk file-like objects, which might be a good idea anyway to capture spurious output happening during a test run. That would permit pdb to Just Work (even if the test process is internally running concurrent tests.. until it has two pdb objects running concurrently
Generation new streams
Should be very easy in anything except shell. For shell, we can have a command line tool that when invoked outputs a subunit stream for one instruction. E.g. test foo completed + some attachments or test foo starting .
case foo inand
[!m]oo)
echo bar
;;
esac
# everything but backups:
for file in documents/*[!~]; do
echo doing something with "$file" ...
done
$ printf "foo\nbar\nbaz\n" grep '^[^b]'
foo
$ printf "foo\nbar\nbaz\n" grep '^[!b]'
bar
baz
What is this?
umockdev is a set of tools and a library to mock hardware devices for programs that handle Linux hardware devices. It also provides tools to record the properties and behaviour of particular devices, and to run a program or test suite under a test bed with the previously recorded devices loaded.
This allows developers of software like gphoto or libmtp to receive these records in bug reports and recreate the problem on their system without having access to the affected hardware, as well as writing regression tests for those that do not need any particular privileges and thus are capable of running in standard make check.
After working on it for several weeks and lots of rumbling on G+, it s now useful and documented enough for the first release 0.1!
Component overview
umockdev consists of the following parts:
umockdev-record program generates text dumps (conventionally called *.umockdev) of some specified, or all of the system s devices and their sysfs attributes and udev properties. It can also record ioctls that a particular program sends and receives to/from a device, and store them into a text file (conventionally called *.ioctl).UMockdevTestbed GObject class which builds sysfs and /dev testbeds, provides API to generate devices, attributes, properties, and uevents on the fly, and can load *.umockdev and *.ioctl records into them. It provides VAPI and GI bindings, so you can use it from C, Vala, and any programming language that supports introspection. This is the API that you should use for writing regression tests. You can find the API documentation in docs/reference in the source directory.umockdev-wrapper program.umockdev-run program builds a sandbox using libumockdev, can load *.umockdev and *.ioctl files into it, and run a program in that sandbox. I. e. it is a CLI interface to libumockdev, which is useful in the debug a failure with a particular device use case if you get the text dumps from a bug report. This automatically takes care of using the preload library, i. e. you don t need umockdev-wrapper with this. You cannot use this program if you need to simulate uevents or change attributes/properties on the fly; for those you need to use libumockdev directly.Bus 001 Device 012: ID 0fce:0166 Sony Ericsson Xperia Mini Pro
$ umockdev-record /dev/bus/usb/001/012 > mobile.umockdev
$ umockdev-record --ioctl mobile.ioctl /dev/bus/usb/001/012 mtp-detect $ umockdev-record --ioctl mobile.ioctl /dev/bus/usb/001/012 mtp-emptyfolders
/dev/bus/usb/001/012 merely echoes what is in mobile.umockdev and it is independent of what is actually in the real /dev directory. You can rename that device in the generated *.umockdev files and on the command line.
$ umockdev-run --load mobile.umockdev --ioctl /dev/bus/usb/001/012=mobile.ioctl mtp-detect $ umockdev-run --load mobile.umockdev --ioctl /dev/bus/usb/001/012=mobile.ioctl mtp-emptyfolders
umockdev-run. As a simple example, let s pretend we want to write tests for upower.
Batteries, and power supplies in general, are simple devices in the sense that userspace programs such as upower only communicate with them through sysfs and uevents. No /dev nor ioctls are necessary. docs/examples/ has two example programs how to use libumockdev to create a fake battery device, change it to low charge, sending an uevent, and running upower on a local test system D-BUS in the testbed, with watching what happens with upower --monitor-detail. battery.c shows how to do that with plain GObject in C, battery.py is the equivalent program in Python that uses the GI binding. You can just run the latter like this:
umockdev-wrapper python3 docs/examples/battery.pyand you will see that upowerd (which runs on a temporary local system D-BUS in the test bed) will report a single battery with 75% charge, which gets down to 2.5% a second later. The gist of it is that you create a test bed with
UMockdevTestbed *testbed = umockdev_testbed_new ();and add a device with certain sysfs attributes and udev properties with
gchar *sys_bat = umockdev_testbed_add_device (
testbed, "power_supply", "fakeBAT0", NULL,
/* attributes */
"type", "Battery",
"present", "1",
"status", "Discharging",
"energy_full", "60000000",
"energy_full_design", "80000000",
"energy_now", "48000000",
"voltage_now", "12000000",
NULL,
/* properties */
"POWER_SUPPLY_ONLINE", "1",
NULL);
You can then e. g. change an attribute and synthesize a change uevent with
umockdev_testbed_set_attribute (testbed, sys_bat, "energy_now", "1500000"); umockdev_testbed_uevent (testbed, sys_bat, "change");With Python or other introspected languages, or in Vala it works the same way, except that it looks a bit leaner due to proper object semantics. Packages I have a packaging branch for Ubuntu and a recipe to do daily builds with the latest upstream code into my daily builds PPA (for 12.10 and raring). I will soon upload it to Raring proper, too. What s next? The current set of features should already get you quite far for a range of devices. I d love to get feedback from you if you use this for anything useful, in particular how to improve the API, the command line tools, or the text dump format. I m not really happy with the split between umockdev (sys/dev) and ioctl files and the relatively complicated CLI syntax of
umockdev-record, so any suggestion is welcome.
One use case that I have for myself is to extend the coverage of ioctls for input devices such as touch screens and wacom tablets, so that we can write some tests for gnome-settings-daemon plugins.
I also want to find a way to pass ioctls back to the test suite/calling program instead of having to handle them all in the preload library, which would make it a lot more flexible. However, due to the nature of the ioctl ABI this is not easy.
Where to go to
The code is hosted on github in the umockdev project; this started out as a systemd branch to add this functionality to libudev, but after a discussion with Kay we decided to keep it separate. But I kept it in git anyway, given how popular it is today. For the bzr lovers, Launchpad has an import at lp:umockdev.
Release tarballs will be on Launchpad as well. Please file bugs and enhancement requests in the git hub tracker.
Finally, if you have questions or want to discuss something, you can always find me on IRC (pitti on Freenode or GNOME).
Thanks for your attention and happy testing!
X-INVOICE-NUMBER:1 X-INVOICE-AMOUNT:NOK1000.00 X-INVOICE-KID:123412341234 X-INVOICE-MSG:Donation to Debian Edu X-BANK-ACCOUNT-NUMBER:16040884339 X-BANK-IBAN-NUMBER:NO8516040884339 X-BANK-SWIFT-NUMBER:DNBANOKKXXXThe X-BANK-ACCOUNT-NUMBER field was proposed in a stackoverflow answer regarding how to put bank account information into a vCard. For payments in Norway, either X-INVOICE-KID (payment ID) or X-INVOICE-MSG could be used to pass on information to the seller when paying the invoice. The complete vCard could look like this:
BEGIN:VCARD VERSION:2.1 ORG:SLX Debian Labs Foundation ADR;WORK:;;Gunnar Schjelderups vei 29D;OSLO;;0485;Norway URL;WORK:http://www.linuxiskolen.no/slxdebianlabs/ EMAIL;PREF;INTERNET:sdl-styret@rt.nuug.no REV:20130212T095000Z X-INVOICE-NUMBER:1 X-INVOICE-AMOUNT:NOK1000.00 X-INVOICE-MSG:Donation to Debian Edu X-BANK-ACCOUNT-NUMBER:16040884339 X-BANK-IBAN-NUMBER:NO8516040884339 X-BANK-SWIFT-NUMBER:DNBANOKKXXX END:VCARDThe resulting QR code created using qrencode would look like this, and should be readable (and thus checkable) by any smart phone, or for example the zbar bar code reader and feed right into the approval and accounting system.
The extension fields will most likely not show up in any normal
vCard reader, so those parts would have to go directly into a system
handling invoices. I am a bit unsure how vCards without name parts
are handled, but a simple test indicate that this work just fine.
Update 2013-02-12 11:30: Added KID to the proposal
based on feedback from Sturle Sunde.
The first time I met my Debian team mates (from the Publicity and WWW
teams) was during DebConf11 in Banja Luka, in 2011.
I was very excited, and being the craft geek I am, I decided to bring them
little handmade gifts.
After some thought, I opted for a Debian swirl plush charm/keychain. And
here is how I made it, in six easy steps.
It's a really simple project, you just need a bit of patience for the
sewing part: the swirl is sewn by hand and it's very important to
make regular stitches.
The finished swirl measures approximately 7.5x12 cm.
What you'll need
Make a sketch of your swirl on paper or, if you don't feel too confident
about your drawing skills, print this pattern I made.
Trace it on the felt and cut it on felt twice (you'll need two
swirl-shaped pieces for the plushie).
You can also fold the felt double and cut just one time (as I did).
Step 2
Pin the two swirl-shaped pieces together, with the wrong sides inside.
With the red thread, using back stitch, sew them together, starting
from the lower end as indicated in the image above.
Step 3
When you reach the point marked in the image above, start
stuffing the filling into the curved part of the swirl (use a
pen to stuff it more easily in the narrow part of the curve).
Step 4
Continue stuffing the swirl and sewing it until you reach the point marked
in the picture above.
Step 5
Now it's time to add the ribbon and the ring: cut 4 cm of ribbon and fold
it lenghtwise. Slide on it the ring and fold it again, then put the two
ends between the folds of the swirl, as shown in the picture above.
Step 6
Continue sewing til the end.
Congratulations, you now have a Debian plushie.
I really loved creating these, and I'm really happy to know that
the swirls I made are now scattered around the world: one in Austria,
one in Australia, one in Denmark, four in Germany, three in Italy and one
in Martinique.
And the one I made for this tutorial will end in Italy as well (Mark, it's
yours).
If you have any doubts about the pattern, you can contact
me. And if you make one, please send me a pic.
This pattern is released under CC-BY-SA 3.0
license: this
means that you can share, distribute, modify and even use it commercially
but you'll need to mention the original author (OH HAI! it's me!) and
maintain the same license.
The above two pictures show me holding a Samsung Galaxy S3 which has a 4.8 display in my left hand and a Sony Ericsson Xperia X10 which has a 4.0 display in my right hand. I am holding both phones in a manner that allows me to touch the top opposite corner with my thumb the position I need for one-handed phone use. The Xperia X10 can be completely enclosed by my hand, when I have a bottom corner resting in my palm it won t slide down while the Galaxy S3 can slide.
Also one thing I didn t realise before having the pictures taken is that my posture is quite different when using the two phones. With the Galaxy S3 my wrist is clearly bent and this seems more likely to cause me to have more problems with Carpal Tunnel Syndrome [1]. I haven t had any serious problems with CTS for the last 2.5 years but I have had minor problems that suggest that I will have to be careful about my posture for the rest of my life.
So it seems that a 4.8 phone is just too big for ideal one-handed use (grasping the vertical phone from the bottom). As I had CTS problems with my left hand I will try to use my new phone with my right hand as much as possible. Also I can reach further than the width of the phone screen when I grasp it from the side, so for me a size of about 5.2 would be better than the 4.8 of the Galaxy S3. It s quite likely that the Samsung Galaxy Note with it s 5.3 screen would be a better device for me to grasp from the side. But the Galaxy Note 2 might be a little too large for me.
Note that I am only considering ways of holding the device that permit full operation. Anything that involves changing position for different uses or occasionally using two hands for a mostly one-hand operation doesn t count.
The above picture is of me holding a 7 Android tablet (which I have just returned to Aldi [2]). When holding it from the sides I can reach more than half the screen with one hand so it seems that the ideal size for a tablet to be held in two hands for me would be 8 or even a little larger. A tablet larger than that could only be properly used if resting on my lap or a desk so for me 8 is the size that differentiates things which can be strictly used as tablets (holding with two hands and using thumbs for input) and things which are more like Netbooks (on desk typing).
Ideal Device Sizes for people based on Height
I am about 190cm tall. If we assume that height and hand size are strongly correlated then we can look at median heights of various age groups and determine what might be a good device size. I am also assuming that everyone wants to have the largest possible device, but some people have other criteria such as the size of their pockets.
For me it s 8 tablet, 5.2 side-grip phone, and 4 bottom-grip phone. I used data from a chart of the average heights of American boys [3] and a chart of the average heights of American girls [4] to determine what size devices might suit children of various ages. Note that before the age of 12 the height of boys and girls is near enough to identical.
| Device | Two Hand Tablet Use | Phone Grasp from Side | Phone Grasp from Below |
|---|---|---|---|
| 7 tablet | 14yo boy or 17yo girl | noone | noone |
| Galaxy Note 5.3 | 10yo | 95th percentile 17yo boy | noone |
| Galaxy S3 4.8 | 6yo | 17yo boy or 95th percentile 17yo girl | almost noone |
| Galaxy S and iPhone 5 4.0 | 3yo | 10yo | 95th percentile 17yo boy |
| iPhone 4 3.5 | noone | 8yo | 14yo boy or 16yo girl |
until a few hours ago I thought there wouldn't be anything for my weekly
RC bug report. then I at least found one bug to work on:
We've just put the new PostgreSQL minor releases live on apt.postgresql.org.
Building 5 major versions for 10 distributions produces quite a lot of stuff:
$ readelf --segments /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
Elf file type is DYN (Shared object file)
Entry point 0x77c00
There are 7 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x00000000001b5ea4 0x00000000001b5ea4 R E 200000
LOAD 0x00000000001b6b60 0x00000000003b6b60 0x00000000003b6b60
0x0000000000028dc4 0x000000000002c998 RW 200000
The permissions to load the code and data segments are given the
Flags output. Code has execute-permissions, and data has
write-permissions. These flags are mapped into flags for the mmap
call, which the loader then uses to map the various segments of the
file into memory.
So, do you actually need execute-permissions on the underlying file
to mmap that segment as executable? No, because you can read it.
If I can read it, then I can copy the segment to another area of
memory I have already mapped with PROT_EXEC and execute it there
anyway.
Googling suggests that some systems do require execute-permissions
on a file if you want to directly mmap pages from it with
PROT_EXEC (and if you dig into the kernel source, there's an
ancient system call uselib that looks like it comes from a.out
days, given it talks about loading libraries at fixed addresses, that
also wants this). This doesn't sound like a terrible hardening step;
I wouldn't be surprised if some hardening patches require it. Maximum
compatability and historical-features such as a.out also probably
explains why gcc creates shared libraries with execute permissions
by default.
Thus, should you feel like it, you can run a shared-library.
Something trivial will suffice:
int function(void)
return 100;
$ gcc -fPIC -shared -o libfoo.so foo.c $ ./libfoo.so Segmentation faultThis is a little more interesting (to me anyway) to dig into. At a first pass, why does this even vaguely work? That's easy -- an ELF file is an ELF file, and the kernel is happy to map those PT_LOAD segments in and jump to the entry point for you:
$ readelf --segments ./libfoo.so
Elf file type is DYN (Shared object file)
Entry point 0x570
There are 6 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x000000000000070c 0x000000000000070c R E 200000
LOAD 0x0000000000000710 0x0000000000200710 0x0000000000200710
0x0000000000000238 0x0000000000000240 RW 200000
What's interesting here is that the shared-library has an entry point
(e_entry) at all. ELF defines it as:
This member gives the virtual address to which the system first transfers control, thus starting the process. If the file has no associated entry point, this member holds zero.First things first, where did that entry point come from? The ld manual tells us that the linker will set the entry point based upon the following hierarchy:
We know we're not specifying an entry point. The ENTRY command is interesting; we can check our default-link script to see if that is specified:
- the -e entry command-line option;
- the ENTRY(symbol) command in a linker script;
- the value of a target specific symbol
- the address of the first byte of the .text section, if present;
- The address 0.
$ ld --verbose grep ENTRY ENTRY(_start)Interesting. Obviously we didn't specify a _start; so do we have one? A bit of digging leads to the crt files, for C Run-Time. These are little object files automatically linked in by gcc that actually do the support-work to get a program to the point that main is ready to run. So, if we go and check out the crt files, one can find a definition of _start in crt1.o
$ nm /usr/lib/x86_64-linux-gnu/crt1.o
crt1.o:
0000000000000000 R _IO_stdin_used
0000000000000000 D __data_start
U __libc_csu_fini
U __libc_csu_init
U __libc_start_main
0000000000000000 T _start
0000000000000000 W data_start
U main
But do we have that for our little shared-library? We can get a feel
for what gcc is linking in by examining the output of
-dumpspecs. Remembering gcc is mostly just a driver that
calls out to other things, a specs file is what gcc uses to
determine which arguments pass around to various stages of a compile:
$ gcc -dumpspecs ... *startfile: % !shared: % pg p profile:gcrt1.o%s;pie:Scrt1.o%s;:crt1.o%s crti.o%s % static:crtbeginT.o%s;shared pie:crtbeginS.o%s;:crtbegin.o%sThe format isn't really important here (of course you can read about it); but the gist is that various flags, such as -static or -pie get passed different run-time initailisation helpers to link-in. But we can see that if we're creating a shared library we won't be getting crt1.o. We can double-confirm this by checking the output of gcc -v (cut down for clarity).
$ gcc -v -fPIC -shared -o libfoo.so foo.c Using built-in specs. ... /usr/lib/gcc/x86_64-linux-gnu/4.4.5/collect2 -shared -o libfoo.so /usr/lib/gcc/x86_64-linux-gnu/4.4.5/../../../../lib/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.4.5/crtbeginS.o /tmp/ccRpsQU3.o /usr/lib/gcc/x86_64-linux-gnu/4.4.5/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/4.4.5/../../../../lib/crtn.oSo this takes us further down ld's entry-point logic to pointing to the first bytes of .text, which is where the entry-point comes from. So that solves the riddle of the entry point. There's one more weird thing you notice when you run the library, which is the faulting address in kern.log:
libfoo.so[8682]: segfault at 1 ip 0000000000000001 sp 00007fffcd63ec48 error 14 in libfoo.so[7f54c51fa000+1000]The first thing is decoding error; 14 doesn't seem to have any relation to anything. Of course everyone has the Intel 64 Architecture Manual (or mm/fault.c that also mentions the flags) to decode this into 1110 which means "no page found for a user-mode write access with reserved-bits found to be set" (there's another post in all that someday!). So why did we segfault at 0x1, which is an odd address to turn up? Let's disassemble what actually happens when this starts.
00000000000004a0 <call_gmon_start>:
4a0: 48 83 ec 08 sub $0x8,%rsp
4a4: 48 8b 05 2d 03 20 00 mov 0x20032d(%rip),%rax # 2007d8 <_DYNAMIC+0x190>
4ab: 48 85 c0 test %rax,%rax
4ae: 74 02 je 4b2 <call_gmon_start+0x12>
4b0: ff d0 callq *%rax
4b2: 48 83 c4 08 add $0x8,%rsp
4b6: c3 retq
$ readelf --relocs ./libfoo.so Relocation section '.rela.dyn' at offset 0x3f0 contains 4 entries: Offset Info Type Sym. Value Sym. Name + Addend 000000200810 000000000008 R_X86_64_RELATIVE 0000000000200810 0000002007d8 000200000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0 0000002007e0 000300000006 R_X86_64_GLOB_DAT 0000000000000000 _Jv_RegisterClasses + 0 0000002007e8 000400000006 R_X86_64_GLOB_DAT 0000000000000000 __cxa_finalize + 0 Relocation section '.rela.plt' at offset 0x450 contains 1 entries: Offset Info Type Sym. Value Sym. Name + Addend 000000200808 000400000007 R_X86_64_JUMP_SLO 0000000000000000 __cxa_finalize + 0Thus call_gmon_start, rather unsurprisingly, checks the value of __gmon_start__ and calls it if it is set. Presumably this is set as part of profiling and called during library initialisation -- but it is clearly not an initialiser by itself. The retq ends up popping a value off the stack and jumping to it, which in this case just happens to be 0x1 -- which we can confirm with gdb by putting a breakpoint on the first text address and examining the stack-pointer:
(gdb) x/2g $rsp 0x7fffffffe7d8: 0x0000000000000000 0x0000000000000001So that gives us our ultimate failure. Of course, if you're clever, you can get around this and initalise yourself correctly and actually make your shared-library do something when executed. The canonical example of this is libc.so itself:
$ /lib/x86_64-linux-gnu/libc-2.13.so GNU C Library (Debian EGLIBC 2.13-37) stable release version 2.13, by Roland McGrath et al. Copyright (C) 2011 Free Software Foundation, Inc. This is free software; see the source for copying conditions. ...You can trace through how this actually does work in the same way as we traced through why the trivial example doesn't work. If you wondering my opinion on executable-bits for shared-libraries; I would not give them execute permissions. I can't see it does anything but open the door to confusion. However, understanding exactly why the library segfaults the way it does actually ends up being a fun little tour around various parts of the toolchain!
Edward Tufte s Speech From Aaron Swartz s Memorial
Speech starts 41:00 [video cuts out in beginning]
We would then meet over the years for a long talk every now and then, and my responsibility was to provide him with a reading list, a reading list for life and then about two years ago Quinn had Aaron come to Connecticut and he told me about the four and a half million downloads of scholarly articles and my first question is, Why isn t MIT celebrating this? .
[Video cuts out again]
Obviously helpful in my career there, he then became president of the Mellon foundation, he then retired from the Mellon foundation, but he was asked by the Mellon foundation to handle the problem of JSTOR and Aaron. So I wrote Bill Bullen(sp?) an email about it, I said first that Aaron was a treasure and then I told a personal story about how I had done some illegal hacking and been caught at it and what happened. In 1962, my housemate and I invented the first blue box, that s a device that allows for free, undetectable, unbillable long distance telephone calls. And we got this up and played around with it and the end of our research came when we concluded what was the longest long distance call ever made, which was from Palo Alto to New York time-of-day via Hawaii, well during our experimentation, AT&T, on the second day it turned out, had tapped our phone and uh but it wasn t until about 6 months later when I got a call from the gentleman, AJ Dodge, senior security person at AT&T and I said, I know what you re calling about. and so we met and he said You what you are doing is a crime that would , you know all that. But I knew it wasn t serious because he actually cared about the kind of engineering stuff and complained that the tone signals we were generating were not the standard because they record them and play them back in the network to see what numbers they we were that you were trying to reach, but they couldn t break though the noise of our signal. The upshot of it was that uh oh and he asked why we went off the air after about 3 months, because this was to make long distance telephone calls for free and I said this was because we regarded it as an engineering problem and we made the longest long distance call and so that was it. So the deal was, as I explained in my email to Bill Bullen, that we wouldn t try to sell this and we were told, I was told that crime significance would pay a great deal for this, we wouldn t do any more of it and that we would turn our equipment over to AT&T, and so they got a complete vacuum tube isolator kit for making long distance phone calls. But I was grateful for AJ Dodge and I must say, AT&T that they decided not to wreck my life. And so I told Bill Bullen that he had a great opportunity here, to not wreck somebody s life, course he thankfully did the right thing.
Aaron s unique quality was that he was marvelously and vigorously different. There is a scarcity of that. Perhaps we can be all a little more different too.
Thank you very much.
This post is a quick status update on DebConf13, for those who aren t following the debconf-team mailing list.
As you may know, this year s DebConf will be held at Le Camp, in Vaumarcus, Switzerland. Our concept for this DebConf is to hold it in a natural environment, away from distractions. We hope you will enjoy spending time together with other Debian collaborators in this beautiful part of Switzerland, on the shore of Lake Neuch tel.
Dates
DebConf13 will take place from Sunday 11. August 2013 to Sunday 18. August 2013. (We will use Saturday 10. August 2013 to prepare the venue for the conference.)
You may notice that the dates don t cover two weeks like in the last few years (there is no separate DebCamp week). For budget reasons, current plans are to merge the two weeks activities into an 8 day period. If you think that s a pity, it s not too late to change it just join the fundraising team and start working quickly!
Travel
If you want to start arranging your travel to attend DebConf, some initial travel suggestions may be useful:
Lots of exciting things are happening with and around Rcpp.
I just added a new page about Upcoming Events to the recently-created
Rcpp site. This events page has lots to cover: an upcoming talk at Columbia on March 8 (details still TBD),
a day-long workshop in New York on March 9, a possible participation at a CERN / ROOT conference in Switzerland on May
11-14, an upcoming talk in May in Milwaukee, and last but not least the tutorial by Romain and Hadley at UseR!
2013 in Spain. Phew!
With that, a few quick words about the
upcoming master class in New York.
It will be a full day, covering an introduction and motivation, details about the core data types, tools for working with and
and extending Rcpp and of course applications galore, including RcppArmadillo and RInside. I have done the same one day
class format a few times before, most recently (with Revolution Analytics) in San Francisco in late 2011, and also as a two-part seminar at UseR!
2012. This time, we plan on providing cloud-hosted RStudio instances for participants. Better still, RStudio's own JJ
Allaire will be on deck as well for RStudio --- and Rcpp Attributes --- questions.
Details and registration information for the New York class are at
this page.
Next.