Search Results: "reg"

13 January 2013

Russell Coker: Android Multitasking

My new Samsung Galaxy S3 has support for Multi Window Mode , here is a video which shows how to use this, Multi Window Mode starts at about 2:30 [1]. A common complaint about Android is the lack of multitasking, which is partly true and only slightly alleviated by Multi Window Mode. Running Multiple Programs Traditionally Android has multitasked with a similar ability to a Unix shell session, you can have applications running in the background and switch between them but you can t see multiple applications running at the same time (apart from seeing notification messages at the top of the screen). The new Multi Window Mode allows two or more applications to share a screen. But it only applies to a small number of applications which support it, on my phone that is applications shipped by Samsung and Google Chrome. Also I can t have multiple copies of Chrome open at the same time which means I can t do the things that I do on every PC that runs Chromium (the non-Google build of Chrome). I have not yet found a situation where Multi Window Mode has been useful to me, the applications I use for most tasks don t support it. So while Android being based on Linux does multitask really well in the technical computer-science definition it doesn t do so well in the user-centric definition. In practice Android multitasking is mostly about task switching and doing things like checking email in the background. Having multiple programs running at once is particularly difficult due to the Android model of applications sometimes terminating when they aren t visible. A common task is to view a message in a MUA and switch between that and another window (EG a web browser). K9 is my preferred MUA for Android which seems to have no option to switch back and still be viewing the same message as before the task switch so at least three actions need to be taken to get back to the same message after I resume K9. One major feature to make multitasking on Android more usable would be a way of rapidly switching between two applications and being certain that each one would be in the same state when the user switches back to it. Copy and Paste Another problem with Android multitasking is the difficulty in copying and pasting text. While copy/paste is not strictly required for multitasking it is a logical requirement when you have multiple applications running on behalf of a single user. For PCs everyone knows that you select text by holding down the SHIFT key while using the cursor control keys or by holding down the mouse button and swiping the mouse cursor over the text and you then use CTRL-C or the Edit menu to copy text. On Android it s a long press to select text which then gives you markers for the start and end, you drag those markers around and then select that you want to copy the text. This is at best a lot more inconvenient than using a high resolution input device like a mouse to select text. At worst it doesn t seem particularly reliable, K9 Mail for example won t let me copy text from a message for unknown reasons on a desktop OS such problems are vanishingly rare such that I can t think of an example of it happening. IO and Multitasking Multitasking for a user (as opposed to the multitasking needed to host dozens or hundreds of concurrent users) on Unix servers was very limited in the days of VT100 terminals and similar devices. Programs such as GNU Screen [2] allowed a text display to perform windowing functions that are similar to a modern GUI. But generally it seems that the ability for a user to run multiple programs at once is largely limited by their ability to see the output and to rapidly switch between programs or sessions. As a keyboard with ~100 keys and a text display with 80*25 characters is a major limitation it s obviously going to be a comparable (and often greater) limitation to have an on-screen keyboard that takes half the screen space, a single program taking all the screen, and a drop-down status bar that might be useful for multitasking. With Android 4.0 and above you can activate a task switcher by holding down the home button for two seconds [3]. There are also a variety of third party task switching programs on the Google Play store which all seem to start by holding down the home button. One problem with these options is that they require an extended press of the home button where the ideal is something that is as quick as ALT-TAB or a single mouse movement on a desktop system. One possible input action would be to switch between the most recent tasks with a palm swipe on the screen an operation that is quick and easy. Currently a palm swipe is used for a screen capture, but as there are four possible directions for swiping the screen one of them could be used for screen capture and another two for task switching. But this wouldn t do a lot of good without the ability to switch tasks without losing the context that either requires Android application changes or having the OS not tell an application that it was occluded. The iPaQ had interesting capabilities for input, it had a main button on the front that could be pushed in four directions (usually for cursor control) as well as being pushed in, four extra buttons on the front, and a button on the side [4]. I don t recall the methods that Familiar Linux on the iPaQ [5] used for task switching, but it was less of an issue as the iPaQ had less RAM, no Internet access and no telephony functions. I think that adding a bunch of extra buttons to an Android phone would make it a lot more useful. The iPaQ method of cursor control is one that could be considered (it could alleviate the copy/paste problems among other things). As an aside in two years of using Android phones I ve done less serious writing on Android than I did in my first two years of using Familiar on the iPaQ largely due to the lack of keys and a stylus on my Android phones. The screen size on Android phones is also a limit for multitasking. The earlier/cheaper phones that have small screens with resolutions such as 320*480 have very limited ability to display two programs at once. The 720*1280 display in the Galaxy S3 has a lot more potential in this regard and the Galaxy Note 2 and the HTC J Butterfly AKA Droid DNA have even more potential. In the future it seems that screen size limitations on phone multitasking will be a solved problem for everyone who can afford one of the high end phones which incidentally are much cheaper than the iPaQ was 10 years ago. Conclusion Checking email in the background etc is very useful on Android systems. But in terms of the user running two programs at once it seems very limited, and that situation seems likely to remain until the vendors adopt multi-window support. This could be difficult given that Google applied a lot of pressure to CyanogenMod to stop them from doing it [6]. Even when a system with a large display (particularly a tablet) runs a version of Android that supports multi-window mode that still won t entirely solve the problem. No matter how big your display is there are occasions when you need to use it all for a single application while still having the ability to rapidly switch to another application. User interface tweaks to allow rapid task switching without losing application context are necessary. Finally for the past two years that I ve been using Android devices I have been disappointed in the ways that they compare poorly to the iPaQ running Familiar I used 10 years ago. I once wrote a feature article for a magazine on an iPaQ while so far I haven t even written a blog post on an Android device. I think that some of the earlier Android devices might have been better in some ways, the trackball on the HTC Nexus One might have made it more suitable for writing long articles than more recent Android devices.

Russ Allbery: afs-monitor 2.4

I love free software. Two more patches from users of my AFS monitoring scripts (primarily for Nagios) on the heels of the last release, so have another release. This one adds another ignore line to check_afs_bos for the output when there are long-running scheduled jobs, and adds an optional regex filter to check_afs_quotas to select the volumes whose quota one wants to check. Thanks to Georg Sluyterman and Christian Ospelkaus for the patches. You can get the latest version from the afs-monitor distribution page. As before, Debian packages have been uploaded to my personal repository. Also, a quick note about a previous journal entry: Philip Martin and Bernhard R. Link pointed me at the relevant part of the gpg source, which will only check signatures if all signatures use the same class and digest. I updated my previous entry. Thank you!

12 January 2013

Sylvestre Ledru: Some more cool stuff with LLVM/Clang

One of the drawback of C is the hard memory management. It is because of this aspect that Clang provides a way to detect, at runtime, memory errors. Called Address Sanitizer, it allows, while the program is running, to keep track of the memory and detect typical errors (out-of-bound accesses, use of a variable after a free, etc).
# With the following packages (version 3.2-1~exp3)
$ sudo apt-get install clang clang-3.2 compiler-rt -t experimental
Taking the simple following example, even if the errors are obvious, compilers will accept this code.
#include <stdlib.h>
int main()
char *x = (char*)malloc(10 * sizeof(char*));
free(x);
return x[5];
Built and run with:
$ clang -O1 -g -fsanitize=address -fno-omit-frame-pointer foo.c -o foo
$ ./foo &> memoryDebug.log
The previous command will generate a log file. Log which can be post processed with the asan_symbolize command.
$ asan_symbolize memoryDebug.log
which will give:
=================================================================
==21368== ERROR: AddressSanitizer: heap-use-after-free on address 0x7fb22e547f45 at pc 0x408c44 bp 0x7ffff60c10b0 sp 0x7ffff60c10a8
READ of size 1 at 0x7fb22e547f45 thread T0
    #0 0x408c43 in main /tmp/foo.c:5
    #1 0x7fb22d99f6ac in __libc_start_main /home/adconrad/eglibc-2.16/csu/libc-start.c:227
0x7fb22e547f45 is located 5 bytes inside of 80-byte region [0x7fb22e547f40,0x7fb22e547f90)
freed by thread T0 here:
    #0 0x408c90 in __interceptor_free ??:0
    #1 0x408c0a in main /tmp/foo.c:4
    #2 0x7fb22d99f6ac in __libc_start_main /home/adconrad/eglibc-2.16/csu/libc-start.c:227
previously allocated by thread T0 here:
    #0 0x408d50 in __interceptor_malloc ??:0
    #1 0x408bff in main /tmp/foo.c:3
    #2 0x7fb22d99f6ac in __libc_start_main /home/adconrad/eglibc-2.16/csu/libc-start.c:227
Shadow byte and word:
  0x1ff645ca8fe8: fd
  0x1ff645ca8fe8: fd fd fd fd fd fd fd fd
More shadow bytes:
  0x1ff645ca8fc8: fa fa fa fa fa fa fa fa
  0x1ff645ca8fd0: fa fa fa fa fa fa fa fa
  0x1ff645ca8fd8: fa fa fa fa fa fa fa fa
  0x1ff645ca8fe0: fa fa fa fa fa fa fa fa
=>0x1ff645ca8fe8: fd fd fd fd fd fd fd fd
  0x1ff645ca8ff0: fd fd fd fd fd fd fd fd
  0x1ff645ca8ff8: fa fa fa fa fa fa fa fa
  0x1ff645ca9000: fa fa fa fa fa fa fa fa
  0x1ff645ca9008: fa fa fa fa fa fa fa fa
Stats: 0M malloced (0M for red zones) by 1 calls
Stats: 0M realloced by 0 calls
Stats: 0M freed by 1 calls
Stats: 0M really freed by 0 calls
Stats: 0M (128 full pages) mmaped in 1 calls
  mmaps   by size class: 8:2047;
  mallocs by size class: 8:1;
  frees   by size class: 8:1;
  rfrees  by size class:
Stats: malloc large: 0 small slow: 1
==21368== ABORTING
The main advantage compare to valgrind is that asan is supposed to be way more faster. Threads can be also tricking to develop.
With the following example from stolen from upstream:
#include <pthread.h>
int Global;
void *Thread1(void *x)
Global = 42;
return x;
int main()
pthread_t t;
pthread_create(&t, NULL, Thread1, NULL);
Global = 43;
pthread_join(t, NULL);
return Global;
$ clang -fsanitize=thread -g -O1 foo2.c -fPIE -pie -o foo
$ ./foo
==================
WARNING: ThreadSanitizer: data race (pid=21416)
  Write of size 4 at 0x7f2f1a214a50 by thread 1:
    #0 Thread1 /tmp/foo2.c:4 (exe+0x00000000f850)
  Previous write of size 4 at 0x7f2f1a214a50 by main thread:
    #0 main /tmp/foo2.c:10 (exe+0x00000000f8a4)
  Thread 1 (tid=21417, running) created at:
    #0 pthread_create ??:0 (exe+0x00000001267e)
    #1 main /tmp/foo2.c:9 (exe+0x00000000f894)
==================
ThreadSanitizer: reported 1 warnings
Note that clang also provides scan-build, a static analyzer for memory issues. Not as powerful as the Address Sanitizer (it only works on a file), it provides some excellent reports. See the Wouter's blog post on this subject or the automatic report of scan-build on Scilab.

Russ Allbery: Review: Asimov's, April/May 2011

Review: Asimov's Science Fiction, April/May 2011
Editor: Sheila Williams
Issue: Volume 35, No. 4 & 5
ISSN: 1065-2698
Pages: 192
Williams's editorial this issue is about the tendency of SF to take a rose-colored view of the world, which on the surface seems odd given the tendency of recent SF towards dystopia. But she makes a good point that the portrayal of the past is rose-colored, linking that into the current steampunk trend. She doesn't take the argument quite as far as I'd like, but I'm glad to see editorials raising points like this. I'm inclined to think that a lot of the rose-colored frame of the past is because few of us want to read about real historic conditions at any length, even for edification, because the stench and discomfort isn't fun to read about. Silverberg's column is another discussion of programmatic plot generators, which mostly makes the point that plot ideas are the easy part of writing. James Gunn contributes an extended biography of Isaac Asimov that probably won't be new to long-time genre readers but may fill in some details (although it politely sticks to mostly flattering material). Spinrad's book review column is one of his better ones; it looks at two novels by China Mi ville and two by Ian McDonald and explores differences in world-building. Spinrad predictably makes the case in favor of science fiction with rules and against the New Weird, but the discussion along the way was worth reading. "The Day the Wires Came Down" by Alexander Jablokov: Speaking of steampunk, here's an example. There is even an airship, although the primary technological focus is suspended street cars. Jablokov postulates a city-wide transportation network of suspended carriages called telpher cars, along with a city built around the telpher cables: stores on roofs, windows displaying merchandise to passing cars, and even a history of heated competition and dirty tricks between competing telpher networks. The story is set, as the title would indicate, on the last day of the network. It's being shut down for cost, with some hints that progress is destroying something precious. There is a plot here, revolving around some mysteries of the history of the telpher network and the roles of several people in that history. But the story is primarily a celebration of old technology. It's a rail fan's story recast with a steampunk technology, featuring the same mix of fascination with mechanics and a sense that the intricate details are falling out of common knowledge (and perhaps usefulness). As a story, it's a bit slow-moving, but I enjoyed the elegiac tone. (7) "An Empty House with Many Doors" by Michael Swanwick: This is a very short story, more of an emotional profile, involving a man's reaction to the death of his wife. Oh, and parallel universes. It's sort of the inverse of Niven's classic "All the Myriad Ways." Similar to Niven's story, I found the idea vaguely interesting but the conclusion and emotional reaction unbelievable and alien. (5) "The Homecoming" by Mike Resnick: Resnick tends to yank on the heart-strings rather sharply in his stories, so I knew roughly what to expect when a father comes home to find his son is visiting. A son who, rather against his father's wishes, has been significantly altered to be able to live with aliens. Throw in a mother with serious dementia, and you can probably predict what Resnick does with this. Still, most of the story is a two-sided conversation, and I thought he succeeded in doing justice to both sides, even though one of them was destined to lose. (6) "North Shore Friday" by Nick Mamatas: Illegal Greek immigrants, a family-run system for getting them married off before the INS catch them, government psi probes and eavesdropping on thoughts, joint projects between computer and religion departments, secret government experiments, and even ghosts... this story is a complex mess, with numerous thoughts stuck into small boxes and scattered through the surface story. It's one of those stories where figuring out what's going on, and even how to read the story in a sensible way, is much of the fun. If you find that fun, that is; if not, it will probably be frustrating. I wished there was a bit more plot, but there's something delightful about how much stuff Mamatas packs into it. (6) "Clockworks" by William Preston: This is a prequel to Preston's earlier "Helping Them Take the Old Man Down". Like that story, it's primarily a pulp adventure, but layered with another level of analysis and thoughtfulness that tries to embed the pulp adventure in our understanding of human behavior and the nature of the world, although this one stays a bit more pulp than its predecessor. As with Preston's other story, we don't get directly in the head of the Old Man (here, just called the man, but identifiable from clues in both stories as Doc Savage); instead, the protagonist is a former villain named Simon Lukic who the man hopes to have fixed by operating on his brain. The undercurrent that lies beneath a more typical pulp adventure is the question of whether Lukic is actually healed. I think there was a bit too much daring-do and human perfection, but it's a perfectly servicable pulp story with some depth. (6) "The Fnoor Hen" by Rudy Rucker: If you've read any of Rucker's work before, you probably know what to expect: a mind-boggling blizzard of mathematically-inspired technobabble that turns into vaguely coherent surrealism. (You can probably tell that I'm not much of a fan, although the clear good humor in these stories makes it hard to dislike them too much.) There's a mutated chicken and some sort of alternate mathematical space and then something that seems like magic... I'd be lying if I said that I followed this story. If you like Rucker, this seems like the sort of thing that you'd like. (4) "Smoke City" by Christopher Barzak: At the start of this story, I thought it was going to be an emotional parable about immigration. The progatonist lives two lives: one in our world, and one in the Smoke City of industry, a world of hard labor, pollution, and little reward, with families in both. But nearly all of the story is set within Smoke City, and the parable turns out to be a caustic indictment of industry and its exploitation of labor. I kind of wish Barzak hadn't used rape as a metaphor, but when the captains of industry show up, I can't argue with how deeply and accurately the story shoves in the knife. There isn't much subtlety here, but it's still one of the better stories in this issue. (7) "A Response from EST17" by Tom Purdom: I'm very happy to see Purdom's writing appearing regularly. His stories are always quiet and matter-of-fact, and at first seem to miss emotional zest, but they almost always grow on me. He lets the reader fill in their own emotional reactions to events, and there's always a lot going on. This story is a first-contact story, except that the "humans" here are not human at all. They're automated probes sent by two separate human civilizations, with different programming and different governance algorithms, and they quickly start competing negotiations. The aliens they've discovered similarly have factions, who start talking to the different probes in an elaborate dance of gathering information without giving too much away. The twist is that this pattern has replayed itself many times in the past, and information itself can be a weapon. I enjoyed this one from start to finish. (7) "The One That Got Away" by Esther M. Friesner: Friesner is best known, at least to me, for humorous fantasy, and this story is advertised as such from early on. The first-person protagonist is a prostitute in a seaside town. She's bemused to finally be invited over by a sailor who's been eyeing her all evening, but that sailor has something else in mind than normal business. For much of this story, the fantasy element is unclear; when it finally comes, it was an amusing twist. (7) "The Flow and Dream" by Jack Skillingstead: This is a mildly interesting variation on the old SF story of hibernating humans (on a generation ship or elsewhere) waking up to a transformed world. Here, it's not a ship, it's a planet, and the hiberation was to wait for terraforming rather than for transit. The twist comes from an excessively literal computer and the fun of putting together the pieces. Sadly, the story trails off at the end without much new to say. (5) "Becalmed" by Kristine Kathryn Rusch: "Becalmed" takes place immediately before "Becoming One with the Ghosts" and explains the incident that created the situation explored in that story. The first-person protagonist of "Becalmed" is a linguist, an expert in learning alien languages so that the Fleet can understand the civilizations that it runs across. But something went horribly wrong at their last stop, something that she's largely suppressed, and now she's confined to quarters and possibly in deep trouble. As is the ship; they're in foldspace, and they have been for days. "Becalmed" is structed like a mystery, centered around recovering the protagonist's memories. It's also a bit of a legal procedural; the ship is trying to determine what to do with her and to what degree she's responsible. But the heart of the story is a linguistic and cultural puzzle. This is another great SF story from Rusch, whose name on a cover will make me eager to start reading a new magazine. I love both angles on the universe she's built, but I think I like the Fleet even better than the divers. The Fleet captures some of the magic of the original Star Trek, but with much more mature characters, more believable situations, and a more sensible and nuanced version of the Prime Directive. Rusch writes substantial, interesting plots that hold my interest. I'd love to see more like this. (8) Rating: 7 out of 10

10 January 2013

Debian Med: Bits from Debian Med team (Posted by Andreas Tille)

This are the new years bits from Debian Med team. We try to make the wishes for a healthy new year becoming true with Free Software. :-)
Debian Med Bug Squashing Advent Calendar 2012 As last year the Debian Med team did some advent calendar bug squashing. The summary states 15 bugs from Debian Med scope which is less than last year but Debian Med team members also did some general bug squashing to help the Wheezy release. Finally this makes a lot of sense also for Debian Med if we can speed up the release. When counting the additional 12 RC bugs fixed in the advent calendar effort we again managed to fix more than one bug per day. Please keep on squashing RC bugs for Wheezy!
Mentoring of Month Last year Andreas Tille started the Mentoring of Month effort to lower the entrance barrier for newcomers. There was no student for every month and not each package was uploaded at end of month but expecting such a high number of newcomers would be unrealistic anyway. So I'd like to draw some positive conclusion that we now have some newcomers friendly invited to the project and some packages uploaded or at least prepared which would not be there without the MoM project. Finally all MoM students liked the project and we will continue it in the future and would like to recommend other teams to do something similar.
Planed Debian Med sprint in Kiel The next Debian Med sprint for real life meeting of Debian Med developers and users will happen in Kiel 23rd/24th of February. All interested people are invited to join to continue the success of the past two sprints. We are currently in the planing phase but there are just some hot topics which are workflows, licensing (specifically bug #694908), cooperation with Ubuntu and BioLinux and other things.
Metapackages targeting at Wheezy Currently Debian Med metapackages version 1.13.2 are in unstable and once the unblock request is accepted by Debian release team this version will target at Wheezy. This means your very last chance to influence the metapackage content regarding some missing dependencies is defacto over now. Please closely observe our web sentinel in case you are interested in the work of Debian Med and tell us if packages are missing from our focus.

Kind regards and see you at Debian Med sprint
Andreas.

9 January 2013

Daniel Kahn Gillmor: universally accessible storage for the wary user

A friend wrote me a simple question today. My response turned out to be longer than i expected, but i hope it's useful (and maybe other people will have better suggestions) so i thought i'd share it here too: Angela Starita wrote:
I'd like to save my work in a location where I can access it from any computer. I'm wary of using the mechanisms provided by Google and Apple. Can you suggest another service?
Here's my reply: I think you're right to be wary of the big cloud providers, who have a tendency to inspect your data to profile you, to participate in arbitrary surveillance regimes, and to try to sell your eyeballs to advertisers. Caveat: You have to trust the client machine too But it's also worth remembering that the network service provider is not the only source of risk. If you really mean "accessing your data from any computer", that means the computer you're using to access the data can do whatever it wants with it. That is, you need to trust both the operator of these "cloud" services, *and* the administrator/operating system of the client computer you're using to access your data. For example, if you log into any "secure" account from a terminal in a web caf , that leaves you vulnerable to the admins of the web caf (and, in the rather-common case of sloppily-administered web terminals, vulnerable to the previous user(s) of the terminal as well). Option 0: Portable physical storage One way to have your data so that you can access it from "any computer" is to not rely on the network at all, but rather to carry a high-capacity MicroSD card (and USB adapter) around with you (you'll probably want to format the card with a widely-understood filesystem like FAT32 instead of NTFS or HFS+ or ext4, which are only understood by some of the major operating systems, but not all). Here is some example hardware: Almost every computer these days has either a microSD slot or a USB port, while some computers are not connected to the network. This also means that you don't have to rely on someone else to manage servers that keep your data available all the time. Note that going the microSD route doesn't remove the caveat about needing to trust the client workstation you're using, and it has another consideration: You'd be responsible for your own backup in the case of hardware failure. You're responsible for your own backup in the case of online storage too, of course -- but the better online companies are probably better equipped than most of us to deal with hardware failure. OTOH, they're also susceptible to some data loss scenarios that we aren't as individual humans (e.g. the company might go bankrupt, or get bought by a competitor who wants to terminate the service, or have a malicious employee who decides to take revenge). Backup of a MicroSD card isn't particularly hard, though: just get a USB stick that's the same size, and regularly duplicate the contents of the MicroSD card to the USB stick. One last consideration is storage size -- MicroSD cards are currently limited to 32GB or 64GB. If you have significantly more data than that, this approach might not be possible, or you might need to switch to a USB hard disk, which would limit your ability to use the data on computers that don't have a USB port (such as some smartphones). Option 1: Proprietary service providers If you don't think this portable physical storage option is the right choice for you, here are a couple proprietary service providers who offer some flavor of "cloud" storage while claiming to not look at the contents of your data: I'm not particularly happy with either of those, though, in part because the local client software they want you to run is proprietary, so there's no way to verify that they are actually unable to access the contents of your data. But i'd be a lot happier with either wuala or spideroak than i would be with google drive, dropbox, or iCloud. Option 2: What i really want I'm much more excited about the network-accessible, free-software, privacy-sensitive network-based storage tool known as git-annex assistant. The project is spearheaded by Joey Hess, who is one of the most skilled and thoughtful software developers i know of. "assistant" (and git-annex, from which it derives) has the advantage of being pretty agnostic about the backend service (many plugins for many different cloud providers) and allows you to encrypt your data locally before sending it to the remote provider. This also means you can put your encrypted data in more than one provider, so that if one of the providers fails for some reason, you can be relatively sure that you have another copy available. But "assistant" won't be ready for Windows or Android for several months (builds are available for Linux and Mac OS X now), so i don't know if it meets the criterion for "accessible from any computer". And, of course, even with the encryption capabilities, the old caveat about needing to trust the local client machine still applies.

8 January 2013

Russell Coker: The Death of the Netbook

The Age has an interesting article about how Apple supposedly killed the Netbook [1]. It s one of many articles with a similar spin on the news that the last two companies making Netbooks are going to cease production. The main point of these articles is that Apple decided that Netbooks were crap and killed the market for them by producing tablets and light laptops that squeeze them out of the market. Is the Macbook Air a Netbook? According to the Wikipedia page the Macbook Air [2] weighs 1080g for the 11 version and 1340g for the 13 version. According to Wikipedia the EeePC 701 (the first EeePC) weighs 922g and the last EeePC weighs 1460g [3]. The last EeePC produced is heavier than ANY Macbook Air while the first (and lightest) EeePC is only 158g lighter than the 11 Macbook Air. The 11 Macbook Air is 300*192*17mm (979cm^3) in size while the EeePC 701 is 225*165*35mm (1299cm^3) and the biggest EeePC was 266*191*38mm (1931cm^3). So the 11 Macbook Air is 13% wider than the widest EeePC but takes less volume than any EeePC. The 13 Macbook Air is 325*227*17mm (1254cm^3) which is still less volume than any EeePC. The Wikipedia page about Netbooks defines them as being small, lightweight, legacy-free (in terms of hardware not software) and cheap [4]. The Macbook Air clearly meets all the criteria apart from price. The Apple US web site offers the version of the 11 Macbook Air with 64G of storage for $999 with free shipping, for comparison the EeePC 701 was on sale in stores for $500 in 2008. The CPI adjusted price for the EeePC 701 would be at least $550 in today s money. The Macbook is a bit less than twice as expensive as the EeePC was, but that s more of an issue of Apple being expensive a few years ago companies like HP were also selling Netbooks that were more expensive than the EeePC. Unless having an awful keyboard is a criteria for being a Netbook I think that the Macbook Air meets the criteria. As an aside, a relative recently asked me for advice on a device that is like a Macbook Air but cheaper. Does anyone know of a good option? Is Netbook Production Ceasing? Officeworks currently sells an ASUS Notebook that has a 11.6 display and weighs 1.3kg for $398, it s got a metal body that looks a bit like a Macbook Air (which is the latest fashion and is good for heat dissipation). That s not advertised as a Netbook or a Eee product but it s cheap, lighter than the heaviest EeePC, and not much bigger than an EeePC. It seems that the general prices of laptops other than Apple products (which have always had higher prices) have been dropping a lot recently. There are lots of good options if you want a laptop that costs $500 or less. Even Thinkpads (one of the most expensive and best designed ranges of laptops) are well below $1000. Do the Articles about Netbooks Make Sense? The claims being made are that Apple skipped Netbooks because they couldn t make a good profit. This disregards the fact that the iPhone and iPad (which are very profitable) are in the high end of the price range that was occupied by Netbooks. While Apple does make a good deal of money from the iPhone App Market it would be possible to make a Netbook with a lower production price than an iPhone because making things smaller requires more engineering work and often more expensive parts. This also disregards the fact that there are a range devices which work as an iPad case with keyboard, an iPad with such a keyboard meets most criteria for being a Netbook, so Apple is one iPad keyboard device away from selling Netbooks. It s interesting to note that I haven t yet seen an article about the profits from Netbooks which didn t make an issue of the MS-Windows license fees. The first Netbooks only ran Linux but later on they switched to Windows, that had to make a big impact on profits. An article about Netbooks which just assumes that everyone has to pay a MS license fee is missing too much of the Netbook history to be useful. I wonder if anyone could make products that are as profitable as the iPhone and Macbook Air if they had to pay for MS license fees and design their hardware to work with MS software (as opposed to Apple who can change their software to allow a cheaper hardware design). The articles also claim that Netbooks give a bad user experience. When I bought my EeePC 701 it was the fastest system I owned for loading OpenOffice, SSD random read speeds were really good (writes sucked but that didn t matter so much). The keyboard on an EeePC 701 is not nearly as good as a full size laptop but it is also a lot better than using a tablet, I ve used both a 10 Android tablet and an EeePC as a ssh client and there is no comparison. When I m going somewhere that requires random sysadmin work (or other serious typing) and I can t carry much weight then I still take my EeePC 701 and I don t consider taking a tablet. The low resolution of the screen is a major issue, but it s about the same as a Macbook Air so that s not an advantage for Apple. I knew some people who used an EeePC 701 for the majority of their work, I couldn t do that but obviously some people have different requirements. I now use my phone for many tasks that I used to do on my EeePC (even light sysadmin work) so my EeePC sometimes goes unused for months. But it s still an important part of my collection of computers. It works well for what it does and I don t feel any need to buy a replacement. When it wears out I ll probably buy something similar to an 11 Macbook Air to replace it unless there s a good option of a tablet with a detachable keyboard. My plans for computer ownership for the near future are based on a reasonably large Android phone (currently a Samsung Galaxy S3 but maybe a Galaxy Note 2 or similar next year), a small laptop or large tablet with hardware keyboard (currently an EeePC 701), a large laptop (currently a Thinkpad T61), and a workstation (currently a NEC system with an Intel E4600 CPU and a Dell U2711 27 monitor). A reasonably small and light system with a hardware keyboard and solid state storage is an important part of my computer needs. If tablet computers with hardware keyboards replace traditional Netbooks that s not really killing Netbooks but introducing a new version of the same thing. But a good way of getting web hits on an article is to claim that a once popular product is dead.

7 January 2013

Rapha&#235;l Hertzog: My Free Software Activities in December 2012

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (836.78 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Debian Packaging I uploaded Zim 0.58, WordPress 3.5 (and I had to file a ticket about the availability of sources of minified javascript files again) and another security update for python-django (696535). Speaking of python-django, I forwarded one bug report of Anders Kaseorg concerning Django s bash completion (#695811). I also sponsored the upload of ledgersmb 1.3.25-1. Other Debian work I contributed some patches to improve debian-installer support in live-build. I also added a work-around for bug #652946 in live-installer and committed a fix for this same bug in a jessie branch of partman-target. I also prepared a bugfix for a counter-productive behavior of choose-mirror (#695261, it was not possible to override the codename of the release to install via preseed if you install from a CD with a full base system). I discovered an oddity in the Packages.diff/Index file for the architectures armhf and s390x, I reported it to ftpmasters in #696792. (All those issues were discovered while working for a customer) Debian France I spent quite some time on Debian France this month again. I started by setting up an internal gitolite to manage our accounting/administrative documents. Then I updated galette, the web application that we are using to manage our database of members. In the process, I filed two bugs that we discovered. I immediately tested Galette by registering 4 members that joined during the former mini-Debconf in Paris. We have plans to automate the membership renewal process so we have opened a Paypal account. This month we also cleared the last steps so that I and Sylvestre Ledru have full control on the Debian France bank account. I also registered the new officers at the Tribunal d instance de Sarreguemines . Salt bug reports During the last mini-debconf, I discovered Salt (thanks to Julien Cristau!) and since I had to switch some servers of mine, I took this opportunity to upgrade to wheezy and try out salt at the same time. It took much more time than expected but the result is pleasant. The configuration of all my servers is now well documented/specified in a central Git repository, and moving services is much easier than before. In the process, I filed quite some bugs (#2865, #2851, #2866 and #2875), most of them have been fixed in the 0.11.1 release that just happened. Thanks I wish you a happy new year and all the best for 2013! See you next month for a new summary of my activities.

4 comments Liked this article? Click here. My blog is Flattr-enabled.

6 January 2013

Neil Williams: bootstrapping arm64

I'm still working on perl-cross-debian (just uploaded 0.0.2) and there's more to do on that with upstream but part of the reason to work on perl cross-building is to do what I can to help with the ARM64 port.

So, I went back to Wookey's talk at DebConf12 which leads to the current list of cross-build results for arm64 and started through the list.

coreutils is listed as failing but that was an archive error (MD5sum hash mismatch), so that just needs a retry. I don't have access to that buildd, yet, so nothing I can do there.

Next on the list (excluding those just waiting for build-deps) was klibc.

aarch64-linux-gnu-gcc -Wp,-MD,usr/klibc/.__static_init.o.d -nostdinc -iwithprefix
include -I/ PKGBUILDDIR /usr/include/arch/x86_64 -Iusr/include/arch/x86_64
-I/ PKGBUILDDIR /usr/include/bits64
-Iusr/include/bits64 -I/ PKGBUILDDIR /usr/klibc/../include
-Iusr/klibc/../include -I/ PKGBUILDDIR /usr/include -Iusr/include
-I/ PKGBUILDDIR /linux/include -Ilinux/include
-I/ PKGBUILDDIR /linux/arch/x86/include -Ilinux/arch/x86/include
-D__KLIBC__=2 -D__KLIBC_MINOR__=0 -D_BITSIZE=64 -fno-stack-protector -fwrapv -m64 -Os
-fno-asynchronous-unwind-tables -fomit-frame-pointer -falign-functions=1 -falign-jumps=1 -falign-loops=1
-W -Wall -Wno-sign-compare -Wno-unused-parameter -c -o usr/klibc/__static_init.o usr/klibc/__static_init.c
aarch64-linux-gnu-gcc: error: unrecognized command line option '-m64'
make[4]: *** [usr/klibc/__static_init.o] Error 1


Turns out that this is a build failure I understood, at least initially. A little digging and a trivial patch was begun:

--- a/usr/klibc/arch/arm/MCONFIG
+++ b/usr/klibc/arch/arm/MCONFIG
@@ -30,6 +30,13 @@
ifeq ($(CONFIG_AEABI),y)
KLIBCREQFLAGS += -mabi=aapcs-linux -mno-thumb-interwork
else
+# aarch64
+ifeq ($(CONFIG_AARCH64),y)
+KLIBCREQFLAGS +=
+KLIBCOPTFLAGS += -mgeneral-regs-only
+else
KLIBCREQFLAGS += -mabi=apcs-gnu -mno-thumb-interwork
endif
endif
+endif
+

Alongside a trivial change to debian/rules

ifeq ($(DEB_HOST_ARCH),arm64)
DEB_MAKE_ENVVARS := ARCH=arm CONFIG_AARCH64=y CPU_ARCH=armv8-a CPU_TUNE=generic
endif

OK, then things get a bit more awkward,

....
code/debian/cross/klibc/klibc-2.0.1/linux/arch/arm/include -Ilinux/arch/arm/include -D__KLIBC__=2
-D__KLIBC_MINOR__=0 -D_BITSIZE=32 -fno-stack-protector -fwrapv -fno-exceptions -Os
-march=armv8-a -mtune=generic -mgeneral-regs-only -W -Wall -Wno-sign-compare -Wno-unused-parameter
-c -o ../foo.o usr/klibc/arch/arm/crt0.S
usr/klibc/arch/arm/crt0.S: Assembler messages:
usr/klibc/arch/arm/crt0.S:19: Error: operand 1 should be an integer register -- mov r0,sp'
usr/klibc/arch/arm/crt0.S:20: Error: operand 1 should be an integer register -- mov r1,#0'

Hmm. Assembly, well, yes, I've done assembly before, I know what mov should normally do, sp is likely to be the stack pointer .... where's my AARCH64 assembly PDF again... PRD03-GENC-010197 ...

OK, so maybe the r0 and r1 should be x0 and x1, hmm, that at least doesn't raise assembly errors. So a tentative change:

--- a/usr/klibc/arch/arm/crt0.S
+++ b/usr/klibc/arch/arm/crt0.S
@@ -15,9 +15,13 @@
#ifdef __thumb__
.thumb_func
#endif
-
+#ifdef __aarch64__
+_start: mov x0, sp
+ mov x1, #0
+ bl __libc_init
+#else
_start: mov r0, sp
mov r1, #0
bl __libc_init
-
+#endif
.size _start,.-_start

Next stage, however, leaves me quite a bit more lost:

....
klibc-2.0.1/linux/arch/arm/include -Ilinux/arch/arm/include
-D__KLIBC__=2 -D__KLIBC_MINOR__=0 -D_BITSIZE=32 -fno-stack-protector -fwrapv -fno-exceptions -Os
-march=armv8-a -mtune=generic -mgeneral-regs-only -W -Wall -Wno-sign-compare -Wno-unused-parameter
-c -o usr/klibc/arch/arm/setjmp.o usr/klibc/arch/arm/setjmp.S
usr/klibc/arch/arm/setjmp.S: Assembler messages:
usr/klibc/arch/arm/setjmp.S:32: Error: unknown mnemonic stmia' -- stmia r0, r4,r5,r6,r7,r8,r9,r10,fp,sp,lr '
usr/klibc/arch/arm/setjmp.S:33: Error: operand 1 should be an integer register -- mov r0,#0'
usr/klibc/arch/arm/setjmp.S:34: Error: operand 1 should be an integer register -- mov pc,lr'
usr/klibc/arch/arm/setjmp.S:42: Error: unknown mnemonic ldmia' -- ldmia r0, r4,r5,r6,r7,r8,r9,r10,fp,sp,lr '
usr/klibc/arch/arm/setjmp.S:43: Error: unknown mnemonic movs' -- movs r0,r1'
usr/klibc/arch/arm/setjmp.S:44: Error: unknown mnemonic moveq' -- moveq r0,#1'
usr/klibc/arch/arm/setjmp.S:45: Error: operand 1 should be an integer register -- mov pc,lr'
make[5]: *** [usr/klibc/arch/arm/setjmp.o] Error 1


So now I'm out of my depth of AARCH64 assembly (apart from the recurrence of mov r0 vs mov x0 etc.). If the above is useful then maybe someone can work out what is wrong with setjmp.S or whether AARCH64 just means that klibc needs to gain a arch/arm64/ directory and not try to duplicate each entire assembly block within #ifdef clauses.

I don't really know where else to put an incomplete investigation like this, so it's here for anyone to find.

(Oh, and if you're reading those arm64 cross-build logs, then a few hundred occurrences of ldconfig: /usr/lib/aarch64-linux-gnu/*.so is for unknown machine 183. in every build log (success or fail) is apparently entirely normal until more packages get fixed. Really slows down scanning the build log. :-(

I may try busybox or libusb next. libusb looks like a classic "you might have told me to cross-compile but I'm going to use g++ anyway because I know best" cross-building problem, indicative of yet another BDBS. sigh.

Resources:
Getting started with 64-bit ARM development
ARMv8 images for developers
AArch64 for everyone, Marcin Juszkiewicz
Howto/HelloAarch64 - Linaro wiki
AArch64 gcc options

5 January 2013

Ian Wienand: Position Independent Code and x86-64 libraries

A comment pointed out that the original article from 2008 made a few simplifications that were a bit misleading, so I have taken some time to update this. Thanks for the feedback. If you've ever tried to link non-position independent code into a shared library on x86-64, you should have seen a fairly cryptic error about invalid relocations and missing symbols. Hopefully this will clear it up a little. Let's start with a small program to illustrate.
int global = 100;
int function(int i)  
    return i + global;
 
Firstly, inspect the disassembley of this function:
$gcc -c function.c
$objdump --disassemble function.o
0000000000000000 <function>:
   0:   55                      push   %rbp
   1:   48 89 e5                mov    %rsp,%rbp
   4:   89 7d fc                mov    %edi,-0x4(%rbp)
   7:   8b 05 00 00 00 00       mov    0x0(%rip),%eax        # d <function+0xd>
   d:   03 45 fc                add    -0x4(%rbp),%eax
  10:   c9                      leaveq
  11:   c3                      retq
Lets just go through that for clarity: The IP relative move is really the trick here. We know from the code that it has to move the value of the global variable here. The zero value is simply a place holder - the compiler currently does not determine the required address (i.e. how far away from the instruction pointer the memory holding the global variable is). It leaves behind a relocation -- a note that says to the linker "you should determine the correct address of foo (global in our case), and then patch this bit of the code to point to that addresss". Relocations with addend The top portion of the image above gives some idea of how it works. We can examine relocations in binaries with the readelf tool.
$ readelf --relocs ./function.o
Relocation section '.rela.text' at offset 0x518 contains 1 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000000009  000800000002 R_X86_64_PC32     0000000000000000 global + fffffffffffffffc
There are many different types of relocations for different situations; the exact rules for different relocation types are described in the ABI documentation for the architecture. The R_X86_64_PC32 relocation is defined as "the base of the section the symbol is within, plus the symbol value, plus the addend". The addend makes it look more tricky than it is; remember that when an instruction is executing the instruction pointer points to the next instruction to be executed. Therefore, to correctly find the data relative to the instruction pointer, we need to subtract the extra. This can be seen more clearly when layed out in a linear fashion (as in the bottom of the above diagram). If you try and build a shared object (dynamic library) with an object file with this type of relocation, you should get something like:
$ gcc -shared function.c
/usr/bin/ld: /tmp/ccQ2ttcT.o: relocation R_X86_64_32 against  a local symbol' can not be used when making a shared object; recompile with -fPIC
/tmp/ccQ2ttcT.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
If you look back at the disassembly, we notice that the R_X86_64_32 relocation has left only 4-bytes (32-bits) of space left for the relocation entry (the zeros in 7: 8b 15 00 00 00 00). So why does this matter when you're creating a shared library? The first thing to remember is that in a shared library situation, we can not depend on the local value of global actually being the one we want. Consider the following example, where we override the value of global with a LD_PRELOAD library.
$ cat function.c
int global = 100;
int function(int i)  
    return i + global;
 
$ gcc -fPIC -shared -o libfunction.so function.c
$ cat preload.c
int global = 200;
$ gcc -shared preload.c -o libpreload.so
$ cat program.c
#include <stdio.h>
int function(int i);
int main(void)  
   printf("%d\n", function(10));
 
$ gcc -L. -lfunction program.c -o program
$ LD_LIBRARY_PATH=. ./program
110
$ LD_PRELOAD=libpreload.so LD_LIBRARY_PATH=. ./program
210
If the code in libfunction.so were to have a fixed offset into its own data section, it will not be able to be overridden at run-time by the value provided by libpreload.so. Additionally, there are only 4-bytes available to patch in for the address of global -- since a shared library could conceivably be loaded anywhere in the 64-bit (8-byte) address space we therefore need 8-bytes of space to cover ourselves for all possible addresses global might turn up at. The two basic possibilities for an object file are to be either linked into an executable or linked into a shared-library. In the executable case, the value of global will be in the exectuable's data section, which should definitely be reachable with a 32-bit offset of the current instruction-pointer. The instruction-pointer relative address can simply be patched in and the executable is finalised. But what about the shared-library case, where we know the value of global could essentially be anywhere within the 64-bit address space? It is possible to leave 8-bytes of space for the address of global, by telling gcc to use the large-code model. e.g.
$ gcc -c -mcmodel=large function.c
$ objdump --disassemble ./function.o
./function.o:     file format elf64-x86-64
Disassembly of section .text:
0000000000000000 <function>:
   0:  55                      push   %rbp
   1:  48 89 e5                mov    %rsp,%rbp
   4:  89 7d fc                mov    %edi,-0x4(%rbp)
   7:  48 b8 00 00 00 00 00    movabs $0x0,%rax
   e:  00 00 00
  11:  8b 10                   mov    (%rax),%edx
  13:  8b 45 fc                mov    -0x4(%rbp),%eax
  16:  01 d0                   add    %edx,%eax
  18:  5d                      pop    %rbp
  19:  c3                      retq
However, this creates a problem if you really want to share this code. By having to patch in an address of global directly, this means the run-time code above does not remain unchanged. Two separate processes therefore can't share this code -- they each need separate copies that are identical but for their own addresses of global patched into it. By enabling Position Independent Code (PIC, with the flag -fPIC) you can ensure the code remains share-able. PIC means that the output binary does not expect to be loaded at a particular base address, but is happy being put anywhere in memory (compare the output of readelf --segments on a binary such as /bin/ls to that of any shared library). This is obviously critical for implementing lazy-loading (i.e. only loaded when required) shared-libraries, where you may have many libraries loaded in essentially any order at any location. Of course, any problem in computer science can be solved with a layer of abstraction and that is essentially what is done when compiling with -fPIC. To examine this case, let's see what happens with PIC turned on.
$ gcc -fPIC -shared -c  function.c
$ objdump --disassemble ./function.o
./function.o:     file format elf64-x86-64
Disassembly of section .text:
0000000000000000 <function>:
   0:   55                      push   %rbp
   1:   48 89 e5                mov    %rsp,%rbp
   4:   89 7d fc                mov    %edi,-0x4(%rbp)
   7:   48 8b 05 00 00 00 00    mov    0x0(%rip),%rax        # e <function+0xe>
   e:   8b 00                   mov    (%rax),%eax
  10:   03 45 fc                add    -0x4(%rbp),%eax
  13:   c9                      leaveq
  14:   c3                      retq
It's almost the same! We setup the frame pointer with the first two instructions as before. We push the first argument into memory in the pre-allocated "red-zone" as before. Then, however, we do an IP relative load of an address into rax. Next we de-reference this into eax (e.g. eax = *rax in C) before adding the incoming argument to it and returning.
$ readelf --relocs ./function.o
Relocation section '.rela.text' at offset 0x550 contains 1 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
00000000000a  000800000009 R_X86_64_GOTPCREL 0000000000000000 global + fffffffffffffffc
The magic here is again in the relocations. Notice this time we have a P_X86_64_GOTPCREL relocation. This says "replace the data at offset 0xa with the global offset table (GOT) entry of global. Global Offset Table operation with data variables As shown above, the GOT ensures the abstraction required so symbols can be diverted as expected. Each entry is essentially a pointer to the real data (hence the extra dereference in the code above). Since we can assume the GOT is at a fixed offset from the program code within plus or minus 2Gib, the code can use a 32-bit IP relative address to gain access to the table entries. So, taking a look a the final shared-library binary we see a final offset hard-coded
$ gcc -shared -fPIC -o libfunction.so function.c
$ objdump --disassemble ./libfunction.so
00000000000006b0 <function>:
 6b0:  55                      push   %rbp
 6b1:  48 89 e5                mov    %rsp,%rbp
 6b4:  89 7d fc                mov    %edi,-0x4(%rbp)
 6b7:  48 8b 05 8a 02 20 00    mov    0x20028a(%rip),%rax        # 200948 <_DYNAMIC+0x1d8>
 6be:  8b 10                   mov    (%rax),%edx
 6c0:  8b 45 fc                mov    -0x4(%rbp),%eax
 6c3:  01 d0                   add    %edx,%eax
 6c5:  5d                      pop    %rbp
 6c6:  c3                      retq
 6c7:  90                      nop
Every process who wants to share this code just needs to make sure they have their unique address of global at 0x20028a(%rip). Since each process has a separate address-space, this means they can all have different values for global but share the same code! Thus the default of the small-code model is sensible. It is exceedingly rare for an executable to need more than 4-byte offsets for a relative access to a variable in it's data region, so using a full 8-byte value would just be a waste of space. Although leaving 8-bytes would allow access to the variable anywhere in the 64-bit address space; when building a shared library, you really want to use -fPIC to ensure the library can actually be shared, which introduces a different relocation and access to data via the GOT. This should explain why gcc -shared function.c works on x86-32, but does not work on x86-64. Inspecting the code reveals why:
$ objdump --disassemble ./function.o
00000000 <function>:
   0:   55                      push   %ebp
   1:   89 e5                   mov    %esp,%ebp
   3:   a1 00 00 00 00          mov    0x0,%eax
   8:   03 45 08                add    0x8(%ebp),%eax
   b:   5d                      pop    %ebp
   c:   c3                      ret
$ readelf --relocs ./function.o
Relocation section '.rel.text' at offset 0x2ec contains 1 entries:
 Offset     Info    Type            Sym.Value  Sym. Name
00000004  00000701 R_386_32          00000000   global
We start out the same, with the first two instructions setting up the frame pointer. However, next we load a memory value into eax -- as we can see from the relocation information, the address of global. Next we add the incoming argument from the stack (0x8(%ebp)) to the value in this memory location; implicitly dereferencing it. But since we only have a 32-bit address-space, the 4-bytes allocated is enough to access any possible address. So while this can work, you're not creating position-independent code and hence not enabling code-sharing. The disadvantage of PIC code is that you require "bouncing" through the GOT, which requires more loads and reads to find an address than directly referencing it. However, if your program is at the point that this is becoming a performance issue you're probably not reading this blog! Hopefully, this helps clear up that possibly cryptic error message. Further searches around position-independent code, global-offset tables and code-sharing should also yield you more information if it remains unclear.

Paul Tagliamonte: Updates to dput-ng since version 1.0

Big release notes since 1.0: We ve got a new list dput-ng-maint@lists.alioth.debian.org feel free to subscribe!
1.3:
  * Avoid failing on upload if a pre/post upload hook is missing from the
    Filesystem.
  * Fix "dcut raises FtpUploadException" by correctly initializing the uploader
    classes from dcut (Closes: #696467)
1.2:
  * Add bash completions for dput-ng (Closes: #695412).
  * Add in a script to set the default profile depending on the building
    distro (Ubuntu support)
  * Fix a bug where meta-class info won't be loaded if the config file has the
    same name.
  * Add an Ubuntu upload target.
  * Added .udeb detection to the check debs hook.
  * Catch the correct exception falling out of bin/dcut
  * Fix the dput manpages to use --uid rather then the old --dm flag.
  * Fix the CLI flag registration by setting required=True
    in cancel and upload.
  * Move make_delayed_upload above the logging call for sanity's sake.
  * Fix "connects to the host even with -s" (Closes: #695347)
Thanks to everone who s contributed!
     7  Bernhard R. Link
     4  Ansgar Burchardt
     3  Luca Falavigna
     2  Michael Gilbert
     2  Salvatore Bonaccorso
     1  Benjamin Drung
     1  Gergely Nagy
     1  Jakub Wilk
     1  Jimmy Kaplowitz
     1  Luke Faraone
     1  Sandro Tosi
This has been your every-once-in-a-while dput-ng update. We re looking for more code contributions (to make sure everyone s happy), doc updates (etc) or ideas.

4 January 2013

David Welton: The Software Millionaire Next Door

I've been reading "The Millionaire Next Door" and have so far found it to be a pleasant book with a good message: don't waste your money on silly things and appearance (fancy suits, fancy cars, expensive boats, etc...), save what you do earn consistently and constantly, invest wisely, and so on. Wikipedia has a good summary:
http://en.wikipedia.org/wiki/The_Millionaire_Next_Door
One of the things I like about it is that it focuses on "ordinary" wealthy people, those with a million or more in the bank, but not the Warren Buffets or Bill Gates types that are extreme statistical outliers. There are plenty of people in the US who have done well by themselves by slowly but surely putting together enough money to be financially independent, without, however, being in the spotlight. As the book says, these are the kind of people who maybe own a local chain of businesses doing something fairly ordinary, but doing it well enough to succeed. They may very well not live in a fancy house, nor drive an expensive car, or otherwise outwardly draw much attention to themselves. The world of software does not revolve around "dressing for success" (you noticed?), but we do tend to focus on the "big winners". Gates, Jobs, Zuckerberg, Larry & Sergey, Larry Ellison, and so on are the stars of the show. Of course, the economics of software being what they are, instances of winner-take-all markets with one big fish and a lot of also-rans are not uncommon. However, that is not the only story, and I think it'd be interesting to know more about those in our industry who have accumulated significant wealth, yet are not the guys with more money than they could possibly ever spend on things that aren't, say, country-sized chunks of real-estate. I'm guessing they'd fall into these categories: It'd be very interesting to gather some actual data on this, although I'm not in a position to do so myself - I wouldn't even really know where to start. As I age, I think the third category has begun to seem appealing in many ways - I'm simply not cut out for the Big Company life, and I'm not interested in living in Silicon Valley and going "all in" on the latest startup - I already did that, and while it was fun and I don't regret it, it's not the kind of thing I'd want to do now that I'm married and have kids. Incidentally, this more relaxed, under the radar approach is exactly what is expoused in one of my favorite books of the past few years, Start Small, Stay Small. Edit : I finished reading the book and reviewed it here: http://davids-book-reviews.blogspot.com/

Stefano Zacchiroli: bits from the DPL for December 2012

Happy new year, Debian! To celebrate, here are some freshly posted, bits from the DPL for December 2012.
Dear Project Members, happy new year! Here goes another report of DPL activities, this time for December 2012. This issue of the DPL-monthly is skinnier than usual: during the past month I've been struck by the catastrophe also known as "family holiday season", enjoying a solid 10 day break from computer-related activities. Talks Assets DPL helpers Collaboration with the outer world That's all for last year, enjoy the new one, which will soon see a new Debian release out of the door. And to make it happen sooner, let's go back fix RC bugs! Cheers.
PS the day-to-day activity log for December 2012 is available at the usual place master:/srv/leader/news/bits-from-the-DPL.txt.201212

Russell Coker: Links January 2013

AreWomenHuman has an interesting article about ViolentAcrez and the wide support for trolling (including by media corporations) [1]. Chrys Stevenson wrote an important article for the ABC about the fundamentalist Christians who are trying to take over the Australian education system [2]. Tavi Gevinson gave an interesting TED talk titled A teen just trying to figure it out about her work starting Rookie magazine and her ideas about feminism [3]. Burt Rutan gave an interesting and inspiring TED talk about the future of space expploration [4]. One of his interesting points is that fun really is defendable in regard to tourism paying for the development of other space industries. Stephen Petranek gave an interesting TED talk about how to prepare for some disasters that could kill a significant portion of the world s population [5]. Some of these are risks of human extinction, we really need to spend some money on it. John Wilbanks gave an intresting TED talk about the way that current informed consent laws prevent large-scale medical research [6]. He says I live in a web world where when you share things beautiful stuff happens, not bad stuff . Joey Hess was interviewed for The Setup and the interview sparked a very interesting Hacker News discussion about workflow for software development [7]. Like most developers I prefer large screens with high resolution, I have an EeePC 701 which works reasonably well for an ultra-portable system but I largely don t use it now I have an Android phone (extremely portable and totally awful input usually beats moderately portable and mostly awful input for me). But Joey s methods are interesting and it seems that for some people different systems give the best result. Jeff Masters gave an insightful TED talk about the weather disasters that may seriously impact the US in the next 30 years [8]. Governments really need to start preparing for such things, some of them are really cheap to mitigate if work is started early. Bryan Stevenson gave an inspiring TED talk about the lack of justice in the US justice system [9]. Wouter Verhelst wrote an insightful article about some of the criticisms of Linux from Windows users [10]. He references a slightly satirical post he previously wrote about why Windows isn t ready for desktop use. Paul Carr wrote an interesting article comparing disruptive business practices of dot-com companies to the more extreme aspects of Ayn Rand s doctrine [11]. In reading some of the links from that article I discovered that Ayn Rand was even more of a sociopath than I had previously realised. Lindy West gave an amazing Back Fence PDX talk about dealing with nasty blog comments from the PUA/MRA communities [12]. After investigating them she just feels sorry for the trolls who s lives suck. Hang from the Vlogbrothers explains gender, sex, sexual orientation, etc [13]. Rick Falkvinge wrote an interesting article about recent political news from Brazil, they had a proposed law that was very positive for liberty on the Internet but it was sabotaged by the media and telcos [14]. We should try to avoid paying any money to the media industry so that they can go away sooner. Amy Cuddy gave an interesting TED talk about body language, power, and the imposter syndrome [15]. Caleb Chung gave an interesting TED talk about toy design which focussed on Pleo a robotic dinosaur with a SD card and USB socket to allow easy reprogramming by the user [16].

1 January 2013

Russ Allbery: 2012 Book Reading in Review

For the year of 2012, I finished and reviewed 60 books, the same number as 2011. Given how stressful and chaotic much of the year was, I count this a major triumph. The hardest part was not the reading but the review writing; the end of the year required a significant push to finish writing reviews of all the books I'd read that year, and at one point I was reviewing things I'd read more than two months earlier. But I can enter 2013 entirely caught-up. This continues to feel like about the right pace, striking a balance between reading enough that I can pursue multiple reading goals at the same time, while leaving enough time for other projects and video games. I gave three novels a 10 out of 10 this year, but the stand-out even among that group was Elizabeth Wein's Code Name Verity. Not only was it my favorite book of the year, but it was one of the best books I've ever read. The other two 10-rated books are also highly recommended, of course: Matt Ruff's fascinating novel of multiple personalities, Set This House in Order; and C.J. Cherryh's SF classic, Cyteen. The latter was a re-read in advance of reading the sequel, Regenesis, which I also recommend if you've read and liked the last third of Cyteen. Other fiction highlights of the year were China Mi ville's Embassytown and Suzanne Collins's The Hunger Games triology, particularly the third book, Mockingjay. Embassytown continues the trend from Mi ville's The City & The City of tighter, faster-moving novels while moving into space opera and first contact territory. It keeps Mi ville in the top rank of current SFF writers, despite having some suspension of disbelief problems. Collins's world-building in The Hunger Games also has suspension of disbelief problems, but I understood why the series was so popular. I was caught by surprise by its look at violence and its after-effects and was particularly impressed by Mockingjay and Collins's chosen ending. I think this is a popular series that lives up to the hype. There were no 10 ratings in non-fiction this year, but several books nonetheless stood out. John Kenneth Galbraith's The Affluent Society is a thought-provoking book that questions the foundation of a consumer-oriented economy while perceptively pointing out how it distorts choices away from public goods. Susan Cain's Quiet is a passionate defense of introversion in an extroverted world. And, finally, Joshua Bloch's Effective Java and Damian Conway's Perl Best Practices are both insightful looks at the good and bad of their respective languages and taught me a great deal as a practicing programmer. One final highlight to mention: Eclipse Phase by Posthuman Studios is an excellent RPG sourcebook, at least from the perspective of interesting world-building. I haven't played it, but I thoroughly enjoyed reading it (and have acquired nearly all of the supplements). The full analysis includes some additional personal reading statistics, probably only of interest to me.

Dirk Eddelbuettel: Ragnar Relay Chicago 2012

One things I never quite got around to during 2012 was to blog about the awesome relay we ran in early June. This was the race formerly known as MC200 (for Madiston, WI, to Chicago, IL, by way of Milwaukee, WI, for about 200 miles) and is now part of the Ragnar Relay series: Ragnar Chicago. We ran as a so-called ultra team of six runners, as opposed to a regular ream of twelve. The course is cut into 36 segments; on regular teams you get 3, we each had twice that. My first leg was a combined 17 miles in what turned out to be pretty blistering heat in mid-to-late afternoon. My fellow team members were awesome in getting me lots of water an ice, and I managed to hold onto a pace of just over 8 min/miles. One of the harder runs I've had. Next was a wonderful run pretty much exactly at midnight under starry skies---about seven or so miles followed by ten more miles the next morning. We ended up coming third (yay!) beating the next time by about six or seven seconds (!!) over a total time of 25 or 26 hours. It was hard. It was fun. It was exhilirating. It may also have broken me as I haven't really run much since. So good intentions for 2013: get back into the groove.

31 December 2012

Russ Allbery: Review: Wasted Heart

Review: Wasted Heart, by Lynn Galli
Publisher: Penikila
Copyright: 2010
ISBN: 1-935611-32-1
Format: Kindle
Pages: 252
Austy is a lawyer working for the US District Attorney's office in Seattle. Six months ago, she moved there from Virginia, in large part because she fell in love with her best friend. Her married best friend. Now, she's trying (mostly unsuccessfully) to deal with the Seattle lesbian scene while becoming (successfully) the go-to lawyer for prosecuting cases with a substantial technology component. Two things are about to complicate that life: meeting and working with FBI special agent Elise Bridie, and working on a case that involves her best friend's company. Yes, that best friend. This is, as you might have guessed, a lesbian romance novel, so with that should come my "reading out of genre disclaimer": it's rather easy, when reading a genre one normally doesn't read, to find books fresh and interesting when they're unoriginal to regular genre readers, or to miss genre conventions or reactions because one has not read the previous work. You should therefore take my opinions with a grain of salt, particularly if you read more romance novels than I do (which wouldn't be hard). I grabbed this book for a few reasons: I felt like reading a warm-hearted story focused on human relationships, I wanted a reasonably fast read (since I'm obsessed with meaningless numeric milestones, like how many books I read in a year), and I wanted something with an upbeat ending. Romance generally delivers on those fronts. Lesbian romance has the added bonus of avoiding a whole bunch of gender essentialism and heterosexual romance tropes that tend to give me hives. Landing was excellent, so I thought I'd see if I could find more of that. Wasted Heart does indeed deliver, although it's not quite as good as Landing. There's a bit too much physical attraction and love based on appearance for my taste, but Galli leavens that with quite a lot of interesting psychology. Both Austy and Elise are private and cautious people, and for me that makes the romance feel more realistic. It also draws out the initial phase of getting to know the other person, which is the part of the romance that I often find the most interesting. Elise in particular is a great character maybe a little too ideal in a few places, but smart and observant in a way that's very fun to read about. Like a lot of romances, there is some annoying miscommunication and misunderstanding (predictably right around the height of the tension), but it's not quite as stupid as those problems usually are, and Galli doesn't drag it out until one starts wanting to hit the characters with blunt objects. As good or better is the community that surrounds the main characters. Galli has a full cast of believable and interesting characters, most notably including Austy's old gang from Virginia. The tensions between having a group of friends one cares about deeply, wanting to build a new life after making a major break and major decision but having difficulty sustaining that under the onslaught of those friends, and being easily tossed about by expressive and outgoing people when one is private and a bit quiet all play out here. And, refreshingly, without a lot of angst, and with quite a bit of intelligent introspection. Both Austy and her friends are smarter and more considerate than most of the characters I've seen in this sort of situation. Another good bit is the little pieces of legal drama. Galli doesn't neglect Austy's work life, and while this is certainly not a legal thriller or procedural, we do get some nice scenes of Austy doing her job. I love courtroom scenes, particularly ones (like these) that are more on the realistic side, so that was a nice bonus. Austy and Elise's jobs also lead to a somewhat predictable bit of drama, and I was a bit less enamored of that; the more I thought about it, the less well it hung together for me. But I did like that it stayed firmly on the realistic side, including a look at psychological aftereffects for real people rather than thriller protagonists. This is not the sort of book that I would tell someone to rush out and buy, but it met my criteria nicely and delivered just the sort of entertainment I was looking for. (And it's not particularly expensive on Kindle, at least in the US.) There was a bit too much focus on physical appearance and physical infatuation, and Elise might be just a little too perfect, but I enjoyed it. Note that I read the special edition, which includes a (much appreciated) epilogue that closes out the story. That seems to be the default edition for sale everywhere I look, but in case you see the older version, I recommend the special edition. Rating: 7 out of 10

Gregor Herrmann: RC bugs 2012/52

hey, this week the list of RC bugs I touched is even shorter than in the last weeks:

Russell Coker: Modern Swap Use

A while ago I wrote a blog post debunking the idea that swap should be twice as large as RAM [1]. The issue of swap size has come up for discussion in a mailing list so it seems that it s worth another blog post. Swap Size In my previous post I suggested making swap space equal to RAM size for systems with less than 1G of RAM and half RAM size for systems with 2-4G of RAM with a maximum of 2G for any system. Generally it s better to have the kernel Oom handler kill a memory hungry process than to have the entire system go so slow that a hardware reset is needed. I wrote memlockd to lock the important system programs and the shared objects and data files they use into RAM to make it easier to recover from some bad paging situations [2], but even that won t always make it possible to login to a thrashing system in a reasonable amount of time. But it s not always good to reduce swap size. Sometimes if you don t have enough swap then performance sucks in situations where it would be OK if there was more swap. One factor in this regard is that pages mapped read-only from files are discarded when there is great memory pressure and no swap space to page-out the read-write pages. This can mean that you get thrashing among memory for executables and shared objects while there s lots of unused data pages in RAM that can t be paged out. One noteworthy corner case in this regard is Chromium which seems to take a lot of RAM if you have many tabs open for a long time. Another factor is that there is a reasonable amount of memory allocated which will almost never get used (EG all the X based workstations which have 6 gettys running on virtual consoles which will almost never be used). So a system which has a large amount of RAM relative to it s use (EG a single-user workstation with 8G of RAM) can still benefit from having a small amount of swap to allow larger caches. One workstation I run has 8G of RAM for a single user and typically has about 200M of the 1G swap space in use. It doesn t have enough swap to make much difference if really memory hungry programs are run, but having 4G of memory for cache instead of 3.8G might make a difference to system performance. So even systems which can run without using any swap can still be expected to give better performance if there is some swap space for programs that are very inactive. I have some systems with as much 4G of swap, but they are for corner cases where swap is allocated but there isn t a lot of paging going on. For example I have a workstation with 3G of RAM and 4G of swap for the benefit of Chromium. Hardware Developments Disk IO performance hasn t increased much when compared to increases in RAM size over the last ~15 years. A low-end 1988 era hard drive could sustain 512KB/s contiguous transfer rates and had random access times of about 28ms. A modern SATA disk will have contiguous transfer rates getting close to 200MB/s and random access times less than 4ms. So we are looking at about 400* the contiguous performance and about 10* the random access since 1988. In 1988 2MB was a lot of RAM for a PC, now no-one would buy a new PC with less than 4G so we are looking at something like 2000* the RAM size in the same period. When comparing with 1993 (when I first ran a Linux server 24*7) it was 4M of RAM, maybe 15ms random access, and maybe 1MB/s contiguous IO approximately double everything I had in 1988. But I ll use the numbers from 1988 because I m more certain of them even though I never ran an OS capable of proper memory management on the 1988 PC. In the most unrealistically ideal situation paging IO will be a factor of 5* more expensive now relatively than it was in 1988 (RAM size increase by a factor of 2000 divided by contiguous IO performance increase by a factor of 400). But in a more realistic situation (random IO) it will be 200* more expensive relatively than it was. In the early 90s a Linux system with swap use equal or greater than RAM could perform well (my recollection is that a system with 4M of RAM and 4M of active swap performed well and the same system with 8M of active swap was usable). But nowadays there s no chance of a system that has 4G of RAM and 4G of swap in active use being usable in any way unless you have some corner case of an application allocating RAM and not using it much. Note that 4G of RAM doesn t make a big system by today s standards, so it might be reasonable to compare 2M in 1988 to 8G or 16G today. But I think that assuming 4G of RAM for the purpose of comparison makes my point. Using Lots of Swap It is probably possible to have some background Chromium tabs using 4G of paged out RAM with good system performance. I ve had a system use 2G of swap for Chromium and perform well overall. There are surely many other ways that a user can launch processes that use a lot of memory and not use them actively. But this will probably require that the user know how the system works and alter their usage patterns. If you have a lot of swap in use and one application starts to run slowly you really don t want to flip to another application which would only make things worse. If you use tmpfs for /tmp then you need enough swap to store everything that is written to /tmp. There is no real upper limit to how much data that could be apart from the fact that applications and users generally don t want to write big data files that disappear on reboot. I generally only use a tmpfs for /tmp on servers. Programs that run on servers tend not to store a large amount of data in /tmp and that data is usually very temporary, unlike workstation users who store large video files in /tmp and leave them there until the next reboot. Are there any other common corner cases that allow using a lot of swap space without serious performance problems? My Swap Space Allocation When I setup Linux workstations for other people I generally use 1G of swap. For a system that often has multiple users (via the user-switch feature) I use 2G of swap if it only has 3G of RAM (I run many systems which are limited to 3G of RAM by their chipset). For systems that I personally use I generally allocate 4G of swap, this is to cater to excessive RAM use by Chromium and also because I test out random memory hungry programs and can t easily predict how much swap space will be needed a typical end-user desktop system is a lot more predictable than a workstation used for software development. For servers I generally allocate 512M of swap space. That s enough to page out things that aren t used and make space for cache memory. If that turns out to be inadequate then it s probably best to just let the watchdog daemon reboot the system.

30 December 2012

Neil McGovern: Fleeing Google

For some time, I have been concerned about the amount of data I store on Google. This post aims to show alternatives to the service. This is primarily for two reasons. The first is simple. I would like to avoid a single point of failure, and the risk that I could be suspended for a number of months with no recourse. The second if more political. I'm worried about the way that certain governments can subpoena information which they may not be able to collect directly. Now, don't get me wrong. Google is good at transparency. They also provide a very useful service that works incredibly well. In fact, the quality of the integration and services provided is the primary reason I have not changed previously. Calendaring is something that I naturally feel shouldn't be hard, but somehow is. Google Calendaring service is rather good, and far outstrips anything open source I've found so far. However, I've managed to migrate to Zoho which offers a very similar level of functionality. It's also very easy to extract your entire calendar from Google, which makes the transition easier. Reader is a service provided to subscribe and read RSS feeds. In the past, I've used Gregarius as a self hosted solution, but this seems to have disappeared recently. I briefly flirted with NetVibes, which works well once you turn off the frustrating 'widget' mode, but has completely failed to address the need to have native tablet apps. In the end, I settled for NewsBlur which also has the clever use of hotkeys for navigation which I found essential in Google's products. Picasa has stood me well for hosting photos for some time. And photo hosting is something I have quite a large amount of professional experience with :) Despite having previously been in competition with them, I can heartily recommend Flickr, even with it's dogmatic reliance to creating a Yahoo! email address for each account. Search still has some way to go to find a viable replacement. I've tried DuckDuckGo, but found it lacking. Much like the mistake that Apple seem to have made with the launch of Apple Maps, there seems to be a misunderstanding that search is actually HARD.
There are two camps when it comes to searching. You can be dumb, or you can be smart. By being dumb, the user can provide enough information to help narrow the results down to a single (or at least manageable) result set. This works fine, and is a traditional model. The opposite is to be smart. Simply get the results right, using a variety of metrics to work out what the user wanted in the first place. This is obviously more difficult, but infinitely more valuable, and is something that Google excels at. The problem lies when you fall in between these stalls. Unfortunately, it seems that there is no viable alternative to the big G at the moment.

Next.

Previous.