Search Results: "ag"

13 January 2013

Bernhard R. Link: some signature basics

While almost everyone has already worked with cryptographic signatures, they are usually only used as black boxes, without taking a closer look. This article intends to shed some lights behind the scenes. Let's take a look at some signature. In ascii-armoured form or behind a clearsigned message one often does only see something like this:
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
iQIcBAABAgAGBQJQ8qxQAAoJEH8RcgMj+wLc1QwP+gLQFEvNSwVonSwSCq/Dn2Zy
fHofviINC1z2d/voYea3YFENNqFE+Vw/KMEBw+l4kIdJ7rii1DqRegsWQ2ftpno4
BFhXo74vzkFkTVjo1s05Hmj+kGy+v9aofnX7CA9D/x4RRImzkYzqWKQPLrAEUxpa
xWIije/XlD/INuhmx71xdj954MHjDSCI+9yqfl64xK00+8NFUqEh5oYmOC24NjO1
qqyMXvUO1Thkt6pLKYUtDrnA2GurttK2maodWpNBUHfx9MIMGwOa66U7CbMHReY8
nkLa/1SMp0fHCjpzjvOs95LJv2nlS3xhgw+40LtxJBW6xI3JvMbrNYlVrMhC/p6U
AL+ZcJprcUlVi/LCVWuSYLvUdNQOhv/Z+ZYLDGNROmuciKnvqHb7n/Jai9D89HM7
NUXu4CLdpEEwpzclMG1qwHuywLpDLAgfAGp6+0OJS5hUYCAZiE0Gst0sEvg2OyL5
dq/ggUS6GDxI0qUJisBpR2Wct64r7fyvEoT2Asb8zQ+0gQvOvikBxPej2WhwWxqC
FBYLuz+ToVxdVBgCvIfMi/2JEE3x8MaGzqnBicxNPycTZqIXjiPAGkODkiQ6lMbK
bXnR+mPGInAAbelQKmfsNQQN5DZ5fLu+kQRd1HJ7zNyUmzutpjqJ7nynHr7OAeqa
ybdIb5QeGDP+CTyNbsPa
=kHtn
-----END PGP SIGNATURE-----
This is actually only a form of base64 encoded data stream. It can be translated to the actual byte stream using gpg's --enarmor and --dearmour commands (Can be quite useful if some tool only expects one BEGIN SIGNATURE/END SIGNATURE block but you want to include multiple signatures but cannot generate them with a single gpg invocation because the keys are stored too securely in different places). Reading byte streams manually is not much fun, so I wrote gpg2txt some years ago, which can give you some more information. Above signature looks like the following:
89 02 1C -- packet type 2 (signature) length 540
        04 00 -- version 4 sigclass 0
        01 -- pubkey 1 (RSA)
        02 -- digest 2 (SHA1)
        00 06 -- hashed data of 6 bytes
                05 02 -- subpacket type 2 (signature creation time) length 4
                        50 F2 AC 50 -- created 1358081104 (2013-01-13 12:45:04)
        00 0A -- unhashed data of 10 bytes
                09 10 -- subpacket type 16 (issuer key ID) length 8
                        7F 11 72 03 23 FB 02 DC -- issuer 7F11720323FB02DC
        D5 0C -- digeststart 213,12
        0F FA -- integer with 4090 bits
                02 D0 [....]
Now, what does this mean. First all gpg data (signatures, keyrings, ...) is stored as a series of blocks (which makes it trivial to concatenate public keys, keyrings or signatures). Each block has a type and a length. A single signature is a single block. If you create multiple signatures at once (by giving multiple -u to gpg) there are simple multiple blocks one after the other. Then there is a version and a signature class. Version 4 is the current format, some really old stuff (or things wanting to be compatible with very old stuff) sometimes still have version 3. The signature class means what kind of signature it is. There are roughly two signature classes: A verbatim signature (like this one), or a signature of a clearsigned signature. With a clearsigned signature not the file itself is hashed, but instead a normalized form that is supposed to be invariant under usual modifications by mailers. (This is done so people can still read the text of a mail but the recipient can still verify it even if there were some slight distortions on the way.) Then the type of the key used and the digest algorithm used for creating this signature. The digest algorithm (together with the signclass, see above) describes which hashing algorithm is used. (You never sign a message, you only sign a hashsum. (Otherwise your signature would be as big as your message and it would take ages to create a signature, as asymetric keys are necessarily very slow)). This example uses SHA1, which is no longer recommended: As SHA1 has shown some weaknesses, it may get broken in the not too distant future. And then it might be possible to take this signature and claim it is the signature of something else. (If your signatures are still using SHA1, you might want to edit your key preferences and/or set a digest algorithm to use in your ~/.gnupg/gpg.conf. Then there are some more information about this signature: the time it was generated on and the key it was generated with. Then, after the first 2 bytes of the message digest (I suppose it was added in cleartext to allow checking if the message is OK before starting with expensive cryptograhic stuff, but it might not checked anywhere at all), there is the actual signature. Format-wise the signature itself is the most boring stuff. It's simply one big number for RSA or two smaller numbers for DSA. Some little detail is still missing: What is this "hashed data" and "unhashed data" about? If the signed digest would only be a digest of the message text, then having a timestamp in the signature would not make much sense, as anyone could edit it without making the signature invalid. That's why the digest is not only signed message, but also parts of the information about the signature (those are the hashed parts) but not everything (not the unhashed parts).

Russell Coker: Android Multitasking

My new Samsung Galaxy S3 has support for Multi Window Mode , here is a video which shows how to use this, Multi Window Mode starts at about 2:30 [1]. A common complaint about Android is the lack of multitasking, which is partly true and only slightly alleviated by Multi Window Mode. Running Multiple Programs Traditionally Android has multitasked with a similar ability to a Unix shell session, you can have applications running in the background and switch between them but you can t see multiple applications running at the same time (apart from seeing notification messages at the top of the screen). The new Multi Window Mode allows two or more applications to share a screen. But it only applies to a small number of applications which support it, on my phone that is applications shipped by Samsung and Google Chrome. Also I can t have multiple copies of Chrome open at the same time which means I can t do the things that I do on every PC that runs Chromium (the non-Google build of Chrome). I have not yet found a situation where Multi Window Mode has been useful to me, the applications I use for most tasks don t support it. So while Android being based on Linux does multitask really well in the technical computer-science definition it doesn t do so well in the user-centric definition. In practice Android multitasking is mostly about task switching and doing things like checking email in the background. Having multiple programs running at once is particularly difficult due to the Android model of applications sometimes terminating when they aren t visible. A common task is to view a message in a MUA and switch between that and another window (EG a web browser). K9 is my preferred MUA for Android which seems to have no option to switch back and still be viewing the same message as before the task switch so at least three actions need to be taken to get back to the same message after I resume K9. One major feature to make multitasking on Android more usable would be a way of rapidly switching between two applications and being certain that each one would be in the same state when the user switches back to it. Copy and Paste Another problem with Android multitasking is the difficulty in copying and pasting text. While copy/paste is not strictly required for multitasking it is a logical requirement when you have multiple applications running on behalf of a single user. For PCs everyone knows that you select text by holding down the SHIFT key while using the cursor control keys or by holding down the mouse button and swiping the mouse cursor over the text and you then use CTRL-C or the Edit menu to copy text. On Android it s a long press to select text which then gives you markers for the start and end, you drag those markers around and then select that you want to copy the text. This is at best a lot more inconvenient than using a high resolution input device like a mouse to select text. At worst it doesn t seem particularly reliable, K9 Mail for example won t let me copy text from a message for unknown reasons on a desktop OS such problems are vanishingly rare such that I can t think of an example of it happening. IO and Multitasking Multitasking for a user (as opposed to the multitasking needed to host dozens or hundreds of concurrent users) on Unix servers was very limited in the days of VT100 terminals and similar devices. Programs such as GNU Screen [2] allowed a text display to perform windowing functions that are similar to a modern GUI. But generally it seems that the ability for a user to run multiple programs at once is largely limited by their ability to see the output and to rapidly switch between programs or sessions. As a keyboard with ~100 keys and a text display with 80*25 characters is a major limitation it s obviously going to be a comparable (and often greater) limitation to have an on-screen keyboard that takes half the screen space, a single program taking all the screen, and a drop-down status bar that might be useful for multitasking. With Android 4.0 and above you can activate a task switcher by holding down the home button for two seconds [3]. There are also a variety of third party task switching programs on the Google Play store which all seem to start by holding down the home button. One problem with these options is that they require an extended press of the home button where the ideal is something that is as quick as ALT-TAB or a single mouse movement on a desktop system. One possible input action would be to switch between the most recent tasks with a palm swipe on the screen an operation that is quick and easy. Currently a palm swipe is used for a screen capture, but as there are four possible directions for swiping the screen one of them could be used for screen capture and another two for task switching. But this wouldn t do a lot of good without the ability to switch tasks without losing the context that either requires Android application changes or having the OS not tell an application that it was occluded. The iPaQ had interesting capabilities for input, it had a main button on the front that could be pushed in four directions (usually for cursor control) as well as being pushed in, four extra buttons on the front, and a button on the side [4]. I don t recall the methods that Familiar Linux on the iPaQ [5] used for task switching, but it was less of an issue as the iPaQ had less RAM, no Internet access and no telephony functions. I think that adding a bunch of extra buttons to an Android phone would make it a lot more useful. The iPaQ method of cursor control is one that could be considered (it could alleviate the copy/paste problems among other things). As an aside in two years of using Android phones I ve done less serious writing on Android than I did in my first two years of using Familiar on the iPaQ largely due to the lack of keys and a stylus on my Android phones. The screen size on Android phones is also a limit for multitasking. The earlier/cheaper phones that have small screens with resolutions such as 320*480 have very limited ability to display two programs at once. The 720*1280 display in the Galaxy S3 has a lot more potential in this regard and the Galaxy Note 2 and the HTC J Butterfly AKA Droid DNA have even more potential. In the future it seems that screen size limitations on phone multitasking will be a solved problem for everyone who can afford one of the high end phones which incidentally are much cheaper than the iPaQ was 10 years ago. Conclusion Checking email in the background etc is very useful on Android systems. But in terms of the user running two programs at once it seems very limited, and that situation seems likely to remain until the vendors adopt multi-window support. This could be difficult given that Google applied a lot of pressure to CyanogenMod to stop them from doing it [6]. Even when a system with a large display (particularly a tablet) runs a version of Android that supports multi-window mode that still won t entirely solve the problem. No matter how big your display is there are occasions when you need to use it all for a single application while still having the ability to rapidly switch to another application. User interface tweaks to allow rapid task switching without losing application context are necessary. Finally for the past two years that I ve been using Android devices I have been disappointed in the ways that they compare poorly to the iPaQ running Familiar I used 10 years ago. I once wrote a feature article for a magazine on an iPaQ while so far I haven t even written a blog post on an Android device. I think that some of the earlier Android devices might have been better in some ways, the trackball on the HTC Nexus One might have made it more suitable for writing long articles than more recent Android devices.

Russ Allbery: afs-monitor 2.4

I love free software. Two more patches from users of my AFS monitoring scripts (primarily for Nagios) on the heels of the last release, so have another release. This one adds another ignore line to check_afs_bos for the output when there are long-running scheduled jobs, and adds an optional regex filter to check_afs_quotas to select the volumes whose quota one wants to check. Thanks to Georg Sluyterman and Christian Ospelkaus for the patches. You can get the latest version from the afs-monitor distribution page. As before, Debian packages have been uploaded to my personal repository. Also, a quick note about a previous journal entry: Philip Martin and Bernhard R. Link pointed me at the relevant part of the gpg source, which will only check signatures if all signatures use the same class and digest. I updated my previous entry. Thank you!

12 January 2013

John Sullivan: Aaron Swartz

Aaron was an inspiration to me personally, politically, and professionally ever since we met (ice cream and word games with a small group in a bank vault at Herrell's in Harvard Square) several years ago. I don't understand how things got to this point, but I know I'm angry along with Lessig. I'm so sorry for all of his family and friends; all the rest of us can do is try to make even a tiny sliver of the difference he did.

Jakub Wilk: Internationalization environment variables

Setting internationalization environment variables is a bit tricky. For example, this:
$ LANG=sv_SE.UTF-8 stat /nonexistent
may look like a way to make stat(1) print the error message in Swedish. Yet there are many ways it could go wrong:
  1. LC_MESSAGES could be set in the environment, overriding LANG.
  2. LC_ALL could be set, overriding both LC_MESSAGES and LANG.
  3. LANGUAGE could be set, overriding LC_ALL, LC_MESSAGES, and LANG. For extra complexity, LANGUAGE has no effect if LC_MESSAGES is effectively set to C. (Also, this variable is a GNUism.)
  4. The locale could be simply missing from the system.
To make these things a little less intricate, I wrote localehelper. It's a bit like env(1), but it takes care of: This does the right thing:
$ localehelper LANG=sv_SE.UTF-8 stat /nonexistent
stat: kan inte ta status p   /nonexistent : Filen eller katalogen finns inte

Sylvestre Ledru: Some more cool stuff with LLVM/Clang

One of the drawback of C is the hard memory management. It is because of this aspect that Clang provides a way to detect, at runtime, memory errors. Called Address Sanitizer, it allows, while the program is running, to keep track of the memory and detect typical errors (out-of-bound accesses, use of a variable after a free, etc).
# With the following packages (version 3.2-1~exp3)
$ sudo apt-get install clang clang-3.2 compiler-rt -t experimental
Taking the simple following example, even if the errors are obvious, compilers will accept this code.
#include <stdlib.h>
int main()
char *x = (char*)malloc(10 * sizeof(char*));
free(x);
return x[5];
Built and run with:
$ clang -O1 -g -fsanitize=address -fno-omit-frame-pointer foo.c -o foo
$ ./foo &> memoryDebug.log
The previous command will generate a log file. Log which can be post processed with the asan_symbolize command.
$ asan_symbolize memoryDebug.log
which will give:
=================================================================
==21368== ERROR: AddressSanitizer: heap-use-after-free on address 0x7fb22e547f45 at pc 0x408c44 bp 0x7ffff60c10b0 sp 0x7ffff60c10a8
READ of size 1 at 0x7fb22e547f45 thread T0
    #0 0x408c43 in main /tmp/foo.c:5
    #1 0x7fb22d99f6ac in __libc_start_main /home/adconrad/eglibc-2.16/csu/libc-start.c:227
0x7fb22e547f45 is located 5 bytes inside of 80-byte region [0x7fb22e547f40,0x7fb22e547f90)
freed by thread T0 here:
    #0 0x408c90 in __interceptor_free ??:0
    #1 0x408c0a in main /tmp/foo.c:4
    #2 0x7fb22d99f6ac in __libc_start_main /home/adconrad/eglibc-2.16/csu/libc-start.c:227
previously allocated by thread T0 here:
    #0 0x408d50 in __interceptor_malloc ??:0
    #1 0x408bff in main /tmp/foo.c:3
    #2 0x7fb22d99f6ac in __libc_start_main /home/adconrad/eglibc-2.16/csu/libc-start.c:227
Shadow byte and word:
  0x1ff645ca8fe8: fd
  0x1ff645ca8fe8: fd fd fd fd fd fd fd fd
More shadow bytes:
  0x1ff645ca8fc8: fa fa fa fa fa fa fa fa
  0x1ff645ca8fd0: fa fa fa fa fa fa fa fa
  0x1ff645ca8fd8: fa fa fa fa fa fa fa fa
  0x1ff645ca8fe0: fa fa fa fa fa fa fa fa
=>0x1ff645ca8fe8: fd fd fd fd fd fd fd fd
  0x1ff645ca8ff0: fd fd fd fd fd fd fd fd
  0x1ff645ca8ff8: fa fa fa fa fa fa fa fa
  0x1ff645ca9000: fa fa fa fa fa fa fa fa
  0x1ff645ca9008: fa fa fa fa fa fa fa fa
Stats: 0M malloced (0M for red zones) by 1 calls
Stats: 0M realloced by 0 calls
Stats: 0M freed by 1 calls
Stats: 0M really freed by 0 calls
Stats: 0M (128 full pages) mmaped in 1 calls
  mmaps   by size class: 8:2047;
  mallocs by size class: 8:1;
  frees   by size class: 8:1;
  rfrees  by size class:
Stats: malloc large: 0 small slow: 1
==21368== ABORTING
The main advantage compare to valgrind is that asan is supposed to be way more faster. Threads can be also tricking to develop.
With the following example from stolen from upstream:
#include <pthread.h>
int Global;
void *Thread1(void *x)
Global = 42;
return x;
int main()
pthread_t t;
pthread_create(&t, NULL, Thread1, NULL);
Global = 43;
pthread_join(t, NULL);
return Global;
$ clang -fsanitize=thread -g -O1 foo2.c -fPIE -pie -o foo
$ ./foo
==================
WARNING: ThreadSanitizer: data race (pid=21416)
  Write of size 4 at 0x7f2f1a214a50 by thread 1:
    #0 Thread1 /tmp/foo2.c:4 (exe+0x00000000f850)
  Previous write of size 4 at 0x7f2f1a214a50 by main thread:
    #0 main /tmp/foo2.c:10 (exe+0x00000000f8a4)
  Thread 1 (tid=21417, running) created at:
    #0 pthread_create ??:0 (exe+0x00000001267e)
    #1 main /tmp/foo2.c:9 (exe+0x00000000f894)
==================
ThreadSanitizer: reported 1 warnings
Note that clang also provides scan-build, a static analyzer for memory issues. Not as powerful as the Address Sanitizer (it only works on a file), it provides some excellent reports. See the Wouter's blog post on this subject or the automatic report of scan-build on Scilab.

Russ Allbery: Review: Asimov's, April/May 2011

Review: Asimov's Science Fiction, April/May 2011
Editor: Sheila Williams
Issue: Volume 35, No. 4 & 5
ISSN: 1065-2698
Pages: 192
Williams's editorial this issue is about the tendency of SF to take a rose-colored view of the world, which on the surface seems odd given the tendency of recent SF towards dystopia. But she makes a good point that the portrayal of the past is rose-colored, linking that into the current steampunk trend. She doesn't take the argument quite as far as I'd like, but I'm glad to see editorials raising points like this. I'm inclined to think that a lot of the rose-colored frame of the past is because few of us want to read about real historic conditions at any length, even for edification, because the stench and discomfort isn't fun to read about. Silverberg's column is another discussion of programmatic plot generators, which mostly makes the point that plot ideas are the easy part of writing. James Gunn contributes an extended biography of Isaac Asimov that probably won't be new to long-time genre readers but may fill in some details (although it politely sticks to mostly flattering material). Spinrad's book review column is one of his better ones; it looks at two novels by China Mi ville and two by Ian McDonald and explores differences in world-building. Spinrad predictably makes the case in favor of science fiction with rules and against the New Weird, but the discussion along the way was worth reading. "The Day the Wires Came Down" by Alexander Jablokov: Speaking of steampunk, here's an example. There is even an airship, although the primary technological focus is suspended street cars. Jablokov postulates a city-wide transportation network of suspended carriages called telpher cars, along with a city built around the telpher cables: stores on roofs, windows displaying merchandise to passing cars, and even a history of heated competition and dirty tricks between competing telpher networks. The story is set, as the title would indicate, on the last day of the network. It's being shut down for cost, with some hints that progress is destroying something precious. There is a plot here, revolving around some mysteries of the history of the telpher network and the roles of several people in that history. But the story is primarily a celebration of old technology. It's a rail fan's story recast with a steampunk technology, featuring the same mix of fascination with mechanics and a sense that the intricate details are falling out of common knowledge (and perhaps usefulness). As a story, it's a bit slow-moving, but I enjoyed the elegiac tone. (7) "An Empty House with Many Doors" by Michael Swanwick: This is a very short story, more of an emotional profile, involving a man's reaction to the death of his wife. Oh, and parallel universes. It's sort of the inverse of Niven's classic "All the Myriad Ways." Similar to Niven's story, I found the idea vaguely interesting but the conclusion and emotional reaction unbelievable and alien. (5) "The Homecoming" by Mike Resnick: Resnick tends to yank on the heart-strings rather sharply in his stories, so I knew roughly what to expect when a father comes home to find his son is visiting. A son who, rather against his father's wishes, has been significantly altered to be able to live with aliens. Throw in a mother with serious dementia, and you can probably predict what Resnick does with this. Still, most of the story is a two-sided conversation, and I thought he succeeded in doing justice to both sides, even though one of them was destined to lose. (6) "North Shore Friday" by Nick Mamatas: Illegal Greek immigrants, a family-run system for getting them married off before the INS catch them, government psi probes and eavesdropping on thoughts, joint projects between computer and religion departments, secret government experiments, and even ghosts... this story is a complex mess, with numerous thoughts stuck into small boxes and scattered through the surface story. It's one of those stories where figuring out what's going on, and even how to read the story in a sensible way, is much of the fun. If you find that fun, that is; if not, it will probably be frustrating. I wished there was a bit more plot, but there's something delightful about how much stuff Mamatas packs into it. (6) "Clockworks" by William Preston: This is a prequel to Preston's earlier "Helping Them Take the Old Man Down". Like that story, it's primarily a pulp adventure, but layered with another level of analysis and thoughtfulness that tries to embed the pulp adventure in our understanding of human behavior and the nature of the world, although this one stays a bit more pulp than its predecessor. As with Preston's other story, we don't get directly in the head of the Old Man (here, just called the man, but identifiable from clues in both stories as Doc Savage); instead, the protagonist is a former villain named Simon Lukic who the man hopes to have fixed by operating on his brain. The undercurrent that lies beneath a more typical pulp adventure is the question of whether Lukic is actually healed. I think there was a bit too much daring-do and human perfection, but it's a perfectly servicable pulp story with some depth. (6) "The Fnoor Hen" by Rudy Rucker: If you've read any of Rucker's work before, you probably know what to expect: a mind-boggling blizzard of mathematically-inspired technobabble that turns into vaguely coherent surrealism. (You can probably tell that I'm not much of a fan, although the clear good humor in these stories makes it hard to dislike them too much.) There's a mutated chicken and some sort of alternate mathematical space and then something that seems like magic... I'd be lying if I said that I followed this story. If you like Rucker, this seems like the sort of thing that you'd like. (4) "Smoke City" by Christopher Barzak: At the start of this story, I thought it was going to be an emotional parable about immigration. The progatonist lives two lives: one in our world, and one in the Smoke City of industry, a world of hard labor, pollution, and little reward, with families in both. But nearly all of the story is set within Smoke City, and the parable turns out to be a caustic indictment of industry and its exploitation of labor. I kind of wish Barzak hadn't used rape as a metaphor, but when the captains of industry show up, I can't argue with how deeply and accurately the story shoves in the knife. There isn't much subtlety here, but it's still one of the better stories in this issue. (7) "A Response from EST17" by Tom Purdom: I'm very happy to see Purdom's writing appearing regularly. His stories are always quiet and matter-of-fact, and at first seem to miss emotional zest, but they almost always grow on me. He lets the reader fill in their own emotional reactions to events, and there's always a lot going on. This story is a first-contact story, except that the "humans" here are not human at all. They're automated probes sent by two separate human civilizations, with different programming and different governance algorithms, and they quickly start competing negotiations. The aliens they've discovered similarly have factions, who start talking to the different probes in an elaborate dance of gathering information without giving too much away. The twist is that this pattern has replayed itself many times in the past, and information itself can be a weapon. I enjoyed this one from start to finish. (7) "The One That Got Away" by Esther M. Friesner: Friesner is best known, at least to me, for humorous fantasy, and this story is advertised as such from early on. The first-person protagonist is a prostitute in a seaside town. She's bemused to finally be invited over by a sailor who's been eyeing her all evening, but that sailor has something else in mind than normal business. For much of this story, the fantasy element is unclear; when it finally comes, it was an amusing twist. (7) "The Flow and Dream" by Jack Skillingstead: This is a mildly interesting variation on the old SF story of hibernating humans (on a generation ship or elsewhere) waking up to a transformed world. Here, it's not a ship, it's a planet, and the hiberation was to wait for terraforming rather than for transit. The twist comes from an excessively literal computer and the fun of putting together the pieces. Sadly, the story trails off at the end without much new to say. (5) "Becalmed" by Kristine Kathryn Rusch: "Becalmed" takes place immediately before "Becoming One with the Ghosts" and explains the incident that created the situation explored in that story. The first-person protagonist of "Becalmed" is a linguist, an expert in learning alien languages so that the Fleet can understand the civilizations that it runs across. But something went horribly wrong at their last stop, something that she's largely suppressed, and now she's confined to quarters and possibly in deep trouble. As is the ship; they're in foldspace, and they have been for days. "Becalmed" is structed like a mystery, centered around recovering the protagonist's memories. It's also a bit of a legal procedural; the ship is trying to determine what to do with her and to what degree she's responsible. But the heart of the story is a linguistic and cultural puzzle. This is another great SF story from Rusch, whose name on a cover will make me eager to start reading a new magazine. I love both angles on the universe she's built, but I think I like the Fleet even better than the divers. The Fleet captures some of the magic of the original Star Trek, but with much more mature characters, more believable situations, and a more sensible and nuanced version of the Prime Directive. Rusch writes substantial, interesting plots that hold my interest. I'd love to see more like this. (8) Rating: 7 out of 10

11 January 2013

Patrick Matth i: What an ugly (PHP) work..

We still have got some more or less webapplications which are not compatible with PHP higher than version 5.2.x, which is the only blocker for the last Lenny servers to upgrade them to Squeeze.. I do not think that I am alone with this ****** topic :) So the new masterplan is to deploy those applications on seperated Wheezy servers, with PHP 5.2.x running as FastCGI, so that most parts of the system are security supported . First; I didn t documented my steps and I am not 100% done (something like 99%) but I have done the following to have it as clean as possible : PHP 5.2.x is just not able to detect the new multiarch paths, it fails at most dir options. Since patching the whole build system would be *too* much work I decided to hack around this (ln -s /usr/lib/x86 ./foo.so /usr/lib/), then some adjustions to the build dependencies, disablieng some modules, like SSL and libdb (incompatible versions), disabling merged patches and refresh the suhosin hardening patch; I get an working PHP 5.2.17 package on Wheezy.
But this is too easy!
I want packages which I could co-install with the PHP 5.4 packages from Wheezy, 5.2 should only used withing vHosts where I have enabled them.. So I rewrote the whole packaging (*burg* IMHO at all) to use php52 instead of php5 as packaging namespace and also everything is put into /opt as prefix. Much painfull work, but yeah it works.. :)
Some packages, like php52-dev or php-pear are broken, but those were not my goal of this action. If someone is interested in those packages please send me an email.
Since PHP 5.2 is not supported any longer (and that this is at all a big hack) I will not publish the source and binaries at all.

Richard Hartmann: Release Critical Bug report for Week 02

The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze release cycle?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56) 339 (195+144) +135 (+47/+90)
51 178 (124+54) 323 (190+133) +145 (+66/+79)
52 115 (78+37) 289 (190+99) +174 (+112/+62)
1 93 (60+33) 287 (171+116) +194 (+111/+83)
2 82 (46+36) 271 (162+109) +189 (+116/+73)
3 25 (15+10)
4 14 (8+6)
5 2 (0+2)
6 release!
Graphical overview of bug stats thanks to azhag:

Julien Danjou: Overriding cl-json object encoding

CL-JSON provides an encoder for Lisp data structures and objects to JSON format. Unfortunately, in some case, its default encoding mechanism for CLOS objects isn't exactly doing the right thing. I'll show you how Common Lisp makes it easy to change that. Identifying the problem CL-JSON & CLOS CL-JSON mechanism encoding CLOS object is really neat. Let's see how it works for a simple case:
(defclass kitten ()
((tail :initarg :tail)))

(json:encode-json-to-string (make-instance 'kitten :tail 'black))

will produce:
 "tail":"black" 

Still using CL-JSON, we can also decode the JSON object to a CLOS object:
(slot-value
(json:with-decoder-simple-clos-semantics
(json:decode-json-from-string " \"tail\":\"black\" "))
:tail)

That code will return "black". Note that it's also possible to specify which class should be used when decoding objects, but that's beyond the purpose of this article. Postmodern Now, let's introduce Postmodern, a wonderful Common Lisp system providing access to the wonderful PostgreSQL database. It also provides a simple system to map rows in a database to CLOS classes, called DAO for Database access objects. With this, we can easily store our kitten into a table.
(defclass kitten ()
((tail :initarg :tail))
(:metaclass postmodern:dao-class))

If we try to encode this to JSON, it will produce the exact same result seen previously. The problem is what happens when one of our column has a NULL value. Postmodern encodes this using the :null symbol. So this code:
(defclass kitten ()
((tail :initarg :tail :col-type (or s-sql:db-null text)))
(:metaclass postmodern:dao-class))

(postmodern:deftable kitten
(postmodern:!dao-def))

(postmodern:connect-toplevel )

(postmodern:create-table 'kitten)

(json:encode-json-to-string
(postmodern:make-dao 'kitten))

will return:
" "tail":"null" "

Fail! The fact that the column is NULL is represented by the :null symbol. And CL-JSON encodes all symbols as string. This is not at all what we want here! Overriding encode-json CL-JSON provides and uses the encode-json method to encode all kind of object. It is defined as a generic function, and a lot of different methods are implemented to handle the different standard Common Lisp types. The one used for standard-object is defined liked that:
(defmethod encode-json ((o standard-object)
&optional (stream *json-output*))
"Write the JSON representation (Object) of the CLOS object O to
STREAM (or to *JSON-OUTPUT*)."
(with-object (stream)
(map-slots (stream-object-member-encoder stream) o)))

All we need to do here, is to create a new method for our kitten objects, that handles correctly the :null case.
(defclass kitten ()
((tail :initarg :tail :col-type (or s-sql:db-null text)))
(:metaclass postmodern:dao-class))

(export 'kitten)

;; Switch package just to define the new method
(in-package :json)
(defmethod encode-json ((o cl-user:kitten)
&optional (stream json:*json-output*))
"Write the JSON representation (Object) of the postmodern DAO CLOS object
O to STREAM (or to *JSON-OUTPUT*)."
(with-object (stream)
(map-slots (lambda (key value)
(as-object-member (key stream)
(encode-json (if (eq value :null) nil value) stream)))
o)))

;; Go back into our package
(in-package :cl-user)

(postmodern:deftable kitten
(postmodern:!dao-def))

(postmodern:connect-toplevel )

(postmodern:create-table 'kitten)

(json:encode-json-to-string
(postmodern:make-dao 'kitten))

With that new method, as soon as we encounter a :null symbol as a value for an object's slot, we replace it by nil. Now if we try to encode another kitten, we'll get:
 "tail":null 

which is far better for our JavaScript data consumers! In the end, I think that this kind of trick is feasible that easily because of the way CLOS provides its generic method implementation. The fact that methods don't belong to any class makes the extension of every program, library and class so much easier. Doing this in another language like Java would likely by impossible, and in Python it would unlikely be as clean as it is done in Common Lisp. The ability to teach any library about how it should handle your class just by defining a new method is really handy!

Benjamin Mako Hill: Goodbye PyBlosxom, Hello WordPress

Since 2004, I ve used the blogging software PyBlosxom. Over that time, the software has served me well and I have even written a series of patches and plugins. PyBloxsom is blog software designed for hackers. It assumes you already have a text editor you love and relies on features of a POSIX filesystem instead of a relational database. It s designed so you can keep your blog under revision control (since 2004 I ve used GNU Arch, baz, bzr and now git). It is also hackers software in the sense that you should expect to write code to use it (e.g., the configuration is pure Python). I love it. What PyBlosxom does not have is a large community. This summer, the most recent long-time maintainer of the project, Will Kahn-Greene, stepped down. Although the project eventually found a new maintainer, the reality is the project entered maintenance mode years ago and many features available in more popular blogging platforms are unlikely to make it PyBlosxom. The situation with comment spam is particularly dire. I ve written several antispam plugins but time has shown that I don t have the either the expertise or the time to make them as awesome as they need to be to really work in today s web. So, after many months of weighing, waffling, and planning I ve switched to WordPress a great piece of free software with an enormous and established community As you ll know if you ve read my interview on The Setup, I think a lot about the technology I surround myself with. I considered WordPress when I started my blog back in 2004 and rejected it soundly. Eight years ago, I would have laughed at you if you told me I d be using it today; If PyBlosxom is for hackers, WordPress is designed for everyone else. But the way I evaluate software has changed over that period. In the nineties, I used to download every new version of the Linux kernel to compile it it took hours! to try out the latest features. Configurabilty, hackability, and the ability to write my own features was after a point more important than the features the software came with. Today, I m much more aware of the fact that for all the freedom that my software gives me, I simply do not have the time, energy, or inclination to take advantage of that freedom to hack very often. Today, I give much more value to software that is not just free, but that is maintained by a community of people who can do all the work that I would do if I had unlimited time. Although I don t have the time or experience to make WordPress do everything I would like, the collective of all WordPress users does. And they ve done a lot of it already! The flip side matters as well: Today, I give more value to other people using my software. When WordPress doesn t do something and I write a plugin or patch, there are tons of people ready to pick it up and use it and perhaps even to collaborate on it with me. Want to guess how many patches my PyBlosxom plugins have received? None, if my memory serves me. In the past, I ve written about how free software is a victory even when it doesn t build a community. I still believe that. But the large communities at the heart of the most successful free software communities (the promise of open source ) are deeply important in a way that I increasingly value. In that spirit: If you want to make the jump from PyBlosxom to WordPress, I ve shared a Git repository with the scripts I used and wrote for the transition.

Julian Andres Klode: Recursive-descent in Python, using generators

Writing recursive descent parsers is easy, especially in Python (just like everything is easy in Python). But Python defines a low limit on the number of recursive calls that can be made, and recursive descent parsers are prone to exceed this limit. We should thus write our parser without real recursion. Fortunately, Python offers us a way out: Coroutines, introduced in Python 2.5 as per PEP 342. Using coroutines and a simple trampoline function, we can convert every mutually recursive set of functions into a set of coroutines that require constant stack space. Let s say our parser looks like this (tokens being an iterator over characters):
def parse(tokens):
    token = next(tokens)
    if token.isalnum():
        return token
    elif token == '(':
         result = parse(tokens)
         if next(tokens) != ')': raise ...
         return result
    else:
         raise ...
We now apply the following transformations:
return X => yield (X)
parse() => (yield parse()) That is, we yield the value we want to return, and we yield a generator whenever we want to call (using a yield expression). Our rewritten parser reads:
def parse(tokens):
    token = next(tokens)
    if token.isalnum():
        yield token
    elif token == '(':
         result = yield parse(tokens)
         if next(tokens) != ')': raise ...
         return result
    else:
         raise ...
We obviously cannot call that generator like the previous function. What we need to introduce is a trampoline. A trampoline is a function that manages that yielded generators, calls them, and passes their result upwards. It looks like this:
def trampoline(generator):
    """Execute the generator using trampolining. """
    queue = collections.deque()
    def schedule(coroutine, stack=(), value=None):
        def resume():
            if 0:
                global prev
                now = stack_len(stack)
                if now < prev:
                    print("Length", now)
                    prev = -1
                elif prev != -1:
                    prev = now
            result = coroutine.send(value)
            if isinstance(result, types.GeneratorType):     # Normal call
                schedule(result, (coroutine, stack))
            elif isinstance(result, Tail):                  # Tail call (if you want to)
                schedule(result.value, stack)
            elif stack:                                     # Return to parent
                schedule(stack[0], stack[1], result)
            else:                                           # Final Return
                return result
        queue.append(resume)
    schedule(generator)
    result = None
    while queue:
        func = queue.popleft()
        result = func()
    return result
This function is based on the code in PEP 342, the difference being that For a more generic version of that, you might want to re-add exception passing, but the exceptions will then have long tracebacks, so I m not sure how well they will work if you have deep recursion. Now, the advantage of that is that our parser now requires constant stack space, because the actual real stack is stored in the heap using tuples which are used like singly-linked lists in scheme here. So, the only recursion limit is available memory. A disadvantage of that transformation is that the code will run slightly slower for small inputs that could be handled using a normally recursive parser.
Filed under: Python

Russ Allbery: release 1.45 and multiple PGP signatures

release is the tool that I use to manage software releases. This version adds support for signing a tarball with multiple PGP keys at once. For WebAuth, we have an official signing key, but I also wanted to add my personal signature. The minimum supported version of Python was also bumped to 2.5 (see below for why). You can get the latest version from my scripts page. Normally, I wouldn't announce this to everyone, since it's a fairly minor change, except I had a question and a quick Python note. The question: how does one get GnuPG to verify multiple signatures? When I run GnuPG on the resulting detached *.asc file created by this new code, I get the following:
gpg: WARNING: multiple signatures detected.  Only the first will be checked.
gpg: Signature made Thu 10 Jan 2013 03:58:43 PM PST using RSA key ID 5736DE75
gpg: Good signature from "Russ Allbery <rra@stanford.edu>"
gpg:                 aka "Russ Allbery <rra@debian.org>"
gpg:                 aka "Russ Allbery <eagle@windlord.stanford.edu>"
gpg:                 aka "Russ Allbery <eagle@eyrie.org>"
That's nice, I suppose, but it's not as helpful as it could be. Why not verify both signatures and report all of the results? I searched both the gpg man page and the Internet for some option I'm missing, without any luck. (Searching Google for that exact message amusingly turns up lots of verbatim output from Debian repository verification and almost nothing else.) If anyone knows, send me email (rra@stanford.edu or any other convenient address), and I'll update this post with the answer. UPDATE: gpg can currently only verify multiple signatures if the signatures are from keys with the same class and digest. I was attempting this with a 2048-bit RSA key and a 1024-bit DSA key, which gpg does not support. (Yet another reason to replace the WebAuth signing key with a newly-generated key.) Thanks to Philip Martin and Bernhard R. Link for the information! The note is that, way back when I was first learning Python, one of the things that bothered me the most was the lack of safe program execution. The best available was os.spawnv(), which left something to be desired. If you too were bothered by this but don't program in Python enough to keep track of the current state of the art, there's now a subprocess module that has much saner interfaces that work properly, do reasonable things, and avoid the shell. It was added in Python 2.4, with some nicer interfaces added in Python 2.5, so it's fairly safe to use at this point.

10 January 2013

Joachim Breitner: Haskell Bytes again

Janis Voigtl nder has invited me again to give a lecture in his Advanced Functional Programming course at the University of Bonn. Like last time, when I talked about performance analysis in Haskell, I chose a topic that is more on implementationishy side to contrast the theoretic content of the rest of the course. I recycled my talk on the memory representation of GHC that I have held at last year s Meta Main-Rhein Chaos Days. As usual I provide a transcript of the talk as intended (not as held). It is slightly revised over the old version and I have added a section on unboxed values in constructors, which I have nevertheless skipped today.

Vincent Fourmond: Introducing dvdcopy, an error-resistant command-line DVD copier

My wife likes watching DVDs a lot, which is why she borrows many from the public library. The problem is that many of those were handled by people who have no clue whatsoever about the proper handling of DVDs, which means that they are in a state ranging from moderately scratched in the best cases to properly scarred in the worst ones... xine or VLC fail miserably to read such DVDs. What's worse is that failures happen right in the middle of DVDs, which is quite frustrating !That's why I've started quite some time ago to write a small C wrapper around libdvdread to first copy the DVD files onto the disk and then read them from there. It was also an easy solution to copy files to a driveless netbook to watch the DVD there. With time, this project evolved into a neat C++ command-line utility that's able to keep track of badly read sectors and come back to them later.Today is the 0.0 release of dvdcopy. You can get the tarball there. It follows the standard autoconf dance:
~ ./configure
~ make
~ make install
Using it is as simple as running:
~ dvdcopy /dev/dvd MyMovieDirectory
If everything went fine, you're set, and you can open the directory created with vlc or xine:
~ xine dvd:// pwd /MyMovieDirectory
(yes, xine needs the full path, I haven't checked with VLC, but from what I remember, you can open DVD directories directly from a menu).If you're not that lucky and dvdcopy found some bad sectors, try reading again using
~ dvdcopy --second-pass /dev/dvd MyMovieDirectory
I found that in many cases, one or two second passes get all the sectors back. While that wasn't really the intended purpose, the fact that most copy protection schemes (in addition to CSS) are implemented using bad sectors make dvdcopy quite resistant to them.I have no plan of making a Debian package for it at the time being, but if there is demand for that, I may reconsider. Enjoy !

Petter Reinholdtsen: Moved the pymissile Debian packaging to collab-maint

As part of my investigation on how to improve the support in Debian for hardware dongles, I dug up my old Mark and Spencer USB Rocket Launcher and updated the Debian package pymissile to make sure udev will fix the device permissions when it is plugged in. I also added a "Modaliases" header to test it in the Debian archive and hopefully make the package be proposed by jockey in Ubuntu when a user plug in his rocket launcher. In the process I moved the source to a git repository under collab-maint, to make it easier for any DD to contribute. Upstream is not very active, but the software still work for me even after five years of relative silence. The new git repository is not listed in the uploaded package yet, because I want to test the other changes a bit more before I upload the new version. If you want to check out the new version with a .desktop file included, visit the gitweb view or use "git clone git://anonscm.debian.org/collab-maint/pymissile.git".

Michal &#268;iha&#345;: Improvements in Weblate master

As there is still some development ongoing in Weblate, it's time to share information about some of the features you can expect in upcoming release. Recently, Weblate gained ability to display changes which caused string to be fuzzy. This currently only works with Gettext PO files as no other format provides this information. If this is available Weblate shows differences between current and previous string, what allows translator to easily spot what has been changed. Example can be seen on Uzbek translation of phpMyAdmin (until somebody translates that). While working on this feature, I've realized, that Django does not include this information in PO files. Fortunately it was pretty easy to fix and I've created a pull request to implement this. Looking at this file made me fix another thing which annoyed me since I've used translations in Django for first time - it generates POT file for every language, making the updates really slow. Fortunately it was also easy to fix and pull request is here. So if you feel makemessages is too slow, this is patch for you. But back to Weblate as there are more changes. For example automatic locking now works more fine grained and uses AJAX to update lock period. This means translation will be unlocked much faster if you close browser with Weblate. Also searching for messages has been updated to include substring search as well as it is sometimes more useful than fulltext search.

Filed under: Django English Weblate 0 comments Flattr this!

Hideki Yamane: 256 -> Debian

I had a cold, so couln't write anything (but back to be normal, almost).

I'm also Ubuntu LoCo team member as Ubuntu Japanese team. Some of team members write articles on gihyo.jp website during the years, and now our team leader Jun Kobayashi(*) celebrate 256 times(!) as Android Ubuntu Debian (Run Ubuntu Debian on Android)... Yes, Debian, you know :)

*) He also wrote very funny article ago, see this...

Debian Med: Bits from Debian Med team (Posted by Andreas Tille)

This are the new years bits from Debian Med team. We try to make the wishes for a healthy new year becoming true with Free Software. :-)
Debian Med Bug Squashing Advent Calendar 2012 As last year the Debian Med team did some advent calendar bug squashing. The summary states 15 bugs from Debian Med scope which is less than last year but Debian Med team members also did some general bug squashing to help the Wheezy release. Finally this makes a lot of sense also for Debian Med if we can speed up the release. When counting the additional 12 RC bugs fixed in the advent calendar effort we again managed to fix more than one bug per day. Please keep on squashing RC bugs for Wheezy!
Mentoring of Month Last year Andreas Tille started the Mentoring of Month effort to lower the entrance barrier for newcomers. There was no student for every month and not each package was uploaded at end of month but expecting such a high number of newcomers would be unrealistic anyway. So I'd like to draw some positive conclusion that we now have some newcomers friendly invited to the project and some packages uploaded or at least prepared which would not be there without the MoM project. Finally all MoM students liked the project and we will continue it in the future and would like to recommend other teams to do something similar.
Planed Debian Med sprint in Kiel The next Debian Med sprint for real life meeting of Debian Med developers and users will happen in Kiel 23rd/24th of February. All interested people are invited to join to continue the success of the past two sprints. We are currently in the planing phase but there are just some hot topics which are workflows, licensing (specifically bug #694908), cooperation with Ubuntu and BioLinux and other things.
Metapackages targeting at Wheezy Currently Debian Med metapackages version 1.13.2 are in unstable and once the unblock request is accepted by Debian release team this version will target at Wheezy. This means your very last chance to influence the metapackage content regarding some missing dependencies is defacto over now. Please closely observe our web sentinel in case you are interested in the work of Debian Med and tell us if packages are missing from our focus.

Kind regards and see you at Debian Med sprint
Andreas.

Russ Allbery: afs-monitor 2.3

Today was a travel day, coming back home from the holidays, so I'm rather stunned that I managed to do yet another software release. But there were a few minor bugs in afs-monitor that didn't take long to fix, and I talked myself into making a release rather than spending more time updating the coding style. afs-monitor is a package of check scripts for AFS servers. They're designed with Nagios in mind, but they should work with most monitoring systems that can call external checks in a similar way as Nagios. This is a bug fix release that fixes a relatively serious problem with check_afs_quotas. It has been there since the beginning, but was reported to me by three separate people in the past month; it's weird how those sorts of things come in spurts. It also teaches check_afs_bos about some standard bos output for a feature I don't use (scheduled jobs). You can get the latest version from the afs-monitor distribution page. New Debian packages (as nagios-plugins-afs) have been uploaded to my personal repository. (If anyone would find it useful to have those in Debian proper, drop me a note and I'll do that.)

Next.

Previous.