Search Results: "ron"

22 January 2013

Russ Allbery: Review: Fantasy & Science Fiction, March/April 2011

Review: Fantasy & Science Fiction, March/April 2011
Editor: Gordon van Gelder
Issue: Volume 120, No. 3 & 4
ISSN: 1095-8258
Pages: 258
Charles de Lint's book review column sticks with the sorts of things he normally reviews: urban and contemporary fantasy and young adult. Predictably, I didn't find that much of interest. But I was happy to see that not all the reviews were positive, and he talked some about how a few books didn't work. I do prefer seeing a mix of positive and negative (or at least critical) reviews. James Sallis's review column focuses entirely on Henry Kuttner and C.L. Moore (by way of reviewing a collection). I'm always happy to see this sort of review. But between that and de Lint's normal subject matter, this issue of F&SF was left without any current science fiction reviews, which was disappointing. Lucius Shepard's movie review column features stunning amounts of whining, even by Shepard's standards. The topic du jour is how indie films aren't indie enough, mixed with large amounts of cane-shaking and decrying of all popular art. I find it entertaining that the F&SF film review column regularly contains exactly the sort of analysis that one expects from literary gatekeepers who are reviewing science fiction and fantasy. Perhaps David Langford should consider adding an "As We See Others" feature to Ansible cataloging the things genre fiction fans say about popular movies. "Scatter My Ashes" by Albert E. Cowdrey: The protagonist of this story is an itinerant author who has been contracted to write a family history (for $100K, which I suspect is a bit of tongue-in-cheek wish fulfillment) and has promptly tumbled into bed with his employer. But he is somewhat serious about the writing as well, and is poking around in family archives and asking relatives about past details. There is a murder (maybe) in the family history, not to mention some supernatural connections. Those familiar with Cowdrey's writing will recognize the mix of historical drama, investigation, and the supernatural. Puzzles are, of course, untangled, not without a bit of physical danger. Experienced fantasy readers will probably guess at some of the explanation long before the protagonist does. Like most Cowdrey, it's reliably entertaining, but I found it a bit thin. (6) "A Pocketful of Faces" by Paul Di Filippo: Here's a bit of science fiction, and another mystery, this time following the basic model of a police procedural. The police in this case are enforcing laws around acceptable use of "faces" in a future world where one can clone someone's appearance from their DNA and then mount it on a programmable android. As you might expect from that setup, the possibilities are lurid, occasionally disgusting, and inclined to give the police nightmares. After some scene-setting, the story kicks in with the discovery of the face of a dead woman who, at least on the surface, no one should have any motive to clone. There were a few elements of the story that were a bit too disgusting for me, but the basic mystery plot was satisfying. I thought the ending was a let-down, however. Di Filippo tries to complicate the story and, I thought, went just a little too far, leaving motives and intent more confusing than illuminating. (6) "The Paper Menagerie" by Ken Liu: Back to fantasy, this time using a small bit of magic to illustrate the emotional conflicts and difficulties of allegiance for second-generation immigrants. Jack is the son of an American farther and a Chinese mother who was a mail-order bride. He's young at the start of the story and struggling with the embarassment and humiliation that he feels at his mother's history and the difficulties he has blending in with other kids, leading to the sort of breathtaking cruelty that comes so easily from teenagers who are too self-focused and haven't yet developed adult empathy. I found this very hard to read. The magic is beautiful, personal, and very badly damaged by the cruelty in ways that can never really be fixed. It's a sharp reminder of the importance of being open-hearted, but it's also a devastating reminder that the lesson is normally learned too late. Not the story to read if you're prone to worrying about how you might have hurt other people. (6) "The Evening and the Morning" by Sheila Finch: This long novella is about a third of the issue and is, for once, straight science fiction, a somewhat rare beast in F&SF these days. It's set in the far future, among humans who are members of the Guild of Xenolinguists and among aliens called the Venatixi, and it's about an expedition back to the long-abandoned planet of Earth. I had a lot of suspension of disbelief problems with the setup here. While Earth has mostly dropped out of memory, there's a startling lack of curiosity about its current condition among the humans. Finch plays some with transportation systems and leaves humanity largely dependent on other races to explain the failure to return to Earth, but I never quite bought it. It was necessary to set up the plot, which is an exploration story with touches of first contact set on an Earth that's become alien to the characters, but it seemed remarkably artificial to me. But, putting that aside, I did get pulled into the story. Its emotional focus is one of decline and senescence, a growing sense of futility, that's halted by exploration, mystery, and analysis. The question of what's happened on Earth is inherently interesting and engaging, and the slow movement of the story provides opportunities to build up to some eerie moments. The problem, continuing a theme for this issue, is the ending. Some of the reader's questions are answered, but most of the answers are old, well-worn paths in science fiction. The emotional arc of the story is decidedly unsatisfying, at least to me. I think I see what Finch was trying to do: there's an attempted undermining of the normal conclusion of this sort of investigation to make a broader point about how to stay engaged in the world. But it lacked triumph and catharsis for me, partly because the revelations that we get are too pedestrian for the build-up they received. It's still an interesting story, but I don't think it entirely worked. (6) "Night Gauntlet" by Walter C. DeBill, Jr., et al.: The full list of authors for this story (Walter C. DeBill, Jr., Richard Gavin, Robert M. Price, W.H. Pugmire, Jeffrey Thomas, and Don Webb) provides one with the first clue that it's gone off the rails. Collaborative storytelling, where each author tries to riff off the work of the previous author while spinning the story in a different direction, is something that I think works much better orally, particularly if you can watch facial expressions while the authors try to stump each other. In written form, it's a recipe for a poorly-told story. That's indeed what we get here. The setup is typical Cthulhu mythos stuff: a strange scientist obsessed with conspiracy theories goes insane, leaving behind an office with a map of linkages between apparently unrelated places. The characters in the story also start going insane for similar reasons, leading up to a typical confrontation with things man was not meant to know, or at least pay attention to. If you like that sort of thing, you may like this story better than I did, but I thought it was shallow and predictable. (3) "Happy Ending 2.0" by James Patrick Kelly: More fantasy, this time of the time travel variety. (I call it fantasy since there's no scientific explanation for the time travel and it plays a pure fantasy role in the story.) That's about as much as I can say without giving away the plot entirely (it's rather short). I can see what Kelly was going for, and I think he was largely successful, but I'm not sure how to react to it. The story felt like it reinforced some rather uncomfortable stereotypes about romantic relationships, and the so-called happy ending struck me as the sort of situation that was going to turn very nasty and very uncomfortable about five or ten pages past where Kelly ended the story. (5) "The Second Kalandar's Tale" by Francis Marion Soty: The main question I have about this story is one that I can't answer without doing more research than I feel like doing right now: how much of this is original to Soty and how much if it is straight from Burton's translation of One Thousand and One Nights. Burton is credited for the story, so I suspect quite a lot of this is from the original. Whether one would be better off just reading the original, or if Soty's retelling adds anything substantial, are good questions that I don't have the background to answer. Taken as a stand-alone story, it's not a bad one. It's a twisting magical adventure involving a djinn, a captive woman, some rather predictable fighting over the woman, and then a subsequent adventure involving physical transformation and a magical battle reminiscent of T.H. White. (Although I have quite likely reversed the order of inspiration if as much of this is straight from Burton as I suspect.) Gender roles, however, are kind of appalling, despite the presence of a stunningly powerful princess, due to the amount of self-sacrifice expected from every woman in the story. Personally, I don't think any of the men in the story are worth anywhere near the amount of loyalty and bravery that the women show. Still, it was reasonably entertaining throughout, in exactly the way that I would expect a One Thousand and One Nights tale to be. Whether there's any point in reading it instead of the original is a question I'll leave to others. (7) "Bodyguard" by Karl Bunker: This is probably the best science fiction of the issue. The first person protagonist is an explorer living with an alien race, partly in order to flee the post-singularity world of uploaded minds and apparent stagnation that Earth has become. It's a personal story that uses his analysis of alien mindsets (and his interaction with his assigned alien bodyguard) to flesh out his own desires, emotional background, and reactions to the world. There are some neat linguistic bits here that I quite enjoyed, although I wish they'd been developed at even more length. (The alien language is more realistic than it might sound; there are some human languages that construct sentences in a vaguely similar way.) It's a sad, elegiac story, but it grew on me. (7) "Botanical Exercises for Curious Girls" by Kali Wallace: One has to count this story as science fiction as well, although for me it had a fantasy tone because the scientific world seems to play by fantasy rules from the perspective of the protagonist. Unpacking that perspective is part of the enjoyment of the story. At the start, she seems to be a disabled girl who is being cared for by a strange succession of nurses who vary by the time of day, but as the story progresses, it becomes clear that something much stranger is going on. There are moments that capture a sense of wonder, reinforced by the persistantly curious and happy narrative voice, but both are undercut by a pervasive sense of danger and dread. This is a light story written about rather dark actions. My biggest complaint with the story is that it doesn't so much end as wander off into the sunset. It set up conflict within a claustrophobic frame, so I can understand the thematic intent of breaking free of that frame, but in the process I felt like the story walked away from all of the questions and structure that it created and ended in a place that felt less alive with potential than formless and oddly pointless. I think I wanted it to stay involved and engaged with the environment it had created. (6) "Ping" by Dixon Wragg: I probably should just skip this, since despite the table of contents billing and the full title introduction, it's not a story. It's a two-line joke. But it's such a bad two-line joke that I had to complain about it. I have no idea why F&SF bothered to reprint it. (1) "The Ifs of Time" by James Stoddard: This certainly fits with the Arabian Nights story in this issue. The timekeeper of a vast and rambling castle (think Gormenghast taken to the extreme) wanders into a story-telling session in a distant part of the castle. The reader gets to listen to four (fairly good) short stories about time, knowledge, and memory, told in four very different genres. All of this does relate to why the timekeeper is there, and the frame story is resolved by the end, but the embedded stories are the best part; each of them is interesting in a different way, and none of them outlast their welcome. This was probably the strongest story of this issue. (7) Rating: 6 out of 10

21 January 2013

Raphael Geissert: January's Debian mirrors update

It's been slightly over a month since December's update to http.debian.net. Since then, Debian's mirrors network has grown by 6 more archive mirrors. Many thanks to the Debian sponsors running them!

There are now about 370 archive mirrors serving it over http, an increase of 40 (12%) since April last year. The number of backports mirrors is now at 82, and 25 for archive.debian.org.

On the http.debian.net front there haven't been many changes since last month. Some major changes are in the works, but they didn't make it into January's code update. There were, however, a few issues with one of the hosts during the first couple of days of January. Apologies for the inconveniences it may have caused.

A new version of ftpsync addressing some issues should hopefully be released some time next month. Stay tuned to the debian-mirrors mailing list for a call for testers and probably a survey for mirror administrators.

19 January 2013

Niels Thykier: Wheezy release progress (January)

In December, I wrote a post on the progress of the Wheezy release. Back then, we had 348 RC bugs in testing and today there are about 249 left. A bit of simple math put us at 99 RC bugs fixed since last and an average of about 2.4 RC bugs fixed per day (up from 1.8). Assuming a constant rate, we would be able to release in about 100 days or 3 months (plus 1 week or two) from now. If you want the release earlier than that, fix RC bugs even faster! :) Though, the data we are looking might be inaccurate. We filed a bug against UDD, which claims that #639407 affects Wheezy while it is in fact fixed as far as the BTS is concerned. If UDD is only wrong in this direction, we can hope for a positive surprise when the UDD bug is fixed. On the other hand, we may also be unpleasently surprised if not. While talking about the UDD, I would like to mention a new feature its bug search script. It is now possible to filter based on whether or not a package is unblocked. If you take this into account (and assume that all unblocked packages will migrate in a timely manner), our RC bug count drops to about 219. It also gives us a much better view of how many packages that could actually use an unblock. which is now down to about 61. In case you have been wondering what is up with the removal of RC buggy leaf packages . We have still been looking at those, but for the last couple of times there have been at most one candidate for a given run (obviously not the same package each time). It felt a bit excessive to send an email to debian-devel for just one RC bug.

Petter Reinholdtsen: Thank you Thinkpad X41, for your long and trustworthy service

This Christmas my trusty old laptop died. It died quietly and suddenly in bed. With a quiet whimper, it went completely quiet and black. The power button was no longer able to turn it on. It was a IBM Thinkpad X41, and the best laptop I ever had. Better than both Thinkpads X30, X31, X40, X60, X61 and X61S. Far better than the Compaq I had before that. Now I need to find a replacement. To keep going during Christmas, I moved the one year old SSD disk to my old X40 where it fitted (only one I had left that could use it), but it is not a durable solution. My laptop needs are fairly modest. This is my wishlist from when I got a new one more than 10 years ago. It still holds true.:) You will notice that there are no RAM and CPU requirements in the list. The reason is simply that the specifications on laptops the last 10-15 years have been sufficient for my needs, and I have to look at other features to choose my laptop. But are there still made as robust laptops as my X41? The Thinkpad X60/X61 proved to be less robust, and Thinkpads seem to be heading in the wrong direction since Lenovo took over. But I've been told that X220 and X1 Carbon might still be useful. Perhaps I should rethink my needs, and look for a pad with an external keyboard? I'll have to check the Linux Laptops site for well-supported laptops, or perhaps just buy one preinstalled from one of the vendors listed on the Linux Pre-loaded site.

Charles Plessy: Debian-Installer in a Cloud, part 2.

I finally manage to create an image of the Debian Installer automatically, with no key or password transiting on the Cloud or the network, and without using the ec2-api-tools, which are non-Free. Thanks to Eucalyptus for implementing the attribute InstanceInitiatedShutdownBehavior in euca2ools 2.1.2. In brief, I start a micro-instance with an additional volume, and I pass it a script for cloud-init, which will download the Installer on the volume and shut down the instance. Then, I register the volume as a machine image. Those machine images of the Installer are to be preseeded via instance metadata to install Debian on another volume. I am still struggling with the pre-configuration. More details will follow in the next article. In the meantime, here are the commands I a using. On the local computer, start an instance on the Cloud. Until #696595 is fixed and cloud-init is installed by default, I use a Ubuntu image.
HELPER_AMI=ami-7609bb77 # Ubuntu 12.04 LTS Precise amd64 EBS (Asia North-East)
Start the instance with an extra volume of 1 GiB, which will persist after termination (/dev/sdb=:1:false). Pass the script debigen-install-installer-cloud (see below).
HELPER_INSTANCE=$( euca-run-instances \
        --instance-initiated-shutdown-behavior terminate \
        --instance-type t1.micro \
        --block-device-mapping /dev/sdb=:1:false \
        --user-data-file debigen-install-installer-cloud \
        $HELPER_AMI  
        tee /dev/stderr   grep INSTANCE   awk ' print $2 ')
Wait that the boot finished.
while [ ! $(euca-describe-instances $HELPER_INSTANCE   grep INSTANCE   cut -f 6   tee /dev/stderr) = "running" ]
    do sleep 30
done
Get the volume's identifier
TARGET_VOLUME=$( euca-describe-volumes  
        grep $HELPER_INSTANCE   grep '/dev/sdb'  
        tee /dev/stderr   awk ' print $2 ')
Wait that the scripts shuts down the instance.
while [ ! $(euca-describe-instances $HELPER_INSTANCE   grep INSTANCE   cut -f 6   tee /dev/stderr) = "terminated" ]
    do sleep 30
done
Snapshot the volume and register it as a machine image.
PV_KERNEL=aki-44992845 # Boot PV-Grub on hd0 (Asia North-East)
TARGET_SNAPSHOT=$( euca-create-snapshot $TARGET_VOLUME  
        tee /dev/stderr   awk ' print $2 ')
while euca-describe-snapshots $TARGET_SNAPSHOT   grep -q pending ; do sleep 30 ; done
euca-register \
    --name test_di \
    --description test_of_debian_installer \
    --snapshot $TARGET_SNAPSHOT \
    --kernel $PV_KERNEL \
    --architecture x86_64
Here is the script called debigen-install-installer-cloud, used above.
#!/bin/sh -ex    
mke2fs -L debian-installer /dev/xvdb -F
mount LABEL=debian-installer /mnt/
cd /mnt    
ARCH=amd64
DIST=squeeze
DI_VERSION=20110106+squeeze4+b2
MIRROR=jp
BASEURL=http://ftp.$MIRROR.debian.org/debian/dists/$DIST/main/installer-$ARCH/$DI_VERSION/images/netboot/xen
wget $BASEURL/initrd.gz $BASEURL/vmlinuz    
mkdir -p boot/grub    
cat > boot/grub/menu.lst <<__END__
default 0
timeout 3
title  Debian Installer ($DI_VERSION $ARCH)
root   (hd0)
kernel /vmlinuz root=LABEL=debian-installer ro console=hvc0 auto=true priority=critical url=http://169.254.169.254/latest/user-data DEBIAN_FRONTEND=text
initrd /initrd.gz
__END__
sleep 30     
halt

18 January 2013

Russell Coker: Phone Calls and Other Distractions

Harald Welte has written about the distraction of phone calls and how it impacts engineering work [1]. He asks why people feel that they are entitled to interrupt him given the cost to his work. Some years ago while working as a programmer I was discussing such things with a colleague who worked for the consulting part of the same company. He was really surprised when I told him that a phone call at the wrong time would cost me at least 30 minutes work and possibly an hour or more. His work was also quite technical and demanding but the difference between software development (where you need to think about a lot of program state to consider where the problem might be) and consulting (where you have to deal with a smaller number of configuration file options and sometimes getting debugging information to someone like me) is considerable. So the attitudes towards receiving calls also tends to be quite different. Computer work requires more concentration, thought, and knowledge of system state than many (most?) career choices. If someone finds that an unexpected phone call costs them no more than a few minutes work then it s quite reasonable of them to phone other people whenever they feel like it generally by default people think that everyone else is just like them. In terms of managing interruptions to my work, I generally encourage people to email me and that works reasonably well. So I don t have too many problems with distracting phone calls. I used Jabber for a while a few years ago but I didn t reinstall my Jabber server after it became corrupt because of the distraction. I believe that was due to using Jabber in the wrong way. I should have just started a Jabber client when I wasn t doing anything important and then killed it when I started doing some serious coding. Having a Jabber message interrupt me when I m watching a TED talk or reading blogs is no big deal. In fact I could tell everyone who has my phone number that if they see me on Jabber then they can just phone me if they wish while knowing that it won t distract me from anything serious. I wonder if I could configure a Jabber client to only receive messages when a program such as mplayer is running. I have configured my laptop and workstation to never alert me for new mail. If I m not concentrating then I ll be checking my email frequently and if I am concentrating I don t want a distraction. I have configured my phone to give one brief vibration when it gets mail and not make any sound, I will only notice that if I m not concentrating on anything. It s a standard Android feature to associate ring tones with phone numbers, it s a pity that the K9 MUA doesn t allow associating email addresses with notifications. There are some people who s email could usefully trigger an audible alert. There is an K9 feature request from 2009 to allow notifications only when the IMAP flag Flagged is set which would allow the mail server to determine which users are important, but there s no sign that it will be implemented soon. I ve started playing with Google+ recently due to Ingress team interaction being managed through it. Google+ seems quite poor in this regard, it defaults to making a ring tone for lots of different events. Turning that off is easy enough but getting notifications only about things that are important to me seems impossible. I would like to get an audible alert when someone makes a Google+ post with an Ingress code (because they expire quickly and because they only seem to be posted at times when I m not busy) but not get audible alerts about anything else. I m sure that most people who use Google+ would like to have different notifications for various types of event. But the Android client has options for whether there should be vibration and/or noise and for which events get the notifications. No options for different notifications for different events and for treating some community posts differently from others. It seems that the default settings for most programs suit people who never need to spend much time concentrating on a task. It also seems that most programs don t offer configuration options that suit the needs of people who do concentrate a lot but who also sometimes receive important phone calls and email. It s ironic that so many applications are designed in the least optimal way for the type of people who develop applications. The Google+ developers have an excuse as doing what I desire would be quite complex. But there are other programs which should deal with such things in a user friendly manner.

Russ Allbery: Review: Soulless

Review: Soulless, by Gail Carriger
Series: Parasol Protectorate #1
Publisher: Orbit
Copyright: October 2009
ISBN: 0-316-07165-X
Format: Kindle
Pages: 357
Alexia Tarabotti is a spinster in London society in the time of Queen Victoria. She is possessed of a hopelessly overwrought and society-focused mother, two half-sisters whose only thoughts are for fashion and gossip, a mixed Italian heritage, an unpopular body shape and complexion, and a secret. In a world in which vampire hives, werewolf packs, and the occasional ghost are out of hiding and openly participating in society, and where the Bureau of Unnatural Registry (headed by a particularly intimidating werewolf who is also an earl) keeps track of the undead, Alexia is something still rarer, and not entirely acceptable. She is preternatural, able to draw the supernatural out of the undead with a touch. She has no soul. In Carriger's Victorian urban fantasy, laced with a gentle helping of steampunk, vampires, werewolves, and ghosts all have similar conditions. All three are undead; even the change for werewolves happens after normal death. But the change doesn't happen to everyone killed by the corresponding type of undead. The requirements are quite exacting and include the quantity of soul in the person being changed. Only those with a surfeit of soul, too much to stay dead, can be successfully changed. Vampires and werewolves both are surrounded by hopeful humans, called drones and clavigers respectively, who hope to be transformed and gain immortality. But no one can tell in advance how much soul someone has. Alexia is the precise opposite of the undead. Rather than a surfeit of soul, she has none at all, a condition inherited from her father but unknown to the rest of her family. Her kind, the preternatural, make up the vampire and werewolf hunters of legend, called soul-suckers, curse-breakers, and exorcists by the supernatural set. She's viewed with great suspicion by most, but with fascination by some for her ability to revert them to humanity with physical contact. Not that this has been helpful in finding a husband, or, really, for much of anything else in her life. Soulless opens with Alexia being accousted by a rogue vampire in the library, where she went for refuge from a private ball. Not only does he not know who she is, despite every hive in the area telling all of their new vampires about her, but he doesn't seem to even know what a preternatural is, and she has to kill him with her parasol. That is the first unusual occurance in an escalating mystery involving new vampires appearing from nowhere, loner vampires and werewolves disappearing, and a truly prodigious use of chloroform. Not to mention quite a few encounters with Lord Maccon, Earl of Woolsey, the Alpha werewolf who runs the Bureau of Unnatural Registry. The start of Soulless feels a bit uneven and stilted until the story gets moving and the reader gets used to Carriger's voice. It's written in the style of a comedy of manners and takes delight in long words and a semi-formal style. For example:
The vampire recovered his equanimity quickly enough. He reared away from Alexia, knocking over a nearby tea trolley. Physical contact broken, his fangs reappeared. Clearly not the sharpest of prongs, he then darted forward from the neck like a serpent, diving in for another chomp. "I say!" said Alexia to the vampire. "We have not even been introduced!"
Sometimes this works. Sometimes it doesn't. The frequency with which it works seemed to me to increase as the book went along. The best thing this book has going for it is the character of Alexia Tarabotti. Rather than the typical kick-ass female protagonist of a modern urban fantasy novel, Alexia is not cynical, battle-hardened, or courageously sarcastic. Her attitude for most of the book is something closer to bemusement crossed with exasperation. She sees herself as an outsider, has largely given up on fitting into any sort of proper society, and has little use for the expectations of her family. But she's still embedded within her culture, still shocked (at least momentarily) by breaches in etiquette, and quite able to fit in, if a bit uncomfortably. Her delightfully logical tone and constant open-minded self-analysis is what made this book for me. The plot is fairly straightforward (and has a bit too much anti-science flavor for me). The romance angle, which is reasonably obvious from early on in the book, was entertaining mostly in Alexia's combination of thoughtful analysis (based in part on her father's extensive and not entirely acceptable library) and runaway physical reactions, although I thought the latter was slightly overdone. But all the elements of the story improve noticably towards the end of the book. I was unenthused by the halfway point, but by the time I finished Soulless, I decided I would probably read the sequel. This is more in the enjoyably fun reading pile than a book that I would go to great effort to acquire, and it has a somewhat rocky start, but I enjoyed it and would mildly recommend it. Followed by Changeless. Rating: 7 out of 10

17 January 2013

Craig Small: Pre-selecting ToscaWidgets jqgrid values in TurboGears

My Turbogears project has recently reached an important milestone; one of the back-end processes now runs pretty much continuously and the plugins it use (or at least the ones I can see) are also working. That means I can turn to the front-end which displays the data the back-end collected. For some of the data I am using a ToscaWidgets (or TW2) widget called a jqGridWidget which is a very nice jquery device that separates the presentation and data using a json query. I ve mentioned previously about how to get a jqGridWidget working but left the pre-filtering out until now. This meant that my grid showed all the items in the database, not just the ones I wanted to see, but it was a start. Now this widget displays things called Attributes which are basically children of another model called Hosts. Basically, Attributes are things you want to check or track about a Host. My widget used to show all Attributes but often on a Host screen, I want to see its hosts only. So, this is how I got my widget to behave; I m not sure this is THE CORRECT way of doing it, but it does work. First, in the Hosts controller you need to create the widget and pass along the host_id that the controller has obtained. I was not able to use the sub-classing trick you see in some TW2 documentation but actually make a widget instance.
class HostsController(BaseController):
    # other stuff here
    class por2(portlets.Portlet):
        title = 'Host Attributes'
        widget = AttributeGrid()
        widget.host_id = host_id
Next, the prepare() method in the Widget needs to get hold of the host_id and put it into the postData list. I needed to do it before calling super() because the options become one long string in the sub-class.
class AttributeGrid(jqgrid.jqGridWidget):
    def prepare(self, **kw):
        if self.host_id is not None:
            self.options['postData'] =  'hostid': self.host_id 
        super(AttributeGrid, self).prepare()
This means the jqgrid when it asks for its JSON data will include the hostid parameter as well. We need that method to see the host ID so we can filter the database access. Finally in the JSON method for the Attribute we pick up and filter on the hostid.
    @expose('json')
    @validate(validators= 'hostid':validators.Int() )
    def jqsumdata(self, hostid=0, page=1, rows=1, *args, **kw):
        conditions = []
        if hostid &gt; 0:
            conditions.append(model.Attribute.host_id == hostid)
        attributes =model.DBSession.query(model.Attribute).filter(and_(*conditions))
From there you run through the attributes variable and build your JSON reply.

Craig Small: Pre-selecting ToscaWidgets jqgrid values in TurboGears

My Turbogears project has recently reached an important milestone; one of the back-end processes now runs pretty much continuously and the plugins it use (or at least the ones I can see) are also working. That means I can turn to the front-end which displays the data the back-end collected. For some of the data I am using a ToscaWidgets (or TW2) widget called a jqGridWidget which is a very nice jquery device that separates the presentation and data using a json query. I ve mentioned previously about how to get a jqGridWidget working but left the pre-filtering out until now. This meant that my grid showed all the items in the database, not just the ones I wanted to see, but it was a start. Now this widget displays things called Attributes which are basically children of another model called Hosts. Basically, Attributes are things you want to check or track about a Host. My widget used to show all Attributes but often on a Host screen, I want to see its hosts only. So, this is how I got my widget to behave; I m not sure this is THE CORRECT way of doing it, but it does work. First, in the Hosts controller you need to create the widget and pass along the host_id that the controller has obtained. I was not able to use the sub-classing trick you see in some TW2 documentation but actually make a widget instance.
class HostsController(BaseController):
    # other stuff here
    class por2(portlets.Portlet):
        title = 'Host Attributes'
        widget = AttributeGrid()
        widget.host_id = host_id
Next, the prepare() method in the Widget needs to get hold of the host_id and put it into the postData list. I needed to do it before calling super() because the options become one long string in the sub-class.
class AttributeGrid(jqgrid.jqGridWidget):
    def prepare(self, **kw):
        if self.host_id is not None:
            self.options['postData'] =  'hostid': self.host_id 
        super(AttributeGrid, self).prepare()
This means the jqgrid when it asks for its JSON data will include the hostid parameter as well. We need that method to see the host ID so we can filter the database access. Finally in the JSON method for the Attribute we pick up and filter on the hostid.
    @expose('json')
    @validate(validators= 'hostid':validators.Int() )
    def jqsumdata(self, hostid=0, page=1, rows=1, *args, **kw):
        conditions = []
        if hostid &gt; 0:
            conditions.append(model.Attribute.host_id == hostid)
        attributes =model.DBSession.query(model.Attribute).filter(and_(*conditions))
From there you run through the attributes variable and build your JSON reply.

Francesca Ciceri: T-Shirt Surgery: DebConf11 t-shirt redux

Please DO TRY this at home debconf11 t-shirt before and after the surgery Here's a little tutorial on how to modify a plain t-shirt, inspired by this one. What you'll need: Step 1: the neckline cutting the neckline Put the t-shirt on a flat surface, then start cutting out the collar, to create a new neckline: cut two slits about 2 cm from the collar (from both sides), then cut out a circle. Flip the tee and do the same on the back side. Step 2: the bottom hem cutting the bottom hem Try your shirt on and mark the desired lenght. I decided to create a curved hem, to make it more feminine: to do it, you'll just need to cut an half-moon shape on the bottom line of the t-shirt. Step 3: the shoulder strap (optional) adding a shoulder strap Just to add a little twist, I decided to sew a ribbon as shoulder strap. You'll just need to measure the required lenght with the tee on, and mark the place to sew the ribbon. After sewing the strap, you can also add a little bow on the front side. Don't throw away the scraps, use them to make a fabric necklace! Some resources on t-shirt surgery:

Matthew Garrett: Experiences with depression

I've been lucky in life. I've known several people who've killed themselves, but I've been friendly with them rather than friends with them. Their deaths have saddened me, but haven't left giant gaping holes that I have trouble imagining filling. I've also been lucky in that I'm not especially predisposed towards depression, and even luckier that the one serious episode I had ended well. That luck means I'm not qualified to speak authoritatively on the topic, but this seems like a good time to share. Most of 2002 wasn't great for me. Many of my friends had graduated and moved away. My PhD, which was what I'd spent the past few years of my life working towards, was going horribly wrong. The girl I liked didn't like me. Lots of individual small things that didn't fundamentally matter, but which build up into overall feelings of isolation and failure at life. I was spending an increasing number of days not leaving bed and not talking to people. It wasn't that I couldn't have fun - socialising was still pleasurable, and I didn't actively avoid people, but it was always tempered with the knowledge that this was temporary and I'd be returning to the unhappiness afterwards. I couldn't see any sequence of events that would turn my life around and restore my happiness. This was how life was, and this was how life was going to be. I'd irrevocably fucked up, and this was my future. Looking back, what strikes me is how reasonable this seemed. I could point at specific things that were making me unhappy, and nobody could argue that it was unreasonable to be unhappy about them. There wasn't any point in speaking to people about it, because what would they be able to do except agree that I was justifiably unhappy? I thought about suicide. Not seriously, because overall life still seemed worthwhile, but I could conceive of a level of further unhappiness that would make it seem like the best choice. I don't think it would ever have occured to me to speak to someone about it first. It seemed like it would be the same argument - I'm justifiably unhappy, I already feel like I'm letting my friends down, what could they do other than tell me that my feelings are wrong or make me feel even more guilty? So, when I saw exhortions for people to speak to someone if they felt suicidal, it seemed like they were talking to people who hadn't thought this through as well as me. It felt like I'd thought this all through carefully and rationally and come to a completely logical decision. If changes in circumstances and further consideration made it seem like suicide was the better choice, that would be because it was the better choice. Maybe other people weren't thinking about this as logically as I was. Maybe they'd have their minds changed by speaking to a friend or a professional. I wouldn't. Of course, with hindsight I was rationalising the way I already felt rather than making entirely rational decisions. I could have rationalised myself to death even though there were (in my case) straightforward ways to make my life better. In any case, I've no idea how close I ever got to that point. Things were at their worst in August - by September I had a new job and new house, and things just got better from there. In the end, the friends I was convinced could do nothing for me ended up giving me the opportunity to find gainful employment and made sure I had somewhere to live. Without them, things might have been different. As it was, I spent less than nine months depressed and it was still the most hellish experience of my life. The thought of returning to that state is terrifying. I was lucky. I might not be again. There's no terribly good moral here. If I'd known more about depression beforehand, I might have been able to identify what was happening to me and seek professional help. Other than that, I didn't learn anything about how to avoid or deal with depression. The experience didn't make me a better person. I've no advice for people in the same situation. The only thing I gained from it all was the realisation that if I'd felt any worse and knew that this was what I faced for the rest of my life, death might not have been the worst choice I had. Depression is a huge social problem and we deal with it badly. We refuse to talk about it, and when we do talk about it we mostly limit ourselves to platitudes about how things will get better or placing the blame on depressed people for not wanting to talk to those around them. Sometimes it doesn't get better. Sometimes talking to those around you will make things worse. People need to be aware of what the effects of depression are and get better at identifying it in others, rather than assuming that they'll be able to ask for help themselves. Society as a whole needs to be better at ensuring that professional support is there for people who need it. And, unless we can make massive improvements in our understanding of the causes of depression and effective mechanisms for countering it, we need to accept that it will cost us friends. Let's redirect the anger we feel at their choice to avoid a lifetime of misery into anger at the society that still hasn't done everything it can to help them.

comment count unavailable comments

Russ Allbery: A few last thoughts on Aaron Swartz

Daniel Kahn Gillmor has a very good blog post. You should read it. It includes a thoughtful rebuttal to some of my earlier thoughts about activism. I think I'm developing a richer understanding of where I see boundaries here, but after my last post, I also realized that by focusing on the specific details of what should have been a minor alleged crime, I'm derailing. Swartz did so much else. I made a note to come back to the more theoretical discussion in six months; now isn't the time. Now is the time to celebrate open content and all of the things Swartz achieved. (But thank you very much to the multiple people who have pointed out flaws in my reasoning and attempted approach.) Hopefully, it's also an opportunity to keep the pressure on for a saner and less abusive judicial system that doesn't threaten people with ridiculous and disproportional punishment in order to terrify them into unwarranted plea bargains. The petition I mentioned has reached nearly 40,000 signatures and passed the threshold (at the time it was posted) for forcing a White House response. Probably more importantly, it also seems to be creating the feedback cycle that I was hoping to see: the popularity of the petition is causing this story to stay in the news cycle and continue to be written about, which in turn drives more signatures to the petition. I'm not particularly hopeful that the Obama administration cares about the vast and deep problems with our criminal justice system, but I'm somewhat more hopeful that they, like most politicians, hate news cycles that they don't control. The longer this goes on, the stronger the incentive to find some way to make it go away, which could lead to real disciplinary action. A key committee in the US House of Representatives is starting a formal investigation. One of my local representatives has proposed modifying the US federal law on computer fraud and abuse to remove violations of terms of service from the definition of the crime. (I don't have much hope that this will pass when proposed by the minority party in a fairly hostile House, but the mere act of proposing it keeps the news focus on.) Glenn Greenwald has a (typically long-winded) round-up of news in the Guardian. Note that both Greenwald and Declan McCullagh link directly to the petition in articles in mainstream news outlets. One thing that slacktivism can do is perpetuate a news cycle until it gets more uncomfortable for people in power. It's still nowhere near as effective as the types of activism that Swartz was so good at, but in this specific case I think one gets a reasonable return on one's five-minute investment of effort. I'm going to stop talking about this now, since other people are a lot better at this sort of post than I am. But one last link: the medical community has a related open content problem, and theirs is also killing people. Possibly people you know. If all of this has inspired you, as it has me, to care even more about open content, be watching the push for open access to clinical trial data. More background is in Ben Goldacre's TED talk.

Benjamin Mako Hill: 1-800-INTERNET.COM

I just returned home from Aaron Swartz s funeral in Chicago. Aaron was a good friend. The home I ve returned to is an apartment that was Aaron s before it was mine, that I have lived in with Aaron during several stints, and that I still share with many of his old books and posters. Although, I ve spent what feels like most of the last five days reading things that people have written about Aaron, I m still processing and digesting myself. Right now, I m very sad and at a loss for words. While I reflect, I wanted to share this video recently put online by Finne Boonen. The video was made in 2006 at a Web 1.0 Elevator Pitch Competition held at Wikimania 2006 about a year after that both Aaron and I moved to Cambridge and met. The goal of the contest was to pitch Web 1.0 DotCom business ideas to a team of real Web 1.0 investors like it was still 1999. Aaron and I formed a team along with SJ Klein (who I traveled to the funeral with this week), and Wikimania general counsel and interim executive director Brad Patrick. The video shows how as Danny O Brien has reminded us Aaron was funny. He came up with many our teams best lines in addition to checking our Web 1.0 boxes for tech guru and Stanford dropout. Our pitch for 1-800-INTERNET.COM is in the video below. The transcript was done by Phoebe Ayers in Facebook and the video is also available in WebM. <iframe allowfullscreen="" frameborder="0" height="315" src="http://www.youtube.com/embed/NzFjWY6Fd_g" width="420"></iframe>
SJ: You know, Mako and I had some pretty good ideas for improving connectivity to the internet, and we think we can reach 90% of the world s population. So think about this. You re sitting in a Starbucks, and you need to connect to the internet. But you can t, because there s no internet. But what is there, near every Starbucks? There s a payphone! You pick up the payphone, and you call . 1-800-INTERNET. You can connect to our bank of researchers on our fast T1 connections and get any information you need! So, we don t actually have 1-800-INTERNET yet, we have 1-800-225-3224, so the first thing we need to do is buy the number. So here s Mako, who is our web designer from UC Santa Cruz and Bradford, our financial guru, and Aaron, who s handling all of our technical implementation. But Mako, you should explain the earballs. Mako: So, so, so yeah, so most people on the Internet are going for the eyeballs, but they ve just left all of these earballs. So I have some experience in web design, and it s true that this isn t really a website, but we still need good web design. So, so, I ve actually got a really experienced team, we can go into later, and we have some really great earcons not icons, but earcons.. And it s going to be all together, not apart like some of the websites. It s going to be together. Brad: so how does this work technically? Aaron: Well, I mean, so I only spent one year at Stanford but that s Ok, because there are new developmental technologies, we re going to throw away all that old stuff, we re going to use really reliable and efficient well-designed code that everyone can clearly understand, and write the whole thing in Perl. I know this is a risk, but I am confident that Perl is going to destroy those old C websites. No one will write websites in C anymore once we do this, it s going to be so much faster, and so dynamic, everythings going to be like, on top of everything. It s going to be great. Bradford: So here s the business model. It s really really simple, and it s a really really great idea. It s all about the licensing. Because what we re going to have are these underlying audio ads, While you re on the phone you re going to hear this subliminal advertising message. And the way it works is really really cool, because it s really really low volume, it s high impact! And it s even better, because we license it, and the way it works is when a caller calls 1-800-Internet, they re hearing the ad, but so is the representative, so we get to bill em twice! So that s it: All: 1-800-INTERNET.COM
We did not win and I still believe that we were robbed.

16 January 2013

Francois Marier: Moving from Blogger to Ikiwiki and Branchable

In order to move my blog to a free-as-in-freedom platform and support the great work that Joey (of git-annex fame) and Lars (of GTD for hackers fame) have put into their service, I decided to convert my Blogger blog to Ikiwiki and host it on Branchable. While the Ikiwiki tips page points to some old instructions, they weren't particularly useful to me. Here are the steps I followed.

Exporting posts and comments from Blogger Thanks to Google letting people export their own data from their services, I was able to get a full dump (posts, comments and metadata) of my blog in Atom format. To do this, go into "Settings Other" then look under "Blog tools" for the "Export blog" link.

Converting HTML posts to Markdown Converting posts from HTML to Markdown involved a few steps:
  1. Converting the post content using a small conversion library to which I added a few hacks.
  2. Creating the file hierarchy that ikiwiki requires.
  3. Downloading images from Blogger and fixing their paths in the article text.
  4. Extracting comments and linking them to the right posts.
The Python script I wrote to do all of the above will hopefully be a good starting point for anybody wanting to migrate to Ikiwiki.

Maintaining old URLs In order to make sure I wouldn't break any existing links pointing to my blog on Blogger, I got the above Python script to output a list of Apache redirect rules and then found out that I could simply email these rules to Joey and Lars to get them added to my blog. My rules look like this:
# Tagged feeds
Redirect permanent /feeds/posts/default/-/debian http://feeding.cloud.geek.nz/tags/debian/index.rss
Redirect permanent /search/label/debian http://feeding.cloud.geek.nz/tags/debian
# Main feed (needs to come after the tagged feeds)
Redirect permanent /feeds/posts/default http://feeding.cloud.geek.nz/index.rss
# Articles
Redirect permanent /2012/12/keeping-gmail-in-separate-browser.html http://feeding.cloud.geek.nz/posts/keeping-gmail-in-separate-browser/
Redirect permanent /2012/11/prefetching-resources-to-prime-browser.html http://feeding.cloud.geek.nz/posts/prefetching-resources-to-prime-browser/

Collecting analytics Since I am no longer using Google Analytics on my blog, I decided to take advantage of the access log download feature that Joey recently added to Branchable. Every night, I download my blog's access log and then process it using awstats. Here is the cron job I use:
#!/bin/bash
BASEDIR=/home/francois/documents/branchable-logs
LOGDIR=/var/log/feedingthecloud
# Download the current access log
LANG=C LC_PAPER= ssh -oIdentityFile=$BASEDIR/branchable-logbot b-feedingthecloud@feedingthecloud.branchable.com logdump > $LOGDIR/access.log
It uses a separate SSH key I added through the Branchable control panel and outputs to a file that gets overwritten every day. Next, I installed the awstats Debian package, and configured it like this:
$ cat /etc/awstats/awstats.conf.local
SiteDomain=feedingthecloud.branchable.com
LogType=W
LogFormat=1
LogFile="/var/log/feedingthecloud/access.log"
Even if you're not interested in analytics, I recommend you keep an eye on the 404 errors for a little while after the move. This has helped me catch a critical redirection I had forgotten.

Limiting Planet feeds One of the most common things that happen right after someone migrates to a new blogging platform is the flooding of any aggregator that subscribes to their blog. The usual cause being the change in post identifiers. Unsurprisingly, Ikiwiki already had a few ways to avoid this problem. I chose to simply modify each tagged feed and limit them to the posts added after the move to Branchable.

Switching DNS Having always hosted my blog on a domain I own, all I needed to do to move over to the new platform without an outage was to change my CNAME to point to feedingthecloud.branchable.com. I've kept the Blogger blog alive and listening on feeding.cloud.geek.nz to ensure that clients using a broken DNS resolver (which caches records for longer than requested via the record's TTL) continue to see the old posts.

15 January 2013

Daniel Kahn Gillmor: in memory of Aaron Swartz

I was upset to learn about Aaron Swartz's death last week. I continue to be upset about his loss, and about our loss. He didn't just show promise of great things to come in the future -- he had already done more work for the public good than many of us will ever do. I'd only met him IRL a couple times, but (like many others) i had encountered him on the 'net in many places. He was a good person, someone i didn't need to always agree with to respect. I read Russ Allbery's posts about Aaron and "slacktivism" with much appreciation. I had been ambivalent about signing the whitehouse.gov petition asking for the removal of the prosecutor for overreach, because I generally distrust the effectiveness of online petitions (and offline petitions, for that matter). But Russ's analysis convinced me to go ahead and sign it. The petition is concrete, clear (despite wanting a grammatical proofread), and actionable. For people willing to go beyond petition signing to civil disobedience, The Aaron Swartz Memorial JSTOR Liberator is an option. It makes it straightforward to potentially violate the onerous JSTOR terms of service by re-publishing a public-domain article from JSTOR to archive.org, where it will be accessible to anyone directly. As someone who builds and maintains information/communications infrastructure, i have very mixed feelings about most online civil disobedience, since it often takes the form of a Distributed Denial of Service (DDoS) attack of some sort. DDoS attacks of public services are notoriously difficult to defend against without having huge resources to throw at the problem, so encouraging participation in a DDoS often feels a little bit like handing out cans of gasoline when you know that everyone is living in a house of straw. However, the JSTOR Liberator is not a DDoS at all -- it's simply a facilitation of people breaking the JSTOR Terms of Service (ToS), some of the same terms that Aaron was facing charges for violating. So it is a well-targeted way to demonstrate that the prosecutions were overreaching. I wanted to take issue with one of Russ' statements, though. In his second post about the situation, Russ wrote:
Social activism and political disobedience are important and often valuable things, but performing your social activism using other people's stuff is just rude. I think it can be a forgivable rudeness; people can get caught up in the moment and not realize what they're doing. But it's still rude, and it's still not the way to go about civil disobedience.
While i generally agree with Russ' thoughtful consideration of consent, I have to take issue with this elevation of some sort of hyper-extended property right over the moral agency that drives civil disobedience. To use someone else's property for the sake of a just cause without damaging the property or depriving the owner of its use is not "forgivable rudeness" -- it's forgivable, laudable even, because it is just. And the person using the property doesn't need to be "caught up in the moment and not realize what they're doing" for it to be acceptable. Civil disobedience often involves putting some level of inconvenience or discomfort on other people, including innocent people. It might be the friends and family of the activist who have to deal with the jail time; it might be the drivers stuck in a traffic jam caused by a demonstration; it might be the people forced to shop elsewhere because the store's doors are barricaded by protestors. All of these people could be troubled by the civil disobedience more than MIT's network users and admins were troubled by Aaron's protest, and that doesn't make the protests described worse or "not the way to go about civil disobedience." The trouble highlights a more significant injustice, and in its troubling way does what it can to help right it. Aaron was a troublemaker, and a good one. He will be missed. Tags: aaronsw

Russ Allbery: More on Aaron Swartz

I got some feedback that the analogy I used in my last post was a bit confusing, and indeed I blew the phrasing of the analogy (also now corrected). So let me try this again, since I think there's a subtlety here that may be missed. I should note for the record that my understanding of what Swartz did that started the process is apparently somewhat based on the description from the prosecution, so it may not be the complete or accurate facts. Since there will now be no trial, we may never find out what the defense was, and whether those facts would be challenged. So it may be best to think of this as a hypothetical. We never established, or will establish, in court exactly what happened. Swartz was, generally speaking, charged with two things that I consider quite distinct, at least from an ethical perspective. Most of the focus is on the copyright part: downloading JSTOR articles with an intent (never acted upon) to distribute them to the world. There are a bunch of reasons why this may or may not be justified, which are tied into the origin of those articles (many of them were publicly funded) and the legitimacy of copyright licensing agreements. I think there is significant room to hold a variety of opinions on this, although I don't believe that "crime worth 35 years in prison" is even remotely close to justifiable under any interpretation. However, there is a second part of what he was charged with, and that's primarily what I was commenting on, not the copyright part. He allegedly hooked up a laptop in an unlocked MIT wiring closet without permission and then used MIT's network and JSTOR license to download information. This is, to me, a subtlely but entirely distinct act from the question of whether taking JSTOR's data was ethically or legally wrong. Whether or not one believes that JSTOR's copyrights are not legitimate, it's still not okay to use someone else's network and license or to trespass in their wiring closets without permission. (I work in central IT for a university, so this strikes closer to home.) And this is where the analogy came in, which I flubbed. I had originally said that this was akin to "traipsing into someone's barn or backyard shed without their permission and taking some of their tools because you want to use them." The word "taking" was wrong; it falsely implies that you weren't going to return the tools. I had been thinking "taking and then returning" when I wrote that, and the important second part didn't make it into the post. So let's try this again: Swartz's actions at MIT, as I understand them (and many things could change this, such as a revelation that he had MIT's prior permission), are akin to going onto someone's property and into their barn without permission, borrowing their hedge trimmer for a while because you want to use it, and returning the hedge trimmer without any damage when you're done (and without them noticing it was gone). I'm quite fond of this analogy, since I think it clearly establishes two things:
  1. Most people are going to be unhappy about this happening and will intuitively feel like it should probably be illegal. Not everyone; there are folks who don't believe in personal property, or at least wouldn't extend it to tools in a barn. But most people will feel that someone should ask first before they come borrow your tools. Even if they don't damage them, even if they return them before you notice they're gone, you might have wanted to use the tool at the same time or they might have damaged them without intending to, and they should just ask first. It's common politeness, and depending on the circumstances, someone who doesn't ask and is covert about borrowing tools might be worth calling the police over.
  2. There is absolutely no way in any reasonable moral system that doing this should result in 35 years in prison. Or even 10, or even 1. Yes, most people would consider this a crime, but most people would consider it a minor crime. It's the sort of thing where you might have to impose some consequences just to make sure the message of "knock it off" is delivered firmly, but someone doing this is rude and inconsiderate, not evil.
The JSTOR copyright stuff is more complex to analyze and is more politically divisive, but for me the key points are (a) Swartz never released the data, and (b) JSTOR declined to press charges. To me, that means the deeper copyright questions, which are quite interesting, were never actually reached in this particular case. The crime that did apparently happen was the trespass at MIT, for which I think the above analogy is the right way to think about it. The point I do want people to take away from this is that one should not overlook the trespass at MIT even if one wants to celebrate the undermining of the copyright regime and doesn't believe JSTOR's data should be considered their private property. Social activism and political disobedience are important and often valuable things, but performing your social activism using other people's stuff is just rude. I think it can be a forgivable rudeness; people can get caught up in the moment and not realize what they're doing. But it's still rude, and it's still not the way to go about civil disobedience. For both ethical and tactical reasons, involving bystanders in your act of social activism without their consent is a bad approach. ETA: The problem, of course, with discussing all of this is that while it's relevant and possibly even somewhat important in the broader sense of how our community acts going forward, it also doesn't capture the fact that this was only one incident in a remarkable life. One of the worst problems with the abusive prosecution of Swartz is that it blew this incident completely out of proportion. A moment of arguable judgement should not dominate one's life or cast a shadow over all of one's other accomplishments; the prosecution tried to make it do just that. That's part of what I'm arguing, but ironically that partly feeds into the lack of proportionality in the discussion. I've gotten kind of far afield here, so let me go back and say explicitly: Swartz was a remarkable person who did much to admire and respect, and the world is a worse place without him.

14 January 2013

Russell Coker: Promoting Enthusiasm

Rusty wrote an insightful post titled What Can I Do To Help? about reactions to new ideas [1]. He suggests that people make an effort to have a positive approach when someone talks about a new idea, it s quite common for people to point out reasons why the new idea might not work out which is discouraging for the person who had the idea. I think that is a really good point. I probably haven t done too well in that regard in the past and will try to do better in future. Code Written by Assholes Rusty previously wrote a post titled If you didn t run code written by assholes, your machine wouldn t boot which implies that we should just let assholes be assholes [2]. That doesn t go well with his What Can I Do To Help? post. Note that I m not accusing Rusty of hypocrisy here, giving advice to help people who want to get along well with others is not in contradiction with refraining from giving unsolicited advice and encouragement to difficult people who have expressed no interest in improving their behavior. A comment on the latter post by Doctor Whom says If I had seen this kind of talk when I was a teenager, I would have thought twice about picking up coding , presumably given the number of people who read Rusty s blog there are some teenagers who experienced some discouragement towards a career in computers (or a hobby in FOSS) from Rusty s post. I ve already written a response to the If you didn t run code written by assholes post, among other things I suggested that people who are minor assholes should be assisted to be less difficult and major assholes should be excluded [3]. In that post I was working on the assumption that for every significant task that needs to be completed (such as making a popular OS bootable) someone will do it, if the person working on it disappears then someone else will take over there is a community of programmers who will work on whatever needs to be done. The Importance of Individuals But in terms of new ideas it really comes down to individuals. Most projects which are significant and important now probably started out as one person or a small group who had an idea that seemed unlikely to succeed at the time. So while any big and successful project can have people replaced (which is among other things a requirement of long-term success) there are situations in which individuals with ideas matter. Another important factor is that even ideas which turn out to be impractical are still useful. Someone who has an impractical idea about a technical issue and investigates it fully will learn a lot and may end up working on the less radical ways of solving similar problems this is good for the individual and the community. Another Way of Promoting Enthusiasm In terms of promoting enthusiasm it seems that one thing that can be done by high profile people is to avoid writing posts like If you didn t run code written by assholes, your machine wouldn t boot . When people in positions of power and influence appear to have no interest in promoting good behavior it really discourages people who are vulnerable to the assholes which among other things means most members of minority groups. Obviously Rusty could t stamp out all asshole behavior, but if he announced a plan to try and make things better in that regard then it would help. It s difficult to be enthusiastic when faced with discrimination from a minority and disinterest from the majority. Of course with the way the Internet works I m sure someone will say what about the assholes who have great ideas, shouldn t we nurture their enthusiasm by letting them keep doing asshole things? . I think that for the major assholes this won t be a problem, for example anyone who s racist will be well aware that many people disagree strongly with them and thus won t be particularly discouraged when they meet more people who disagree. For the minor assholes (people who don t want to be assholes) it will be somewhat discouraging to be corrected, but that could be a learning experience for them that s worth more than support in implementing their latest technical idea. Update: Why Rusty is Important In response to a comment by private mail I ve added this section after publication. Firstly I think that the opinions of all members of the community matter as they all affect the social environment which determines what types of behavior are encouraged and discouraged. But Rusty is more important than most people. Firstly Rusty has a Wikipedia page [4], that alone is an objective criteria indicating his importance. But in terms of influencing people in the FOSS community the most important things are that he s a high profile Linux kernel programmer (which alone gives significant status and influence) and that he s the founder of the first Linux conference in Australia (which is now known as Linux.conf.au AKA LCA). When issues such as the anti-harassment policy for LCA are being discussed any opinion that Rusty offered would be taken very seriously. But so far he doesn t seem to be involved in any of the public discussions.

Robert Collins: Multi-machine parallel testing of nova with testrepository

I recently added a formal interface to testrepository to enable cross-machine scaling of test runs. As testrepository is still a static scheduler, this isn t perfect, but its quite a minimal interface, which makes it easy to implement. I will likely evolve it in reaction to feedback and experience. In the long term I d love to have a super generic tool that matches that interface, so the project VCS copy of .testr.conf can just call out to it. However I don t yet have that, but I do have a simple by-hand implementation that I use to run nova s tests across my personal laptop, desktop and work laptop. Testr models this by assuming each test running process can be mapped to a single instance id (which could be a chroot, vm, cloud instances, ) and then running one or more commands in the instance, before disposing of it. This by hand implementation consists of 4 things:
  1. A tiny script to rsync my source directory to the relevant places before I run tests. (This takes <2seconds on my home wifi).
  2. A script to allocate instance ids (I just use ints)
  3. A script to discard them
  4. And a script to copy tempfiles onto the target machine and run a given command.
I do my testing in lxc containers, because I like my primary environment to be free of project-specific quirks and workarounds. lxc is not needed though, if you don t want it. So, to set this up for yourself:
  1. on each host, make an lxc container (e.g. following) http://wiki.openstack.org/DependsOnUbuntu
  2. start them all (lxc-start -n nova -d)
  3. Make SSH config entries for the lxc containers, so you can get at them remotely. (make sure your host * rules are at the end of the file otherwise the master overrides won t work [and you might not notice for some time...]):
    Host desktop-nova.lxc
    # lxc addresses may be present on localhost too, so namespace the control
    # path to avoid connecting to the wrong container.
      ControlPath ~/.ssh/master-lxc-%r@%h:%p
      hostname 10.0.3.19
      ProxyCommand ssh 192.168.1.106 nc -q0 %h %p
    Host hplaptop-nova.lxc
    # lxc addresses may be present on localhost too, so namespace the control
    # path to avoid connecting to the wrong container.
      ControlPath ~/.ssh/master-lxc-%r@%h:%p
      hostname 10.0.3.244
      ProxyCommand ssh 192.168.1.116 nc -q0 %h %p
  4. make a script to copy your nova source tree to each test location. I called mine sync
    #!/bin/bash           
    cd $(dirname $0)
    echo syncing in $(pwd) 
    (rsync -a . desktop-nova.lxc:source/openstack/nova --delete-after && echo dell done) &
    (rsync -a . hplaptop-nova.lxc:source/openstack/nova --delete-after && echo hp done)
  5. Make sure you have the base directory on each location
    ssh desktop-nova.lxc mkdir -p source/openstack
    ssh hplaptop-nova.lxc mkdir -p source/openstack
  6. Sync your code over.
    ./sync
  7. And check tests run by running a few.
    ssh hplaptop-nova.lxc "cd source/openstack/nova && ./run_tests.sh compute"
    ssh hplaptop-nova.lxc "cd source/openstack/nova && ./run_tests.sh compute"
    This will check the test environment: we re not going to be running tests on each node via run-tests or even testr (because it gets immediately meta), but if this fails, later attempts won t work. Your test virtualenv is inside the source tree, so it is copied implicitly by the sync.
  8. Decide what concurrency you want. For me, I picked 12: I have a desktop i7 with 4 cores, and two laptops with 2 cores each, and hyperthreads are on on all of them I m going to set a concurrency figure of 12 between the cores (8) and threads (16) counts, and possibly balance it more in future. A higher number assumes less contention between ALU s and other elements of the core pipeline, and I expect quite some contention because most of nova s unittests are CPU bound not I/O. If the test servers are not busy, I can always raise it later.
  9. Create scripts to create / dispose / execute logical worker threads.
  10. Creation. I call this instance-provision and all it does is find the lowest ints not currently allocated and return them.
    #!/usr/bin/env python
    import os.path
    import sys
    if not os.path.isdir('.instances'):
        os.mkdir('.instances')
    running_ids = os.listdir('.instances')
    count = int(sys.argv[1])
    top = count + len(running_ids)
    ids = [str(i) for i in range(top)]
    new = set(ids) - set(running_ids)
    for id in new:
        file('.instances/%s' % id, 'w').close()
    print(' '.join(new))
  11. Disposal is easy: remove the file marking the instance as in-use.
    #!/bin/bash
    echo freeing $@
    cd .instances
    rm $@
  12. Execution is a little trickier. We need to run some commands locally, and other ones by copying in temp files that testr has setup to the machine sshing to the remote machine, cd ing to the right directory, sourcing the virtual env, and finally running the command.
    #!/bin/bash
    instance="$(($1 % 4))"
    case $instance in
    [0]) node=
         local="true"
         ;;
    [1]) node=hplaptop-nova.lxc
         local=""
         ;;
    [2-3]) node=desktop-nova.lxc
         local=""
         ;;
    *)   echo "Unknown instance $instance" >&2
         exit 1
         ;;
    esac
    shift
    files=
    # accumulate files to copy
    while [ "--" != "$1" ]; do 
    files="$files $1"
    shift ; done 
    shift   
    if [ -n "$files" -a -z "$local" ]; then
        echo copying $files to node.
        for f in $files; do
            rsync $f $node:$(dirname $f) ;
        done
    fi  
    if [ -n "$local" ]; then
        eval $@
    else
        echo ssh to $node
        ssh $node "cd source/openstack/nova && . .venv/bin/activate && $@"
    fi
  13. Finally, tell testr how to use this. (Don t commit this change to nova, as it would break other people). Add this to your .testr.conf.
    test_run_concurrency=echo 12
    instance_provision=./instance-provision $INSTANCE_COUNT
    instance_execute=./instance-execute $INSTANCE_ID $FILES -- $COMMAND
    instance_dispose=./instance-dispose $INSTANCE_IDS
Now, when you run testr run parallel, it will run across your machines. Just do a ./sync before running tests to get the code out there. It is possible to wrap all of this up via automation (or to include just-in-time provisioned cloud instances), but I like the results of still rough scripts here it strikes a good balance between effort, reliability and performance.

13 January 2013

Russ Allbery: Slacktivism that may actually help

Normally, I'm not that much of a fan of the slacktivist trend of signing pre-canned petitions, signing things that political organizations send in the mail, and so forth. It takes very little effort and therefore carries very little weight. A lot of those efforts are more exercises in helping the signers feel better about themselves. But there is the occasional exception. Most of you probably already know about the death of Aaron Swartz. For those who aren't familiar, see Larry Lessig's article. Swartz suffered from depression (and, seriously, fuck depression it's an awful, horrible disease), but there's little doubt that the ongoing federal prosecution using the full weight of the US district attorney's office to hound him for what amounts to political trespassing was part of what led to his suicide. As it happens, I personally believe that Swartz committed a crime, and probably should have paid some consequence for it (on the order of a fine or some community service). Hooking your devices up to someone else's network without their permission and messing around in their wiring closets (locked or not) is, for me, akin to traipsing into someone's barn or backyard shed without their permission and using some of their tools because you want to use them. And whether or not one likes the current copyright regime (and I don't like it at all), MIT is still in the awkward position of having to work with it. Abusing their license for your political goals is effectively recruiting them into your activism without their permission, and as I've mentioned before, I have a mild obsession with consent. It was a crime. However, it was a minor crime, and that's where this whole situation went completely off the rails. One aspect of a justice system is fairness: uniform application of the laws to everyone. However, another aspect of a justice system is proportionality: punishments that fit the crime. Without ever having been convicted, Swartz was already punished completely out of proportion to what he actually did, both financially and emotionally, by prosecution by the US attorney that went far, far beyond zealous into actively abusive. This despite the fact that the owner of the academic papers that he was downloading as an act of open access activism stated they did not want the case to proceed and asked the US government to drop the charges. I don't think what Swartz did was right, or legal. However, the correct reaction was "look, involuntarily recruiting MIT as an accomplice to your act of civil disobedience is not okay don't ever do that again." Not "you are evil and should be locked up in prison for 35 years." And I'm completely fed up with the disproportionality of our justice system and the practice of ridiculous over-charging of crimes in an attempt to terrify people into bad plea bargains. Which brings me to the slacktivism. This is exactly the sort of situation where popular opinion matters. US attorneys who have lost the faith and support of the population they serve won't keep their jobs. And no one in government particularly wants this case to be splashed across the front pages, or to have to answer questions about the appropriateness or proportionality of the prosecution while people are mourning a dead young man. If we make it clear enough to the Obama administration that people are watching, that this matters, and that we're angry about it, not only is it quite likely there will be consequences for this prosecutor, but it may serve as a deterrant for other prosecutors in the future. There is a petition on WhiteHouse.Gov to remove the district attorney for prosecutorial overreach, and for once taking five minutes to create an account and clicking on a petition may be both useful and appropriate. Taking disciplinary action here is, unlike with a lot of petitions, something that the Obama administration can actually do, directly, without involving the rest of the dysfunctional US government, and without making new law. If you are a US citizen, please consider going to this site and signing it to say that this matters to you and you believe this prosecution was excessive and inappropriate. Please note: you do not have to believe that Swartz died solely or even primarily because of this prosecution to do this. We'll never really know the complex factors behind his death. But this was a burden that he shouldn't have had to deal with. Please also note that you do not have to think he was justified in his behavior to sign this petition. This is not a question of whether what he did should be legal or was ethical. Rather, even assuming it was illegal, it's a question of appropriate punishment and proportionality of response. Whether or not you agree with his political cause, there simply was not enough damage done, to anyone, to warrant this kind of aggressive prosecution. And the direct victims agreed, which was the point at which the district attorney should have scaled way back on their actions or dropped the matter entirely. Not doing so was an abuse of office and position, and that should have consequences. ETA: Corrected "taking" to "using" in the analogy about tools. I was actually thinking "taking and then returning" when I wrote that, but only the first word made it into the post, and ended up creating a confusing parallel with theft that wasn't intended.

Thorsten Glaser: PSA: Unicode codepoints, referring to

PSA: Referring to Unicode codepoints. If your Unicode codepoint is, numerically, between 0 and 65533, inclusive, convert it to hexadecimal and zero-pad it to four nibbles. For example, the Euro sign is Unicode codepoint #8364 which is 20AC hex; the Eszett is 223 which is DF hex, padded 00DF.
Then write an uppercase U , a plus sign + , and the four nibbles: U+20AC U+00DF
In mksh, JSON, etc. it s a backslash \ , a lower-case u and four nibbles. Otherwise, your Unicode codepoint will be, numerically, between 65536 and 1114111, inclusive, that is hex 10000 to 10FFFF. (There s nothing on 65534 and 65535, nor above these figures.) In this case, convert it to hex, zero-pad it to eight nibbles and write it as an uppercase U , a hyphen-minus - and the eight nibbles. In C-like escapes for environments supporting the Unicode SMP, that s a backslash \ , an upper-case U and eight nibbles. Do not, in either case, use less (or more) hex digits than specified here. For example, there s a famous Unicode codepoint U-0001F4A9 PILE OF POO . That s not the same as U+1F4A9. The latter reads as U+1F4A GREEK CAPITAL LETTER OMICRON WITH PSILI AND VARIA and a digit 9 ( 9). Be educated.

Next.

Previous.