Search Results: "spa"

3 December 2012

Pablo Lorenzzoni: Nostalgia time

My parents will soon be moving to a smaller home, so they are digging up a lot of stuff of my sister and mine past. Among my stuff, they just sent me my first computer (which was, of course, the first computer of my father s company I was using in the spare time). I couldn t believe they kept that. It was an Unitron Apple ][ 64K!! I just had it cleaned and took this picture: Due to the closed informatics market Brazilians were subject to at the time, it came with a full set of manuals in Portuguese which taught me how to code in Basic (I was too young to learn English at the time) Interesting how a bad policy like that can result in a Good Thing sometimes. :-) Is it just me or does this picture made you nostalgic also?

Joey Hess: hledger

Apologies in advance for writing a long blog post about the dull and specialised subject of double-entry accounting from the Unix tools perspective, that ends up talking about Monads to boot. I can't believe I'm about to write such a thing, but I'm finding it an oddly interesting subject. double-entry accounting I've known of, though probably not really understood double entry accounting for a while, thanks to GnuCash. I think GnuCash did something rather special in making such a subject approachable to the layman, and I've been happy to recommend GnuCash to friends. I was stoked to find a chapter in my sister Anna's new book that happily and plausibly suggests readers use GnuCash. But for my personal use, GnuCash is clunky. I mean, I'm a programmer, but I can't bring any of my abilities to bear on it, without a deep dive into the code. The file format is opaque (and a real pain to keep checked into git with all the log files); the user interface is often confusing, but there's no benefit to its learning curve, it never seems to get better than manually entering data into its ledger, or perhaps importing data from a bank. I've never found the reports very useful. I've got perhaps a year of data in GnuCash, but it's fragmented and incomplete and not something I've been able to motivate myself to keep up with. So I have less financial data than I'd like. I'm hoping ledger will change that. ledger I've known about ledger for a while, at least since This Linux Weekly News article. It's a quintessential Unix tool, that simply processes text files. The genius of it is the simplicity of the file format, that gets the essence and full power of double entry bookeeping down to something that approaches a meme. Once you get the file format stuck in your head, you're done for.
2004/05/27 Book Store
      Expenses:Books                 $20.00
      Liabilities:Visa
starting to use hledger Now as a Haskell guy, I was immediately drawn to the Haskell clone, hledger. It's nice that there are two (mostly) compatable implementations of ledger too. So from here on, I'll be talking about hledger. I sat down and built a hledger setup for myself the other day. I started by converting the GnuCash books I have been keeping up-to-date, for a small side business (a rental property). It quickly turned into something like programming, in the best way, as I used features like: a modern unix program While I've been diving into hledger, I've been drawing all kinds of parallels between it and other modern Unix-friendly programs I use lately. I think we've gone over a bit of a watershed recently. Unix tools used to be either very simple and crude (though often quite useful), or really baroque and complex with far too many options (often due to 10-30 years of evolution). Or they were a graphical user interface, like GnuCash, and completely divorced from Unix traditions. The new unix programs have some commonalities... importing data After manually converting my GnuCash data, I imported all my PayPal history into hledger. And happily it calculates the same balance Paypal does. It also tells me I've paid PayPal $180 in transaction fees over the years, which is something PayPal certianly doesn't tell you on their website. (However, my current import from PayPal's CSV files is a hackish, and only handles USD currency, so I miss some currency conversion fees.) I also imported my Amazon Payments history, which includes all the Kickstarter transactions. I almost got this to balance, but hledger and Amazon disagree about how many hundreths of a cent remain in my account. Still, pretty good, and I know how much I paid Amazon in fees for my kickstarter, and how much was kicked back to Kickstarter as well. (Look for a cost breakdown in some future blog post.) At this point, hledger stats says I have 3700 transactions on file, which is not bad for what was just a few hours work yesterday. One problem is hledger's gotten a little slow with this many transactions. It takes 5 seconds to get a balance. The ledger program, written in C, is reportedly much faster. hledger recently had a O(n^2) slowdown fixed, which makes me think it's probably only starting to be optimised. With Haskell code, you can get lucky and get near C (language, not speed of light) performace without doing much, or less lucky and get not much better than python performance until you dive into optimising. So there's hope. harnessing haskell If there's one place hledger misses out on being a state of the art modern Unix program, it's in the rules files that are used to drive CSV imports. I found these really hard to use; the manual notes that "rules file parse errors are not the greatest"; and it's just really inflexible. I think the state of the art would be to use a Domain Specific Language here. For both my Amazon and PayPal imports I had CVS data something like:
date, description, amount, fees, gross
I want to take the feeds into account, and make a split transaction, like this:
date description
    assets:accounts:PayPal             $9.90
    expenses:transaction fees:PayPal   $0.10
    income:misc:PayPal                 -$10.00
This does not seem possible with the rules file. I also wanted to combine multiple CVS lines, to do with currency conversions, into a single transaction, and couldn't. The problem is that the rules file is an ad-hoc format, not a fully programmable one. If instead, hledger's rules files were compiled into standalone haskell programs that performed the import, arbitrarily complicated conversions like the above could be done. So, I'm thinking about something like this for a DSL.. I doubt I'll get much further than this, since I have a hacked together multi-pass importer that meets my basic needs. Still, this would be nice, and being able to think about adding thing kind of thing to a program cleanly is one of the reasons I reach for the Haskell version when possible these days. First, here's part of one of my two paypal import rules files (the other one extracts the transaction fees):
amount-field 7
date-field 0
description-field %(3) - %(4) - %(11)
base-account assets:accounts:PayPal
Bank Account
assets:accounts:checking
.*
expenses:misc:PayPal
That fills out the basic fields, and makes things with "Bank Account" in their description be categorised as bank transfers. Here's how it'd look as Haskell, carefully written to avoid the $ operator that's more than a little confusing in this context. :)
main :: IO ()
main = convert paypalConveter
paypalConverter :: [CSVLine] -> [Maybe Transaction]
paypalConverter = map convert
  where
    convert = do
        setAmount =<< field 7
        setDate =<< field 0
        setDescription =<< combine " - " [field 3, field 4, field 11]
        defaultAccounts
            "assets:accounts:PayPal" ==> "expenses:misc:PayPal"
        inDescription "Bank Account" ?
            "assets:accounts:PayPal" ==> "assets:accounts:checking"
That seems like something non-haskell people could get their heads around, especially if they didn't need to write the boilerplate function definitions and types at the top. But I may be biased. :) Internally, this seems to be using a combination Reader and Writer monad that can get at fields from a CSV line and build up a Transaction. But I really just made up a simple DSL as I went along and thew in enough syntax to make it seem practical to implement. :) Of course, a Haskell programmer can skip the monads entirely, or use others they prefer. And could do arbitrarily complicated stuff during imports, including building split transactions, and combining together multiple related CVS lines into a single transaction.

Erich Schubert: ResearchGate Spam

Update Dec 2012: ResearchGate still keeps on sending me their spam. Most of the colleagues I had that tried out RG now deleted their account there, apparently, so the invitation mails become fewer. Please do not try to push this link on Wikipedia just because you are also annoyed by their emails. My blog is not a "reliable source" by Wikipedia standards. It solely reflects my personal view of that web site, not journalistic or scientific research. The reason why I call ResearchGate spam is the weasel words they use to trick authors into sending the invitation spam. Here's the text coming with the checkbox you need to uncheck (from the ResearchGate "blog")
Add my co-authors that are already using ResearchGate as contacts and invite those who are not yet members.
See how it is worded so it sounds much more like "link my colleagues that are already on researchgate" instead of "send invitation emails to my colleagues"? It deliberately avoids the mentioning of "email", too. And according to the researchgate news post, this is hidden in "Edit Settings", too (I never bothered to try it -- I do not see any benefit to me in their offers, so why should I?). Original post below:
If you are in science, you probably already received a couple of copies of the ResearchGate spam. They are trying to build a "Facebook for scienctists", and so far, their main strategy seems to be aggressive inivitation spam. So far, I've received around 5 of their "inivitations", which essentially sound like "Claim your papers now!" (without actually getting any benefit). When I asked my colleagues about these invitations none actually meant to invite me! This is why I consider this behaviour of ResearchGate to be spam. Plus, at least one of these messages was a reminder, not triggered by user interaction. Right now, they claim to have 1.9 million users. They also claim "20% interact at least once a month". However, they have around 4000 Twitter followers and Facebook fans, and their top topics on their network are at like 10000-50000 users. That is probably a much more real user count estimation: 4k-40k. And these "20%" that interact, might just be those 20% the site grew in this timeframe and that happened to click on the sign up link. For a "social networking" site, these numbers are pointless anyway. That is probably even less than MySpace. Because I do not see any benefit in their offers! Before going on an extremely aggressive marketing campaign like this, they really should consider to actually have something to offer... And the science community is a lot about not wasting their time. It is a dangerous game that ResearchGate is playing here. It may appeal to their techies and investors to artificially inflate their user numbers in the millions. But if you pay for the user numbers with your reputation, that is a bad deal! Once you have the reputation as being a spammer (and mind it, every scientist I've talked to so far complained about the spam and "I clicked on it only to make it stop sending me emails") it's hard to be taken serious again. The scientific community is a lot about reputation, and ResearchGate is screwing up badly on this. In particular, according to researchgate founder on quora, the invitations are opt-out on "claiming" a paper. Sorry, this is wrong. Don't make users annoy other users by sending them unwanted invitations to a worthless service! And after all, there are alternatives such as Academia and Mendeley that do offer much more benefit. (I do not use these either, though. In my opinion, they also do not offer enough benefit to bother going to their website. I've mentioned the inaccuracy of Mendeleys data - and the lack of an option to get them corrected - before in an earlier blog post. Don't rely on Mendeley as citation manager! Their citation data is unreviewed. I'm considering to send ResearchGate (they're Berlin based, but there maybe also is a US office you could direct this to) a cease and desist letter, denying them to store personal information on me, and to use my name on their websites to promote their "services". They may have visions of a more connected and more collaborative science, but they actually don't have new solutions. You can't solve everything by creating yet another web forum and "web2.0izing" everything. Although many of the web 2.0 bubble boys don't want to hear it: you won't solve world hunger and AIDS by doing another website. And there is a life outside the web.

2 December 2012

Dirk Eddelbuettel: RQuantLib 0.3.9

A minor feature release RQuantLib 0.3.9 is now on CRAN and in Debian. RQuantLib combines (some of) the quantitative analytics of QuantLib with the R statistical computing environment and language. Bryan Lewis had suggested to enable another pricing engine for American Options in order to get (at least some) Greeks. This is now supported by picking engine="CrankNicolson" as shown in the default example for the AmericanOption function:
R> library(RQuantLib)
R> example(AmericanOption)
AmrcnOR> # simple call with unnamed parameters
AmrcnOR> AmericanOption("call", 100, 100, 0.02, 0.03, 0.5, 0.4)
Concise summary of valuation for AmericanOption 
  value   delta   gamma    vega   theta     rho  divRho 
11.3648      NA      NA      NA      NA      NA      NA 
AmrcnOR> # simple call with some explicit parameters
AmrcnOR> AmericanOption("put", strike=100, volatility=0.4, 100, 0.02, 0.03, 0.5)
Concise summary of valuation for AmericanOption 
  value   delta   gamma    vega   theta     rho  divRho 
10.9174      NA      NA      NA      NA      NA      NA 
AmrcnOR> # simple call with unnamed parameters, using Crank-Nicolons
AmrcnOR> AmericanOption("put", strike=100, volatility=0.4, 100, 0.02, 0.03, 0.5, engine="CrankNicolson")
Concise summary of valuation for AmericanOption 
  value   delta   gamma    vega   theta     rho  divRho 
10.9173 -0.4358  0.0140      NA      NA      NA      NA 
R> 
Thanks to CRANberries, there is also a diff to the previous release 0.3.8. Full changelog details, examples and more details about this package are at my RQuantLib page.

28 November 2012

Russ Allbery: Review: Emissaries from the Dead

Review: Emissaries from the Dead, by Adam-Troy Castro
Series: Andrea Cort #1
Publisher: Eos
Copyright: March 2008
ISBN: 0-06-144372-7
Format: Mass market
Pages: 387
As a small child, Andrea Cort was a colonist on Bocai, part of a utopian project to create a mixed human and Bocaian community. Until, that is, all the members of the community woke up one day and started horrifically murdering each other for reasons that none of the survivors could ever explain. Now, she's protected from extradition by a life contract with the Diplomatic Corps and serves them as a special investigator. The scene of the investigation of this book is the strange habitat One One One. The habitat was created by the AIsource, an alliance of artificial intelligences now independent of their original creators. It's in deep space, far from any sun, and would have remained entirely unknown to the other inhabitants of the galaxy if the AIsource hadn't announced its presence. The choice to announce it was baffling, since it contains a sentient species entirely under the control of the AIsource, something that's against the shared rules of all the other species, but which they wouldn't have known about if the AIsource hadn't drawn their attention to it. After extensive negotiations, one unofficial human research group has been allowed access, and now two of the members of that group have been murdered. That's where Andrea comes in. This is Castro's first original solo novel, but he has a background in short fiction in the SF, fantasy, and horror genres and some previous collaborations and shared universe writing. The horror shows through a bit more than I'd personally prefer: the preface briefly describing Andrea's past would have been sufficient, but Castro goes back and shows it in horrific and gruesome detail as a flashback. His fight scenes are also strangely and disturbingly detailed. If you have a sensitive stomach, be aware that you may want to skim some parts of this. The overall construction of this novel, though, is not horror. It's an SF murder mystery, about halfway between a procedural and a noir detective story. Andrea does follow a coherent procedure and uses an investigative technique more sophisticated than stirring the pot and seeing what jumps out, making it not quite noir, but it has the noir tendency towards story progress via attacks on the detective and quite a lot of ongoing tension. It also involves quite a lot of setup; be aware going in that we get over a hundred pages of description of the various research group members, the details of the crimes, possible motives, and the details of the artificial habitat without a lot of forward progress on understanding the crime. Castro puts a lot of work into the design of the habitat despite leaving it mostly in the background. It could easily serve as the backdrop of a more typical Big Dumb Object SF novel. The habitable zone where the research team is and where all of Andrea's investigations take place is the effective "top" of the habitat, covered in dense foilage to which the native sentient species clings and crawls its entire life. Below is an effectively infinite fall into crushing acidic atmosphere. Castro managed to not set off my dislike of heights too badly, in part because he focuses more on Andrea's own dislike of heights rather than vividly describing the drop, but it's a constant presence throughout the novel and adds a persistant background tension. It also leads to some minor but neat clashes of perspective in conversations with the native species. For most of the book, I would have described Emissaries from the Dead as an enjoyable but not horribly original detective novel with a neat science fiction background. But there are several interesting ending twists that I won't spoil and some decisions that quite surprised me. They don't have a large effect on the plot of this book, but they make for a fascinating hook and setup for later books in the series (and I see there is a sequel). It's definitely not the sort of thing that you want spoiled before reading this book, so avoid reading blurbs and details about the sequel before reading this one. I wouldn't go out of my way to hunt down this book, but it's solid, enjoyable detective SF with a surprisingly good ending. I can see why it won the Philip K. Dick award (for best SF paperback original). Mildly recommended. Followed by The Third Claw of God. Rating: 7 out of 10

26 November 2012

John Sullivan: Santiago de Compostela

I've put up my nearly unedited, unsorted, and uncommented photos from my recent trip to Santiago de Compostela, Spain, for the Libre Software World Conference. It was a beautiful place (and a great conference) -- I hope to write more about it soon.

Steve Langasek: Upstart in Debian

Good news, everyone! So as of last Sunday, this works on all Linux archs in Debian unstable and gives you a modern version of upstart:
echo 'Yes, do as I say!'   apt-get -o DPkg::options=--force-remove-essential -y --force-yes install upstart
Thanks to the ifupdown, sysvinit, and udev maintainers for their cooperation in getting upstart support in place; to the Debian release team for accomodating the late changes needed for upstart to be supported in wheezy; and to Scott for his past maintenance of upstart in Debian. Benchmarking One of the consequences is that it's now possible to do meaningful head-to-head comparisons of boot speed between sysvinit (with startpar), upstart, and systemd. At one time or another people have tested systemd vs. sysvinit when using bash as /bin/sh, and upstart vs. sysvinit, and systemd vs. sysvinit+startpar, and there are plenty of bootcharts floating around showing results of one init system or another on one distro or another, but I'm not aware of anyone having done a real, fair comparison of the three solutions, changing nothing but the init system. I've done some initial comparisons in a barebones sid VM, and the results are definitely interesting. Sysvinit with startpar (the default in Debian) can boot a stock sid install, with no added services, in somewhere between 3.37 and 3.42 seconds (three runs). That's not a whole lot, but on the other hand this is a system with a single filesystem and no interesting services yet. Is this really as fast as we can boot? No, even this minimal system can boot faster. Testing with upstart shows that upstart can do the same job in between 3.03 and 3.19 seconds (n=3, mean=3.09). This confirms what we'd already seen in Ubuntu, that it makes a difference to boot speed to have filesystem mounting handled by an integrated process that understands the whole system instead of as a group of serialized shell scripts. What about systemd? The same test gives a boot time between 2.32 and 2.85 seconds (n=4, mean=2.48). Interesting; what would make systemd faster than upstart in sid? Well, a quick look at the system shows one possible contributing factor: the rsyslog package in Debian has a systemd unit file, but not an upstart job file. Dropping in the /etc/init/rsyslog.conf from Ubuntu has a noticeable impact, and brings the upstart boot time down nearer to that of systemd (2.78-3.03s, n=5, mean=2.92). Besides telling me that it's time to start spamming Debian maintainers with wishlist bugs asking them to include upstart jobs in their packages, this suggests that the remaining difference in boot time may be due to the outstanding init scripts in rcS.d that are made built-in no-ops by systemd but not (yet) by upstart in Debian (e.g., hwclock, hostname, udev-mtab). (In Ubuntu, /etc/rcS.d has long since been emptied out in favor of upstart jobs in the common case, since the time it takes to get to runlevel=2 is definitely a major issue for boot speed and boot parallelization.) It also gives the lie to the claim that's been made in various places that spawning shells is a major bottleneck for upstart vs. systemd. More study is certainly needed to confirm this, but at least this naive first test suggests that in spite of the purported benefits of hard-coding boot-time policies in C code, upstart with its default degree of runtime configurability is at least in the ballpark of systemd. Indeed, when OpenSuSE switched from upstart to systemd, it seems that something else in the stack managed to nullify any benefit from improving the boot-time performance of apparmor. Contrary to what some would have you believe, systemd is not some kind of silver bullet for boot speed. Upstart, with its boot-time flexibility and its long history of real-world testing in Ubuntu, is a formidable competitor to systemd in the boot speed department - and a solid solution to the many longstanding boot-time ordering bugs in Debian, which still affect users of sysvinit. I've published the bootcharts for the above tests here. Between the fact that Debian's bootchart package logs by default to /var/log/bootchart.tgz (thus overwriting on every boot) and the fact that these tests are in a VM, I haven't bothered to include the raw data, just the bootcharts themselves. The interested reader can probably generate more interesting boot charts of their own anyway - in particular, it will be interesting to see how the different init systems perform with more complicated filesystem layouts, or when booting a less trivial set of services. Other musings The boot charts have been created with the bootchart package rather than with bootchart2. For one thing, it turns out that bootchart2 includes systemd units, not init scripts; so when replacing bootchart with bootchart2, the non-matching init script is left behind and systemd in particular gets terribly confused. This is now reported as Bug #694403. In an amusing twist, while I was experimenting with bootchart2, I also noticed that having systemd installed would slow down booting with other init systems, because systemd installs udev rules which take a noticeable amount of time to run a helper command at boot even though the helper should be a no-op. So if you're doing boot speed testing of other init systems, be sure you don't have systemd on the system at the time!

Russell Coker: Links November 2012

Julian Treasure gave an informative TED talk about The 4 Ways Sound Affects US [1]. Among other things he claims that open plan offices reduce productivity by 66%! He suggests that people who work in such offices wear headphones and play bird-songs. Naked Capitalism has an interesting interview between John Cusack and Jonathan Turley about how the US government policy of killing US citizens without trial demonstrates the failure of their political system [2]. Washington s blog has an interesting article on the economy in Iceland [3]. Allowing the insolvent banks to go bankrupt was the best thing that they have ever done for their economy. Clay Shirky wrote an insightful article about the social environment of mailing lists and ways to limit flame-wars [4]. ZRep is an interesting program that mirrors ZFS filesystems via regular snapshots and send/recv operations [5]. It seems that it could offer similar benefits to DRBD but at the file level and with greater reliability. James Lockyer gave a movingTEDx talk about his work in providing a legal defence for the wrongly convicted [6]. This has included overturning convictions after as much as half a century in which the falsely accused had already served a life sentence. Nathan Myers wrote an epic polemic about US government policy since 9-11 [7]. It s good to see that some Americans realise it s wrong. There is an insightful TED blog post about TED Fellow Salvatore Iaconesi who has brain cancer [8]. Apparently he had some problems with medical records in proprietary formats which made it difficult to get experts to properly assess his condition. Open document standards can be a matter of life and death and should be mandated by federal law. Paul Wayper wrote an interesting and amusing post about Emotional Computing which compares the strategies of Apple, MS, and the FOSS community among other things [9]. Kevin Allocca of Youtube gave an insightful TED talk about why videos go viral [10]. Jason Fried gave an interesting TED talk Why Work Doesn t Happen at Work [11]. His main issues are distraction and wasted time in meetings. He gives some good ideas for how to improve productivity. But they can also be used for sabotage. If someone doesn t like their employer then they could call for meetings, incite managers to call meetings, and book meetings so that they don t follow each other and thus waste more of the day (EG meetings at 1PM and 3PM instead of having the second meeting when the first finishes). Shyam Sankar gave an interesting TED talk about human computer cooperation [12]. He describes the success of human-computer partnerships in winning chess tournaments, protein folding, and other computational challenges. It seems that the limit for many types of computation will be the ability to get people and computers to work together efficiently. Cory Doctorow wrote an interesting and amusing article for Locus Magazine about some of the failings of modern sci-fi movies [13]. He is mainly concerned with pointless movies that get the science and technology aspects wrong and the way that the blockbuster budget process drives the development of such movies. Of course there are many other things wrong with sci-fi movies such as the fact that most of them are totally implausible (EG aliens who look like humans). The TED blog has an interesting interview with Catarina Mota about hacker spaces and open hardware [14]. Sociological Images has an interesting article about sporting behaviour [15]. They link to a very funny youtube video of a US high school football team who make the other team believe that they aren t playing until they win [16] Related posts:
  1. Links April 2012 Karen Tse gave an interesting TED talk about how to...
  2. Links March 2012 Washington s Blog has an informative summary of recent articles about...
  3. Links November 2011 Forbes has an interesting article about crowd-sourcing by criminals and...

25 November 2012

Pietro Abate: Mini Debian Conf 2012 in Paris : Bootstrapping Debian for a new architecture

I just finished to address the awesome debian crowd at the Mini Deb conf in paris. My presentation was about a few challenges we have ahead to bootstrap debian on a new architecture. Johannes Schauer and Wookey did a lot of work in the last few months particularly focusing on Linaro/Ubuntu. After Wheezy I think it is important to catch up with their work and integrate it in debian. The two main take away messages from my presentation : This week we just reached an important milestone toward a fully automatic bootstrap procedure. Hopefully we are going to tell you more about this work during fosdem 2013 My slides are attached.
AttachmentSize
debminiconf2012.pdf271.56 KB

24 November 2012

Russ Allbery: Review: Dune

Review: Dune, by Frank Herbert
Series: Dune #1
Publisher: Ace
Copyright: 1965
Printing: September 1990
ISBN: 0-441-17271-7
Format: Mass market
Pages: 537
The Atreides family, distant cousins to the imperial family, have ruled the planet of Caladan for twenty generations. Caladan is a wet farming world, comfortable and pleasant, but not horribly important. But House Atreides is feuding with House Harkonnen, and, at the start of Dune, that feud maneuvers Duke Leto into giving up his holdings and moving his family to take possession of Arrakis. Arrakis is a desert planet, previously controlled by Baron Harkonnen. It is unrelentingly hostile, home to smugglers and dangerous local desert dwellers called Fremen. But it's also one of the most important planets in the galaxy, since it's the sole origin of the chemical called melange, or spice. Spice permits a limited form of prescience, which allows the navigators of the Spacing Guild to successfully steer ships across the interstellar void. Arrakis's production of spice is what makes interstellar travel, and therefore all of interstellar civilization, possible. Dune is the story of Paul Atreides, son and heir to Duke Leto Atreides. His mother, Lady Jessica, is one of the Bene Gesserit, a secretive order of women devoted to mental and physical discipline and to the long-term genetic improvement of mankind. He is not supposed to exist; Lady Jessica was supposed to only bear a daughter of Leto. But he may be something special, the long-sought (but also dangerous) Kwisatz Haderach who can unite male and female Bene Gesserit powers. The Bene Gesserit take great interest in him from the start of Dune. More surprisingly, so do the Fremen of Arrakis; from the moment he arrives there, he seems to be fulfilling prophecies of theirs that are partly, but not entirely, ones planted by the Bene Gesserit long ago. The feud with the Harkonnens, the unstable place of Arrakis in galactic politics, the dreams of the Fremen and the Imperial ecologist on Arrakis of terraforming, Bene Gesserit plans, Paul's abilities, and the legends of the Fremen all combine in a complex mix of politics, battle, and clashes of culture. Dune is an acknowledged SF masterpiece, one of the best-known classics of the genre. It's usually found in short lists of the best SF novels ever written. It spawned five sequels by Frank Herbert (about which more in a moment), as well as numerous additional sequels and prequels by Kevin J. Anderson and Brian Herbert. It's been adopted for the screen twice, not to mention board games, video games, and numerous other projects. This is my second reading, the first in about twenty years, but the story was still immediately familiar from having seen films and having discussed and read about the universe. This is not science fiction in any strict sense. Dune is science fiction in the same way that Star Wars is: a futuristic gloss on top of power structures inspired by feudalism, heavily mixed with mysticism, mental powers, magic, and implausible but convenient science that creates the story effects the author wants. Both Bene Gesserit powers in general and Paul's abilities in particular are effectively magic. There is some hand-waving explanation of their ability to verbally control other people as taking advantage of specific pitches and intonations that people are vulnerable to, but it's effectively spell-casting (and is a direct inspiration for Jedi mind tricks). All of the mysticism (and there's quite a lot of it in Dune, including race memory, precognition, and even molecular transformation) resembles the Force from Star Wars more than anything scientific. Dune is epic fantasy told on a science fiction stage, complete with a young protagonist coming into his powers and dangerous and sometimes hostile mentors. What Dune gets right, and what has put it so high in the pantheon of great science fiction, is the world building. Herbert sets the story tens of thousands of years into the future of humanity and then effectively projects the feeling of deep history over everything in the novel. This is the kind of book that has appendices with more background information; more to the point, it's the kind of book where you may actually read them out of curiosity. Mankind has a vast interstellar empire (Herbert's universe, like Asimov's Foundation universe, admits no aliens) governed by a system akin to the early British monarchy. An emperor rules in balance with the Great Houses, who meet in a sort of parliament. But against both is a third force: the Spacing Guild, who maintains a monopoly over all interstellar travel. (And the Bene Gesserit form an underground, secretive fourth power base.) Herbert plays with vast swaths of time and great forces of history as well as very good epic fantasy and better than nearly all SF I've read. The detailed world-building is equally good. Nearly all of Dune takes place on the desert planet of Arrakis, which has a lovingly-described ecology and local culture built entirely around scarcity of water. (The details of that ecology are much of the plot and mystery of the book, so I won't spoil them further.) While I doubt the precise details hold up to close scientific scrutiny, this is an obvious precursor to the great ecological stories of later SF, such as Kim Stanley Robinson's Mars trilogy. The details all feel right and hang together in satisfying ways, while also generating the great Sand Worms of Arrakis, a key ingredient in several of the best set pieces in the history of SF. This is the sort of book where the fascinating details and discoveries about the world do as much to keep one turning the pages as the plot, although the plot is also satisfyingly twisty and tense. Unfortunately, Dune doesn't get everything right. The amount of mysticism involved is a bit much, and at times the drug-trip mystical experiences of viewpoint characters turn into excessively purple prose and nearly incomprehensible descriptions. Those mystical experiences also involve race and genetic memory, a concept that's just scientific enough to be unbelievable. A few of the other scentific cheats are also rather blantant; for example, Herbert constructs an elaborate, artificial technology of shielding that seems designed primarily as an excuse to add sword combat to a futuristic story, and I have always struggled to suspend disbelief about the way lasers and shields interact in Dune. The Spacer Guild's monopoly on interstellar travel can be explained; their monopoly on local orbital space, or even the high stratosphere, both vital to allow certain things on Arrakis to remain secret, are much more dubious. Herbert mostly doesn't try to explain these things, and as with Star Wars the less explained the cheats are, the better they work as part of the story. But the technological background doesn't hold up against much examination. Worse, for me, is the general quality of the writing. Herbert does some things very well, such as world-building, and avoids awkward infodumps. Characterization and pacing are both fairly solid; he does a good job with Paul and Jessica in particular, and I've always liked the Fremen. But he wants to put the reader in everyone's head, frequently by giving character thoughts as italicized dialogue, and to enable that he uses a perspective that I always find distracting. Most fiction is written in tight third person. This means that the viewpoint character for any given section of the book is referred to in the third person, like all the other characters, but the reader has special access to their thoughts and emotions. We get to know what they're really thinking and feeling, not just the impressions they give to others, while the non-viewpoint characters are shown only from external appearances and the thoughts of the viewpoint character. Some books hold to the same viewpoint character throughout, but more commonly books move between viewpoint characters at scene breaks to provide more angles on the book's events. First person, in which the story is told by a specific character as if they were telling a story or writing it down, is the most common alternative. Third person objective, in which we don't get any special insight into the internal thoughts of any of the characters, is less common but still unsurprising. Dune does not use any of those perspectives. Instead, Dune uses wandering third-person omniscient, in which we get the inner thoughts and emotions of a character in a scene and then a few lines later the inner thoughts and emotions of a different character. This is the sort of thing that may or may not bug you depending on how much you've read, how deep the expectations of perspective are ingrained, and how much you notice perspective. It drives me nuts. I subconsciously align with the viewpoint character of a section, and pay attention to the ways that authors indicate which character will be the viewpoint character at the start of a scene. Herbert's constant flitting from character to character makes me dizzy. We get the verbatim thoughts of everyone almost indiscriminately, making me feel like I'm randomly hopscotching through the scene. For me, this does two things: it hurts my ability to get engrossed in the story, since I'm constantly thrown out of my normal reading mode when the viewpoint unexpectedly shifts, and it makes the writing feel repetitive. One keeps hearing about the same thing from multiple perspectives, and at times the story bogs down in everyone's internal dialogues rather than showing character reactions and letting the reader draw their own conclusions. I think it tries for a cinematic perspective, but ends up making the story feel muddled. The other flaw, which I didn't notice originally but which leaped out at me during this re-read, is that Herbert's world-building uses quite a few stereotypes. The most notorious, and most widely discussed, is of course the Fremen. Herbert draws heavily on Arab and Islamic culture even beyond the obvious similarities of people living in a harsh, arid climate. He borrows some rather loaded terms and cultural markers, such as jihad, to construct a culture of potential religious fanatics. This is not all bad; the Fremen are clearly portrayed as the good guys, which is a refreshing change from more typical current portrayals of Islam. But it becomes clear that they have aligned their entire culture around influences from outside, and the whole plot of Dune can be fairly characterized as an instance of "what these people need is a white man." Paul (and Kynes before him) joins their culture as well, but Paul becomes a better native than the natives, while simultaneously bringing his outside perspective. It's the sort of plot that is more widely noticed today than it would have been in 1965. Another major example of this, and one that I found more blatant, is that Herbert turns the Harkonnen into hissable, one-sided villains and uses some nasty stereotypes to do it. The insane torturer is consistently and repeatedly described as effeminate, fat is used as a marker of moral inferiority and evil, and the primary villain is homosexual and prefers drugged young male slaves. Here too, this sort of characterization short-cut was more common in 1965, but it's not appealing and makes the (already rather camp) scenes set among the Harkonnen even less enjoyable. Less clear-cut is the way women are handled throughout Dune. I do have to give Herbert some credit, particularly for the era in which he was writing. There are powerful female characters in Dune, including both Jessica and Alia, who have their own independent power and successfully pursue their own agendas throughout. The effectively all-female Bene Gesserit is a major political power in the story and is treated by the other players with respect as well as fear. But it's hard not to also notice the general position of women as subservient to men, not only in the general culture of the Great Houses but also in the more positively-portrayed Fremen culture. Indeed, the subservience of women is even worse in Fremen culture, where they're treated like property and where being killed by a woman is a sign of shame. Again, Herbert deserves some credit for doing better than a lot of 1960s fiction, but the sexism fairy has still been at work here. None of these flaws change the fact that Dune is a masterpiece. Herbert brings together history, world building, ecology, politics, and a compelling coming-of-age story about a messiah figure into a fast-paced, sweeping epic with a thoroughly satisfying conclusion. I think they do make it a flawed masterpiece, but it's still one of those SF novels that everyone should read at least once. Sadly, it's also a masterpiece that I think has suffered from its own success in the form of sequels, prequels, and a ton of supporting material. This is one of the problems that truly excellent world building can lead to. Human history is fractal: any specific detail can be examined in more depth and will usually lead (provided that information is available at all) to even more fascinating detail. The best world building conveys that impression of depth. That's what Herbert achieves here with hints, notes, and asides: the sense that galactic history is a vast ediface with the same fractal complexity as real human history. It makes for a compelling background, but it also inspires people to dig into that background and flesh out all of the details the way that we do with human history. But this doesn't actually work; invented history created by one person simply cannot be fractal in the same way. Human history is endlessly complex because it was generated by the complex interactions of many people. Invented history is an illusion that hints at complexity by building the same surface, but one mind, or even a small number of minds, cannot generate the same depth. The result is that if one digs too deep, one removes that convincing surface and ends up with a mundane, simplistic, and unsatisfyingly fake set of events. I think that's what's happened with all of the supporting material that's been written around Dune since its original publication. Dune is of a piece, a single story that's deeply enjoyable on its own terms and leaves the reader with a satisfying impression of complexity. The systemic excavation of that complexity lessens it and reveals too much of the illusion. Yes, I want to know more about the Butlerian Jihad, but that's the point: the wanting is the sign of succesful crafting of imagined history. Reading the definitive account is more likely to leave me unsatisfied than to lead to the recursive curiosity that human history can create. The sequels to Dune written by Herbert himself are, for me, another matter. Some reviewers level the same criticism at them: that Herbert dives too far into background best left unexplored. But they have the advantage of moving forward, telling more of the story set off by Paul, and the end of Dune is a clear setup for a sequel. One of Paul's goals throughout most of the book has been left unaccomplished. I don't think Herbert dove too deep into his creation; rather, my problem with his sequels (all of which I've read, although it's been some years now) is that he took the story in a direction that I actively disliked and found painful to read. Regardless, the general consensus is that the sequels aren't as good as the original, and while Dune doesn't fully resolve its story, it's complete enough that it's possible to stop here. Stopping is the general recommendation, although I still may re-read and review the sequels at some point. Followed by Dune Messiah. Rating: 8 out of 10

22 November 2012

Philipp Kern: OpenClonk

The Free and Open successor to the Clonk computer game series OpenClonk is now available in Debian sid and Ubuntu raring: just use sudo apt-get install openclonk to install it (backports to Ubuntu precise and quantal can be found in the OpenClonk Releases PPA).

<iframe allowfullscreen="allowfullscreen" frameborder="0" height="236" src="https://www.youtube.com/embed/ykJZ9WsevGE" width="420">OpenClonk Beyond the Rocks Trailer</iframe>

It's still under development, but as you can see it's also already quite functional. The engine was made available freely under the ISC license after the original author of Clonk was no longer interested in creating new successors. Many of the old volunteer contributors then picked it up. Be aware that same computer multiplayer, which was available in the older parts of the series, is gone. If you play on the network, you need to ensure that both of you have the same version of the game.

Ian Wienand: Upgrading Samsung Series 9 Laptop BIOS without Windows

Having recently acquired a Samsung Series 9 laptop, I'd like to update the BIOS. This is proving rather difficult without Windows. The first thing you download from the Samsung website isn't actually BIOS update at all; it's a Windows application which then goes and finds a BIOS update for your machine. Some time with a Windows VM, tcpdump and you'll end up at http://sbuservice.samsungmobile.com/. I'll leave interested parties to play around at that website, but with a little fiddling you can find the BIOS you're looking for. In this case, it is ITEM_20121024_754_WIN_P08AAH.exe which is the latest version, P08AAH. Unfortunately, this still won't run in any useful way -- in a virtual-machine it errors out saying the battery isn't connected. On previous endeavors such as this, running various incarnations of zip on the .exe file has found interesting components, but not in this case. Not to be deterred, some work with strings will yield the following
$ strings ./ITEM_20121024_754_WIN_P08AAH.exe   grep Copyright
inflate 1.2.7 Copyright 1995-2012 Mark Adler
deflate 1.2.7 Copyright 1995-2012 Jean-loup Gailly and Mark Adler
Now this is interesting, because it tells us that the components aren't using some fancy compression scheme we have no hope of figuring out. Checking RFC1952 we see the following header mentioned
Each member has the following structure:
+---+---+---+---+---+---+---+---+---+---+
 ID1 ID2 CM  FLG      MTIME      XFL OS   (more-->)
+---+---+---+---+---+---+---+---+---+---+
We can easily look for that in the file. Expanding the values, what we're looking for is 0x1f 0x8b 0x08 0x0 0x0 0x0 0x0 0x0 0x0 0x0 which represents the compressed stream (the OS byte is 0xb for NTFS which we can just ignore). A quick hack will get that done and give us the offsets we need
$ ./look-for-headers
Found potential header at 455212
Found potential header at 677136
Found potential header at 1293930
Found potential header at 1294980
Found potential header at 2771193
Found potential header at 2779874
Found potential header at 2789142
Found potential header at 2935778
Found potential header at 3078639
Found potential header at 3083039
Found potential header at 3338552
Found potential header at 3339443
Found potential header at 3352440
Found potential header at 3366757
Found potential header at 3391734
Found potential header at 3444255
Found potential header at 3494372
So, we can feed this into a shell script and extract the streams
FILE=ITEM_20121024_754_WIN_P08AAH.exe
for offset in 455212 677136 1293930 1294980 \
    2771193 2779874 2789142 2935778 3078639 \
    3083039 3338552 3339443 3352440 3366757 \
    3391734 3444255 3494372
do
    echo "create $offset.gz"
    dd if=$FILE  bs=1 skip=$offset   zcat > $offset.data
done
We don't have to worry about the end-point, because zcat will handily stop outputting and ignore junk at the end of the stream. That will yield the following output files:
$ md5sum *
c014e900b02ae39d6e574c63154809b2  1293930.data
982747cd5215c7d51fa1fba255557fef  1294980.data
942c6a8332d5dd06d8f4b2a9cb386ff4  2771193.data
d98d2f80b94f70780b46d1f079a38d93  2779874.data
33ce3174b0e205f58c7cedc61adc6f32  2789142.data
1d6851288f40fa95895b2fadee8a8b82  2935778.data
3a0a9b2fa9b40bdf5721068acde351ce  3078639.data
fc166141d06934e9c7238326f18a4e68  3083039.data
8cf7f86f84afa81363d3cef44b257d50  3338552.data
85e09e18fd495599041e13b4197e1746  3339443.data
0c72004eae0131d52817f23faf8a46a7  3352440.data
26347f93721e7ac49134a17104f56090  3366757.data
26973e301da4948a0016765e40700e8c  3391734.data
c8e4675cda3aa367d523a471bb205fba  3444255.data
7bf5ea753d4cc056b9462a02ac51b160  3494372.data
a2c6562f7f43aaa0e636cc1995b7ee3d  455212.data
d5c91b007dac3a5c6317f8a8833f9304  677136.data
65c0dcd1e7e9f8f87a8e9fb706eb8768  ITEM_20121024_754_WIN_P08AAH.exe
$ file *
1293930.data:                     MS-DOS executable
1294980.data:                     data
2771193.data:                     PE32 executable (native) Intel 80386, for MS Windows
2779874.data:                     PE32+ executable (native) x86-64, for MS Windows
2789142.data:                     PE32 executable (console) Intel 80386 (stripped to external PDB), for MS Windows
2935778.data:                     PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows
3078639.data:                     MS-DOS executable
3083039.data:                     PE32 executable (GUI) Intel 80386, for MS Windows
3338552.data:                     ASCII text, with CRLF line terminators
3339443.data:                     PE32 executable (native) Intel 80386, for MS Windows
3352440.data:                     PE32+ executable (native) x86-64, for MS Windows
3366757.data:                     PE32 executable (DLL) (GUI) Intel 80386, for MS Windows
3391734.data:                     PE32 executable (GUI) Intel 80386, for MS Windows
3444255.data:                     PE32 executable (GUI) Intel 80386, for MS Windows
3494372.data:                     PE32 executable (DLL) (GUI) Intel 80386, for MS Windows
455212.data:                      PE32 executable (DLL) (console) Intel 80386, for MS Windows
677136.data:                      data
ITEM_20121024_754_WIN_P08AAH.exe: PE32 executable (GUI) Intel 80386, for MS Windows
This is where I get a bit stuck. There is clearly a bunch of WinFlash stuff that Wine kind-of recognises, but so far I haven't managed to figure out how I might go about flashing this from a DOS boot disk. So far I know it's a "Phoenix SecureCore Tiano" based BIOS, and there is some indication that there is a UEFI app to flash the BIOS, but unfortunately I didn't set the laptop up with UEFI (note to anyone setting this up; probably a good idea to do that). I believe that if you have Windows, you can halt the install part-way through and see these files extracted in a temp directory. Possibly if we can put some file-names to the blobs it might help. There may also be a much easier way to do this that I'm completely missing. If anything comes up, I'll update this post; hopefully a future Googlenaut who enjoys BIOS hacking will drop a comment... Update 2012-11-27 I'm unfortunately less optimistic about being able to update this. Certainly the files identified above as MS-DOS executable are EFI binaries; however they are both 32-bit binaries. You can easily get an EFI shell up and running by putting a 64-bit EFI shell binary on a FAT formatted USB disk in the efi/boot/bootx64.efi but it will not run either of those binaries, presumably because they're 32-bit. A 32-bit EFI binary doesn't seem to be recognised for boot, so no luck there. A comment also mentioned some issues with Samsung's and UEFI boot in this bug though it's not clear what models are affected. Another comment mentioned binwalk which also does a nice job of finding the embedded zip streams.
$ strings ./1293930.data
...
RSDS
D:\flash\Source\ENV\src\Efi863\edk\phoenix\edk\Sample\Platform\DUET\ldr16\IA32\oemslp20.pdb
oemslp20.dll
InitializeDriver
2n3u3
$ strings ./3078639.data
en-us;zh-Hans
slp20.marker
slp20.size
slp20.offset
slp20.offset2
read marker fail!
Marker file '%s''s size(%d)!=%d
Not a valided OemMarker!
Update image data error.
marker file is updated.
marker file size too big!
Not a valided OemMarker in ROM.
Update image data error.
SLP marker is reserved.
-PROT
Prot flag is found, mProgFlag is true.
Prot flag is not found, mProgFlag is false.
Please specify input file name.
-PROT
slpFile
OEM Activation 2.0 processing
Failed to Locate Flash Support protocol.
...
D:\flash\Source\ENV\src\Efi863\edk\phoenix\edk\Sample\Platform\DUET\ldr16\IA32\SLP20.pdb

21 November 2012

Dirk Eddelbuettel: Rcpp attributes: A simple example 'making pi'

We introduced Rcpp 0.10.0 with a number of very nice new features a few days ago, and the activity on the rcpp-devel mailing list has been pretty responsive which is awesome. But because few things beat a nice example, this post tries to build some more excitement. We will illustrate how Rcpp attributes makes it really easy to add C++ code to R session, and that that code is as easy to grasp as R code. Our motivating example is everybody's favourite introduction to Monte Carlo simulation: estimating . A common method uses the fact the unit circle has a surface area equal to . We draw two uniform random numbers x and y, each between zero and one. We then check for the distance of the corresponding point (x,y) relative to the origin. If less than one (or equal), it is in the circle (or on it); if more than one it is outside. As the first quadrant is a quarter of a square of area one, the area of the whole circle is -- so our first quadrant approximates over four. The following figure, kindly borrowed from Wikipedia with full attribution and credit, illustrates this: Example of simulating pi Now, a vectorized version (drawing N such pairs at once) of this approach is provided by the following R function.
piR <- function(N)  
    x <- runif(N)
    y <- runif(N)
    d <- sqrt(x^2 + y^2)
    return(4 * sum(d < 1.0) / N)
 
And in C++ we can write almost exactly the same function thanks the Rcpp sugar vectorisation available via Rcpp:
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
double piSugar(const int N)  
    RNGScope scope;		// ensure RNG gets set/reset
    NumericVector x = runif(N);
    NumericVector y = runif(N);
    NumericVector d = sqrt(x*x + y*y);
    return 4.0 * sum(d < 1.0) / N;
 
Sure, there are small differences: C++ is statically typed, R is not. We need one include file for declaration, and we need one instantiation of the RNGScope object to ensure random number draws remain coordinated between the calling R process and the C++ function calling into its (compiled C-code based) random number generators. That way we even get the exact same draws for the same seed. But the basic approach is identical: draw a vector x and vector y, compute the distance to the origin and then obtain the proportion within the unit circle -- which we scale by four. Same idea, same vectorised implementation in C++. But the real key here is the one short line with the [[Rcpp::export]] attribute. This is all it takes (along with sourceCpp() from Rcpp 0.10.0) to get the C++ code into R. The full example (which assumes the C++ file is saved as piSugar.cpp in the same directory) is now:
#!/usr/bin/r
library(Rcpp)
library(rbenchmark)
piR <- function(N)  
    x <- runif(N)
    y <- runif(N)
    d <- sqrt(x^2 + y^2)
    return(4 * sum(d < 1.0) / N)
 
sourceCpp("piSugar.cpp")
N <- 1e6
set.seed(42)
resR <- piR(N)
set.seed(42)
resCpp <- piSugar(N)
## important: check results are identical with RNG seeded
stopifnot(identical(resR, resCpp))
res <- benchmark(piR(N), piSugar(N), order="relative")
print(res[,1:4])
and it does a few things: set up the R function, source the C++ function (and presto: we have a callable C++ function just like that), compute two simulations given the same seed and ensure they are in fact identical -- and proceed to compare the timing in a benchmarking exercise. That last aspect is not even that important -- we end up being almost-but-not-quite twice as fast on my machine for different values of N. The real takeaway here is the ease with which we can get a C++ function into R --- and the new process completely takes care of passing parameters in, results out, and does the compilation, linking and loading. More details about Rcpp attributes are in the new vignette. Now enjoy the .

Update:One somewhat bad typo fixed.

Update:Corrected one background tag.

Axel Beckert: zutils: zcat and friends on Steroids

I recently wrote about tools to handle archives conveniently. If you just have to handle compressed text files, there are some widely known shortcut commands to mimic common commands on files compressed with a specific compression format.
gzip bzip2 lzma xz
cat zcat bzcat lzcat xzcat
cmp zcmp bzcmp lzcmp xzcmp
diff zdiff bzdiff lzdiff xzdiff
grep zgrep bzgrep lzgrep xzgrep
egrep zegrep bzegrep lzegrep xzegrep
fgrep zfgrep bzfgrep lzfgrep xzfgrep
more zmore bzmore lzmore xzmore
less zless bzless lzless xzless
In Debian and derivatives, those tools are part of the according package for that compression utility, i.e. the zcat command is part of the gzip package and the xzfgrep command is part of the xz-utils package. But despite this matrix is quite easy to remember, the situation has a few drawbacks: This is where the zutils project comes in: zutils provides the functionality of most of these utilities, too, but with one big difference: You don t have to remember, think about or type which compression method has been used for your data, just use zcat, zcmp, zdiff, zgrep, zegrep, or zfgrep and it works independently of what compression method has been used if any or if there are different compression types mixed in the parameters to the same command:
$ zfgrep foobar bla.txt fnord.gz hurz.xz quux.lz bar.lzma
Especially if you use logrotate and let logrotate compress old logs, it s very comfortable that one command suffices to concatenate all the available logfiles, including the current uncompressed one:
$ zcat /var/log/syslog*    
Additionally, zutils versions of these tools also support lzip-compressed files. The zutils package is available in Debian starting with Wheezy and in Ubuntu since Oneiric. When being installed, it replaces the original z* utilities from the gzip package by diverting them away. The only drawback so far is that there neither a zless nor a zmore utility from the zutils project, so zless bla.txt fnord.gz hurz.xz quux.lz bar.lzma will not work as expected even after installing zutils as it is still the one from the gzip package and hence it will show you just the first two files in plain text, but not the remaining ones.

17 November 2012

Craig Sanders: done

OK, I m finished my experiment. It s over now and I can break character . First, I have to offer special thanks to tshirtman for being the first to unambiguously exemplify one of my main points. well done! In case it s not blindingly obvious (as it should be), the reason for my post was that I was outraged by the spectacle of one fairly high-profile member of the linux community trying to rally support to shun and exclude another fairly high-profile member because a nightmare had upset her. WTF? Is that really all it s going to take to destroy someone s reputation and perhaps their career? even with the shunning target s own words available and archived to disprove the ridiculous straw-man mis-characterisations of what he actually said? Not one of the arguments against him actually addressed anything he said, they ALL attacked him for things he didn t say, for things that other people claimed he said. I was further outraged by seeing everyone who even suggested that questioning of stats (or, indeed, ANY claim of fact or evidence) may be, in some small way, a valid and reasonable thing to do get instantly put in their place and dismissed as Yet Another Rape Apologist. Are we supposed to be anti-science, anti-scientific method now? or are rape stats a special case like religion where we are just supposed to switch off our analytical brains and accept what we are told on faith, without question? Surely we are capable of better than that? i know we are capable of better than that. Or, at least, i used to know that. Now i m not so sure. In 2011, all it took was Ted Tso ( TT ) making some fairly reasonable statements about the need for any claimed evidence or statistics to be viewed skeptically and that dissenting research should also be considered and he was instantly vilified as a rape apologist . Sorry, but questioning extremely dodgy stats (that even in feminist circles are viewed more as ideological propaganda than as serious research) is NOWHERE NEAR SUFFICIENT to earn the label of rape-apologist. That is not how debate works you can t just refuse to engage with someone s point and simply accuse them of being the enemy for not agreeing 100% with whatever you say at least, not if you have any intellectual honesty or self-respect. (sure, some people are complete jerks and deserve to be told to FOAD in no uncertain terms but a) jerks like that are self-evident and obvious, and b) TT s participation in that thread was at all times civil and reasonable) But that thread is ancient history it was over and done with nearly two years ago.
In October this year, for reasons which are not at all clear, Valerie Aurora ( VA ) decided to revive the issue (which had been resolved back in 2011 with a resounding fuck no, we don t want misogynist shit or porn in our conferences from pretty much the entire linux community including near-universal support for improved anti-harassment policies both for linux.conf.au and for geek conferences in general) and use it to attack TT. And she did so by twisting his words and claiming in a post on The Ada Initiative blog that he said something which he didn t, that rape was impossible if both people were drunk enough . If he chose, he could quite easily win a libel case against her and TAI on that. It s not what he said, it s not even close to what he said, and VA is clearly too smart to honestly believe that it is what he said. In another post on her personal, blog she talks about how what he said was so terrible that it even now gives her nightmares, and that she can t bear the thought of working with him. Again, WTF? VA can say I had nightmares and was upset and furious and THAT is enough to justify a call to shun TT?? He didn t attack her, or threaten her (explicitly or implicitly), he was polite and civil. What he did was disagree with VA by referring to other research that disputed VA s preferred studies. I agree with and support many (perhaps most) of VA s and The Ada Initiative s aims, I certainly believe that linux and open source etc should be very welcoming and supportive of human diversity (including gender and sexuality, identity, religion, politics and so on), believe that it s a good thing that The Ada Initiative exists as part of that diversity welcome to be particularly supportive of women in geekdom. And i wholeheartedly agree that Linux leaders should not make public statements belittling and condoning rape BUT: a) I haven t seen one instance of that happening, ever and b) I find VA s choice of tactics here to be despicable. as i do when anyone else uses similar tactics, because they ARE despicable tactics. They are exactly the same as accusing someone of being a child pornographer for being against net censorship: You dared to disagree with me so I m going to accuse you of being a monster. (and, i must admit, the enthusiasm level of my support for The Ada Initiative is somewhat .diminished by this tactical blunder by the spokesperson and co-founder) There are far more deserving targets of VA s ire than TT. And there are far better ways for the Ada Initiative to achieve their aims.
Why? So why did I decide to comment when I knew that I was inevitably going to be accused of hating women and being a rape apologist? Mostly because I thought it would be gutless of me not to. Hardly anyone else had, and they quickly backed down under the accusations of misogyny and since I consider myself to be psychologically fairly strong, I felt that I am capable of wearing a little shit (or even a lot) for a while. In my egocentric fashion, I thought if I can t do then it s no wonder that no one else dares . And also because anyone who cared to make even the slightest effort to find out what my actual views on sexual harassment, rape, women s rights and numerous inter-related issues are can fairly easily see a very consistent record of the kinds of things I argue for and against, and my scathing responses to actual misogynists when they appear on lists that I participate in. they are not my kind of people. and even then, i hesitated. it s scary and intimidating to be putting yourself forward to be accused of being one of the things you hate. this is, of course, an instance of the chilling effect. So, I found the prospect scary, almost terrifying .and I can t think of a single person who has met me online or in real life who would even remotely describe me as being any kind of delicate or sensitive wall-flower. again, if i can t do it, it s no wonder no one else dares . so i clicked the Publish button. an ego is useful for some things. Also, I took VA s words but don t be silent as inspiration. In the process, i discovered why it is that some people just simply refuse to engage in rational discourse. I ve seen it many times from the other side, but i ve never experienced the seductive pleasure of indulging in it myself before there s a liberating freedom in just ignoring any and every point that someone makes and simply accuse them of being the enemy. You don t have to try to understand what they wrote, hell you don t even have to really read it you just need to quickly scan it for overall tone and if they don t seem like they re 100% supportive, you just accuse them of being the enemy or an apologist for the enemy. it s that fucking simple and easy. well, sort of easy. easy for some, perhaps. i personally found it extremely difficult a struggle to refrain from engaging, to remain in character (i m not much of an actor). especially when i kind of agreed with whoever was arguing against my experimental character or if i thought they made a good point. and even more so when i thought that some comments skirted a bit too close to being the kind of misogynist crap that i didn t want to tolerate having on MY blog. (i resolved that issue by just approving any reply that didn t squick me or that i could squint at and think hmmm borderline, give benefit of doubt ) but even though i don t like it, i can recognise the attraction it holds for some people.
Other thoughts: I m particularly disgusted by the men who intervene way too early without an explicit invitation or request for help or a clear need such as an immediate threat of violence in womens issues. Many or maybe even most may not realise it, but they are just taking over and asserting male strength and control by protecting women rather than giving them the support and space to discover and practice their own strength and their own voices. These uninvited interventions do not help women, they weaken and undermine them, they perpetuate dependence, they steal strength from the movement. It is patronising and enfeebling. But mostly, they just re-assert male dominance and are an attempt to make women s spaces more comfortable, more palatable, for men.(*) (it s also quite often very transparent self-serving and ingratiating behaviour from blokes who want to lay the groundwork for perhaps getting laid one day) IMO this goes far beyond a problem with men over-involving themselves in feminist causes i feel the same way for any relatively less-privileged group with a need to find their own voice and their own power they ll never find it if members of the privileged class (i.e. white males like me) just ride roughshod over the movement and speak FOR them rather then just silently lending their strength in support. For the most part, they (we) should just shut up and listen we already have more than enough opportunities to have our say. (*) yes, i m well aware of the difficulty in writing something like that paragraph as a member of a privileged class, without coming across as either self-hating or patronising or both. if i ve failed here, it s not for want of trying. done is a post from: Errata

Antoine Beaupr : First stable release of the Drupal OpenID Provider

A short announcement to mark the release of the first stable OpenID provider for Drupal. Started by James Walker around 4 years ago, I have since then got commit access and taken care of the module for the last 2 years. And while it was not an easy task, with the help of the community, this module was able to mature to a fully featured, flexible and standard implementation during that time. One has to wonder however, is it too late for OpenID?

A bit of history Drupal was one of the pioneers in OpenID support. Back in 2006 when the protocol was still in its infancy (less than a year old!), Drupal was adding OpenID support to its 4.7 release through a contrib module and two years later, the Drupal 6.0 release was sporting OpenID in core, although not without significant interoperability issues. Still, we could say that rapid adoption by such a popular CMS as Drupal really boosted OpenID's adoption. At about the same time, Yahoo, Sourceforge, Myspace, Microsoft and Google would all join the OpenID bandwagon. On the OpenID "provider" side of things, things were not so great however. Back when I started experimenting with this, it wasn't actually that easy to host your own OpenID provider, to allow your users to login to other OpenID-enabled sites. While there are a lot of different libraries to provide OpenID support in your application there was not (and still isn't) any software that gives you an OpenID provider and is dedicated to that. No official reference implementation either, other than the code from the one of the founding companies of OpenID, JanRain. There was an OpenID provider module in Drupal, but there was no official release done until a two beta releases in 2010.

Koumbit's community involvement It is about that time I started getting involved in the project. Koumbit needed an easy way for our workers to log into multiple sites without too much trouble, so I started looking at OpenID provider implementations. The lack of dedicated software somehow made the Drupal implementation a natural fit, and I started experimenting with the module, filing bugs and reviewing patches, as there were a lot of interoperability and design issues with the provider. After a while, walkah was kind enough to give me full access to the project and I started doing regular releases (every few months), culminating in today's official 1.0 release for both Drupal 6 and Drupal 7, which I am very proud and happy about. While I am here, I want to send many thanks to the people behind the Drupal 7 port: paranojik, mikecar, xamanu and testing from neizod. A lot of issues were also patched, reviewed and tested by numerous contributors that are unfortunately difficult to track down other than reading the commit log. So we at Koumbit have been running our own OpenID provider for a good while now, and it has the key advantage of being connected to our LDAP directory (with the D6 LDAP integration module so it is, in a sense, a central point for authentication at Koumbit. Very often people don't clearly understand what their LDAP password does, and I can easily remind them "it's the OpenID thing", or "try your password in the OpenID site" if they think they have forgotten it. So much goes on in the web browser these days that it makes everything much easier.

The problem with Drupal's OpenID provider However, it is kind of embarrassing that it took so long. More than 6 years after the OpenID protocol was written, we have a stable OpenID provider. Now, this may be because I am overly conservative about stable releases, but I don't think so: the OpenID provider module had, for a long time, serious interoperability issues with OpenID sites. There was also a clear lack of features on the OpenID provider, a gap that various third party modules, now documented on the OpenID provider homepage are now trying to fill. There is also serious security issues with the protocol itself. The complexity and lack of clarity of the OpenID specification have rendered my work quite difficult at time, as the protocol has significant ambiguities and no clear reference implementation. OpenID is unfortunately renowned to be vulnerable to phishing and while Koumbit's use of the HTTPS protocol alleviates some of those issues (especially the redirect session hijack), tools like sslstrip can bypass those protections easily. The privacy aspect is also problematic, as the user is usually not free to choose which information from his or her profile is divulged to the site they are logging into. Of course those issues may be overlooked if the security trade off is not worth it. In the case of Koumbit, we use OpenID on a lot of client sites, but usually nothing critical, and we train our users to pay attention to the HTTPS icon when login in their OpenID account.

Are there alternatives? But unfortuantely, the reality is that there are not really any alternatives to OpenID out there. Mozilla's new BrowserID initiative (now rebranded Persona, boy those guys sure love rebranding..) is interesting, but actually has even more flaws than OpenID itself. Not only is it not resolving the phishing problem, it actually creates new problems, including my personal favorite, trusting server-side Javascript code, which OpenID is not using that much. I should also mention the Shibboleth module which has a surprising number of installs recorded on Drupal.org, but that depends on the much more complicated Shibboleth federation, for which there is no server implementation for Drupal, from what I understand. Oauth, on the other end, is even more widely used, and has server-side components like Oauth Login Provider and Services 3.x. We could also simply give up and use our souls stored at Facebook, Google and Twitter for authentication everywhere. Maybe OpenID opened the door for those corporations to see that there is a real opportunity in leveraging user's profile to track their online activity. It is not something that OpenID was necessarily designed for but it's an idea that certainly permeates Facebook connect and "like" buttons or Twitter and Google online business strategies, where you are the product sold to advertisers. I personally believe we should keep the struggle for distributed and standard identity mechanisms for the web that respect users' privacy. With Drupal's OpenID Provider, it is possible to create a secure, privacy-aware identity provider, although I still have to perform that damn security audit. Experiments with those third party modules should also be encouraged and I welcome any feedback on those modules. While I do not plan to attend OpenID conferences or do significant development on the OpenID provider right now, I will certainly make sure Koumbit keeps its commitment to maintaining stewardship on the OpenID provider module in Drupal. I encourage people to try it out, submit issues and patches to make it even better. Long life to OpenID! PS: if people are interested in funding that security audit, I would be happy to perform this through my regular work at Koumbit.org, just file a request at services@koumbit.org.

Michal &#268;iha&#345;: phpMyAdmin documentation now in Spanish

Since converting the phpMyAdmin documentation to new format and updating it's content, I was quite frightened how this will affect translated versions of it. Fortunately giving it more attractive format and more advertisement lead to increased contributions. Since Friday, we already have one language which has reached 100% - Spanish. Mat as Bellone simply did there awesome job and went through all fuzzy and new strings in just few days. Now I'm looking forward to other contributors as well, I hope we will be able to deliver translated manual for phpMyAdmin 4.0 in many languages. I'll definitely try to translate to Czech, but any help with that is welcome.

Filed under: English Phpmyadmin 2 comments Flattr this!

Dirk Eddelbuettel: RcppArmadillo 0.3.4.4

A minor bug-fix release 3.4.4 of Armadillo came out upstream a few days ago. RcppArmadillo, our wrapper for R and Armadillo, is now on CRAN with its corresponding version 0.3.4.4. No R level or interface changes were made and the upstream changes are summarized below.
Changes in RcppArmadillo version 0.3.4.4 (2012-11-15)
  • Upgraded to Armadillo release 3.4.4
    • fix for handling complex numbers by sparse matrices
    • fix for minor memory leak by sparse matrices
Courtesy of CRANberries, there is also a diffstat report for 0.3.4.4 relative to 0.3.4.3 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

16 November 2012

Michal &#268;iha&#345;: Weblate 1.3

Quite on updated schedule (ie. one month later than originally planned), Weblate 1.3 has been released today. It comes with lot of improvements, especially in support of non gettext files, new quality checks and improved performance. Full list of changes for 1.3: You can find more information about Weblate on it's website, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Ready to run appliances will be soon available in SUSE Studio Gallery. Weblate is also being used https://l10n.cihar.com/ as official translating service for phpMyAdmin, Gammu, Weblate itself and others. If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you (this will be decided case by case as my hosting space is limited).

Filed under: English Phpmyadmin Weblate 0 comments Flattr this!

Julien Danjou: Logitech Unifying devices support in UPower

A few months ago, I wrote about my reverse engineering attempt to Logitech Unifying devices. Back then, I concluded my post with big hopes on the future after receiving a document with some part of the specification of the HID++ 2.0 from Logitech. A couple of weeks ago, some of my summer work has been merged to UPower, adding battery support for some Logitech devices. HID++ UPower As I discovered late in my first reverse engineering attempt, Logitech developed a custom HID protocol named HID++. This protocol exists in two versions, 1.0 and 2.0. Some devices talk with version 1 of the protocol (like my M705 mouse) and some others talk with version 2 of the protocol (like my K750 keyboard). Recently, I've been able to be in touch with a Logitech engineer who worked on the Linux support for the Unifying receiver, and he has been really helpful and exposed me some details about this protocol. Logitech made the decision to publish their HID++ specification publicly about a year ago, but still didn't do it. The internal review needed to publish such documents hasn't be done yet. The only published draft is just an extract of the specification, with even some typo in it as I discovered. Some other documents have been recently published, but I didn't have the time to review them. They contains HID++ 1.0 specifications and some details I asked for about the K750 keyboard. UPower support UPower It took me sometime to get a full understanding of the protocol, its different version etc. After reverse engineering my K750 keyboard, I've also reverse engineered the data stream used to get my M705 mouse battery status. I've also received some information about the HID++ 1.0 protocol, so I've been able to discover a bit more on what the packets mean. Most of my discoveries are now used to do proper #define in up-lg-unifying.c so the code makes more sense. My first patch implements a new property for UPower devices, named luminosity, that use with K750 keyboard to report the light level received. The second patch add support for Logitech Unifying devices (over USB only) and should work with at least Logitech M705 and K750 devices. This should be available with the next version of UPower, which should be 0.9.19. gnome-power-statistics for K750 So far, Logitech has been kind enough to help me understanding part of the protocol and even sent me a few devices so I can play and test my work with them. Unfortunately, this will probably requires some work and time, and so far Logitech was not able to help with that. There should be enough information out there to at least add support for battery to HID++ 2.0 devices, and probably a few other things too. I hope I'd get the time do this at some point, but feel free to beat me in this race!

Next.

Previous.