Search Results: "ema"

3 December 2012

Raphael Geissert: Some things you wanted to know about http.debian.net

After quite a bit of, very welcome, feedback I've put together a FAQ page in an attempt to respond to the most common questions about http.debian.net.

Emails have been accumulating for a few weeks now, but I will get to them. So please be patient if you send me an email, or if you have sent me one.

1 December 2012

Raphaël Hertzog: My Free Software Activities in November 2012

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (692.20 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Misc packaging I updated the publican package (a tool for publishing material authored in DocBook XML) with version 3.0, a major new upstream version. As with any important update, it had its share of problems and I created two patches that I sent upstream. I uploaded the package to experimental since we re in freeze. The Debian Administrator s Handbook Since the translation teams have been working for a few months, I wanted to put the result of their work online. I did it and I blogged about it on debian-handbook.info. By the way, we have a Polish translation that just started. This took quite some time because many translators were not well versed with Docbook XML and its structure. So I fixed their mistakes and asked the Weblate developer (Michal Cihar) to implement new checks to avoid those basic XML mistakes. I also added a couple of build scripts to the git repository to make it easier to rebuild translations in multiple formats. I used this opportunity to file a couple of bugs I encountered with Publican (concerning ePub output mainly, and custom brands). I also blogged about our plans to update the book for Wheezy. Roland started to work on it but I did not have the time yet. Debian France The officers (president, treasurer, secretary) have just changed and we had to organize the transition. As the new president, I got administrator access on our Gandi virtual machine (france.debian.net) as well as access to our bank account. I got also got a bunch of administrative papers retracing the history of the association. Carl Chenet (the former president) gave them to me during the mini-debconf that was organized in Paris. Indeed, Sylvestre Ledru and Mehdi Dogguy organized our second mini-debconf Paris and they did it very well. It was a great success with over 100 attendants each of the 2 days it lasted (November 24-25th). Carl managed a merchandising booth that was well stuffed (Luca Capello also brought goodies of Debian.ch) I gave small lightning talk to present the ideas behind my Librement project (it s about funding free software developers). BTW I have not been very good at it, it was only my second lightning talk and I have been a bit too verbose. The talk did not fit in my 5 minutes time slot ;-) Back from the mini-debconf, I have been trying to delegate some projects (like get a real website, improve the work-flow of members management, update our server which was still running Lenny). Julien Cristau was willing to upgrade the server did not exactly knew how to upgrade the kernel (it s a bit special since Gandi manages the kernel on the Xen hypervisor side). So I took care of this part and also did some cleanup (adding a backup with its associated remote disk, tweaking the email configuration). And Julien completed the upgrade on November 30th. Alexandre Delano volunteered to have a try at the website and Emmanuel Bouthenot has been looking a bit to see if there was something better than Galette to handle our members. It looks like we ll stay with Galette but have to take care of upgrading it to a newer version. I also processed the first membership applications and organized a vote to extend the board of administrators (since we have two vacant seats). On Monday, we should be back to 9 administrators. Librement Except for the talk during the mini-debconf, I did not do much on this project. That said I got an answer from the Autorit de Contr le Prudentiel saying that I might be eligible for the exemption case (see discussion of last month) and that I should fill out a form to get a confirmation. I also contacted Tunz.com who might be able to provide the services I need (their E-money manager product in particular). They have the required accreditation as a banking/credit institution and are willing to partner with enterprises who setup platforms where you must manage flows of money between several parties. I m now waiting for details such as the cost of their various services. I expect to have much more to show next month I m working with two developers to implement the first building blocks of all this. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

30 November 2012

Gunnar Wolf: Becoming a teacher!

For many years, I have aspired being a university teacher. I remember asking at different universities as early as ten years ago But I didn't have the needed papers. And yes, I have been a "Licenciado en Ingenier a en Software" for one year already. Anyway, I won't go into the details on why I didn't do this earlier on. But this time, I did get my act together in time, and by mid October, I contacted Juan Carre n, an enthusiastic teacher I met a long time ago as he helped a lot for the formation and cohesion of the (now defunct?) LIDSOL group (Laboratorio de Investigaci n en Software Libre, Free Software Research Laboratory), a group of very worthy students, mostly from the Engineering careers. Juan Carre n had long offered me help in getting to the right people in Facultad de Ingenier a as soon as I had my formal requirements ready. I just didn't expect it to be so swift! Within two days of my initial mail, he contacted me back asking me to look at the subjects in Computing Engineering and choosing some I would be willing to teach Yes, understanding that due to my time (as I'm already full-time employed in UNAM) would allow me to take only one group. Rush of excitement, of course. I promptly looked at the program, and answered with a list of 12 subjects I would be confident to teach. Next day I was contacted by the Chief of Computing Engineering Department, offering me to dictate the Operating Systems course. A subject that has always motivated me, and towards which I feel confident. A fifth semester course (from 9 semesters in the career), with around 30 students in the classroom. And I'm very happy with this! Yes, this will be my first formal university course experience (either as a student or a teacher), and I'm quite nervous on how this will turn out. But I'm already reading again my books on the subject, starting to structure a set of teaching notes, and... Lets see what comes next! So, I will be teaching this course starting January 28, three times a week for 1.5hr, for a formal theric total of 72 hours. We shall see how this results six months from today! :-D

29 November 2012

Wouter Verhelst: HELMo talk about Debian

About a month and a half ago, I got an email from someone studying at the HELMo institute for higher eduction in Li ge, Belgium, inviting me to talk at their conference which was held today. In the email, they asked me to talk about "Debian and Open Source", which is such a wide subject that I could talk about it all day; and then they gave me 20 minutes to do it. So I picked "Debian", and focused a bit more on that, only to find out when I saw their schedule that they'd put "Open Source" there instead. Hrm. Oh well. The talk itself went pretty good. I believe I managed a good balance between introducing Debian to those who've not heard of it before, and explaining to those who have, how this Debian thing works (or is supposed to work). The fact that the audience consisted of first- and second-year IT students made this easier (in case you were wondering: the third-year students, who're supposed to graduate at the end of this eyar, organized the thing). I got some interesting questions afterwards (which is always a good sign), and only positive comments as well (which is even better). After me, Lionel Dricot held a talk which was titled "Pirate Party", but which could be summarized as his personal history into the open source world, followed by some parallels between the open source world and the pirate party.That, too, was an interesting talk, even if I'm not sure I'd vote for the pirate party (which isn't even possible in Mechelen, but ignoring that). All in all, a nice use of my time.

28 November 2012

Russ Allbery: Review: Emissaries from the Dead

Review: Emissaries from the Dead, by Adam-Troy Castro
Series: Andrea Cort #1
Publisher: Eos
Copyright: March 2008
ISBN: 0-06-144372-7
Format: Mass market
Pages: 387
As a small child, Andrea Cort was a colonist on Bocai, part of a utopian project to create a mixed human and Bocaian community. Until, that is, all the members of the community woke up one day and started horrifically murdering each other for reasons that none of the survivors could ever explain. Now, she's protected from extradition by a life contract with the Diplomatic Corps and serves them as a special investigator. The scene of the investigation of this book is the strange habitat One One One. The habitat was created by the AIsource, an alliance of artificial intelligences now independent of their original creators. It's in deep space, far from any sun, and would have remained entirely unknown to the other inhabitants of the galaxy if the AIsource hadn't announced its presence. The choice to announce it was baffling, since it contains a sentient species entirely under the control of the AIsource, something that's against the shared rules of all the other species, but which they wouldn't have known about if the AIsource hadn't drawn their attention to it. After extensive negotiations, one unofficial human research group has been allowed access, and now two of the members of that group have been murdered. That's where Andrea comes in. This is Castro's first original solo novel, but he has a background in short fiction in the SF, fantasy, and horror genres and some previous collaborations and shared universe writing. The horror shows through a bit more than I'd personally prefer: the preface briefly describing Andrea's past would have been sufficient, but Castro goes back and shows it in horrific and gruesome detail as a flashback. His fight scenes are also strangely and disturbingly detailed. If you have a sensitive stomach, be aware that you may want to skim some parts of this. The overall construction of this novel, though, is not horror. It's an SF murder mystery, about halfway between a procedural and a noir detective story. Andrea does follow a coherent procedure and uses an investigative technique more sophisticated than stirring the pot and seeing what jumps out, making it not quite noir, but it has the noir tendency towards story progress via attacks on the detective and quite a lot of ongoing tension. It also involves quite a lot of setup; be aware going in that we get over a hundred pages of description of the various research group members, the details of the crimes, possible motives, and the details of the artificial habitat without a lot of forward progress on understanding the crime. Castro puts a lot of work into the design of the habitat despite leaving it mostly in the background. It could easily serve as the backdrop of a more typical Big Dumb Object SF novel. The habitable zone where the research team is and where all of Andrea's investigations take place is the effective "top" of the habitat, covered in dense foilage to which the native sentient species clings and crawls its entire life. Below is an effectively infinite fall into crushing acidic atmosphere. Castro managed to not set off my dislike of heights too badly, in part because he focuses more on Andrea's own dislike of heights rather than vividly describing the drop, but it's a constant presence throughout the novel and adds a persistant background tension. It also leads to some minor but neat clashes of perspective in conversations with the native species. For most of the book, I would have described Emissaries from the Dead as an enjoyable but not horribly original detective novel with a neat science fiction background. But there are several interesting ending twists that I won't spoil and some decisions that quite surprised me. They don't have a large effect on the plot of this book, but they make for a fascinating hook and setup for later books in the series (and I see there is a sequel). It's definitely not the sort of thing that you want spoiled before reading this book, so avoid reading blurbs and details about the sequel before reading this one. I wouldn't go out of my way to hunt down this book, but it's solid, enjoyable detective SF with a surprisingly good ending. I can see why it won the Philip K. Dick award (for best SF paperback original). Mildly recommended. Followed by The Third Claw of God. Rating: 7 out of 10

NOKUBI Takatsugu: BPS in Tokyo

I had attend to Debian Bug Squashing Party in Tokyo at 24th Nov. 2012. The place was provided by Plat Home Co., Ltd. The attendee was about 7 people, it s not so many men.
My big problem is unblocking file-mmagic package. I sent an unblocking request mail, but it has invalid From email address, so I couldn t trace the discussion.
My fault is changing source format to 3.0(quilt). It is a big change for testing.
Now I am making a new package for testing-proposed-update.
It seems about 10 RC bugs had closed by all attendees. Thank you the attendees, especially the cordinator Hideki Yamane.
The network in the place had provied by OpenBlocks, the product of Plat Home.

Dirk Eddelbuettel: Rcpp 0.10.1

A the new Rcpp release 0.10.1 arrived this morning on CRAN (as already has Windows binaries) and in Debian. This is a follow-up to the recent 0.10.0 release which extends the exciting new Rcpp-attributes and Rcpp-sugar work further, and as in a number of other areas as detailed below in the NEWS sections. This release brings an change to some of the binary interfaces. If you have packages using Rcpp, you will most likely have to reinstall them from source. Some change were made to const correctness as well as other aspects, and it seems that we have temporarily broken the excellent RcppEigen and RcppOctave packages. We are looking into this, and are sorry about the bug. The complete NEWS entry for 0.10.1 is below; more details are in the ChangeLog file in the package and on the Rcpp Changelog page.
Changes in Rcpp version 0.10.1 (2012-11-26)
  • Changes in Rcpp sugar:
    • New functions: setdiff, union_, intersect setequal, in, min, max, range, match, table, duplicated
    • New function: clamp which combines pmin and pmax, e.g. clamp( a, x, b) is the same as pmax( b, pmin(x, a) )
    • New function: self_match which implements something similar to match( x, unique( x ) )
  • Changes in Rcpp API:
    • The Vector template class (hence NumericVector ...) get the is_na and the get_na static methods.
    • New helper class no_init that can be used to create a vector without initializing its data, e.g. : IntegerVector out = no_init(n) ;
    • New exception constructor requiring only a message; stop function to throw an exception
    • DataFrame gains a nrows method
  • Changes in Rcpp attributes:
    • Ability to embed R code chunks (via specially formatted block comments) in C++ source files.
    • Allow specification of argument defaults for exported functions.
    • New scheme for more flexible mixing of generated and user composed C++ headers.
    • Print warning if no export attributes are found in source file.
    • Updated vignette with additional documentation on exposing C++ interfaces from packages and signaling errors.
  • Changes in Rcpp modules:
    • Enclose .External invocations in BEGIN_RCPP/END_RCPP
  • Changes in R code :
    • New function areMacrosDefined
    • Additions to Rcpp.package.skeleton:
      • attributes parameter to generate a version of rcpp_hello_world that uses Rcpp::export.
      • cpp_files parameter to provide a list of C++ files to include the in the src directory of the package.
  • Miscellaneous changes:
    • New example 'pi simulation' using R and C++ via Rcpp attributes
Thanks to CRANberries, you can also look at a diff to the previous release 0.10.0. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

27 November 2012

Daniel Kahn Gillmor: more proprietary workarounds, sigh

In supporting a labful of Debian GNU/Linux machines with NFS-mounted home directories, i find some of my users demand a few proprietary programs. Adobe Flash is one of the most demanded, in particular because some popular streaming video services (like Amazon Prime and Hulu) seem to require it. I'm not a fan of proprietary network services, but i'm happy to see that Amazon Prime takes Linux support seriously enough to direct users to Adobe's Linux Flash "Protected Content" troubleshooting page (Amazon Prime's rival NetFlix, by comparison, has an abysmal record on this platform). Of course, none of this will work on any platform but i386, since the flash player is proprietary software and its proprietors have shown no interest in porting it or letting others port it :( One of the main issues with proprietary network services is their inclination to view their customer as their adversary, as evidenced by various DRM schemes. In two examples, the Flash Player's DRM module appears to arbitrarily break when you use one home directory across multiple machines. Also, the DRM module appears to depend on HAL, which is being deprecated by most of the common distributions. Why bother with this kind of gratuitous breakage? We know that video streaming can and does work fine without DRM. With modern browsers, freely-formatted video, and HTML5 video tags, video just works, and it works under the control of the user, on any platform. But Flash appears to throw up unnecessary hurdles, requiring not only proprietary code, but deprecated subsystems and fiddly workarounds to get it functional. I'm reminded of Mako's concept of "antifeatures" -- how much engineering time and effort went into making this system actually be less stable and reliable than it would have otherwise been? How could that work have been better-directed? Tags: antifeatures, flash, hal, proprietary, streaming

26 November 2012

Michal Čihař: Weblate 1.3 at SUSE Studio

Finally, I've managed to update SUSE Studio images with Weblate to recently released 1.3. So you can now fetch there various images for your virtual machines or clouds. There is also new support for SUSE Cloud or OpenStack in SUSE Studio, so you can enjoy these as well. Within this version, I've upgraded the system to openSUSE 12.2 and included MTA in the image, so that you can send out registration emails. You can find more information about Weblate at http://weblate.org/.

Filed under: English Suse Weblate 0 comments Flattr this!

Steve Langasek: Upstart in Debian

Good news, everyone! So as of last Sunday, this works on all Linux archs in Debian unstable and gives you a modern version of upstart:
echo 'Yes, do as I say!'   apt-get -o DPkg::options=--force-remove-essential -y --force-yes install upstart
Thanks to the ifupdown, sysvinit, and udev maintainers for their cooperation in getting upstart support in place; to the Debian release team for accomodating the late changes needed for upstart to be supported in wheezy; and to Scott for his past maintenance of upstart in Debian. Benchmarking One of the consequences is that it's now possible to do meaningful head-to-head comparisons of boot speed between sysvinit (with startpar), upstart, and systemd. At one time or another people have tested systemd vs. sysvinit when using bash as /bin/sh, and upstart vs. sysvinit, and systemd vs. sysvinit+startpar, and there are plenty of bootcharts floating around showing results of one init system or another on one distro or another, but I'm not aware of anyone having done a real, fair comparison of the three solutions, changing nothing but the init system. I've done some initial comparisons in a barebones sid VM, and the results are definitely interesting. Sysvinit with startpar (the default in Debian) can boot a stock sid install, with no added services, in somewhere between 3.37 and 3.42 seconds (three runs). That's not a whole lot, but on the other hand this is a system with a single filesystem and no interesting services yet. Is this really as fast as we can boot? No, even this minimal system can boot faster. Testing with upstart shows that upstart can do the same job in between 3.03 and 3.19 seconds (n=3, mean=3.09). This confirms what we'd already seen in Ubuntu, that it makes a difference to boot speed to have filesystem mounting handled by an integrated process that understands the whole system instead of as a group of serialized shell scripts. What about systemd? The same test gives a boot time between 2.32 and 2.85 seconds (n=4, mean=2.48). Interesting; what would make systemd faster than upstart in sid? Well, a quick look at the system shows one possible contributing factor: the rsyslog package in Debian has a systemd unit file, but not an upstart job file. Dropping in the /etc/init/rsyslog.conf from Ubuntu has a noticeable impact, and brings the upstart boot time down nearer to that of systemd (2.78-3.03s, n=5, mean=2.92). Besides telling me that it's time to start spamming Debian maintainers with wishlist bugs asking them to include upstart jobs in their packages, this suggests that the remaining difference in boot time may be due to the outstanding init scripts in rcS.d that are made built-in no-ops by systemd but not (yet) by upstart in Debian (e.g., hwclock, hostname, udev-mtab). (In Ubuntu, /etc/rcS.d has long since been emptied out in favor of upstart jobs in the common case, since the time it takes to get to runlevel=2 is definitely a major issue for boot speed and boot parallelization.) It also gives the lie to the claim that's been made in various places that spawning shells is a major bottleneck for upstart vs. systemd. More study is certainly needed to confirm this, but at least this naive first test suggests that in spite of the purported benefits of hard-coding boot-time policies in C code, upstart with its default degree of runtime configurability is at least in the ballpark of systemd. Indeed, when OpenSuSE switched from upstart to systemd, it seems that something else in the stack managed to nullify any benefit from improving the boot-time performance of apparmor. Contrary to what some would have you believe, systemd is not some kind of silver bullet for boot speed. Upstart, with its boot-time flexibility and its long history of real-world testing in Ubuntu, is a formidable competitor to systemd in the boot speed department - and a solid solution to the many longstanding boot-time ordering bugs in Debian, which still affect users of sysvinit. I've published the bootcharts for the above tests here. Between the fact that Debian's bootchart package logs by default to /var/log/bootchart.tgz (thus overwriting on every boot) and the fact that these tests are in a VM, I haven't bothered to include the raw data, just the bootcharts themselves. The interested reader can probably generate more interesting boot charts of their own anyway - in particular, it will be interesting to see how the different init systems perform with more complicated filesystem layouts, or when booting a less trivial set of services. Other musings The boot charts have been created with the bootchart package rather than with bootchart2. For one thing, it turns out that bootchart2 includes systemd units, not init scripts; so when replacing bootchart with bootchart2, the non-matching init script is left behind and systemd in particular gets terribly confused. This is now reported as Bug #694403. In an amusing twist, while I was experimenting with bootchart2, I also noticed that having systemd installed would slow down booting with other init systems, because systemd installs udev rules which take a noticeable amount of time to run a helper command at boot even though the helper should be a no-op. So if you're doing boot speed testing of other init systems, be sure you don't have systemd on the system at the time!

25 November 2012

Lucas Nussbaum: Half of the package maintainers are not DDs or DMs

During the Paris Mini-Debconf, Nicolas Dandrimont talked about The state of mentors.debian.net: GSoC and beyond. He said that Half of Debian s packages are maintained by sponsored maintainers. That statement was actually wrong, as he confirmed later. However, using a few UDD queries, I could come up with: Full UDD notes:
all packages in sid:
select source, version from sources_uniq where release = 'sid'
packages in sid known to upload_history:
select source, version from upload_history where
(source, version) in (select source, version from sources_uniq where release = 'sid')
packages that were uploaded by the changed_by person:
create temporary table sources_not_sponsored as select distinct source, version
 from upload_history, carnivore_keys, carnivore_emails
 where (source, version) in (select source, version from sources_uniq where release = 'sid')
 and fingerprint = key
 and carnivore_keys.id = carnivore_emails.id
 and carnivore_emails.email = changed_by_email;
packages not uploaded by the changed_by person:
create temp table uh_sid as select source, version, fingerprint, changed_by_email
from upload_history
where (source, version) in (select source, version from sources_uniq where release = 'sid');
create temp table uh_sid_sponsored as select source, version, fingerprint, changed_by_email from uh_sid
where (source, version) not in (select source, version from sources_not_sponsored);
list with sponsor login:
select distinct source, version, fingerprint, changed_by_email, login
from uh_sid_sponsored
left join carnivore_keys on fingerprint = key
left join carnivore_login on carnivore_keys.id = carnivore_login.id;
=> 4188 sponsored packages. some of them are in a strange state (changed_by is a DD, but uploaded by another DD). excluding those:
create temp table sponsored_but_dds as select distinct source, version, fingerprint, changed_by_email, login
from uh_sid_sponsored, carnivore_emails, carnivore_login
where changed_by_email = carnivore_emails.email
and carnivore_emails.id = carnivore_login.id;
create temp table really_sponsored as select distinct source, version, fingerprint, changed_by_email, login
from uh_sid_sponsored
left join carnivore_keys on fingerprint = key
left join carnivore_login on carnivore_keys.id = carnivore_login.id
where (source, version) not in (select source, version from sponsored_but_dds);
=> 3147 sponsored packages
select distinct changed_by_email from really_sponsored ;
=> 963 different sponsorees
select distinct changed_by_email from upload_history where
(source, version) in (select source, version from sources_uniq where release = 'sid');
=> 2015 distinct emails.
no DD amongst maintainer or uploader:
create temp table dds_emails as select email from carnivore_emails, carnivore_login
where carnivore_emails.id = carnivore_login.id;
select source, version, maintainer, uploaders from sources_uniq
where release='sid'
and maintainer_email not in (select * from dds_emails)
and not exists (select * from uploaders where release = 'sid' and sources_uniq.source = uploaders.source and sources_uniq.version = uploaders.version and email in (select * from dds_emails))
and maintainer_email != 'packages@qa.debian.org'
and (source, version) in (select source, version from really_sponsored);

Steve Kemp: A busy few weeks - bah humbug

The following companies are amongst those showing Christmas Adverts on television before the start of December: I will boycott these companies until next year. In happier news I've spent the past week or two replacing the monitoring system that we use at work. Our previous monitoring system had been struggling to keep up with the sheer number of tests it was being asked to process. This was partly because we carry out many ping-tests, ssh-tests, http-tests, dns-tests, etc. The other reason was that our monitoring system was a behemoth of threaded-ruby, which all ran upon a single host. This made adding another monitoring host a complex undertaking. The new solution uses a work-queue: The code is open-source, written in Ruby, and available here: I've completed the process of tidying up the code to the extent I'm happy with it, and I believe I've also abstracted away the work-specific pieces of the code. That said I'd not be surprised if it needs a few minor tweaks before it it useful for other people.

Olivier Berger: Presented Generating Linked Data descriptions of Debian packages in the Debian PTS at the Paris Mini DebConf

I have made a presentation at the Paris MinDebconf 2012 about the work I ve done to bring more semantic meta-data to the Debian PTS (see previous posts). Here are my slides :
<iframe allowfullscreen="allowfullscreen" frameborder="0" height="356" marginheight="0" marginwidth="0" mozallowfullscreen="mozallowfullscreen" scrolling="no" src="http://www.slideshare.net/slideshow/embed_code/15336720" style="border:1px solid #CCC;border-width:1px 1px 0;margin-bottom:5px" webkitallowfullscreen="webkitallowfullscreen" width="427"> </iframe>
Generating Linked Data descriptions of Debian packages in the Debian PTS from olberger
Also available here as PDF.

24 November 2012

Jon Dowland: Backing up Twitter tweets and mentions

I mentioned ifttt last year: it's a web automation tool that lets you glue different web services together. I used it to build a 6music bot this year which was very well received. I also use it to back up my own twitter interactions. All of my tweets and mentions are emailed to me. I can then archive those mails with the rest of my mail. I wrote a small python script to clean up the mails that ifttt spits out. The resulting mails are now only about twice as large as the maximum length of a tweet, rather than orders of magnitude larger. I've published it here: twitter.

Russ Allbery: Review: Dune

Review: Dune, by Frank Herbert
Series: Dune #1
Publisher: Ace
Copyright: 1965
Printing: September 1990
ISBN: 0-441-17271-7
Format: Mass market
Pages: 537
The Atreides family, distant cousins to the imperial family, have ruled the planet of Caladan for twenty generations. Caladan is a wet farming world, comfortable and pleasant, but not horribly important. But House Atreides is feuding with House Harkonnen, and, at the start of Dune, that feud maneuvers Duke Leto into giving up his holdings and moving his family to take possession of Arrakis. Arrakis is a desert planet, previously controlled by Baron Harkonnen. It is unrelentingly hostile, home to smugglers and dangerous local desert dwellers called Fremen. But it's also one of the most important planets in the galaxy, since it's the sole origin of the chemical called melange, or spice. Spice permits a limited form of prescience, which allows the navigators of the Spacing Guild to successfully steer ships across the interstellar void. Arrakis's production of spice is what makes interstellar travel, and therefore all of interstellar civilization, possible. Dune is the story of Paul Atreides, son and heir to Duke Leto Atreides. His mother, Lady Jessica, is one of the Bene Gesserit, a secretive order of women devoted to mental and physical discipline and to the long-term genetic improvement of mankind. He is not supposed to exist; Lady Jessica was supposed to only bear a daughter of Leto. But he may be something special, the long-sought (but also dangerous) Kwisatz Haderach who can unite male and female Bene Gesserit powers. The Bene Gesserit take great interest in him from the start of Dune. More surprisingly, so do the Fremen of Arrakis; from the moment he arrives there, he seems to be fulfilling prophecies of theirs that are partly, but not entirely, ones planted by the Bene Gesserit long ago. The feud with the Harkonnens, the unstable place of Arrakis in galactic politics, the dreams of the Fremen and the Imperial ecologist on Arrakis of terraforming, Bene Gesserit plans, Paul's abilities, and the legends of the Fremen all combine in a complex mix of politics, battle, and clashes of culture. Dune is an acknowledged SF masterpiece, one of the best-known classics of the genre. It's usually found in short lists of the best SF novels ever written. It spawned five sequels by Frank Herbert (about which more in a moment), as well as numerous additional sequels and prequels by Kevin J. Anderson and Brian Herbert. It's been adopted for the screen twice, not to mention board games, video games, and numerous other projects. This is my second reading, the first in about twenty years, but the story was still immediately familiar from having seen films and having discussed and read about the universe. This is not science fiction in any strict sense. Dune is science fiction in the same way that Star Wars is: a futuristic gloss on top of power structures inspired by feudalism, heavily mixed with mysticism, mental powers, magic, and implausible but convenient science that creates the story effects the author wants. Both Bene Gesserit powers in general and Paul's abilities in particular are effectively magic. There is some hand-waving explanation of their ability to verbally control other people as taking advantage of specific pitches and intonations that people are vulnerable to, but it's effectively spell-casting (and is a direct inspiration for Jedi mind tricks). All of the mysticism (and there's quite a lot of it in Dune, including race memory, precognition, and even molecular transformation) resembles the Force from Star Wars more than anything scientific. Dune is epic fantasy told on a science fiction stage, complete with a young protagonist coming into his powers and dangerous and sometimes hostile mentors. What Dune gets right, and what has put it so high in the pantheon of great science fiction, is the world building. Herbert sets the story tens of thousands of years into the future of humanity and then effectively projects the feeling of deep history over everything in the novel. This is the kind of book that has appendices with more background information; more to the point, it's the kind of book where you may actually read them out of curiosity. Mankind has a vast interstellar empire (Herbert's universe, like Asimov's Foundation universe, admits no aliens) governed by a system akin to the early British monarchy. An emperor rules in balance with the Great Houses, who meet in a sort of parliament. But against both is a third force: the Spacing Guild, who maintains a monopoly over all interstellar travel. (And the Bene Gesserit form an underground, secretive fourth power base.) Herbert plays with vast swaths of time and great forces of history as well as very good epic fantasy and better than nearly all SF I've read. The detailed world-building is equally good. Nearly all of Dune takes place on the desert planet of Arrakis, which has a lovingly-described ecology and local culture built entirely around scarcity of water. (The details of that ecology are much of the plot and mystery of the book, so I won't spoil them further.) While I doubt the precise details hold up to close scientific scrutiny, this is an obvious precursor to the great ecological stories of later SF, such as Kim Stanley Robinson's Mars trilogy. The details all feel right and hang together in satisfying ways, while also generating the great Sand Worms of Arrakis, a key ingredient in several of the best set pieces in the history of SF. This is the sort of book where the fascinating details and discoveries about the world do as much to keep one turning the pages as the plot, although the plot is also satisfyingly twisty and tense. Unfortunately, Dune doesn't get everything right. The amount of mysticism involved is a bit much, and at times the drug-trip mystical experiences of viewpoint characters turn into excessively purple prose and nearly incomprehensible descriptions. Those mystical experiences also involve race and genetic memory, a concept that's just scientific enough to be unbelievable. A few of the other scentific cheats are also rather blantant; for example, Herbert constructs an elaborate, artificial technology of shielding that seems designed primarily as an excuse to add sword combat to a futuristic story, and I have always struggled to suspend disbelief about the way lasers and shields interact in Dune. The Spacer Guild's monopoly on interstellar travel can be explained; their monopoly on local orbital space, or even the high stratosphere, both vital to allow certain things on Arrakis to remain secret, are much more dubious. Herbert mostly doesn't try to explain these things, and as with Star Wars the less explained the cheats are, the better they work as part of the story. But the technological background doesn't hold up against much examination. Worse, for me, is the general quality of the writing. Herbert does some things very well, such as world-building, and avoids awkward infodumps. Characterization and pacing are both fairly solid; he does a good job with Paul and Jessica in particular, and I've always liked the Fremen. But he wants to put the reader in everyone's head, frequently by giving character thoughts as italicized dialogue, and to enable that he uses a perspective that I always find distracting. Most fiction is written in tight third person. This means that the viewpoint character for any given section of the book is referred to in the third person, like all the other characters, but the reader has special access to their thoughts and emotions. We get to know what they're really thinking and feeling, not just the impressions they give to others, while the non-viewpoint characters are shown only from external appearances and the thoughts of the viewpoint character. Some books hold to the same viewpoint character throughout, but more commonly books move between viewpoint characters at scene breaks to provide more angles on the book's events. First person, in which the story is told by a specific character as if they were telling a story or writing it down, is the most common alternative. Third person objective, in which we don't get any special insight into the internal thoughts of any of the characters, is less common but still unsurprising. Dune does not use any of those perspectives. Instead, Dune uses wandering third-person omniscient, in which we get the inner thoughts and emotions of a character in a scene and then a few lines later the inner thoughts and emotions of a different character. This is the sort of thing that may or may not bug you depending on how much you've read, how deep the expectations of perspective are ingrained, and how much you notice perspective. It drives me nuts. I subconsciously align with the viewpoint character of a section, and pay attention to the ways that authors indicate which character will be the viewpoint character at the start of a scene. Herbert's constant flitting from character to character makes me dizzy. We get the verbatim thoughts of everyone almost indiscriminately, making me feel like I'm randomly hopscotching through the scene. For me, this does two things: it hurts my ability to get engrossed in the story, since I'm constantly thrown out of my normal reading mode when the viewpoint unexpectedly shifts, and it makes the writing feel repetitive. One keeps hearing about the same thing from multiple perspectives, and at times the story bogs down in everyone's internal dialogues rather than showing character reactions and letting the reader draw their own conclusions. I think it tries for a cinematic perspective, but ends up making the story feel muddled. The other flaw, which I didn't notice originally but which leaped out at me during this re-read, is that Herbert's world-building uses quite a few stereotypes. The most notorious, and most widely discussed, is of course the Fremen. Herbert draws heavily on Arab and Islamic culture even beyond the obvious similarities of people living in a harsh, arid climate. He borrows some rather loaded terms and cultural markers, such as jihad, to construct a culture of potential religious fanatics. This is not all bad; the Fremen are clearly portrayed as the good guys, which is a refreshing change from more typical current portrayals of Islam. But it becomes clear that they have aligned their entire culture around influences from outside, and the whole plot of Dune can be fairly characterized as an instance of "what these people need is a white man." Paul (and Kynes before him) joins their culture as well, but Paul becomes a better native than the natives, while simultaneously bringing his outside perspective. It's the sort of plot that is more widely noticed today than it would have been in 1965. Another major example of this, and one that I found more blatant, is that Herbert turns the Harkonnen into hissable, one-sided villains and uses some nasty stereotypes to do it. The insane torturer is consistently and repeatedly described as effeminate, fat is used as a marker of moral inferiority and evil, and the primary villain is homosexual and prefers drugged young male slaves. Here too, this sort of characterization short-cut was more common in 1965, but it's not appealing and makes the (already rather camp) scenes set among the Harkonnen even less enjoyable. Less clear-cut is the way women are handled throughout Dune. I do have to give Herbert some credit, particularly for the era in which he was writing. There are powerful female characters in Dune, including both Jessica and Alia, who have their own independent power and successfully pursue their own agendas throughout. The effectively all-female Bene Gesserit is a major political power in the story and is treated by the other players with respect as well as fear. But it's hard not to also notice the general position of women as subservient to men, not only in the general culture of the Great Houses but also in the more positively-portrayed Fremen culture. Indeed, the subservience of women is even worse in Fremen culture, where they're treated like property and where being killed by a woman is a sign of shame. Again, Herbert deserves some credit for doing better than a lot of 1960s fiction, but the sexism fairy has still been at work here. None of these flaws change the fact that Dune is a masterpiece. Herbert brings together history, world building, ecology, politics, and a compelling coming-of-age story about a messiah figure into a fast-paced, sweeping epic with a thoroughly satisfying conclusion. I think they do make it a flawed masterpiece, but it's still one of those SF novels that everyone should read at least once. Sadly, it's also a masterpiece that I think has suffered from its own success in the form of sequels, prequels, and a ton of supporting material. This is one of the problems that truly excellent world building can lead to. Human history is fractal: any specific detail can be examined in more depth and will usually lead (provided that information is available at all) to even more fascinating detail. The best world building conveys that impression of depth. That's what Herbert achieves here with hints, notes, and asides: the sense that galactic history is a vast ediface with the same fractal complexity as real human history. It makes for a compelling background, but it also inspires people to dig into that background and flesh out all of the details the way that we do with human history. But this doesn't actually work; invented history created by one person simply cannot be fractal in the same way. Human history is endlessly complex because it was generated by the complex interactions of many people. Invented history is an illusion that hints at complexity by building the same surface, but one mind, or even a small number of minds, cannot generate the same depth. The result is that if one digs too deep, one removes that convincing surface and ends up with a mundane, simplistic, and unsatisfyingly fake set of events. I think that's what's happened with all of the supporting material that's been written around Dune since its original publication. Dune is of a piece, a single story that's deeply enjoyable on its own terms and leaves the reader with a satisfying impression of complexity. The systemic excavation of that complexity lessens it and reveals too much of the illusion. Yes, I want to know more about the Butlerian Jihad, but that's the point: the wanting is the sign of succesful crafting of imagined history. Reading the definitive account is more likely to leave me unsatisfied than to lead to the recursive curiosity that human history can create. The sequels to Dune written by Herbert himself are, for me, another matter. Some reviewers level the same criticism at them: that Herbert dives too far into background best left unexplored. But they have the advantage of moving forward, telling more of the story set off by Paul, and the end of Dune is a clear setup for a sequel. One of Paul's goals throughout most of the book has been left unaccomplished. I don't think Herbert dove too deep into his creation; rather, my problem with his sequels (all of which I've read, although it's been some years now) is that he took the story in a direction that I actively disliked and found painful to read. Regardless, the general consensus is that the sequels aren't as good as the original, and while Dune doesn't fully resolve its story, it's complete enough that it's possible to stop here. Stopping is the general recommendation, although I still may re-read and review the sequels at some point. Followed by Dune Messiah. Rating: 8 out of 10

23 November 2012

Benjamin Mako Hill: Cultivated Disinterest in Professional Sports

Like many of my friends, I have treated professional sports with cultivated indifference. But a year and a half ago, I decided to become a football fan. Several years ago, I was at a talk by Michael Albert at MIT where he chastised American intellectuals for what he claimed was cultivated disdain of professional sports. Albert suggested that sports reflect the go-to topic for small talk and building rapport across class and context. But he suggested that almost everybody who used the term "working class struggle" was incapable of making small talk with members of the working class because unlike most working class people (and most people in general) educated people systematically cultivate ignorance in sports. Professional sports are deeply popular. In the US, Sunday Night Football is now the most popular television show among women in its time slot and the third most popular television in America among 18-49 year old women. That it is also the most popular television show in general is old news. There are very few things that anywhere near half of Americans have in common. Interest in football is one of them. An enormous proportion of the US population watches the Superbowl each year. I recognized myself in Albert's critique. So I decided to follow a local team. I picked football because it is the most popular sport in America and because their strong revenue sharing system means that either team has a chance to win any given match. My local team is the New England Patriots and I've watched many of the team's games or highlights over the last season and a half. I've also followed a couple football blogs. A year and half in, I can call myself a football fan. And I've learned a few things in the process:
  1. With a little effort, getting into sports is easy. Although learning the rules of a sport can be complicated, sports are popular because people, in general, find them fun to watch. If you watch a few games with someone who can explain the rules, and if you begin to cheer for a team, you will find yourself getting emotionally invested and excited.
  2. Sports really do, as Albert implied, allow one to build rapport and small talk across society. I used to dread the local cab driver who would try to make small talk by mentioning Tom Brady or the Red Sox. No more! Some of these conversations turn into broader conversations about life and politics.
  3. Interest in sports can expand or shrink to fill the time you're willing to give it. It can mean just glancing through the sports sections of the paper and watching some highlights here or there. Or it can turn into a lifestyle.
  4. It's not all great. Football, like most professional sports, is deeply permeated with advertisements, commercialism, and money. Like other sports, it is also violent. I don't think I could ever get behind a fight sport where the goal is to hurt someone else. The machoness and absence of women in the highest levels of most professional sports bothers me deeply.
I've also tried to think a lot about why I, like most of my friends, avoided sports in the past. Disinterest in sports among academics and the highly educated is, in my experience, far from passive. I've heard people almost compete to explain the depth of their ignorance in sports one doesn't even know the rules, one doesn't own a television, one doesn't know the first thing about the game. I did the same thing myself. Bethany Bryson, a sociologist at JMU has shown that increased education is associated with increased inclusiveness in musical taste (i.e., highly educated people like more types of music) but that these people are most likely to reject music that is highly favored by the least educated people. Her paper's title sums up the attitude: "Anything But Heavy Metal". For highly educated folks, it's a sign of cultivation to be eclectic in one's tastes. But to signal to others that you belong in the intellectual elite, it can pay in cultural capital to dislike things, like sports, that are enormously popular among the least educated parts of society. This ignorance among highly educated people limits our ability to communicate, bond, and build relationships across different segments of society. It limits our ability to engage in conversations and build a common culture that crosses our highly stratified and segmented societies. Sports are not politically or culturally unproblematic. But they provide an easy and enjoyable way to build common ground with our neighbors and fellow citizens that transcend social boundaries.

Keith Packard: MicroPeak

MicroPeak tiny peak-recording altimeter now available MicroPeak is a miniature peak-recording altimeter. About the same size and weight as a US dime (with battery), MicroPeak offers fabulous accuracy (20cm or 8in at sea level) and wide range (up to 31km or 101k ). The size of the board was predicated with the premise that we needed a battery included to avoid having wiring running between the altimeter and the board, we found some small lithium coin-cell battery holders for the CR1025 battery. These battery holders are rated to hold the battery secure up to 150gs. We d already started playing with the Measurement Specialties MS5607 pressure sensor which offers amazing accuracy while using very little power. Taking full-precision measurements every 96ms consumes about .2mA on average. Once on the ground, we stop taking measurements entirely, dropping the power use to around 1 A. It s also pretty small, measuring only 5mm x 3mm. For a CPU, this little project didn t need much. The 8-bit ATtiny85 comes in a 20qwfn package which is only 4mm x 4mm. When run at full speed (8MHz), it consumes a couple of mA of power. Reduce the clock to a pokey 250kHz and the CPU has enough CPU power to track altitude while consuming less than .2mA on average. To avoid losing the battery, we wanted to avoid having it removed while the board wasn t in use. So, we added a little power switch to the board. The one we found is good to at least 50g. Finally, we wanted to find a nice bright LED to show the state of the device and to blink out the final altitude. The OSRAM LO T67K are bright-orange surface-mount LEDs that run happily on 2mA. We used OSHPark.com to create prototype circuit boards for this project. Because of the small size of the board, each prototype run cost only $2 for three boards. It takes a couple of weeks to get boards, but it s really hard to beat the price. All of the schematic and circuit board artwork are published under the TAPR Open Hardware License and are available via git. All of the source code is published under the GPLv2 and is included in the main AltOS source repository.

21 November 2012

Dirk Eddelbuettel: Rcpp attributes: A simple example 'making pi'

We introduced Rcpp 0.10.0 with a number of very nice new features a few days ago, and the activity on the rcpp-devel mailing list has been pretty responsive which is awesome. But because few things beat a nice example, this post tries to build some more excitement. We will illustrate how Rcpp attributes makes it really easy to add C++ code to R session, and that that code is as easy to grasp as R code. Our motivating example is everybody's favourite introduction to Monte Carlo simulation: estimating . A common method uses the fact the unit circle has a surface area equal to . We draw two uniform random numbers x and y, each between zero and one. We then check for the distance of the corresponding point (x,y) relative to the origin. If less than one (or equal), it is in the circle (or on it); if more than one it is outside. As the first quadrant is a quarter of a square of area one, the area of the whole circle is -- so our first quadrant approximates over four. The following figure, kindly borrowed from Wikipedia with full attribution and credit, illustrates this: Example of simulating pi Now, a vectorized version (drawing N such pairs at once) of this approach is provided by the following R function.
piR <- function(N)  
    x <- runif(N)
    y <- runif(N)
    d <- sqrt(x^2 + y^2)
    return(4 * sum(d < 1.0) / N)
 
And in C++ we can write almost exactly the same function thanks the Rcpp sugar vectorisation available via Rcpp:
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
double piSugar(const int N)  
    RNGScope scope;		// ensure RNG gets set/reset
    NumericVector x = runif(N);
    NumericVector y = runif(N);
    NumericVector d = sqrt(x*x + y*y);
    return 4.0 * sum(d < 1.0) / N;
 
Sure, there are small differences: C++ is statically typed, R is not. We need one include file for declaration, and we need one instantiation of the RNGScope object to ensure random number draws remain coordinated between the calling R process and the C++ function calling into its (compiled C-code based) random number generators. That way we even get the exact same draws for the same seed. But the basic approach is identical: draw a vector x and vector y, compute the distance to the origin and then obtain the proportion within the unit circle -- which we scale by four. Same idea, same vectorised implementation in C++. But the real key here is the one short line with the [[Rcpp::export]] attribute. This is all it takes (along with sourceCpp() from Rcpp 0.10.0) to get the C++ code into R. The full example (which assumes the C++ file is saved as piSugar.cpp in the same directory) is now:
#!/usr/bin/r
library(Rcpp)
library(rbenchmark)
piR <- function(N)  
    x <- runif(N)
    y <- runif(N)
    d <- sqrt(x^2 + y^2)
    return(4 * sum(d < 1.0) / N)
 
sourceCpp("piSugar.cpp")
N <- 1e6
set.seed(42)
resR <- piR(N)
set.seed(42)
resCpp <- piSugar(N)
## important: check results are identical with RNG seeded
stopifnot(identical(resR, resCpp))
res <- benchmark(piR(N), piSugar(N), order="relative")
print(res[,1:4])
and it does a few things: set up the R function, source the C++ function (and presto: we have a callable C++ function just like that), compute two simulations given the same seed and ensure they are in fact identical -- and proceed to compare the timing in a benchmarking exercise. That last aspect is not even that important -- we end up being almost-but-not-quite twice as fast on my machine for different values of N. The real takeaway here is the ease with which we can get a C++ function into R --- and the new process completely takes care of passing parameters in, results out, and does the compilation, linking and loading. More details about Rcpp attributes are in the new vignette. Now enjoy the .

Update:One somewhat bad typo fixed.

Update:Corrected one background tag.

Axel Beckert: zutils: zcat and friends on Steroids

I recently wrote about tools to handle archives conveniently. If you just have to handle compressed text files, there are some widely known shortcut commands to mimic common commands on files compressed with a specific compression format.
gzip bzip2 lzma xz
cat zcat bzcat lzcat xzcat
cmp zcmp bzcmp lzcmp xzcmp
diff zdiff bzdiff lzdiff xzdiff
grep zgrep bzgrep lzgrep xzgrep
egrep zegrep bzegrep lzegrep xzegrep
fgrep zfgrep bzfgrep lzfgrep xzfgrep
more zmore bzmore lzmore xzmore
less zless bzless lzless xzless
In Debian and derivatives, those tools are part of the according package for that compression utility, i.e. the zcat command is part of the gzip package and the xzfgrep command is part of the xz-utils package. But despite this matrix is quite easy to remember, the situation has a few drawbacks: This is where the zutils project comes in: zutils provides the functionality of most of these utilities, too, but with one big difference: You don t have to remember, think about or type which compression method has been used for your data, just use zcat, zcmp, zdiff, zgrep, zegrep, or zfgrep and it works independently of what compression method has been used if any or if there are different compression types mixed in the parameters to the same command:
$ zfgrep foobar bla.txt fnord.gz hurz.xz quux.lz bar.lzma
Especially if you use logrotate and let logrotate compress old logs, it s very comfortable that one command suffices to concatenate all the available logfiles, including the current uncompressed one:
$ zcat /var/log/syslog*    
Additionally, zutils versions of these tools also support lzip-compressed files. The zutils package is available in Debian starting with Wheezy and in Ubuntu since Oneiric. When being installed, it replaces the original z* utilities from the gzip package by diverting them away. The only drawback so far is that there neither a zless nor a zmore utility from the zutils project, so zless bla.txt fnord.gz hurz.xz quux.lz bar.lzma will not work as expected even after installing zutils as it is still the one from the gzip package and hence it will show you just the first two files in plain text, but not the remaining ones.

20 November 2012

Pablo Lorenzzoni: Decision-making by flipping a coin

Yesterday I was discussing in an online board how to break a decision deadlock in life. There are real deadlocks, but I don t think they are very frequent most of the time, doing a pros and cons analysis is enough to decide what to do. Sometimes our judgement is impaired by lack of objectivity or by our inability to see things from a different point of view (it s hard to think straight when everything seems to be falling apart around you); talking to a friend or relative can help in these situations. But there are times when nothing helps. When you are really stuck and nothing seems to break the deadlock. At these times I give it one or two days, sleep on the issue and if I cannot come up with a decision I assume the alternatives are equivalent to me and just flip a coin. Of course, this will do if you can postpone the decision, giving yourself (and your inner self) time to decide on a course-of-action. But is there some way to speed up the process? During the discussion, someone came up with a rather smart quote by Rothstein character in Boardwalk Empire TV series:
Flip a coin. When it s in the air, you ll know what side you re hoping for.
I found it an interesting way to give your inner self an ultimatum: decide or luck will decide instead. I ve never did it like that, for I would not flip a coin unless it s my last resort. Being such a clever psychological idea, I doubted it originated in the TV series itself, so I went on pursue of the original idea. I found a Danish poet and mathematician called Piet Hein, who wrote a poem about it circa 1969:
A PSYCHOLOGICAL TIP Whenever you re called on to make up your mind,
and you re hampered by not having any,
the best way to solve the dilemma, you ll find,
is simply by spinning a penny.
No not so that chance shall decide the affair
while you re passively standing there moping;
but the moment the penny is up in the air,
you suddenly know what you re hoping.
And also, there are a Donald Duck comic called Flip Decision circa 1953 that introduces Flipism philosophy, which supports a rather radical anecdotal variant: make all decisions by flipping a coin. Piet Hein or Rothstein character idea is much better than Flipism, of course. Does anyone knows any older sources of the same idea?

Next.

Previous.