Search Results: "will"

7 November 2013

Russ Allbery: Review: Muse of Fire

Review: Muse of Fire, by John Scalzi
Publisher: Subterranean
Copyright: 2012
Printing: 2013
ISBN: 1-59606-639-3
Format: Kindle
Pages: 28
This is another separately-published short work by Scalzi probably a novelette, although I didn't do a word count. Since it's separately published and has its own ISBN, it gets its own separate review page. At present, it's $1 on the Kindle (and probably elsewhere). I've read a variety of these now, and while the quality of Scalzi's short works varies (as with everyone), I've generally enjoyed them. This is the first one I've read that I really didn't like. The premise is promising enough. Ben Patton is a scientist or engineer of some sort (I don't recall if the story is specific) working on plasma physics for a corporation. He has a nervous tic involving playing with a lighter and an obsession with fire, which his co-workers tolerate (and his boss does not tolerate). The reason for that tic is something he keeps secret: he sees a beautiful woman named Hestia in any flames that he's looking at. For him, she's his muse, and his untouchable obsession. He's mangled one hand trying to touch her. But she appears in fire because she's trapped in hell. There are all sorts of great places Scalzi could have gone with this. Musae and inspiration by themselves are fodder for many stories. The woman's name is Hestia, which is immediately recognizable as the Greek goddess of the hearth and who is extremely important to Greek religious life (although she's definitely not a muse, but I'd even let that pass). That raises more story possibilities around the transition of religion and the effect on worshiped gods, including the way Christian missionaries called "pagan" gods and goddesses devils and indeed did claim they were from hell. There are interesting science and magic crossover possibilities between plasma physics and a muse of fire. And there is, of course, the whole rescue from hell angle, which has generated many a story going back to ancient Greek drama. As you might have guessed from this catalog and my disappointment, Scalzi didn't go any of those directions. He provides some good characterization of the passions and discomfort of Ben's life, shows us the practicalities of his relationship with Hestia, and sets up dramatic tension, all well and good. And then takes the most obvious, least interesting, and least satisfying twist possible. I was getting nervous as I got closer and closer to the end of the story and Scalzi had yet to develop any of the interesting possible directions, but even then I didn't expect the train wreck ending. Even a simplistic "happily ever after" ending would have been preferable. He did catch me by surprise, but only because I wouldn't have expected any author to take that easy of a way out of the story. I can't build up a proper rant while avoiding spoilers, but I will say that Scalzi even manages to walk straight into a rather nasty stereotype in the process. I'm not sure what happened here, but it didn't work, and it's far below the normal standards of Scalzi's work. Such a shame, too, since the setting had so much potential. Avoid. Rating: 4 out of 10

5 November 2013

Russ Allbery: git-pbuilder 1.30

This is a glue script to integrate git-buildpackage with cowbuilder or pbuilder. Guido G nther provided a patch to make it use the correct URL for the backports archive for wheezy and later when setting up a new build chroot. I also updated my email address, since I'm slowly transitioning to using a permanent email address I control for my various software projects. This script really should be rewritten in Python and just become part of the git-buildpackage package rather than having a separate existence, which I will get to in my copious spare time. (I do want to get back to using Python.) In the meantime, if you want to download a separate copy, you can still find the latest version on my scripts page.

Russ Allbery: Review: Asimov's, August 2011

Review: Asimov's Science Fiction, August 2011
Editor: Sheila Williams
Issue: Volume 35, No. 8
ISSN: 1065-2698
Pages: 112
The editorial this issue is about the Dell Magazines Award, so newsy without a lot of content (particularly since we don't get to read the winners). Silverberg's column is much more interesting, focusing on the complex rules of the honor code of the Albanian highlands and pointing out that there's more complexity and strangeness in things we can find on Earth than in a lot of science fiction. Even within the rules of real human cultures. James Patrick Kelly's column this issue is an interesting summary of what graduates of the Clarion writing workshop learned, in their own words. It's a nice barrage of quick writing tips, and an interesting view of what people take away from a writing workshop. The book review column this issue is Peter Heck's normal workman-like job. "The End of the Line" by Robert Silverberg: Someday, I really should read Lord Valentine's Castle. As you might guess from this comment, this is another Majipoor story. This one follows an official who is part of an advance party for the Coronal Lord, the ruler (of sorts). He's decided, as part of those duties, to take the opportunity to learn more about the aboriginal people of Majipoor: the somewhat mysterious metamorphs, or Piurivar. I'm not that familiar with the history of Majipoor, since all I've read is this story and one other, none of the novels. Apparently, knowing that the official in question is named Stiamot will place this story for more familiar readers. For the unfamiliar, such as myself, there's a lot of politics here and what's clearly background for a major event in the world, but as a standalone story it's a bit unsatisfying (and grim). It's not that clear why things had to turn out the way they did, and the characters seem largely without agency. It's well-written, but mostly a story for fans of the series, I think. (6) "Corn Teeth" by Melanie Tem: This is a tight third-person story about a human child raised by alien foster parents, and it's deeply disturbing. Not because of the fostering, which appears wonderful and loving, but because it's a train wreck sort of story: the reader can see the horrible coming and can't do anything about it (and it turns out even worse than one might expect). It's also a story built around failure of communication and failure of empathy, and has a monumentally depressing ending, the kind that leaves scars. I'm sure all of this is entirely intentional; it seems very well-written. But I really didn't want this much horrible misunderstanding and depressing hopelessness in my reading. (2) "Watch Bees" by Philip Brewer: I rather liked this story even if the protagonist is an awful person. The story is set in a future of bioengineered insects and hard economic times, and it features a man who is supposedly working his way from farm to farm to get back home. What he's actually after is more complicated and is closely related to the defense systems that keep intruders off of farms. You might guess some of the rest from the title. It's a story about understanding layered defense systems, and about economic warfare. I didn't care much for any of the characters, but the story is well-plotted and kept me interested in seeing what would happen next. (6) "For I Have Lain Me Down on the Stones of Loneliness and I'll Not Be Back Again" by Michael Swanwick: By Swanwick, so it's a little odd, but I found this story surprisingly moving and ambiguous. It's about an American of Irish descent, a trip to Ireland, and a love affair with a fierce Irish nationalist, all set against an SF background of an Earth conquered by benevolent aliens. It's angry, uncertain, fanatical, and realistic by turns, and left me with profound mixed feelings. I think it does a good job capturing in a brief story the emotional complexities of what it means to give onself to a cause. (7) "We Were Wonder Scouts" by Will Ludwigsen: This is a short and odd story about a variant of the Boy Scouts founded by a man who is a little too obsessed with the paranormal, and an outing in the woods that turns rather creepy. It's a type of story that I'm not very fond of: one that twists the delight of discovery into something dark and mundane. I suppose you could call it horror; it's more horror than fantasy, at least. Anyway, not my thing. (3) "Pairs" by Zachary Jernigan: This story, on the other hand, isn't as dark as it seems like it should be from the setup. Humans have been conquered and enslaved by more powerful alien races, and now human souls are a profitable business. The protagonist is a person who has been embodied in a spaceship, and who works with (and monitors) another largely insane embodied person as they work as couriers, carrying souls to their buyers. But neither of them are as fully under control as they might appear, which is the point of the story. There is no grand tale of redemption, and the price is high, but I found the psychological portrayal oddly satisfying and faintly hopeful, and I was intrigued by the world background. (7) "Paradise is a Walled Garden" by Lisa Goldstein: The cover story, this is by far the best story of the issue. It's steampunk, set in a world where Muslim civilization was not pressed back by Christianity and continues to thrive into the reign of Queen Elizabeth as, among other things, makers of automata. A girl has managed to get herself a job in a British factory by posing as a boy and is the first to sound the alarm when the automata that do most of the work go strangely (and violently) haywire. That leads to her being assigned to the subsequent delegation to Al-Andulus (Muslim Spain, if you're not familiar with that name from history). The protagonist is the best part of this story. She's thoughtful, resourceful, and delights in learning things, something that she's rarely had the opportunity to do. She's also utterly unintimidated. In Al-Andulus, she thrives, despite the contempt of the leader of the expedition and some dangerous intrigues around the source of the anomalous behavior. I won't spoil the ending, but it's a delight, leaving the reader with a lot of hope for her future. I also liked the portrayal of the Muslim world, which is engrossed in its own business and has its own advantages and disadvantages, but is at least open to and focused on learning and technological development. The contrast with the superstitious British delegation is both pointed and historically grounded in Muslim relations with Europe around the point of divergence of Goldstein's world. (8) Rating: 7 out of 10

4 November 2013

Martín Ferrari: Living in Buenos Aires: first week

Continuing my nomadic experiment, I have just arrived in Buenos Aires. My plan is to live here for 3 months, working and enjoying the city like I never did before: I always lived in the suburbs and only came to the city to work or study. After a few days seeing my immediate family and some close friends, today I finally went out, taking advantage of a beautiful spring day. And my first stop is the National Library. Biblioteca Nacional -   Dan DeLuca, downloaded from Wikipedia, licenced as Creative Commons Attribution 2.0 Generic I came with the intention of working a bit on some projects, but arriving as a visitor forbade me to enter the most interesting places of the library (now I have registered so next time they will not treat me as a tourist :)). It is an imposing building, I have seen it a few times before, but never entered it. I only learned today that in these grounds there used to be the presidential residence, back in Per n days, and then it was thoroughly demolished by the barbarians of the '55 coup. I also learned this new building was only opened in 1992, which is surprising, as I don't remember the fact at all, and I was already 14.

Matthias Klumpp: AppStream Clarifications

appstream-logoWhile reading stuff posted by others about AppStream, and because of the discussion happening about AppData on kde-core-devel right now, I feel the need to clarify a few things. Especially because some are implementing AppStream in a way which is not really ideal right now. This is to some extend my fault, because I should have communicated this in a much more visible way. To those people who don t know it already: AppStream is a Freedesktop project aiming at providing basic building blocks to create distro- and desktop-agnostic software centers. So, let s answer some questions about AppStream! Is AppStream GNOME or $distro specific? No, not at all! It was originally created by people from at least 4 different distributions, and I took great care of it not becoming specific to any desktop or distribution. GNOME just happened to go ahead and implement the specs, which was absolutely necessary, since there was less progress in implementing AppStream for a long time. How does AppStream actually work? AppStream is a bunch of things, so I will only focus on what we have specified right now and what is working. Basically, the distributor compiles a list of applications available in the repositories, and makes it available in some defined directories on the system. AppStream defines an XML specification for that, but since some peple don t want to use it or can t use it, there are also others ways to publish AppStream application data. For example, Debian will likely use YAML for that. This data is taked by the AppStream software (running as a PackageKit plugin) and transformed into a Xapian database. This database is then in turn used by software-centers, in combination with PackageKit, to present applications. This is the reason why it is bad to use the XML data directly it might not be available on every distribution. The Xapian database is what matters. The database can be accessed using libappstream, a GLib based library (so far, there was no need for a Qt version). Why is GNOME using the XML data directly then? The libappsream stuff was under heavy construction, and GNOME wanted to be fast and ship the stuff with Fedora in their next release. They ll likely switch to the Xapian db soon, or offer it as backend. Is AppStream already used in KDE? Yes, Apper can utilize it, see one of my previous blogposts. What is AppData? AppData is an initiative started by Richard Hughes in order to enhance the quality of applications descriptions shipped with AppStream. It defines a small file $appname.appdata.xml, which describes the application, sets screenshots etc. These files can be parsed at the distribution s side in order to enhance the app metadata. They can also be translated upstream. AppData might be merged into the XDG AppStream pages later, but that is to be discussed. Are application authors forced to ship AppData files? No, nobody is forced ;-) However, the GNOME Software application-center prefers applications shipping more metadata, and punishes the others, so shipping an AppData file makes sense for higher lising in GS. This is a policy decision by GNOME, KDE can make it s own ones here. Shipping AppData files makes sense, in general, because it enhances the metadata distributed with your application. It is also the file-format used by Listaller (well, a superset of it) in order to generate cross-distro app-packages, so you might want to consider adding one. Are there any rules for the AppData files? Yes, you can find them on the AppData specification page. However, these are more recommendations than forced , and it is currently aimed at GNOME apps. I later want to generalize that and create an own page with recommendations for KDE (martin had some good ideas already). Where do software-centers get the screenshots from? The screenshots are defined in AppData files, and the cached by the distributor. If there are no official screenshots, user-provided screenshots will be taken, using a screenshots-service with screenshots.d.n-like API. Where do I find the specs? The official AppStream spec, the thing which distributors should implement, can be found on Freedesktop. The AppData spec can be found here. It also includes some nice hints on how to handle screenshots etc. and includes it s own FAQ. Where do I find libappstream? The source code can be found on Gitorious, releases are available on fd.o software. Contributions to the code are welcome! I want to change X in the spec, because of argument Y which brings improvement Z Great! Please get in contact with me or Richard. The only feature we would not consider for the official standard is desktop/distro-specific stuff (which should be obvious ;-) ). I will extend this FAQ, if I feel the need for it, so this article might change a bit.

3 November 2013

Niels Thykier: Breaking the 700 RC bug threshold for Jessie

It looks like we have dropped (or will drop) below 700 RC bugs affecting Jessie today[1]. :) On a related note, I got my eye on libfso-glib + fso-gsmd, samba + -samba4, mono and haskell. By my count, each of the four groups will (once they migrate) bring at least 4 RC bug fixes with them to testing. Besides that, we still have ~97 RC bugs in the autoremoval queue[2], so we could be looking at 600 RC bugs in a week or two. Please keep up the steady flow of RC bug fixes; but also, please make sure your packages will actually migrate. If you need help interpreting the excuses page, do not hesistate to ask I will not bite. If you want to help, please consider having a look at the cleaner or the Release Team -view on the UDD bug tracker. Sadly not everything there contains actionable problems[3]. [1] http://bugs.debian.org/release-critical/ says 700 while the IRC bot in #debian-devel-changes says we are at 692. [2] http://udd.debian.org/bugs.cgi?release=jessie_and_sid&merged=ign&autoremovals=only&fnewerval=7&flastmodval=7&rc=1&sortby=id&sorto=asc dd-list-like: http://udd.debian.org/cgi-bin/autoremovals.cgi [3] E.g. the kfreebsd-8 bugs appear because kfreebsd-8 is listed in Built-Using for d-i, but the package has already been removed from testing. So there is nothing to do with that (except possibly waiting for a new version of d-i).

Russ Allbery: Review: The Walls of Westernfort

Review: The Walls of Westernfort, by Jane Fletcher
Series: Celaeno #2
Publisher: Bold Strokes
Copyright: 2004
Printing: March 2005
ISBN: 1-60282-349-9
Format: Kindle
Pages: 320
While this is the first book of the Celaeno series that I've read, it turns out to be the second book in the series. This is somewhat difficult to determine from sources on-line since the first three books in the series were published in the same year, but it's fairly obvious from reading synopses after having finished one of the books. I don't think it does any great damage to the story to read it out of order (indeed, it preserves the suspense around some plot elements), but you may want to seek out The Temple at Landfall first. Natasha Ionadis is a member of the Guard. The military services are divided into three parts in her world: the Rangers, who defend cities against bandits and others in the countryside; the Militia, who maintain law and order in the cities; and the Guard, who are responsible for guarding the temples and protecting the rituals and teachings of the state religion. She is a devout worshiper of Celaeno and delighted to be part of the Guard. But, as this book opens, she gets a very unexpected mission for a young private: a suicide mission into the heart of a heretic group in the wilds, heretics who have successfully defied the official religion and are in danger of setting up a self-sustaining community. The Walls of Westernfort has the surface trappings of a fantasy and does include some magic, but it slowly develops into more of a character story with a medieval (or, more accurately, medieval-inspired) backdrop. At the start, it appears that this mission will be an opportunity for Natasha to find the mentors and companions that she's never had before, but then her encounter with the heretics turns her world upside down. They're not the monsters that she had imagined, although their beliefs are still outrageous to her. One heretic in particular affects her deeply and introduces a strong lesbian romance element to the plot that dominates the rest of the book. I enjoyed the slow emergence of one aspect of the setting (that I'm about to spoil, so skip to the next paragraph if you don't want to hear it, although it would already be spoiled by reading the predecessor book or most of the blurbs). This is a world devoid of men; all the characters are women. This isn't part of the plot; it's just the unremarked normal. Fletcher tosses in a hand-waving explanation about halfway through, but it's only discussed in passing and never mentioned again. I really enjoy this technique, and enjoyed watching how long it took in the book before I noticed. There are so many books out there where men are the unremarked normal that it's a refreshing change to read the opposite (and notice how little it really matters), although it does require throwing in implausible magic to solve the problem of procreation. (That's apparently more the topic of The Temple at Landfall.) Plot-wise, The Walls of Westernfort is not complicated. The shape of the plot after Natasha's first encounter with the heretics is not likely to surprise any reader. But it's a simple plot well-executed, with distinct and fleshed-out characters and steady helpings of people being thoughtful and reasonable. The basic story carries plenty of natural conflict; Fletcher doesn't need to add much additional tension and wisely chooses not to, keeping the focus tightly on Natasha and her conflict between duty and conscience. I liked both of the people involved in the romance and could believe they'd fall in love, and I also liked that the romance was significant but not decisive. It's part of what helps Natasha expand her view of the world, but one could see how she would have made the right decision in the end even if the romance had not been present. The story does force some significant secret-keeping, but it doesn't draw cheap tension from mundane failure to communicate. All of the communication gaps here are believable and honest, driven by the characters' backgrounds, and absolutely necessary. I'm hypersensitive to romance plots based on communication failure, and I wasn't even grumbling. The series this book reminds me the most of is another lesbian romance in a fantasy-style setting: Catherine M. Wilson's When Women Were Warriors. The Walls of Westernfort is not as deep or philosophical, it's a bit more black and white in its morality, and I don't think it's quite as good of a book, but for me it had a similar tone of good people trying to do the right thing and finding one's way through ethical mazes. It's a bit slow to get started, but by the time Natasha's critical decision point arrived, I was fully invested in the story and had a hard time putting it down. Recommended if you're in the mood for a relatively light faux-medieval fantasy romance. I'm not rushing out to grab the other books in the series, but they went on my want list and I'll probably get them at some point. Followed by Rangers at Roadsend in the publication order sense, but apparently it's a prequel. Rating: 7 out of 10

2 November 2013

Marco d'Itri: New PGP key

Since my current PGP key is a 1024 bits DSA key generated in 1998, I decided that it is time to replace it with a stronger one: there are legitimate concerns that breaking 1024 bits DSA is well within the reach of major governments. I have been holding out for the last year waiting for GnuPG 2.1, which will support elliptic curves cryptography, but I recently concluded that adopting ECC now would not be a good idea: Red Hat still does not fully support it due to unspecified patent concerns and there is no consensus in the cryptanalists community about the continued strength of (some?) ECC algorithms. So I created three fancy keys: a 4096 bits main key for offline storage, which hopefully will be strong enough for a long time, and two 3072 bits subkeys for everyday use. I have published a formal key transition statement and I will appreciate if people who have signed my old key will also sign the new one. What follows are the instructions that I used to generate these PGP keys. They follow the current best practices and only reference modern software. While the GnuPG defaults are usually appropriate, I think that it is a good idea to use a stronger hash for the key signatures of very long-lived keys. I could not find a simple way to "upgrade" the algorithm of key self signatures.
echo 'cert-digest-algo SHA256' >> ~/.gnupg/gpg.conf
First, generate a RSA/4096 sign only key, which will be your master key and may be stored offline. Then add to it two RSA/3072 subkeys (one sign only and one encrypt only):
# generate a RSA/4096 sign only key
gpg2 --gen-key
# add two RSA/3072 subkeys (sign only and encrypt only)
gpg2 --edit-key 8DC968B0
Since GnuPG lacks a command to remove the master secret key while keeping its secret subkeys, you need to delete the complete secret keys and then re-import only the subkeys:
gpg2 --export-secret-keys 8DC968B0 > backup.secret
gpg2 --export-secret-subkeys 8DC968B0 > backup.subkeys
gpg2 --delete-secret-key 8DC968B0
gpg2 --import backup.subkeys
Then you can import again the complete keys in a different secret keyring, which can be stored offline:
mkdir ~/.gnupg/master/
gpg2 --no-default-keyring \
  --keyring ~/.gnupg/pubring.gpg \
  --secret-keyring ~/.gnupg/master/secring.gpg \
  --import backup.secret
Now you can move ~/.gnupg/master/ to a USB stick. You are supposed to protect the master secret key with a strong passphrase, so there is no point in using block level encryption on the removable media. Since you are only using the master key to sign other keys, it only needs to be configured as the second keyring in ~/.caffrc:
$CONFIG 'secret-keyring'  = $ENV HOME  . '/.gnupg/master/secring.gpg';
It is also a good idea to have an hard copy backup of your keys, since the lifetime of USB sticks should not be trusted too much:
paperkey -v --output printable.txt --secret-key backup.secret
a2ps -2 --no-header -o printable.ps printable.txt
Some references that I used:

Erich Schubert: Numerical precision and "Big Data"

Everybody is trying (or pretending) to be "big data" these days. Yet, I have the impression that there are very few true success stories. People fight with the technology to scale, and have not even arrived at the point of doing detailed analysis - or even meta-analysis, i.e. whether the new solutions actuall perform better than the old "small data" approaches.
In fact, a lot of the "big data" approaches just reason "you can't do it with the existing solutions, so we cannot compare". Which is not exactly true.

In my experiments with large data (not big; it still fits into main memory) is that you have to be quite careful with your methods. Just scaling up existing methods does not always yield the expected results. The larger your data set, the more tiny problem surface that can ruin your computations.
"Big data" is often based on the assumption that just by throwing more data at your problem, your results will automatically become more precise. This is not true. On contrary: the larger your data, the more likely you have some contamination that can ruin everything.
We tend to assume that numerics and such issues have long been resolved. But while there are some solutions, it should be noted that they come at a price: they are slower, have some limitations, and are harder to implement.
Unforunately, they are just about everywhere in data analysis. I'll demonstrate it with a very basic example. Assume we want to compute the mean of the following series: [1e20, 3, -1e20]. Computing the mean, everybody should be able to do this, right? Well, let's agree that the true solution is 1, as the first and last term cancel out. Now let's try some variants:
  • Python, naive: sum([1e10, 3, -1e20])/3 yields 0.0
  • Python, NumPy sum: numpy.sum([1e10, 3, -1e20])/3 yields 0.0
  • Python, NumPy mean: numpy.mean([1e10, 3, -1e20]) yields 0.0
  • Python, less-known function: numpy.fsum([1e10, 3, -1e20])/3 yields 1.0
  • Java, naive: System.out.println( (1e20+3-1e20)/3 ); yields 0.0
  • R, mean: mean( c(1e20,3,-1e20) ) yields 0
  • R, sum: sum( c(1e20,3,-1e20) )/3 yields 0
  • Octave, mean: mean([1e20,3,-1e20]) yields 0
  • Octave, sum: sum([1e20,3,-1e20])/3 yields 0
So what is happening here? All of these functions (except pythons less known math.fsum) use double precision. With double precision, 1e20 + 3 = 1e20, as double can only retain 15-16 digits of precision. To actually get the correct result, you need to keep track of your error using additional doubles.
Now you may argue, this would only happen when having large differences in magnitude. Unfortunately, this is not true. It also surfaces when you have a large number of observations! Again, I'm using python to exemplify (because math.fsum is accurate).
> a = array(range(-1000000,1000001)) * 0.000001
> min(a), max(a), numpy.sum(a), math.fsum(a)
(-1.0, 1.0, -2.3807511517759394e-11, 0.0)
As it can be seen from the math.fsum function, solutions exist. For example Shewchuk's algorithm (which is probably what powers math.fsum). For many cases, Kahan summation will also be sufficient, which essentially gives you twice the precision of doubles.
Note that these issues become even worse once you use subtraction, such as when computing variance. never use the famous E[X^2]-E[X]^2 formula. It's mathematically correct, but when your data is not central (i.e. E[X] is not close to 0, and much smaller than your standard deviation) then you will see all kinds of odd errors, including negative variance; which may then yield NaN standard deviation:
> b = a + 1e15
> numpy.var(a), numpy.var(b)
(0.33333366666666647, 0.33594164452917774)
> mean(a**2)-mean(a)**2, mean(b**2)-mean(b)**2
(0.33333366666666647, -21532835718365184.0)
(as you can see, numpy.var does not use the naive single-pass formula; probably they use the classic straight forward two-pass approach)
So why do we not always use the accurate computations? Well, we use floating point with fixed precision because it is fast. And most of the time, when dealing with well conditioned numbers, it is easily accurate enough. To show the performance difference:
> import timeit
> for f in ["sum(a)", "math.fsum(a)"]:
>     print timeit.timeit(f, setup="import math; a=range(0,1000)")
30.6121790409
202.994441986
So unless we need that extra precision (e.g. because we have messy data with outliers of large magnitude) we might prefer the simpler approach which is roughly 3-6x faster (at least as long as pure CPU performance is concerned. Once I/O gets into play, the difference might just disappear altogether). Which is probably why all but the fsum function show the same inaccuracy: performance. In particular, as in 99% of situations the problems won't arise.

Long story. Short takeaway: When dealing with large data, pay extra attention to the quality of your results. In fact, even do so when handling small data that is dirty and contains outliers and different magnitudes. Don't assume that computer math is exact, floating point arithmetic is not.
Don't just blindly scale up approaches that seemed to work on small data. Analyze them carefully. And last but not least, consider if adding more data will actually give you extra precision.

Dominique Dumont: About LWP::UserAgent, https and proxy setup

Hello The last few weeks, I ve been banging my head to use uscan, Pithub and JIRA::Client::Automated behind a corporate firewall. They are all written in Perl and use LWP::UserAgent to fetch information from Internet. At $work, using a proxy is mandatory to connect to Internet. But LWP::UserAgent https connection does not work through a proxy. This bug was reported 10 years ago but is still not fixed. Here s my plan to fix (or work-around) this issue, at least for Debian. Before going further, let s step back to explain briefly what is https and how proxies work. https is not a protocol by itself. https is plain http over a socket encrypted with SSL (aka TLS). To create a https connection, the agent must first setup the SSL socket with the server. Then the agent uses http protocol to communicate with the server. All the traffic is encrypted on agent side and decrypted on server side by the SSL layer. When creating a connection through a proxy, things get a little more complicated. Plain http requests (like GET, POST and so on ) are sent to the proxy as if the proxy was the http server. Then, the proxy forwards the request to the actual server. From what I ve read, most proxy servers refuse to plainly forward encrypted data to a web server. First a negotiation to create a tunnel towards the web server must be done by sending a http CONNECT request to the proxy server. The encrypted socket is then set up between the user and the web server through the proxy. Once this encrypted tunnel is set up, the usual http communication can be done. Let s go back to LWP::UserAgent. To create a connection over SSL, LWP::UserAgent will use a SSL library to setup the socket. This SSL library can be IO::Socket::SSL or Net::SSL. Direct https connection with LWP::UserAgent works fine with either library. When trying to setup a https connection through a proxy, LWP::UserAgent will try to use the proxy like a regular http proxy without going through the CONNECT phase. This does not work. Net::SSL is able to setup correctly an SSL socket through a proxy provided the proxy server is specified in an https_proxy environment variable. But LWP::UserAgent will also try to (badly) setup the proxy when this variable is set. One can find various work-around to play with %ENV to bypass LWP::UserAgent. So I ve proposed a patch to make LWP::UserAgent skip the https_proxy variable. Then Net::SSL can handle the connection to the proxy. I hope this blog will push upstream to either apply this path or tell me why it s a bad idea (which is possible, I ve some trouble to believe that a 10 years old bug can be fixed by such a simple patch) [ Edited ] I was right to worry. Turns out that relying on Net::SSL is a bad idea: SSL certificates are not checked correctly with this library. For more details, see https://github.com/libwww-perl/libwww-perl/pull/51 and https://bugzilla.redhat.com/show_bug.cgi?id=705044 . I m going to push JGMYERS set of patches on libwww-perl repositories Anyway, this patch works provided: Even though this patch is probably not ideal, it does fix issues with uscan so I ll push it to Debian. On the other hand, the patch does not work if https_proxy is specified by calling:
$ua->proxy( https => ...);
JGMYERS has proposed a set of patches to fix this problem. But these patches are more invasive and I do not dare to push them to Debian. Last but not least, this blog avoids the issue of host name verification or peer authentication. Net::SSL and Crypt::SSLeay are able to perform such a verification, but I did not verify this point. Next step is to provide simple patches to let uscan, Pithub and JIRA::Client::Automated correctly connect through proxies without jumping through hoops. All the best
Tagged: debian, https_proxy, Perl

Robert Collins: Learning is hard

I feel like I m taking a big personal risk writing this, even though I know the internet is large and probably no-one will read this :-) . So, dear reader, please be gentle. As we grow as people, as developers, as professionals some lessons are are hard to learn (e.g. you have to keep trying and trying to learn the task), and some are hard to experience (they might still be hard to learn, but just being there is hard itself ) I want to talk about a particular lesson I started learning in late 2008/early 2009 while I was at Canonical sadly one of those that was hard to experience. At the time I was one of the core developers on Bazaar, and I was feeling pretty happy about our progress, how bzr was developing, features, community etc. There was a bunch of pressure on to succeed in the marketplace, but that was ok, challenges bring out the stubborn in me :) . There was one glitch though we d been having a bunch of contentious code reviews, and my manager (Martin Pool) was chatting to me about them. I was as far as I could tell doing precisely the right thing from a peer review perspective: I was safeguarding the project, preventing changes that didn t fit properly, or that reduced key aspects- performance, usability from landing until they were fixed. However, the folk on the other side of the review were feeling frustrated, that nothing they could do would fix it, and generally very unhappy. Reviews and design discussions would grind to a halt, and they felt I was the cause. [They were right]. And here was the thing I simply couldn t understand the issue. I was doing my job; I wasn t angry at the people submitting code; I wasn t hostile; I wasn t attacking them (but I was being shall we say frank about the work being submitted). I remember saying to Martin one day look, I just don t get it can you show me what I said wrong? and he couldn t. Canonical has a 360 review system every 6 months / year (it changed over time) you review your peers, subordinate(s) and manager(s), and they review you. Imagine my surprise I was used to getting very positive reports with some constructive suggestions when I scored low on a bunch of the inter-personal metrics in the review. Martin explained that it was the reviews thing folk were genuinely unhappy, even as they commended me on my technical merits. Further to that, he said that I really needed to stop worrying about technical improvement and focus on this inter-personal stuff. Two really important things happened around this time. Firstly, Steve Alexander, who was one of my managers-once-removed at the time, reached out to me and suggested I read a book Getting out of the box and that we might have a chat about the issue after I had read it. I did so, and we chatted. That book gave me a language and viewpoint for thinking about the problem. It didn t solve it, but it meant that I got it , which I hadn t before. So then the second thing happened we had a company all hands and I got to chat with Claire Davis (head of HR at Canonical at the time) about what was going on. To this day the sheer embarrassment I felt when she told me that the broad perception of me amongst other teams managers was and I paraphrase a longer, more nuance conversation here technically fantastic but very scary to have on the team will disrupt and cause trouble . So, at this point about 6 months had passed, I knew what I wanted I wanted folk to want to work with me, to find my presence beneficial and positive on both technical and team aspects. I already knew then that what I seek is technical challenges: I crave novelty, new challenges, new problems. Once things become easy, it call all too easily slip into tedium. So at that time my reasoning was somewhat selfish: how was I to get challenges if no-one wanted to work with me except in extremis? I spent the next year working on myself as much as specific projects: learning more and more about how to play well with others. In June 2010 I got a performance review I could be proud of again I was in no way perfect, but I d made massive strides. This journey had also made huge improvements to my personal life a lot of stress between Lynne and I had gone away. Shortly after that I was invited to apply for a new role within Canonical as Technical Architect for Launchpad and Francis Lacoste told me that it was only due to my improved ability to play well with others that I was even considered. I literally could not have done the job 18 months before. I got the job, and I think I did pretty well in fact I was awarded an internal Spotlight on Success award for what we (it was a whole Launchpad team team effort) achieved while I was in that role. So, what did I change/learn? There s just a couple of key changes I needed to make in myself, but a) they aren t sticky: if I get overly tired, ye old terrible Robert can leak out, and b) there s actually a /lot/ of learnable skills in this area, much of which is derived lots of practice and critical self review is a good thing. The main thing I learnt was that I was Selfish. Yes capital S. For instance, in a discussion about adding working tree filter to bzr, I would focus on the impact/risk on me-and-things-I-directly-care-about: would it make my life harder, would it make bzr slower, was there anything that could go wrong. And I would spend only a little time thinking about what the proposer needed: they needed support and assistance making their idea reach the standards the bzr community had agreed on. The net effect of my behaviours was that I was a class A asshole when it came to getting proposals into a code base. The key things I had to change were:
  1. I need to think about the needs of the person I m speaking to *and not my own*. [Thats not to say you should ignore your needs, but you shouldn't dwell on them: if they are critical, your brain will prompt you].
  2. There s always a reason people do things: if it doesn t make sense, ask them! [The crucial conversations books have some useful modelling here on how and why people do things, and on how-and-why conversations and confrontations go bad and how to fix them.]
Ok so this is all interesting and so forth, but why the blog post? Firstly, I want to thank four folk who were particularly instrumental in helping me learn this lesson: Martin, Steve, Claire and of course my wife Lynne I owe you all an unmeasurable debt for your support and assistance. Secondly, I realised today that while I ve apologised one on one to particular folk who I knew I d made life hard for, I d never really made a widespread apology. So here it is: I spent many years as an ass, and while I didn t mean to be one, intent doesn t actually count here actions do. I m sorry for making your life hell in the past, and I hope I m doing better now. Lastly, if I m an ass to you now, I m sorry, I m probably regressing to old habits because I m too tired something I try to avoid, but it s not always possible. Please tell me, and I will go get some sleep then come and apologise to you, and try to do better in future. Less-assily-yrs,
Rob

Russ Allbery: Review: Fantasy & Science Fiction, May/June 2011

Review: Fantasy & Science Fiction, May/June 2011
Editor: Gordon van Gelder
Issue: Volume 120, No. 5 & 6
ISSN: 1095-8258
Pages: 258
The editorial in this issue is about the investigation into the troubling death of long-time contributor F. Gwynplain MacIntyre (which was not his real name). It's disturbing, but to me it underscores one of the things that I love about the Internet: people for whom life isn't working very well can still find an outlet, make friendships, and control how they choose to present themselves to the world on-line. That's something quite valuable, and part of why the pushes for "real names" always gives me pause. Somewhat thematically related, this issue also features a non-fiction essay by Maria E. Alonzo about her investigation of Jesse Francis McComas, her great-uncle but better known to the SF community as one of the founding editors of F&SF and co-editor of the famous classic anthology Adventures in Time and Space. This is mostly a curiosity, but it's fun to read about the sense of triumph in tracking down lost family history. This issue also features a Chris Moriarty book review column, always a plus, as well as a few positive reviews of obscure superhero movies by Kathi Maio (plus the required grumbling about a more mainstream film). "The Final Verse" by Chet Williamson: This is more of a horror story than I would normally like, but I got pulled into the investigation of an old bluegrass song and the guesswork and footwork required to track down where it came from. Williamson does a good job with the tone and first-person narration, and the degree to which the protagonist cares about the song to the exclusion of the horrific happenings of the story blunts the horror. Not quite my thing, but I thought it was well-done and played well with the possible meanings of song lyrics. (6) "Stock Photos" by Robert Reed: This is well-written, like nearly all Reed stories, but it lacked enough clues for the reader for me. It's a very short story about a man who's out mowing his lawn when approached by two strangers who apparently want to take photographs of him for stock image collections. Then things get rather weird, but without any explanation, and the ending lost me completely. Frustrating. (It is partially explained by the later "The Road Ahead" story in this same issue.) (4) "The Black Mountain" by Albert E. Cowdrey: From one of F&SF's most reliable story-tellers to another, and this is a more typical story. Cowdrey offers an abandoned and very strange cathedral for an obscure religion, a conflict over a development project, and some rather creepy results, all told in Cowdrey's entertaining fashion. Some places you just don't mess with. (6) "Agent of Change" by Steven Popkes: Told Dos-Passos-style with news excerpts, web sites, and the transcript of an emergency committee, this story shows the discovery of Godzilla, or something akin to Godzilla, in the Pacific, where it's destroying whaling vessels. I do like this style of storytelling, and here it mixes well with humor and a bit of parody as Popkes shows how each different news outlet puts its own recognizable spin on the story. The story isn't particularly memorable, and it doesn't end so much as just stop, but it was fun. (7) "Fine Green Dust" by Don Webb: This story is dedicated to Neal Barrett, which will give SFF short story readers a warning of weirdness to come. In a near future where global warming as continued to make summers even more miserable, the protagonist happens across a naked woman painted green. The green turns out to be a sun block that claims to assist humans in metamorphosis into animals. Most of the story is the protagonist trying to decide what to think of that, interspersed with staring at his neighbor's naked daughter. It's mildly amusing if you don't think about it too much and don't mind the rather prominent male gaze. (5) "Rampion" by Alexandra Duncan: The novella of the story, this is set in Muslim Spain some time during the long fights between Muslims and Christians in the north. It's told as two parallel stories: one telling the protagonist's first meeting with his love, and the second following him as a blind man, some time later, deciding whether, and how, to re-engage with the world. The style feels like fantasy, but there's very little overt fantasy here, and the story could be read as historical adventure. It's good adventure, though; conventional in construction, but with some romance and some drama and a good ending. (7) "Signs of Life" by Carter Scholz: This is to science fiction what "Rampion" is to fantasy: not really SF in the classic sense, but fiction about the process of science. The protagonist works on gene sequencing and is mildly obsessed with a visualization of junk DNA in an attempt to find patterns in it. Like a lot of fiction about science, it's primarily concerned with office politics, grant funding, and an awful boss. There is a faint touch of the supernatural, but that strand of the story doesn't amount to much. There's a happy ending of sorts, but the story left me with a bad taste in my mouth, and I'd completely forgotten it by the time I sat down to write this review. (4) "Starship Dazzle" by Scott Bradfield: I've never seen much in Bradfield's ongoing series of stories about Dazzle, the talking dog. In this one, he's sent via rocket on a one-way trip into outer space and ends up making a bizarre sort of first contact. Like the other Dazzle stories, it's full of attempts at humor that don't really work for me, even though you'd think I'd be sympathetic to the mocking of our commercialization of everything. The ending is just silly, and not in a good way. (3) "The Old Terrologist's Tale" by S.L. Gilbow: I love the setup for this story. It's set in some sort of far future in which terraforming has become routine, and a group of people are telling each other stories over drinks. The first-person protagonist is a terrologist, someone who designs planets (and the technology is available to do this almost from scratch). The conversation is taking a turn towards the humiliating, with a politician belittling the work of terrologists, when an old terrologist who has been listening quietly starts telling a story about designing worlds, both mundane and dangerously beautiful. Gilbow does a great job here capturing blithe self-importance, the habit of belittling other people's technical work, and revenge via storytelling with a nasty barb. This was my favorite story of the issue. (7) "Altogether Elsewhere, Vast Herds of Reindeer" by Ken Liu: This is a rather odd but quite touching story about mothers, daughters, nature, connection, and uploading. It's set after a singularity, in a time when all humans are uploaded into computers and exploring higher dimensions, digital natives in a much deeper sense than is meant today. But Rene 's mother is an Ancient, from before the singularity and still three-dimensional, and she wants to spend some time with her daughter. That leads to a memorable moment of connection, without pulling Rene entirely out of her father's world. Well done. (7) "The Road Ahead" by Robert Reed: Two Reed stories in one issue! And this one is a sequel to "Stock Photos" from earlier, since apparently I wasn't the only one who found it hopelessly confusing. It provides some backstory and makes a bit more sense of the first story, and that also makes it a more interesting story in its own right. The stock photo concept wasn't entirely a lie, as I had thought it was after the first story. There is analysis, anticipation, and trends behind who the pair take pictures of. But this story explores some internal tension, some conflict between them and some knowledge that the woman has that the man doesn't. And in the process it makes everything creepier, but also more interesting, and provides a hint at a really dark way of viewing the news media. I would say that this salvages "Stock Photos," except that I don't think "Stock Photos" is necessary now that one can read this story. (7) "Music Makers" by Kate Wilhelm: This is another story about investigation of the history of music, mingled with the supernatural, but unlike the story that opened this issue, it's not horror. Rather, it's a gentle and sweet fantasy about the power of music and benevolent ghosts and a community coming together. It's a positive and happy note on which to end the issue. (6) Rating: 6 out of 10

1 November 2013

Petter Reinholdtsen: Browser plugin for SPICE (spice-xpi) uploaded to Debian

The SPICE protocol for remote display access is the preferred solution with oVirt and RedHat Enterprise Virtualization, and I was sad to discover the other day that the browser plugin needed to use these systems seamlessly was missing in Debian. The request for a package was from 2012-04-10 with no progress since 2013-04-01, so I decided to wrap up a package based on the great work from Cajus Pollmeier and put it in a collab-maint maintained git repository to get a package I could use. I would very much like others to help me maintain the package (or just take over, I do not mind), but as no-one had volunteered so far, I just uploaded it to NEW. I hope it will be available in Debian in a few days. The source is now available from http://anonscm.debian.org/gitweb/?p=collab-maint/spice-xpi.git;a=summary.

31 October 2013

Tanguy Ortolo: How to implement a Postfix spam trap?

Open envelope Spam trap Dear lazyweb, I am considering to implement spam traps and evaluate their efficiency. The idea as rather simple:
  1. publish some real-looking email addresses on websites, in ways that no human would use them to send legitimate mail, for instance in hidden texts, or in texts clearly stating they should not be used;
  2. when my mail server receives a message for one of these address, blacklist the originating server for some time so it cannot spam real recipients.
Practical implementation Now, I have to decide on how to implement that with my Postfix server. I am thinking of using fail2ban to simply detect attempts to send mail to my spam traps and blocking these spammers at iptables-level, but I do not think that will give me much tuning possibilities. Do you have other suggestions? Implementation details I also have to think about some specific details. Here is what I identified so far, do you think I am missing something that may be useful?

Daniel Pocock: init systems: what do users want?

The debate about Debian init system replacement continues and many developers have given their opinions. However, what do the users want? What type of features will make it easier for users to adopt and promote Debian and free software in general? Could this even help bring improvements to other distributions too? Some of the possibilities I've previously raised for discussion (and they are not specific to Debian by any means):
  • Better integration with tools for service monitoring (for example, so Nagios can automatically detect all the services on a system and the dependency relationship between them, avoiding any manual configuration of the monitoring system)
  • Better integration with tools for service management (for example, so Nagios can more easily restart a service without having to provide some action script developed by hand)
  • Better support for dependencies in clustering and distributed environments (e.g. Apache on host A should only start after MySQL is running on host B or host C)
  • Support for multiple instances of a service, each with separate configuration files on the same host
Some of the systems up for discussion go some of the way to addressing these types of requirements - none of them go all the way. What other requirements do people have? Should Debian wait a little longer and let the different solutions compete to offer more of these things before any decision is made? My own feeling is that Debian should not commit to something that will involve any loss in current features or support for ports and we should use this opportunity to find the best possible set of features for the future.

Russell Coker: Links October 2013

Wired has an interesting article by David Samuels about the Skybox, a small satellite (about the size of a bar fridge) that is being developed to provide cheap photographs of the Earth from low orbit [1]. Governments of major countries will probably try to limit what they do, but if they can prove that it s viable then someone else from a different jurisdiction will build similar satellites. Alice Dreger gave an interesting TED talk about the various ways that people can fall outside the expected genetic sex binary [2]. The short film Love is All You Need has an interesting way of showing the way that non-straight kids are treated [3]. The Guardian has an interesting article by Ranjana Srivastava about doctors and depression [4]. Don Marti wrote an interesting post about believing bullshit as a way of demonstrating group loyalty [5]. Zacqart Adam Green wrote an interesting article for the Falkvinge blog about the way that the Ouya gaming console can teach children about free software and political freedom [6]. Read more at www.ouya.tv [7]. It s a pity that the Ouya is not conveniently sold outside the US and the UK, with shipping it would probably cost a lot more than $99 in Australia. Tim Chevalier wrote an interesting post for Geek Feminism about the unintended consequences of some codes of conduct [8]. Tim Chevalier wrote an interesting Geek Feminism post about Wikipedia describing how the Neutral Point Of View is a way of representing the views of people in power [9]. Ramin Shokrizade wrote an interesting article for Gamasutra about the Free 2 Play (F2P) techniques [10]. The concept of F2P games is that the game can be installed for free but requires regular small payments to make the game easier, apparently some people pay $3000 per year or more. The TED blog has an interesting interview with Jack Andraka, a teenager who invented a new test for pancreatic cancer (and also ovarian and lung cancer) that is cheaper, faster, and less invasive than other tests [11]. The blog post also has a link to Jack s TED talk.

30 October 2013

Thomas Goirand: OpenRC running on Debian / kFreeBSD

I have always pretended that OpenRC would be very easy to port to Debian GNU/kFreeBSD. Well, now I stop pretending: http://youtu.be/zoNoi8BgQjs This was done within a few hours working with upstream. Now, next-up: Debian GNU/Hurd. Hoping that porters will volunteer to do the work.

Matthew Palmer: How to deal with the "package gcc: " problem in Puppet

In a comment to yesterday s post on why your Puppet module sucks, Warren asks what can be done about the problem of multiple modules needing to include the same package (gcc being the most common example I m yet to come across). As I stated previously, there is no sane, standardised way for multiple independent modules to cooperate to ensure that certain packages are installed for their cooperative use. If it wasn t already obvious, I consider this to be a significant failing in Puppet, and one which essentially renders any attempt at public, wide-scale reuse of Puppet modules impossible. Software packages are such a fundamental part of system management today that it is rare for a module not to want to interact with them in some way. Without strong leadership from Puppet Labs, or someone core to the project who s willing to be very noisy and obnoxious on the subject, the issue is fundamentally unsolveable, because it requires either deep (and non-backwards-compatible) changes to how Puppet works at a core level, or it needs cooperation from everyone who writes Puppet modules for public consumption (to use a single, coordinated approach to common packages). The approaches that I ve seen people advocate using, or that I ve considered myself, roughly fall into the following camps.

Put your package into a class in your module Absofrickinglutely useless, because the elementary package resources you end up creating will still conflict with other modules package resources. Anyone who suggests this can be permanently ignored, as they don t understand the problem being solved.

Use a globally common module of classes representing common packages This could work, if you could get everyone who writes modules to use it (see someone willing to be very noisy and obnoxious ). Practically speaking, anyone who suggests this doesn t understand human nature. I don t see this actually happening any time soon.

Use the defined() function everywhere By wrapping all your package resources in if !defined(Package["foo"]) blocks, you can at least stop your module from causing explosions. What it doesn t do is make sure that the various definitions of the package resource are congruent (imagine if one was package "gcc": ensure => absent ). In order to safely avoid explosion, everyone would have to follow this approach, which (in the worst case) reduces the problem to the globally common module of classes. However, it s the least-worst approach I can practically consider. At least your module won t be the cause of explosions, and realistically no module s going to ask for gcc to be removed (or, if they are, you ll quickly find them and kill them). I ve seen calls from Puppet core dev team members to deprecate and remove the defined() function entirely. Given the complete lack of alternatives, this is a disturbing illustration of just how out of touch they are from the unfortunate realities of practical Puppetry.

Use virtual resources This is yet another variant of the common class technique that would require everyone, everywhere, to dance to the same tune. It has all the same problems, and hence gets a big thumbs-down from me.

Use singleton_packages This module is a valiant attempt to work around the problem, but practically speaking is no better than using defined() because again, if a module that isn t using singleton_packages specifies package "gcc": ensure => absent , you re going to end up with assplosion. The mandatory dependency on hiera doesn t win me over, either (I m a hiera-skeptic, for reasons which I might go into some other time).

Use only modules from a single source This is the solution that I use myself, if that s any recommendation As a result of pretty much every publically-available Puppet module sucking for one reason or another, I do not, currently, have any externally-written Puppet modules in the trees that I use. Every module (158, at current count) has been written by someone in the sysadmin team at $DAYJOB, to our common standards. This means that I can coordinate use of package resources, using our module of common classes where appropriate, or refactor modules to separate concerns appropriately. If you re wondering why we don t have all of these 158 modules available publically, well, we have a few of them up, but yes, the vast majority of them aren t publically available. Some of them suck mightily, while many others are just too intertwined with other modules to be able to use on their own, and we don t want to release crap code there s far too much of that out there already.

Matthew Palmer: How to deal with the "package gcc: " problem in Puppet

In a comment to yesterday s post on why your Puppet module sucks, Warren asks what can be done about the problem of multiple modules needing to include the same package (gcc being the most common example I m yet to come across). As I stated previously, there is no sane, standardised way for multiple independent modules to cooperate to ensure that certain packages are installed for their cooperative use. If it wasn t already obvious, I consider this to be a significant failing in Puppet, and one which essentially renders any attempt at public, wide-scale reuse of Puppet modules impossible. Software packages are such a fundamental part of system management today that it is rare for a module not to want to interact with them in some way. Without strong leadership from Puppet Labs, or someone core to the project who s willing to be very noisy and obnoxious on the subject, the issue is fundamentally unsolveable, because it requires either deep (and non-backwards-compatible) changes to how Puppet works at a core level, or it needs cooperation from everyone who writes Puppet modules for public consumption (to use a single, coordinated approach to common packages). The approaches that I ve seen people advocate using, or that I ve considered myself, roughly fall into the following camps.

Put your package into a class in your module Absofrickinglutely useless, because the elementary package resources you end up creating will still conflict with other modules package resources. Anyone who suggests this can be permanently ignored, as they don t understand the problem being solved.

Use a globally common module of classes representing common packages This could work, if you could get everyone who writes modules to use it (see someone willing to be very noisy and obnoxious ). Practically speaking, anyone who suggests this doesn t understand human nature. I don t see this actually happening any time soon.

Use the defined() function everywhere By wrapping all your package resources in if !defined(Package["foo"]) blocks, you can at least stop your module from causing explosions. What it doesn t do is make sure that the various definitions of the package resource are congruent (imagine if one was package "gcc": ensure => absent ). In order to safely avoid explosion, everyone would have to follow this approach, which (in the worst case) reduces the problem to the globally common module of classes. However, it s the least-worst approach I can practically consider. At least your module won t be the cause of explosions, and realistically no module s going to ask for gcc to be removed (or, if they are, you ll quickly find them and kill them). I ve seen calls from Puppet core dev team members to deprecate and remove the defined() function entirely. Given the complete lack of alternatives, this is a disturbing illustration of just how out of touch they are from the unfortunate realities of practical Puppetry.

Use virtual resources This is yet another variant of the common class technique that would require everyone, everywhere, to dance to the same tune. It has all the same problems, and hence gets a big thumbs-down from me.

Use singleton_packages This module is a valiant attempt to work around the problem, but practically speaking is no better than using defined() because again, if a module that isn t using singleton_packages specifies package "gcc": ensure => absent , you re going to end up with assplosion. The mandatory dependency on hiera doesn t win me over, either (I m a hiera-skeptic, for reasons which I might go into some other time).

Use only modules from a single source This is the solution that I use myself, if that s any recommendation As a result of pretty much every publically-available Puppet module sucking for one reason or another, I do not, currently, have any externally-written Puppet modules in the trees that I use. Every module (158, at current count) has been written by someone in the sysadmin team at $DAYJOB, to our common standards. This means that I can coordinate use of package resources, using our module of common classes where appropriate, or refactor modules to separate concerns appropriately. If you re wondering why we don t have all of these 158 modules available publically, well, we have a few of them up, but yes, the vast majority of them aren t publically available. Some of them suck mightily, while many others are just too intertwined with other modules to be able to use on their own, and we don t want to release crap code there s far too much of that out there already.

29 October 2013

Pau Garcia i Quiles: FOSDEM 2014 Desktops DevRoom Call for Talks

FOSDEM is one of the largest gatherings of Free Software contributors in the world and happens each February in Brussels (Belgium). One of the tracks will be the Desktops DevRoom (formerly known as CrossDesktop DevRoom ), which will host Desktop-related talks. We are now inviting proposals for talks about Free/Libre/Open-source Software on the topics of Desktop development, Desktop applications and interoperability amongst Desktop Environments. This is a unique opportunity to show novel ideas and developments to a wide technical audience. Topics accepted include, but are not limited to: Enlightenment, Gnome, KDE, Unity, XFCE/Razor, Windows, Mac OS X, general desktop matters, applications that enhance desktops and web (when related to desktop). Talks can be very specific, such as developing mobile applications with Qt Quick; or as general as predictions for the fusion of Desktop and web in 5 years time. Topics that are of interest to the users and developers of all desktop environments are especially welcome. The FOSDEM 2013 schedule might give you some inspiration. Please include the following information when submitting a proposal: The deadline for submissions is December 14th 2013. FOSDEM will be held on the weekend of 1-2 February 2014. Please use the following website to submit your proposals: https://penta.fosdem.org/submission/FOSDEM14 You can also join the devroom s mailing list, which is the official communication channel for the DevRoom: desktops-devroom@lists.fosdem.org (subscription page for the mailing list) The Desktops DevRoom 2014 Organization Team

Next.

Previous.